11 - 1 © 2003 pearson prentice hall chapter 11 simple linear regression

11 - 11 - 11

© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall

Chapter 11Chapter 11

Simple Linear Regression Simple Linear Regression

11 - 11 - 22


Learning ObjectivesLearning Objectives

1.1. Describe the Linear Regression ModelDescribe the Linear Regression Model

2.2. State the Regression Modeling StepsState the Regression Modeling Steps

3.3. Explain Ordinary Least SquaresExplain Ordinary Least Squares

1.1. Understand and check model assumptionsUnderstand and check model assumptions

4.4. Compute Regression CoefficientsCompute Regression Coefficients

5.5. Predict Response VariablePredict Response Variable

6.6. Interpret Computer OutputInterpret Computer Output

11 - 11 - 33


ModelsModels

11 - 11 - 44


ModelsModels

1.1. Representation of Some PhenomenonRepresentation of Some Phenomenon

2.2. Mathematical Model Is a Mathematical Mathematical Model Is a Mathematical Expression of Some PhenomenonExpression of Some Phenomenon

3.3. Often Describe Relationships between Often Describe Relationships between VariablesVariables

4.4. TypesTypes Deterministic ModelsDeterministic Models Probabilistic ModelsProbabilistic Models

11 - 11 - 55


Deterministic Deterministic ModelsModels

1.1. Hypothesize Exact RelationshipsHypothesize Exact Relationships

2.2. Suitable When Prediction Error is Suitable When Prediction Error is NegligibleNegligible

3.3. Example: Force Is Exactly Example: Force Is Exactly Mass Times AccelerationMass Times Acceleration FF = = mm··aa

© 1984-1994 T/Maker Co.

11 - 11 - 66


Probabilistic ModelsProbabilistic Models

1.1. Hypothesize 2 ComponentsHypothesize 2 Components DeterministicDeterministic Random ErrorRandom Error

2.2. Example: Sales Volume Is 10 Times Example: Sales Volume Is 10 Times Advertising Spending + Random ErrorAdvertising Spending + Random Error YY = 10 = 10X X + + Random Error May Be Due to Factors Random Error May Be Due to Factors

Other Than AdvertisingOther Than Advertising

11 - 11 - 77


Types of Types of Probabilistic ModelsProbabilistic Models

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

11 - 11 - 88


Regression ModelsRegression Models

11 - 11 - 99



ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

11 - 11 - 1010


Regression ModelsRegression Models

1.1. Answer ‘What Is the Relationship Answer ‘What Is the Relationship Between the Variables?’Between the Variables?’

2.2. Equation UsedEquation Used 1 Numerical Dependent (Response) Variable1 Numerical Dependent (Response) Variable

What Is to Be PredictedWhat Is to Be Predicted 1 or More Numerical or Categorical 1 or More Numerical or Categorical

Independent (Explanatory) VariablesIndependent (Explanatory) Variables

3.3. Used Mainly for Prediction & EstimationUsed Mainly for Prediction & Estimation

11 - 11 - 1111


Regression Modeling Regression Modeling Steps Steps

1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component

2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters

3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error

4.4. Evaluate ModelEvaluate Model

5.5. Use Model for Prediction & Estimation Use Model for Prediction & Estimation

11 - 11 - 1212


Model SpecificationModel Specification

11 - 11 - 1313





3.3. Specify Probability Distribution of Random Specify Probability Distribution of Random Error TermError Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error



11 - 11 - 1414


Specifying the Specifying the ModelModel

1.1. Define VariablesDefine Variables

2.2. Hypothesize Nature of RelationshipHypothesize Nature of Relationship Expected Effects (i.e., Coefficients’ Signs)Expected Effects (i.e., Coefficients’ Signs) Functional Form (Linear or Non-Linear)Functional Form (Linear or Non-Linear) InteractionsInteractions

11 - 11 - 1515


Model Specification Model Specification Is Based on TheoryIs Based on Theory

1.1. Theory of Field (e.g., Sociology)Theory of Field (e.g., Sociology)

2.2. Mathematical TheoryMathematical Theory

3.3. Previous ResearchPrevious Research

4.4. ‘Common Sense’‘Common Sense’

11 - 11 - 1616


Advertising

Sales

Advertising

Sales

Advertising

Sales

Advertising

Sales

Advertising

Sales

Advertising

Sales

Advertising

Sales

Advertising

Sales

Thinking Challenge: Thinking Challenge: Which Is More Which Is More

Logical?Logical?

11 - 11 - 1717


Types of Types of Regression ModelsRegression Models

11 - 11 - 1818



RegressionModels

11 - 11 - 1919



RegressionModels

Simple

1 Explanatory1 ExplanatoryVariableVariable

11 - 11 - 2020



RegressionModels

2+ Explanatory2+ ExplanatoryVariablesVariables

Simple Multiple


11 - 11 - 2121



RegressionModels

Linear


Simple Multiple


11 - 11 - 2222



RegressionModels

LinearNon-

Linear


Simple Multiple


11 - 11 - 2323



RegressionModels

LinearNon-

Linear


Simple Multiple

Linear


11 - 11 - 2424



RegressionModels

LinearNon-

Linear


Simple Multiple

Linear


Non-Linear

11 - 11 - 2525


Linear Regression Linear Regression ModelModel

11 - 11 - 2626



RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

11 - 11 - 2727


Y

Y = mX + b

b = Y-intercept

X

Changein Y

Change in X

m = Slope

Linear EquationsLinear Equations

High School TeacherHigh School Teacher© 1984-1994 T/Maker Co.

YY XXii ii ii 00 11

Linear Regression Linear Regression ModelModel

1.1. Relationship Between Variables Is a Relationship Between Variables Is a Linear FunctionLinear Function

Dependent Dependent (Response) (Response) VariableVariable(e.g., income)(e.g., income)

Independent Independent (Explanatory) (Explanatory) Variable Variable (e.g., education)(e.g., education)

Population Population SlopeSlope

Population Population Y-InterceptY-Intercept

Random Random ErrorError

11 - 11 - 2929


Population & Population & Sample Regression Sample Regression

ModelsModels

11 - 11 - 3030



ModelsModels

PopulationPopulation

$ $

$

$

$

11 - 11 - 3131



ModelsModels

Unknown Relationship

PopulationPopulation

Y Xi i i 0 1

$

$

$

$ $

11 - 11 - 3232



ModelsModels


PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

$ $$

$

$ $$

$$ $$

11 - 11 - 3333



ModelsModels


PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

Y Xi i i 0 1Y Xi i i 0 1

$ $$

$

$ $$

$$ $$

11 - 11 - 3434


Y

X

Y

X

Population Linear Population Linear Regression ModelRegression Model


iXYE 10 iXYE 10

ObservedObservedvaluevalue

Observed valueObserved value

ii = Random error= Random error

11 - 11 - 3535


Y

X

Y

X


Sample Linear Sample Linear Regression ModelRegression Model

Y Xi i 0 1 Y Xi i 0 1

Unsampled Unsampled observationobservation

ii = Random = Random

errorerror

Observed valueObserved value

^

11 - 11 - 3636


Estimating Parameters:Estimating Parameters:Least Squares MethodLeast Squares Method

11 - 11 - 3737







5.5. Use Model for Prediction & EstimationUse Model for Prediction & Estimation

11 - 11 - 3838


0204060

0 20 40 60

X

Y

ScattergramScattergram

1.1. Plot of All (Plot of All (XXii, , YYii) Pairs) Pairs

2.2. Suggests How Well Model Will FitSuggests How Well Model Will Fit

11 - 11 - 3939


0204060

0 20 40 60

X

Y

Thinking ChallengeThinking Challenge

How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

11 - 11 - 4040


0204060

0 20 40 60

X

Y



11 - 11 - 4141


0204060

0 20 40 60

X

Y



11 - 11 - 4242


0204060

0 20 40 60

X

Y



11 - 11 - 4343


0204060

0 20 40 60

X

Y



11 - 11 - 4444


0204060

0 20 40 60

X

Y



11 - 11 - 4545


0204060

0 20 40 60

X

Y



11 - 11 - 4646


Least SquaresLeast Squares

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Actual Y Values & Predicted Y Values Are a MinimumAre a Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative

11 - 11 - 4747



1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Actual Y Values & Predicted Y Values Are a MinimumAre a Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative

n

ii

n

iii YY

1

2

1

2ˆˆ

n

ii

n

iii YY

1

2

1

2ˆˆ

11 - 11 - 4848



1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimuma Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative

2.2. LS Minimizes the Sum of the Squared LS Minimizes the Sum of the Squared Differences (SSE)Differences (SSE)

n

ii

n

iii YY

1

2

1

2ˆˆ

n

ii

n

iii YY

1

2

1

2ˆˆ

11 - 11 - 4949


Least Squares Least Squares GraphicallyGraphically

2

Y

X

1 3

4

^^

^2

Y

X

1 3

4

^^

^^

Y X2 0 1 2 2 Y X2 0 1 2 2

Y Xi i 0 1 Y Xi i 0 1

LS minimizes ii

n2

112

22

32

42

LS minimizes ii

n2

112

22

32

42

11 - 11 - 5050


Coefficient Coefficient EquationsEquations

Sample SlopeSample Slope

Sample Y-interceptSample Y-intercept

Prediction EquationPrediction Equation

xy 10 ˆˆ

21

xx

yyxxSS

SS

i

ii

xx

xy

ii xy 10 ˆˆˆ

11 - 11 - 5151


Computation TableComputation Table

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

XiYi

Xi2 Yi

2 XiYi

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

XiYi

Xi2 Yi

2 XiYi

11 - 11 - 5252


Interpretation of Interpretation of CoefficientsCoefficients

11 - 11 - 5353



1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 for Each 1

Unit Increase in Unit Increase in XX If If 11 = 2, then Sales ( = 2, then Sales (YY) Is Expected to ) Is Expected to

Increase by 2 for Each 1 Unit Increase in Increase by 2 for Each 1 Unit Increase in Advertising (Advertising (XX))

^

^

^

11 - 11 - 5454



1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 for Each 1

Unit Increase in Unit Increase in XX If If 11 = 2, then Sales ( = 2, then Sales (YY) Is Expected to Increase ) Is Expected to Increase

by 2 for Each 1 Unit Increase in Advertising (by 2 for Each 1 Unit Increase in Advertising (XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Value of Average Value of YY When When XX = 0 = 0

If If 00 = 4, then Average Sales ( = 4, then Average Sales (YY) Is Expected to ) Is Expected to Be 4 When Advertising (Be 4 When Advertising (XX) Is 0) Is 0

^

^

^^

^

11 - 11 - 5555


Parameter Parameter Estimation ExampleEstimation Example

You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You gather the following data:You gather the following data:

Ad $Ad $ Sales (Units)Sales (Units)11 1122 1133 2244 2255 44

What is the What is the relationshiprelationship between sales & advertising?between sales & advertising?

11 - 11 - 5656


0

1

2

3

4

0 1 2 3 4 5

Scattergram Scattergram Sales vs. AdvertisingSales vs. Advertising

Sales

Advertising

11 - 11 - 5757


Guess The Parameters!Guess The Parameters!

11 - 11 - 5858


0

1

2

3

4

0 1 2 3 4 5

Scattergram Scattergram Sales vs. AdvertisingSales vs. Advertising

Sales

Advertising

11 - 11 - 5959


Parameter Parameter Estimation Solution Estimation Solution

TableTableXi Yi Xi

2 Yi2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

11 - 11 - 6060


Parameter Parameter Estimation SolutionEstimation Solution

10.0370.02ˆˆ

70.0

515

55

51015

37ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii

10.0370.02ˆˆ

70.0

515

55

51015

37ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii

11 - 11 - 6161


Coefficient Coefficient Interpretation Interpretation

SolutionSolution

11 - 11 - 6262



SolutionSolution1.1. Slope (Slope (11))

Sales Volume (Sales Volume (YY) Is Expected to Increase ) Is Expected to Increase by .7 Units for Each $1 Increase in by .7 Units for Each $1 Increase in Advertising (Advertising (XX))

^

11 - 11 - 6363



SolutionSolution1.1. Slope (Slope (11))

Sales Volume (Sales Volume (YY) Is Expected to Increase ) Is Expected to Increase by .7 Units for Each $1 Increase in Advertising by .7 Units for Each $1 Increase in Advertising ((XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Value of Sales Volume (Average Value of Sales Volume (YY) Is ) Is

-.10 Units When Advertising (-.10 Units When Advertising (XX) Is 0) Is 0 Difficult to Explain to Marketing ManagerDifficult to Explain to Marketing Manager Expect Some Sales Without AdvertisingExpect Some Sales Without Advertising

^

^

11 - 11 - 6464


Parameter EstimatesParameter Estimates

ParameterParameter Standard T for H0: Standard T for H0:

VariableVariable DF DF EstimateEstimate Error Param=0 Prob>|T| Error Param=0 Prob>|T|

INTERCEPINTERCEP 1 1 -0.1000-0.1000 0.6350 -0.157 0.8849 0.6350 -0.157 0.8849

ADVERTADVERT 1 1 0.70000.7000 0.1914 3.656 0.0354 0.1914 3.656 0.0354

Parameter Parameter Estimation Computer Estimation Computer

OutputOutput

0^ 1

^

k^

11 - 11 - 6565


Derivation of Derivation of Parameter Parameter EquationsEquations

Goal: Minimize squared errorGoal: Minimize squared error

xnnyn

xy

xy

ii

iii

10

10

0

210

0

2

ˆˆ2

ˆˆ2

ˆˆˆ

ˆˆ

0

xy 10 ˆˆ

Derivation of Derivation of Parameter Parameter EquationsEquations

iii

iii

iii

xxyyx

xyx

xy

11

10

1

210

1

2

ˆˆ2

ˆˆ2

ˆˆˆ

ˆˆ

0

xx

xy

iiii

iiii

SS

SS

yyxxxxxx

yyxxxx

1

1

1

ˆ

ˆ

ˆ

11 - 11 - 6767


Parameter Parameter Estimation Thinking Estimation Thinking

ChallengeChallengeYou’re an economist for the county You’re an economist for the county cooperative. You gather the following data:cooperative. You gather the following data:

Fertilizer (lb.)Fertilizer (lb.) Yield (lb.)Yield (lb.) 4 4 3.03.0 6 6 5.55.51010 6.56.51212 9.09.0

What is the What is the relationshiprelationship between fertilizer & crop yield?between fertilizer & crop yield?

© 1984-1994 T/Maker Co.

11 - 11 - 6868


02468

10

0 5 10 15

02468

10

0 5 10 15

Scattergram Scattergram Crop Yield vs. Crop Yield vs.

Fertilizer*Fertilizer*

Yield (lb.)Yield (lb.)Yield (lb.)Yield (lb.)

Fertilizer (lb.)Fertilizer (lb.)Fertilizer (lb.)Fertilizer (lb.)

11 - 11 - 6969


Parameter Parameter Estimation Solution Estimation Solution

Table*Table*

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

11 - 11 - 7070


Parameter Parameter Estimation Solution*Estimation Solution*

80.0865.06ˆˆ

65.0

432

296

42432

218ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii

80.0865.06ˆˆ

65.0

432

296

42432

218ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii

11 - 11 - 7171



Solution*Solution*

11 - 11 - 7272



Solution*Solution*

1.1. Slope (Slope (11)) Crop Yield (Crop Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase in Fertilizer by .65 lb. for Each 1 lb. Increase in Fertilizer ((XX))

^

11 - 11 - 7373



Solution*Solution*

1.1. Slope (Slope (11)) Crop Yield (Crop Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase in Fertilizer by .65 lb. for Each 1 lb. Increase in Fertilizer ((XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Crop Yield (Average Crop Yield (YY) Is Expected to Be ) Is Expected to Be

0.8 lb. When No Fertilizer (0.8 lb. When No Fertilizer (XX) Is Used) Is Used

^

^

11 - 11 - 7474


Probability Distribution Probability Distribution

of Random Errorof Random Error

11 - 11 - 7575








11 - 11 - 7676


Linear Regression Linear Regression Assumptions Assumptions

1.1. Mean of Probability Distribution of Mean of Probability Distribution of Error Is 0Error Is 0

2.2. Probability Distribution of Error Has Probability Distribution of Error Has Constant VarianceConstant Variance1.1. Exercise: Constant across what?Exercise: Constant across what?

3.3. Probability Distribution of Error is Probability Distribution of Error is NormalNormal

4.4. Errors Are Independent Errors Are Independent

11 - 11 - 7777


Error Error Probability Probability DistributionDistribution

Y

f()

X

X 1X 2

Y

f()

X

X 1X 2

^

11 - 11 - 7878


Random Error Random Error VariationVariation

11 - 11 - 7979



1.1. Variation of Actual Variation of Actual YY from Predicted from Predicted YY

11 - 11 - 8080




2.2. Measured by Standard Error of Measured by Standard Error of Regression ModelRegression Model Sample Standard Deviation of Sample Standard Deviation of , , ss^

11 - 11 - 8181




2.2. Measured by Standard Error of Measured by Standard Error of Regression ModelRegression Model Sample Standard Deviation of Sample Standard Deviation of , , ss

3. 3. Affects Several FactorsAffects Several Factors Parameter SignificanceParameter Significance Prediction AccuracyPrediction Accuracy

^

11 - 11 - 8282


Evaluating the ModelEvaluating the Model

Testing for SignificanceTesting for Significance

11 - 11 - 8383







5.5. Use Model for Prediction & EstimationUse Model for Prediction & Estimation

11 - 11 - 8484


Test of Slope Test of Slope CoefficientCoefficient

1.1. Shows If There Is a Linear Relationship Shows If There Is a Linear Relationship Between Between XX & & YY

2.2. Involves Population Slope Involves Population Slope 11

3.3. Hypotheses Hypotheses HH00: : 1 1 = 0 (No Linear Relationship) = 0 (No Linear Relationship)

HHaa: : 11 0 (Linear Relationship) 0 (Linear Relationship)

4.4. Theoretical Basis Is Sampling Distribution Theoretical Basis Is Sampling Distribution of Slopeof Slope

11 - 11 - 8585


Sampling Sampling Distribution Distribution

of Sample Slopesof Sample Slopes

11 - 11 - 8686


Y

Population LineX

Sample 1 Line

Sample 2 Line

Y

Population LineX

Sample 1 Line

Sample 2 Line



11 - 11 - 8787


Y

Population LineX

Sample 1 Line

Sample 2 Line

Y

Population LineX

Sample 1 Line

Sample 2 Line



All Possible All Possible Sample SlopesSample Slopes

Sample 1:Sample 1: 2.52.5

Sample 2:Sample 2: 1.6 1.6


Sample 4:Sample 4: 2.12.1 : : : :Very large number of Very large number of sample slopessample slopes

11 - 11 - 8888


Y

Population LineX

Sample 1 Line

Sample 2 Line

Y

Population LineX

Sample 1 Line

Sample 2 Line



11

All Possible All Possible Sample SlopesSample Slopes


Sample 2:Sample 2: 1.6 1.6


Sample 4:Sample 4: 2.12.1 : : : :Very large number of Very large number of sample slopessample slopes

Sampling DistributionSampling Distribution

11

11SS

^

^

11 - 11 - 8989


Slope Coefficient Slope Coefficient Test StatisticTest Statistic

n

X

X

SS

St

n

iin

ii

n

2

1

1

2

ˆ

ˆ

112

1

1

where

ˆ

n

X

X

SS

St

n

iin

ii

n

2

1

1

2

ˆ

ˆ

112

1

1

where

ˆ

11 - 11 - 9090


Test of Slope Test of Slope Coefficient ExampleCoefficient Example

You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You find You find bb00 = -.1 = -.1,, bb11 = .7 = .7 & & ss = .60553= .60553..


Is the relationship Is the relationship significantsignificant at the at the .05.05 level? level?

11 - 11 - 9191


Solution TableSolution Table

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

11 - 11 - 9292


Test of Slope Test of Slope Parameter Parameter

SolutionSolutionHH00: : 11 = 0 = 0

HHaa: : 11 0 0

.05.05

df df 5 - 2 = 35 - 2 = 3

Critical Value(s):Critical Value(s):

Test Statistic: Test Statistic:

Decision:Decision:

Conclusion:Conclusion:

t0 3.1824-3.1824

.025

Reject Reject

.025

t0 3.1824-3.1824

.025

Reject Reject

.025

tS

.

..

1 1

1

0 70 001915

3 656tS

.

..

1 1

1

0 70 001915

3 656

Reject at Reject at = .05 = .05

There is evidence of a There is evidence of a relationshiprelationship

11 - 11 - 9393


Test StatisticTest StatisticSolutionSolution

1915.0

515

55

60553.0

where

656.31915.0

070.0ˆ

32

1

1

2

ˆ

ˆ

112

1

1

n

X

X

SS

St

n

iin

ii

n

1915.0

515

55

60553.0

where

656.31915.0

070.0ˆ

32

1

1

2

ˆ

ˆ

112

1

1

n

X

X

SS

St

n

iin

ii

n

11 - 11 - 9494


Test of Slope Test of Slope ParameterParameter

Computer OutputComputer Output Parameter EstimatesParameter Estimates

Parameter Standard Parameter Standard T for H0:T for H0:

VariableVariable DF Estimate Error DF Estimate Error Param=0 Prob>|T|Param=0 Prob>|T|

INTERCEP 1 -0.1000 0.6350 -0.157 0.8849INTERCEP 1 -0.1000 0.6350 -0.157 0.8849

ADVERTADVERT 1 0.7000 0.1914 1 0.7000 0.1914 3.6563.656 0.03540.0354

t = k / S

P-Value

Skk k

^^^^

11 - 11 - 9595


Measures of Measures of Variation Variation

in Regression in Regression 1.1. Total Sum of Squares (SSTotal Sum of Squares (SSyyyy))

Measures Variation of Observed Measures Variation of Observed YYii Around the MeanAround the MeanYY

2.2. Explained Variation (SSR)Explained Variation (SSR) Variation Due to Relationship Between Variation Due to Relationship Between

XX & & YY

3.3. Unexplained VariationUnexplained Variation (SSE) (SSE) Variation Due to Other FactorsVariation Due to Other Factors

11 - 11 - 9696


Y

X

Y

X i

Y

X

Y

X i

Variation MeasuresVariation Measures

Y Xi i 0 1 Y Xi i 0 1

Total sum Total sum

of squares of squares

(Y(Yii - -Y)Y)22

Unexplained sum Unexplained sum

of squares (Yof squares (Yii - -

YYii))22

^

Explained sum of Explained sum of

squares (Ysquares (Yii - -Y)Y)22 ^

YYii

11 - 11 - 9797


1.1. ProportionProportion of Variation ‘Explained’ by of Variation ‘Explained’ by Relationship Between Relationship Between XX & & YY

Coefficient of Coefficient of DeterminationDetermination

n

ii

n

ii

n

ii

YY

YYYY

r

1

2

1

2

1

2

2

ˆ

Variation Total

Variation Explained

n

ii

n

ii

n

ii

YY

YYYY

r

1

2

1

2

1

2

2

ˆ

Variation Total

Variation Explained

0 r2 1

11 - 11 - 9898


Y

X

Y

X

Y

X

Coefficient of Coefficient of Determination Determination

ExamplesExamplesY

X

r2 = 1 r2 = 1

r2 = .8 r2 = 0

11 - 11 - 9999


Coefficient of Coefficient of Determination Determination

ExampleExampleYou’re a marketing analyst for Hasbro You’re a marketing analyst for Hasbro

Toys. You find Toys. You find 00 = -0.1 & = -0.1 & 11 = 0.7. = 0.7.


Interpret a Interpret a coefficient of coefficient of determination determination ofof 0.8167.0.8167.

^^

11 - 11 - 100100


r r 22 Computer Output Computer Output

Root MSE 0.60553Root MSE 0.60553 R-square 0.8167R-square 0.8167

Dep Mean 2.00000 Dep Mean 2.00000 Adj R-sq 0.7556Adj R-sq 0.7556

C.V. 30.27650 C.V. 30.27650

r2 adjusted for number of explanatory variables & sample size

S

r2

11 - 11 - 101101


Using the Model for Using the Model for Prediction & EstimationPrediction & Estimation

11 - 11 - 102102








11 - 11 - 103103


Prediction With Prediction With Regression ModelsRegression Models

1.1. Types of PredictionsTypes of Predictions Point EstimatesPoint Estimates Interval EstimatesInterval Estimates

2.2. What Is PredictedWhat Is Predicted Population Mean Response Population Mean Response EE((YY) for ) for

Given Given XX Point on Population Regression LinePoint on Population Regression Line

Individual Response (Individual Response (YYii) for Given ) for Given XX

11 - 11 - 104104


What Is PredictedWhat Is Predicted

Mean Y, E(Y)

Y

Y i= 0

+ 1X

^Y Individual

Prediction, Y

E(Y) = 0 + 1X

^

XXP

^^

Mean Y, E(Y)

Y

Y i= 0

+ 1X

^Y Individual

Prediction, Y

E(Y) = 0 + 1X

^

XXP

^^

11 - 11 - 105105


ConfidenceConfidence Interval Interval Estimate of Mean Estimate of Mean YY

n

ii

p

Y

YnYn

XX

XX

nSS

StYYEStY

1

2

2

ˆ

ˆ2/,2ˆ2/,2

1

where

ˆ)(ˆ

n

ii

p

Y

YnYn

XX

XX

nSS

StYYEStY

1

2

2

ˆ

ˆ2/,2ˆ2/,2

1

where

ˆ)(ˆ

11 - 11 - 106106


Factors Affecting Factors Affecting Interval WidthInterval Width

1.1. Level of Confidence (1 - Level of Confidence (1 - )) Width Increases as Confidence IncreasesWidth Increases as Confidence Increases

2.2. Data Dispersion (Data Dispersion (ss)) Width Increases as Variation IncreasesWidth Increases as Variation Increases

3.3. Sample SizeSample Size Width Decreases as Sample Size IncreasesWidth Decreases as Sample Size Increases

4.4. Distance of Distance of XXpp from Mean from MeanXX Width Increases as Distance IncreasesWidth Increases as Distance Increases

11 - 11 - 107107


Why Distance from Why Distance from Mean?Mean?

Sample 2 Line

Y

XX1 X2

Y_ Sample 1 Line

Sample 2 Line

Y

XX1 X2

Y_ Sample 1 Line

Greater Greater dispersion dispersion than than XX11

XX

11 - 11 - 108108


ConfidenceConfidence Interval Interval Estimate ExampleEstimate Example

You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You find You find bb00 = -.1 = -.1,, bb11 = .7 = .7 & & ss = .60553= .60553..


Estimate the Estimate the meanmean sales when sales when advertising is advertising is $4$4 at the at the .05.05 level. level.

11 - 11 - 109109


Solution TableSolution Table

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

11 - 11 - 110110


ConfidenceConfidence Interval Interval Estimate SolutionEstimate Solution

7553.3)(6445.1

3316.01824.37.2)(3316.01824.37.2

3316.010

34

5

160553.

7.247.01.0ˆ

ˆ)(ˆ

2

ˆ

ˆ2/,2ˆ2/,2

YE

YE

S

Y

StYYEStY

Y

YnYn

7553.3)(6445.1

3316.01824.37.2)(3316.01824.37.2

3316.010

34

5

160553.

7.247.01.0ˆ

ˆ)(ˆ

2

ˆ

ˆ2/,2ˆ2/,2

YE

YE

S

Y

StYYEStY

Y

YnYn

XX to be predicted to be predictedXX to be predicted to be predicted

11 - 11 - 111111


n

ii

PYY

YYnPYYn

XX

XX

nSS

StYYStY

1

2

2

ˆ

ˆ2/,2ˆ2/,2

11

where

ˆˆ

n

ii

PYY

YYnPYYn

XX

XX

nSS

StYYStY

1

2

2

ˆ

ˆ2/,2ˆ2/,2

11

where

ˆˆ

PredictionPrediction Interval Interval of Individual of Individual

ResponseResponse

Note!Note!

11 - 11 - 112112


Why the Extra ‘SWhy the Extra ‘S’’??

Expected(Mean) Y

Y

Y i= 0

+ 1X i

^

Y we're trying to predict

Prediction, Y

E(Y) = 0 + 1X

^

XXP

^

^Expected(Mean) Y

Y

Y i= 0

+ 1X i

^

Y we're trying to predict

Prediction, Y

E(Y) = 0 + 1X

^

XXP

^

^

11 - 11 - 113113


Interval Estimate Interval Estimate Computer OutputComputer Output

Dep Var Pred Std Err Dep Var Pred Std Err Low95% Upp95% Low95% Upp95%Low95% Upp95% Low95% Upp95%

Obs SALES Value Predict Obs SALES Value Predict Mean Mean Predict PredictMean Mean Predict Predict

1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037 1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037

2 1.000 1.300 0.332 0.244 2.355 -0.897 3.4972 1.000 1.300 0.332 0.244 2.355 -0.897 3.497

3 2.000 2.000 0.271 1.138 2.861 -0.111 4.1113 2.000 2.000 0.271 1.138 2.861 -0.111 4.111

4 2.000 4 2.000 2.700 0.332 1.644 3.755 0.502 4.897 2.700 0.332 1.644 3.755 0.502 4.897

5 4.000 3.400 0.469 1.907 4.892 0.962 5.8375 4.000 3.400 0.469 1.907 4.892 0.962 5.837

Predicted Predicted YY when when XX = 4 = 4

Confidence Confidence IntervalInterval

SSYYPrediction Prediction IntervalInterval

11 - 11 - 114114


Hyperbolic Interval Hyperbolic Interval BandsBands

X

Y

X

Y i= 0

+ 1X i

^

XP

_

^^

X

Y

X

Y i= 0

+ 1X i

^

XP

_

^^

11 - 11 - 115115


Correlation ModelsCorrelation Models

11 - 11 - 116116



ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

11 - 11 - 117117


Correlation ModelsCorrelation Models

1.1. Answer ‘Answer ‘How Strong How Strong Is the Linear Is the Linear Relationship Between 2 Variables?’Relationship Between 2 Variables?’

2.2. Coefficient of Correlation UsedCoefficient of Correlation Used Population Correlation Coefficient Denoted Population Correlation Coefficient Denoted

(Rho) (Rho) Values Range from -1 to +1Values Range from -1 to +1 Measures Degree of AssociationMeasures Degree of Association

3.3. Used Mainly for UnderstandingUsed Mainly for Understanding

11 - 11 - 118118


1.1. Pearson Product Moment Coefficient of Pearson Product Moment Coefficient of Correlation, Correlation, rr::

Sample Coefficient Sample Coefficient of Correlationof Correlation

n

ii

n

ii

n

iii

YYXX

YYXX

r

1

2

1

2

1

ionDeterminat oft Coefficien

n

ii

n

ii

n

iii

YYXX

YYXX

r

1

2

1

2

1

ionDeterminat oft Coefficien

11 - 11 - 119119


Coefficient of Correlation Coefficient of Correlation ValuesValues

-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

11 - 11 - 120120



-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

No No CorrelationCorrelation

11 - 11 - 121121



-1.0-1.0 +1.0+1.000

Increasing degree of Increasing degree of negative correlationnegative correlation

-.5-.5 +.5+.5

No No CorrelationCorrelation

11 - 11 - 122122



-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

Perfect Perfect Negative Negative

CorrelationCorrelationNo No

CorrelationCorrelation

11 - 11 - 123123



-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5




Increasing degree of Increasing degree of positive correlationpositive correlation

11 - 11 - 124124



-1.0-1.0 +1.0+1.000

Perfect Perfect Positive Positive


-.5-.5 +.5+.5




11 - 11 - 125125


Coefficient of Coefficient of CorrelationCorrelation ExamplesExamples

Y

X

Y

X

Y

X

Y

X

r = 1 r = -1

r = .89 r = 0

11 - 11 - 126126


Test of Test of Coefficient of Coefficient of Correlation Correlation

1.1. Shows If There Is a Linear Relationship Shows If There Is a Linear Relationship Between 2 Numerical VariablesBetween 2 Numerical Variables

2.2. Same Conclusion as Testing Same Conclusion as Testing Population Slope Population Slope 11

3.3. Hypotheses Hypotheses HH00: : = 0 (No Correlation) = 0 (No Correlation)

HHaa: : 0 (Correlation) 0 (Correlation)

11 - 11 - 127127


ConclusionConclusion

1.1. Described the Linear Regression ModelDescribed the Linear Regression Model

2.2. Stated the Regression Modeling StepsStated the Regression Modeling Steps

3.3. Explained Ordinary Least SquaresExplained Ordinary Least Squares

4.4. Computed Regression CoefficientsComputed Regression Coefficients

5.5. Predicted Response VariablePredicted Response Variable

6.6. Interpreted Computer OutputInterpreted Computer Output

End of Chapter

Any blank slides that follow are blank intentionally.

11 - 1 © 2003 pearson prentice hall chapter 11 simple linear regression

Documents

pearson prentice hall

pearson prentice hall

simple linear regression

explanatory variable

advertising slide

prediction estimation

common sense slide

computer output slide