multiple regression 2
TRANSCRIPT
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 1/30
Business Research Methods
Multiple Regression
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 2/30
Multiple Regression
• Multiple Regression Model
• Least Squares Method
• Multiple Coefficient of Determination
• Model Assumptions• Testing for Significance
• Using the Estimated Regression Equation
for Estimation and Prediction• Qualitative Independent Variables
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 3/30
The Multiple Regression Model
• The Multiple Regression Model
y = 0 + 1 x1 + 2 x2 + . . . + p x p +
• The Estimated Multiple Regression Equation
y = b0
+ b1 x
1+ b
2 x
2+ . . . + b
p x
p
^
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 4/30
The Least Squares Method
• Least Squares Criterion
• Computation of Coefficients’ Values
The formulas for the regression coefficients b0, b1, b2, . .
. b p involve the use of matrix algebra. We will rely on
computer software packages to perform the calculations.
• A Note on Interpretation of Coefficients
bi represents an estimate of the change in y
corresponding to a one-unit change in xi when all other independent variables are held constant & is known as
partial regression coefficient
min ( iy y i )2
min ( iy y i
)
2
^
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 5/30
The Multiple Coefficient of Determination
• Relationship Among SST, SSR, SSE
SST = SSR + SSE
• Multiple Coefficient of Determination
R 2 = SSR/SST
• Adjusted Multiple Coefficient of Determination
( ) ( ) ( )y y y y y yi i i i 2 2 2( ) ( ) ( )y y y y y yi i i i 2 2 2
^^
R Rn
n pa2 21 1
1
1
( )R Rn
n pa2 21 1
1
1
( )
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 6/30
The Multiple Coefficient of Determination
• Multiple Coefficient of Determination R 2:It is the
square of multiple correlation coefficient R & itmeasures strength of association in multiple
regression
• Adjusted Multiple Coefficient of Determination:
R 2 is adjusted for the number of independent
variables & the sample size to account for
diminishing returns. After the first few variables
the additional variables do not make much
contribution
^^
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 7/30
Model Assumptions• Assumptions About the Error Term
– The error is a random variable with mean of zero.
– The variance of , denoted by 2, is the same for all
values of the independent variables.
– The values of are independent.
– The error is a normally distributed random variable
reflecting the deviation between the y value and the
expected value of y given by 0 + 1 x1 + 2 x2 + . . . + p x p
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 8/30
Testing for Significance: F Test
• Hypotheses
H 0: 1 = 2 = . . . = p = 0 H a: One or more of the parameters
is not equal to zero.
• Test Statistic F = MSR/MSE
• Rejection Rule
Reject H 0 if F > F
where F is based on an F distribution with p d.f.
in the numerator and n - p - 1 d.f. in the
denominator. PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 9/30
Testing for Significance: t Test
• Hypotheses
H 0: i = 0
H a: i = 0
• Test Statistic
• Rejection RuleReject H 0 if t < -t or t > t
where t is based on a t distribution with
n - p - 1 degrees of freedom.
t b
s
i
bi
t b
s
i
bi
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 10/30
Testing for Significance: Multicollinearity
• The term multicollinearity refers to thecorrelation among the independent variables.
• When the independent variables are highly
correlated (say, |r | > .7), it is not possible todetermine the separate effect of any
particular independent variable on the
dependent variable.
• Every attempt should be made to avoid
including independent variables that are
hi hl correlated.PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 11/30
Using the Estimated Regression Equation
for Estimation and Prediction
• The procedures for estimating the mean value of y
and predicting an individual value of y in multiple
regression are similar to those in simple regression.
• We substitute the given values of x1, x2, . . . , x p into
the estimated regression equation and use the
corresponding value of y as the point estimate. .
• Software packages for multiple regression will often
provide the interval estimates for an individual value
of y
^
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 12/30
Example: Programmer Salary Survey
A software firm collected data for a sample of
20computer programmers. A suggestion was made
that regression analysis could be used to determine
if salary was related to the years of experience and
the score on the firm’s programmer aptitude test.The years of experience, score on the aptitude
test, and corresponding annual salary (Rs10000) for
a sample of 20 programmers is shown on the nextslide.
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 13/30
Example: Programmer Salary Survey
Exper. Score Salary Exper. Score Salary4 78 24 9 88 38
7 100 43 2 73 26.6
1 86 23.7 10 75 36.2
5 82 34.3 5 81 31.6
8 86 35.8 6 74 29
10 84 38 8 87 34
0 75 22.2 4 79 30.1
1 80 23.1 6 94 33.9
6 83 30 3 70 28.2
6 91 33 3 89 30PIYOOSH BAJORIA
E l P S l S
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 14/30
Example: Programmer Salary Survey
• Multiple Regression Model
Suppose we believe that salary ( y) is related to theyears of experience ( x1) and the score on the
programmer aptitude test ( x2) by the following
regression model:
y = 0 + 1 x1 + 2 x2 +
where
y = annual salary (Rs.0000)
x1 = years of experience
x2
= score on programmer aptitude testPIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 15/30
Example: Programmer Salary Survey
• Estimated Regression Equationb0, b1, b2 are the least squares estimates of
0, 1, 2
Thus y = b0 + b1 x1 + b2 x2
^
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 16/30
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 17/30
Example: Programmer Salary Survey
• Computer Output
Predictor Coef Stdev t-ratio p
Constant 3.174 6.156 .52 .613
Exper 1.4039 .1986 7.07 .000
Score .25089 .07735 3.24 .005
PIYOOSH BAJORIA
INTERPRETATION
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 18/30
INTERPRETATIONPartial regression coefficients are 1.40 for
experience and .251 for score
Increase in salary per unit increase in experienceis 1.40 units when test score is constant
Increase in salary per unit increase in test score
is .251 units when experience is constantThe regression equation is
Salary = 3.17 + 1.40 Exper + 0.251 Score
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 19/30
The p ratio for
Experience is .000. This is less than .05.Hence experience is a significant variablein explaining salary
The p ratio for
Score is .005 which is also less than .05.
Hence score is also a significant variable inexplaining salary
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 20/30
Example: Programmer Salary Survey
• Computer Output (continued)
Analysis of Variance SOURCE DF SS MS F P
Regression 2 500.33 250.16 42.76 0.000
Error 17 99.46 5.85Total 19 599.79
R-sq = 83.4% R-sq(adj) = 81.5%
Interpretation : The p ratio of overall model is.000.This is less than .05. Hence overall regression
model is significant at 5% level of significance
Since R-Sqare is 81.5 %,the regression model explains
81.5% variation in salaryPIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 21/30
Example: Programmer Salary Survey
• F Test
– Hypotheses H 0: 1 = 2 = 0
H a: One or both of the parameters
is not equal to zero.
– Rejection Rule
For = .05 and d.f. = 2, 17: F .05 = 3.59
Reject H 0 if F > 3.59.
– Test Statistic
F = MSR/MSE = 250.16/5.85 = 42.76
– Conclusion
We can reject H 0.PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 22/30
Qualitative Independent Variables
• In many situations we must work with qualitativeindependent variables such as gender (male, female),method of payment (cash, check, credit card), etc.
• For example, x2 might represent gender where x2 = 0indicates male and x2 = 1 indicates female.
• In this case, x2 is called a dummy or indicator variable.• If a qualitative variable has k levels, k - 1 dummy
variables are required, with each dummy variable beingcoded as 0 or 1.
• For example, a variable with levels A, B, and C would berepresented by x1 and x2 values of (0, 0),
(1, 0), and (0,1), respectively.
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 23/30
Example: Programmer Salary Survey (B)
As an extension of the problem involving thecomputer programmer salary survey, suppose that
management also believes that the annual salary is
related to whether or not the individual has a graduate
degree in computer science or information systems.
The years of experience, the score on the programmer
aptitude test, whether or not the individual has a
relevant graduate degree, and the annual salary (Rs.0000)
for each of the sampled 20 programmers are shown on
the next slide.PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 24/30
Example: Programmer Salary Survey (B)
Exp. Score Degr. Salary Exp. Score Degr. Salary4 78 No 24 9 88 Yes 38
7 100 Yes 43 2 73 No 26.6
1 86 No 23.7 10 75 Yes 36.2
5 82 Yes 34.3 5 81 No 31.6
8 86 Yes 35.8 6 74 No 29
10 84 Yes 38 8 87 Yes 34
0 75 No 22.2 4 79 No 30.11 80 No 23.1 6 94 Yes 33.9
6 83 No 30 3 70 No 28.2
6 91 Yes 33 3 89 No 30PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 25/30
Example: Programmer Salary Survey (B)
• Multiple Regression Equation
E( y ) = 0 + 1 x1 + 2 x2 + 3 x3 • Estimated Regression Equation
y = b0 + b1 x1 + b2 x2 + b3 x3 where
y = annual salary (Rs.0000) x1 = years of experience
x2 = score on programmer aptitude test
x3 = 0 if individual does not have a grad. degree
1 if individual does have a grad. degree
Note: x3 is referred to as a dummy variable.
^
PIYOOSH BAJORIA
E l P S l S (B)
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 26/30
Example: Programmer Salary Survey (B)
• Computer Output
The regression is Salary = 7.95 + 1.15 Exp + 0.197 Score + 2.28 Deg
Predictor Coef Stdev t-ratio p
Constant 7.945 7.381 1.08 .298
Exp 1.1476 .2976 3.86 .001
Score .19694 .0899 2.19 .044
Deg 2.280 1.987 1.15 .268
SE=S =√MSE= √5.74= 2.396 R-sq = 84.7% R-sq(adj) = 81.8%
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 27/30
Example: Programmer Salary
Survey (B)• Computer Output (continued)
Analysis of Variance
SOURCE DF SS MS F P
Regression 3 507.90 169.30 29.48 0.000
Error 16 91.89 5.74
Total 19 599.79
Since p value is < .05 , we reject hypothesis 1= 2= 3=0Regression model is significant & explains = 81.8% of total
variation
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 28/30
Partial CorrelationA partial correlation coefficient measures the
association between two variables after controlling for,
or adjusting for, the effects of one or more additional
variables.
• Partial correlations have an order associated with them. Theorder indicates how many variables are being adjusted or controlled.
• The simple correlation coefficient, r , has a zero-order, as it doesnot control for any additional variables while measuring theassociation between two variables.
r x y . z = r x y - ( r x z ) ( r y z ) 1 - r x z 2 1 - r y z 2
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 29/30
Nonmetric Correlation
• If the nonmetric variables are ordinal and numeric,
Spearman's rho, , and Kendall's tau, , are two measuresof nonmetric correlation, which can be used to examinethe correlation between them.
• Both these measures use rankings rather than the absolutevalues of the variables, and the basic concepts underlying
them are quite similar. Both vary from -1.0 to +1.0 .• In the absence of ties, Spearman's yields a closer
approximation to the Pearson product moment correlationcoefficient, , than Kendall's . In these cases, theabsolute magnitude of tends to be smaller than Pearson's
.• On the other hand, when the data contain a large number of tied ranks, Kendall's seems more appropriate.
s
s
PIYOOSH BAJORIA
7/28/2019 Multiple Regression 2
http://slidepdf.com/reader/full/multiple-regression-2 30/30
Regression with Dummy Variables
Product Usage Original Dummy Variable Code
Category VariableCode D1 D2 D3
Nonusers............... 1 1 0 0
Light Users........... 2 0 1 0
Medium Users....... 3 0 0 1
Heavy Users.......... 4 0 0 0
i = a + b1 D1 + b2 D2 + b3 D3
• In this case, "heavy users" has been selected as a reference categoryand has not been directly included in the regression equation.
• The coefficient b1 is the difference in predictedi
for nonusers, ascompared to heavy users.
Y
Y
PIYOOSH BAJORIA