Download - Introductory Econometrics: Chapter 2 Slides
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 1/31
REVISION MULTIPLE R EGRESSION
Econometrics I - Exercise 2
Lubomır Cingl
March 4, 2014
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 2/31
REVISION MULTIPLE R EGRESSION
REVISION
Simple regression model
y = β 0 + β 1x + u (1)
y dependent / explained
x independent / explanatory / control
u error term
β 0 intercept
β 1 slope
we want to know the relationship
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 3/31
REVISION MULTIPLE R EGRESSION
ASSUMPTIONS
Average value of u in population is 0: E(u) = 0
not very restrictive Zero conditional mean
knowing sth about x does not give any info about u E(u|x) = E(u) = 0 which implies E( y|x) = β 0 + β 1x
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 4/31
REVISION MULTIPLE R EGRESSION
E( y|x) as a linear function of x;distribution of y centered aroundexp. value
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 5/31
REVISION MULTIPLE R EGRESSION
ESTIMATION FROM A SAMPLE
we have a random sample of observations
for each observation holds:
Sample regression line
yi = β 0 + β 1xi + ui (2)
we want ”best” estimates of parameters
β 0, β 1 3 ways how to find them: MoM, OLS, ML
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 6/31
REVISION MULTIPLE R EGRESSION
FORMULAS TO KNOW
interceptβ 0 = ¯ y− β 1x (3)
slope
β 1 =
ni=1(xi − x)( yi − ¯ y)ni=1
(xi − x)2
(4)
if
ni=1(xi − x)
2
> 0
β 1 is sample covar bw x and y div by variance of x
If x and y are positively correlated, the slope positive
R M R
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 7/31
REVISION MULTIPLE R EGRESSION
SAMPLE REGRESSION LINE
e.g. ˆinc = 0.33 + 0.56edu
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 8/31
REVISION MULTIPLE R EGRESSION
R SQUARED
each observation can be made up by explained andunexplained part
yi = ˆ yi + ui we can define following:
( yi − ¯ y)2 is total sum of squares (Var of y)
( ˆ yi − ¯ y)2 explained sum of squares
(ui)
2 residual sum of squares
SST = SSE + SSR
R2 = SSE/SST = 1 − SSR/SST
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 9/31
REVISION MULTIPLE R EGRESSION
PROPERTIES OF OLS ESTIMATOR
Unbiased expected value of estimator is its true value β 1 = β 1 +
(xi−x)ui(xi−x)2 = β 1
Variance Assume homoskedasticity Var(u|x) = σ2
σ2 is the error variance Var β 1 = σ2/
(xi − x) = σ2/sx
2
We also have to estimate σ2
σ2 = 1/(n− 2)u12 = SSR/(n− 2)
Var β 1 = σ2/
(xi − x) = 1/(n− 2)u1
2/sx2
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 10/31
REVISION MULTIPLE R EGRESSION
EXAMPLE 2.7
Consider the savings function
sav = β 0 + β 1inc + u,u =√ inc ∗ e (5)
where e is a random variable with E(e) =
0 and Var(e) = σ2
e .Assume that e is independent of inc.
1. Show that E(u|inc) = 0, so that the key zero conditionalmean assumption is satisfied. [Hint: if e independent of inc, then E(e
|inc) = E(e) and Var(e
|inc) = Var(e)]
2. Show that Var(u|inc) = σ2e ∗ inc.3. Provide a discussion that supports the assumption that the
variance of savings increases with family income.
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 11/31
REVISION MULTIPLE R EGRESSION
INTERPRETATION
linear function y = β 0 + β 1x
marginal effect of x on y
natural log
y = log(x); x > 0
good to know: log(1 + x) ≈ x for x ≈ 0 and
log(x1) − log(x0) ≈ (x1 − x0)/x0 = ∆x/x0 therefore ∆log(x)
∗100
≈%∆x
constant elasticity model
log( y) = β 0 + β 1log(x); y, x > 0
β 1 is elasticity of y w.r.t. x
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 12/31
REVISION MULTIPLE R EGRESSION
EXAMPLE 2.6
Using data from 1988 for houses sold close to garbage mill,following equation relates housing price to distance fromgarbage incinerator:
log( ˆ price) = 9.4 + 0.312log(dist);n = 135;R2 = 0.162 (6)
1. Interpret the coefficient on log(dist). Is the sign of thisestimate what you expect it to be?
2. Do you think simple regression provides an unbiased
estimator of the ceteris paribus elasticity of price withrespect to dist? (Think about the citys decision on where toput the incinerator.)
3. What other factors about a house affect its price? Mightthese be correlated with distance from the incinerator?
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 13/31
REGRESSION THROUGH ORIGIN
restriction: x = 0 if y = 0 then we estimate y = β 1x
we know that
β 1 =
ni=1
(xi − x)( yi − ¯ y)
ni=1
(xi − x)2(7)
if we set x = 0, ¯ y = 0 we get possibly biased estimator:
β 1 =
ni=1
(xi)( yi)
ni=1
(xi)2
(8)
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 14/31
EXAMPLE 2.8
Consider standard simple regression model with standardassumptions met. Let β 1 be the estimator of β 1 obtained byassuming the intercept is zero.
1. Find E( β 1) in terms of the xi, β 0, and β 1. Verify that β 1 is
unbiased for β 1 when the population intercept β 0 is zero.Are there other cases where β 1 is unbiased?
2. Find the variance of β 1. Hint: It does not depend on β 0.
3. Show that Var( β 1) ≤ Var( β 1). Hint:
(xi)2 ≥(xi − x)2
with strict inequality unless x = 0.4. Comment on the trade-off between bias and variance
when choosing between β 1 and β 1.
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 15/31
EXAMPLE 2.9
1. Let β 0 and β 1 be the intercept and slope from the regressionof yi on xi, using n observations. Let c1 and c2, with c2 = 0, be constants. Let β 0 and β 1 be the intercept and slope from
the regression c1 yi on c2xi. Show that β 1 = (c1/c2) β 1 andβ 0 = c1 β 0, thereby verifying the claims on units of measurement in Section 2.4. [Hint: To obtain β 1, β 0 plugthe scaled versions of x and y into the OLS formulas.]
2. Now let and β 0 and β 1 be from the regression (c1 + yi) on(c2 + xi) (with no restriction on c1 or c2). Show that β 1 = β 1and β 0 = β 0 + c1 − c2 β 1.
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 16/31
EXAMPLE 2.9
3. Now let and β 0 and β 1 be the OLS estimates from theregression log( yi) on (xi), y > 0. For c1 > 0 let β 0 and β 1 be the
intercept and slope from the regression of log(c1 yi) on (xi).Show that β 1 = β 1 and β 0 = β 0 + log(c1).4. Now, assuming that xi > 0, let β 0 and β 1 be the intercept andslope from the regression of yi on log(c2xi). How do β 0 and β 1compare with the intercept and slope from the regression of yi
on log(xi)?
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 17/31
Multiple regression
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 18/31
MULTIPLE REGRESSION MODEL
paralels with simple reg model
y = β 0 + β 1x1 + β 2x2 + ... + u (9)
y dependent / explained
x independent / explanatory / control u error term
similar assumptions about the error term
β 0 intercept
linear - linear in parameters β 1,β 2, ... slopes
Interpretation: ∆ˆ y = ∆ β 1x1 ceteris paribus: holding other factors constant
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 19/31
EXAMPLE I
effect of education on wage
wage = β 0 + β 1educ + β 2exper + u (10)
we take exper out from error term
ceteris paribus: holding experience constant, we canmeasure the effect of education on wage
partial effect
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 20/31
EXAMPLE II
effect of income on consumption
cons = β 0 + β 1inc + β 2(inc)2 + u (11)
same way of getting parameters interpretation different: alone inc non-sense
∆cons
∆inc ≈ β 1 + 2β 2inc
marginal effect of income on consumption depends on both terms
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 21/31
ASSUMPTIONS
Expected value of u is 0: E(u) = 0 Zero conditional mean (ASS3)
knowing sth about x does not give any info about u
E(u|x1, x2,..., xk ) = 0 all unobserved factors in error term uncorrelated with all xi
No perfect collinearity (ASS4) no independent variable is constant no exact linear relationship among them holds
x’s can be correlated, but not perfectly we need more observations than parameters: n ≥ k + 1
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 22/31
ESTIMATION FROM A SAMPLE
we have a random sample of observations (ASS2)
for each observation holds (ASS1):
yi = β 0 + β 1xi1 + β 2xi2 + ... + β k xik + ui (12)
we want ”best” estimates of parameters
β 0, β 1, ...
3 ways how to find them: MoM, OLS, ML
we will get the estimates ⇒ Sample regression ft
ˆ yi = β 0 + β 1xi2 + β 2xi2 + ... + β k xik (13)
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 23/31
PROPERTIES
if ASS1 thru ASS4 hold, then our OLS estimator isunbiased
i.e. the procedure of getting the estimate is unbiased
if we add irrelevant variable no effect on parameters of relevant variables - still unbiased slight increase in R2
if we omit important variable OLS will be biased, usually ”omitted variable bias”
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 24/31
OMMITED VARIABLE BIAS
y = β 0 + β 1x1 + β 2x2 + u
˜ y = β 0 + β 1x1
E( β 1) = β 1 + β 2
(xi1 − x1)xi2(xi1 − x1)2 = β 1 + β 2 δ 1
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 25/31
VARIANCE
Assume homoskedasticity Var(u|x1, x2...xk ) = σ2
variance of u same for all combinations of outcomes of x’s
Var(β j) = σ2
(xij−
x j)(1−R2
j
) =
σ2
SST j(1−R2
j
) (14)
R2 j is R2 from the regression of x j on all other x’s size important
We also have to estimate σ2
σ2 = 1/(n
−k
−1) u1
2= SSR/df
Varβ j = σ
2
SST j(1−R2 j )
seβ j = σ SST j(1−R2 j )
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 26/31
EXAMPLE 3.9
The following equation describes the median housing price in acommunity in terms of amount of pollution (nox for nitrousoxide) and the average number of rooms in houses in thecommunity (rooms):
log( price) = β 0 + β 1log(nox) + β 2rooms + u
1. What are the probable signs of the regression slopes?Interpret β 1, explain.
2. Why might nox [more precisely, log(nox)] and rooms benegatively correlated? If this is the case, does the simpleregression of log( price) on log(nox) produce an upward ordownward biased estimator of β 1?
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 27/31
EXAMPLE 3.9
3. Using the data in HPRICE2.RAW, the following equationswere estimated:
log( ˆ price) = 11.71
−1.043log(nox),n = 506,R2 = 0.264
log( ˆ price) = 9.23−0.718log(nox)+0.306rooms,n = 506,R2 = 0.514
Is the relationship between the simple and multiple regressionestimates of the elasticity of price with respect to nox what you
would have predicted, given your answer in part 2.? Does thismean that -0.718 is definitely closer to the true elasticity than-1.043?
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 28/31
EXAMPLE 3.12
1. Consider the simple regression model y = β 0 + β 1x+ u underthe first four Gauss-Markov assumptions. For some functiong(x), for example g(x) = x2 or g(x) = log(1 + x2) define zi = g(xi). Define a slope estimator as
β 1 =
( zi − ¯ z) yi/
( zi − z)xi
Show that β 1 is linear and unbiased. Remember, becauseE(u|x) = 0, you can treat both xi and zi as nonrandom in yourderivation.2. Add the homoskedasticity assumption, MLR.5. Show that
Var( β 1) = σ(
( zi − ¯ z)2)/(
( zi − z)xi)2
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 29/31
EXAMPLE 3.12
3. Show directly that, under the Gauss-Markovassumptions,Var( β 1) ≤ Var( β 1), where β 1 is the OLS estimator.
[Hint: The Cauchy-Schwartz inequality in Appendix B impliesthat
(n−1
( zi − z)(xi − x))2 ≤ σ(n−1
( zi − ¯ z)2)(n−1
(xi − x)2)
notice that we can drop ¯x from the sample covariance.]
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 30/31
EXAMPLE 3.15
The file CEOSAL2.RAW contains data on 177 chief executiveofficers, which can be used to examine the effects of firmperformance on CEO salary.
1. Estimate a model relating annual salary to firm sales and
market value. Make the model of the constant elasticityvariety for both independent variables. Write the resultsout in equation form.
2. Add profits to the model from part 1. Why can this
variable not be included in logarithmic form? Would yousay that these firm performance variables explain most of the variation in CEO salaries?
REVISION MULTIPLE R EGRESSION
7/17/2019 Introductory Econometrics: Chapter 2 Slides
http://slidepdf.com/reader/full/introductory-econometrics-chapter-2-slides 31/31
EXAMPLE 3.15
3. Add the variable ceoten to the model in part 2. What is theestimated percentage return for another year of CEO tenure,
holding other factors fixed?4. Find the sample correlation coefficient between the variableslog(mktval) and profits. Are these variables highly correlated?What does this say about the OLS estimators?