gls heteros kedas t i city

7/30/2019 Gls Heteros Kedas t i City

1/22

OLS Under HeteroskedasticityTesting for Heteroskedasticity

Heteroskedasticity and Weighted Least Squares

Walter Sosa-Escudero

Econ 507. Econometric Analysis. Spring 2009

April 14, 2009

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares
http://find/http://goback/


2/22


The Classical Linear Model:

1 Linearity: Y = X+ u.

2 Strict exogeneity: E(u) = 0

3 No Multicollinearity: (X) = K.

4 No heteroskedasticity/ serial correlation: V(u) = 2In.

Gauss/Markov Theorem: = (XX)1XY is best linearunbiased.

V() = S2(XX)1 is an unbiased estimate ofV() = 2(XX)1.

http://find/


3/22


What happens if we drop the homoskedasticity assumption?

(the OLS estimator) is still linear and unbiased (Why?).

Though linear and unbiased, is not the minimum varianceestimate (inefficient).

V() = S2(XX)1 is biased. This makes standard t andF tests invalid.

http://find/


4/22


Intuition

The presence of heteroskedastic errors should not alter the centralposition of the OLS line (unbiasedness).

OLS weigths all observations equally, but in this case it makes moresense to pay more attention to observations where the variance issmaller.


OLS U d H k d i i
http://find/


5/22


The plan: what to do with heteroskedasticity.

1 Before abandoning OLS we will see how to test forheteroskedasticity.

2 Strategy 1: Propose another more efficient and unbiased

estimator for (weighted least squares (WLS)) and a suitableestimator for its variance.

3 Strategy 2: Keep using OLS (it is still unbiased, thoughinefficient), but find a replacement for its variance (the old

one is biased under heteroskedasticity).


OLS U d H t k d ti it


6/22


Testing for heteroscedasticity

a) The White test

H0 : no heteroscedasticity, HA : there is heterocedasticity of someform.

Consider a simple case with K = 3:

Yi = 1 + 2X2i + 3X3i + ui 1, . . . , n


OLS Under Heteroskedasticity


7/22


Steps to implement the test:

1 Estimate by OLS, save squared residuals in e2.

2 Regress e2 on all variables, their squares and all possiblenon-redundant cross-products. In our case, regress e2 on

1, X2, X3, X22 , X23 , X2X3, and obtain R2 in this auxiliarregression.

3 Under H0, nR2 2(p). p = number of explanatory variables

in the auxiliar regression minus one.

4 Reject Ho if nR2

is too large.


http://find/


8/22


Intuition: The auxiliar model can be seen as trying to model the

variance of the error term. If the R2

of this auxiliar regression werehigh, then we could explain the behavior of the squared residuals,providing evidence that they are not constant.

Caveats:

Valid for large samples.

Informative if we do not reject the null (no heterocedasticity).

When it rejects the null: there is heterocedasticity. But we donot have any information regarding what causes

heterocedasticity. This will cause some trouble when trying toconstruct a GLS estimator, for which we need to know in avery specific way what causes heterocedasticity.


http://find/


9/22


b) The Breusch-Pagan/Godfrey/Koenker test

Mechanically very similar to Whites test. Checks if certainvariables cause heterocedasticity. Consider the followingheteroscedastic model:

Y = X+ u, ui

normal, with E(u) = 0 and

V(ui) = h(1 + 2Z2i + 3Z3i + . . . + pZpi)

where h( ) is any positive function with two derivatives.

When 2 = . . . = p = 0, V(ui) = h(1), a constant!!Then, homoscedasticity H0 : 2 = 3 = . . . = p = 0 ,andHA : 2 = 0 3 = 0 . . . p = 0.


http://find/


10/22


Steps to implement the test:

1 Estimate by OLS, and save squared residuals e2i .

2 Regresss e2i on the Zik variables, k = 2, . . . , p and get (ESS).

The test statistic is:1

2ESS 2(p 1) 2(p)

under H0, asymptotically. We reject if it is too large.


http://find/


11/22

yTesting for Heteroskedasticity

Comments:

Intuition is as in the White test (a model for the variance).

By focusing on a particular group, if we reject the null wehave a better idea of what causes heterocedasticity.

Accepting the null does not mean there isnt heterocedasticity(why?).

Also a large sample test.

Koenker (1980) has proposed to use nR2A as a test, which is

still valid if errors are non-normal.


http://find/


12/22

yTesting for Heteroskedasticity

Estimation and inference under heteroscedasticity

For simplicity, consider the two variable case

Yi = 1 + 2Xi + ui

where now the error term is heteroskedastic, that is

V(ui) = 2

i , i = 1, . . . , n

We will assume all the other classical assumptions still hold


http://find/


13/22

Testing for Heteroskedasticity

Divide each observation of the linear model by i:

Yii

= 11

i+ 2

Xii

+uii

Yi = 1X

1i + 2X

2i + kX

ki + u

i

Note that V(ui ) = V(ui/i) = 1, then the residuals of thistransformed model are homoscedastic.

Then, if we know 2i , the BLUE is simply the OLS estimator usingthe transformed variables.


OLS Under Heteroskedasticityf


14/22


In our case, the OLS with the transfored model is

2,wls =

ni=1 x

i y

ini=1 x

2i

1,wls = Y 2,wlsXwith Yi = Yi/i, X

i = Xi/i and lowercase letters are deviationsfrom sample means, as usual.

This is the weighted least squares estimator.


OLS Under HeteroskedasticityT i f H k d i i


15/22


The name weighted least squares come from the fact that theestimator can be obtained by solving the following minimizationproblem

min

n

i=1

1

2i e2

i

that is, errors enter the SSR weighted by the inverse of thevariance for each observation: we pay more attention toobservations with smaller variance.


OLS Under HeteroskedasticityT ti f H t k d ti it
http://find/


16/22


Problem: in practice we do not know 2i . This leads to two

strategies1 Use WLS: Pros: estimates will be unbiased and efficient.

Cons: we need to know the variances in advance (or makeassumptions)

2 Keep OLS but change its variance estimator: Think againabout the effects of heteroscedasticity on standard estimationprocedures. OLS is still unbiased though not efficient (notthat bad...). But, S2(XX)1 is biased, which invalidatesinference (this is bad!). Then, a second strategy: keeping

OLS for and look for a valid estimator for its variance. Pros:no assumptions needed, unbiased. Cons: we will loseefficiency with respect to the WLS case (if available).


http://find/


17/22


1) Known variance structure: WLS

Consider our simple two-variable case: Yi = 1 + 2Xi + ui

Strategy: assume some particular forms of heteroscedasticity.

a) V(ui) = 2X2i

2 is an unknown constant. Divide all observations by Xi

YiXi

= 11

Xi+ 2 +

uiXi

Y

i

= 1X

0i

+ 2 + u

i

Note E(ui ) = E(ui/Xi)2 = 2

X2i

X2i

= 2

Errors of the transformed model are homocedastic. Do OLS on thetransformed model!




18/22


We do not need to know 2. What have just divided by thepart of the standard error that varies over observations, thatis, by Xi.

This strategy provides a WLS.Careful with interpretations. The intercept of the transformedmodel is the slope of the original model and that the slope ofthe transformed model is the intercept of the original one.


http://find/


19/22


b) V(u) = 2Xi

YiXi

= 11Xi

+ 2

Xi +uiXi

Yi = 1X

0i + 2X

1i + u

i

As for implementation and interpretation, note that thetransformed model has no intercept, and that the coefficient of thefirst explanatory variable corresponds to the intercept of theoriginal model, and the coefficient of the second variable

corresponds to the slope of the original model.

Problem with these strategies: it is difficult to find an exact formfor heterocedasticity




20/22


2) Unknown variance structure

Alternative strategy: retain OLS (still unbiased though notefficient) and look for valid estimators for its variance. Variance

matrix of OLS under heteroscedasticity can be shown to be:

V(OLS) = (XX)1XX(XX)1

= diag(21

, 22

, . . . , 2n).


http://find/


21/22

g y

White (1980): a consistent estimator for X

X is X

DX,D = diag(e21

, e22

, . . . , e2n), eis OLS residuals

Then, a heteroscedasticity consistent estimator of the variancematrix is:

V(OLS)HC = (XX)1XDX(XX)1

Strategy: use OLS but replace S2(XX)1 by Whites consistentestimator. This strategy is not efficient, but it

does not require assumptions about the structure ofheteroscedasticity.


http://find/


22/22

Summary

Heteroskedasticity makes OLS inefficient and invalidates thestandard estimator of its variance, and hence invalidatesinference (t tests, F tests, etc.).

The WLS estimator is efficient and unbiased but it depends onknowing the variance structure. In practice is seldom available.

In practice it is more common to keep OLS and replace itsvariance estimator by Whites consistent method, which doesnot require any assumptions.

http://find/

gls heteros kedas t i city

Documents