applied econometrics qem meeting 3: heteroskedasticity and

HeteroskedasticityAutocorrelation

Applied Econometrics QEM

Meeting 3: Heteroskedasticity and Autocorrelation

Chapter 8 and 9 in PoE

Michaª Rubaszek

Warsaw School of Economics

Michaª Rubaszek AE QEM: Meeting 3


Outline

1. Heteroskedasticity

2. Autocorrelation



Denition of heteroskedasticity

Heteroskedasticity

The error term of the econometric model is said to be heteroskedastic ifits variance depend on observation

One of the assumptions for the linear regression model washomoskedasticity:

var(yi |xi ) = var(εi ) = σ2

For many models this assumption is violated and the error term isheteroskedastic:

var(yi |xi ) = var(εi ) = σ2i .



Example of heteroskedasticity

• A model in which y is the house value and the only explanatory variable x is itssurface, so that ε = y − α1 − α2x

• It seems intuitive that the probability of getting large positive or negative valuesof ε for large houses is higher than for small ones.

• This means that the variance of ε depends on x

Let us analyze the relationship between (yi ) and (xi ) with the data from thestockton4.wf1 le (1500 houses sold in Stockton during 1996-1998). We start withthe linear LS regression:

yi = −30.1(3.2)

+ 9.2(0.2)

xi

Next, for the squared residuals (e2i ) - which are the proxy for var(εi ) - we estimate theregression:

e2i = −4199.3(881.1)

+ 339.4(50.1)

xi

that show that the surface has a signicant and positive impact on squared residuals



Consequences for LS estimator

The consequence of heteroskedasticity for the LS estimator are twofold:

i. The LS estimator for α is still unbiased and consistent, but no longer eective(there are other unbiased and consistent estimators with a smaller variance)

ii. The LS estimator for σ2 is biased and hence the estimator of the parametersvariance Σ is also biased (condence intervals and hypotheses tests that use thestandard errors, e.g. t-Student and F tests for coecients signicance, aremisleading)

a. The explanation of (i.): in the case of heteroskedasticity it is more reasonable tominimize a weighted sum of squared residuals, where the weights are equal towi = 1/σ2i , so that each observation is equally treated

b. For (ii), it can be shown that for a linear model with one explanatory variableyi = α1 + α2xi + εi the variance of α2 is:

var(α2) =

∑Ni=1

(xi − x)2σ2i(∑Ni=1

(xi − x)2)2.



Derivation for a general case

LS estimator:

b =(∑

xix′i

)−1 (∑xiyi)

Sustitute yi = x ′i β + εi to get:

b = β +(∑

xix′i

)−1 (∑xiεi)

We can calculate the variance:

var(b) = E [(b−β)(b−β)′) =(∑

xix′i

)−1 (∑xiεi)(∑

εix′i

)(∑xix′i

)−1For homoskedastic (and not autocorrelated) error term this simplies to:

var(b) = E [(b − β)(b − β)′) = σ2(∑

xix′i

)−1Michaª Rubaszek AE QEM: Meeting 3


Detecting heteroskedasticity:

Cross-plot of residuals on explanatory variable

The rst method is rather informal and relies on analyzing a cross-plot of residuals (orsquared residuals) from a given regression on the explanatory variable. If there is someclear pattern showing that the variance of residuals increases with the value of theexplanatory variable, we can suspect heteroskedasticity.

Figure: Cross plot of residuals on house surface form example 1




Lagrange Multiplier test for heteroskedasticity

The most popular test of heteroskedasticity checks whether the variance of the errorterm depends on a linear combination of some variables zsi , where the general form ofsuch a relationship is:

var(εi ) = σ2i = h(β1 + β2z2i + . . .+ βSzSi ) = h(β′zi ).

Usually, it it is assumed that function h is linear i.e. h(β′zi ) = β′zi or exponentialh(β′zi ) = exp(β′zi ).The general form of test for heteroskedasticity is to verify the following hypotheses:

H0 :∀ih(β′zi ) = σ2

H1 :∃ih(β′zi ) 6= σ2.

The above hypotheses can be represented in a more testable form, namely:

H0 :∀s>1βs = 0

H1 :∃s>1βs 6= 0.





In the empirical test, we dene the dierence between the squared errorsand its expected value by:

νi = ε2i − σ2i

so that ε2i = σ2i + νi . For the linear function h, substituting to σ2i = β′ziyields:

ε2i = β1 + β2z2i + . . .+ βSzSi + νi .

We can test the null (H0 : ∀s>1βs = 0) with the standard tests ofsignicance for models with multiple variables (F -test or LM-test).However, to do that we need to replace unobservable errors εt withresiduals et .





• The nal question: which variables should be used in the role of explanatoryvariables zi s?

• The most common choice, proposed by Hal White, is to use variables that werepresent in the original regression, its squares and interaction. In regression withtwo explanatory variables x2 and x3, we have:z2 = x2, z3 = x3, z4 = x2

2, z5 = x2

3, z6 = x2x3

For the house-price model e estimate the auxiliary model of the form:

e2i = β1 + β2xi + β3x2

i + νi

The values of the test statistics for the null:

H0 : β2 = β3 = 0

are F (2, 1497) = 33.7 (p = 0.000) and LM = 33.7 (p = 0.000). This indicates thatthere is a problem of heteroskedasticity.



Tackling heteroskedasticity

Three solutions to tackle heteroskedasticity:

1 Change the specication of the model so that the error term washomoskedastic (for example, if we model the value of houses we canregress the price of squared meter on various determinants instead ofhouse price)

2 Use the estimator that accounts for heteroskedasticity, e.g. FeasibleGeneralized Least Squares

3 Estimate the parameters of the model with the LS and account forheteroskedasticity while calculating standard errors

We discuss that last two options below.



Tackling heteroskedasticity:

Generalized Least Squares

Let us consider a linear, heteroskedastic model:

yi = α′xi + εi , var(εi ) = σ2i .

We can transform the model by dividing both sides by σi :

yi

σi= α′

xi

σi+εi

σi; var

(εi

σi

)= 1.

Let y∗i = yi/σi , x∗i = xi/σi and ε

∗i = εi/σi , so that the transformed model:

y∗i = α′x∗i + ε∗i , var(ε∗i ) = 1.

is homoskedastic and can be estimated with LS (LS estimator is unbiased, consistentand eective). This estimator is known as Generalized Least Squares (GLS) estimator.

Remark: in a model with a constant (x1i = 1) the transformed variable is x∗1i = 1/σi .

That means that one have to be very careful to replace a constant from the originalregression with x∗

1i .




Generalized Least Squares

So far we have assumed that the values σi are known. In practise,however, this is not the case and these values have to be estimated. Thiscan be done auxiliary regression, so that:

σ2i = h(β′zi )

Important: it is required that σ2i are positive, which can be achieved byassuming exponential form of h, hence:

lnσ2i = β′zi .




feasible Generalized Least Squares

The following feasible algorithm can be applied (feasible GLS):

a. Estimate the parameters of regression yi = α′xi + εi with LS

b. Compute squared residuals e2i .

c. Estimate the parameters of regression ln e2i = β′zi + νi .

d. Compute variance estimates σ2i = exp(β′zi ).

e. Apply (d.) to calculate transformed variables y∗i and x∗i .

f. Estimate the parameters of transformed regression y∗i = α′x∗i + ε∗iwith LS.




feasible Generalized Least Squares - example

Let us apply the following algorithm to the house price model. The LS estimates were:

yi = −30.1(3.2)

+ 9.2(0.2)

xi ,

We generate the logs of squared residuals and regress it on the variable describinghouse surface xi . The estimates are:

ln(e2)i = 2.34(0.19)

+ 0.17(0.01)

xi ,

The estimation results for the transformed model:

y∗i = 5.7(3.1)

1

σi+ 6.9

(0.2)x∗i .

are visibly dierent from the LS estimates. Moreover, the results of the LM test:F = 0.14 (p = 0.97) indicate that the transformed error term is homoskedastic.




Heteroskedasticity consistent standard errors

The last option was to use LS and account for the fact that the LS estimator of thecovariance matrix Σ = cov(α) is biased. For heteroskedastic model the consistentestimator for Σ is:

= (N∑i=1

xix′i )−1(

N∑i=1

σ2i xix′i )(

N∑i=1

xix′i )−1.

The standard errors computed with the above formula are known asheteroskedasticity-consistent (HC) standard errors or heteroskedasticity robuststandard errors (notice that for constant σ2i this expression shrinks to the formulafrom the previous meeting).

Since σ2i are not observable, the White standard errors are obtained by substituting σ2iwith squared residuals e2i and adjusting for the degrees of freedom N/(N − K). Theresulting formula is:

Σ =N

N − K(

N∑i=1

xix′i )−1(

N∑i=1

e2i xix′i )(

N∑i=1

xix′i )−1.




Heteroskedasticity consistent standard errors - example

Let us apply the following algorithm to the house price model. The LSestimates were:

yi = −30.1(3.2)

+ 9.2(0.2)

xi ,

The application of formula for White standard errors yields:

yi = −30.1(6.2)

+ 9.2(0.4)

xi .

The heteroskedasticity robust standard errors are about twice larger thanthe unadjusted ones. Of course, the estimates of model coecients areexactly the same.



Outline

1. Heteroskedasticity

2. Autocorrelation



Autocorrelation: denition

Autocorrelation = serial correlation

The error term of the econometric model is said to be autocorrelated ifits realizations from dierent observations are correlated

One of the assumptions for the linear regression model was:

cov(εt , εs) = 0 for all s 6= t

For many time series models, however, this assumption is violatedbecause changes in many macro-variables are gradual, hence their valuesin the current period depend on what happened in the previous one. Forexample, if the level of interbank interest rates is high today, it is verylikely that it will remain elevated tomorrow.



Consequences for LS estimator

The consequence of autocorrelation for LS estimator are exactly the sameas the consequences of heteroskedasticity, i.e.:

i. The LS estimator for α is still unbiased and consistent, but nolonger eective.

ii. The LS estimator for σ2 is biased and hence the estimator of theparameters variance Σ is also biased.



Some derivation

LS estimator:

b =(∑

xix′i

)−1 (∑xiyi)

Substitute yi = x ′i β + εi to get:

b = β +(∑

xix′i

)−1 (∑xiεi)

We can calculate the variance:

var(b) = E [(b−β)(b−β)′) =(∑

xix′i

)−1 (∑xiεi)(∑

εix′i

)(∑xix′i

)−1For autocorrelated errors this does not simplies to:

var(b) = E [(b − β)(b − β)′) = σ2(∑

xix′i

)−1Michaª Rubaszek AE QEM: Meeting 3


Detecting autocorrelation

Two methods of detecting autocorrelation:

1 Analysis of the sample autocorrelation of model residuals(correlogranm)

2 Formal test for autocorrelation




Correlogram

• Autocorrelation coecient of k-th order:

ρk =cov(yt , yt−k )

var(yt).

• Sample autocorrelation coecient of k-th order:

ρk =cov(yt , yt−k )

var(yt)=

∑Tt=k+1

(yt − y)(yt−k − y)∑Tt=1

(yt − y)2.

• Test for autocorrelation coecient signicance:

H0 :ρk = 0

H1 :ρk 6= 0

Under the null, the test statistic Z =√T ρk has normal distribution N (0, 1).




Correlogram

Correlogram

A graph of the sample autocorrelations ρk versus the time lags k , oftensupplemented by the 95% condence interval (bands are equal to±1.96/

√T )




Correlogram - example

Model for the relationship between the interest rate (it), year-on-year ination (πt)and year-on-year GDP growth rate (yt) (data from the MPdata.wf1 le cover theperiod 1974-2011 and relate to the U.S. economy). The LS estimates are:

it = 1.41(0.46)

+ 0.97(0.07)

πt + 0.26(0.09)

yt .

Figure: Correlogram of residuals of the interest rate model




Lagrange Multiplier test for autocorrelation

The algorithm of LM test for autocorrelation of order k :

1 Calculate the residuals of the original regression: et = yt − α′xt .2 Estimate the auxiliary regression:

et = γxt + ρ1et−1 + ρ2et−2 + . . . ρket−k + νt .

3 Test the null of no autocorrelation:

H0 : ρ1 = 0 ∧ ρ2 = 0 ∧ . . . ∧ ρk = 0

H1 : ρ1 6= 0 ∨ ρ2 6= 0 ∨ . . . ∨ ρk 6= 0

with the standard tests of signicance for models with multiplevariables (F -test or LM-test).




Lagrange Multiplier test for autocorrelation - example

For the model analyzing the relationship between the interest rate (it),year-on-year ination (πt) and year-on-year GDP growth rate (yt) in theU.S. we have estimated the parameters of the auxiliary model for k = 1.The results are:

et = −0.09(0.20)

+ 0.01(0.03)

πt + 0.00(0.04)

yt + 0.91(0.04)

et−1.

The values of the tests statistics of the null ρ1 = 0 at:F (1, 148) = 608.7 (p = 0.000) and LM = 122.7 (p = 0.000)unambiguously indicate that there is a problem of autocorrelation in thismodel.



Tackling autocorrelation

Three methods to tackle autocorrelation:

1 Change the specication of the model so that the error term was notautocorrelated.

2 Use an estimator that accounts for autocorrelation, e.g. NonlinearLeast Squares or Cochrane-Orcutt iterative method.

3 Estimate the parameters of the model with the LS and account forautocorrelation while calculating standard errors (HAC standarderrors).

We discuss these options below.




HAC consistent standard errors

The LS estimator of the covariance matrix Σ = cov(α) is biased. For heteroskedasticand autocorrelated model the consistent estimator for this covariance matrix is:

Σ = (T∑

t=1

xtx′t)−1ΩLR(

T∑t=1

xtx′t)−1,

where 1

TΩLR is the estimator of the long-run covariance matrix of the error term et :

ΩLR =T∑

t=1

e2t xtx′t +

S∑s=1

ws

T∑t=s+1

etet−s(xtx′t−s + xt−sx

′t).

Here S is a truncation parameter and ws = 1− s/(S + 1) is given by the Barlettweight function.

The standard errors computed with the above formula are known as heteroskedasticityand autocorrelation consistent (HAC) standard errors or Newey-West standard errors(after the authors of the formula).

Remark: in case of no autocorrelation, the second component drops out, and the HACstandard errors shrink to the HC standard errors.




HAC consistent standard errors

The LS estimates of the interest rate model were:

it = 1.41(0.46)

+ 0.97(0.07)

πt + 0.26(0.09)

yt .

The application of HAC formula yields:

it = 1.41(0.74)

+ 0.97(0.12)

πt + 0.26(0.17)

yt .

The HAC standard errors are almost twice larger than the unadjustedones. Of course, the estimates of model coecients are exactly the same.




Nonlinear Least Squares

Let us consider a following autocorrelated model:

yt = α′xt + εt , εt = ρεt−1 + νt

By noticing that εt = yt − α′xt , we can rewrite it as:

yt = ρyt−1 + α′xt − ρα′xt−1 + νt .

The above model is a linear function of the explanatory variables, but anonlinear function of the parameters. The usual LS estimation cannot beapplied. However, the sum of squared errors numerically, and we willhave Nonlinear Least Squares (NLS) estimates.




iterative Cochrane-Orcutt procedure

The alternative to the NLS estimator is the iterative Cochrane-Orcuttprocedure:

1 LS estimate α in the original model:

yt = α′xt + εt

2 Calculate the residuals and estimate ρ

3 Conditional on the estimate of ρ, estimate α with the model:

yt = ρyt−1 + α′xt − ρα′xt−1 + νt .

4 Repeat (2) and (3) until convergence

The estimates obtained with the Cochrane-Orcutt procedure converge tothe NLS estimates.




NLS estimates

The LS estimates of the interest rate model were:

it = 1.41(0.46)

+ 0.97(0.07)

πt + 0.26(0.09)

yt .

The NLS we estimates

it = (1−0.96) 3.31(2.06)

+ 0.96(0.03)

it−1+ 0.36(0.10)

πt−0.96×0.36πt−1+ 0.19(0.19)

yt−0.96×0.19yt−1.

are visibly dierent




Changing the specication of the model

The last method (the recommended one) is to estimate a model ofspecication that takes into account dynamic relations between variables,e.g. the AutoRegressive Distributed Lag (ARDL) model. The generalform of the ARDL(Q,K,P) model with K independent variables thatenters the regression with lags up to P and Q lags:

yt = α0 +Q∑

q=1

ρyt−q +K∑

k=1

P∑p=0

αkpxk,t−p + εt .

If the values of P, K and Q are correctly chosen, the error term εt shouldnot be serially correlated.




Changing the specication of the model - example

For the interest rate model we estimate the ARDL(1,2,1) model of theform:

it = α0 + ρit−1 + α11πt + α12πt−1 + α21yt + α22yt−1 + νt .

The estimation results are:

it = −0.35(0.19)

+ 0.89(0.03)

it−1 + 0.27(0.10)

πt − 0.12(0.11)

πt−1 + 0.07(0.26)

yt − 0.14(0.07)

yt−1.

The tests statistics: F (1, 144) = 2.68 (p = 0.10) andLM = 2.76 (p = 0.10) indicate that we cannot reject the null of noautocorrelation of order one



Multiplier analysis

Consider a Distributed Lags model:

yt = α + β0xt + β1xt−1 + . . .+ εt

We can calculate the following multipliers:

Delay multiplier: βs = ∂yt∂xt−s

Interim multiplier:∑s

j=0βj

Total multiplier:∑s

j=0βj

Interpretation?



Multiplier analysis

Consider an Autoregressive Distributed Lags - ARDL(p,q) - model:

yt = δ + θ1yt−1 + . . .+ θpxt−p + δ0xt + δ1xt−1 + . . .+ δqxt−q + εt

Let L be the lag operator so that Lsyt = yt−s and:

θ(L) = 1− θ1L− . . .− θpLp

δ(L) = δ + δ0 + δ1L + . . .+ δqLq

We have:

θ(L)yt = δ(L)xt + εt ⇔ yt = θ(L)−1δ(L) + θ(L)−1εt

this can be transformed into DL(∞) model:

yt = α + β0xt + β1xt−1 + . . .+ εt



Multiplier analysis

For ARDL(1,0) model:

yt = δ + θyt−1 + γxtεt

the relationship is:

yt = α + β0xt + β1xt−1 + . . .+ εt

α = δ/(1− θ)

βs = γ ∗ θs


applied econometrics qem meeting 3: heteroskedasticity and

Documents