charles university

Charles University

FSV UK

STAKAN III

Institute of Economic Studies

Faculty of Social Sciences Institute of Economic Studies

Faculty of Social Sciences

Jan Ámos VíšekJan Ámos Víšek

Econometrics Econometrics

Tuesday, 12.30 – 13.50

Charles University

Third Lecture (summer term)

Plan of the whole year

Regression models for various situations

● Division according to character of data (with respect to time):

* Cross-sectional data (winter term)

* Panel data (summer term)



● Division according to character of variables

* Continuous response (and nearly arbitrary) explanatory variables (winter and part of summer term)

* Qualitative and limited response (and nearly arbitrary) explanatory variables (summer term)



● Division according to contamination of data

* Classical methods, neglecting contamination (winter and most of of summer term)

* Robust methods (three lectures in summer term)

Schedule of today talk

● The Generalized Least Squares

● Modeling time series by AR(p) and MA(q)

* Stationarity, Dickey-Fuller tests of unit roots

* Convertibility

* Moments and covariance matrices

The Generalized Least Squares

Let us assume that ,T - regular,

i.e. homoscedasticity is broken.

- regular and symmetric IPP:P T

1T1 PP

and put

P~,PXX~

,PYY~

multiplying the basic model from the left 0XYby

.

P ~X

~PXPYY

~ 00 .

IPPPP~~ TTTT

For we have ~

,

i.e. we have reached homoscedasticity. Then

PYPXPXPXY~

X~

X~

X~~ TT1TTT1TSLO

.

Recalling that

1T1 PP PP T1 ,

i. e. SLG1T11TSLO ˆYXXX

~

Generalized Least Squares

The Generalized Least Squares continued

The Generalized Least Squares continued

GLSWhat is problem with application of ?

contains of unknown elements n2

)1n(n

which cannot be estimated due to the fact that we have

at hand only observations ! n

above the diagonal on the diagonal

But sometimes we know the structure of and moreover it can be determined by a few parameters !!

Modeling time series by stochastic models - Box-Jenkins methodology

Box, G. E. P., G. M. Jenkins: Time Series Analysis, Forecasting and Control. Holden Day, San Francisco, 1970.

Judge, G.,G., W.,E. Griffiths, R.C. Hill, H. Lutkepohl, T.,C. Lee: The Theory and Practice of Econometrics. J.Wiley and Sons, New York, 1985.

Cipra, T.: Analýza časových řad s aplikacemi v ekonomii. SNTL/ALFA, Praha, 1986.

Brockwell, P. J., R. A. Davis: Time Series: Theory and Methods. Springer Verlag, New York, 1991.

Recommended

Modeling time series by AR(p) and MA(q)

Let be a sequence of i.i.d. r.v.’s. with zero mean and

tptp2t21t1t v...

1ttv

with . The sequence of r.v.’s given by (1) is called 10

(1)

autoregressive process of order p and denoted by AR(p).

Put also

qtq2t21t1tt vvvv

(3)

The sequence of r.v.’s given by (2) is called moving -average

process of order q and denoted MA(q). Finally, put

qtq1t1tptp1t1t vvv...

continued

(2)

variance equal to . Then put 2v


The sequence of r.v.’s given by (3) is called autoregressive

moving -average process (of order (p,q)) and denoted by

ARMA(p,q).

continued

If the process is ARMA(p,q), then 1t1ttt)1(d

1ttthe original process is called the integrated

autoregressive moving-average process (of order (p,1,q) )

and denoted by ARIMA(p,1,q).

Modeling time series by AR(p) and MA(q) continued

If the process is ARMA(p,q), then the original 1tt)h(d

process is called the integrated autoregressive

moving-average process of order (p,h,q) and denoted

1t1t

)1h(t

)1h(t

)h( ddd Put

1tt

by ARIMA(p,1,q).Assumption:

1ttv is i.i.d. r.v.’s with and . 0v t 2v

2tv


A very first question is, of course, how far the autoregressive

processes can be expressed as moving average and vice versa ?

t1t2tt1tt vvv

t1t2t3t2

t1t2t2 vvvvv

.....vvvvvvv 3t3

2t2

1ttt1t2t2

3t3

For simplicity, consider AR(1) :

)(MA)1(AR

( Notice that the “dual” description is much more complicated. )

continued

So we may say that is “dual” to . )(MA )1(AR


0.....vvvv 3t3

2t2

1ttt

It immediately gives two results:

23t3

2t2

1tt2tt .....vvvv)var(

.....vvvv 23t

622t

421t

22t

.....vv...vvvv rtr

sts

2t2

t1tt

.....2v

62v

42v

22v

2definition2

2v6422

v1

.....1

Firstly (moments of ) t

for 1


Secondly (conditions of stationarity for ) 1tt

Let’s recall stationarity

DEFINITION

ttThe sequence of r.v.’s is called stationary if

)Nj(,...}),2,1,0,1,2{...,,...,,(),Nk( k21

)x,,x,x(P kk2211

)x,,x,x(P kjk2j21j1

alternatively

(this definition is easier to understand)


DEFINITION

ttThe sequence of r.v.’s is called stationary if

)x,,x,x(,,,

F k21

k21

)x,,x,x(j,,j,j

F k21

k21

)Nj(,...}),2,1,0,1,2{...,,...,,(),Nk( k21

(this definition is usually employed) 1ttOf course, our sequence is not infinite on both sides,

hence the definition is to be applied in a bit modifies way.

n0tt

Remark Assuming sequence to be stationary, of course requires

some modification of the definitions.


.....vvvv 3t3

2t2

1ttt

Returning to

,

we immediately observe that only for the variance is

.....1)var( 6422vt

1

finite and hence it has any sense to speak about some distribution.

Now, return to

,

consider any k-tuple of indices and corresponding

k21,,,

k21 ,...,,

k-tuple of r.v.’s and find ( it is sufficient in mind )

the structure of r.v.’s which generated . tvk21

,,,


the both structures of r.v.’s are the same but shifted about tv

j. Since are i.i.d., the d.f. of both k-tuples, tv

k21,,, jjj k21

,,, and ,

are the same ( for any fixed j ). So, if , the sequence

,....2,1,1 tvttt ,

is stationary.

1

Finally, do the same for and find that jjj k21,,,


We may take an analogy to the equation

t1tt v

has to be in absolute value larger that 1.

z1

Then , if , the solution of the polynomial ( in z )1

Let us look for a general condition for stationarity.

the polynomial .

z1

In other words, if solution of , is larger than 1, z1

t ‘s are stationary.


Similarly (and alternatively), the solution of the equation 0x

has to be in absolute value less that 1.t1tt v

which can be viewed as an analogy to

So again, if solution of , is less than 1,

t ‘s are stationary.

0x


For general we conclude, in analogy with )1(p

tptp2t21t1t v... ,

that all roots of the polynomial

pp

221 z...zz1

have to be in absolute value ( notice that they are generally complex numbers ) larger than 1.

( 4 )


Again alternatively,

0...xxx p2p

21p

1p

have to be in absolute value less than 1.

( 5 )

“Conditions of stationarity”

( of course, they are equivalent).

tptp2t21t1t v...

all roots of the polynomial

The conditions ( 4) and (5) are called


0ˆ...xˆxˆx p2p

21p

1p

We have not at hand ‘s but “only” ‘s, so that we solve tt

and obtain, say, instead of . But p21 x,,x,x p21 x,,x,x

even if , we can have . Hence 1xpj1

max j

1xpj1

max j

we have to test whether statistically significantly. 1xpj1

max j

The test is known as the “Test on unit roots”. The best known

is Dickey-Fuller test.


Dickey-Fuller test – for AR( 1 )

t1tt v

t-test of significance that . Since and 1 t 1t

are not independent, we cannot use “classical” student test.

D. A. Dickey and W. A. Fuller (1979) made Monte Carlo study

and tabulated the critical values. An alternative

Augmented Dickey-Fuller test – for AR( 1 )

t1t*

t1t1ttt vv)1(

and test of significance whether . 0*

t1tt v

1t2t1t v t1tt rˆˆ

1t2t1t rˆˆ


We already know that for AR( 1)

0t and for 2

2

2v

t1

)var(

1t

That is why we define (frequently)

0v1

1t121

and 01 and .

2v1 )var(

and 2

2

2v

11

)var(

.

Moreover,

.....)vvvv(},cov{ 3t3

2t2

1ttst

.....)vvvv( 3s3

2s2

1ss


2

st2v

4st2stst2vst

1},cov{

.

So the covariance matrix is given as

1,,,

,,,

,,1,

,,,1

1}cov{

1TT

2T2

1T

T

2

2vT

.


and the inversion as

PPT

vv

2

2

2

2

2

21 1

1,,,0,0

,1,

0,0,,,0

0,0,,1,

0,0,,,1

1

where

1,,,0,0

0,1,

0,0,,,0

0,0,,1,

0,0,,0,1

P

2

( We shall need it later. )


It is easy to verify that the inversion matrix given on the previous

slide is really inversion of . We have for the product of k-th1

0,,,1,,0,,0,0 2

line of and of the ( transposed ) j-th row of ( )

2

jT

2

1kj

2

kj

2

1kj

2

2kj

2

2j

2

1j

1,,

1,

1,

1,

1,,

1,

1

011

)1(

1 2

kj

2

kj2

2

2kj

1st coor. 2nd coor. coor.

thk coor.

1k th coor.1k th

kj

kj Similarly for .

.


For , i.e. for the product of k-th line of 1

0,,,1,,0,,0,0 2

and of the ( transposed ) k-th row of we have

2

jT

2222

2

2

2k

2

1k

1,,

1,

1

1,

1,

1,,

1,

1

111

)1(

1 2

2

2

2

2

2

1st coor. 2nd coor. coor.

thk coor.

1k th coor.1k th

kj

.

PP T1 Along similar lines we verify that .


Let us move to MA( 1 ).

1ttt vv and

2t1t1t vv 2t1t1t vv

3t2t2t vv

t2t2

1t2t1t1t1t vv)v(v

,

,

but and hence

t3t3

2t2

1t1t vv

t5t5

4t4

3t3

2t2

1t vv etc.

So, )(AR)1(MA .( Notice that the “dual” description is again much more complicated. )


An analogy (or counterpart?) to the condition of stationarity for

AR( p ), there is a condition of invertibility of MA( q ) which reads:

The condition has following sense:

DEFINITION

Let L be operator of the back-shift, i.e. for any we have t1ttL .

( The letter “L” went from “lagged” value of .) t

All roots of the polynomial

0zzz1 qq

221

have to be outside unit circle.


We shall use the operator L (rather formally) in the following way

1ttL .

Returning to the MA( 1 ) and changing the sign of (but only

t1ttt v)L1(vv

)L1(v t

t

and then

.

Assuming now that , we can the sum of the geometric series, 1

namely , write as L1

1

443322 LLLL1 ,

i.e. t443322

t LLLL1v

this moment of explanation of condition of invertibility), we have


4t4

3t3

2t2

1tttv and finally

1

.

During the derivation of the result, we have needed ,

i.e. solution of 0z1

has to be larger than 1.

Unlike for AR( p ), for MA( q ) we can easy ( without any “dual”

representation ) evaluate moments and covariance matrix.

Clearly, 0vvvv qtq2t21t1tt

2qtq2t21t1t2t vvvv

and


2qt

2q

22t

22

21t

21

2t vvvv

qttq2tt21tt1 vvvvvv(2

qt1tq13t1t312t1t21 vvvvvv

qt1qtq1q vv

2q

22

21

2v 1

In a similar way ( assume )

)vvvv(},cov{ qtq2t21t1tst

)vvvv( qsq2s21s1s

st


2qtstqq

21s11st

2sst vvv

stqq11stst2v

for qst

0

,

otherwise.

Specifying it for MA( 1 )

0t

and

)1()var( 22v

2tt ,

2v1tt },cov{ .


There are at least two or three problems:

Why we study both AR( p ) and MA( q ), when we can convert

Firstly

How to recognize that there is some dependence in the series ?

Secondly

Which type of dependency took place? How large p or q is ?

Thirdly

on and vice versa, i.e. on ? )(MA )p(AR

We’ll answer them successively in the next lecture.

)q(MA )(AR

What is to be learnt from this lecture for exam ?

All what you need is on http://samba.fsv.cuni.cz/~visek/

• The Generalized Least Squares

• AR(p), MA(q), ARMA(p,q), ARIMA(p,h,q)

• Stationarity - conditions for stationarity, - Dickey-Fuller test – for AR( 1 ), - augmented Dickey-Fuller test – for AR( 1 )

• Moments and covariance matrices

• Convertibility

charles university

Documents

average process of order

squares modeling time

original process

time series analysis

denoted maq

summer term schedule

panel data summer term

sequence of r