Download - Charles University
Charles University
FSV UK
STAKAN III
Institute of Economic Studies
Faculty of Social Sciences Institute of Economic Studies
Faculty of Social Sciences
Jan Ámos VíšekJan Ámos Víšek
Econometrics Econometrics
Tuesday, 12.30 – 13.50
Charles University
Third Lecture (summer term)
Plan of the whole year
Regression models for various situations
● Division according to character of data (with respect to time):
* Cross-sectional data (winter term)
* Panel data (summer term)
Plan of the whole year
Regression models for various situations
● Division according to character of variables
* Continuous response (and nearly arbitrary) explanatory variables (winter and part of summer term)
* Qualitative and limited response (and nearly arbitrary) explanatory variables (summer term)
Plan of the whole year
Regression models for various situations
● Division according to contamination of data
* Classical methods, neglecting contamination (winter and most of of summer term)
* Robust methods (three lectures in summer term)
Schedule of today talk
● The Generalized Least Squares
● Modeling time series by AR(p) and MA(q)
* Stationarity, Dickey-Fuller tests of unit roots
* Convertibility
* Moments and covariance matrices
The Generalized Least Squares
Let us assume that ,T - regular,
i.e. homoscedasticity is broken.
- regular and symmetric IPP:P T
1T1 PP
and put
P~,PXX~
,PYY~
multiplying the basic model from the left 0XYby
.
P ~X
~PXPYY
~ 00 .
IPPPP~~ TTTT
For we have ~
,
i.e. we have reached homoscedasticity. Then
PYPXPXPXY~
X~
X~
X~~ TT1TTT1TSLO
.
Recalling that
1T1 PP PP T1 ,
i. e. SLG1T11TSLO ˆYXXX
~
Generalized Least Squares
The Generalized Least Squares continued
The Generalized Least Squares continued
GLSWhat is problem with application of ?
contains of unknown elements n2
)1n(n
which cannot be estimated due to the fact that we have
at hand only observations ! n
above the diagonal on the diagonal
But sometimes we know the structure of and moreover it can be determined by a few parameters !!
Modeling time series by stochastic models - Box-Jenkins methodology
Box, G. E. P., G. M. Jenkins: Time Series Analysis, Forecasting and Control. Holden Day, San Francisco, 1970.
Judge, G.,G., W.,E. Griffiths, R.C. Hill, H. Lutkepohl, T.,C. Lee: The Theory and Practice of Econometrics. J.Wiley and Sons, New York, 1985.
Cipra, T.: Analýza časových řad s aplikacemi v ekonomii. SNTL/ALFA, Praha, 1986.
Brockwell, P. J., R. A. Davis: Time Series: Theory and Methods. Springer Verlag, New York, 1991.
Recommended
Modeling time series by AR(p) and MA(q)
Let be a sequence of i.i.d. r.v.’s. with zero mean and
tptp2t21t1t v...
1ttv
with . The sequence of r.v.’s given by (1) is called 10
(1)
autoregressive process of order p and denoted by AR(p).
Put also
qtq2t21t1tt vvvv
(3)
The sequence of r.v.’s given by (2) is called moving -average
process of order q and denoted MA(q). Finally, put
qtq1t1tptp1t1t vvv...
continued
(2)
variance equal to . Then put 2v
Modeling time series by AR(p) and MA(q)
The sequence of r.v.’s given by (3) is called autoregressive
moving -average process (of order (p,q)) and denoted by
ARMA(p,q).
continued
If the process is ARMA(p,q), then 1t1ttt)1(d
1ttthe original process is called the integrated
autoregressive moving-average process (of order (p,1,q) )
and denoted by ARIMA(p,1,q).
Modeling time series by AR(p) and MA(q) continued
If the process is ARMA(p,q), then the original 1tt)h(d
process is called the integrated autoregressive
moving-average process of order (p,h,q) and denoted
1t1t
)1h(t
)1h(t
)h( ddd Put
1tt
by ARIMA(p,1,q).Assumption:
1ttv is i.i.d. r.v.’s with and . 0v t 2v
2tv
Modeling time series by AR(p) and MA(q)
A very first question is, of course, how far the autoregressive
processes can be expressed as moving average and vice versa ?
t1t2tt1tt vvv
t1t2t3t2
t1t2t2 vvvvv
.....vvvvvvv 3t3
2t2
1ttt1t2t2
3t3
For simplicity, consider AR(1) :
)(MA)1(AR
( Notice that the “dual” description is much more complicated. )
continued
So we may say that is “dual” to . )(MA )1(AR
Modeling time series by AR(p) and MA(q) continued
0.....vvvv 3t3
2t2
1ttt
It immediately gives two results:
23t3
2t2
1tt2tt .....vvvv)var(
.....vvvv 23t
622t
421t
22t
.....vv...vvvv rtr
sts
2t2
t1tt
.....2v
62v
42v
22v
2definition2
2v6422
v1
.....1
Firstly (moments of ) t
for 1
Modeling time series by AR(p) and MA(q) continued
Secondly (conditions of stationarity for ) 1tt
Let’s recall stationarity
DEFINITION
ttThe sequence of r.v.’s is called stationary if
)Nj(,...}),2,1,0,1,2{...,,...,,(),Nk( k21
)x,,x,x(P kk2211
)x,,x,x(P kjk2j21j1
alternatively
(this definition is easier to understand)
Modeling time series by AR(p) and MA(q) continued
DEFINITION
ttThe sequence of r.v.’s is called stationary if
)x,,x,x(,,,
F k21
k21
)x,,x,x(j,,j,j
F k21
k21
)Nj(,...}),2,1,0,1,2{...,,...,,(),Nk( k21
(this definition is usually employed) 1ttOf course, our sequence is not infinite on both sides,
hence the definition is to be applied in a bit modifies way.
n0tt
Remark Assuming sequence to be stationary, of course requires
some modification of the definitions.
Modeling time series by AR(p) and MA(q) continued
.....vvvv 3t3
2t2
1ttt
Returning to
,
we immediately observe that only for the variance is
.....1)var( 6422vt
1
finite and hence it has any sense to speak about some distribution.
Now, return to
,
consider any k-tuple of indices and corresponding
k21,,,
k21 ,...,,
k-tuple of r.v.’s and find ( it is sufficient in mind )
the structure of r.v.’s which generated . tvk21
,,,
Modeling time series by AR(p) and MA(q) continued
the both structures of r.v.’s are the same but shifted about tv
j. Since are i.i.d., the d.f. of both k-tuples, tv
k21,,, jjj k21
,,, and ,
are the same ( for any fixed j ). So, if , the sequence
,....2,1,1 tvttt ,
is stationary.
1
Finally, do the same for and find that jjj k21,,,
Modeling time series by AR(p) and MA(q) continued
We may take an analogy to the equation
t1tt v
has to be in absolute value larger that 1.
z1
Then , if , the solution of the polynomial ( in z )1
Let us look for a general condition for stationarity.
the polynomial .
z1
In other words, if solution of , is larger than 1, z1
t ‘s are stationary.
Modeling time series by AR(p) and MA(q) continued
Similarly (and alternatively), the solution of the equation 0x
has to be in absolute value less that 1.t1tt v
which can be viewed as an analogy to
So again, if solution of , is less than 1,
t ‘s are stationary.
0x
Modeling time series by AR(p) and MA(q) continued
For general we conclude, in analogy with )1(p
tptp2t21t1t v... ,
that all roots of the polynomial
pp
221 z...zz1
have to be in absolute value ( notice that they are generally complex numbers ) larger than 1.
( 4 )
Modeling time series by AR(p) and MA(q) continued
Again alternatively,
0...xxx p2p
21p
1p
have to be in absolute value less than 1.
( 5 )
“Conditions of stationarity”
( of course, they are equivalent).
tptp2t21t1t v...
all roots of the polynomial
The conditions ( 4) and (5) are called
Modeling time series by AR(p) and MA(q) continued
0ˆ...xˆxˆx p2p
21p
1p
We have not at hand ‘s but “only” ‘s, so that we solve tt
and obtain, say, instead of . But p21 x,,x,x p21 x,,x,x
even if , we can have . Hence 1xpj1
max j
1xpj1
max j
we have to test whether statistically significantly. 1xpj1
max j
The test is known as the “Test on unit roots”. The best known
is Dickey-Fuller test.
Modeling time series by AR(p) and MA(q) continued
Dickey-Fuller test – for AR( 1 )
t1tt v
t-test of significance that . Since and 1 t 1t
are not independent, we cannot use “classical” student test.
D. A. Dickey and W. A. Fuller (1979) made Monte Carlo study
and tabulated the critical values. An alternative
Augmented Dickey-Fuller test – for AR( 1 )
t1t*
t1t1ttt vv)1(
and test of significance whether . 0*
t1tt v
1t2t1t v t1tt rˆˆ
1t2t1t rˆˆ
Modeling time series by AR(p) and MA(q) continued
We already know that for AR( 1)
0t and for 2
2
2v
t1
)var(
1t
That is why we define (frequently)
0v1
1t121
and 01 and .
2v1 )var(
and 2
2
2v
11
)var(
.
Moreover,
.....)vvvv(},cov{ 3t3
2t2
1ttst
.....)vvvv( 3s3
2s2
1ss
Modeling time series by AR(p) and MA(q) continued
2
st2v
4st2stst2vst
1},cov{
.
So the covariance matrix is given as
1,,,
,,,
,,1,
,,,1
1}cov{
1TT
2T2
1T
T
2
2vT
.
Modeling time series by AR(p) and MA(q) continued
and the inversion as
PPT
vv
2
2
2
2
2
21 1
1,,,0,0
,1,
0,0,,,0
0,0,,1,
0,0,,,1
1
where
1,,,0,0
0,1,
0,0,,,0
0,0,,1,
0,0,,0,1
P
2
( We shall need it later. )
Modeling time series by AR(p) and MA(q) continued
It is easy to verify that the inversion matrix given on the previous
slide is really inversion of . We have for the product of k-th1
0,,,1,,0,,0,0 2
line of and of the ( transposed ) j-th row of ( )
2
jT
2
1kj
2
kj
2
1kj
2
2kj
2
2j
2
1j
1,,
1,
1,
1,
1,,
1,
1
011
)1(
1 2
kj
2
kj2
2
2kj
1st coor. 2nd coor. coor.
thk coor.
1k th coor.1k th
kj
kj Similarly for .
.
Modeling time series by AR(p) and MA(q) continued
For , i.e. for the product of k-th line of 1
0,,,1,,0,,0,0 2
and of the ( transposed ) k-th row of we have
2
jT
2222
2
2
2k
2
1k
1,,
1,
1
1,
1,
1,,
1,
1
111
)1(
1 2
2
2
2
2
2
1st coor. 2nd coor. coor.
thk coor.
1k th coor.1k th
kj
.
PP T1 Along similar lines we verify that .
Modeling time series by AR(p) and MA(q) continued
Let us move to MA( 1 ).
1ttt vv and
2t1t1t vv 2t1t1t vv
3t2t2t vv
t2t2
1t2t1t1t1t vv)v(v
,
,
but and hence
t3t3
2t2
1t1t vv
t5t5
4t4
3t3
2t2
1t vv etc.
So, )(AR)1(MA .( Notice that the “dual” description is again much more complicated. )
Modeling time series by AR(p) and MA(q) continued
An analogy (or counterpart?) to the condition of stationarity for
AR( p ), there is a condition of invertibility of MA( q ) which reads:
The condition has following sense:
DEFINITION
Let L be operator of the back-shift, i.e. for any we have t1ttL .
( The letter “L” went from “lagged” value of .) t
All roots of the polynomial
0zzz1 qq
221
have to be outside unit circle.
Modeling time series by AR(p) and MA(q) continued
We shall use the operator L (rather formally) in the following way
1ttL .
Returning to the MA( 1 ) and changing the sign of (but only
t1ttt v)L1(vv
)L1(v t
t
and then
.
Assuming now that , we can the sum of the geometric series, 1
namely , write as L1
1
443322 LLLL1 ,
i.e. t443322
t LLLL1v
this moment of explanation of condition of invertibility), we have
Modeling time series by AR(p) and MA(q) continued
4t4
3t3
2t2
1tttv and finally
1
.
During the derivation of the result, we have needed ,
i.e. solution of 0z1
has to be larger than 1.
Unlike for AR( p ), for MA( q ) we can easy ( without any “dual”
representation ) evaluate moments and covariance matrix.
Clearly, 0vvvv qtq2t21t1tt
2qtq2t21t1t2t vvvv
and
Modeling time series by AR(p) and MA(q) continued
2qt
2q
22t
22
21t
21
2t vvvv
qttq2tt21tt1 vvvvvv(2
qt1tq13t1t312t1t21 vvvvvv
qt1qtq1q vv
2q
22
21
2v 1
In a similar way ( assume )
)vvvv(},cov{ qtq2t21t1tst
)vvvv( qsq2s21s1s
st
Modeling time series by AR(p) and MA(q) continued
2qtstqq
21s11st
2sst vvv
stqq11stst2v
for qst
0
,
otherwise.
Specifying it for MA( 1 )
0t
and
)1()var( 22v
2tt ,
2v1tt },cov{ .
Modeling time series by AR(p) and MA(q) continued
There are at least two or three problems:
Why we study both AR( p ) and MA( q ), when we can convert
Firstly
How to recognize that there is some dependence in the series ?
Secondly
Which type of dependency took place? How large p or q is ?
Thirdly
on and vice versa, i.e. on ? )(MA )p(AR
We’ll answer them successively in the next lecture.
)q(MA )(AR
What is to be learnt from this lecture for exam ?
All what you need is on http://samba.fsv.cuni.cz/~visek/
• The Generalized Least Squares
• AR(p), MA(q), ARMA(p,q), ARIMA(p,h,q)
• Stationarity - conditions for stationarity, - Dickey-Fuller test – for AR( 1 ), - augmented Dickey-Fuller test – for AR( 1 )
• Moments and covariance matrices
• Convertibility