STAD57 Time Series Analysis

Lecture 6

1

Time Series Modeling Have seen how to check for stationarity of TS

data & ways to “convert” them to stationarity if necessary (de-trending, differencing, etc)

Next step in analysis is to assume a model for the stationary TS data Need model to proceed with estimation / forecasting Ideally, model should accurately describe TS

dependence structure (i.e. sample ACF) For tractability, only consider linear models w.r.t.

series’ lagged values (AR) and/or past WN (MA)2

Stationary Linear Processes

Important result (Wold’s theorem) states that every zero-mean stationary TS can be expressed as:

{Xt} is 1-sided linear function of WN {Wt} Process does not depend on future values of {Wt} Only present & lagged values of {Wt} appear in Xt

Square summability of ψ’s ensures process is stable (variance does not explode to ∞) 3

20

0 2,

{ } ~ (0, )where jj

t j t jjt w

X WW WN

Stationary Linear Processes

Converse is also true, i.e. every Wold linear process is stationary Proof:

4

5

Time Series Modeling

So, why not model every stationary TS as stationary 1-sided linear process?

Because model is not tractable in practice: Need to estimate infinite # of parameters ψj

But only have finite # of data Can only estimate ACF γ(h) for h=0,…,n−1

Instead, will try to use simpler (finite) AR and/or MA models to describe dependence structure of TS

6

Autoregressive Models

Autoregressive (AR) models express current value as linear function of past values of TS

AR model of order p, or AR(p), is of the form

where φ1,…, φp are constant, {Wt} ~ WN(0,σw2),

and {Xt} is zero-mean If E[Xt]=µ≠0, can substitute (Xt−µ) in place of Xt :

7

1 1 2 2t t t p t p tX X X X W

1 1( ) ( ) ( )t t p t p tX X X W

Autoregressive Operators

Simplify representation of AR(p) model using polynomials of backshift operator

The AR operator φ(B) is defined as:

Can write AR(p) model as:

Operator φ(Β) & corresponding polynomial are important for properties of AR model 8

21 2( ) 1 p

pB B B B

1 1 1

1 1

(1 )pp t t t t p t p t

t t p t p t

B B X W X X X WX X X W

( ) t tB X W

Autoregressive Models

Not all AR models are stationary. To check stationarity, express model as linear process E.g. use substitution on AR(1)

9

1t t tX X W

Autoregressive Models

AR(1) can be written as When is this process stationary (stable)? → Need

coefficients to be square summable For AR(1) model, this translates to |φ|<1 Proof:

10

0j

t t jjX W

Example

11

0 10 20 30 40 50 60 70

-20

24

6

0 10 20 30 40 50 60 70

-4-2

02

4

0 5 10 15 20 25 30

0.0

0.2

0.4

0.6

0.8

1.0

0 5 10 15 20 25 30

-0.5

0.0

0.5

1.0

1.9t t tX X W 1.9t t tX X W

Simulated series:

Sample ACF:

( )Theoretical

ACF: hh

Autoregressive Models

For general AR(p) model, find linear representation through AR operator

Try to find a polynomial operator so that:

To find ψ(B), multiply both sides of by ψ(B):

12

1( ) ( ) 1, where pt t pB X W B B B

0( ) j

jjB B

00( ) ( 1)t t j t jj

X B W W

( ) ( ) ( )( ) ( ) 1t t tB B X B W XB B

( ) t tB X W

Inverse Operators

Solve for coefficients ψj of ψ(Β) so that:

Operator ψ(B) is called the inverse of φ(Β), and is denoted by φ−1(B) or 1/φ(Β)

13

21 2 1

21 1 2 1 1 2

1 1 1 1

2 1 1 2 2 1 1 2

( ) ( ) 1 (1 )(1 ) 1

1 ( ) ( ) 100

ppB B B B B B

B B

Example

For AR(1) model ,show that the inverse of is:

14

( ) 1B B 1 ( )t t t t tX X W B X W

2 2 3 3( ) 1B B B B

Autoregressive Models

Assume you express AR(p) model as linear process Can you tell if is stationary?

Important result: if AR(p) characteristic polynomial, for complex z, has all roots outside the unit circle, i.e. , then the inverse

exists and has absolutely summable coefficients, i.e.

15

( ) t tB X W 1( ) ( )t t tX B W B W

( )t tX B W

21 2( ) 1 p

pz z z z

( ) 0 | | 1 for z z 1( ) ( )B B

0| |jj

Causal Processes

1-sided linear process is called causal if Every causal process is stationary, because

An AR(p) process is causal (→ stationary) if and only if the roots of its characteristic polynomial all lie outside the unit circle

16

0( )t t j t jj

X B W W

0| |jj

20 0| |j jj j

21 2( ) 1 0 | | 1for all p

pz z z z z

Example

Show that AR(1) model is causal (stationary) iff

17

1t t tX X W | | 1

Moving Average Models

18

Moving average (MA) models express TS as linear function of current & past values of WN

MA model of order q, or MA(q), is of the form

where θ1,…, θq are constant & {Wt} ~ WN(0,σw2)

Can write MA(q) using the MA operator θ(Β)

1 1 2 2t t t t q t qX W W W W

21 2( ) 1 ( )q

q t tB B B B X B W

Moving Average Models

Every MA(q) process is stationary Follows because it is finite linear process →

coefficients are always summable However, can have two different MA(q)

processes with the same autocovariance MA models are not uniquely defined

To avoid this problem, we restrict attention to invertible MA processes

19

Invertible Models

Consider MA(q) model . Using the inverse operator of θ(B), call it π(B)=θ−1(B), can express {Wt} as (infinite) AR model of {Xt}

To ensure AR representation is stable, we need πj coefficients to be summable As before, necessary & sufficient condition for this

is that the MA characteristic equation satisfies:

20

( )t tX B W

1 1 2( ) ( )t t t t t t t tX B W W B X X X X

( ) 0 | | 1 for z z

Invertible Models

An MA(q) process is invertible if and only if the roots of its characteristic polynomial all lie outside the unit circle

Invertible MA(q) process can be expressed as

Invertibility is useful: estimation algorithms rely on writing unobserved WN {Wt} as function of TS {Xt} For equivalent MA(q) models (same ACF), always use

the one that is invertible 21

21 2( ) 1 0 | | 1for all q

qz z z z z

10 0

( ) ( ) | |, for t t t j t j jj jW B X B X X

Example

Show that following MA models are equivalent

22

1

21

, { } ~ (0,1)1 , { } ~ (0, )

A: for

B: for

t t t t

t t t t

X W W W WN

X W W W WN

Example (cont’d)

If θ=.5, show that only model A is invertible, and find its infinite AR representation

23