sta 6857---difference equations & autocorrelation and
TRANSCRIPT
STA 6857—Difference Equations & Autocorrelation andPartial Autocorrelation Functions (§3.3 & 3.4)
Difference Equations Autocorrelation Function Homework 3c
Outline
1 Difference Equations
2 Autocorrelation Function
3 Homework 3c
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 2/ 23
Difference Equations Autocorrelation Function Homework 3c
Outline
1 Difference Equations
2 Autocorrelation Function
3 Homework 3c
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 3/ 23
Difference Equations Autocorrelation Function Homework 3c
ACF of AR(1)
From last time, we found the ACF of the AR(1) model given by
xt = φxt−1 + wt
to be ρ(h) = φ|h| which satisfies the difference equation
un = φun−1
with initial condition u0 = 1.
This difference equation is classified as homogeneous of order 1.
In the following, we will consider homogeneous difference equations of ageneral order p.
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 4/ 23
Difference Equations Autocorrelation Function Homework 3c
Difference Equation
Definition (Difference Equation)An equation which expresses a value of a sequence as a function of the otherterms in the sequence is called a difference equation.
ExampleConsider the difference equation
an = an−1 + an−2
where a0 = 0 and a1 = 1. The solution to this is the famed Fibonacci numbersgiven by
an =(1 +
√5)n − (1−
√5)n
2n√
5
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 5/ 23
Difference Equations Autocorrelation Function Homework 3c
Difference Equations are Fun!
ExampleConsider the difference equation
an = an−1 + n
where a0 = 0.What’s the solution to this difference equation?Prize awarded to the first correct respondent!!
Answer:
an = 1 + 2 + · · ·+ n =n(n + 1)
2Winner receives ...
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 6/ 23
Difference Equations Autocorrelation Function Homework 3c
Difference Equations are Fun!
ExampleConsider the difference equation
an = an−1 + n
where a0 = 0.What’s the solution to this difference equation?Prize awarded to the first correct respondent!!Answer:
an = 1 + 2 + · · ·+ n =n(n + 1)
2
Winner receives ...
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 6/ 23
Difference Equations Autocorrelation Function Homework 3c
Difference Equations are Fun!
ExampleConsider the difference equation
an = an−1 + n
where a0 = 0.What’s the solution to this difference equation?Prize awarded to the first correct respondent!!Answer:
an = 1 + 2 + · · ·+ n =n(n + 1)
2Winner receives ...
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 6/ 23
Difference Equations Autocorrelation Function Homework 3c
Second-Order Homogeneous Difference Equation
Solving difference equations often similar to solving differential equations.We start with a general second-order homogeneous difference equation given by
un = α1un−1 + α2un−2 (α2 6= 0)
which can be written as
(1− α1B− α2B2)︸ ︷︷ ︸α(B)
un = 0
Here α(z) is called the characteristic polynomial.
Since deg(α(z)) = 2, α(z) will have two complex roots by the fundamentaltheorem of algebra.
Call these two roots z1 and z2.
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 7/ 23
Difference Equations Autocorrelation Function Homework 3c
Two Cases
General second order difference equation:
un = α1un−1 + α2un−2 (α2 6= 0) (?)
Case 1: z1 6= z2 In this case, the solution to (?) is
un = c1
(1z1
)n
+ c2
(1z2
)n
Case 2: z1 = z2 In this case, the solution to (?) is
un =(
1z1
)n
(c1 + c2n)
In each case, the constants c1 and c2 are determined from the initial conditions.Feel free to check these solutions by plugging into (?)!
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 8/ 23
Difference Equations Autocorrelation Function Homework 3c
Back to the Fibonacci Numbers
an − an−1 − an−2 = 0⇒ (1− B− B2)︸ ︷︷ ︸α(B)
an = 0
The characteristic equation is α(z) = 1− z− z2 which has two roots
z1 = −√
5 + 12
and z2 =√
5− 12
Notice the following interesting relationships:
1z1
= − 21 +√
5·
(1−√
51−√
5
)=−2 + 2
√5
1− 5=
1−√
52
= −z2
This follows from the general fact that if z1 and z2 are roots of the quadraticequation ax2 + bx + c, then z1z2 = c
a (and also z1 + z2 = −ba ).
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 9/ 23
Difference Equations Autocorrelation Function Homework 3c
Back to the Fibonacci Numbers
an − an−1 − an−2 = 0⇒ (1− B− B2)︸ ︷︷ ︸α(B)
an = 0
The characteristic equation is α(z) = 1− z− z2 which has two roots
z1 = −√
5 + 12
and z2 =√
5− 12
Notice the following interesting relationships:
1z1
= − 21 +√
5·
(1−√
51−√
5
)=−2 + 2
√5
1− 5=
1−√
52
= −z2
This follows from the general fact that if z1 and z2 are roots of the quadraticequation ax2 + bx + c, then z1z2 = c
a (and also z1 + z2 = −ba ).
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 9/ 23
Difference Equations Autocorrelation Function Homework 3c
Fibonacci Recursion – Solved
Utilizing the general solution of a second-order homogeneous differenceequation with two distinct roots gives
an = c1
(1z1
)n
+ c2
(1z2
)n
= c1(−z2)n + c2(−z1)n
a0 = c1 + c2 = 0⇒ c2 = −c1
a1 = −c1z2 − c2z1 = 1
⇒ −c1z2 + c1z1 = 1
⇒ c1(z1 − z2) = 1
⇒ c1 =1
z1 − z2
=−1√
5Note that
z1 − z2 = −√
5 + 12
−√
5− 12
= −√
5
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 10/ 23
Difference Equations Autocorrelation Function Homework 3c
Fibonacci Recursion – Solved
Utilizing the general solution of a second-order homogeneous differenceequation with two distinct roots gives
an = c1
(1z1
)n
+ c2
(1z2
)n
= c1(−z2)n + c2(−z1)n
a0 = c1 + c2 = 0⇒ c2 = −c1
a1 = −c1z2 − c2z1 = 1
⇒ −c1z2 + c1z1 = 1
⇒ c1(z1 − z2) = 1
⇒ c1 =1
z1 − z2
=−1√
5
Note that
z1 − z2 = −√
5 + 12
−√
5− 12
= −√
5
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 10/ 23
Difference Equations Autocorrelation Function Homework 3c
Fibonacci Recursion – Solved
Utilizing the general solution of a second-order homogeneous differenceequation with two distinct roots gives
an = c1
(1z1
)n
+ c2
(1z2
)n
= c1(−z2)n + c2(−z1)n
a0 = c1 + c2 = 0⇒ c2 = −c1
a1 = −c1z2 − c2z1 = 1
⇒ −c1z2 + c1z1 = 1
⇒ c1(z1 − z2) = 1
⇒ c1 =1
z1 − z2=−1√
5Note that
z1 − z2 = −√
5 + 12
−√
5− 12
= −√
5
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 10/ 23
Difference Equations Autocorrelation Function Homework 3c
Fibonacci Solution Simplified
an = c1(−z2)n + c2(−z1)n
= c1(−z2)n − c1(−z1)n
=(−z1)n − (−z2)n
√5
=
(1+√
52
)n−(
1−√
52
)n
√5
=(1 +
√5)n − (1−
√5)n
2n√
5
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 11/ 23
Difference Equations Autocorrelation Function Homework 3c
Homogeneous Difference Equation of Order pNow consider the general equation:
un − α1un−1 − · · · − αpun−p = 0 (αp 6= 0)
The associated characteristic equation is
α(z) = 1− α1z− · · · − αpzp
Suppose α(z) has r distinct roots: z1 with multiplicity m1, z2 withmultiplicity m2,. . . , and zr with multiplicity mr. (So thatm1 + m2 + · · ·+ mr = p.)Here are the roots of α(z) listed out:
z1, z1, . . . , z1︸ ︷︷ ︸m1
, z2, z2, . . . , z2︸ ︷︷ ︸m2
, . . . , zr, zr, . . . , zr︸ ︷︷ ︸mr
Then the solution is
un = z−n1 P1(n) + z−n
2 P2(n) + · · ·+ z−nr Pr(n)
where Pj(n) is a polynomial in n of degree mj − 1 whose coefficients arededuced from the initial conditions.
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 12/ 23
Difference Equations Autocorrelation Function Homework 3c
Our Applications of Difference Equations
Determine the causal representation of an ARMA(p,q) — cf. Example 3.10(p.102–103)
Compute the ACF of AR(p) and ARMA(p,q) model...next section.
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 13/ 23
Difference Equations Autocorrelation Function Homework 3c
Outline
1 Difference Equations
2 Autocorrelation Function
3 Homework 3c
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 14/ 23
Difference Equations Autocorrelation Function Homework 3c
ACF of an MA(q)
We computed the ACF of an MA(1) last time and also computed theautocovariance function for a general MA(∞) last Friday. Brushing over thecomputations, the ACF of an MA(q) with representation
xt = µ+ wt + θ1wt−1 + θ2wt−2 + · · ·+ θqwt−q
is given as
ρ(h) =
∑q−h
j=0 θjθj+h
1+θ21+···+θ2
q1 ≤ h ≤ q
0 h > q
No statistical significance in the correlelogram (ACF plot) after lag q indicatesthe process may be an MA(q) process.
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 15/ 23
Difference Equations Autocorrelation Function Homework 3c
Autocovariance function of an ARMA(p,q)
The computation of the autocovariance function of the ARMA modelφ(B)xt = θ(B)wt begins as
γ(h) = cov(xt+h, xt)
= E
p∑j=1
φjxt+h−j +q∑
j=0
θjwt+h−j
xt
=
p∑j=1
φjγ(h− j) +q∑
j=0
θj E(wt+h−jxt).
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 16/ 23
Difference Equations Autocorrelation Function Homework 3c
Autocovariance function of a Causal ARMA(p,q)
For a causal ARMA(p,q) model, we have
xt =∞∑
j=0
ψjwt−j
so that
E(wt+h−jxt) = E
[wt+h−j
( ∞∑k=0
ψkwt−k
)]= ψj−hσ
2
Therefore we can continue the calculation of the autocovariance function as
γ(h) =p∑
j=1
φjγ(h− j) +q∑
j=0
θj E(wt+h−jxt)
=p∑
j=1
φjγ(h− j) + σ2q∑
j=h
θjψj−h
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 17/ 23
Difference Equations Autocorrelation Function Homework 3c
Autocovariance Recurrence
Note that θj = 0 when j > q + 1, so we have the following recurrencerelationship
γ(h)− φ1γ(h− 1)− · · · − φp(h− p) = 0, h ≥ max(p, q + 1)
with initial conditions
γ(h)−p∑
j=1
φjγ(h− j) = σ2q∑
j=h
θjψj−h, 0 ≤ h < max(p, q + 1)
Dividing through by γ(0) gives a similar recursion on ρ(h) = γ(h)/γ(0).
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 18/ 23
Difference Equations Autocorrelation Function Homework 3c
ACF of Causal ARMA(1,1)
Example (ACF of Causal ARMA(1,1))Start with the causal ARMA(1,1) given by xt = φxt−1 + θwt−1 + wt.From the previous slide, the following recursion holds
γ(h)− φγ(h− 1) = 0
so the general solution is γ(h) = cφ|h|.
We use the initial conditions from the previous slide to determine theconstant c.
To continue, we need to know the values of ψ0 and ψ1.
Recall ψ(z)φ(z) = θ(z).Multiplying out this equation gives the relationships ψ0 = 1 andψ1 = θ1 + φ1φ0. (Refer Example 3.10 pp.102-103 for more details.)
This gives two equations and two unknowns (γ(0) and γ(1))...
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 19/ 23
Difference Equations Autocorrelation Function Homework 3c
ACF of Causal ARMA(1,1) (cont.)
Example ((cont.))...
γ(0) = φγ(1) + σ2(1 + θφ+ θ2)
γ(1) = φγ(0) + σ2θ
we can continue to solve for these unknowns to produce
γ(h) = σ2 (1 + θφ)(φ+ θ)1− φ2 φ|h|−1,
and for h ≥ 1,
ρ(h) =(1 + θφ)(φ+ θ)
1 + 2θφ+ θ2 φ|h|−1
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 20/ 23
Difference Equations Autocorrelation Function Homework 3c
Outline
1 Difference Equations
2 Autocorrelation Function
3 Homework 3c
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 21/ 23
Difference Equations Autocorrelation Function Homework 3c
Textbook Reading
Read the following section from the textbook§3.5 (Forecasting)
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 22/ 23
Difference Equations Autocorrelation Function Homework 3c
Textbook Problems
Do the following exercise from the textbook3.6 (a)
3.8
Arthur Berg STA 6857—Difference Equations & Autocorrelation and Partial Autocorrelation Functions (§3.3 & 3.4) 23/ 23