forecasting: principles and practice · estimation if we minimize pn2 t (by using ordinary...
TRANSCRIPT
![Page 1: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/1.jpg)
Forecasting: principles and practice 1
Forecasting: principlesand practice
Rob J Hyndman
3.2 Dynamic regression
![Page 2: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/2.jpg)
Outline
1 Regression with ARIMA errors
2 Stochastic and deterministic trends
3 Periodic seasonality
4 Lab session 14
5 Dynamic regression models
Forecasting: principles and practice Regression with ARIMA errors 2
![Page 3: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/3.jpg)
Regression with ARIMA errorsRegression models
yt = β0 + β1x1,t + · · · + βkxk,t + et,
yt modeled as function of k explanatory variablesx1,t, . . . , xk,t.In regression, we assume that et was WN.Now we want to allow et to be autocorrelated.
Example: ARIMA(1,1,1) errorsyt = β0 + β1x1,t + · · · + βkxk,t + nt,(1− φ1B)(1− B)nt = (1 + θ1B)et,
where et is white noise .Forecasting: principles and practice Regression with ARIMA errors 3
![Page 4: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/4.jpg)
Regression with ARIMA errorsRegression models
yt = β0 + β1x1,t + · · · + βkxk,t + et,
yt modeled as function of k explanatory variablesx1,t, . . . , xk,t.In regression, we assume that et was WN.Now we want to allow et to be autocorrelated.
Example: ARIMA(1,1,1) errorsyt = β0 + β1x1,t + · · · + βkxk,t + nt,(1− φ1B)(1− B)nt = (1 + θ1B)et,
where et is white noise .Forecasting: principles and practice Regression with ARIMA errors 3
![Page 5: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/5.jpg)
Residuals and errors
Example: nt = ARIMA(1,1,1)yt = β0 + β1x1,t + · · · + βkxk,t + nt,(1− φ1B)(1− B)nt = (1 + θ1B)et,
Be careful in distinguishing nt from et.Only the errors nt are assumed to be white noise.In ordinary regression, nt is assumed to be whitenoise and so nt = et.
Forecasting: principles and practice Regression with ARIMA errors 4
![Page 6: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/6.jpg)
Residuals and errors
Example: nt = ARIMA(1,1,1)yt = β0 + β1x1,t + · · · + βkxk,t + nt,(1− φ1B)(1− B)nt = (1 + θ1B)et,
Be careful in distinguishing nt from et.Only the errors nt are assumed to be white noise.In ordinary regression, nt is assumed to be whitenoise and so nt = et.
Forecasting: principles and practice Regression with ARIMA errors 4
![Page 7: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/7.jpg)
Estimation
If we minimize ∑ n2t (by using ordinary regression):
1 Estimated coefficients β̂0, . . . , β̂k are no longeroptimal as some information ignored;
2 Statistical tests associated with the model (e.g.,t-tests on the coefficients) are incorrect.
3 p-values for coefficients usually too small (“spuriousregression”).
4 AIC of fitted models misleading.
Minimizing ∑ e2t avoids these problems.Maximizing likelihood is similar to minimizing ∑ e2t .
Forecasting: principles and practice Regression with ARIMA errors 5
![Page 8: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/8.jpg)
Estimation
If we minimize ∑ n2t (by using ordinary regression):
1 Estimated coefficients β̂0, . . . , β̂k are no longeroptimal as some information ignored;
2 Statistical tests associated with the model (e.g.,t-tests on the coefficients) are incorrect.
3 p-values for coefficients usually too small (“spuriousregression”).
4 AIC of fitted models misleading.
Minimizing ∑ e2t avoids these problems.Maximizing likelihood is similar to minimizing ∑ e2t .
Forecasting: principles and practice Regression with ARIMA errors 5
![Page 9: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/9.jpg)
Stationarity
Model with ARIMA(1,1,1) errorsyt = β0 + β1x1,t + · · · + βkxk,t + nt,(1− φ1B)(1− B)nt = (1 + θ1B)et,
Equivalent to model with ARIMA(1,0,1) errorsy′t = β1x
′1,t + · · · + βkx′k,t + n′t,
(1− φ1B)n′t = (1 + θ1B)et,
where y′t = yt − yt−1, x′t,i = xt,i − xt−1,i and n′t = nt − nt−1.
Forecasting: principles and practice Regression with ARIMA errors 6
![Page 10: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/10.jpg)
Stationarity
Model with ARIMA(1,1,1) errorsyt = β0 + β1x1,t + · · · + βkxk,t + nt,(1− φ1B)(1− B)nt = (1 + θ1B)et,
Equivalent to model with ARIMA(1,0,1) errorsy′t = β1x
′1,t + · · · + βkx′k,t + n′t,
(1− φ1B)n′t = (1 + θ1B)et,
where y′t = yt − yt−1, x′t,i = xt,i − xt−1,i and n′t = nt − nt−1.
Forecasting: principles and practice Regression with ARIMA errors 6
![Page 11: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/11.jpg)
Regression with ARIMA errorsAny regression with an ARIMA error can be rewritten as aregression with an ARMA error by differencing all variableswith the same differencing operator as in the ARIMAmodel.Original data
yt = β0 + β1x1,t + · · · + βkxk,t + ntwhere φ(B)(1− B)dnt = θ(B)et
After differencing all variablesy′t = β1x
′1,t + · · · + βkx′k,t + n′t.
where φ(B)nt = θ(B)etand y′t = (1− B)dyt
Forecasting: principles and practice Regression with ARIMA errors 7
![Page 12: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/12.jpg)
Regression with ARIMA errorsAny regression with an ARIMA error can be rewritten as aregression with an ARMA error by differencing all variableswith the same differencing operator as in the ARIMAmodel.Original data
yt = β0 + β1x1,t + · · · + βkxk,t + ntwhere φ(B)(1− B)dnt = θ(B)et
After differencing all variablesy′t = β1x
′1,t + · · · + βkx′k,t + n′t.
where φ(B)nt = θ(B)etand y′t = (1− B)dyt
Forecasting: principles and practice Regression with ARIMA errors 7
![Page 13: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/13.jpg)
Regression with ARIMA errorsAny regression with an ARIMA error can be rewritten as aregression with an ARMA error by differencing all variableswith the same differencing operator as in the ARIMAmodel.Original data
yt = β0 + β1x1,t + · · · + βkxk,t + ntwhere φ(B)(1− B)dnt = θ(B)et
After differencing all variablesy′t = β1x
′1,t + · · · + βkx′k,t + n′t.
where φ(B)nt = θ(B)etand y′t = (1− B)dyt
Forecasting: principles and practice Regression with ARIMA errors 7
![Page 14: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/14.jpg)
Model selection
Fit regression model with automatically selectedARIMA errors.Check that et series looks like white noise.Note that estimation is done on the differenced seriesto ensure consistent estimators.
Selecting predictorsAICc can be calculated for final model.Repeat procedure for all subsets of predictors to beconsidered, and select model with lowest AICc value.
Forecasting: principles and practice Regression with ARIMA errors 8
![Page 15: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/15.jpg)
US personal consumption & incomeC
onsumption
Income
Production
Savings
Unem
ployment
1970 1980 1990 2000 2010
−2−1
012
−2.50.02.5
−5.0−2.5
0.02.5
−50−25
02550
−1.0−0.5
0.00.51.01.5
Year
Quarterly changes in US consumption and personal income
Forecasting: principles and practice Regression with ARIMA errors 9
![Page 16: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/16.jpg)
US personal consumption & income
−2
−1
0
1
2
−2.5 0.0 2.5
Income
Con
sum
ptio
n
Quarterly changes in US consumption and personal income
Forecasting: principles and practice Regression with ARIMA errors 10
![Page 17: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/17.jpg)
US Personal Consumption and income
No need for transformations or further differencing.Increase in income does not necessarily translate intoinstant increase in consumption (e.g., after the loss ofa job, it may take a few months for expenses to bereduced to allow for the new circumstances). We willignore this for now.
Forecasting: principles and practice Regression with ARIMA errors 11
![Page 18: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/18.jpg)
US personal consumption & income
(fit <- auto.arima(uschange[,"Consumption"],xreg=uschange[,"Income"]))
## Series: uschange[, "Consumption"]## Regression with ARIMA(1,0,2) errors#### Coefficients:## ar1 ma1 ma2 intercept xreg## 0.6922 -0.5758 0.1984 0.5990 0.2028## s.e. 0.1159 0.1301 0.0756 0.0884 0.0461#### sigma^2 estimated as 0.3219: log likelihood=-156.95## AIC=325.91 AICc=326.37 BIC=345.29
Forecasting: principles and practice Regression with ARIMA errors 12
![Page 19: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/19.jpg)
US personal consumption & income
ggtsdisplay(residuals(fit, type='regression'),main="ARIMA errors")
−2
−1
0
1
1970 1980 1990 2000 2010
ARIMA errors
−0.1
0.0
0.1
0.2
0.3
4 8 12 16 20
Lag
AC
F
−0.1
0.0
0.1
0.2
0.3
4 8 12 16 20
Lag
PAC
F
Forecasting: principles and practice Regression with ARIMA errors 13
![Page 20: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/20.jpg)
US personal consumption & income
ggtsdisplay(residuals(fit),main="ARIMA residuals")
−2
−1
0
1
1970 1980 1990 2000 2010
ARIMA residuals
−0.1
0.0
0.1
4 8 12 16 20
Lag
AC
F
−0.1
0.0
0.1
4 8 12 16 20
Lag
PAC
F
Forecasting: principles and practice Regression with ARIMA errors 14
![Page 21: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/21.jpg)
US Personal Consumption and Income
A Ljung-Box test shows the residuals are uncorrelated.
checkresiduals(fit, plot=FALSE)
#### Ljung-Box test#### data: Residuals from Regression with ARIMA(1,0,2) errors## Q* = 5.8916, df = 3, p-value = 0.117#### Model df: 5. Total lags used: 8
Forecasting: principles and practice Regression with ARIMA errors 15
![Page 22: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/22.jpg)
US Personal Consumption and Income
fcast <- forecast(fit,xreg=rep(mean(uschange[,"Income"]),8), h=8)
autoplot(fcast) + xlab("Year") +ylab("Percentage change") +ggtitle("Forecasts from regression with ARIMA(1,0,2) errors")
−2
−1
0
1
2
1970 1980 1990 2000 2010 2020
Year
Per
cent
age
chan
ge
level
80
95
Forecasts from regression with ARIMA(1,0,2) errors
Forecasting: principles and practice Regression with ARIMA errors 16
![Page 23: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/23.jpg)
Forecasting
To forecast a regression model with ARIMA errors, weneed to forecast the regression part of the model andthe ARIMA part of the model and combine the results.
Some explanatory variable are known into the future (e.g.,time, dummies).Separate forecasting models may be needed for otherexplanatory variables.
Forecasting: principles and practice Regression with ARIMA errors 17
![Page 24: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/24.jpg)
Outline
1 Regression with ARIMA errors
2 Stochastic and deterministic trends
3 Periodic seasonality
4 Lab session 14
5 Dynamic regression models
Forecasting: principles and practice Stochastic and deterministic trends 18
![Page 25: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/25.jpg)
Stochastic & deterministic trends
Deterministic trend
yt = β0 + β1t + nt
where nt is ARMA process.Stochastic trend
yt = β0 + β1t + nt
where nt is ARIMA process with d ≥ 1.Difference both sides until nt is stationary:
y′t = β1 + n′t
where n′t is ARMA process.Forecasting: principles and practice Stochastic and deterministic trends 19
![Page 26: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/26.jpg)
Stochastic & deterministic trends
Deterministic trend
yt = β0 + β1t + nt
where nt is ARMA process.Stochastic trend
yt = β0 + β1t + nt
where nt is ARIMA process with d ≥ 1.Difference both sides until nt is stationary:
y′t = β1 + n′t
where n′t is ARMA process.Forecasting: principles and practice Stochastic and deterministic trends 19
![Page 27: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/27.jpg)
Stochastic & deterministic trends
Deterministic trend
yt = β0 + β1t + nt
where nt is ARMA process.Stochastic trend
yt = β0 + β1t + nt
where nt is ARIMA process with d ≥ 1.Difference both sides until nt is stationary:
y′t = β1 + n′t
where n′t is ARMA process.Forecasting: principles and practice Stochastic and deterministic trends 19
![Page 28: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/28.jpg)
International visitors
2
4
6
1980 1990 2000 2010
Year
mill
ions
of p
eopl
e
Total annual international visitors to Australia
Forecasting: principles and practice Stochastic and deterministic trends 20
![Page 29: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/29.jpg)
International visitorsDeterministic trend
(fit1 <- auto.arima(austa, d=0, xreg=1:length(austa)))
## Series: austa## Regression with ARIMA(2,0,0) errors#### Coefficients:## ar1 ar2 intercept xreg## 1.1127 -0.3805 0.4156 0.1710## s.e. 0.1600 0.1585 0.1897 0.0088#### sigma^2 estimated as 0.02979: log likelihood=13.6## AIC=-17.2 AICc=-15.2 BIC=-9.28
yt = 0.4173 + 0.1715t + ntnt = 1.0371nt−1 − 0.3379nt−2 + etet ∼ NID(0, 0.02854).
Forecasting: principles and practice Stochastic and deterministic trends 21
![Page 30: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/30.jpg)
International visitorsDeterministic trend
(fit1 <- auto.arima(austa, d=0, xreg=1:length(austa)))
## Series: austa## Regression with ARIMA(2,0,0) errors#### Coefficients:## ar1 ar2 intercept xreg## 1.1127 -0.3805 0.4156 0.1710## s.e. 0.1600 0.1585 0.1897 0.0088#### sigma^2 estimated as 0.02979: log likelihood=13.6## AIC=-17.2 AICc=-15.2 BIC=-9.28
yt = 0.4173 + 0.1715t + ntnt = 1.0371nt−1 − 0.3379nt−2 + etet ∼ NID(0, 0.02854).
Forecasting: principles and practice Stochastic and deterministic trends 21
![Page 31: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/31.jpg)
International visitorsStochastic trend
(fit2 <- auto.arima(austa,d=1))
## Series: austa## ARIMA(0,1,1) with drift#### Coefficients:## ma1 drift## 0.3006 0.1735## s.e. 0.1647 0.0390#### sigma^2 estimated as 0.03376: log likelihood=10.62## AIC=-15.24 AICc=-14.46 BIC=-10.57
yt − yt−1 = 0.1537 + etyt = y0 + 0.1537t + ntnt = nt−1 + etet ∼ NID(0, 0.03241).Forecasting: principles and practice Stochastic and deterministic trends 22
![Page 32: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/32.jpg)
International visitorsStochastic trend
(fit2 <- auto.arima(austa,d=1))
## Series: austa## ARIMA(0,1,1) with drift#### Coefficients:## ma1 drift## 0.3006 0.1735## s.e. 0.1647 0.0390#### sigma^2 estimated as 0.03376: log likelihood=10.62## AIC=-15.24 AICc=-14.46 BIC=-10.57
yt − yt−1 = 0.1537 + etyt = y0 + 0.1537t + ntnt = nt−1 + etet ∼ NID(0, 0.03241).Forecasting: principles and practice Stochastic and deterministic trends 22
![Page 33: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/33.jpg)
International visitors
2.5
5.0
7.5
1980 1990 2000 2010 2020
Year
level
80
95
Forecasts from linear trend with AR(2) error
2.5
5.0
7.5
10.0
1980 1990 2000 2010 2020
Year
level
80
95
Forecasts from ARIMA(0,1,0) with drift
Forecasting: principles and practice Stochastic and deterministic trends 23
![Page 34: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/34.jpg)
Forecasting with trend
Point forecasts are almost identical, but predictionintervals differ.Stochastic trends have much wider predictionintervals because the errors are non-stationary.Be careful of forecasting with deterministic trends toofar ahead.
Forecasting: principles and practice Stochastic and deterministic trends 24
![Page 35: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/35.jpg)
Outline
1 Regression with ARIMA errors
2 Stochastic and deterministic trends
3 Periodic seasonality
4 Lab session 14
5 Dynamic regression models
Forecasting: principles and practice Periodic seasonality 25
![Page 36: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/36.jpg)
Fourier terms for seasonalityPeriodic seasonality can be handled using pairs of Fourierterms:
sk(t) = sin(2πkt
m
)ck(t) = cos
(2πktm
)
yt =K∑k=1
[αksk(t) + βkck(t)] + nt
nt is non-seasonal ARIMA process.Every periodic function can be approximated by sumsof sin and cos terms for large enough K.Choose K by minimizing AICc.
Forecasting: principles and practice Periodic seasonality 26
![Page 37: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/37.jpg)
US Accidental Deaths
fit <- auto.arima(USAccDeaths,xreg=fourier(USAccDeaths, 5),seasonal=FALSE)
fc <- forecast(fit,xreg=fourier(USAccDeaths, 5, 24))
Forecasting: principles and practice Periodic seasonality 27
![Page 38: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/38.jpg)
US Accidental Deaths
autoplot(fc)
8000
10000
12000
1973 1975 1977 1979 1981
Time
US
Acc
Dea
ths
level
80
95
Forecasts from Regression with ARIMA(0,1,1) errors
Forecasting: principles and practice Periodic seasonality 28
![Page 39: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/39.jpg)
Outline
1 Regression with ARIMA errors
2 Stochastic and deterministic trends
3 Periodic seasonality
4 Lab session 14
5 Dynamic regression models
Forecasting: principles and practice Lab session 14 29
![Page 40: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/40.jpg)
Lab Session 14
Forecasting: principles and practice Lab session 14 30
![Page 41: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/41.jpg)
Outline
1 Regression with ARIMA errors
2 Stochastic and deterministic trends
3 Periodic seasonality
4 Lab session 14
5 Dynamic regression models
Forecasting: principles and practice Dynamic regression models 31
![Page 42: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/42.jpg)
Dynamic regression models
Sometimes a change in xt does not affect ytinstantaneously
yt = sales, xt = advertising.yt = stream flow, xt = rainfall.yt = size of herd, xt = breeding stock.
These are dynamic systems with input (xt) and output(yt).xt is often a leading indicator.There can be multiple predictors.
Forecasting: principles and practice Dynamic regression models 32
![Page 43: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/43.jpg)
Dynamic regression models
Sometimes a change in xt does not affect ytinstantaneously
yt = sales, xt = advertising.yt = stream flow, xt = rainfall.yt = size of herd, xt = breeding stock.
These are dynamic systems with input (xt) and output(yt).xt is often a leading indicator.There can be multiple predictors.
Forecasting: principles and practice Dynamic regression models 32
![Page 44: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/44.jpg)
Dynamic regression models
Sometimes a change in xt does not affect ytinstantaneously
yt = sales, xt = advertising.yt = stream flow, xt = rainfall.yt = size of herd, xt = breeding stock.
These are dynamic systems with input (xt) and output(yt).xt is often a leading indicator.There can be multiple predictors.
Forecasting: principles and practice Dynamic regression models 32
![Page 45: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/45.jpg)
Lagged explanatory variablesThe model include present and past values of predictor:xt, xt−1, xt−2, . . . .
yt = a + ν0xt + ν1xt−1 + · · · + νkxt−k + nt
where nt is an ARIMA process.Rewrite model as
yt = a + (ν0 + ν1B + ν2B2 + · · · + νkBk)xt + nt= a + ν(B)xt + nt.
ν(B) is called a transfer function since it describeshow change in xt is transferred to yt.x can influence y, but y is not allowed to influence x.Forecasting: principles and practice Dynamic regression models 33
![Page 46: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/46.jpg)
Lagged explanatory variablesThe model include present and past values of predictor:xt, xt−1, xt−2, . . . .
yt = a + ν0xt + ν1xt−1 + · · · + νkxt−k + nt
where nt is an ARIMA process.Rewrite model as
yt = a + (ν0 + ν1B + ν2B2 + · · · + νkBk)xt + nt= a + ν(B)xt + nt.
ν(B) is called a transfer function since it describeshow change in xt is transferred to yt.x can influence y, but y is not allowed to influence x.Forecasting: principles and practice Dynamic regression models 33
![Page 47: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/47.jpg)
Lagged explanatory variablesThe model include present and past values of predictor:xt, xt−1, xt−2, . . . .
yt = a + ν0xt + ν1xt−1 + · · · + νkxt−k + nt
where nt is an ARIMA process.Rewrite model as
yt = a + (ν0 + ν1B + ν2B2 + · · · + νkBk)xt + nt= a + ν(B)xt + nt.
ν(B) is called a transfer function since it describeshow change in xt is transferred to yt.x can influence y, but y is not allowed to influence x.Forecasting: principles and practice Dynamic regression models 33
![Page 48: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/48.jpg)
Example: Insurance quotes and TV advertsQ
uotesT
V.advert
2002 2003 2004 2005
8
10
12
14
16
18
6
7
8
9
10
11
Year
Insurance advertising and quotations
Forecasting: principles and practice Dynamic regression models 34
![Page 49: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/49.jpg)
Example: Insurance quotes and TV adverts
Advert <- cbind(insurance[,2], c(NA,insurance[1:39,2]))colnames(Advert) <- paste("AdLag",0:1,sep="")(fit <- auto.arima(insurance[,1], xreg=Advert, d=0))
## Series: insurance[, 1]## Regression with ARIMA(3,0,0) errors#### Coefficients:## ar1 ar2 ar3 intercept AdLag0 AdLag1## 1.4117 -0.9317 0.3591 2.0393 1.2564 0.1625## s.e. 0.1698 0.2545 0.1592 0.9931 0.0667 0.0591#### sigma^2 estimated as 0.2165: log likelihood=-23.89## AIC=61.78 AICc=65.28 BIC=73.6
yt = 2.04 + 1.26xt + 0.16xt−1 + ntnt = 1.41nt−1 − 0.93nt−2 + 0.36nt−3 + et
Forecasting: principles and practice Dynamic regression models 35
![Page 50: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/50.jpg)
Example: Insurance quotes and TV adverts
Advert <- cbind(insurance[,2], c(NA,insurance[1:39,2]))colnames(Advert) <- paste("AdLag",0:1,sep="")(fit <- auto.arima(insurance[,1], xreg=Advert, d=0))
## Series: insurance[, 1]## Regression with ARIMA(3,0,0) errors#### Coefficients:## ar1 ar2 ar3 intercept AdLag0 AdLag1## 1.4117 -0.9317 0.3591 2.0393 1.2564 0.1625## s.e. 0.1698 0.2545 0.1592 0.9931 0.0667 0.0591#### sigma^2 estimated as 0.2165: log likelihood=-23.89## AIC=61.78 AICc=65.28 BIC=73.6
yt = 2.04 + 1.26xt + 0.16xt−1 + ntnt = 1.41nt−1 − 0.93nt−2 + 0.36nt−3 + et
Forecasting: principles and practice Dynamic regression models 35
![Page 51: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/51.jpg)
Example: Insurance quotes and TV adverts
fc <- forecast(fit, h=20,xreg=cbind(c(Advert[40,1],rep(10,19)), rep(10,20)))
autoplot(fc)
8
10
12
14
16
18
2002 2003 2004 2005 2006 2007
Time
insu
ranc
e[, 1
]
level
80
95
Forecasts from Regression with ARIMA(3,0,0) errors
Forecasting: principles and practice Dynamic regression models 36
![Page 52: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/52.jpg)
Example: Insurance quotes and TV adverts
fc <- forecast(fit, h=20,xreg=cbind(c(Advert[40,1],rep(8,19)), rep(8,20)))
autoplot(fc)
8
10
12
14
16
18
2002 2003 2004 2005 2006 2007
Time
insu
ranc
e[, 1
]
level
80
95
Forecasts from Regression with ARIMA(3,0,0) errors
Forecasting: principles and practice Dynamic regression models 37
![Page 53: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/53.jpg)
Example: Insurance quotes and TV adverts
fc <- forecast(fit, h=20,xreg=cbind(c(Advert[40,1],rep(6,19)), rep(6,20)))
autoplot(fc)
8
10
12
14
16
18
2002 2003 2004 2005 2006 2007
Time
insu
ranc
e[, 1
]
level
80
95
Forecasts from Regression with ARIMA(3,0,0) errors
Forecasting: principles and practice Dynamic regression models 38
![Page 54: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/54.jpg)
Dynamic regression modelsyt = a + ν(B)xt + nt
where nt is an ARMA process. So
φ(B)nt = θ(B)et or nt =θ(B)φ(B)
et = ψ(B)et.
yt = a + ν(B)xt + ψ(B)et
ARMA models are rational approximations to generaltransfer functions of et.We can also replace ν(B) by a rational approximation.There is no R package for forecasting using a generaltransfer function approach.Forecasting: principles and practice Dynamic regression models 39
![Page 55: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/55.jpg)
Dynamic regression modelsyt = a + ν(B)xt + nt
where nt is an ARMA process. So
φ(B)nt = θ(B)et or nt =θ(B)φ(B)
et = ψ(B)et.
yt = a + ν(B)xt + ψ(B)et
ARMA models are rational approximations to generaltransfer functions of et.We can also replace ν(B) by a rational approximation.There is no R package for forecasting using a generaltransfer function approach.Forecasting: principles and practice Dynamic regression models 39
![Page 56: Forecasting: principles and practice · Estimation If we minimize Pn2 t (by using ordinary regression): 1 Estimated coe˚cients βˆ 0,...,βˆ k are no longer optimal as some information](https://reader035.vdocuments.net/reader035/viewer/2022071001/5fbe6183ab91f102c40bffea/html5/thumbnails/56.jpg)
Dynamic regression modelsyt = a + ν(B)xt + nt
where nt is an ARMA process. So
φ(B)nt = θ(B)et or nt =θ(B)φ(B)
et = ψ(B)et.
yt = a + ν(B)xt + ψ(B)et
ARMA models are rational approximations to generaltransfer functions of et.We can also replace ν(B) by a rational approximation.There is no R package for forecasting using a generaltransfer function approach.Forecasting: principles and practice Dynamic regression models 39