![Page 1: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/1.jpg)
Census Bureau Seasonal Adjustment Software and Research
U S C E N S U S B U R E A UU S C E N S U S B U R E A U
![Page 2: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/2.jpg)
Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software
Genhol (holiday regressors)SAS Software:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses sets of series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)
![Page 3: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/3.jpg)
Outline of Talk: Research
TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)– Filters and Filter Diagnostics– Automatic modeling: TRAMO vs. X-12’s “TRAMO”– Revisions
State-Space Models using Sampling Error Data
Non-Gaussian “Structural” State-Space Models for More Stable Resistance to “Outliers”
![Page 4: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/4.jpg)
Statistical Research Division Time Series Group Research and “X-12-ARIMA+” Programming
Brian.C.MonsellKellie.C.WillsWilliam.R.Bell (honorary)David.F.Findley (honorary)Donald.E.Martin (Part-time, Howard University)Trang.Ta.Nguyen (1-year in-house visitor)John.Alexander.Aston (2-year Post-Doc from
Imperial College, London)S.J.M. Koopman (Fellow, Free Univ. of Amsterdam)
![Page 5: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/5.jpg)
Economic and Statistical Programming DivisionTime Series Methods Branch
Research and SAS, Excel Programming
Catherine.C.HoodKathleen.M.McDonald.JohnsonGolam.FarooqueRoxanne.Feldpausch
![Page 6: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/6.jpg)
Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software
Genhol (holiday regressors)SAS Software:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses sets of series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)
![Page 7: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/7.jpg)
X-12-ARIMA
Improvements over StatsCan’s X-11-ARIMA• regARIMA models (including outliers, user-defined
regressors, etc.) vs. ARIMA models• Much more extensive automatic options for
modeling, including trading day, holiday est., additive vs. multiplicative adjustment
• More diagnostics (e.g. spectra, revisions)• Specialized output files, e.g. log files for users
favorite diagnostics, from many X-12-Graph (SAS, but for non-SAS-users)
![Page 8: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/8.jpg)
X-11 Seasonal Adjustment
RegARIMA Models(Forecasts, Backcasts, and Preadjustments)
Modeling and Model Comparison Diagnostics and Graphs
Seasonal Adjustment Diagnostics and Graphs
![Page 9: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/9.jpg)
REGARIMA Model
transformation ARIMA Process
Regressors for trading day and holiday or calendar effects, additive outliers,temporary changes, level shifts, rampsuser-defined effects
Leap-year adjustment, or“subjective” strike adjustment, etc.
ttt
t ZXDY
log
Xt
Dt
![Page 10: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/10.jpg)
Types of Regression Variables Available in X-12-ARIMA
Outlier and Trend-Change EffectsAdditive (or Point) OutliersTemporary Change Outliers
Level shifts, Ramps Seasonal Effects
Calendar month indicators*Trigonometric Seasonal (Sines-Cosines)*
Calendar EffectsTrading Day (Flows or Stocks)*Leap-year February*, Length of Month*Shifting Holidays (e.g. Easter)
Constant Term User-Defined Effects
*Two-regime option availableNote: Regression coefficients can be fixed
![Page 11: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/11.jpg)
X-12-ARIMA Releases
Ver. 0.2.10 July (Statistics Canada options)Ver. 0.3 Summer (TRAMO-type automatic
ARIMA model selection)-based on information gleaned from
TRAMO code provided by Victor Gomez
Ver. 1.0 End of year (Better organized output and manual, more testing etc.)
![Page 12: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/12.jpg)
“X-12-ARIMA/SEATS”• Offers both x11{ } and seats{ } commands to provide “X-
11” or SEATS type seasonal adjustments with X-12-ARIMA diagnostics as well as SEATS diagnostics
• Is being updated from SEATS2000 to SEATS2001&2002 (with support from Agustin Maravall and Gianluca Caporello)
• Schizophenic (duplicate) output, currently• Distribution for research and testing to statistical
agencies and central banks in 2003
![Page 13: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/13.jpg)
Diagnoses from X-12-ARIMA/SEATS
1. Spectrum diagnostic reveals source of “Invalid Decomposition” problem
![Page 14: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/14.jpg)
X-12-A/SEATS COMMAND FILE
series {file="serie.txt"format="tramo"}
transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}check{}#x11{}seats{}
![Page 15: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/15.jpg)
Message from seats{ } run:
• NOTE: Spectral plot for the seasonally adjusted series cannot be done when SEATS cannot perform a signal extraction.
![Page 16: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/16.jpg)
Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.3846 0.12087
Seasonal MA Lag 12 -0.3665 0.12612
![Page 17: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/17.jpg)
10*LOG(SPECTRUM) of the regARIMA model residuals Spectrum estimated from 1990.Jan to 1995.Oct.+++++++I+++++++++++++++++++++++++++++ -22.11I * I * I * I * * -23.34I * * I * * * I * * T I * * *T -24.57I * * *T I * * *T I * * * *T I * * * *T* -25.81I * * * *T* I * * * *T*
![Page 18: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/18.jpg)
series {file="serie.txt"format="tramo"}
transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}check{}x11{}#seats{}
![Page 19: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/19.jpg)
X-12-ARIMA/SEATS Seasonal Adjustment ProgramVersion Number 0.3s Build 24
WARNING: At least one visually significant trading day peak has been found in one or more of the estimated spectra.
![Page 20: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/20.jpg)
G.1 10*LOG(SPECTRUM) of the differenced, transformed seasonally adjusted data. Spectrum estimated from 1990.Jan to 1995.Oct. ++++++++++I+++++++++++++++++++++++++++++++++++ I T I T I T -20.10I T I T I T I T -22.01I T I T I T
![Page 21: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/21.jpg)
series {file="serie.txt"format="tramo"}
transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}regression{variables=td}check{}seats{}
![Page 22: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/22.jpg)
X-12-ARIMA/SEATS Seasonal Adjustment ProgramVersion Number 0.3s Build 24
Reading input spec file from metalss.spc Storing any program output into metalss.out Storing any program error messages into metalss.err
WARNING: At least one visually significant seasonal peak has been found in one or more of the estimated spectra.
![Page 23: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/23.jpg)
Standard Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.1995 0.12871
Seasonal MA Lag 12 0.3843 0.15795
![Page 24: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/24.jpg)
X-12-ARIMA Diagnoses for SEATS
2. T/S Practice of adding outliers to improve kurtosis, etc. can substantially increase the size of revisions of the initial seasonal adjustments:
Example (from Catherine Hood) US Exports of Passenger Cars: History diagnostic shows cost to revisions of adding outlier regressors to reduce kurtosis
![Page 25: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/25.jpg)
![Page 26: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/26.jpg)
![Page 27: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/27.jpg)
Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software
Genhol (holiday regressors)SAS Utilities:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses of many series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)
![Page 28: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/28.jpg)
Genhol
• From holiday date file: Generates regressor matrices and associated command files to enable X-12-ARIMA estimation of complex moving holiday effects (e.g. for Easter, Ramadan, etc.).
• Regressors for up to three intervals:– before-the-holiday interval– surrounding-the-holiday interval– past-the-holiday interval (“recovery” interval)
![Page 29: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/29.jpg)
Proportionality Regressors: An Example
• Assume :– An effect interval is 10 days long, and this year
2 of its days fall in January and 8 in February.The interval regressor’s values for this year will
be:– 0.2 in January– 0.8 in February– 0.0 for the rest of the year
![Page 30: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/30.jpg)
Interface Program (SAS): for seasonal adjustment of sets of series
Example: Seasonally Adjusted Total U.S. Imports = sum of 140 component series, c. 80% of which are seasonally adjusted.
What is the effect on the month-to-month changes and quality diagnostics of the S. A. Total Imports if the seasonal adjustment options are changed for 5 of the component series?
![Page 31: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/31.jpg)
Outline of Talk: Research
TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)– Filter Diagnostics – Automatic modeling: TRAMO vs. X-12’s “TRAMO”– Revisions
![Page 32: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/32.jpg)
Filters and Filter Diagnostics
• Filter (spectral) diagnostics needed– To understand limitations/issues with short
series (finite filter diagnostics, also for concurrent adjustments, trends)
– To decide between closely competitive models
Paper by David Findley and Donald Martin.
![Page 33: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/33.jpg)
0 1 2 3 4 5 6
Cycles per year
0.0
0.5
1.0
1.5
Squ
ared
gai
nSquared gain of symmetric SEATS filters
Parameter values -- 0.4,0.8
infinite109 months49 months
![Page 34: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/34.jpg)
Outline of Talk: Research
TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)
• Automatic modeling: TRAMO vs. X-12’s “TRAMO”
• Accuracy– Results from simulated series
• Revisions– Results from Census Bureau series
![Page 35: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/35.jpg)
ESMPD’s Automatic Modeling Study
• First presented at the International Forecasters Symposium, June 2001
• Continuation of this work to appear at the ASA meetings, August 2002, in a paper by Kathleen McDonald-Johnson, et al.
![Page 36: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/36.jpg)
Series
306 time series from the US Census Bureau’s Import/Export series and Retail Sales
![Page 37: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/37.jpg)
Results
• 88 series (29%) with same regARIMA model• 27 series (9%) with same differencing and same
regressors but different ARMA choices• 123 series (40%) with same differencing, but
different regressors• 32 series (10%) with different nonseasonal
differencing order (but sometimes offset by a constant)
• 36 series (12%) with different seasonal differencing order
![Page 38: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/38.jpg)
Conclusions
• TRAMO’s weakness is the procedure for deciding about trading day modeling– TRAMO developers are aware of our results
• X-12-ARIMA has a problem with choosing nonparsimonious models– Monsell has already implemented some
changes, including a unit root test.
![Page 39: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/39.jpg)
Why Are Different Models Chosen?
• Model estimation method is different – TRAMO : Hannan-Rissanen and m.l.e conditional on
AR part of model– X-12-ARIMA : Exact MLE
• Model residuals are different, which can lead to different choices of outliers
• Outlier procedure itself is different – TRAMO removes insignificant outliers after each
iteration• TRAMO uses approximate BIC
![Page 40: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/40.jpg)
Accuracy: X-12-ARIMA vs T/S (ESMPD)
• Results from 54 simulated series were first presented at the ASA meetings, August 2000– Continuation of the first SEATS studies,
beginning in 1997
![Page 41: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/41.jpg)
The Simulated Series
• Fifty-four series– Six different trends – three from SEATS and
three from X-12– Six different seasonal factors – three from
SEATS and three from X-12– Irregular sampled from three sets of irregular
factors combined from SEATS and X-12
![Page 42: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/42.jpg)
Results of Accuracy Study
• SEATS performed better on the majority of series with large irregulars if the series are 9+ years long, but most adjustments were not acceptable.
• Both programs did better than expected on the short series, but X-12-ARIMA adjustments were usually better than SEATS adjustments on series 4-7 years long
![Page 43: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/43.jpg)
Revisions: X-12-ARIMA vs T\S
• New ESMPD study using X-12-SEATS on Census series. “Final” results will be presented at the ASA meetings, August 2002.– Can we identify characteristics in the series
that will indicate if its “linearized” series will be a better candidate for a model-based adjustment than for an X-11 filter adjustment or vice versa?
![Page 44: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/44.jpg)
Methods
• Use X-12-SEATS to get revision diagnostics from both an X-11/X-12-type adjustment and a SEATS adjustment– Used TRAMO to get the ARIMA model, and
then used either an x11 or a seats “spec”
![Page 45: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/45.jpg)
Very Preliminary Results
• 260 US Import/Export series• Only a very small subset (18 series) where
we can see definite differences in the revision diagnostics for the seasonal adjustment
![Page 46: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/46.jpg)
An Observation: Series with
– Large revisions in X-12 and smaller revisions in SEATS had generally large values for 12 (most greater than 0.95) and values for X-12’s I/S ratio < 5.
– Large revisions in SEATS and smaller revisions in X-12 had generally 0.4 < 12 < 0.6 and values for I/S > 6.
In both cases, smaller revisions are associated with more constant seasonal factor estimates
![Page 47: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/47.jpg)
Next Steps
• Look at more series• Look at more diagnostics/characteristics of
the series to try to find patterns, not just revisions
![Page 48: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/48.jpg)
Outline of Talk: Research
Projects almost ready to yield results:
State-Space Models using Sampling Error Data Non-Gaussian “Structural” State-Space Models
for More Stable Resistance to “Outliers”
![Page 49: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/49.jpg)
State-Space Models with Sampling Error Statistics: Bell and Nguyen
100+ Disaggregate Construction series with “high” sampling error variancesConsider model-based adjustment with
regARIMA+observation errormodels that incorporate sampling error variance and autocovariance estimates to achieve acceptable or better seasonal adj’s.
(Need state-space for model est. & seas adj.)
![Page 50: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/50.jpg)
Non-Gaussian “Structural” State-Space Models for More Stable Resistance to
“Outliers” Koopman and Aston X-12-ARIMA and T\S use outlier regressors
identified by t-statistics and critical values. Identifications can change as new data arrive, causing seasonal adjustment revisions.
Use heavy tailed non-Gaussian models instead of critical values. (Hard to estimate such models, simplest for “Harvey’s structural models”)
![Page 51: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/51.jpg)
More Information
WWW site for X-12-ARIMA (papers and software):
www.census.gov/srd/www/x12a
![Page 52: Census Bureau Seasonal Adjustment Software and Research](https://reader033.vdocuments.net/reader033/viewer/2022061511/56815df1550346895dcc218c/html5/thumbnails/52.jpg)
Thanks to Catherine Hood for supplying some of these slides.