computational issues related to sequential monte carlo filter and
TRANSCRIPT
Institute of Statistical MathematicsResearch Organization of Information and Systems 1
Genshiro Kitagawa Research Organization of Information and Systems
Computational Issues Related to
Sequential Monte Carlo Filter and Smoother
NUS-UTokyo Workshop on Quantitative FinanceSeptember 26, 2013
Institute of Statistical MathematicsResearch Organization of Information and Systems 2
Paradigm Change of Modeling
Expansion of the object of sciences
Physical Life Human/Social Cyber-PhysicalSciences Sciences Sciences Sciences
From “science for knowledge” to “science for design”.
・From the “quest for truth” to “fulfillment of objectives (prediction, simulation, knowledge creation, decision making, control)”
・From the “physical (first principles) model” to “modeling for attaining the objectives
CPS = Cyber Physical System
Flexible platform for modeling
Institute of Statistical MathematicsResearch Organization of Information and Systems 3
Outline
Sequential Monte Carlo Filter/Smoother• State-Space Model and the Kalman Filter• General State-Space Model• Sequential Monte Carlo Filter/Smoother
Computational aspects• Use of Huge Number of Particles• Two-filter Formula• Parallel MCF’s• Posterior Mean Filter/Smoother
Example• Sovereign Credit Default Swap
Institute of Statistical MathematicsResearch Organization of Information and Systems 4
Ordinary State Space Model
x Fx G vy H x w
n n n
n n n
1 System ModelObservation Model
n
n
xy Time Series
State Vector n
n
wv System Noise
Observation Noise
y nx nvn
w nF
G H
Institute of Statistical MathematicsResearch Organization of Information and Systems 5
State Estimation Problem
n
Prediction
Filter
Smoothing
xn
Yj
j
}, ... ,{ 1 jj yyY Observation
Obtain conditional distribution of given observationsxn Yj
Applications
• Prediction, smoothing• Likelihood, parameter estimation• Missing observations• Outliers• Signal extraction
njnjnj
for for for
Institute of Statistical MathematicsResearch Organization of Information and Systems 6
Kalman Filter/Smoother
Prediction
Filter
x F xV F V F G Q G
n n n n n
n n n n n nT
n n nT
| |
| |
1 1 1
1 1 1
K V H H V H Rx x K y H xV I K H V
n n n nT
n n n nT
n
n n n n n n n n n
n n n n n n
| |
| | |
| |
( )( )
( )
1 11
1 1
1
SmoothingA V F Vx x A x xV V A V V A
n n n nT
n n
n N n n n n N n n
n N n n n n N n n nT
| |
| | | |
| | | |
( )( )
11
1 1
1 1
FilterFilter
PredictionPrediction
InitialInitial
xN, VNxN, VN
SmoothingSmoothing
n=n1
n=N
Kalman (1960)
yn
n n 1
n 1
Institute of Statistical MathematicsResearch Organization of Information and Systems 7
Necessity of Non-Gaussian Modeling
Abrupt structural changes Outliers Non-Gaussian distributions
Nonlinear processes Discrete processes On-line parameter estimation
nnn vxfx )( 1Poisson processBernoulli process
Institute of Statistical MathematicsResearch Organization of Information and Systems 8
Generalization of State Space Models
Nonlinear Non-Gaussiannnn
nnn
wHxyGvFxx
1
General
),(),( 1
nnn
nnn
wxhyvxfx
)| (~)| (~ 1
nn
nn
xHyxFx
Linear Gaussian
NonlinearNon-Gaussian
Discrete stateDiscrete obs.
Institute of Statistical MathematicsResearch Organization of Information and Systems 9
General Recursive Filter/Smoother
Prediction
p x Y p x Y p x x p x Yp x Y
dxn N n nn n n N
n nn( | ) ( | ) ( | ) ( | )
( | )
1 1
11
p x Y p y x p x Yp y Yn n
n n n n
n n
( | ) ( | ) ( | )( | )
1
1
p x Y p x x p x Y dxn n n n n n n( | ) ( | ) ( | )
1 1 1 1 1
Filter
Smoother
Institute of Statistical MathematicsResearch Organization of Information and Systems 10
Implementation of Filtering/Smoothing
• Linear-Gaussian SSM p(xn|Yk): Gaussian Kalman Filter/Smoother
• Nonlinear or Non-Gaussian p(xn|Yk): Non-Gaussian Need approximations of
non-Gaussian distributions
Prediction
Filter
)|( 1nn Yxp
)|( nn Yxp
1|1| , nnnn Vx
nnnn Vx || ,
Linear Gaussian General
Institute of Statistical MathematicsResearch Organization of Information and Systems 11
Approximations of Distributions
0. Gaussian Approximation(Extended) Kalman filter/smoother
1. Piecewise-linear (Step) Approx.Non-Gaussian filter/smoother
True
Normal approx.
PiecewiseLinear
Step functionKitagawa G. JASA(1987)
Piecewise linear (or step) function approximation of the distribution and numerical integration.
Institute of Statistical MathematicsResearch Organization of Information and Systems 12
Gaussian vs. Non-Gaussian Smoothing
-4
-3
-2
-1
0
1
2
3
4
1 101 201 301 401
-3
-2
-1
0
1
2
3
0 100 200 300 400
Exact Non-Gaussian Smoother
-3
-2
-1
0
1
2
3
1 101 201 301 401
Kalman Smoother
nnn
nnn
wtyvtt
1
Trend Model
Noise Distribution
),0(~),0(~
2
2
NwNv
n
n ),0( 2Cor
Kitagawa (1987)
0 100 200 300 400 500
0 100 200 300 400 5000 100 200 300 400 500
3
2
1
0
-1
-2
-3
3
2
1
0
-1
-2
-3
4
3
2
1
0
-1
-2
-3
-4
Institute of Statistical MathematicsResearch Organization of Information and Systems 13
Estimation of Volatility (Time-varying Variance)
ApproximationTrue
Model 2 AIC
Gauss 0.04318 1861.6
D-Exp 0.00011 1718.2
8
6
4
2
0
-2
-4 0 100 200 300 400 500
Gaussian Model
0 100 200 300 400 500
8
6
4
2
0
-2
-4
Cauchy Model
Institute of Statistical MathematicsResearch Organization of Information and Systems 14
PTGMPPTGMK
PTTGMMKTGGMMKKMMMKKK
k
11111000300131030100300
10100110100110100130130130130
100101100101100107654321
Amount of Computation for Numerical Integration
State DimensionDimension of System NoiseNumber of NodesNodes of System Noise
jq
i
kk
m
nknkkkk mqqm
11 ~))((~
1
# of computation
Institute of Statistical MathematicsResearch Organization of Information and Systems 15
Sequential Monte Carlo Methods
Gordon, Salmond and Smith (1993) Bootstrap Filter
Kitagawa (1993,1996)Monte Carlo Filter/Smoother
Doucet, de Freitas and Gordon (2001) “Sequential Monte Carlo Methods in Practice”
Frequently called “Particle Filter”
Institute of Statistical MathematicsResearch Organization of Information and Systems 16
Approximations of Distributions
0. Gaussian Approximation(Extended) Kalman filter/smoother
1. Piecewise-linear (Step) Approx.Non-Gaussian filter/smoother
2. Gaussian Mixture ApproximationGaussian-sum filter/smoother
3. Particle ApproximationSeqential Monte Carlo filter/smoother
True
Normal approximation
PiecewiseLinear
Step function
Normal mixture
Particle approximation.
Institute of Statistical MathematicsResearch Organization of Information and Systems 17
Approximation of Distributions by Particles
m1
Distribution function
Empirical distribution function
)|(~,, 1)()1(
nnm
nn Yxppp
F xm
I x pn nj
j
m
( ) ( ; )( )1
1
Pr( | )( )X p Ymn n
jn 1
1
)(~,,
)|(~,,
)|(~,,
)()1(
)()1(
1)()1(
nm
nn
nnm
nn
nnm
nn
vpvv
Yxpff
Yxppp
Predictive Distribution
Filter Distribution
Noise Distribution
Institute of Statistical MathematicsResearch Organization of Information and Systems 18
Prediction Step
),( 1 nnn vxFx System Model
v p vnj( ) ~ ( )
f p x Ynj
n n 1 1 1( ) ~ ( | )
p F f vnj
nj
nj( ) ( ) ( )( , ) 1
11111
11111111
1111
)|()()),((
)|(),|(),,|(
)|,,()|(
nnnnnnnn
nnnnnnnnnnn
nnnnnnnn
dvdxYxpvpvxFx
dvdxYxpYxvpYvxxp
dvdxYvxxpYxp
)(~
)|(~)(
11)(
1
nj
n
nnj
n
vpv
Yxpf
p F f v p x Ynj
nj
nj
n n( ) ( ) ( )( , ) ~ ( | ) 1 1
Institute of Statistical MathematicsResearch Organization of Information and Systems 19
Filtering Step (Resampling)
:)( jn Importance weight of particle pn
j( )
nj
n n njp y X p( ) ( )( | )
Pr( | ) Pr( | , )Pr( | ) Pr( | )
Pr( | ) Pr( | )
( ) ( )
( ) ( )
( ) ( )
( )
( )
( )
( )
X p Y X p Y yy X p X p Y
y X p X p Y
n nj
n n nj
n n
n n nj
n nj
n
i
m
n n ni
n ni
n
nj
m
nj
i
mm
nj
nj
i
m
1
1
11
1
11
1
nj( )
yn
p y pn nj( | )( )
pnj( )
f nj( )
f nj( )
m1
)( jn
resampling
Institute of Statistical MathematicsResearch Organization of Information and Systems 20
One Cycle of Monte Carlo Filtering
)|( 11 nn Yxp
n=n+1
xSt
ate
spac
e
)|( 1nn Yxp
Resampling
)|( 1nn Yxp )|( nn Yxp
Filtering
Importance weight
X perish
X perish
)|( )()( jnn
jn xyr)( nvp
Prediction
),( )(1
)(n
jn
jn vxfx
new data yn
Institute of Statistical MathematicsResearch Organization of Information and Systems 21
Computational Cost of Filtering
Sequential filtering: O(n)n: data length
Numerical integration: O(nkl+1)l: dimension of the statek: number of nodes (>100)
MCF: O(nm) , O(nmlog(m)), O(nm2)m: number of particles
m does not increase so rapidly as kl+1
Institute of Statistical MathematicsResearch Organization of Information and Systems 22
Accuracy in Computing Log-Likelihood
Gaussian model Cauchy modelm Log-L S.D. CPU-time Log-L S.D. CPU-time
102 -750.859 2.287 0.02 -752.207 6.247 0.02
103 -748.529 1.115 0.06 -743.244 2.055 0.06
104 -748.127 0.577 0.58 -742.086 0.429 0.63
105 -747.960 0.232 5.84 -742.024 0.124 6.27
106 -747.931 0.059 59.41 -742.029 0.038 62.73
107 -747.926 0.023 591.04 -742.026 0.013 680.33
108 -747.930 0.008 5906.62 -742.026 0.003 6801.55
109 -747.928 0.002 59077.35 -742.026 0.001 69255.03
Institute of Statistical MathematicsResearch Organization of Information and Systems 23
Fixed-lag Smoothing (by Storing Particles)
Filter p nj( ) f n
j( )
Fixed-interval Smoothing
L-lag Smoothing
s s pnj
n nj
nj
1 1 1 1|( )
|( ) ( ), , , s sn
jn n
j1 |( )
|( ), ,
s s pn L nj
n nj
nj
|( )
|( ) ( ), , ,1 1 1 s sn L n
jn n
j |
( )|
( ), ,
L particles
Institute of Statistical MathematicsResearch Organization of Information and Systems 24
Number of Different Particles in Fixed-Lag Smoothing
1
10
100
1000
10000
100000
1000000
0 25 50 75 100 125 150 175 200
M=100 M=1000 M=10000M=10^5 M=10^6
1
10
100
1000
10000
100000
1000000
0 25 50 75 100 125 150 175 200
M=100 M=1000 M=10000M=10^5 M=10^6
Gaussian model Cauchy model
Lag Lag
m=100m=105
m=1000m=106
m=10000 m=10000m=1000m=106
m=100m=105
Institute of Statistical MathematicsResearch Organization of Information and Systems 25
Single MCF
-3
-2
-1
0
1
2
3
0 100 200 300 400
-3
-2
-1
0
1
2
3
1 101 201 301 401
m=100,000 m=10,000
Exact Non-Gaussian Smoother
-3
-2
-1
0
1
2
3
1 101 201 301 401
-3
-2
-1
0
1
2
3
1 101 201 301 401
m=100
m=1,000
-3
-2
-1
0
1
2
3
1 101 201 301 401
m=10,000
Institute of Statistical MathematicsResearch Organization of Information and Systems 26
Accuracy vs. Number of Particles
0.0001
0.001
0.01
0.1
1
10
100
1 2 3 4 5 60.001
0.01
0.1
1
10
100
1 2 3 4 5 6
Filter Smoother Fixed-interval
102 103 104 105 106 107 102 103 104 105 106 107
Number of Particles Number of Particles
Gaussian model Cauchy model
Smoothing (max lag) Filter xnxDnxDDDI
N
n
L
jjj
2
1 1),(),(),(
Smoothing (best lag)
Institute of Statistical MathematicsResearch Organization of Information and Systems 27
Accuracy of Fixed-Lag Smoother
Gaussian model Cauchy model
m=100
m=103
m=107
m=106
m=105
m=104
m: Number of particles
Institute of Statistical MathematicsResearch Organization of Information and Systems 28
Accuracy of Fixed-Lag Smoother
Gaussian model Cauchy model
m=100 m=103 m=105 m=106m=104 m=107
m=100
m=103
m=104
m=105
m=106
m=107
Institute of Statistical MathematicsResearch Organization of Information and Systems 29
Summary of MCF (Particle Filter/Smoother)
• Very flexible and easy-to-implement method for nonlinear non-Gaussian time series modeling.
• Due to the collapsing caused by repeated re-sampling, it is difficult to get precise posterior distribution (in particular, the quantile or percentile points), for small number of particles.
• This collapsing of smoothed distribution can be mitigated by using huge number of particles (at the expense of computational cost).
Institute of Statistical MathematicsResearch Organization of Information and Systems 30
)|()|(
)|,(
),|()|(
1
1
1
nnnn
N
nn
Nn
nNnnNn
YxpxYp
YYxp
YYxpYxp
Smoothing by Two Filter Formula
Smoother
Filter
)|()|(
)|,(
),|()|(
1
1
1
nnnn
nnn
nnnnn
Yxpxyp
Yyxp
yYxpYxp Nn
nN
nn
yyY
yyY
,,
,,1
)|(
)|(
nn
N
nn
xYp
xyp
nn
n yY
Institute of Statistical MathematicsResearch Organization of Information and Systems 31
Backward Filtering
)|()|( NNNN
N xypxYp
Backward Filtering
)|()|()|(
)|()|()|(1
11111
nn
Nnnnn
N
nnnnn
Nnn
N
xYpxypxYp
dxxxpxYpxYp
Initialization
Institute of Statistical MathematicsResearch Organization of Information and Systems 32
Fixed-Lag with LAG=500 Two-Filter Formula
m=100,000
32
m=10,000
m=1,000
Institute of Statistical MathematicsResearch Organization of Information and Systems 33
mGaussian Model Cauchy Model
Fixed-lag
Fixed-interval
Two-filter
Fixed-lag
Fixed-interval
Two-filter
102 8.693 41.723 6.913 21.248 47.881 26.440103 2.259 16.275 1.399 6.042 23.654 4.870104 0.717 5.547 0.333 1.001 3.679 0.378105 0.185 1.448 0.118 0.140 0.380 0.072
Accuracy of Smoothing
Institute of Statistical MathematicsResearch Organization of Information and Systems 34
Two-Filter Formula for Smoothing
Backward FilteringFiltering
)|( nn
N xYp
Smoothing
n-1n-2 n+2n+1n
Institute of Statistical MathematicsResearch Organization of Information and Systems 35
Accuracy of Two-Filter Smoothing Algorithm
Gaussian model Cauchy model
1 10 102 103 104
Acc
urac
y
Acc
urac
y
ms: Number of evaluated particles ms: Number of evaluated particles1 10 102 103 104
ms=100 is sufficient.ms=10 might be reasonable
Institute of Statistical MathematicsResearch Organization of Information and Systems 36
Summary of Two-Filter Formula for Smoothing
• Collapse of smoothing distribution by resampling can be mitigated by the two-filter formula.
• Strict realization of the formula is time-consuming for large m.
• Thinning (using only 10-100 particles for evaluating the particles) yields reasonable results.
Institute of Statistical MathematicsResearch Organization of Information and Systems 37
Application to High-dim. Problems
Accuracy ~ O(m-1/2)
but amount of computation increases
Parallel computation
High-dimensional cases:• NGF(Integration): Impossible• MCF(Monte Carlo): Possible
Institute of Statistical MathematicsResearch Organization of Information and Systems 38
0.001
1
1000
1000000
1E+09
1E+12
1E+15
1960 1970 1980 1990 2000 2010 20200.001
1
1000
1000000
1E+09
1E+12
1E+15
1960 1970 1980 1990 2000 2010 2020
History of Computers in ISM
Eflops
Pflops
Tflops
GFlops
MFlops
KIPS
EB
PB
TB
GB
MB
KB
Speed Memory
K computer K computer
NGF MCF
Parallel processor
ISM = Institute of Statistical Mathematics, Japan
Institute of Statistical MathematicsResearch Organization of Information and Systems 39
Parallel Computation
Direct Parallel Implementation of MCF Simple Parallel MCF
• Simply run k MCF’s in parallel • Take an average of posteriors
Weighted Parallel MCF• Weighted average of posteriors
Weighted parallel MCF with Crossover• Crossover between MCFs, for MCF with low weight.
Institute of Statistical MathematicsResearch Organization of Information and Systems 40
Direct Parallel Implementation of MCF
)()1( ,, knn ff
)2()1( ,, kn
kn ff )()( ,, m
nkm
n ff
)()1( ,, knn pp )2()1( ,, k
nk
n pp )()( ,, mn
kmn pp
Local sum of
Total sum of )( jn
)( jn
)(0
)1(0 ,, mff
)(0
)1(0 ,, kff )2(
0)1(
0 ,, kk ff )(0
)1(0 ,, mkm ff Initialization
Prediction
Filter
Institute of Statistical MathematicsResearch Organization of Information and Systems 41
Communication Between Cores
1. Likelihood for j=1,…,m; i=1,…,k 2. Importance weight of i-th MCF
for i=1,…,k
3. Cumulative weight
for i=1,…,k
4. Starting points
),( ijn
m
jij
ni
n 1),()(
k
ji
n
i
ji
nin
1)(
1)(
)(
kiJ
kipI
q
pIp
nIi
n
pin
ii
1),()1(
n)(
)(n
)(
)(1
)(1 s.t. qsmallest the:
s.t. smallest the:
0
0.002
0.004
0.006
0.008
0.01
0.012
4 8 16 32
Gauss Cauchy
Number of CoresR
atio
of C
omm
unic
atio
ns B
etw
een
Cor
es
Stratified search
Institute of Statistical MathematicsResearch Organization of Information and Systems 42
Efficiency of Parallel Computation
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5 6 7 8 9
4-Core 8-Core16-Core 32-Core64-Core 128-Core
0.01
0.1
1
10
100
1000
10000
100000
1000000
1 2 3 4 5 6 7 8 9
1-Core 4-Core 8-Core16-Core 32-Core 64-Core128-Core
102 103 104 105 106 107 108 109 1010 102 103 104 105 106 107 108 109 1010
Number of Particles Number of Particles
Elapsed Time Relative Efficiency
Amdahl’s lawParallel portion of thealgorithm: 0.989-0.980
),(),1(),(REmkTk
mTmk
),( mkT
m
Institute of Statistical MathematicsResearch Organization of Information and Systems 43
Simple Parallel MCF
MCP1
MCF1
MCP2
MCF2
MCPk
MCFk
P-MCF
・・・
・・・
)|(1)|(1
)(n
k
i
inkn Yxp
kYxp
)|( )1(kn Yxp )|( )2(
kn Yxp )|( )(k
kn Yxp
Institute of Statistical MathematicsResearch Organization of Information and Systems 44
Accuracy of Simple Parallel MCF
0.00001
0.0001
0.001
0.01
0.1
1
10
1 10 100 1000
Number of Parallel MCF Number of Parallel MCF
m=102 m=103 m=105 m=106m=104
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1 10 100 1000
Cauchy ModelGaussian Model
Institute of Statistical MathematicsResearch Organization of Information and Systems 45
Gaussian model Cauchy model
Number of parallel MCF, k Number of parallel MCF, k
m 1 10 100 100 1 10 100 1000
102 3.22273 1.11807 0.90401 0.90421 21.72463 11.32517 10.44000 10.26888
103 0.56848 0.23308 0.18685 0.18304 4.01454 1.01333 0.72036 0.72802
104 0.11893 0.02979 0.02150 0.01916 0.37586 0.03984 0.00677 0.00479
105 0.02650 0.00377 0.00141 0.00081 0.03416 0.00327 0.00034 0.00008
106 0.00396 0.00042 0.00004 0.00334 0.00032 0.00004
107 0.00039 0.00004 0.00031 0.00003
108 0.00017 0.00003
Accuracy of Simple Parallel MCF
Institute of Statistical MathematicsResearch Organization of Information and Systems 46
Accuracy of Simple Parallel MCF
m 100 1,000 10,000 100,000
GaussianBias(m) 0.9109 0.1883 0.0200 0.0011
Variance(m) 2.2741 0.3329 0.0931 0.0240
CauchyBias(m) 10.2735 0.6985 0.0034 0.0000
Variance(m) 10.9688 3.3143 0.3724 0.0342
)(variance1)(bias )(Accuracy mk
mD
k: number of parallel MCF
Institute of Statistical MathematicsResearch Organization of Information and Systems 47
Weighted Parallel MCF
MCP1
MCF1
MCP2
MCF2
MCPk
MCFk
WP-MCF
・・・
・・・
m
j
jnm 1
)1,(1
1
m
j
jnm 1
)2,(2
1
m
j
kjnk m 1
),(1
k
i i
iiw
1
k
iNniiNn YxpwYxp
1)|()|(
Institute of Statistical MathematicsResearch Organization of Information and Systems 48
• Weighted parallel MCF surrogates the MCF with particles.
• However, weights of some MCF’s may become very small and that will eventually deteriorate the efficiency of the total filter.
• To alleviate this problem, perform the following crossover between MCFs
If , for j=2,4,…,m, interchange particles and weights
Weighted Parallel MCF with Crossover
),(),(
),(),(
maxmin
maxmin
ijn
ijn
ijn
ijn pp
m
j
ijn
k
iinn
ik
iinn w
mYypwYyp
1
),(
11
)(
11
1)|()|(
Cww
i
i minmax
km
Institute of Statistical MathematicsResearch Organization of Information and Systems 49
100 parallel MCF
-3
-2
-1
0
1
2
3
1 101 201 301 401
-3
-2
-1
0
1
2
3
1 101 201 301 401
-3
-2
-1
0
1
2
3
1 101 201 301 401
Weighted Parallel MCF(with crossover)
-3
-2
-1
0
1
2
3
1 101 201 301 401
10 parallel MCF
-3
-2
-1
0
1
2
3
1 101 201 301 401
-3
-2
-1
0
1
2
3
1 101 201 301 401
m=1,000
m=10,000
m=100,000
-3
-2
-1
0
1
2
3
1 101 201 301 401
-3
-2
-1
0
1
2
3
1 101 201 301 401
-3
-2
-1
0
1
2
3
1 101 201 301 401
1000 parallel MCF
Institute of Statistical MathematicsResearch Organization of Information and Systems 50
Simple parallel Weighted Parallel
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1 10 100 1000
Accuracy (Divergence from “True” Densities)
MCF with m=100MCF with m=1,000MCF with m=10,000MCF with m=100,000MCF with m=1,000,000
D(q(x);p(x))q(x): “true” distributionp(x): estimated distribution
k : number of parallel MCF’s
D(q
;p)
m: number of particles in each MCF
?
Institute of Statistical MathematicsResearch Organization of Information and Systems 51
Summary of Parallel Computation
• In parallel computation, reduction of communications between cores is crucial and stratified sampling is necessary. Parallel computation with 64 (or more) cores is efficient only for the number of particles m is 106 or more.
• Simple parallel MCF is perfectly computationally efficient but increase of the accuracy is limited for m=104 or smaller. This is because bias of the MCF cannot be reduced by simple parallel MCF.
• Weighted parallel MCF is efficient both computationally and in accuracy.
Institute of Statistical MathematicsResearch Organization of Information and Systems 52
Posterior Mean Filter/Smoother
1
1)(1)(
)(1|
)1(1|
)(|
)()1()(
)()1(
)( re whe
)1( )iii(
)1( )i(
)0(
in
in
int
int
jnt
in
in
jn
mnn
u
sss
ppf
pp
MCF PMF
1
1)(1)(
)(1|
)(|
)()(|
)()(1
)(1)(1
1)(1
1)(
)(
)()(
)()()(
)(
)( re whe
)iii(
)ii(
(i)
)1,0(~ )4(
)|( )3(
) ,( )2(
)(~ )1(
in
in
jnt
jnt
jn
jnn
in
jn
in
jn
in
mn
jn
jnn
jn
jn
jn
jn
jn
u
ss
fs
pf
cuc
c
Uu
pyp
vfFp
vqv
Institute of Statistical MathematicsResearch Organization of Information and Systems 53
Posterior Mean Filter/Smoother (Gauss)
-3
-2
-1
0
1
2
3
1 101 201 301 401
-2
-1
0
1
2
1 101 201 301 401
-2
-1
0
1
2
0 100 200 300 400
MCF PMF
-2
-1
0
1
450 460 470 480 490
Institute of Statistical MathematicsResearch Organization of Information and Systems 54
Posterior Mean Filter/Smoother (Cauchy)
-3
-2
-1
0
1
2
3
1 101 201 301 401
MCF PMF
-2
-1
0
1
2
1 101 201 301 401
-2
-1
0
1
2
0 100 200 300 400
-1
0
1
450 460 470 480 490
Institute of Statistical MathematicsResearch Organization of Information and Systems 55
-2
-1
0
1
2
3
0 10 20 30 40 50
-2
-1
0
1
2
0 10 20 30 40 50
-2
-1
0
1
2
0 10 20 30 40 50
Posterior Mean Filter/Smoother
5.0
random },{1,
mj 5.0 ,1 jj
(0,1)~
random },{1,
U
mj
i.i.d. case
-2
-1
0
1
2
3
0 10 20 30 40 50
data ordered5.0 ,1 jj
Institute of Statistical MathematicsResearch Organization of Information and Systems 56
Posterior Mean Filter/Smoother
2/)ZZ( )(1
)(1
)( jjn
in
jnZ
)( Z,ZCov ,ZVar ,ZE )()()()( kjcv nk
nj
nnj
nnj
n
0 , , 02
00 cv
11
11
431
43
112111
21
nnn
nnn
cm
vm
c
cm
vm
v
21
21
41
213 3 )12(
41
21)1(23 )12(
n
n
n
n
mmc
mmmv
i.i.d. case: average
123limlim
m
cv nnnn
Institute of Statistical MathematicsResearch Organization of Information and Systems 57
Posterior Mean Filter/Smoother
(0,1)~ ,ZZ)1( )(1
)(1
)( UZ jjn
in
jn
)( Z,ZCov ,ZVar ,ZE )()()()( kjcv nk
nj
nnj
nnj
n
0 , , 02
00 cv
11
11
431
43
31
32
nnn
nnn
cm
vm
c
cvv
21
21
43
323 9 )94(
43
3249 )94(
n
n
n
n
mmc
mmmv
i.i.d. case: random mean
949limlim
m
cv nnnn
Institute of Statistical MathematicsResearch Organization of Information and Systems 58
Posterior Mean vs. Fixed-Lag SmootherCauchy modelGaussian model
m=102 m=103 m=105 m=106m=104m=102 m=103 m=105 m=106m=104
Institute of Statistical MathematicsResearch Organization of Information and Systems 59
Summary of Posterior Mean Smoother
• Posterior mean smoother can provide only the posterior mean.
• But its accuracy is significantly higher than the fixed-lag smoother for any lag length.
Institute of Statistical MathematicsResearch Organization of Information and Systems 60
Applications of MCF
Gordon et al. (1993), Kitagawa (1996)Doucet, de Freitas and Gordon (2001) “Sequential Monte Carlo Methods in Practice”
1. Non-Gaussian smoothingLevel shiftNon-Gaussian seasonal adjustmentStochastic volatility models
2. Nonlinear smoothingTrackingPhase-unwrapping
3. Signal extraction problems4. Modeling count data5. Self-organizing state space model6. High-dimensional filtering/smoothing
Institute of Statistical MathematicsResearch Organization of Information and Systems 61
SovCDS Index and Analysis of Relationship between Regional Sovereign Risk the Power
Contribution.
Presented at the 59th World Statistics Congress, Hong Kong, China onAugust 26, 2013
Yoko Tanokura Meiji University, TokyoHiroshi Tsuda Doshisha University, KyotoSeisho Sato The University of Tokyo, TokyoGenshiro Kitagawa ROIS, Tokyo
Institute of Statistical MathematicsResearch Organization of Information and Systems 62
Outline of the Example
Introduce Sovereign Credit Default Swap (SovCDS) and problems on SovCDS data for constructing a market index as a proxy of sovereign risk
Propose a method of index construction based on time series analysis
Construct five regional sovereign risk indices Investigate the relationships between regional sovereign risks by
using power contribution analysis and detect spillover effects of the European sovereign debt crisis
Institute of Statistical MathematicsResearch Organization of Information and Systems 63
CDSbuyer
CDSseller
periodically pay a CDS spread quoted as annual rate
make a payment on the occurrence of the credit event
Sovereign Credit Default Swap (SovCDS)
...is an insurance contract that protects the buyer against the issuer’s credit risk of the country’s debt.
... can be regarded as the market evaluation on the credit risk for the country’s economy.
Daily composite spreads (provided by Markit) 82 US dollar-denominated sovereign CDS 5-year issues September 11, 2003 – March 29, 2013
Construct a Sovereign Risk Index Based on SovCDS Price Data
Institute of Statistical MathematicsResearch Organization of Information and Systems 64
Problems: Heavy-tailed distributions Time-varying number of observations
Time Series of SovCDS Price Distributions
3/12/09
3/8/12
0% 51%
3/29/13
97% 144% 191% 232%
11/18/11
7/9/10
2/27/09
10/19/07
6/9/06
1/28/05
9/19/03
Number of observations
Institute of Statistical MathematicsResearch Organization of Information and Systems 65
Method of Market Index Construction*
is estimated by a time-varying variance model (Kitagawa 1987).
0)(log0)1)((
))(()(1
,
npnp
nphnqi
iii
Apply Box-Cox transformation (Box and Cox 1964) to the prices
Determine an optimal λ by minimizing : modified (Akaike 1973) to the original prices (Kitagawa 2010)
)()()()()1()(
nwnxHnynvGnxFnx
))()(,0(~)(),0(~)(
),()()(),(
2
2
nknNnwNnv
nwntnynvtl
)1()()( ntntnt
Obtain the posterior distribution by applying state-space modeling (Kitagawa 2010)
For each λ, fit the following trend model to the mean time series of
* Improved version of Tanokura et al. (2012)
)(1
log2AIC'AICnzzdz
dhT
n
))(()(
Jacobian:1 nyhnz
dzdh
An index is defined by the inverse Box-Cox transformation of the optimal trend.
)(ny
2)(n
'AIC
nsobservatio ofnumber :)(,...,1time:,...,1
nkiTn
AIC
)(, nqi
1
2
3
4
5
Institute of Statistical MathematicsResearch Organization of Information and Systems 66
Countries of Five Regions
LAAP EE
MADE
SovCDS Distributions
Institute of Statistical MathematicsResearch Organization of Information and Systems 67
AIC-Optimal Transformation
For all regions, we select , which is the reciprocal square root transformation. 5.0
'AIC for
Institute of Statistical MathematicsResearch Organization of Information and Systems 68
Examples of the Estimated Trend
Asia Pacific
Developed Europe
Institute of Statistical MathematicsResearch Organization of Information and Systems 69
Asia Pacific (AP) Index & price distributions
Developed Europe (DE) Index & price distributions
Sovereign Risk Index and the Price Distributions
Institute of Statistical MathematicsResearch Organization of Information and Systems 70
Five Regional Sovereign Risk Indices
RISK
Institute of Statistical MathematicsResearch Organization of Information and Systems 71
Relationships between Cyclical Fluctuations around the Trends
Cyclical component: extracted by the program package Web DECOMP developed by ISM (Gersch and Kitagawa 1983, Kitagawa and Gersch 1984)
Post-Greece: 11/17/09 -3/8/12Post-Lehman: 9/15/08 -11/16/09 Current: 3/12/12 -3/29/13
Institute of Statistical MathematicsResearch Organization of Information and Systems 72
Multivariate AR Model and Power Contributions(PC)
nmn
M
mm vxAx n
1 )( )( ),( )(
,)( ,)(
mnOxvEmnOvvE
WvvEOvE
T
T
T
mn
mn
nnn
nxmA
: 5-dim stationary time series: AR coefficient matrix: 5-dim white noise: Variance covariance matrix
nvW
PC measures the influence between variable fluctuations of the noise at a frequency.
Power spectrum (PS) of A:decomposes the fluctuation by frequency
PC of A: decomposes PS of A into components of variable combinations
(Akaike 1968, Tanokura and Kitagawa 2004)
Institute of Statistical MathematicsResearch Organization of Information and Systems 73
Power Contributions (%)
Post-Greece: 11/17/09 -3/8/12
Post-Lehman: 9/15/08 -11/16/09
Current: 3/12/12 -3/29/13
Asia Pacific (AP) Dev. Europe (DE) Emerg. Europe (EE)Mid. East/Africa (MA) Latin America (LA)
Institute of Statistical MathematicsResearch Organization of Information and Systems 74
Contribution Score(CS)
...is defined as the quantity (% of the total) between two variables based on the sum of the equally allocated the PC value to variables concerned at the dominant frequency domain of the power spectrum each region.
CS from Developed Europe can be regarded as the influence of the European debt crisis. CSs from Developed Europe for the current period become higher!
Institute of Statistical MathematicsResearch Organization of Information and Systems 75
Contribution Score (Cont.)The mutual strong contribution scores between Asia Pacific and Emerging Europe are detected for the Post-Greece period.
Institute of Statistical MathematicsResearch Organization of Information and Systems 76
Summary of this Example
We presented a method constructing an index where the price distributions are heavy-tailed in the market.
We showed the effectiveness of our method by applying to the sovereign Credit Default Swap (SovCDS) market.
We detect the worldwide spillover effects of the European debt crisis.
Applying our method to the markets with insufficient information such as fast-growing or immature markets can be effective.
Institute of Statistical MathematicsResearch Organization of Information and Systems 77
Summary
• General state-space model and sequential Monte Carlo filter/smoother.
• Computational aspect of sequential MonteCarlo method are presented
• MCF with huge number of particles• Two-filter formula• Parallel MCF• Posterior mean smoother
• Application: Sovereign risk index