a localized markov chain particle filter (lmcpf) for the ... · a localized markov chain particle...
TRANSCRIPT
A Localized Markov Chain Particle Filter (LMCPF)
for the Global Weather Prediction Model ICON
Anne Sophie Walter and Roland Potthast and Andreas Rhodin
German Meteorological Service (DWD)
Data Assimilation Unit
7th International Symposium on Data Assimilation 2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 2
What is new?
• Two localized particle filters working stably in an operational setup: LAPF and
LMCPF
• Strong improvements of scores by including model error via a Gaussian
mixture approach for the prior distribution
• LMCPF partly decouples spread and assimilation functionality (via model error)
• First particle filter tests with full global operational resolution and setup,
including two-way European nest in global DA
• Particle Filter shows behavior similar to LETKF, with some scores better than
LETKF, some worse (o-b and forecast!)
• LMCPF also works similarly for COSMO convective scale
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 3
Content
1. The ICON-Model and DA-System
2. The Localized Markov Chain Particle Filter (LMCPF)
3. Numerical Testing
4. Summary
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 4
1. The ICON-Model
• Operational NWP model of DWD: ICON (ICOsahedral Nonhydrostatic)
• Two-way NEST over Europe (~6.5 km)
• Resolution: 13 km deterministic
40 km ensemble (40 member)
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 5
1. DA System at DWD
• Hybrid EnVar operational since
January 20, 2016
𝑋𝑎 = ҧ𝑥𝑏 + 𝑋𝑏𝑊
• Following Hunt et al. (2007),
(see also Schraff et al., 2016)
• EnVar-B-Matrix: 70% LETKF,
30% Climatology
22.01.2019
B-Matrix
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 6
2. The LMCPF
• Our Particle Filters are based on the LETKF philosophy
➢ Localization: as LETKF in ensemble space
➢ Adaptivity: tools for spread control
22.01.2019
PRIOR
DATA
Posterior
Analysis
Ensemble
➢ Able to handle with multi-modal
distributions
➢ Able to handle with Non-Gaussianity
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 7
2. The LMCPF
22.01.2019
LETKF
Classical PF
DATA
Prior
Posterior
DATA
Prior
Posterior
LMCPF
DATA
Prior
Posterior
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 8
2. The LMCPF
• Employ Bayes formula to calculate new analysis distribution
𝑝𝑘𝑎
𝑥 := 𝑝 𝑥 𝑦𝑘 = 𝑐 𝑝 𝑦𝑘 𝑥 𝑝𝑘𝑏
𝑥 , 𝑥 ∈ ℝ𝑛
𝑐 is a normalization factor: 𝑋 𝑝𝑘𝑎
𝑥 𝑑𝑥 = 1
• To carry out the analysis step at time 𝑡𝑘 a posteriori weights 𝑤𝑘,𝑙𝑎
are
calculated
𝑤𝑘,𝑙(𝑎)
= 𝑐 𝑒−12 𝑦−𝐻𝑥 𝑙 𝑇
𝑅−1(𝑦−𝐻𝑥 𝑙 )
𝑐 is chosen such that σ𝑙=1𝐿 𝑤𝑘,𝑙
(𝑎)= 𝐿
22.01.2019
First Step: The Classical Particle Filter
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 9
2. The LMCPF
• Accumulated weights 𝑤𝑎𝑐 are defined:
𝑤𝑎𝑐0 = 0
𝑤𝑎𝑐𝑙 = 𝑤𝑎𝑐𝑙−1 +𝑤𝑙𝑎 , 𝑙 = 1,… , 𝐿
where 𝐿 denotes the ensemble size
• Drawing 𝑟𝑙~𝑈 0,1 , 𝑙 = 1,… , 𝐿, set 𝑅𝑙 = 𝑙 − 1 + 𝑟𝑙 and define transform
matrix ෲ𝑾 for the particles by:
ෲ𝑊𝑖,𝑙 = ൝1 𝑖𝑓 𝑅𝑙 ∈ 𝑤𝑎𝑐𝑙−1 , 𝑤𝑎𝑐𝑙 ,
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒,
𝑖, 𝑙 = 1,… , 𝐿 with 𝑊 ∈ ℝ𝐿𝑥𝐿, (𝑠, 𝑡] denotes the interval of values 𝑠 < 𝜂 ≤ 𝑡.
22.01.2019
Second Step: Classical Resampling
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 10
2. The LMCPF
• Based on the adaptive multiplicative inflation factor 𝝆 determined by the
LETKF
𝜌 =Ε 𝒅𝑜−𝑏
𝑇 𝒅𝑜−𝑏 − Tr(𝐑)
𝑇𝑟(𝑯𝑷𝑏𝑯𝑇)
• Weighting factor 𝜶 has been chosen, due to the small ensemble size (𝐿 = 40)
𝜌𝑘 = 𝛼 𝜌𝑘 + 1 − 𝛼 𝜌𝑘−1
22.01.2019
Third Step: Spread Control
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 11
2. The LMCPF
• Perturbation factor 𝜎 is used to add spread to the system
𝜎 =
𝑐0, 𝜌 < 𝜌(0)
𝑐0 + 𝑐1 − 𝑐0 ∗𝜌 − 𝜌(0)
𝜌(1) − 𝜌(0), 𝜌(0) ≤ 𝜌 ≤ 𝜌(1)
𝑐1, 𝜌 > 𝜌(1)
where 𝑐0 = 0.02, 𝑐1 = 1.5, 𝜌(0) = 1.0 and 𝜌(1) = 1.4, with 𝜎 = 𝑐1if 𝜌 ≥ 𝜌(1) and
𝜎 = 𝑐0 if 𝜌 ≤ 𝜌(0)
22.01.2019
Third Step: Spread Control
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 12
2. The LMCPF
෨𝐵 = 𝐼 − 𝐾𝐻 𝐵
= 𝐼 − 𝐵𝐻𝑇 𝑅 + 𝐻𝐵𝐻𝑇 −1𝐻 𝐵
= 𝐼 − 𝛾𝑋 𝑋𝑇𝐻𝑇 𝑅 + 𝛾 𝐻𝑋 𝑋𝑇𝐻𝑇 −1𝐻 𝛾𝑋𝑋𝑇
= 𝑋(𝐼 − 𝛾 𝑌𝑇(𝑅 + 𝛾𝑌𝑌𝑇)−1𝑌)𝛾𝑋𝑇
= 𝑋 𝐼 − 𝛾 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝑌𝑇𝑅−1𝑌 𝛾X𝑇
= 𝑋 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1(𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 − 𝛾𝑌𝑇𝑅−1𝑌) 𝛾𝑋𝑇
= 𝑋 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝛾𝑋𝑇
= 𝛾𝑋 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝑋𝑇
22.01.2019
Fourth Step: Determine posterior B ≡ ෨𝐵
𝐾 = 𝐵𝐻𝑇 𝑅 + 𝐻𝐵𝐻𝑇 −1
𝐵 = 𝛾𝑋𝑋𝑇
𝐻𝑋 = 𝑌
𝑌𝑇 𝑅 + 𝛾𝑌𝑌𝑇 −1 = 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝑌𝑇𝑅−1
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 13
2. The LMCPF
𝛽(𝑠ℎ𝑖𝑓𝑡,𝑙) = 𝐼 + 𝛾Y𝑇𝑅−1𝑌 −1𝑌𝑇𝑅−1 𝐶 − 𝑒𝑙
with 𝐴 = 𝑌𝑇𝑅−1𝑌, 𝐶 = 𝐴−1𝑌𝑇𝑅−1(𝑦0 − ത𝑦𝑏)
𝑊(𝑠ℎ𝑖𝑓𝑡) ≔ 𝛽 𝑠ℎ𝑖𝑓𝑡,1 , … , 𝛽(𝑠ℎ𝑖𝑓𝑡,𝐿) ∈ ℝ𝐿×𝐿
Important effects of 𝛽(𝑠ℎ𝑖𝑓𝑡,𝑙):
• it moves the particles towards the observations in ensemble space
• by the use of model error, it constitutes a further degree of freedom which can be used for
tuning of a real system
22.01.2019
Fifth Step: Calculation of the Shift-Vector 𝛽(𝑠ℎ𝑖𝑓𝑡,𝑙)
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 14
2. The LMCPF
• Weights W are calculated by drawing from the posterior.
𝑊 = 𝑊 +𝑊(𝑠ℎ𝑖𝑓𝑡) ∗ 𝑊 + ෨𝐵 ∗ 𝑅𝑛𝑑 ∗ 𝜎
22.01.2019
Sixth Step: Gaussian Resampling
with 𝑅𝑛𝑑 normally distributed random numbers,
𝑊(𝑠ℎ𝑖𝑓𝑡) and ෨𝐵 calculated with Gaussian radial
basis function (rbf) Approximation for prior
density and observation error
DATA
Prior
Posterior
➢ It is an explicit calculation of the Bayes posterior based on radial basis
function approximation of the prior.
Centers of posterior RBFs Shape of posterior RBFs
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 16
• Full ensemble: 40 members
• Reduced resolution:
• 26km deterministic
• 52km ensemble
• Period:
06.05.2016 – 31.05.2016
• Full ensemble: 40 members
• Routine resolution:
• 13km deterministic
• 40km ensemble
➢ two-way NEST:
• 6.5km deterministic
• 20km ensemble
• Period: 01.09.2018 – 04.09.2018
• Lot more satellite data: SEVIRI, IASI
& ATMS humidity channels
Experimental setup (A) Experimental setup (B)
3. Numerical Testing
T [K] at 03.05.2016 15 UTC ~1000 hPa
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 18
3. Numerical Testing – (A)
22.01.2019
RMSE
LETKF
LMCPF
RMSE
p-level [h
Pa]
Global RMSE for obs-fg statistics (Radiosondes vs. Model)
Period: 10.05.2016 – 31.05.2016
Relative humidityTemperature
~3%
~7%
~4%
~2% better
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 19
3. Numerical Testing – (A)
22.01.2019
RMSE
LETKF
LMCPF
RMSE
Channel num
ber
Global RMSE for obs-fg statistics
Period: 10.05.2016 – 31.05.2016
Bending Angles (GPSRO)𝑇𝐵𝑟𝑖𝑔ℎ𝑡𝑛𝑒𝑠𝑠 (IASI)
~6%~15%
Heig
ht
[m]
~17%
~3%
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 21
3. Numerical Testing – (B)
22.01.2019
RMSE
LETKF
LMCPF
RMSE
p-level [h
Pa]
Global RMSE for obs-fg statistics (Radiosondes vs. Model)
Period: 01.09.2018 – 04.09.2018
Relative humidityTemperature
~3%
~2%
~3% better
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 22
3. Numerical Testing – (B)
22.01.2019
RMSE
LETKF
LMCPF
RMSE
Channel num
ber
Global RMSE for obs-fg statistics
Period: 01.09.2018 – 04.09.2018
AIREP (WIND)IASI (𝑇𝐵𝑟𝑖𝑔ℎ𝑡𝑛𝑒𝑠𝑠)
~2%
~2% p-level [h
Pa]
~1% better
~6%
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 23
3. Numerical Testing
22.01.2019
mean
min
max
LETKF
LAPF
LMCPF
Global spread of T [K] ~ 500 hPa
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 25
3. Numerical Testing – (A)
22.01.2019
Global verification of different lead times against Radiosondes
Period: 10.05.2016 – 24.05.2016
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 26
3. Numerical Testing – (A)
22.01.2019
Global verification of different lead times against Radiosondes
Period: 10.05.2016 – 24.05.2016
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 27
4. Summary
• Two localized particle filters working stably in an operational setup: LAPF and
LMCPF
• Strong improvements of scores by including model error via a Gaussian
mixture approach for the prior distribution
• LMCPF partly decouples spread and assimilation functionality (via model error)
• Particle Filter shows behavior similar to LETKF, with
some scores better than LETKF, some worse (o-b and forecast!)
• First particle filter tests with full global operational resolution and setup,
including two-way European nest in global DA
• LMCPF also works similarly for COSMO convective scale
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 28
References
• Gerald Desroziers and Serguei Ivanov. “Diagnosis and adaptive tuning of observation-error
parameters in a variational assimilation”. Quarterly Journal of the Royal Meteorological Society,
2001
• Brian Hunt, Eric Kostelich and Istvan Szunyogh. “Efficient data assimilation for spatiotemporal
chaos: A local ensemble transform Kalman filter”. Physica D: Nonlinear Phenomena, 2007
• Gen Nakamura and Roland Potthast. “Inverse Modeling”. IOP Publishing, 2015
• Roland Potthast, Anne Walter and Andreas Rhodin. “A Localized Adaptive Particle Filter (LAPF)
within an operational NWP Framework”. MWR, 2019
• Sebastian Reich and Colin Cotter. “Probabilistic Forecasting and Bayesian Data Assimilation”.
Cambridge University Press, 2015
• Christoph Schraff, Hjendrick Reich, Andreas Rhodin, Annika Schomburh, Klaus Stehphan, Africa
Periáñez and Roland Potthast. “Kilometre-scale ensemble data assimilation for the cosmo model
(kenda)”. Quarterly Journal of the Royal Meteorological Society, 2016
• Peter Jan van Leeuwen, Yuan Cheng and Sebastian Reich. “Nonlinear Data Assimilation”. Frontiers
in Applied Dynamical Systems: Reviews and Tutorials. Springer, 2015
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 31
Appendix
Global histogram of number of particles with weight ≥ 1
31.05.2016 21 UTC ~1000 hPa
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 32
Appendix
31.05.2016 21 UTC ~ 100 hPa
31.05.2016 21 UTC ~ 500 hPa
31.05.2016 21 UTC ~1000 hPa
Global histograms of number of
particles with weight ≥ 1
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 33
Global Model Verification Comparison
Created: 17.01.2019
EnVarICON
22.01.2019
[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 34
Global Model Verification Comparison -
EPS
Created: 17.01.201922.01.2019