1 lam eps workshop, madrid, 3-4 october 2002 ken mylne and kelvyn robertson met office poor...
Post on 25-Dec-2015
217 Views
Preview:
TRANSCRIPT
1 LAM EPS Workshop, Madrid, 3-4 October 2002
Ken Mylne and Kelvyn Robertson
Met Office
Poor Man's EPS experiments and LAMEPS plansat the Met Office
2 LAM EPS Workshop, Madrid, 3-4 October 2002
Why PEPS (Poor Man’s EPS)? Storms of Dec 1999 over Europe were poorly
forecast by most deterministic models, even at 24h– Need for effective short-range ensemble to reduce risk of
missing severe weather events
Existing operational ensembles (eg ECMWF) designed for medium-range (3-10days)
– some evidence of poor performance for severe events in short-range
PEPS is an ensemble formed by combining the operational output from several NWP centres
– provides a relatively cheap way of obtaining short-range ensemble forecasts
3 LAM EPS Workshop, Madrid, 3-4 October 2002
Why PEPS might work Multi-model multi-analysis ensemble
– experiments in USA have shown this is important (eg Hou et al, 2001; Stensrud et al, 1999)
Random sampling of initial condition errors– may be important for estimating probabilities
at short-range
Previous studies (eg Ziehmann, 2000) have shown encouraging results
4 LAM EPS Workshop, Madrid, 3-4 October 2002
Preliminary system 9 models Low-res (5x5°) H500 and pmsl
only Output every 24h Data stored and
used by VT, not DT
5 LAM EPS Workshop, Madrid, 3-4 October 2002
Verification - Brier Skill
Brier Skill Scores, using the ECMWF EPS as ref.
Several PEPS configurations– all available models
– one model removed (all versions)
– all plus 6 members of EPS
– reduced combinations
Range of PMSL thresholds 126 days from 7th Feb to 12th
June 2001
6 LAM EPS Workshop, Madrid, 3-4 October 2002
Hi-Res PEPSSuccess of the preliminary system
encouraged us to set up a much larger PEPS system:
Larger ensemble – around 15 members from 9 models
Higher resolution– tests at 1.25x1.25°
– output every 12h
More fields– PMSL
– H500
– T850
– 2m Temp
– 10m Windspeed
– Precipitation
7 LAM EPS Workshop, Madrid, 3-4 October 2002
Data Exchange 9 centres agreed to supply forecast data
Data are pulled from FTP sites in near-real time– European data via ECMWF fast link
– Other centres via the internet
– Met Office UM
– ECMWF
– DWD
– Meteo-France
– BoM
– JMA
– KMA
– CMC
– NCEP
– Russia
8 LAM EPS Workshop, Madrid, 3-4 October 2002
Brier Skill - Winter DJF 2001/02 Results similar
to preliminary experiments
Reference EPS is 12 hours older due to late data cut-off
– provides the gain which could be achieved operationally
9 LAM EPS Workshop, Madrid, 3-4 October 2002
Effect of 12h Advantage Re-ran verification
without giving PEPS the 12h advantage
Apparent PEPS skill mostly comes from the 12h advantage
Without:– No skill at T+24
– Slight advantage at T+84
With 12h Without 12h
10 LAM EPS Workshop, Madrid, 3-4 October 2002
BSS - Different Weather Parameters
PMSL H500 T850 T 2m 10m WS
T+24
T+72
Results similar for all weather parameters:-
11 LAM EPS Workshop, Madrid, 3-4 October 2002
BSS - PMSL in Regions
N. Hem. Europe N. Am. S. Hem.
PMSL results poor over S. Hemisphere.
T+72
T+24
12 LAM EPS Workshop, Madrid, 3-4 October 2002
BSS - 2m Temperature in Regions
N. Hem. Europe N. Am. S. Hem.
T2m results poor over S. Hemisphere.
Best over continents but still poorer than EPS.
T+72
T+24
13 LAM EPS Workshop, Madrid, 3-4 October 2002
BSS - Wind Speed in Regions
N. Hem. Europe N. Am. S. Hem.
Benefit for more extreme events in all regions:-
T+72
T+24
14 LAM EPS Workshop, Madrid, 3-4 October 2002
Rank Histograms PMSL over Northern
Hemisphere– over-spread at 24-
48h
– good spread but slight bias at longer lead-times
– EPS underdispersive at all times to T+120
15 LAM EPS Workshop, Madrid, 3-4 October 2002
Rank Histograms Focus on over-
spreading at T+24-48
– Northern hemisphere average hides strong regional bias over Europe
– still some over-spreading
– And an opposite regional bias over N. America
16 LAM EPS Workshop, Madrid, 3-4 October 2002
Rank Histograms Focus on over-
spreading at T+24-48
– Southern hemisphere shows stronger over-spreading
– probably due to analysis biases
Difficult to separate ensemble spread from differences in model biases
Some apparent over-spreading may be due to biases in the verifying ECMWF analysis
Need for bias correction
17 LAM EPS Workshop, Madrid, 3-4 October 2002
Rank Histograms Weather parameters PMSL
500hPa Height– Strong bias (analysis?)
– Some over-spreading
T850– Over-spreading
18 LAM EPS Workshop, Madrid, 3-4 October 2002
Rank Histograms Weather parameters PMSL
2m Temperature– Over-spreading
10m Wind Speed– Over-spreading
– Bias
19 LAM EPS Workshop, Madrid, 3-4 October 2002
Reliability Diagrams PMSL<970mb over
Northern Hemisphere– reliability good for
PEPS and for EPS
20 LAM EPS Workshop, Madrid, 3-4 October 2002
Reliability Diagrams H500<480dm over
Northern Hemisphere– some general
under-forecasting - possibly bias in ECMWF analysis, as seen in Rank Histograms
21 LAM EPS Workshop, Madrid, 3-4 October 2002
Reliability Diagrams 2m Temperature
– <260 deg C
– better reliability than EPS for all thresholds
– <280 deg C
– <300 deg C
22 LAM EPS Workshop, Madrid, 3-4 October 2002
Conclusions on PEPS PEPS advantage over EPS was due to the 12h
lag applied to EPS– little scientific advantage of PEPS method at T+24
– slight advantage at T+84 (multi-model?)
PEPS over-spread at short-range– regional biases make interpretation difficult
– some evidence for better reliability for extreme events
Experiments with bias-corrected PEPS should clarify results– set up to run over the coming winter
23 LAM EPS Workshop, Madrid, 3-4 October 2002
Plans for LAMEPS The Met Office is devising plans for a short-range
ensemble based on a LAM covering the Atlantic and Europe. Aims:
– Risk assessment for rapid cyclogenesis
– Uncertainty of sub-synoptic systems
– assess probability forecasts of precipitation, low cloud and visibility
– LBCs for future storm-scale ensembles
24 LAM EPS Workshop, Madrid, 3-4 October 2002
LAMEPS Perturbation Strategy
To be fully effective LAMEPS will need perturbations to:
Initial conditions Model physics parametrizations Lateral boundaries Surface parameters
25 LAM EPS Workshop, Madrid, 3-4 October 2002
LAMEPS Perturbation Strategy
To be fully effective LAMEPS will need perturbations to:
Initial conditions Model physics parametrizations Surface parameters Lateral boundaries
26 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations
Options: Singular vectors (as used at ECMWF) Error breeding (Toth and Kalnay, 1993) (as
used at NCEP) Ensemble data assimilation (CMC,
Houtekamer et al, 1996) Ensemble Kalman Filter (Bishop et al, 2001) Multi-analysis (INM)
27 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations Singular
vectors Error breeding Ensemble
data assimilation
Ensemble Kalman Filter
Multi-analysis
Maximise ensemble growth over early forecast range (48h at ECMWF)
Possibility of combining SVs optimised at 6h, 12 and 18h (Hollingsworth, personal communication)
Some evidence that SVs only provide reliable probabilities for severe weather events well after the optimisation period
28 LAM EPS Workshop, Madrid, 3-4 October 2002
Early Warnings of Severe Weather from EPS
Verification of severe weather warnings based on the EPS
– Discrimination of events is best at 4 days (ROC)
– Better discrimination is independent of calibration
– Reliability is best at day 4 and useless at days 1-2
4 days
1 day 2 days
3 days
29 LAM EPS Workshop, Madrid, 3-4 October 2002
Early Warnings -Brier Skill Scores
Brier Skill also tends to increase after day 2.
Heavy Rain Severe Gales
30 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations Singular
vectors Error breeding Ensemble
data assimilation
Ensemble Kalman Filter
Multi-analysis
Relatively simple to implement Identifies modes growing
rapidly at analysis time– may provide a more random
sampling in the early forecast
But… bred vectors are not
orthogonal– tend to converge
– not worth running more than 5-8 cycles
31 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations Singular
vectors Error breeding Ensemble
data assimilation
Ensemble Kalman Filter
Multi-analysis
Multiple data assimilation cycles with perturbed observations
– computationally expensive
Accounts for model errors Monte-Carlo method
– random sampling, so should provide reliable probabilities
In practice did not perform very well at CMC
– insufficient spread to scale with forecast errors
32 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations Singular
vectors Error breeding Ensemble
data assimilation
Ensemble Kalman Filter
Multi-analysis
Various configurations exist Theoretically optimal
– not tested in full NWP models
– difficulties with some obs types
– computationally expensive
Ensemble Transform Kalman Filter (Bishop et al, 2001) may provide the best system in the long-term
33 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations Singular
vectors Error breeding Ensemble
data assimilation
Ensemble Kalman Filter
Multi-analysis
Relatively cheap and simple– reliability may be a problem
Accounts for model errors No attempt to identify
rapidly growing modes Monte-Carlo method
– random sampling, so should provide reliable probabilities
PEPS results suggest:– over-spreading
– need for bias corrections
34 LAM EPS Workshop, Madrid, 3-4 October 2002
Initial Condition Perturbations Singular
vectors Error breeding Ensemble
data assimilation
Ensemble Kalman Filter
Multi-analysis
Initially we will use Error Breeding
Later we hope to develop EnKF
35 LAM EPS Workshop, Madrid, 3-4 October 2002
LAMEPS Perturbation Strategy
To be fully effective LAMEPS will need perturbations to:
Initial conditions Model physics parametrizations Surface parameters Lateral boundaries
36 LAM EPS Workshop, Madrid, 3-4 October 2002
Model Physics Perturbations
Again many options… main priorities: Convection Cloud/microphysics
– impact on radiation
Surface roughness
37 LAM EPS Workshop, Madrid, 3-4 October 2002
Model Physics PerturbationsApproaches: Multi-model
– effective – opportunity for effective collaboration
Multi-scheme– eg. Kain-Fritsch or Betts-Miller convection
Perturbed tendency– as used at ECMWF
Stochastic physics schemes– conceptually and theoretically elegant– research required - role for universities
38 LAM EPS Workshop, Madrid, 3-4 October 2002
LAMEPS Perturbation Strategy
To be fully effective LAMEPS will need perturbations to:
Initial conditions Model physics parametrizations Surface parameters Lateral boundaries
39 LAM EPS Workshop, Madrid, 3-4 October 2002
Surface Parameters
Surface Roughness– fixed but uncertain - perturb between members
– variable over sea
– impact through windspeed, heat and moisture fluxes
Soil moisture, SST, snow cover etc– analysed
– could be perturbed randomly
40 LAM EPS Workshop, Madrid, 3-4 October 2002
LAMEPS Perturbation Strategy
To be fully effective LAMEPS will need perturbations to:
Initial conditions Model physics parametrizations Surface parameters Lateral boundaries
41 LAM EPS Workshop, Madrid, 3-4 October 2002
Lateral Boundary Conditions
Large domain designed to allow uncertainties to grow within the domain, but...– By T+72 significant uncertainty may emanate
from beyond the western boundary
– Error breeding will grow modes over the previous 24h, so important even for 48h forecasts
42 LAM EPS Workshop, Madrid, 3-4 October 2002
Lateral Boundary ConditionsOptions: ECMWF Ensemble Random perturbations Global model breeding at low resolution
43 LAM EPS Workshop, Madrid, 3-4 October 2002
Lateral Boundary Conditions ECMWF Random Global
breeding
Readily available– especially if use member-state
time on ECMWF computers
But… Possible balance problems
using LBCs from different model
Each new EPS run has new perturbations - no continuity with the LAM bred modes
– likely generate noise
44 LAM EPS Workshop, Madrid, 3-4 October 2002
Lateral Boundary Conditions ECMWF Random Global
breeding
Simple to apply Usual problem of random
perturbations - not focussing on the growing modes
45 LAM EPS Workshop, Madrid, 3-4 October 2002
Lateral Boundary Conditions ECMWF Random Global
breeding
Avoids problems of others:– identifies growing modes
– continuity from run to run
But… Expensive, unless run at low
resolution– grid-length for LBCs should
not be more than 4-5 times longer
46 LAM EPS Workshop, Madrid, 3-4 October 2002
Lateral Boundary ConditionsOptions: ECMWF Ensemble Random perturbations Global model breeding at low resolution
No decision has been taken
47 LAM EPS Workshop, Madrid, 3-4 October 2002
Outline of LAMEPS Plans Ensemble based on European Mesoscale
– 20km grid-length initially– Minimum 10 members– Run to T+48, possibly to T+72 later
Error breeding - possibly EnKF later Multi-schemes for convection
– research into stochastic physics Perturbed Surface Roughness Perturbed LBCs
– ECMWF EPS or low-resolution Global breeding
48 LAM EPS Workshop, Madrid, 3-4 October 2002
Collaboration Opportunity
Dispersed multi-model ensemble Relatively simple approach to model
errors Share computing demands Share system maintenance demands Option to run multiple components at
ECMWF on member-states’ time
49 LAM EPS Workshop, Madrid, 3-4 October 2002
Planned Time-Scales Start work April 2003 Year 1 (incl. Relocation):
– Error breeding system– Convection perturbations– First test run
Year 2 (2004-2005): – Version 1 of full perturbation system– System set up for real-time running
Year 3 (2005-2006): – Verification report on real-time performance
New Met Office HQ, Exeter
top related