a.montani; the cosmo-leps system. cosmo-leps and cosmo-s14-eps: what is old, what is new. andrea...
TRANSCRIPT
A.Montani; The COSMO-LEPS system.
COSMO-LEPS and COSMO-S14-EPS:
what is old, what is new.
Andrea Montani,
C. Marsigli, T. Paccagnella
ARPA-SIMC
HydroMeteoClimate Regional Service of Emilia-Romagna, Bologna,
Italy
COSMO General meetingSibiu, 2-5 September 2013
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-LEPS: about the operational verification, about the calibrated precipitation, about the convection schemes, about the clustering technique, about the future plans.
• Present status of COSMO-S14-EPS: about the main features, about case-study assessment, about the plans.
A.Montani; The COSMO-LEPS system.
COSMO-LEPS suite @ ECMWF: present status
d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3
older EPSolder EPS
younger EPSyounger EPS
clustering clustering periodperiod
0000
1212
Cluster Analysis and RM identificationCluster Analysis and RM identification
4 variables4 variables
Z U V QZ U V Q
3 levels3 levels
500 700 850 hPa500 700 850 hPa
2 2 time time stepssteps
Cluster Analysis and RM identificationCluster Analysis and RM identification
European European areaarea
Complete Complete LinkageLinkage
16 Representative Members driving the 16
COSMO-model integrations (weighted according to the cluster
populations)
using either Tiedtke or Kain-Fristch convection scheme (members 1-8 T,
members 9-16 KF) +
perturbations in turbulence scheme and
in physical parameterisations
COSMO-LEPS
clustering area
• suite runs twice a day (00 and 12UTC) as a “time-critical application” managed by ARPA-SIMC;
• Δx ~ 7 km; 40 ML; fc+132h;• COSM0 v4.26 since Jan 2013;• computer time (50 million
BUs for 2013) provided by the ECMWF member states in COSMO.
COSMO-LEPS
Integration Domain
A.Montani; The COSMO-LEPS system.
Main changes during the COSMO year
• 16 January 2013: COSMO upgrade: 4.21 4.26;
int2lm upgrade: 1.18 1.20.
• 17 January 2013: operational dissemination implemented for ARPA-Veneto.
• 22 January 2013, technical changes at ECMWF:
• change of ECMWF super-computer and of the user running the suite: itm zcl;
• introduction of a new “dissemination stream” for COSMO-LEPS: “ad-hoc” initial and boundary conditions do not have to be retrieved any more, but are prepared on a dedicated file system; product dissemination starts about 40 minutes earlier than before (at 9UTC and 21UTC).
• 7 May 2013: enriched test dissemination implemented for HNMS.
• 13 May 2013: in the framework of GEOWOW research project, COSMO-LEPS was the first system (!!!) to populate TIGGE-LAM archive at ECMWF (high-priority parameters in grib2 format).
• 25 June 2013: tests with Fieldextra 11.1.0 started.
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-LEPS: about the operational verification,
A.Montani; The COSMO-LEPS system.
– SYNOP on the GTS
Time-series verification of COSMO-LEPS
Main features:
variable: 12h cumulated precip (18-06, 06-18
UTC);
period : from Dec 2002 to Jul 2013;
region: 43-50N, 2-18E (MAP D-PHASEPHASE
area);
method: nearest grid point; no-weighted fcst;
obs: synop reports (about 470
stations/day);
fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h;
thresholds: 1, 5, 10, 15, 25, 50 mm/12h;
system: COSMO-LEPS;
scores: ROC area, BSS, RPSS, Outliers, …
both monthly and seasonal scores were computed
A.Montani; The COSMO-LEPS system.
Time series of ROC area (6-month running mean)
Area under the curve in the HIT rate vs FAR diagram; the higher, the better …Valuable forecast systems have ROC area values > 0.6.
Highest scores in the 2nd part of 2011 and, for the highest threshold, in 2013.
Drier seasons during 2011 and 2012 with few heavy-precipitation events: limited significance of the results for the 15mm threshold.
fc 30-42h: ROC area is high for last winter and spring. Positive trend can be noticed.
fc 78-90h: the best scores date back to the end of 2011.
Limited loss of predictability with increasing forecast range.
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-LEPS: about operational verification (time-series scores show improvements),
about the calibrated precipitation;
A.Montani; The COSMO-LEPS system.
about calibrated precipitation
• For each COSMO-LEPS member, calibrated precipitation is operationally generated over Germany, Switzerland and Emilia-Romagna; the calibration technique is based on CDF-based corrections, making use of COSMO-LEPS reforecast.
• For MAM2013, inter-comparison between raw and calibrated 24h TP forecast.Main features:
variable: 24h cumulated precip (06-06 UTC);
period : DJF 2012-13 and MAM 2013;
region: Germany, Switzerland, Emilia-
Romagna;
method: nearest grid point; no-weighted fcst;
obs: synop reports (about 300 stations/day);
fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h;
thresholds: 1, 5, 10, 15, 25, 50 mm/12h;
system: opecleps and Calibcleps;
scores: ROC area, BSS, RPSS, Outliers, RelDiag, …
A.Montani; The COSMO-LEPS system.
opecleps vs Calibclepsfc 42-66h; 10mm/24h
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-LEPS: about operational verification (time-series scores show improvements),
about calibration (positive impact, especially over Emilia-Romagna!);
about convection schemes,
members 1-8 use Tiedtke convection scheme (8TD),
members 9-16 use Kain-Fritsch (8KF).
MAM 2013: compare cleps16, 8TD, 8KF.
A.Montani; The COSMO-LEPS system.
about the convection scheme
BSS, tp > 1mm
ROC, tp > 10mm
ROC, tp > 1mm
BSS, tp > 10mm
• As expected, best performance by the full ensemble (cleps16).• Tiedtke-members better than Kain-Fritsch members, but NOT for
all scores.
___ cleps16___ 8TD ___ 8KF
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-LEPS:about operational verification (time-series scores show improvements);
about calibration (positive impact, especially over Emilia-Romagna!);
about convection schemes (Tiedtke slightly superior to Kain-Fritsch);
about the clustering technique
A.Montani; The COSMO-LEPS system.
about the clustering technique
• Consider distances between ECMWF EPS members according to “COSMO-LEPS metric” (Z, U, V, Q in the mid-lower troposphere over the clustering domain).
• Look at distances between pairs of ECMWF EPS members: to what extent do these distances grow with forecast range, using “COSMO-LEPS metric”?
• Study a number of seasons.• Compare against random choice. Outcome: modifications to the number of clusters / number of EPS
considered / clustering intervals.
AIM: provide limited-area ensembles (either convection-parameterised or convection-permitting) with the best set of boundary conditions.
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-LEPS: about operational verification (time-series scores show improvements),
about calibration (clear positive impact of calibration);
about convection schemes (Tiedtke slightly superior to Kain-Fritsch);
about the clustering technique (work in progress);
about the future plans.
A.Montani; The COSMO-LEPS system.
• Adapt COSMO-LEPS suite to ECWMF forthcoming upgrades:– increase of vertical resolution in ECMWF-EPS: 62 91;
– change of Member-State server: ecaccess ecgb;
– change of super-computer: IBM Cray;
• Carry on study about the clustering methodology.
• Increase of COSMO-LEPS vertical resolution (40 50ML): tests start in October.
• LAMEPS_BC project: test with high-resolution ECMWF-EPS boundaries start by
the end of 2013.
• Analysis of the performance of COSMO-HYBEPS (COSMO-LEPS + 2-3 COSMO
runs nested on IFS/GME/GFS).
• Strengthen links with Fieldextra.
about the future plans
Any request for modifications to the present configuration of COSMO-LEPS?
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-S14-EPS: about the main features, about case-study assessment, about the plans.
A.Montani; The COSMO-LEPS system.
Milestones of COSMO-S14-EPS
• 11/3/2011. Nothing present.
• 2/5/2011. Submission of a new ECMWF Special Project (Title: “Implementation of a limited-area ensemble prediction system for Sochi Olympic Games”; Project investigators: Majewski, Montani, Steiner; duration: 3 years) for provision of computer time to run the system on ECMWF super-computers;
• 5-9/9/2011. Discussion during the COSMO meeting about the system set-up;
• 6/12/2011. Approval of the Special Project by ECMWF Council;
• 6-9/12/2011. Visit of Russian colleagues at ARPA-SIMC to define specifics of the new ensemble system;
• 19/12/2011. Beginning of provision of COSMO-S14-EPS products on a daily basis.
• 1/11/2012. Initial conditions of soil fields (temperature, moisture, snow cover) are no more interpolated from ECMWF EPS, but provided by COSMO run in hindcast mode (approach already tested for COSMO-LEPS in 2011) .
A.Montani; The COSMO-LEPS system.
COSMO-S14-EPS @ ECMWF: present status
d+3d+3dd d+2d+2d+1d+1
ECMWF EPS
clustering interval
Cluster Analysis and RM identificationCluster Analysis and RM identification
4 variables
Z U V Q
3 levels
500 700 850 hPa
2 time steps
Cluster Analysis and RM identificationCluster Analysis and RM identification
Black-Sea area
Complete Linkage
10 Representative Members driving the 10
COSMO-model integrations (weighted according to the cluster
populations)
employing either Tiedtke or Kain-Fristch
convection scheme (randomly choosen)
+perturbations in
turbulence scheme and in physical
parameterisations
clustering area
• Δx ~ 7 km; 40 ML; fc+72h;• initial time: 00/12 UTC;• computer time (~ 4.5 million
BUs for 2013) is provided by an ECMWF Special Project;
• contributions from Switzerland national allocation were needed;
• suite managed by ARPA-SIMC.
Integration Domain
A.Montani; The COSMO-LEPS system.
Disseminated products
post-processing uses COSMO-software fieldextra:
probability fields for the exceedance of thresholds for surface fields;
ensemble mean and ensemble standard deviation for some fields;
individual ensemble member runs (ICs and BCs from 10 selected EPS members): start at 00UTC and 12UTC; t = 72h;
1 deterministic run (ICs and BCs from the deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 00UTC and 12UTC; t = 72h;
provision of hourly boundary conditions (from fc+0h to fc+48h) for convective-resolving ensemble (RDP part);
provision of hourly boundary conditions (from fc+0h to fc+48h) for higher-resolution deterministic modelling (RDP part).
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-S14-EPS: about the main features, about case-study assessment,
A.Montani; The COSMO-LEPS system.
Case-study assessment
1. Heavy-precipitation event on 13 Jan 2013: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana);
2. Foehn event on 14-15 Feb 2013: a sudden 10-degree warming, which forecasters regard as hardly predictable.
Sochi local time = UTC + 4 hours
UTC= Sochi local time – 4 hours
The fields of the next slides are operationally delivered to FROST-server
A.Montani; The COSMO-LEPS system.
Heavy precipitation event (1)
OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana)
runs start at 00UTC of 11/1/2013 and verify at 12UTC of 13/1/2013
(fc+48-60h)
sfp12h_gt_15cm
tpp12h_gt_20mm
rain12h_gt_20mm
A.Montani; The COSMO-LEPS system.
Heavy precipitation event (2)
OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana)
runs start at 12UTC of 11/1/2013 and verify at 12UTC of 13/1/2013
(fc+36-48h)
tpp12h_gt_20mm
rain12h_gt_20mm sfp12h_gt_15cm
A.Montani; The COSMO-LEPS system.
Heavy precipitation event (3)
OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana)
runs start at 00UTC of 12/1/2013 and verify at 12UTC of 13/1/2013
(fc+24-36h)
rain12h_gt_20mm sfp12h_gt_15cm
tpp12h_gt_20mm
A.Montani; The COSMO-LEPS system.
Heavy precipitation event: consistency
OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana)
Fix event (probability of 12h snowfall exceeding 15 mm of equivalent water) and verification time (12UTC of 13/1/2013): consider model runs with different forecast ranges.
fc +48-60h
fc +36-48h fc +24-36h
A.Montani; The COSMO-LEPS system.
Outline
• Present status of COSMO-S14-EPS:about the main features, about case-study assessment (good performance of the system),
about the plans.
A.Montani; The COSMO-LEPS system.
• “Survive” ECMWF upgrades (increase of vertical resolution, change
of member-state server, change of super-computer).
• Develop new products “on demand” (in the next weeks, “multi-
model” ensemble products will be tested).
• Use good-quality observations at high resolution (and there are) to
perform statistical verification.
about the plans
•To what extent are forecasters
using COSMO-S14-EPS products?
•How to strengthen links between
forecasters and ensemble
developers?
training with forecasters in Sochi next October
A.Montani; The COSMO-LEPS system.
Thank you !
European Conference on Applications of Meteorology / EMS annual meeting
09 – 13 September 2013, Reading (UK)
Session NWP4 (on 13 September): Probabilistic and ensemble forecasting at short and and medium-range
http://www.ems2013.net/home.html
A.Montani; The COSMO-LEPS system.
Extra slides on configuration
A.Montani; The COSMO-LEPS system.
Dim
2
Initial conditions Dim 1
Dim
2
Possible evolution scenarios
Dim 1 Initial conditions
ensemble size reduction
Cluster members chosen as representative members (RMs)
LAM integrations driven byRMs
LAM scenario
LAM scenario
LAM scenario
COSMO-LEPS methodologyCOSMO-LEPS methodology
A.Montani; The COSMO-LEPS system.
COSMO-HYBrid Ensemble Prediction System
From the results of CONSENS PP, come to a synthesis with the different ensemble systems / strategies, considering scientific, implementation, solidity aspects.
Generate 20-member hybrid ensemble (COSMO-HYBEPS) , where:
a) 16 members comes from COSMO-LEPS,
b) 1 member is nested on IFS (uses Tiedtke scheme),
c) 1 member is nested on IFS (uses Kain-Fritsch scheme),
d) 1 member is nested on GME,
e) 1 member is nested on GFS.
already existing taken from CONSENS.
All members have Δx ~ 7 km; 40 ML; fc+132h;
Study performance of different members’ combinations with the same ensemble size.
“20-members esuite” implemented on 7/9/2012;
will be run up to the end of the year
A.Montani; The COSMO-LEPS system.
COSMO-LEPS (developed at ARPA-SIM)
• What is it?It is a Limited-area Ensemble Prediction System (LEPS),
based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland).
• Why?It was developed to combine the advantages of global-
model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of severe and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)
generation of COSMO-LEPS to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of
severe weather events.
A.Montani; The COSMO-LEPS system.
Operational set-up
Core products:16 perturbed COSMO-model runs (ICs and 3-
hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h;
Additional products: 1 deterministic run (ICs and 3-hourly BCs from the
high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h;
1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h.
A.Montani; The COSMO-LEPS system.
Types of perturbations
As for types and values, the results from CSPERT experimentation
were followed (* denotes default values for COSMO v4.26 ):
•convection_scheme: Tiedtke* (members 1-8), Kain-Fritsch (members 9-16),
•tur_len (either 150, or 500*, or 1000),
•pat_len (either 500*, or 2000),
•crsmin (either 50, or 150*, or 200),
•rat_sea (either 1, or 20*, or 40),
•rlam_heat (either 0.1, or 1*, or 5),
•mu_rain : either 0.5* (with rain_n0_factor =0.1) or 0 (with rain_n0_factor =1.0),
•cloud_num (either 5x10^8* or 5x10^7).
• convection scheme: T=Tiedtke KF=Kain-Fritsch;
• tur_len: maximal turbulent length scale (default 500m); this parameter is used mainly in the
calculation of the characteristic length scale for vertical mixing and thus into the calculation of
the vertical transport momentum coefficient;
• pat_len: length scale of thermal surface patterns (default 500m); this parameter is mainly
used in the calculation of the large-scale part of the equation addressing the heat flux
parameterisation; horizontal length;
• rlam_heat: scaling factor of the laminar layer depth (default 1); it defines the layer with non-
turbulent characteristics (molecular diffusion effects only);
• rat_sea: ratio of laminar scaling factors for heat over sea (default 20);
• crsmin: minimal stomata resistance (default 150);
• Cloud_num: Cloud droplet number concentration;
• Mu_rain: Exponent of the raindrop size distribution;
•( gscp: Switch on/off of the graupel scheme).
A.Montani; The COSMO-LEPS system.
Main results
Time-series verificationECMWF EPS changed substantially in the last years (more and more weight to EDA-
based perturbations) and it is hard to disentangle improvements related to COSMO-
LEPS upgrades from those due to better boundaries; nevertheless:– high values of BSS and ROC area for the probabilistic prediction of 12-h precipitation for
autumn 2011;
– poor performance in the first months of 2012, then recovery. Need to investigate what
happened.
Case-study verificationConsistent signal for different forecast ranges of a high-impact weather event for the
snowfalls of February 2012.
A.Montani; The COSMO-LEPS system.
Extra slides on COSMO-S14-EPS
A.Montani; The COSMO-LEPS system.
Important ingredients (from 1st and 2nd FROST meetings)
1. Provide reasonable “numbers”. addressed
2. Develop experience with probabilities. ?
3. Feedback on the top-priority products. being addressed
4. Snow analysis. ?
5. Soil-field initialisation. addressed
6. High-res obs to assess the quality of the system. being addressed
7. Computer time. addressed
8. Timeliness in product delivery. addressed
9. ......
anything to add/remove?
A.Montani; The COSMO-LEPS system.
FROST-2014 vs SOCHMEL
1) Introduction to FROST-2014:
a) What is it?
b) What has to do with COSMO?
2) COSMO ensemble activities within FROST-2014:
a) introduction to SOCHMEL (the SOCHi-targeted Mesoscale EnsembLe system)
b) methodology;
c) phases of development;
d) planned activity.
3) Final remarks.
A.Montani; The COSMO-LEPS system.
Extra slides on verification
A.Montani; The COSMO-LEPS system.
Seasonal scores of ROC and BSS: last 4 springs
Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges.
Valuable forecast systems have ROC area values > 0.6 and BSS > 0.
Need to take into account the different statistics for each season (MAM 2011 was the driest).
Best performance for the spring 2011 and 2013, but less marked diurnal cycle in 2013.
Spring 2013: BSS is positive for all forecast ranges
Similar results for the other thresholds (not shown).
A.Montani; The COSMO-LEPS system.
Outliers: time series + ………seas scores (DJF)?
How many times the analysis is out of the forecast interval spanned by the ensemble members. … the lower the better … Performance of the system assessed as time series and for the last 4 winters.
Evident seasonal cycle (more outliers in winter).
Overall reduction of outliers in the years up to 2007; then, again in 2009 and 2010, but later.
Need to take into account the different statistics for each season.
In the short range, best results for winter 2010-2011.
For longer ranges, the performance of the system is “stable”.
Outliers before 10% from day 3 onwards.
A.Montani; The COSMO-LEPS system.
Time series of Brier Skill ScoreBSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if
BSS > 0.BS measures the mean squared difference between forecast and observation in probability
space. BS equivalent to MSE for deterministic forecast. DA FINIREEEEEEEEEEEEEEEEEE
fc 30-42h: very good scores in 2010 and 2011; BSS positive for all thresholds since April 2009; fewer and fewer problems with high thresholds.
fc 78-90h: good trend in 2010 and 2011 for all thresholds.
In the last months, “spread” in BSS values for the different threshold values, possibly due to the lack of events.
Month-to-month variability is higher than for the ROC area.
A.Montani; The COSMO-LEPS system.
Seasonal scores of BSS: ……last 4 winters
Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges.
Fixed forecast range (fc 30-42h): consider the performance of the system for increasing thresholds.Need to take into account the different statistics for each season (last DJF was the driest).
Fixed event: best performance for the last two winters (ECMWF EPS had a record performance for winter 2009-2010): BSS positive for all forecast ranges.
Fixed forecast range: similar results as before.
Similar results for longer forecast ranges and for higher thresholds.
A.Montani; The COSMO-LEPS system.
Ranked Probability Skill Score: time series + …….. seasonal scores (MAM)
A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.
Useful forecast systems for RPSS > 0. Performance of the system assessed as time series and for the last 4 springs (MAM).
the increase of the COSMO-LEPS skill is detectable for 3 out of 4 forecast ranges along the years, BUT
low skill in the first months of 2012 (the problem comes from MAM), then recovery.Best results for MAM 2011; quick decrease of RPSS with fcst range for MAM 2012.
A.Montani; The COSMO-LEPS system.
Bias and rmse of T2M Ensemble Mean
Consider bias (the closer to zero, the better) and rmse (the lower the better).
Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite. The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time
verification. The signal is stable (similar scores for 1-month or 3-month verification).Need to correct T2M forecasts with height to assess the impact more clearly.
A.Montani; The COSMO-LEPS system.
Overestimation of Td2m and soil moisture (1)
Verification period: MAM07 and MAM08.
Obs: synop reports (about 470 stations x day).
Region: 43-50N, 2-18E (MAP D-PHASE area).
Larger bias and larger rmse in MAM08 rather than in MAM07 for COSMO-LEPS deterministic run (in 2007, no multi-layer soil model).
A.Montani; The COSMO-LEPS system.
Score dependence on the domain size (1)
Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom):
different statistics of the verification samples; up to now, performance of the system over the 2 domains assessed only for 6
months (March-August 2007). difficult to draw general conclusions
A.Montani; The COSMO-LEPS system.
Score dependence on the domain size (2)
RPSS
RPSS score… the higher the better… (and positive).
ROC area… the higher the better… (and above 0.6).Smoother transitions from month to month in “fulldom” scores.
Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month.
Higher predictability with orographic forcing?
Need to check individual regions and/or to stratify for type of stations.
OUTL
ROC
Outliers percentage … the lower the better.
A.Montani; The COSMO-LEPS system.
Semi-diurnal cycle in COSMO-LEPS scores
BSS score … the higher the better … Performance of the system assessed for 5 different Summers (JJA).
BSS Evident 12-hour cycle in
BSS scores (the same holds for RPSS, while less evident for ROC area scores).
Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …).
The amplitude of the cycle is somewhat reduced throughout the years and with increasing forecast range.
The bad performance in Summer 2006 is confirmed.
A.Montani; The COSMO-LEPS system.
about eps16
Period: March 2013 (62 cases)
Variable: Z500 ensemble mean
Area: 60./-10./30./30.
Common negative bias (too cycclonic) The operational selection choice may not be the optimal one!
A.Montani; The COSMO-LEPS system.
Extra slides on LAMEPS-BC
A.Montani; The COSMO-LEPS system.
Test data for LAMEPS Boundary Conditions
A.Montani; The COSMO-LEPS system.
Proposed region for archiving LAMEPS BC data
A.Montani; The COSMO-LEPS system.
Outline
• Introduction:
migration to the 7-km system.
COSMO-LEPS 10 km (old)
COSMO-LEPS 7 km (new)
A.Montani; The COSMO-LEPS system.
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
tp > 1mm/24h
tp > 5mm/24h
Average values (boxes 0.5 x 0.5)MAM06
As regards AVERAGE precipitation above these two threshols, the 3 systems have similar performance.
A.Montani; The COSMO-LEPS system.
ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (1)
A.Montani; The COSMO-LEPS system.
ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (2)
Observed precipitation between 15-11-2002 12UTC and 16-11-2002 12 UTC
Piedmont case
A.Montani; The COSMO-LEPS system.
Why Limited Area Ensemble Prediction?
• Global Ensemble Prediction Systems
– have become extremely important tools to tackle the problem of predictions beyond day 2
– are usually run at a coarser resolution with respect to deterministic global predictions → skill in forecasting intense and localised events is currently still limited.
A.Montani; The COSMO-LEPS system.
Why Limited Area Ensemble Prediction?(2)
As regards high resolution deterministic forecast in the short range, where limited-area models play the major role, a “satisfactory” QPF is still one of the major challenges. The same can be said for other local parameters.
This is due, among other reasons, to the inherently low degree of predictability typical of severe and localised events.
Probabilistic/Ensemble approach is so required also for the short range at higher resolution
A.Montani; The COSMO-LEPS system.
In the last period the verification package is being developed keeping into account two measure of precipitation:
the cumulative volume of water deployed over a specific region
the rainfall peaks which occur within this region
OBJECTIVE VERIFICATION OF COSMO-LEPS
COSMO observations
A.Montani; The COSMO-LEPS system.
CLEPS EPS
Verification grid
OBS MASK
A.Montani; The COSMO-LEPS system.
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
tp > 1mm/24h
tp > 5mm/24h
Average values boxes 0.5x0.5 deg3 sis
A.Montani; The COSMO-LEPS system.
COSMO-LEPS
16-MEMBER EPS
tp > 1mm/24h
NOCC=610
NOCC=1195
tp > 5mm/24h
NOCC=2671
tp > 10mm/24h
ave 0.5
A.Montani; The COSMO-LEPS system.
Maximum values boxes 0.5x0.5 deg
tp > 1mm/24h
tp > 5mm/24h
tp > 10mm/24h
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
3 sis
A.Montani; The COSMO-LEPS system.
COSMO-LEPS
16-MEMBER EPS
tp > 20mm/24hNOCC=227
Average values boxes 0.5x0.5 deg
A.Montani; The COSMO-LEPS system.
COSMO-LEPS NW
COSMO-LEPS W
tp > 1mm/24h
COSMO-LEPS weighting procedure
maximum values (boxes 0.5x0.5 deg)
A.Montani; The COSMO-LEPS system.
COSMO-LEPS_10 (Old) vs COSMO-LEPS_7 (New)
• Deterministic verification of T2M ensemble mean
Variable: 2-metre temperature. Period: from June to November 2009. Forecast ranges: fc+6h, fc+12h, …, fc+132h. Scores: root-mean-square error, bias.
• Probabilistic verification of 12-hour cumulated precipitation
Variable:12h cumulated precipitation (18-06, 06-18 UTC). Period: from June to November 2009. Forecast ranges: fc 6-18h, fc 18-30h, …, fc 114-126h. Scores: ROC area, BSS, RPSS, Outliers. Thresholds: 1, 5, 10, 15, 25, 50 mm/12h.
Observations: SYNOP reports over either MAP D-PHASE region (450 reports/day) or the FULL-DOMAIN (1400 reports/day).
Method: nearest grid point; no-weighted fcst.
A.Montani; The COSMO-LEPS system.
Bias and rmse of T2M Ensemble Mean
Consider bias and rmse for 3 months (24/5 24/8/2009) over MAPDOM (∼ 450 synop). T2m forecasts are corrected with height.
Bias closer to zero and lower rmse for the 7-km suite. Improvement is not “massive”, but detectable for all forecast ranges, especially for day-time
verification.Similar results over MAPDOM and over FULLDOM (not shown). The signal is stable (same scores also for 6-month verification).
---- OLD rmse (10 km)---- NEW rmse (7 km)—— OLD bias (10 km)—— NEW bias (7 km)
A.Montani; The COSMO-LEPS system.
COSMO-LEPS_10 (Old) vs COSMO-LEPS_7 (New)
• Deterministic verification of T2M ensemble mean
Variable: 2-metre temperature. Period: from June to November 2009. Forecast ranges: fc+6h, fc+12h, …, fc+132h. Scores: root-mean-square error, bias.
• Probabilistic verification of 12-hour cumulated precipitation
Variable:12h cumulated precipitation (18-06, 06-18 UTC). Period: from June to November 2009. Forecast ranges: fc 6-18h, fc 18-30h, …, fc 114-126h. Scores: ROC area, BSS, RPSS, Outliers. Thresholds: 1, 5, 10, 15, 25, 50 mm/12h.
Observations: SYNOP reports over either MAP D-PHASE region (450 reports/day) or the FULL-DOMAIN (1400 reports/day).
Method: nearest grid point; no-weighted fcst.
A.Montani; The COSMO-LEPS system.
opecleps vs Calibcleps
fc 42-66h; 1mm/24h
fc 42-66h; 10mm/24h