![Page 1: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/1.jpg)
© ECMWF October 12, 2015
Verification at ECMWF
Thomas Haiden and Martin JanousekEuropean Centre for Medium-RangeWeather Forecasts
Forecast Analysis
![Page 2: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/2.jpg)
October 29, 2014
2
Anomaly correlation of 500 hPa geopotential
![Page 3: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/3.jpg)
October 29, 2014
3
Anomaly correlation of 500 hPa geopotential
![Page 4: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/4.jpg)
October 29, 2014
4
Anomaly correlation of 500 hPa geopotential
![Page 5: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/5.jpg)
October 29, 2014
5
Anomaly correlation of 500 hPa geopotential
![Page 6: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/6.jpg)
October 29, 2014
6
Improvement relative to ERA-Interim
Upper-air
Surface
![Page 7: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/7.jpg)
October 29, 2014
7
2-m temperature error growth: DJF 2016-17
NH Extratropics, 12 UTC
RMSE
against SYNOP
![Page 8: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/8.jpg)
October 29, 2014
8
2-m temperature error growth: DJF 2016-17
NH Extratropics, 12 UTC
RMSE
against SYNOP
against analysisobs+repres error,unresolved processes
![Page 9: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/9.jpg)
October 29, 2014
9
2-m temperature error growth: DJF 2016-17
MSE
NH Extratropics, 12 UTC(2.8 K)2
against SYNOPagainst analysis
![Page 10: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/10.jpg)
October 29, 2014
10
2-m temperature error growth: DJF 2016-17
MSE
NH Extratropics, 12 UTC
against SYNOP
T850
against analysis
![Page 11: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/11.jpg)
October 29, 2014
11
Regional variations
RMSE
![Page 12: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/12.jpg)
October 29, 2014
12
Regional variations
Stations excluded where ∆z>150m
RMSE
![Page 13: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/13.jpg)
October 29, 2014
13
Regional variations
SDEV
![Page 14: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/14.jpg)
October 29, 2014
14
Europe
SDEV
Problem: strong surface inversions over snow
Problem: low stratus boundaries and persistence
![Page 15: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/15.jpg)
October 29, 2014
T2m forecast skill evolution (ENS) – Day 5
All errors
CRPS
Percentage of large errors
P(CRPS > 5 K)
Horizontal resolution upgrades
CRPS decreased by ~10%Frequency of large errors decreased by ~20%
![Page 16: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/16.jpg)
TIGGE scores (CRPSS) – DJF 2016 and 2017
CMCJMANCEPUKMOECMWF
Z500 T850
T2M PRECIP
![Page 17: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/17.jpg)
Precipitation forecast skill
HRES ENS
Overestimation of frequency of light precipitation still an issue, especially in summer (convection)
![Page 18: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/18.jpg)
10 m wind speed verification
Systematic error (bias) ‘Random’ error (standard deviation)
![Page 19: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/19.jpg)
Systematic error (bias) ‘Random’ error (standard deviation)
10 m wind speed verification
![Page 20: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/20.jpg)
High-impact weather
2m temperature 10m wind speed
24-h precipitation
Tropical cyclones Extreme forecast index (EFI)
![Page 21: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/21.jpg)
Verification on the ECMWF webpage
![Page 22: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/22.jpg)
Model upgrade 11 July 2017 (Cycle 43r3)
https://software.ecmwf.int/wiki/display/USS/IFS+cycle+43r3+scorecard
Verification against analysis
![Page 23: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/23.jpg)
Model upgrade 11 July 2017 (Cycle 43r3)
https://software.ecmwf.int/wiki/display/USS/IFS+cycle+43r3+scorecard
Verification against
observations
![Page 24: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/24.jpg)
Conclusions
• Medium-range NWP further improving – ECMWF leading the way
• Regional-scale systematic errors in surface parameters, need to be addressed
• Frequency of large errors in ENS 2-m temperature: additional headline score
• Promising developments for future cycles (e.g. ocean coupling, cloud physics)
![Page 25: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/25.jpg)
Random error,
normalized
Random error
10 m wind speed verification
![Page 26: Verification at ECMWF...•Regional-scale systematic errors in surface parameters, need to be addressed •Frequency of large errors in ENS 2-m temperature: additional headline score](https://reader035.vdocuments.net/reader035/viewer/2022081619/60f7f1c68915007231685916/html5/thumbnails/26.jpg)
Random error,
normalized
Mean wind speed (observed)
10 m wind speed verification