![Page 1: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/1.jpg)
1
Statistical Post Processing
Hong Guan Bo Cui and Yuejian Zhu
EMC/NCEP/NWS/NOAA
Presents for NWP Forecast Training ClassMarch 31, 2015, Fuzhou, Fujian, China
![Page 2: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/2.jpg)
2
1. Background
![Page 3: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/3.jpg)
North American Ensemble Forecast System (NAEFS)
International project to produce operational multi-center ensemble products
Bias correction and combines global ensemble forecasts from Canada &
USA
Generates products for:Weather forecasters
Specialized usersEnd users
Operational outlet for THORPEX research using TIGGE archive
![Page 4: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/4.jpg)
StatementThe North American Ensemble Forecast System (NAEFS) combines state of the art
weather forecast tools, called ensemble forecasts, developed at the US National Weather Service (NWS) and the Meteorological Service of Canada (MSC). When combined, these tools (a) provide weather forecast guidance for the 1-14 day period that is of higher quality than the currently available operational guidance based on either of the two sets of tools separately; and (b) make a set of forecasts that are seamless across the national boundaries over North America, between Mexico and the US, and between the US and Canada. As a first step in the development of the NAEFS system, the two ensemble generating centers, the National Centers for Environmental Prediction (NCEP) of NWS and the Canadian Meteorological Center (CMC) of MSC started exchanging their ensemble forecast data on the operational basis in September 2004. First NAEFS probabilistic products have been implemented at NCEP in February 2006. The enhanced weather forecast products are generated based on the joint ensemble which has been undergone a statistical post-processing to reduce their systematic errors.
![Page 5: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/5.jpg)
Summary of 6th NAEFS workshop1-3 May, 2012 Monterey, CA
6th NAEFS workshop was held in Monterey, CA during 1-3 May 2012. There were about 50 scientists to attend this workshop whose are from Meteorological Service of Canada, Mexico Meteorological Service, UKMet, NAVY, AFWA and NOAA.
Following topics have been presented and discussed during workshop:• Review the current status of the contribution of each
NWP center to NAEFS• For each NWP center, present plans for future model and
product updates, for both the base models and ensemble system (including regional ensembles)
• Decide on coordination of plans for the overall future NAEFS ensemble and products (added variables, data transfer for increased resolution grids, FNMOC ensemble added to NAEFS, especially for mesoscale ensemble-NAEFS-LAM)
• Learn about current operational uses of ensemble forecast guidance, including military and civilian applications.
![Page 6: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/6.jpg)
7th NAEFS Workshop in Montreal, Canada
• Time: 17-19 June 2014• Locations:
– 17-18 June – Biosphere, Montreal, Canada– 19 June – CMC, Dorval, Canada
• Co-chairs: Andre Methot and Yuejian Zhu• Topics (or sessions)
– Status and plan of Global ensemble forecast systems;– Operational data management and distribution; – Ensemble verification and validation metrics; – Reforecast, bias correction and post process; – Regional ensemble and data exchange; – Wave ensembles; – Integration of ensemble in forecasts: user feedback and recommendation; – Products – hazard weather, high impact weather and diagnostic variables; – Open discussion of the NAEFS research, development, implementation and operation
plan
![Page 7: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/7.jpg)
7
NCEP CMC NAEFSModel GFS GEM NCEP+CMC
Initial uncertainty ETR EnKF ETR + EnKF
Model uncertainty/Stochasti
c
Yes (Stochastic Pert) Yes (multi-physicsand stochastic)
Yes
Tropical storm Relocation None
Daily frequency 00,06,12 and 18UTC 00 and 12UTC 00 and 12UTC
Resolution T254L42 (d0-d8)~55kmT190L42 (d8-16)~70km
About 50kmL72
1*1 degree
Control Yes Yes Yes (2)
Ensemble members 20 for each cycle 20 for each cycle 40 for each cycle
Forecast length 16 days (384 hours) 16 days (384 hours) 16 days
Post-process Bias correction(same bias for all
members)
Bias correction for each member
Yes
Last implementation February 14th 2012 November 18th 2014
NAEFS Current StatusUpdated: November 18th 2014
![Page 8: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/8.jpg)
Milestones• Implementations
– First NAEFS implementation – bias correction Version 1.00 - May 30 2006– NAEFS follow up implementation – CONUS downscaling Version 2.00 - December 4 2007– Alaska implementation – Alaska downscaling Version 3.00 - December 7 2010– Implementation for CONUS/Alaska expansion Version 4.00 - April 8 2014 – Implementation 2.5km/3km NDGD products for CONUS/Alaska Version 5.00 – August 2015
• Applications:– NCEP/GEFS and NAEFS – at NWS– CMC/GEFS and NAEFS – at MSC– FNMOC/GEFS – at NAVY– NCEP/SREF – at NWS
• Publications (or references):– Cui, B., Z. Toth, Y. Zhu, and D. Hou, D. Unger, and S. Beauregard, 2004: “ The
Trade-off in Bias Correction between Using the Latest Analysis/Modeling System with a Short, versus an Older System with a Long Archive” The First THORPEX International Science Symposium. December 6-10, 2004, Montréal, Canada, World Meteorological Organization, P281-284.
– Zhu, Y., and B. Cui, 2006: “GFS bias correction” [Document is available online]– Zhu, Y., B. Cui, and Z. Toth, 2007: “December 2007 upgrade of the NCEP Global Ensemble Forecast System (NAEFS)” [Document is
available online]– Cui, B., Z. Toth, Y. Zhu and D. Hou, 2012: "Bias Correction For Global Ensemble Forecast" Weather and Forecasting, Vol. 27 396-410 – Cui, B., Y. Zhu , Z. Toth and D. Hou, 2013: "Development of Statistical Post-processor for NAEFS"
Weather and Forecasting (In process)– Zhu, Y., and B. Cui, 2007: “December 2007 upgrade of the NCEP Global Ensemble Forecast System (NAEFS)” [Document is available
online]– Zhu, Y, and Y. Luo, 2014: “Precipitation Calibration Based on Frequency Matching Method (FMM)”. Weather and Forecasting (doi:
http://dx.doi.org/10.1175/WAF-D-13-00049.1)– Glahn, B., 2013: “A Comparison of Two Methods of Bias Correcting MOS Temperature and Dewpoint Forecasts” MDL office note, 13-
1– Guan, H, B. Cui and Y. Zhu, 2014: “Improvement of Statistical Post-processing Using GEFS Reforecast Information”, Weather and
Forecasting (in process)
![Page 9: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/9.jpg)
Bias correction: • Bias corrected NCEP/CMC GEFS and NCEP/GFS forecast (up to 180 hrs)• Combine bias corrected NCEP/GFS and NCEP/GEFS ensemble forecasts• Dual resolution ensemble approach for short lead time• NCEP/GFS has higher weights at short lead time
NAEFS products (global) and downstream applications• Combine NCEP/GEFS (20m) and CMC/GEFS (20m) • Produce Ensemble mean, spread, mode, 10% 50%(median) and 90% probability forecast at 1*1
degree resolution• Climate anomaly (percentile) forecasts• Wave ensemble forecast system• Hydrological ensemble forecast system
Statistical downscaling • Use RTMA as reference - NDGD resolution (5km/6km), CONUS and Alaska• Generate mean, mode, 10%, 50%(median) and 90% probability forecasts
NAEFS Statistical Post-Processing System
9
![Page 10: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/10.jpg)
10
Variables pgrba_bc file Total 51
GHT 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 10
TMP 2m, 2mMax, 2mMin, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa
13
UGRD 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 11
VGRD 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 11
VVEL 850hPa 1
PRES Surface, PRMSL 2
FLUX (top) ULWRF (toa - OLR) 1
Td and RH 2m 2
Notes CMC and FNMOC do not apply last upgrade yet
NAEFS bias corrected variables
Last upgrade: April 8th 2014 - (bias correction)
10
![Page 11: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/11.jpg)
11
Variables Domains Resolutions Total 10/10
Surface Pressure CONUS/Alaska 5km/6km 1/1
2-m temperature CONUS/Alaska 5km/6km 1/1
10-m U component CONUS/Alaska 5km/6km 1/1
10-m V component CONUS/Alaska 5km/6km 1/1
2-m maximum T CONUS/Alaska 5km/6km 1/1
2-m minimum T CONUS/Alaska 5km/6km 1/1
10-m wind speed CONUS/Alaska 5km/6km 1/1
10-m wind direction CONUS/Alaska 5km/6km 1/1
2-m dew-point T CONUS/Alaska 5km/6km 1/1
2-m relative humidity CONUS/Alaska 5km/6km 1/1
NAEFS downscaling parameters and productsLast Upgrade: April 8 2014 (NDGD resolution)
All downscaled products are generated from 1*1 degree bias corrected fcst. globally Products include ensemble mean, spread, 10%, 50%, 90% and mode
![Page 12: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/12.jpg)
12
2. NAEFS bias correction
![Page 13: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/13.jpg)
NAEFS bias correction
13
eaf
eaeaafN
afNt
t
1
)(1
For any given forecast f , it could express as :
Where a is truth (or replaced as best analysis), e is systematic error and ℇ is random error
Therefore, systematic error (or accumulated bias) could be a time average difference of forecast and truth:
In fact, an accumulated bias is changed from time to time which means e is not exactly systematic error, and ℇ is not exactly random error based on the time window for average.
![Page 14: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/14.jpg)
NAEFS Bias Correction
)()()( 0,,, tatftb jijiji
2). Decaying Average (or Kalman Filter method): Average bias will be updated by considering prior period bias and current bias by using decaying average (or Kalman Filter method ) with weight coefficient (w).
)()1()1()( ,,, tbwtBwtB jijiji
1). Bias Estimation: We assume a bias (b) for each lead-time (t) (6-hour interval up to 384 hours), each grid point (i, j) is defined as the different of best analysis (a) and forecast (f) at the same valid time (t0) which is up on latest available analysis.
![Page 15: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/15.jpg)
NAEFS Bias Correction 3). Decaying Weight: Through many experiments for different weights (w = 0.01, 0.02, 0.05, 0.1 and etc…), and different parameters, and different lead times, overall, w equals to 0.02 has been used for GEFS bias correction which is mainly using past 50-60 days information (see figure).
![Page 16: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/16.jpg)
NAEFS Bias Correction
4). Bias corrected forecast: The new (or bias corrected) forecast (F) will be generated by applying decaying average bias (B) to current raw forecast (f) for each lead time, at each grid point, and each parameter.
)()()( ,,, tBtftF jijiji
Simple Accumulated Bias
Assumption: Forecast and analysis (or observation) is fully correlated
![Page 17: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/17.jpg)
NAEFS Bias Correction 5). Performance: The performance is estimated by applying NAEFS bias correction method. The bias is calculated at each grid point for raw forecast (f) and bias corrected forecast (F), then using decaying average method (w=0.02) to get current average bias, taking absolute bias for each grid point, each lead time to generate domain average absolute error (bias) which smaller value is better (see figure: example for Northern Hemisphere 2 meter temperature, decaying average (w=0.2) about 2 months period ended by April 27, 2007).
![Page 18: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/18.jpg)
18
500hPa height: 120 hours forecast (ini: 2006043000)Shaded: left – raw bias right – bias after correction
Forecast becomes worse after bias
correction
![Page 19: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/19.jpg)
19
2 meter temperature: 120 hours forecast (ini: 2006043000)Shaded: left – raw bias right – bias after correction
Positive biasNegative bias
![Page 20: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/20.jpg)
Comparison of raw and bias corrected T2m for Summer and Fall 2011
RMS and Spread (NH – Fall)RMS and Spread
(NH – Summer)
ME and ABSE (NH – Fall)
ME and ABSE (NH – Summer)
Get worse from bias correction
Raw forecast is bias free
No carry on bias from summer
![Page 21: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/21.jpg)
NAEFS Bias Correction
• Several questions left behind– The correlation of prior joint sample
• Assume samples are fully correlated
– The weight to calculate decaying average• Optimum weights are functions of geographic and forecast
lead times (should be)• Currently, w (weight) is fixed (w=0.02)
– Systematic error for seasonal• Current method is lagged for seasonal information
– 2nd moment adjustment for current method• Slightly adjust for CMC’s ensembles• N/A for single model ensemble
![Page 22: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/22.jpg)
22
3. Using ensemble reforecast
![Page 23: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/23.jpg)
GEFS Reforecast
Configurations (Hamill et al, 2013)
• Model version– GFS v9.01 – last implement – May 2011– GEFS v9.0 – last implement – Feb. 2012
• Resolutions– Horizontal – T254 (0-192hrs– 55km) T190 (192-384hrs – 70km) – Vertical – L42 hybrid levels
• Initial conditions– CFS reanalysis– ETR for initial perturbations
• Memberships – 00UTC - 10 perturbations and 1 control
• Output frequency and resolutions– Every 6-hrs, out to 16 days– Most variables with 1*1 degree
• Data is available – 1985 - current
23
Using Reforecast Data
• Bias over 24 years (24X1=24) 25 years (25x1=25)• Bias over 25 years and 31day window (25x31)• Bias over recent 2, 5, 10, and 25 years within a window of 31day (2x31, 5x31, 10x31, 25x31)• Bias over 25 years with a sample interval of 7days within a window of 31days and 61days (~25x4 and ~25x8)
.
.
.
1985 1986
20102009
dayday-15 day+15
![Page 24: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/24.jpg)
Winter 2010
Spring 2010
Using 25-year reforecast bias (1985-2009) to calibrate 2010 forecast
24
Fall 2010
Summer 2010Solid line – RMSEDash line - Spread
![Page 25: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/25.jpg)
2% decaying is best for all lead time
Winter 2009
Spring 2009
Summer 2009 Fall 2009
Decaying averages are not good except for day 1-2
Decaying average is equal good as reforecast, except for week-2
forecast
Decaying averages are not good except for day 1-3
Using 24-year reforecast bias (1985-2008) to calibrate 2009 forecast
25
Another year
![Page 26: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/26.jpg)
26
Reforecast bias (2-m temperature)
Perfect bias
warm bias
cold biascold bias
![Page 27: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/27.jpg)
Long training period (10 or 25 years) is necessary to help avoid a large impact to bias correction from a extreme year case and keep a broader diversity of weather scenario!!
Skill for 25y31d’s running mean is the best. 25y31d’s thinning mean (every 7 days) is very similar to 25y31d’s running mean. 25y31d’s thinning mean can be a candidate to reduce computational expense and keep a broader diversity of weather scenario!!!
T2m calibration for different reforecast sample sizes
sensitivity on the number of training years (2, 5, 10, and 25 years)
sensitivity on the interval of training sample (1 day and 7 days)
27
![Page 28: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/28.jpg)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
r2, NH, 2010
Winter
Spring
Summer
Fall
Day
Bias corrected forecast: The new (or bias corrected) forecast (F) will be generated by applying decaying average bias (B) and reforecast bias (b) to current raw forecast (f) for each lead time, at each grid point, and each parameter.
r could be estimated by linear regression from joint samples, the joint sample mean could be generated from decaying average (Kalman Filter average) for easy forward.
Using reforecast to improve current bias corrected product
Additional term (bias from reforecast) is added if r (correlation coefficient) is not equal one. This adjustment is expected to benefit for longer lead time forecast
mjijiji
mji
mji DRBrbrfF
jiji )1()1( ,,
2,
2,, ,,
![Page 29: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/29.jpg)
Using reforecast to improve current bias corrected product (24-hr forecast, 2010 )
Spring Summer
FallWinter
Perfect bias
![Page 30: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/30.jpg)
Using reforecast to improve current bias corrected product (240-hr forecast, 2010 )
Perfect biasWinter Fall
SummerSpring
![Page 31: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/31.jpg)
500hPa height
Winter 2010
Spring 2010
Using 25-year reforecast bias (1985-2009) to calibrate 2010 forecast
Very difficult to improve the skills
31
![Page 32: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/32.jpg)
32
4. 2nd moment adjustment
![Page 33: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/33.jpg)
33
Make Up This Deficient ?Improving surface perturbations
Or Post processing
Bias correction does not change the spreads
![Page 34: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/34.jpg)
2nd moment adjustment
N
t
tatfN
E1
2))()((1
N
t
M
m
m tftfMN
S1 1
2))()((1
11
E
SR ))1()1(( tftfD mm R=1 if E=0
Ensemble skill
Ensemble spread
mji
mji
mji DRFF )1(* ,,,
1st moment adjusted forecast 2nd moment adj.
34
Estimated by decaying averaging
![Page 35: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/35.jpg)
Almost Perfect!!!
Winter Spring
FallSummer
35
After 2nd momentadjustment
![Page 36: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/36.jpg)
Surface Temperature (T2m) for Winter (Dec. 2009 – Feb. 2010), 120-hr Forecast
25% spread increased
7% RMSE reduced
SPREAD
RMSE
SPREAD/RMSE
36
![Page 37: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/37.jpg)
0
0.2
0.4
0.6
0.8
1120-hr forecast NA
Spre
ad R
MSE
ratio
SPP2
Winter Spring Summer Autumn
0
0.2
0.4
0.6
0.8
1240-hr forecast NA
Spre
ad R
MSE
ratio
Winter Spring Summer Autumn
improving under-dispersion for all seasons with a maximum benefit in summer.
Improving Under-dispersion for North America (Dec. 2009 – Nov. 2010)
37
SPP2
RAW
RAW
![Page 38: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/38.jpg)
38
5. Multi-model ensemble
![Page 39: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/39.jpg)
39
ts
jtsk
jk z
nw
,,,ˆ
1
Bayesian Model Average
Weights and standard deviations for each model (k - ensemble member) at step j
Finally, the BMA predictive variance is
2
1 1 1
2,,,,,,,,1, )
~~()
~,...,
~|( k
K
k
K
i
K
kktsiitskktsKtsts wfwfwffyVar
ts
jtsk
tstskts
jtsk
jk
z
fyz
,,,
,
2,,,,,
2
ˆ
)~
(ˆ
Between-forecast variance Within-forecast variance
Sum of (s,t) represents the numbers of obs.
It is good for perfect bias corrected forecast,Or bias-free ensemble forecast, but we do not
![Page 40: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/40.jpg)
40Courtesy of Dr. Veenhuis
Model 1
Model 2
Model 3
![Page 41: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/41.jpg)
41
Bias freeEns forecasts
Observation orBest analysis
Variance Weights
BMA, Decaying process, and adjustment
New weights
New variance
Prior weights
Prior variance
Err and sprd
New Err and sprd
Prior Err and sprd
Flow Chart of Recursive Bayesian Model Process (RBMP)
2nd moment adjustment Adjusted PDF
(We thanks to Dr. Veenhuis for allowing us to adopt his BMA codes).
![Page 42: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/42.jpg)
42
T2m for summer and Fall 2014
![Page 43: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/43.jpg)
43
The results demonstrate:1. BMA could improve 3 ensemble’s mean, but
spread could be over if original spread is larger2. RBMP could keep similar BMA average future,
but 2nd moment will be adjusted internally3. All important time average quantities are
decaying average (or recursive – save storage)
NUOPCIBC – simple combine three bias corrected ensembles
DCBMA – decaying based Bayesian Model Average
RBMP – Recursive Bayesian Model Process (built in decaying average and internal 2nd-moment adjustment)
Solid line – RMS errorDash line - Spread Over-dispersion
![Page 44: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/44.jpg)
44
Summer 2013
ROC are better for all leads
![Page 45: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/45.jpg)
45
The results demonstrate:1. BMA could improve 3 ensemble’s mean, but
spread could be over if original spread is larger2. RBMP could keep similar BMA average future,
but 2nd moment will be adjusted internally3. All important time average quantities are
decaying average (or recursive – save storage)
NUOPCIBC – simple combine three bias corrected ensembles
DCBMA – decaying based Bayesian Model Average
RBMP – Recursive Bayesian Model Process (built in decaying average and internal 2nd-moment adjustment)
Solid line – RMS errorDash line - Spread
Over-dispersion
![Page 46: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/46.jpg)
46
Fall 2013
Don’t understand this over-dispersion
Slightly degradation
ROC are better for all leads
![Page 47: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/47.jpg)
47
U10m for summer and Fall 2014
Assume V10m has the similar statistics as U10m
The difference of error/skills is very smaller, it is very hard to identify the improvement.
Need to consider to have additional plots for difference of NUOPC and DCBMA and NUOPC and RBMA
![Page 48: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/48.jpg)
48
RBMP is perfect for 1st moment and 2nd moment adjustment, especially for
2nd moment.
![Page 49: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/49.jpg)
49
Summer 2013
Southern hemisphere
Tropical
NH CRPS NH ROC
![Page 50: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/50.jpg)
50
The same conclusion as summerPerfect for 2nd moment adjustment
![Page 51: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/51.jpg)
51
NH CRPS NH ROC
Southern hemisphere
Tropical
![Page 52: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/52.jpg)
52
6. Statistical downscaling
![Page 53: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/53.jpg)
Statistical downscaling for NAEFS forecast
• Proxy for truth– RTMA at 5km resolution– Variables (surface pressure, 2-m temperature, and 10-meter
wind)
• Downscaling vector– Interpolate GDAS analysis to 5km resolution– Compare difference between interpolated GDAS and RTMA– Apply decaying weight to accumulate this difference –
downscaling vector
• Downscaled forecast– Interpolate bias corrected 1*1 degree NAEFS to 5km resolution – Add the downscaling vector to interpolated NAEFS forecast
• Application– Ensemble mean, mode, 10%, 50%(median) and 90% forecasts
53
![Page 54: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/54.jpg)
![Page 55: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/55.jpg)
12hr 2m temperature forecast Mean Absolute Error (MAE)
w.r.t RTMA for CONUSaverage for September 2007
GEFS raw forecast
NAEFS forecast
GEFS bias-corr. & down scaling fcst.
![Page 56: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/56.jpg)
56
NCEP/GEFS raw forecast
NAEFS final products
4+ days gain from NAEFS
From Bias correction (NCEP, CMC) Dual-resolution (NCEP
only) Combination of NCEP and
CMC Down-scaling (NCEP, CMC)
![Page 57: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/57.jpg)
57
From: Bias correction (NCEP, CMC) Dual-resolution (NCEP only) Combination of NCEP and
CMC Down-scaling (NCEP, CMC)
NAEFS final products
NCEP/GEFS raw forecast
8+ days gain
![Page 58: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/58.jpg)
GMOS forecast
NAEFS final productsFrom :Bias correction (NCEP, CMC)Dual-resolution (NCEP only)Down-scaling (NCEP, CMC)Combination of NCEP and CMC
From Valery Dagostaro (MDL)
CONUS 2m Temperature
For September 2007
Verify against RTMA
Verify against observation
BACK
![Page 59: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/59.jpg)
59
6. Statistical downscaling
![Page 60: Statistical Post Processing Hong Guan Bo Cui and Yuejian Zhu EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 31, 2015, Fuzhou, Fujian,](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649e255503460f94b1497e/html5/thumbnails/60.jpg)
60
Bias correction for each
ensemble member
+
High resolution deterministic
forecast
Mixed Multi- Model
Ensembles(MMME)
Probabilistic products at 1*1 (and/or) .5*.5
degree globally
Down-scaling (based on RTMA)
Probabilistic products at NSGD
resolution(e.g. 2.5km – CONUS)
NCEP
Others
CMC
Reforecast
RBMPFor
blenderVaried
decaying weights
Auto-adjustment
of 2nd moment
Smartinitialization
Future NAEFS Statistical Post-Processing System