towards an object-oriented assessment of high resolution precipitation forecasts janice l. bytheway...
TRANSCRIPT
Towards an object-oriented assessment of high resolution precipitation forecasts
Janice L. Bytheway CIRA Council and Fellows Meeting
May 6, 2015
CIRA Council and Fellows Meeting, 6 May 2015
2
IntroductionNWP models are continuously undergoing
improvementModel verification studies determine the skill of
model performanceFuture improvements to NWP relies on knowing
not only how well the model performs, but what processes are the cause of a successful or failed forecast
Introduce an additional step to verification:Assessment – validation of the model with the intent
to determine which variables or model processes are likely related to the model’s performance.
CIRA Council and Fellows Meeting, 6 May 2015
3
GoalUse observations (Stage IV MPE) to verify
HRRR model precipitation forecasts with assimilated reflectivities
Use object-oriented validation and track features through time to evaluate model performance through forecast period
Relate validation to other observations or model variables to determine why model does/does not perform well.
CIRA Council and Fellows Meeting, 6 May 2015
4
Based on the MODE method described in Davis et al. [2006; 2009] Identify precipitating features of interest in model and observations
in Central US during 2013 warm season (May-Aug) Apply 15km smoothing to rain field and identify areas where hourly
accumulation exceeds a selected threshold (1 mm/hr) Feature is present in observations one hour prior to forecast
initialization Maximum observed hourly rainfall 1 hour prior to forecast
initialization exceeds 10 mm/hr (obs only) Area within a selected isohyet exceeds 5000 km2 (obs only)
Track features through 15 hours of forecast Validate those observed for at least 70% of the forecast run
(12+hours)Find forecast/observed feature pairs at forecast hour 1.Create a database of precipitating features and associated
properties.
Object Oriented Validation
CIRA Council and Fellows Meeting, 6 May 2015
5
HR
RR
S
tag
e I
V
CIRA Council and Fellows Meeting, 6 May 2015
6
Based on the MODE method described in Davis et al. [2006; 2009] Identify precipitating features of interest in model and observations
in Central US during 2013 warm season (May-Aug) Apply 15km smoothing to rain field and identify areas where hourly
accumulation exceeds a selected threshold (1 mm/hr) Feature is present in observations one hour prior to forecast
initialization Maximum observed hourly rainfall 1 hour prior to forecast
initialization exceeds 10 mm/hr (obs only) Area within a selected isohyet exceeds 5000 km2 (obs only)
Track features through 15 hours of forecast Validate those observed for at least 70% of the forecast run
(12+hours)Find forecast/observed feature pairs at forecast hour 1.Create a database of precipitating features and associated
properties.
Object Oriented Validation
CIRA Council and Fellows Meeting, 6 May 2015
7
HR
RR
S
tag
e I
V
CIRA Council and Fellows Meeting, 6 May 2015
8
Based on the MODE method described in Davis et al. [2006; 2009] Identify precipitating features of interest in model and observations
in Central US during 2013 warm season (May-Aug) Apply 15km smoothing to rain field and identify areas where hourly
accumulation exceeds a selected threshold (1 mm/hr) Feature is present in observations one hour prior to forecast
initialization Maximum observed hourly rainfall 1 hour prior to forecast
initialization exceeds 10 mm/hr (obs only) Area within a selected isohyet exceeds 5000 km2 (obs only)
Track features through 15 hours of forecast Validate those observed for at least 70% of the forecast run
(12+hours)Find forecast/observed feature pairs at forecast hour 1.Create a database of precipitating features and associated
properties.
Object Oriented Validation
CIRA Council and Fellows Meeting, 6 May 2015
9
Match model feature to observedDo any model features overlap the radar
feature?
yesno yes
Match
found
Select feature with maximum overlap
Do any model objects have centroids within effective radius of
observed centroid?
no
More than one?
yes
yes
No match exists
More than one?
no
Select feature with most similar
total rainfall
no
Match
found
CIRA Council and Fellows Meeting, 6 May 2015
10
Based on the MODE method described in Davis et al. [2006; 2009] Identify precipitating features of interest in model and observations
in Central US during 2013 warm season (May-Aug) Apply 15km smoothing to rain field and identify areas where hourly
accumulation exceeds a selected threshold (1 mm/hr) Feature is present in observations one hour prior to forecast
initialization Maximum observed hourly rainfall 1 hour prior to forecast
initialization exceeds 10 mm/hr (obs only) Area within a selected isohyet exceeds 5000 km2 (obs only)
Track features through 15 hours of forecast Validate those observed for at least 70% of the forecast run
(12+hours)Find forecast/observed feature pairs at forecast hour 1.Create a database of precipitating features and associated
properties.
Object Oriented Validation
CIRA Council and Fellows Meeting, 6 May 2015
11
Assigned feature numbersCoordinates of feature center of massFeature size (area)Feature mean, maximum, and total hourly
rainfallPDFs and CDFs of rain rateUse these statistics along with feature maps and
masks to calculateLocation offsetBiases“Standard” validation statistics (FAR, POD, RMSE)
Stored Attributes
CIRA Council and Fellows Meeting, 6 May 2015
12
Location Offset
W E
N S
Model Spin-up andLag CorrelationsModel takes a few hours to spin up to optimum validation results
Model concentrates rainfall into area similar to what was observed at assimilation time, leading to low biases in areal extent.
Model appears to over-concentrate assimilated latent heating, resulting in high biases in mean hourly rainfall and maximum hourly intensity.
Result is good representation of total system rainfall
0 hour lag1 hour lag2 hour lag
Composite Rainfall Feature
Forecast Hour 1
Forecast Hour 3
Area Bias -67% -22%
Mean Bias +61% +25%
Max Bias +303% +125%
Total Bias +8% +2%
CIRA Council and Fellows Meeting, 6 May 2015
15
Probability of PrecipitationGiven 1mm/hr observed
CIRA Council and Fellows Meeting, 6 May 2015
16
TPW and Cloud thickness in near-storm environment
CIRA Council and Fellows Meeting, 6 May 2015
17
Features based validation methods allow evaluation of model performance of specific precipitating objects through time.
Even with assimilated radar, 1-2 hours of spin-up before most accurate QPF.
HRRR placement relatively good, but eastward propagation may be too slow.
HRRR tendency to concentrate convective rainfall in intense cores
HRRR appears to require large amounts of moisture to produce moderate rainfall/deep convection.
Conclusions
CIRA Council and Fellows Meeting, 6 May 2015
18
Future Potential3D model output will allow for additional
evaluation of performance2014/2015 HRRR data will allow for comparison
with previous years to monitor improvementsUse of field projects (IFloodS, IPHEX) for point
validation and reference observations of cloud properties
Further exploration of results using satellite dataRapidly updated, high resolution geostationary3D reflectivity profiles from GPM
CIRA Council and Fellows Meeting, 6 May 2015
19
Backup
CIRA Council and Fellows Meeting, 6 May 2015
20
Examining the tails of the PDFs