Download - Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory
![Page 1: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/1.jpg)
Feature-based(object-based)
Verification
Nathan M. HitchensNational Severe Storms Laboratory
![Page 2: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/2.jpg)
Introduction
• Feature-based verification approaches identify “objects” within forecast and observed fields– Attributes related to the objects from each field
are compared• e.g. size, location, intensity, orientation angle, etc.
– Precipitation most common variable– Summary of approaches• Gilleland et al. 2009 and Gilleland et al. 2010
![Page 3: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/3.jpg)
Example
1-hr precipitation (Stage II) Precipitation Objects
![Page 4: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/4.jpg)
Approaches
• Contiguous Rain Areas (CRAs)– “The area of contiguous
observed and/or forecastrainfall enclosed within aspecified isohyet”
– CRAs are the union of forecastand observed rain entities Ebert and McBride 2000
![Page 5: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/5.jpg)
Approaches
– Verification Statistics• Mean horizontal displacement of the forecast• Error in forecast and observed rain area• Error in mean and maximum rain rates• Error in rain volume• Pattern correlation of the corrected forecast
![Page 6: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/6.jpg)
Approaches
• Baldwin et al. 2005– Features-based technique to classify rainfall
systems• Non-convective subclass (stratiform)• Convective subclasses (linear and cellular)
– First identify objects similar to Ebert and McBride 2000
![Page 7: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/7.jpg)
Approaches
– Use manual expert classification of system type on “training” dataset
– Apply cluster analysisto training dataset• Gamma-scale parameter
and object eccentricityfound to have mostdetermining power
Baldwin et al. 2005
![Page 8: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/8.jpg)
Approaches
• Method for Object-based Diagnostic Analysis (MODE)– Smoothing of fields
to filter out small-scale variations
Davis et al. 2006
![Page 9: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/9.jpg)
Approaches
– Smoothed fields are thresholded to allow object boundaries to be detected
– Identified objects may also be “associated” into simple shapes for better evaluation of some attributes (aspect ratio, angle, etc)
Davis et al. 2006
![Page 10: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/10.jpg)
Approaches
– Observed and forecasted objects can be “matched” based on the distance between two objects (relative to their size)
– Object attributes are compared (either with or without matching)
![Page 11: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/11.jpg)
My Research
• Used Baldwin’s approach to identify objects– 6.0 mm threshold applied to 1-hr Stage II
precipitation
• Identified threshold for “extreme” as 99th percentile value of maximum precip in objects
• Used WRF to simulate selected events
![Page 12: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/12.jpg)
28 August 1998ST2
NARR
60-km
90-km
120-km
150-km
180-km
R1
![Page 13: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/13.jpg)
Methods
• BOOIA applied to ST2 product and precipitation from each simulation– Simulated objects compared to observed using Euclidean
distance approach– Object dissimilarity score formula:
• where s is areal size, me is mean precipitation value, ma is maximum precipitation value, x is the x-direction coordinate, y is the y-coordinate value, and the subscripts O and F represent observed and forecast objects
• Coefficients A through E are for weighting purposes
![Page 14: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/14.jpg)
Methods
– Each attribute is scaled based on the formula:
• where z is the scaled attribute, z0 is the non-scaled attribute, z10 is the attribute’s 10th percentile value, and z90 is the attribute’s 90th percentile value
![Page 15: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/15.jpg)
28 August 1998
• BOOIA attributes for observed and forecast objects
![Page 16: Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory](https://reader035.vdocuments.net/reader035/viewer/2022062315/5697bfd81a28abf838caece4/html5/thumbnails/16.jpg)
Questions?