“probability forecast use” - initial findings - jan verkade (deltares, delft, the netherlands)...
TRANSCRIPT
“Probability forecast use”- initial findings -
Jan Verkade (Deltares, Delft, The Netherlands)
Edwin Welles (Deltares – USA, Silver Spring MD)
May 14, 2012
May 14, 2012
Brief introduction to presenters…
Edwin Welles• hydrologist• PhD on forecast verification (University of Arizona, 2005)• Executive Director, Deltares USA, Inc.
Jan Verkade• hydrologist; focus on real-time flood forecasting• PhD researcher: predictive hydrological uncertainty• Research interests: benefits, use and evaluation of
probability forecasts
May 14, 2012
Outline
• Probability forecast study – overview• Initial findings• Summary and conclusions
But first: thanks to…
May 14, 2012
Thank-you to
• Diane Cooper, Service Hydrologist @ Chanhassen WFO• Reggina Cabrera, Chief, Hydrologic Services Division @ NWS
Eastern Region• John Blood, Senior Planner @ State of Minnesota Homeland
Security Emergency Management Services• Allen Glass, Bruce Elder, Tom Miller and Heather Winkleglass @
City of St Paul Emergency Management / Dept of Public Works• Scott Jutila and Patrick Foley @ USACE, St Paul District• Chris Franks, Meteorologist @ Chanhassen WFO
May 14, 2012
Thank-you to
• Greg Kruse, Chief of Hydrologic Monitoring Unit @ MN Dept of Natural Resources
• Marc Deutschmann, VP @ Houston Engineering• Greg Spoden, Climatologist @ State of Minnesota Climatology
Office, University of Minnesota• Walter Potts @ ESRI• Bill McAuliffe, Jim Kern, Ray Grumney and David Braunger @ Star
Tribune • Jim Bodensteiner and Jeff Berrington @ Xcel Energy
May 14, 2012
Last, but certainly not least, many thanks to…
Steve BuanService Coordination Hydrologist @ NWS NCRFC
May 14, 2012
Probability forecast use study – purpose
• General move towards probabilistic forecasting, varying reasons• To capture perceived benefits, simply having the probability
forecast is not enough• Additional effort may be
required:
• visualisation
• communication
• decision-making
• verification
• “downstream” DSSs
• adaptation of businessprocesses / procedures
May 14, 2012
Probability forecast study – method
• Desk research
• Forecasting exercise with Dutch Waterboard
• Interviews with
• US NWS forecasters and forecast users
• Dutch Meuse River forecasters and forecast users
• Scottish Environment Protection Agency forecasters and forecast users
• Possible wrap up workshop with all participants
May 14, 2012
Probability forecast use study – purpose
forecast responsedecision
Findings
May 14, 2012
May 14, 2012
May 14, 2012
May 14, 2012
Preliminary findings: themes
• Visualisation and communication• Reflection of physical processes• Decision making• Training and education• Use of and rationale for probability forecasting
May 14, 2012
Why people use forecasts
• planning• planning• planning• awareness raising• decision making
May 14, 2012
Preliminary findings: visualisation
• Users may have issues with visualisation of probability forecasts• Main issue: probability forecasting adds a dimension to the
problem• In addition to:
• space (2 dimensions already)
• time
• a variable such as either flow or stage• A 5th dimension is added: probability
visualisation not trivial
forecast responsedecision
May 14, 2012
Preliminary findings: visualisation• Suggested approach for visualisation:• As we can visualise only 2 (graphs) or 3 (maps) dimensions, we
need to “fix” the others• Choices for fixing vary
by user and use
this “dimensionality issue” requires somewhat sophisticated tools to visualise probability forecasts(e.g. GIS, programming languages)
May 14, 2012
Preliminary findings: communication
• Ties in with visualisation• Additional issue: “flow” and “stage” may not be meaningful to many
end users• Variables such as “inundated area”, “number of properties
affected”, “reference floods” may be effective to communicate risks
may require post-processing
forecast responsedecisiondecisionsupport
May 14, 2012
Preliminary findings: physical processes
• Often heard: users compare forecasts with their knowledge of the physical system
• This is quite difficult when “abstract probabilities” come into play these cannot be confirmed with knowledge of hydrology
• some indication may be needed that the physics are okay
not an easy task; maybe solution is found inprovision of metadata
providing information on past performanceof probability forecasts will be helpful(for this: forecast verification is required)
May 14, 2012
Preliminary findings: decision-making
• Decisions are made based on a consequence, not on a hazard
• Must transform probabilities to make a binary decision
> Often times: intuition only
• Decision criteria can now include explicit expression of risk
• Risk = Probability x Consequence
• NWS concentrates on forecasting a possible hazard; decision support is needed to translate this into meaningful consequences
forecast responsedecisiondecisionsupport
May 14, 2012
Preliminary findings: training and education
• Information overload is a real issue
• Trying to understand the forecast is not conducive to good decision making
• This should then be done prior to a “crisis situation”• Links between “hydrological variables” and real issues not always clear
for non-experts• Everyone found the NWS webinars extremely useful• Need to explain how to access the information in the forecasts by fixing
a dimension.
May 14, 2012
Preliminary findings: why probabilities
• Rationale for moving towards probability forecasting:
• “More realistic forecasts” widely accepted> Deterministic forecasts are “over confident”
• “Risk based” decision making easier said than done> Support systems not generally in place
• Benefits may not reside with both forecaster and end user> Forecasters provide more complete information> Users must do more interpretation and post-processing
Some ideas…
May 14, 2012
Decision support: from outlooks to planning (1)
Exc.probs Max stage[-] [ft]
98% 1395% 1490% 1580% 1750% 2520% 3210% 345% 352% 381% 40
May 14, 2012
Decision support: from outlooks to planning (2)
Determine potential consequences from inundation
maps
Do this for all relevant stages
May 14, 2012
Decision support: from outlooks to planning (2)
May 14, 2012
Decision support: from outlooks to planning (2)
Max stage #houses affected[ft] [-]13 014 015 017 025 032 534 2535 5738 10040 250
May 14, 2012
Max stage #houses affected[ft] [-]13 014 015 017 025 032 534 2535 5738 10040 250
Exc.probs #houses affected[-] [-]
98% 095% 090% 080% 050% 020% 510% 255% 572% 1001% 250
Decision support: from outlooks to planning (3)Exc.probs Max stage
[-] [ft]98% 1395% 1490% 1580% 1750% 2520% 3210% 345% 352% 381% 40
May 14, 2012
Decision support: from outlooks to planning (4)#houses affected [-]
0
50
100
150
200
250
300
0%10%20%30%40%50%60%70%80%90%100%
Exceedence probability [-]
Hou
ses
affec
ted
[ft]
May 14, 2012http://wim.usgs.gov/FIMI/FloodInundationMapper.html – via Kris Lander @ NWS Central Region
Another idea…
May 14, 2012
36 Serious flood damage occurs at the University of Iowa campus.
34 Flood protection becomes necessary at the University of Iowa. Water affects several industrial businesses and warehouses along Commercial Drive.
32 Water floods county road bridge approaches along the river and affects streets and parking lots along Commercial Drive.
30 Flooding occurs in Coralville. Water inundates the Cedar Rapids and Iowa City Rail line near Coralville.
25.5 Water affects the north Wastewater Treatment Plant. Considerable flooding occurs at the University of Iowa.
25 Water floods Hancher Auditorium at the University of Iowa. Flooding problems occur elsewhere on the University of Iowa campus.
22 Urban flood damage occurs in Iowa City. Water enters homes along Edgewater Drive.
21 Flooding occurs in homes near Taft Speedway in Iowa City. Homes on east side of Quarry Road in Coralville require protection. Edgewater Drive becomes impassable.
20.5 Water affects the north Wastewater Treatment Plant.
19 Lowland flooding occurs in the Iowa City Park area.
Gather the impacts.
May 14, 2012
Assign probabilities to the stages.
21 ft 70%
25 ft 50%
30 ft 30%
34 ft 10%
May 14, 2012
36 Serious flood damage occurs at the University of Iowa campus.
34 Flood protection becomes necessary at the University of Iowa. Water affects several industrial businesses and warehouses along Commercial Drive.
32 Water floods county road bridge approaches along the river and affects streets and parking lots along Commercial Drive.
30 Flooding occurs in Coralville. Water inundates the Cedar Rapids and Iowa City Rail line near Coralville.
25.5 Water affects the north Wastewater Treatment Plant. Considerable flooding occurs at the University of Iowa.
25 Water floods Hancher Auditorium at the University of Iowa. Flooding problems occur elsewhere on the University of Iowa campus.
22 Urban flood damage occurs in Iowa City. Water enters homes along Edgewater Drive.
21 Flooding occurs in homes near Taft Speedway in Iowa City. Homes on east side of Quarry Road in Coralville require protection. Edgewater Drive becomes impassable.
20.5 Water affects the north Wastewater Treatment Plant.
19 Lowland flooding occurs in the Iowa City Park area.
70%
50%
30%
10%
Assign probabilities to the stages.
And some ideas about visualisation…
May 14, 2012
Recommendations: map-type visualisation
flood stage [ft](or: flood flow [volume/time])
(or: ….)
high/medium/lowexceedence prob
“Play” button fortime dimension
Fix any continuous variable;or define the event otherwise
May 14, 2012
May 14, 2012
Recommendations: graph-type visualisation
variable = f(time)
Fixed:•location: White Cart river @ Overlee•probability: median, x% quantile, …
Q.median @ Lobith
0
500
1000
1500
2000
2500
3000
3500
12/03/199000:00
14/03/199000:00
16/03/199000:00
18/03/199000:00
20/03/199000:00
22/03/199000:00
24/03/199000:00
May 14, 2012
Recommendations: graph-type visualisation
Exceedence probability = f(time)
Fixed:•location: White Cart river @ Overlee•variable / event: flood stage, h = 1.5m
May 14, 2012
Conclusions• Multi-dimensional nature of probability forecasts makes them hard
to understand, (as opposed to the probabilities being too complicated).
• Everyone needs to fix at least one dimension to make sense of the probability forecasts
• Impacts/ consequences
• Planning Scenarios
• Analagous Events
• Background information on the
forecast origins helps. So does verification.
• Probabililty forecasts are more informative! And useful!
May 14, 2012
Contact details
Edwin Welles
[email protected] , 301-642-2505
Jan Verkade
[email protected] , +31.88.335.8348
On the web:
http://www.deltares.nl/en
This presentation can be downloaded from:
http://publicwiki.deltares.nl/display/~verkade
Thank you for your attention