communicating risk
TRANSCRIPT
Communicating Uncertainty
Andy Hart
Food and Environment Research Agency, York
BSA Science Communication Conference, 1 May 2014
Why communicate uncertainty?
Red River Flood, Grand Forks USA, 1997•Levee height: 51 feet•River height prediction: 49 feet
•Uncertainty: plus or minus 9 feet
5149
±9
After: Nate Silver, The Signal and the Noise, 2012
Why communicate uncertainty?
Red River Flood, Grand Forks USA, 1997•Levee height: 51 feet•River height prediction: 49 feet
•Uncertainty: plus or minus 9 feet
•Actual flood height: 54 feet
5149
±9
After: Nate Silver, The Signal and the Noise, 2012
Why communicate uncertainty?
Red River Flood, Grand Forks USA, 1997
•Estimated damages: ca. $3.5 billion
5149
±9
Why communicate uncertainty?
‘Credibility requires trust…
…trust requires openness…
…openness requires recognition of uncertainty’
(Phillips Report on BSE, 2000)
What to communicate?
• What information is needed to communicate uncertainty?
Reasons for uncertainty
• Red River Flood height affected by:− Temperature (snow melt)− Rainfall− Bridges, sandbags etc. (retard flow)− Volume of water downstream− Extrapolation beyond previous data
Reasons are important, but not enough
5149
Confidence in predictions?
• 12th April 2013 – US Intelligence reports ‘moderate confidence’ that North Korea has nuclear weapons
• They say: ‘Moderate confidence generally means the information is interpreted in various ways, we have alternative views, or the information is credible and plausible but not corroborated sufficiently to warrant a higher level of confidence’
Qualitative expressions of confidence are not enough
5149
The range of plausible outcomes
• Tells you what might happen
• But not how likely that is
• Okay if all the outcomes are okay
• Not enough if bad outcomes are possible but costly to avoid or mitigate
Also need information on the relative likelihoods of different outcomes
5149
±9
The range of plausible outcomes
• Also need information on the relative likelihoods of different outcomes
TV forecasts for Hurricane Irene, 2011
299600
299700
299800
299900
300000
1870 1890 1910 1930 1950
Mea
sure
d sp
eed
of li
ght (
km/s
ec)
Confidence intervals
Include information on likelihood butonly for uncertainties that have been quantified
Historical measurements of the speed of light, ± measurement error (1 standard deviation)
After Henrion and Fischoff, 1986
Generally accepted
value (1986)
Confidence intervals
• Confidence intervals from data or modelling only express uncertainty quantified in the data or model
Need to:− say which uncertainties have been quantified− identify any additional uncertainties− indicate their potential impact on the range and
likelihood of alternative outcomes− acknowledge any deep uncertainties for which
this is not possible
Additional uncertainties
• Impact of additional uncertainties
e.g. Bank of England fan charts take account of additional uncertainties when forecasting inflation
% increase on prices a year
earlier
past future
Qualitative expressions of likelihood?
• The same word means different things to different people
• Words are not enough
5149
±9
Macleod & Pietravalle
Subjective probabilities
“Some of our intelligence officers thought that it was only a 40 or 30% chance that Bin Laden was in the compound. Others thought that it was as high as 80 or 90%. At the conclusion of a fairly lengthy discussion where everybody gave their assessments I said: this is basically 50-50.” Barack Obama speaking on ‘Bin Laden:
Shoot to Kill’, Channel 4, 7 September 2011
Words with quantitative meaning
• E.g. IPCC Likelihood scale:
• Verbal definitions just transfer the problem
5149
±9
Words with quantitative meaning
‘The UNEP report puts the average global cooling that could be achieved with black-carbon control measures at 0.3 watts per square metre…
... the range of possible effects runs from twice that much to nothing at all
...there is even an outside chance that the proposed action could lead to a little warming...
...though if that were the case the actions on ozone would still provide a net cooling for the actions as a whole…’
NASA GSFC
The Economist, 19 Feb. 2011
Deep uncertainty
• If it’s not possible to quantify likelihood even approximately then it is misleading to use undefined verbal expressions
• Indicates presence of deep uncertainties− describe them− explain the prediction is just one
possibility− if nothing can be said about the range
and/or likelihood of other outcomes then say so
5149
What is being done about it
• Who are the relevant authorities?
• What are they doing to decrease the chance of adverse outcomes?
• What are they doing to monitor the situation?
• What contingency plans do they have?
• What options can others consider?
5149
±9
Outcomes are always uncertain
express confidence in the chosen strategy
(if warranted)
but
acknowledge the outcome is still uncertain
5149
±9
Summary
Information needed for communicating uncertainty:
•Reasons for uncertainty
•Range of plausible outcomes
•Likelihood of adverse outcomes
− As quantitative as possible
•Nature of deep uncertainties, if present
•What is being done about it?
If this information isn’t provided, ask for it!
5149
±9