communicating uncertainty with probability phrases
TRANSCRIPT
![Page 1: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/1.jpg)
Communicating Uncertainty with Probability Phrases
David V. Budescu Fordham University, NYC, USA
IPCC Communication MeetingOslo, Feb 9-10, 2016
I
![Page 2: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/2.jpg)
Translation Table for IPCC Authors
Phrase Likelihood Conveyed
Virtually certain > 99%
Very likely > 90%
Likely > 66%
About as likely as not 33% to 66%
Unlikely < 33%
Very unlikely < 10%
Exceptionally unlikely < 1%5
![Page 3: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/3.jpg)
Reservations
• The ranges of the terms are not mutually exclusive
• The ranges associated with the terms are arbitrary and are not related to people's “natural” understanding of the language
• People prefer to receive numerical information if available
• Large variability in interpretation of the terms
![Page 4: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/4.jpg)
The 2014 Intl Study
• Read 8 statements extracted from the IPCC executive summary containing 4 terms: very likely, likely, unlikely, and very unlikely (2 statements for each term)
• Provide Best estimate and Lower and Upper bounds for each term (as intended by authors)
• Presentation format: Translation or Dual (VN)
• 27 samples in 25 countries and 17 languages (~400 /country) • All materials were translated and back translated by native
speakers• Analyze n=10,239 in 25 samples (51% men; mean age = 41)
7
![Page 5: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/5.jpg)
Translation Screen shot
8
![Page 6: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/6.jpg)
Translation screen shot with guidelines
9
![Page 7: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/7.jpg)
Verbal-Numerical screen shot
10
![Page 8: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/8.jpg)
Estimates are Regressive, but the VN Presentation Makes Them More Extreme
12
T VN T VN T VN T VNVery Unlikely
<10%Unlikely
<33%Likely>66%
Very Likely>90%
0
20
40
60
80
100
Bes
t Est
imat
es
![Page 9: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/9.jpg)
Consistency with IPCC Guidelines Improves (especially for extremes)
13
Very Unlikely Unlikely Likely Very Likely0
10
20
30
40
50
16.13
36.62 37.64
18.95
31.76
43.1447.59
36.65
Overall TranslationOverall VN
% o
f Est
imat
es C
ompl
ying
with
G
uide
lines
![Page 10: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/10.jpg)
IndiaNetherland
UKBrazil
Hong KongItaly
SpainCanadaQuebecSweden
AustraliaOverall
GermanyJapan
TaiwanRussiaPolandFrance
SlovakiaUSA
ChileChina
RSATurkeyKoreaIsrael
21%24%25%25%25%26%27%27%27%27%27%27%27%28%28%28%28%28%29%30%31%31%31%32%33%35%
28%32%
43%36%
41%39%
37%46%
38%41%
37%40%
43%35%
43%39%40%
38%44%45%
43%40%
48%46%
38%54%
% Compliance Translation Series2
14
![Page 11: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/11.jpg)
The Benefits of Multiple Modalities
• The dual scale was superior in several senses: – Better differentiation and discrimination, – Higher agreement with the prescribed meaning, – More uniform interpretations across various groups– It works for people with different preferences
• It makes communicators more mindful of the terms’ intended meaning
• It can be used in more creative and flexible ways to accommodate, and signal, different levels of (im)precision in various cases
![Page 12: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/12.jpg)
The Evidence Based Approach• Identify the target audience• Identify a subset of terms that are reasonable
candidates for inclusion in the lexicon• Establish empirically how the target population
understands, interprets and uses these terms (using single estimates, ranges, MFs)
• Use statistical methods to Identify optimal cutoff points that archive the highest consensus in interpretation in the target population
![Page 13: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/13.jpg)
Evidence-Based Lexicons
Group Membership FunctionsDistribution of Estimates
![Page 14: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/14.jpg)
Application to the IPCC
Sample Lexicon Consistency (%)UK IPCC 27
Peak 44Membership 50
Australia IPCC 25Peak 41Membership 45
![Page 15: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/15.jpg)
Evidence-Based Lexicons
• There is a clear advantage to deriving evidence - based lexicons over “intuitive” ones
• Such lexicons are constructed to take full advantage of the common norms and meanings shared by a majority of participants in the target group
• As such they are less sensitive to idiosyncratic preferences of the committees that construct them and they are easier to implement and maintain
• There are many possible methods one can use in this context.
![Page 16: Communicating Uncertainty with Probability Phrases](https://reader035.vdocuments.net/reader035/viewer/2022081605/58ee37ad1a28ab8a788b4645/html5/thumbnails/16.jpg)
Evidence-Based Lexicons
• There is no single universal lexicon that serves all purposes
• It makes sense to choose the size and composition of the lexicon to – Fit the needs of the target population – Be sensitive to the precision and discrimination of
the information prevalent in the decision environment
– Be sensitive to the probabilities of interest