Evolving from WAS to IS:

Societal Impacts in NWS Service Assessments

Evolving from WAS to IS:

Societal Impacts in NWS Service Assessments

Kevin Barjenbruch, NWS, Salt Lake City, UTDaniel Nietfeld, NWS, Omaha, NE

Julie Demuth, NCAR Societal Impacts Program

Kevin Barjenbruch, NWS, Salt Lake City, UTDaniel Nietfeld, NWS, Omaha, NE

Julie Demuth, NCAR Societal Impacts Program

2008 Summer WAS*IS WorkshopAugust 9, 2008

2008 Summer WAS*IS WorkshopAugust 9, 2008

February 5-6, 2008, tornado outbreak

• 82 tornadoes, 5 rated a 4 on Enhanced Fujita Tornado Scale

• 57 fatalities; most since May 31, 1985, and 13th of all time

• 350+ injuries

• Over $400M in property damage

Dubbed Super Tuesday Tornado Outbreak

Overarching Goal/Activation:

• Conducted to evaluate NWS performance during significant (high-impact) events– Major economic impacts– Multiple fatalities or numerous serious injuries– Unusually high public or media interest

• Usually convened just once or twice a year

NWS Service Assessments

Team Composition:• Experts from both inside and outside the

National Weather ServiceProcess:

• Study the event itself and as well as actions before, during, and immediately after

• Work with designated officials at National Weather Service Headquarters to author report– Recommend changes in NWS procedures, products,

and services to improve future performance

• Briefing provided to NWS Corporate Board• Service Assessment posted for public

consumption

NWS Service Assessments

• Inward Focus on NWS Procedures, actions, and equipment through– Internal review of

operations in: National Centers, Weather Forecast Offices, Center Weather Service Units, and River Forecast Centers)

– External assessment with various customers of the information – Trending toward IS!

• Emphasis on quantitative assessment– Fatalities and injuries– Damage– Verification, etc.

NWS Service AssessmentsService assessments moving from WAS…

• 2008 assessments have included a Societal Impacts Analysis Component– Super Tuesday Tornado

Outbreak– Mother’s Day Weekend

Tornadoes in Oklahoma and Missouri, May 10, 2008

– Midwest Floods of June 2008

• Increased emphasis on qualitative assessment– Actions taken– Changes in information

delivery, etc.

NWS Service AssessmentsTo IS…

• Utilize lessons learned, best practices, and analysis of customers/partners of weather information to– Improve delivery of hazard information (format

and content) to our customers and partners – Improve clarity of hazard information– Focus research and training– Allocate resources

NWS Service AssessmentsDesired outcomes

Provide better and more understandable weather information so that people will

take action to protect life and property!

Societal Impacts in Super Tuesday

Societal Impacts in Super Tuesday

Motivation• This was a well-warned event

…with such good information...

• Why did so many people die?

• Why don’t people do what they’re “supposed” to do … to make the “right” decision?

We get frustrated when we put “good” weather information out there and people don’t make the

“right” decisions!

We get frustrated when we put “good” weather information out there and people don’t make the

“right” decisions!

Societal impacts scope• The task – To try to understand why so many

people died and the details of those fatalities– E.g., age, gender, whether warning was heard, warning

source, whether they heeded the warning, structure where they died, whether they sought shelter, whether safer shelter was available

Can learn so much by having people walk you through all this … by letting

them tell you their stories!

Can learn so much by having people walk you through all this … by letting

them tell you their stories!

• An opportunity – To gather empirical info about people’s actual warning response behaviors– what info people had, how they interpreted it (knowledge)– how people perceived the situation (perceptions)– what decisions people made (decision-making)

Methodology• Semi-structured interviews with the public

• Targeted, convenience, and snowball sampling

• 41 interviews in the 6 WFOs visited (assessment team broke into 3 sub-teams)– Kevin and I did 17 public interviews over 4 days in

the field, another day on the phone

Some of the questions• When did you first realize there was a threat of a

tornado in this area? – How did you learn about the threat? (Sources, environmental

cues) – What were you thinking after you received that information?

(Trust? Confusion? Uncertainty? Barriers to action?) – What did you do next? (Confirmation?)– Were there any unique circumstances about your day that

affected your experience during the tornado event? What?

• Have you ever been in a similar type of extreme-weather situation in the past? (Experience, false alarms)– Did anything from that experience influence what you did

during this most recent event?– Have you ever been warned about an extreme weather event

in the past that did not occur?

• Think back over the entire tornado event, from the time you learned there was a tornado threat through when the tornado actually occurred.– Is there any other information you would have liked to have

had?

A Few Key Results• Knowledge

– People get information from multiple sources• Most commonly via television• Also commonly from other people

– People get information multiple times– Misconceptions about sirens

• Perceptions– Integration of seasonality, weather salience,

situational awareness about the event– Personalization of the threat

• Seeking confirmation of the threat (e.g., Atkins woman, couple)

• Personal risk perception and optimism bias (e.g., Hardin Co. family, Arkansas family)

A Few Key Results• Decision-making and sheltering

– Decision-making is not a singular event; it happens numerous times throughout the warning process

– Vast majority of people who received warning information sought shelter in best location available to them

– Most people heeded the warning and sought shelter in the best available location, but…

– Most people did not have an underground shelter or safe room available to them• Fatalities

– Collected as much good data as we could– Nearly 2/3 of victims were in mobile homes

Reflections and Wisdom from a Veteran WAS*ISerReflections and Wisdom from a Veteran WAS*ISer

WAS*IS Folded into Super Tuesday

The value of WAS*IS

• Community to fall back on for support/knowledge

• Recognition that the NWS is not “all that”– Through interaction at workshop– Through partnership projects– Through the assessment

• Some exposure to quantitative and qualitative research

• Visibility=Opportunity

• Coping with sleep deprivation

WAS*IS Folded into Super Tuesday

The OMG now what!

• Surveys

• The art of interviewing– Opportunity to observe and learn from Julie

• Excel…data analysis

Thoughts and Inspirations from a New WAS*ISer

Thoughts and Inspirations from a New WAS*ISer

I was a ship without a sail…

• Hurricane Charley Service Assessment – 2004– EM Interviews– Media Interviews– Hours and hours of transcribing– Had a good “teacher” (Sociologist Betty Morrow)– NO knowledge of good wine

• Super Tuesday Service Assessment – 2008– Learned from a great “mentor” (Julie and I suppose

Kevin)– Was inflicted by passion for doing this work– Truly connected my tornado warnings with the impacts of

them– Became a layexpert in Italian Reds

Adrift on the open sea of societal impacts…

Storm-Based Warnings

• Beneficial ?

• Confusing ?

• Media interpretation ?

• Implications to NWS ?

My Job… • Where should I focus my energy in the next 20

years?

• How can I influence the forecasters in my office?

• The internal workings of the WFO should be driven by the external impacts of our actions

• How should my seminars be structured?

• When I collaborate with UNL and Creighton University, which research projects should I emphasize and pursue?

Little Sioux Boy Scout Camp• June 11, 2008 EF3 Tornado

• 4 children killed

• 12 minute lead time

• Sirens sounded

• Sought shelter

• We now have a forecaster who wants to do a study on this event, from a societal impacts perspective

• I can at least now call Julie and say “HELP!”

My Other Job… • Teach “Severe Storms” at UNL

– Shared my Service Assessment experience with students

– Wow – student interest!

• UNL Alumni Advisory Board

• Short Course?

• Curriculum?

June 3, 1980• 28 years later

• Community rebuilding

• Economic Impact

• Psychological Impact

• Sheltering

• New perspective

Julie’s Reflections and Lessons Learned

Julie’s Reflections and Lessons Learned

The essentials• Partnerships among social scientist, research

meteorologists, operational meteorologist, policy makers, practitioners, etc. – Kevin links operational meteorologists, users– Julie links users, social science research– Dan links research meteorologists, operational

meteorologists– The three of us are so much more effective together

• Interest and willingness to work together, to listen, learn, exchange ideas! To co-produce knowledge.

• Passion!

• 3 Huskers + ISTJ + ENFP + INFJ = LATE NIGHTS!

• But I function well (had some of my best thoughts even) at 6:30 a.m. … after going to bed at 5:00 a.m.!

• Even still, it’s important to know who else on the team needs Starbucks daily (or twice daily, or …)

• Speaking of Starbucks, there are a lot of cops in Little Rock … and they’ll pull you over for *almost* running a red light.

• Speaking of Kevin’s driving, have a happy place that you can go to in an instant…Opportunities like this can change

your life in ways you never dreamed possible…

Opportunities like this can change your life in ways you never

dreamed possible…