1

AMS Commission on the Weather and Climate

EnterpriseBoard on Enterprise Communication

Ad Hoc Committee on Uncertainty in Forecasts

(ACUF)

January 23, 2008

AMS Annual Meeting

Town Hall

2

Welcome and IntroductionGeorge Frederick, CWCE Commissioner

• Providing forecast uncertainty information to Nation– Challenge Enterprise should take on together

• CWCE and BEC– Provides forum to bring Enterprise partners together

• ACUF not starting from scratch– Good work going on in community

– 2006 NRC “Completing the Forecast” Report

3

Welcome and IntroductionMuch Interest in Committee

ACUF MembersTotal = 58

0%

5%10%

15%20%

25%30%

35%40%

45%

Private Sector Government AcademiaPe

rce

nta

ge

of

En

tire

C

om

mit

tee

4

Welcome and Introduction Town Hall Purpose

• For Committee to share their initial plans

• For you to share your thoughts and comments

• Committee will meet later today to digest what they’ve heard and kickoff their work

– Those interested welcome to attend

5

Agenda

• Remarks (J. Hayes, J. Myers, B. Ryan, P. Stephens)

• Motivation (E. Abrams)

• Committee overview (P. Hirschberg)– Workgroup 1: National needs, opportunities and benefits (B. Morrow)

– Workgroup 2: Enterprise Goals (A. Stewart)

– Workgroup 3: Needed Capabilities and Capacities (T. Hamill)

– Workgroup 4: Roles and Responsibilities (B. Philips)

– Workgroup 5: Roadmap (S. Tracton)

• Open discussion

• Contacts and additional information (A. Bleistein)

6

Remarks

• Jack Hayes, NWS

• Joel Myers, President and Founder, Accuweather

• Bob Ryan, WNBC-4 in Washington, D.C

• Pam Stephens, NSF

7

Motivation Elliot Abrams (ACUF Co-Chair, AccuWx)

Motivation Elliot Abrams (ACUF Co-Chair, AccuWx)

8

Characterizing and Communicating Uncertainty…

Characterizing and Communicating Uncertainty…

To promote better forecast-based decisions

To reduce losses caused by weather and climate

To promote better forecast-based decisions

To reduce losses caused by weather and climate

9

Are probability forecasts based on meteorological

quicksand, rendering products made from them of

dubious, uncertain value?

Are probability forecasts based on meteorological

quicksand, rendering products made from them of

dubious, uncertain value?

10

Current public forecast probabilities:

Provide the percentage chance of measurable precipitation at a fixed location during a specified time period, but

- Give no information about how long it will rain- Give no information about how much it will rain

-Give no contingency information (ex: the chance of rain is 50%, but if it does rain, the chance of it lasting more than 30

minutes is 5%)

11

InspirationInspiration: BaseballBatting averages are taken

to three decimal places (.334)

Statistical verifications are conducted to measure past player performance in order

predict future strategies

12

We are assigned a daunting, challenging task

But we have a large group of interested, smart, engaged

and determined people

We are assigned a daunting, challenging task

But we have a large group of interested, smart, engaged

and determined people

13

Can we forge a way forward, get our questions addressed, and understand what comes next…and do all of this today?

14

Certainly!

Let’s get started!

Certainly!

Let’s get started!

15

Committee OverviewPaul Hirschberg (ACUF Co-Chair, NWS)

• Vision: Enterprise-wide partnership that generates and communicates forecast uncertainty information meeting Nation’s needs for informed decisions:

– protecting life and property,

– supporting national defense and homeland security,

– enhancing the economy, and

– meeting specific needs of partners, users, and customers.

• Mission: Develop Enterprise-wide goals and roadmap for providing forecast uncertainty information, building off NRC recommendations

• Deliverable: An Enterprise implementation (action) plan for forecast uncertainty that has been reviewed and coordinated with partners

• Goal: Enterprise partners put into effect

16

Committee Overview Enterprise Plan Components

• Needs, opportunities, and benefits of providing hydrometeorological forecast uncertainty products and services to the Nation

– Why is uncertainty information important?

• Enterprise Goals for forecast uncertainty products and services

– Specifics – What will Nation get and how good will it be?

• Description of what is needed to meet goals and reach vision

– Solutions: What is needed to do this?

• Suggested roles and responsibilities of enterprise partners

– Who should do what and with who?

• Enterprise-wide Roadmap

– How will all the pieces fit together?

17

Committee Overview Committee Structure

• Organized into Topic Workgroups along major elements of plan deliverable

• Work must be done in parallel– Coordination, Communication is key

– Looking to use new collaboration technology, e.g., Wikis

Integration

Needs Goals SolutionsRoles and

ResponsibilitiesRoadmap

18

Topic Workgroup 1: National Needs and Benefits

Workgroup Composition

• Co-leads: Lee Anderson (NWS) and Betty Morrow (FIU)

• Members: – Dan Bickford (WSPA – TV)

– Julie Demuth (NCAR)

– Jayant Deo

– Jun Du (NCEP)

– Bob Glahn (NWS)

– Jenifer Martin (NCAR)

– Dan Satterfield (WHNT – TV)

– Paul Schultz (NOAA/GSD)

– Dick Westergard (Shade Tree Meteorology, LLC)

19

• Needs/requirements/opportunities – categorized • Opportunities -- what types/forms of information can be

provided?

• Needs

– End users and decision makers (weather and water)

Public

Emergency managers

Energy Sector

– Service Providers

Private Sector

– Academia – educational initiatives as proposed by NRC

• National Benefits of providing forecast uncertainty information

Topic Workgroup 1: National Needs and Benefits

Deliverables

• Construction

• DOD

• DHS

• Aviation

• Government

20

Topic Workgroup 1: National Needs and Benefits

Methodology

• Review, consolidate and complement existing sources of information:

– Obtain an initial information set from various sources (e.g., surveys, research)

– Identify major user-groups that use hydrometeorological information (e.g., forecasts)

– Use various methods (e.g., surveys, conferences) to obtain users’ needs.

Inform users’ about available hydrometeorological information while obtaining their uncertainty needs.

– Collaborate with ACUF sub-groups, and other related groups, to obtain additional requirements, and information resources.

– Obtain and develop benefits of using uncertainty in products.

21

Topic Workgroup 2: Enterprise GoalsWorkgroup Composition

• Co-leads: John Gaynor (OAR) and Alan Stewart (UGA)

• Members– Barbara Brown (NCAR)

– Chris Elfring (NAS)

– Greg Fishel (WRAL – TV)

– Pat Hayes (Northrup Grumman)

– Carlie Lawson (Natural Hazards Consulting)

– Chris Maier (NWS)

– Bernard Meisner (NWS)

– David Novak (NWS)

– Scott Sandgathe (APL – UW)

– Robyn Weeks (The Weather Channel)

22

Topic Workgroup 2: Enterprise Goals

Deliverables

• Enterprise Goals for forecast uncertainty products and services (threshold and longer-term)

– Outcome Goals

– Output Goals New products and services (collaboration with Subgroup 1)

with performance goals

– Awareness/Education/Credibility Goals

23

Topic Workgroup 2: Enterprise Goals

Outcome Goals

• Lives saved and injuries prevented

• Improve economic competitiveness and reduce weather-related costs

• Enhance productivity and efficiencies in weather-sensitive industries

• Some specific examples:

– More efficient surface and air transportation

– More effective decisions and timing for evacuations

– Decreased impacts of severe weather

– More effective decisions on fuel use for power generation

– More effective decisions on water resources

– Better educated citizenry in understanding and on the use of uncertainty in forecasts

24

Topic Workgroup 2: Enterprise Goals

Output Goals (Collaboration with Subgroup 1)

• By 2015, the routine dissemination and public communication of probabilistic weather and climate forecast products that are:

– timely availability of data and information

– highly reliable

– sharp, to the extent possible

– competitive in skill with the best offered worldwide;

– available for both routine and high-impact events;

– available in user-defined formats and in displays intuitive to all users

– with forecast verification information readily accessible.

25

Topic Workgroup 2: Enterprise Goals

Awareness/Education/Credibility Goals

• Probabilistic forecasting – standard in undergraduate and graduate education

• Continuing education of meteorological community

• Potential AMS and NWS certification

• Public education, K-12, severe weather awareness talks, web pages

• Continual innovation of product suite in response to user needs

• User-defined verification routinely available

26

Topic Workgroup 2: Enterprise Goals

Methodology

• Develop a vision and set goals as a group

• Vet vision and goals

– ACUF meetings

– Appropriate AMS meetings

– Extensive outreach to users and experts

– Other coordination efforts

27

Topic Workgroup 3:

SolutionsWorkgroup Composition

• Co-chairs: Tom Hamill (OAR) and Ross Hoffman (AER)

• Members:

– Tom Dulong (UCAR)

– Paul O. G. Heppner (3SI)

– Eddie Holmes (CBM)

– David Myrick (NWS)

– Carla Roncoli (UGA)

– John Schaake (retired NOAA)

– Dan Stillman (Inst. for Global Env. Strategies)

– Zoltan Toth (NCEP)

– Bill Bua (NCEP)

28

Topic Workgroup 3: SolutionsDeliverables

• Description of what is needed to meet the goals and reach vision

Topic/Subtopics

1. Education and Outreach A. Outreach (what products? what formats?) B. Test beds (provider / user interaction) C. Undergraduate and graduate education D. K-12 education E. Customer education (public & prof. end-users) F. Forecaster training 2. Probabilistic Forecast System Development A. Basic predictability research B. Ensemble forecast technique development C. Statistical post-processing D. Verification E. Tailored product development 3. Operational Technology Requirements A. Supercomputers B. Forecast, observation, analysis, user info. archival C. Bandwidth and data transport tools D. AWIPS and other display systems

29

Topic Workgroup 3: Solutions Methodology

• Workgroup co-leads interact with ACUF leads and other groups leads to formulate coordinated overall direction.

– Subgroup 1, 2 will provide requirements; subgroup 3 will determine implementation strategies; subgroup 4 will suggest who should implement which parts of the solution.

• Workgroup co-leads, with help from subgroup members, prepares section material. As needed, workgroup members interact with broader community to flesh out details. Regular telecons for coordination.

– Existing examples analyzed–what works, what doesn’t

• Workgroup’s material is synthesized by ACUF co-leads and workgroup leads.

– Workgroup also vets ACUF reports.

• Overall roadmap is critiqued by broader community.

• Iterate based on feedback from enterprise members.

30

Topic Workgroup 4: Roles and Responsibilities

Workgroup Composition

• Co-chairs: John Sokich (NWS) and Brenda Philips (UMASS)

• Members:

– Matthew Biddle (OU)

– David Bright (SPC)

– Gordon Brooks (AFWA)

– Gina Eosco (Cornell U)

– John Ferree (NWS)

– Jim Hansen (NRL)

– Paul Nutter (Univ. No. CO)

– Dan O’Hair (OU)

31

Topic Workgroup 4:

Roles and ResponsibilitiesDeliverables

• List of all Enterprise Partners with contact info and relevant areas of expertise for use by sub-groups

• Compilation of ACUF Best Practices

– Best practices for cross-enterprise collaboration

– Impediments for collaboration

• Implementation roles/responsibilities of enterprise members

• Advocacy plan

• After ACUF – how to move forward

32

Topic Workgroup 4: Roles and Responsibilities

Methodology

• Information gathering

– Town Hall meeting at AMS

– Other Weather and Climate Enterprise meetings

– Weather and Climate Enterprise database ?

• Interviews with selected ACUF enterprise members for case studies/testbeds

• Literature/policy review (eg: NRC report “Crossing the Valley of Death,” “Fair Weather”)

33

Topic Workgroup 5: Roadmap

Workgroup Composition

• Co-chairs: Steve Tracton (retired NOAA) and Neil Stuart (NWS)

• Members:

– Jon Ahlquist (FSU)

– Peter Browning (NWS)

– Dan C. Collins (NCEP)

– Chuck Doswell (OU)

– Mary Erickson (NOS)

– Rebecca Morss (UCAR)

– Leonard Smith (London School of Economics)

– Bernadette Woods (WJZ – TV)

34

Topic Workgroup 5: RoadmapDeliverables

• Enterprise-Wide, “Strategic Roadmap” (including alternatives)

– THE primary overall ACUF deliverable

– “End-to-End” milestones and timelines

– Relevant, effective and meaningful metrics of progress

35

Topic Workgroup 5: RoadmapMethodology

• Integrate input from subgroups in context of appropriate strategic planning and roadmapping templates

– Pro-active interaction/coordination between subgroups

– and outreach to Enterprise members and constituents (push and pull perspectives; achieve consensus)

• “Strategic Roadmapping”

– Link strategies to realistically achievable action plans ( “Threshold Objectives)

– Re-evaluate and re-visit pathway as warranted (not static)

– “Off ramps” to actions/implementations (“low-hanging fruit”)

– “On ramps” to accommodate newly identified opportunities, strategies, etc.“

36

Topic Workgroup 5: Roadmap

Methodology (cont’d)

• “Critical Path Analysis”

– Temporal ordering of sequential and parallel activities

represent stages of project co-dependencies

Identify critical activities (“predecessors”) required before other action can continue

Identify “float” activities; can be extended or delayed without jeopardizing project

Identify “what if” scenarios and linkages

StartEnd

SIMPLISTIC SCHEMATIC

Transformation from traditional to ensemble based forecast system where uncertainty is assessed, propagated and conveyed throughout entire forecast process (“End to End”)

37

ACUFTimeline

• Finalize Committee Charter; identify workgroup leads and memberships (Nov. 07)

• Develop draft Committee work plan including workgroup deliverables, methodologies, tasks, timelines, and resource needs (Dec. 07)

• Discuss and review work plan at AMS Annual Meeting – Town Hall and Full Committee Meeting (Jan. 08)

• Finalize Committee work plan (Feb. 08)

• Workgroups meet as necessary accomplish tasks (Jan.–Aug. 08)

• Full Committee review of preliminary workgroup deliverables via briefings (Sept. 08)

• First draft of Workgroup Sections (Oct. 08)

• First draft of complete Enterprise Implementation Plan (Dec. 08)

• Full Committee review of first draft Enterprise Implementation Plan - at AMS Annual Meeting (Jan. 09)

• Stakeholder review and comment of draft Implementation Plan (Feb.-May 09)

• Final draft of Enterprise Implementation Plan (Sept. 09)

38

ACUF Town Hall

Open Discussion

39

ACUF Contact and Information

• Please contact Andrea Bleistein andrea.bleistein@noaa.gov

• ACUF Town Hall slides are posted at: http://www.ametsoc.org/dl

• Full Committee Meeting Today – all welcome– 4:30 – 5:30 Hilton Riverside, Rosedown Room