linked environments for atmospheric discovery (lead): web services for meteorological research and...

Linked Environments for Atmospheric Discovery (LEAD):

Web Services for Meteorological Research and Education

What Would YOU Do if These Were About to Occur?

What THEY Do to Us!!! Each year in the US, mesoscale weather –

local floods, tornadoes, hail, strong winds, lightning, and winter storms – causes hundreds of deaths, routinely disrupts transportation and commerce, and results in annual economic losses > $13B.

What Weather Technologies Do…

Forecast ModelsNEXRAD Radar

Decision Support Systems

Virtually Nothing!!!

Tornadic Storms

Moderate Rain

Radars Do Not Adaptively Scan

Operational Models Run Largely on Fixed Schedules in Fixed Domains

Cyberinfrastructure is Virtually Static

ENIAC (1948)

ARPANET (1980)

Abilene Backbone (2005)

Earth Simulator (2005)

National Lambda Rail (2005)

We Teach Using Current Weather Data But Students Don’t Interact

With It

So What??? Weather is Local, High-Impact, Heterogeneous and Rapidly Evolving…Yet Our

Technologies and Thinking are Static

SevereThunderstorms

FogRain and

Snow

Rain andSnow

IntenseTurbulence

Snow andFreezing

Rain

The Reality for Society: Dynamic, Local and High

Impact

A Fundamental Research Question

Can we better understand the atmosphere, educate more effectively about it, and forecast more accurately if we adapt our technologies and approaches to the weather as it occurs?

People, even animals adapt/respond: Why don’t our resources???

Sponsored by the National Science Foundation

The LEAD VisionRevolutionize the ability of scientists, students,

and operational practitioners to observe, analyze, predict, understand, and respond to intense local weather by interacting with it

dynamically and adaptively in real time

What Does Adaptation Really Mean?

What Does it Buy?

Charles Darwin

Sample Problem: March 2000 Fort Worth Tornadic Storm

Tornado

Local TV Station Radar

NWS 12-hr Computer Forecast Valid at 6 pm CDT (near tornado time)

No Explicit Evidence of Precipitation in North Texas

Reality Was Quite Different!

LEAD Approach

StreamingObservations

Storms Forming orConditionsFavorable

Forecast Model

On-DemandGrid Computing

Data Mining

6 pm 7 pm 8 pmR

adar

Xue et al. (2003)

Fort Worth

6 pm 7 pm 8 pmR

adar

Fcs

t W

ith

Rad

ar D

ata

2 hr 3 hr 4 hr

Xue et al. (2003)

Fort Worth

Fort Worth

What Does it Take to Make This Possible?

Adaptive weather tools Adaptive sensors Adaptive cyberinfrastructure

In a User-CenteredFramework

Where EverythingCan

Mutually Interact

How Does LEAD Do It? The Notion of a Web Service

Web Service: A program that carries out a specific set of operations based upon requests from clients

The LEAD architecture is a “Service Oriented Architecture” (SOA), which means that all of the key functions are represented as a set of services.

Service-Oriented Architecture

Service A(Analysis)

Service B(Model)

Service C(Radar Stream)

Service D(Work Space)

Service E(VO Catalog)

Service F(Viz Engine)

Service G(Monitoring)

Service H(Scheduling)

Service I(Decoder)

Service J(Repository)

Service K(Mining)

Service L(Decoder)

Many others…

Service B(Model)

Service A(Analysis)

Service C(Radar Stream)

Service D(Work Space)

Service K(Mining)

Service L(Decoder)

Service J(Repository)

Can Solve Broad Classes of Problemsby Linking Services Together in Workflows

A LEAD Weather Prediction Workflow

Fault Tolerance in Action

Back to the Earlier Example…

As a Forecaster Worried About This Reality…

7 pm

As a Forecaster Worried About This Reality…

How Much Trust Would You

Place in This Model

Forecast?

3 hr

7 pm

Actual Radar

Ensemble Member #1 Ensemble Member #2

Ensemble Member #3 Ensemble Member #4Control Forecast

Actual Radar

Probability of Intense Precipitation

Model Forecast Radar Observations

Hazardous Weather Test Bed Collaboration with the

NOAA Storm Prediction Center and National Severe Storms Laboratory

Mid-April through early June Goals – to begin

understanding…– The fundamental

predictability of intense thunderstorms

– The utility of ensemble forecasts compared to single fine-grids

– Value of on-demand forecasts launched manually and automatically

The Value of Adaptation: Forecaster-Initiated Predictions

Brewster et al. (2008)

Observed Radar Echoes 20 hr Pre-ScheduledForecast

5 hr LEAD Dynamic WRF-ARF With RadarData Assimilation

Centers of On-Demand Forecast Grids Launched Automatically at NCSA During 2007

Spring Experiment

Launched automatically in response to hazardous weather messages (tornado watches, mesoscale discussions)

Launched based on forecaster guidance

Graphic Courtesy Jay Alameda and Al Rossi, NCSA

Real Impact!

Adaptive Observing Systems: Current Operational Radar System in

USNEXRAD Doppler Radar Network

#2. Earth’s curvature prevents 72% of the atmosphere below 1 km from being

observed

#1. Operates largely independentof the prevailing weather conditions

#3. Operates entirely independent fromthe models and algorithms that use its

data

The Limitations of NEXRAD

Data Courtesy Brenton MacAloney II, National Weather Service

Tornado Warnings

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006

Year (calander)

Rat

io

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

Min

ute

s

Probability of Detection (POD)

False Alarm Ratio (FAR)

Lead Time

NEX

RA

D

© 1998 Prentice-Hall, Inc. -- From: Lutgens and Tarbuck, The Atmosphere, 7th Ed.

NEXRAD/MPAR ($$$)

0-3 km

CASA ($)

NSF Engineering Research Center

for Collaborative Adaptive Sensing of the Atmosphere

(CASA)

Example of Adaptive Sampling

Oklahoma Test Bed

NWS Operational (NEXRAD)

Experimental(CASA)

NWS Operational (NEXRAD)

Real Time Testing Today

Radar Observations

Real Time Testing Today

9-Hour Forecast

9-Hour Forecast

9-Hour Forecast

The Million Dollar Question: Will

Computer Models Ever Be Able to Predict

Tornadoes?

Warn on Explicit Forecast?

Be careful what you wish for! A one-hour model-based “tornado warning” would be a game changer

Social and behavioral science elements are critical– Why did 550 people die in the US last year from

tornadoes? Our ability to effectively warn the public and

understand its response is relatively crude This is an area ripe for additional research –

and it is ESSENTIAL for making progress

Challenges

Each set of forecasts (ensemble and individual)– produces 6 TB of output PER DAY– Requires 9000 cores (750 nodes) of the Kraken

Cray XT5 at Oak Ridge– Takes 6.5 hours to run

Provisioning of data in real time Management in a repository – retention

time? Experiment reproducibility!! Creating products that will benefit the public

(smart device location-based warnings)

Other Challenges

LEAD: Potential to Transform Meteorological Research And Education