Sponsored by the National Science Foundation

GENI Alpha Demonstration

Nowcasting: UMass/CASA Weather Radar Demonstration

David IrwinNovember 3, 2010

http://geni.cs.umass.edu/visehttp://geni.cs.umass.edu/dicloud

http://www.geni.net

Sponsored by the National Science Foundation 2November 3, 2010

Problem

• CASA (an NSF ERC) is studying experimental networks of small controllable weather radars– Better data is the foundation of better hazardous

weather detection and earlier warnings– Complex modeling to detect inclement weather requires

many resources: sensors, bandwidth, storage, and computation

• Costly to dedicate resources for rare events– How do we generate accurate, short-term “nowcasts”

using these new distributed radar systems?

Sponsored by the National Science Foundation 3November 3, 2010

Solution

• Today: only a few large NEXRAD radars (100s)

• Tomorrow: many (1000s) smaller, less expensive radars produce data close to the ground where weather happens

• Requires a flexible infrastructure for coordinated provisioning of shared sensing, networking, storage, and computing resources on-demand

Sponsored by the National Science Foundation 4November 3, 2010

Example: Puerto Rico Testbed

• UPRM Student Testbed – Led by Jorge Trabal, Prof. Sandra Cruz-Pol,

and Prof. Jose Colom– http://www.youtube.com/watch?v=7TR64BhwMlI

Sponsored by the National Science Foundation 5November 3, 2010

Demo Background

• Dynamic end-to-end Nowcasting on GENI– Use GENI/Orca Control Framework (RENCI/Duke)

• https://geni-orca.renci.org/trac/• http://geni-ben.renci.org:11080/orca/

– Reserve heterogeneous slice of resources• Sensing Slice: UMass ViSE radars• Networking Slice: NLR, BEN-RENCI• Computation Slice: Amazon EC2 + UMass and RENCI VMs• Storage Slice: Amazon S3

Sponsored by the National Science Foundation 6November 3, 2010

Demo Data Flow

• Dynamic end-to-end Nowcasting – Mapping Nowcast Workflows onto GENI

Archival Storage

Radar Nodes

“raw” live data

Upstream LDM feed

archived netcdf data

Nowcast Processing

aggregatedmulti-radar data

Post to Web

Nowcast images for display

Sponsored by the National Science Foundation 7November 3, 2010

Multi-radar NetCDF Data

Nowcast Processing

1. DiCloud Archival Service (S3)2. LDM Data Feed (EC2)

“raw” live data

Generate “raw” live dataViSE/CASA radar nodes

http://stb.ece.uprm.edu/current.jsp

Use proxy to track usage-based spending on Amazon and enforce quotas and limits

http://geni.cs.umass.edu/vise/dicloud.php

1. Ingest mulit-radar data feeds2. Merge and grid multi-radar data2. Generate 1min, 5min, and 10min Nowcasts3. Send results over NLR to Umass4. Repeat

ViSE views steerable radars as shared, virtualized resources

http://geni.cs.umass.edu/vise

Nowcast images for display

Sponsored by the National Science Foundation 8November 3, 2010

GENI Technologies and Credits

• UMass-Amherst– ViSE and DiCloud projects

• University of Puerto Rico, Mayaguez– Jorge Trabal, Prof. Cruz-Pol, and Prof. Colom– OTG Radars

• Colorado State University– Prof. V. Chandrasekar– Nowcasting Software

• RENCI/Duke– Orca Control Framework– BEN network

• Starlight

Sponsored by the National Science Foundation 9November 3, 2010

Conclusion

• GENI is critical for next-generation applications– Enable nowcasting in experimental radar systems– GENI capabilities: “sliceability”/virtualization, federation,

network programmability

• Provide domain scientists a new platform– Experiment with tightly integrated systems combining

sensing, storage, networking, computing– Engage domain scientists in CASA and elsewhere

• Extend GENI network to Puerto Rico

Sponsored by the National Science Foundation 10November 3, 2010

Wrap-up

Sponsored by the National Science Foundation 11November 3, 2010

Demo Data Flow

• Dynamic end-to-end Nowcasting on GENI

Archival Storage

Radar Nodes

Generate “raw” dataViSE/CASA radar nodes

“raw” live data

http://stb.ece.uprm.edu/current.jsp

DiCloud ArchivalService on Amazon S3

Upstream LDM feed

archived netcdf data

Data publicly availableto downstream nodes

Use proxy to track usage-based spending on Amazon and enforce quotas and limits

http://geni.cs.umass.edu/vise/dicloud.php

ViSE views steerable radars as shared, virtualized resources

http://geni.cs.umass.edu/vise

Nowcast Processing

aggregatedmulti-radar data

GenerateNowcasts

1. Ingest data feeds from multiple radars2. Merge multi-radar data3. Generate 1min, 5min, and 10min

Nowcasts4. Repeat

http://vise-testbed.cs.umass.edu/nowcast/nowcast.html

Post to Web

Nowcast images for display

Sponsored by the National Science Foundation 12November 3, 2010

Thank you

UMass:Emmanuel

CecchetPrashant ShenoyJim KuroseEric Lyons

UPRM:Gianni PablosJosé OrtizWilson CastellanosMelissa AcostaJosé CorderoBenjamín De JesúsSandra Cruz-PolJosé Colom

CSU:V. ChandrasekarEvan RuzanskiYanting Wang RENCI:

Ilia BaldineJeff ChaseAnirban Mandel

Sponsored by the National Science Foundation 13November 3, 2010

Demo Overview

• Dynamic end-to-end Nowcasting on GENI – Slice of sensing, networking, computing, and storage

Archival Storage

Radar Nodes

Upstream LDM feed

Nowcast Processing

Post to Web

Generate “raw” dataViSE/CASA radars

Archive radar dataAmazon S3

“raw” live data

Archived data availableto downstream nodes

archived netcdf data

aggregatedmulti-radar data

Nowcast images for display

GenerateNowcasts

Sponsored by the National Science Foundation 14November 3, 2010

Demo Resource Listing in Orca/GENI