"Cyberinfrastructure forEnvironmental Observations"

Invited Talk to Symposium on

Science and Technology in GEOSS: The Role of Universities

Hosted by Calit2@UCSD

for the SIO Center for Earth Observations and Applications

La Jolla, CA

November 21, 2005

Dr. Larry Smarr

Director, California Institute for Telecommunications andInformation Technology

Harry E. Gruber Professor,

Dept. of Computer Science and Engineering

Jacobs School of Engineering, UCSD

Components of a Future Global System for Earth Observation(Sensor Web)

Vision for Creating an Integrated InteractiveInformation System for Earth Exploration

Major Obstacle: Trying to Do Global Earth SciencesOn a Shared Internet Designed for Email and FTP

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,0002

00

1

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

20

13

20

14

Calendar Year

Cu

mu

lati

ve T

era

Byte

s

Other EOS

HIRDLS

MLS

TES

OMI

AMSR-E

AIRS-is

GMAO

MOPITT

ASTER

MISR

V0 Holdings

MODIS-T

MODIS-A

Other EOS =

• ACRIMSAT

• Meteor 3M

• Midori II

• ICESat

• SORCE

file name: archive holdings_122204.xls

tab: all instr bar

Terra EOM

Dec 2005

Aqua EOM

May 2008

Aura EOM

Jul 2010

NOTE: Data remains in the archive pending transition to LTA

Source: Glenn Iona, EOSDIS Element Evolution Technical Working Group January 6-7, 2005

AddingSeveralTBs per

Day

Challenge: Average Throughput of NASA Data Productsto End User is < 50 Mbps

TestedOctober 2005

http://ensight.eos.nasa.gov/Missions/icesat/index.shtml

Internet2 Backbone is 10,000 Mbps!Throughput is < 0.5% to End User

fc *=

Dedicated Optical Channels MakesHigh Performance Cyberinfrastructure Possible

(WDM)

Source: Steve Wallach, Chiaro Networks

“Lambdas”Parallel Lambdas are Driving Optical Networking

The Way Parallel Processors Drove 1990s Computing

10 Gbps per User ~ 200x Shared Internet Throughput

San Francisco Pittsburgh

Cleveland

National Lambda Rail (NLR) and TeraGrid ProvidesCyberinfrastructure Backbone for U.S. Researchers

San Diego

Los Angeles

Portland

Seattle

Pensacola

Baton Rouge

HoustonSan Antonio

Las Cruces /El Paso

Phoenix

New York City

Washington, DC

Raleigh

Jacksonville

Dallas

Tulsa

Atlanta

Kansas City

Denver

Ogden/Salt Lake City

Boise

Albuquerque

UC-TeraGridUIC/NW-Starlight

Chicago

International Collaborators

NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout

NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone

Links TwoDozen State

and RegionalOptical

Networks

DOE, NSF,& NASA

Using NLR

September 26-30, 2005Calit2 @ University of California, San Diego

California Institute for Telecommunications and Information Technology

Global ConnectionsBetween University Research Centers at 10Gbps

iGrid 2005T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y

Maxine Brown, Tom DeFanti, Co-Chairs

www.igrid2005.org

21 Countries Driving 50 Demonstrations1 or 10Gbps to Calit2@UCSD Building

Sept 2005--A Number of Projects are SensorNets

Prototyping Cabled Ocean Observatories EnablingHigh Definition Video Exploration of Deep Sea Vents

Source John Delaney & Deborah Kelley, UWash

Canadian-U.S. Collaboration

A Near Future Metagenomics Fiber Optic Cable Observatory

Source John Delaney, UWash

Calit2 is Partnering with the new SIOCenter for Earth Observations and Applications

• Viewing and Analyzing Earth Satellite Data Sets

• Earth Topography

• Project Atmospheric Brown Clouds

• Climate Modeling

• Ocean Observatories

• Coastal Zone Data Assimilation

• Marine Microbial Ecology Metagenomics

• Earth Sciences Collaboratory

ROADnet and HiSeasNet are Prototypes of theFuture of In Situ Earth Observing Systems

http://roadnet.ucsd.edu

ROADNet Architecture: SensorNets, Storage Research Broker, Web Services, Work Flow

KeplerWeb ServicesSRBAntelope

Frank Vernon, SIO; Tony Fountain, Ilkay Altintas, SDSC

12

14

16

18

20

22

22-Aug 23-Aug 24-Aug 25-Aug 26-Aug 27-Aug 28-Aug

Date

Wate

r T

em

pera

ture

(°C

)

0

2

4

6

8

10

12

14

16

18

20

Pre

cip

itati

on

(mm

per

5 m

inu

te i

nte

rval)

Surface

0.5 meters

1 meter

1.5 meters

2 meters

2.5 meters

3 meters

Precipitation

Typhoon

Yuan Yang Lake, Taiwan – August 2004

Part of a growing global lake observatory network -

http://lakemetabolism.org

Access can be

difficult during the

most interesting times

Photo by Peter Arzberger, October 2004

Used

by

NS

F D

irecto

r Feb

2005

Source: Tim Kratz

Remote Observation of Episodic Eventsin Water-Based Ecological Systems

Supported by Moore Foundation

LOOKING:(Laboratory for the Ocean Observatory

Knowledge Integration Grid)

Adding Web & Grid Services to Optical Channelsto Provide Real Time Control of Ocean Observatories

• Goal:

– Prototype Cyberinfrastructure forNSF’s Ocean Research InteractiveObservatory Networks (ORION)

• LOOKING NSF ITR with PIs:

– John Orcutt & Larry Smarr - UCSD

– John Delaney & Ed Lazowska –UW

– Mark Abbott – OSU

• Collaborators at:

– MBARI, WHOI, NCSA, UIC, CalPoly,UVic, CANARIE, Microsoft, NEPTUNE-Canarie

LOOKING is Driven By NEPTUNE CI Requirements

http://lookingtosea.ucsd.edu/

Making Management of Gigabit Flows Routine

Pilot Project ComponentsPilot Project Components

LOOKING Builds on the Multi-InstitutionalSCCOOS Program, OptIPuter, and CENIC-XD

• SCCOOS is Integrating:

– Moorings

– Ships

– Autonomous Vehicles

– Satellite Remote Sensing

– Drifters

– Long Range HF Radar

– Near-ShoreWaves/Currents (CDIP)

– COAMPS Wind Model

– Nested ROMS Models

– Data Assimilation andModeling

– Data Systems

www.sccoos.org/

www.cocmp.org

Yellow—Initial LOOKING OptIPuter Backbone Over CENIC-XD

The OptIPuter Project –Linking Global Scale Science Resources to User’s Linux Clusters

• NSF Large Information Technology Research Proposal– Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI

– Partnering Campuses: USC, SDSU, NW, TA&M, UvA, SARA, NASA

• Industrial Partners– IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent

• $13.5 Million Over Five Years—Entering 4th Year

• Creating a LambdaGrid “Web” for Gigabyte Data Objects

NIH Biomedical Informatics NSF EarthScopeand ORION

Research Network

NSF is Launchinga New Cyberinfrastructure Initiative

www.ctwatch.org

“Research is being stalled by ‘information overload,’ Mr. Bement said, becausedata from digital instruments are piling up far faster than researchers can study.In particular, he said, campus networks need to be improved. High-speed datalines crossing the nation are the equivalent of six-lane superhighways, he said.But networks at colleges and universities are not so capable. “Those massiveconduits are reduced to two-lane roads at most college and universitycampuses,” he said. Improving cyberinfrastructure, he said, “will transform thecapabilities of campus-based scientists.”

-- Arden Bement, the director of the National Science Foundation

UCSD is Prototyping Campus-Scale On-Rampsto the National LambdaRail, TeraGrid, and GLIF

SIO Ocean SupercomputerIBM Storage Cluster

2 Ten GbpsCampus Lambda

Raceway

StreamingMicroscope

Source: Phil Papadopoulos, SDSC, Calit2

UCSD Campus LambdaStore Architecture

Global LambdaGrid

Flat File

Server

Farm

W E B PORTAL

Traditional

User

Response

Request

Dedicated

Compute Farm

(100s of CPUs)

TeraGrid: Cyberinfrastructure Backplane(scheduled activities, e.g. all by all comparison)

(10000s of CPUs)

Web(other service)

Local

Cluster

LocalEnvironment

Direct

Access

Lambda

Cnxns

OptIPuter Cluster Cloud

Data-

Base

Farm

10 GigE

Fabric

Calit2/SDSC Direct Access Core ArchitectureSupporting Massive Instrumental Datasets

Source: Phil Papadopoulos, SDSC, Calit2

EmergingServices Oriented

ArchitectureEnabling Use of

LambdaGrids + W

eb

Se

rvic

es

Tiled Walls for Interactive Exploration of Large Earth SciencesData Sets With Integration of Streaming High Resolution Video

Calit2@UCI Apple Tiled Display WallDriven by 25 Dual-Processor G5s

50 Apple 30” Cinema Displays200 Million Pixels of Viewing Real Estate!

Source: Falko Kuester, Calit2@UCINSF Infrastructure Grant

Data—One Foot Resolution USGS Images of La Jolla, CA

HDTV

Digital Cameras Digital Cinema

Combining Telepresence with Remote InteractiveAnalysis of Earth Sciences Data Over NLR

HDTV Over Lambda

OptIPuter Visualized

Data

SIO/UCSD

NASA Goddard

http://www.calit2.net/articles/article.php?id=660August 8, 2005