insar working group meeting, oxnard, oct 2004 1 it discussion day 1 afternoon breakout session

InSAR Working Group Meeting, Oxnard, Oct 2004

1

IT discussion

Day 1 afternoon breakout session

http://solidearth.jpl.nasa.gov/insar


2

GDS Data Flow

UpdateCatalog

Data FTP

Stanford University

Caltech MIT NSIDC Howard University

SDSC Long Term Archive5 years near-line

EDC Permanent Archive

University of Miami

Alaska SARFacility (ASF)

Mission Control Center (MCC)

DLR-GSOCGround StationWeilheim (WHM)

Short Term Archive 1 year on-line

CatalogScience Team

Archive Manager

Collection requests, acquisition requests, data requests

Acquisition plan, status, holdings, software, ancillary data

Catalog redirects user to appropriate FTP address for L0 data retrieval

L1 Data

Internet - 2

User

Internet - 2

Internet and Internet-2

Network Transfer Subsystem (NTS)Stanford University & NSIDC•Archive Manager

Caltech

UserCatalog

Archive Manager

User

NSIDC

User

NSIDCStanford University

SDSC Long Term Archive5 years near-line

Archive Manager

Alaska SAR Facility (ASF)

JPL Mission Ops

300 Mb/sec downlink

Populate STArchive

Catalog

Archive Manager

UserRedirect

PopulateLT Archive

InBackground

User

NSIDC

Catalog

MIT EDC Permanent Archive

SIO Science Planning Bd.

User Acq.Request

Catalog

SIO SciencePlanning Bd.

JPL Mission Ops

Mission ControlCenter (MCC)

DLR-GSOC Ground StationWeilheim (WHM)

Mission ControlCenter (MCC)

JPL Mission Ops

Status toUser

Catalog

Network Transfer Subsystem (NTS)Stanford University & NSIDC•Archive Manager

User Data.Request

Transfer toNTS

PopulatePermanentArchive in

Background


3

Solid Earth Research Virtual Observatory (SERVO)Solid Earth Research Virtual Observatory (SERVO)

Tier2 Center

Archive

SERVO

…Goddard JPL Ames

InstituteInstituteInstituteInstitute

Fully functional problem solving environment

•Plug and play composing of parallel programs from algorithmic modules

•On-demand downloads of 100 GB in 5 minutes•106 volume elements rendering in real-time

•Program-to-program communication in milliseconds

•Approximately 100 model codesData cache

~TBytes/day

Tier2 CenterTier2 CenterTier2 Center

Tier 0 +1

Tier 1

Tier 3

Tier 4

Tier2 Center

•1 PB per year data rate in 2010

•Distributed Heterogeneous Real-Time Datasets

Observations

Archive

Downlink

Archive Downlink

Downlink

…

……

…… …

…

100 TeraFLOPs sustained

Tier 2

Workstations, other portals

100 - 1000 Mbits/sec


4

National Lambda Rail (NLR)

15808 Terminal, Regen or OADM siteFiber route

NLRComing Up NowInitially 4 10G

WavelengthsFull Footprint Ops

by 3Q or 4Q04 Internet2 HOPI Initiative (w/HEP) To 40 10G Waves in Future

Transition beginning now to optical, multi-wavelength Community owned or leased fiber networks for R&E

PITPIT

PORPOR

FREFRE

RALRAL

WALWAL

NASNASPHOPHO

OLGOLGATLATL

CHICHI

CLECLE

KANKAN

OGDOGD

SACSAC BOSBOSNYCNYC

WDCWDC

STRSTR

DALDAL

DENDEN

LAXLAX

SVLSVL

SEASEA

SDGSDG

JACJAC Regional Dark Fiber Initiatives

in 18 U.S. States


5

UCSD

StarLightChicago

UIC EVL

NU

Solution: Move to Internet Protocol Over Dedicated Optical Lightpaths

NASA JPL

NASA Ames

PNWGPSeattle

Pacific Wave

NLR

NLR

NASA Goddard2Level 3

Level 3

4

CalREN-XD

4

CENICSan DiegoGigaPOP

2

UCI

ISI CENICLos Angeles

GigaPOP

2

U Amsterdam

NetherLightAmsterdam

CA*net4

1GE lambda

10GE lambda

NASA Goddard IT Pathfinder Working GroupEarth and Climate Scientists-

Creating a Virtual Collaboratory Between

Goddard and SIO (UCSD)

Coupling NASA Centers

to NSF OptIPuter


6

Database Database

Analysis and VisualizationPortal

RepositoriesFederated Databases

Data Filter

Services

Field Trip DataStreaming Data

Sensors

?DiscoveryServices

SERVOGrid

ResearchSimulations

Research Education

CustomizationServices

From Research

to Education

EducationGrid ComputerFarmGeoscience Research and Education Grids

GISGrid

Sensor GridDatabase Grid

Compute Grid


7

Basic System Architecture


8

Schematic of a Typical Ontology Manager

DataResource

Grids

MetadataManagement

TopicMining

Authoring ToolOntology

Architecture

DatabaseArchitecture

Task Pool ManagerSemantic -basedQuery Generator

DB1 DB2Data

Source iDBm

DataSource1

Conn.Pool

É É

DBMS1 DBMSn

Conn.Pool

Conn.Pool

Conn.Pool

FederatedDatabase Services

Onto-logy

Onto-logy

Onto-logy

Onto-logy

Onto-logy

Data Integration API (Semantic-based Wrapper)

Meta-ontologies(metadata of ontologies

assocaited with local databases)


9

What will be needed as technology for:

• Access to data through data bases and the web

• Software for data mining and data exploration

• Software for simulating, interpreting, visualizing data


10

Who are the End Users outside the InSAR Research Community?

• Engineering• Education• Government/Policy decision makers• Operational/Hazards• Commercial


11

Issues Associated with Access to data through databases

• Standards for domestic and international missions

• Security• Quality and Validation• Bandwidth• Interoperability – XML & GML• Content addressable storage• Near real time or real time

– Rapid response tasking of satellite


12

S/w and H/w

• Data exploration and analysis– Higher level functionality is desirable– Portal architecture– Peer to peer– Collaborative– Error quantification– Graduated authentication

• S/w development– Curation– Reliability and robustness– Fusion


13

Simulating, interpreting, and visualizing data

• Large scale simulation & modeling capability is needed & should be part of mission plan

• Centralized and Grid computing resources

• Code sharing (open source)• Code validation and curation• Reliability and robustness


14

Remaining questions

• Commercial component?• Operational versus Science

mission?

insar working group meeting, oxnard, oct 2004 1 it discussion day 1 afternoon breakout session

Documents