“OptIPuter Tech Transfer to the Broader e-Science and HPC Communities"
OptIPuter All Hands Meeting
Calit2@UCSD
La Jolla, CA
December 20, 2006
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology;
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
Informing the Cyberinfrastructure Initiative
www.ctwatch.org
Global Lambda Integrated Facility (GLIF)International Group Innovating the LambdaGrid
• GLIF Technical and Control Plane Working Groups– Interim Meetings on 8-9 February 2006 – University of New Mexico, Albuquerque, United States
• GLIF Research and Applications Working Group– Maxine Brown and Larry Smarr– Abstract Taxonomy from 50 iGrid Demos
• GLIF Full Meeting – Tokyo, Japan– September 11-15, 2006– Interim Deadline for SC06 Demos
OptIPuter Will Have Coordinated Approach to SC06
chance2 10Gig (eth1 Intel Pro/10GbE)5 August 2005
chance1 10Gig (eth1 Intel Pro/10GbE)5 August 2005
DRAGON 10Gig DWDM XFP 5 August 2005
5
GSFC Scientific and Engineering Network (SEN)Mrtg-based `Daily' Graph (5 Minute Average)
Bits per second In and Out On Selected Interfaces
On August 5, 2005, GSFC’s Bill Fink simultaneously conducted two 15-minute-duration UDP-based 4.5-Gbps flow tests, with one flow between GSFC-UCSD and the other between GSFC-StarLight/Chicago. This filled both the NLR/WASH-STAR and DRAGON/channel49 lambdas to 90% of capacity. Flows were also tested in both directions. He measured greater than 9-Gbps aggregate in each direction and no-to-negligible packet losses.
Lambdas Give End Users Sustained ~ 10 Gbps Data Flow Rates
200 Times Faster Than Standard
Internet2!
Source: Pat Gary, NASA GSFC
ARC/NGIX-WestARC/NGIX-West
JPLJPL
GSFCGSFC
NGIX-EastNGIX-East
NLR SunnyvaleNLR Sunnyvale
NLR Los AngelesNLR Los Angeles
NLR ChicagoNLR Chicago
StarLightStarLight
LRCLRC
NLR ClevelandNLR ClevelandGRCGRC
NLR HoustonNLR HoustonJSCJSC
NLR MSFCNLR MSFC
NREN SitePeering Points1 GE10 GE
MAXMAX
NLR Baton RougeNLR Baton RougeSSCSSC
NLR JacksonvilleNLR Jacksonville
KSCKSC
OptIPuter Experiments AcceleratedFuture NASA NREN Over NLR
Linking OptIPuter to the DRAGON TestbedWashington, D.C. Metropolitan Area
HOPI / NLR
CLPK
ARLG
DCGW
MCLN
MIT Haystack Observatory(HAYS)U. S. Naval Observatory
(USNO)
University of Maryland College Park(UMCP)
Goddard Space Flight Center(GSFC)
National Computational Science Aliance (NCSA)
Univ of Southern California/Information Sciences Institute
(ISIE)
DCNE
DCNE
MAX
Bossnet
Global e-VLBIiGrid / SC05
• Goal: Real-Time VLBI Radio Telescope Data Correlation – from the USA (MIT Haystack, GGAO),
– Japan (Kashima) and
– Europe (Onsala in Sweden, Jodrell in the UK, Westerbork in The Netherlands)
• Achieved 512Mb Transfers from USA and Sweden to MIT,
• Results Streamed to iGrid.
Sloan Sky SurveyData Mining and Visualizing Data Using OptIPuter
• SDSS-I – Imaged > 8,000 Square Degrees of the Sky in Five Bandpasses– Detecting Nearly 200 Million Celestial Objects – Measured Spectra Of:
– > 675,000 galaxies
– 90,000 quasars
– 185,000 stars
• SDSS-II– Underway till 2008
www.sdss.org
iGRID2005From Federal Express to Lambdas:
Transporting Sloan Digital Sky Survey Data Using UDT
Robert Grossman, UIC
Evolution is the Principle of Biological Systems:Most of Evolutionary Time Was in the Microbial World
You Are
Here
Source: Carl Woese, et al
Much of Genome Work Has
Occurred in Animals
PI Larry Smarr
$24.5M Over 7 Years
Announcing Tuesday January 17, 2006
The Sargasso Sea Experiment The Power of Environmental Metagenomics
• Yielded a Total of Over 1 billion Base Pairs of Non-Redundant Sequence
• Displayed the Gene Content, Diversity, & Relative Abundance of the Organisms
• Sequences from at Least 1800 Genomic Species, including 148 Previously Unknown
• Identified over 1.2 Million Unknown Genes
MODIS-Aqua satellite image of ocean chlorophyll in the Sargasso Sea grid about the BATS site from
22 February 2003
J. Craig Venter, et al.
Science 2 April 2004:
Vol. 304. pp. 66 - 74
Marine Genome Sequencing ProjectMeasuring the Genetic Diversity of Ocean Microbes
CAMERA will include All Sorcerer II Metagenomic Data
Prochlorococcus Microbacterium
Burkholderia
Rhodobacter SAR-86
unknown
unknown
Metagenomics “Extreme Assembly” Requires Large Amount of Pixel Real Estate
Source: Karin RemingtonJ. Craig Venter Institute
Flat FileServerFarm
W E
B P
OR
TA
L
TraditionalUser
Response
Request
DedicatedCompute Farm(100s of CPUs)
TeraGrid: Cyberinfrastructure Backplane(scheduled activities, e.g. all by all comparison)
(10000s of CPUs)
Web(other service)
Local Cluster
LocalEnvironment
DirectAccess LambdaCnxns
Data-BaseFarm
10 GigE Fabric
Calit2’s Direct Access Core Architecture Will Create Next Generation Metagenomics Server
Source: Phil Papadopoulos, SDSC, Calit2+
We
b S
erv
ice
s
Sargasso Sea Data
Sorcerer II Expedition (GOS)
JGI Community Sequencing Project
Moore Marine Microbial Project
NASA Goddard Satellite Data
Community Microbial Metagenomics Data
First Implementation of the CAMERA Complex
Compute Database &Storage
Calit2/SIO will Establish Persistent OptIPuter Collaboratory with Venter Institute
HDTV Over Lambda
OptIPuter Visualized
Data
SIO/UCSD
NASA Goddard
www.calit2.net/articles/article.php?id=660
August 8, 2005
25 Miles
Venter Institute
Calit2/SDSC Proposal to Create a UC Cyberinfrastructure
of “On-Ramps” to National LambdaRail ResourcesOptIPuter + CalREN-XD + TeraGrid = “OptiGrid”
Source: Fran Berman, SDSC , Larry Smarr, Calit2
Creating a Critical Mass of End Users on a Secure LambdaGrid
UC San Francisco
UC San Diego
UC Riverside
UC Irvine
UC Davis
UC Berkeley
UC Santa Cruz
UC Santa Barbara
UC Los Angeles
UC Merced