optiputer larry smarr, pi tom defanti, jason leigh, mark ellisman, phil papadopoulos, co-pis
DESCRIPTION
OptIPuter Larry Smarr, PI Tom DeFanti, Jason Leigh, Mark Ellisman, Phil Papadopoulos, Co-PIs Maxine Brown, Project Manager. A -> Need full Internet routing B -> Need VPN services on/and full Internet routing C -> Need very fat pipes, limited multiple Virtual Organizations. A. B. C. - PowerPoint PPT PresentationTRANSCRIPT
OptIPuter
Larry Smarr, PITom DeFanti, Jason Leigh, Mark Ellisman, Phil Papadopoulos, Co-PIs
Maxine Brown, Project Manager
Knowing the User’s Bandwidth Requirements
DSL GigE LAN
C
A
B
A -> Need full Internet routing
B -> Need VPN services on/and full Internet routing
C -> Need very fat pipes, limited multiple Virtual Organizations
Source: Cees de Laat, UvA
Number of users
Bandwidth consumed
The OptIPuter is a Distributed “Infostructure” forData-Intensive Scientific Research and Collaboration
“A global economy designed to waste transistors, power, and silicon area – and conserve bandwidth above all – is breaking apart and reorganizing itself to waste bandwidth and conserve power, silicon area, and transistors.”— George Gilder, Telecosm (2000)
The OptIPuter: A Philosophy• Bandwidth is getting cheaper, faster than storage.• Storage is getting cheaper, faster than computing.• The exponentials are crossing – we are moving from a processor-
centric world, to one centered on optical bandwidth, where the networks will be faster than the computational resources they connect.
The OptIPuter: A Paradigm• Supercomputers maximize computing, and minimize bandwidth use.• The OptIPuter maximizes bandwidth use: A 32-node IA-64 Linux
cluster assumes 1GigE per node (32 Gbps I/O). When clusters are optically connected at 40 Gbps (for example), bandwidth is >100% the throughput!
A Useful and Usable Tool for Data-IntensiveApplication Drivers: BioScience and GeoScience
The OptIPuter project has two application drivers where scientists are generating multi-gigabytes of 3D volumetric data objects that reside on distributed archives that they want to correlate, analyze and visualize.
NIH Biomedical Informatics Research NetworkInitially a multi-scale brain imaging federated repository, but to be expanded to other organs of the body.
NSF EarthScopeThe acquisition, processing and scientific interpretation of satellite-derived remote sensing, near-real-time environmental, and active source data.
http://ncmir.ucsd.edu/gallery.html
siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml
A Radical New Architecture forScientific Cyber “Infostructure”
The OptIPuter is a “virtual” parallel computer in which the individual “processors” are widely distributed clusters; the “memory” is in the form of large distributed data repositories; “peripherals” are very-large scientific instruments, visualization displays and/or sensor arrays; and the “motherboard” uses standard IP delivered over multiple dedicated lambdas.
Think of the OptIPuter as a giant graphics card, connected to a giant disk system, via a system bus that happens to be an extremely high-speed optical network. One major design goal is to provide scientists with advanced interactive querying and visualization tools, to enable them to explore massive amounts of previously uncorrelated data in near real time.
Computing ● Data ● Visualization ● Networking
A 21st Century Amplified Work Environment: The Continuum
Passive stereo display AccessGrid Digital white board
Tiled display
Cluster Visualization
San Diego OptIPuter Design:LambdaGrid Enabled by Chiaro Router (Spring 2003)
switch switch
switchswitch
Medical Imaging and Microscopy
Chemistry, Engineering, Arts
San Diego Supercomputer Center
Scripps Institution of Oceanography
ChiaroEnstara
SL 6509
16-processor cluster
8-processor cluster
ONI DWDM
SL OPSW
NL 15454
16
8
2
8
8
8
2xOC-192
128x128 MEMSOptical Switch
OMNInet
810Gb
10Gb
Layer2/3 Switch/Router
EVL 6509
16-processor cluster
16
16
EVL OPSW
64x64 MEMSOptical Switch
16
ONI DWDM
OMNInet
8
8
Layer2/3 Switch/Router 16
UIC/EVL LAN
StarLightStarLight LAN
NetherLight/UvA
NL 15454
NL OPSWNL Force10Layer2/3 Switch/Router
8
2
32-processor cluster
32
32
Fiber Use in Pairs
GE ElectronicallySwitched
GE OpticallySwitched
64x64 MEMSOptical Switch
OptIPuter ConnectionsEVL/StarLight/NetherLight/UvA
Summer 2003 Plan
SL Force10
6x10GE24x1GE
N
2x10GE80x1GE
2
2x10GE48x1GE
What is a Lambda?
• A lambda, in networking, is a fully dedicated wavelength of light in an optical network, typically used today for 1-10Gbps.
• Lambdas are circuit-based technology, but can carry packet-based information.
• We are now mostly working with 1Gb dedicated layer2 circuits that act like lambdas
Switching Lambdas make the OptIPuter Possible
• Visualization getting very good on PCs• Networking getting very fast on PCs (10GigE)• Disk getting cheaper and faster• PCIX bus, 64-bit architectures available• Cost of bandwidth is plummeting• Optical switching is coming• We don’t have to scale to reach everybody in
their homes
Why Optical Switching?
• No need to look at every packet when transferring a terabyte of information– 1% the cost of routing– 10% the cost of switching– 64x64 10Gb:
• $100,000 O-O-O switched• $1,000,000 O-E-O switched• $10,000,000 O-E-O Routed
• Spend the savings on computing and collaboration systems instead!
• Replaces patch panels; allows rapid reconfiguration of 1 and 10Gb experiments
Large-Scale International Application Development
Guaranteed Latency
GuaranteedScheduling
Guaranteed Bandwidth
Access GridUSA, Canada, The Netherlands, UK, Italy, Germany, Russia, Australia, China, Korea, Thailand, Taiwan, Japan, Brazil
●
BABARUSA and internationally
●
The D0 ExperimentUSA, CERN, Germany, France, Japan and other worldwide collaborators
●
GiDVN: Global Internet Digital Video NetworkCCIRN DVWG worldwide membership
● ●
Hubble Space TelescopeUSA, France and others worldwide
●
SC GlobalUSA and International
●
Sloan Digital Sky Survey (SDSS)USA, France and worldwide
●
Virtual Room Videoconferencing System (VRVS)CERN, Switzerland; Caltech, USA; Others
●
vlbiGridUSA, The Netherlands, Finland, UK and worldwide
●
Large-Scale International Middleware and Toolkit Development
Guaranteed Latency
GuaranteedScheduling
Guaranteed Bandwidth
EU DataGridCERN, France, Italy, The Netherlands, UK, Czech Republic, Finland, Germany, Hungary, Spain, Sweden (in cooperation with US grid projects, notably GriPhyN, PPDG and iVGL
● ● ●
EU DataTAGUSA and Europe
●
Globally Interconnected Object Databases (GIOD)USA and CERN
●
GlobusUSA, Sweden, others internationally
● ● ●
MONARC for LHC ExperimentsCERN, Switzerland; Caltech, USA; Others
●
UK e-Science ProgrammeUK and USA
● ● ●
StarLight :Perhaps the World’s Largest 1GigE and 10GigE Exchange
Abbott Hall, Northwestern University’sChicago downtown campus
StarLight is an experimental optical infrastructure and proving ground for network services optimized for high-performance applications.
Operational since summer 2001, StarLight is a 1GigE and 10GigE switch/router facility for high-performance access to participating networks and is becoming a true optical switching facility for wavelengths.
StarLight’s Dutch and Canadian Partners
Kees Neggers, SURFnet Bill St. Arnaud, CA*net4
TransPAC/APAN
Euro-LinkSurfNET
Euro-LinkCERN
GÉANT
CA*net4
Abilene
Euro-Link Funded 10Gb LinkEuro-Link Co-Funded 10Gb LinkTransPAC Co-Funded 10Gb LinkPartner Matching 10Gb LinkGÉANT Matching 2.5Gb Links
Euro-Link/Trans-Light andTransPAC/Trans-Light SupplementFunded, Co-Funded and Matching Links
Hard Problems
• Internet is not designed for single large-scale users—TCP is not usable for long fat applications
• Circuits are not “scalable”• All intelligence has to be on the edge• Tuning compute, data, visualization,
networking using clusters to get order of magnitude improvement
• Security at 10Gb line speed
The OptIPuter Project will
• Investigate, procure and install optical and electronic switches.• Explore protocol stack development (for VLAN/optical management,
control and data planes). • Make progress in removing hierarchical protocol layers, moving toward
network transparency.• Optimize network-attached computational PC clusters for large-scale data
mining and visualization.• Facilitate measuring and monitoring 1-10Gb networks in experimental and
production configurations.• Compare strategies for grids moving large data over moderately
congested links vs. dedicated links.• Investigate the dynamic provisioning of light paths by high-performance
applications on optical networks.• Explore mechanisms to communicate and deliver Class of Service in
high-speed networks.• Debate the security advantages of switched lambdas
Questions?