ca*net 4 optical networking and third wave of the internet
DESCRIPTION
CA*net 4 Optical Networking and Third Wave of the Internet. Bill St. Arnaud CANARIE Inc – www.canarie.ca [email protected]. Overview. 1. CA*net 4 Network Architecture 2. Applications that may drive require a new network architecture. CA*net 4 Drivers-1 Reduce cost of Internet. - PowerPoint PPT PresentationTRANSCRIPT
CA*net 4 Optical Networking and Third Wave
of the Internet
Bill St. Arnaud
CANARIE Inc – www.canarie.ca
Overview
> 1. CA*net 4 Network Architecture> 2. Applications that may drive require a new
network architecture
CA*net 4 Drivers-1Reduce cost of Internet
> Set up lightpaths to no cost peering exchanges– Most lambda sales in Canada and USA are for “Remote peering” to
no cost peering points– Allows for considerable savings in Internet transit costs– Each lightpath is directly connected to a high volume peer and
bypasses peering router– Good example is “STAR LIGHT” where high volume peers have
direct connect and small volume peers use a router– CA*net 4 “customer controlled patch panel” allows peers to change
peering relationship remotely without contacting technical staff at peering exchange• Very similar in concept to WorldCom “Peermaker” at MAEs which use
e-mail to setup peers
CA*net 4 Drivers-2Reduce cost of routers
> Eliminate expensive high end routers and replace them with lower cost optical switches
– But circuits are NOT intended to replace packet networks– Use rich mesh of circuits between edge routers to eliminate high
cost of 10GbE core routers• 10Gbe routers ~ $500K with interfaces at ~$200k each
• 10Gbe switches ~$25K with interfaces at ~$20k each
– Trade off between cost of multiple lightpaths and loss of multiplexing versus cost of high end core routers
– 10Gbe wavelengths ~$1000/km for 5 years (lifetime of router)– Assume 1 GbE lightpaths per edge institution then
• One 5000km Gbe lightpath (or 8 x 600km GbE) lightpaths per institution is cheaper than routers
CA*net 4 Drivers-3A VPN alternative to MPLS
> Allows customer to create “customer owned and managed” networks with resource heterogeneity– Integration of wavelengths and dark fiber from different carriers– Create customer controlled VPNs for downstream users and
overlay networks across multiple suppliers– Customers can manage their own restoral and protection schemes– Allows for inter-domain end to end setup of VPNs– End users do not need to to signal carrier for VPN management
• Create VPNs
• Cross connect VPNs from independent users
• Partition or spawn VPNs
• Establish VPNs across multiple management domains
CA*net 4 Drivers-4Application Specific Networks
> Lambda Grids - “Underlay” networks to support Grids and overlay projects like PlanetLab and Oceanstore– A lot of exciting research into overlay networks – At some point in time when traffic volume is sufficient in overlay network
to setup its own direct path> Soon high end grid applications will have sufficient traffic volume to
require their own underlay networks ”Complementing” routed networks – Not a replacement for routed networks – only increasing the direct
peering mesh of the routed network for specific applications– But peering may be more dynamic (and not globally advertised) than
traditional IP BGP peering> Discipline or applications specific networks
– VBLI grids like European EVN – High energy physics grid – Ultralight– NEES grid, Bio-informatics Grid, etc
Example – EVN traffic flows over GEANT
UK
SE
FR
NLBE
DE1DE2
CZ
PL
CH
IT
AT
SURFnet
JANET
GARR
PSNC
DFN
NORDUnet
2.5G10G
JIVE
Provided courtesy of Dai Davies
Issues
> How do you charge for bandwidth and usage when single application traffic dwarfs all other IP traffic?
> Who pays for the traffic volume when it sinks into one NREN?> Possible solutions:
1. GMPLS (with QoS)• Requires expensive routers and complex coordinated central
management to setup and tear down tunnels
• Does not address issue of traffic charging
• Interdomain still unproven
2. Optical overlay/underlay –ASON – same problems as GMPLS
3. Application specific optical BGP networks– Increase BGP mesh for specific applications or disciplines
OBGP applied to EVN
UK
SE
FR
NLBE
DE1DE2
CZ
PL
CH
IT
AT
SURFnet
JANET
GARR
PSNC
DFN
NORDUnet
JIVE
EVN sites will see 2 BGP routes to SURFnet:-the normal IP route over GEANT-Express route using dedicated lightpaths (in green)
Express route
CA*net 4 Drivers-5QoS
> Spatial QoS – TCP throughput over long fat pipes very susceptible to packet
loss, MTU, TCP kernel, Buffer memory, AQM optimized for commodity Internet, Auto negotiating Ethernet, etc
– May also require consistent and similar TCP throughput for multiple sites to maintain coherency for grids and SANs
– Some exciting new TCP protocols like FAST, XCP, etc• Mice and Elephant problem• Without careful design may look like a DOS attack on a router network
– Many commercial SAN/Grid products will only work with QoS network
– Some users want to have super jumbo MTU (64K) or protocols other than IP
Spatial QoS
Application or end user controls peering of BGP optical paths for transfer of elephants!!!
Normal BGP pathx.x.x.1 y.y.y.1
OBGP pathOnly y.y.y.1 advertised to x.x.x.1 via OBGP path
Only x.x.x.1 advertised to y.y.y.1 via OBGP path
Optical “Peermaker”
CA*net 4 Drivers-6Extend E2E principle to circuits
> Extend the Internet end to end principle to circuit based networks– The success of the Internet is largely attributable to the classic e2e principle– Allowed development of exciting new applications or services
> E2E principle presumes all users behave honorably– Otherwise you get spam and DDoS attacks– This ungentlemanly behavior can be reasonably contained on commercial
Internet but pose big challenge on research networks> What happens if you have a malicious implementation of FAST or XCP on a
big pipe network?> Can the Internet e2e principle be applied to circuit based networks?
– Will it engender the same creativity in new applications and services?> MPLS and ASON are classic network state based solutions for VPNs
– CA*net 4 architecture is an alternate approach– All VPNs are BGP direct static routes using lightpaths that are setup and
controlled by end user
CA*net 4 is NOT a network
> It is an aggregation of point to point 10 Gbps wavelengths from a number of carriers
> CA*net 4 is made up of may parallel application or discipline specific networks that may (or may not) BGP peer with each other
> The wavelengths and switches are partitioned into smaller lightpaths with user control of the switch partition which are used for a variety of applications particularly grids– International Grid Testbed – 10 Gbe server to server to CERN– WESTgrid – 1 Gbe lightpaths for distributed backplane– Numerous lightpaths to support direct peering between regional networks
and universities– Lightpaths to support TransLight projects between North America, Europe
and Asia– Many, many more coming – Virtual Astronomy, HDTV video walls, etc
Applications for E2E Lightpaths
The three waves
– The first wave of the Internet consisted primarily of text and data services such as e-mail and FTP.
– The second wave was the web which improved ease of use and facilitated the transfer of images, sound and video.
– The third wave is the integration of applications, p2p networking, open source, distributed computing enabled by next generation web services, semantic web and high speed networks
What is the Third Wave?
> Before the Web on-line information was only available through a small number of information providers who charged high fees
– Compuserve, Dialogic, etc> The Web allowed millions of creators of information to make it
easily accessible to all others at very low cost, bypassing the information middleman
> The Third Wave proposes to extend the WEB paradigm to processes, applications and content
> Third Wave is about creation of tools and applications (i.e. services) in variety of fields such as eLearning, eBusiness, eScience, eHealth, etc that can make these services easily available to all others
– At there are millions of web sites, there will be millions of Third wave services
Today’s Network
Application
OS
Data
Application
OS
Data
Network
User User
The application is tightly bound to the OS
The network is a mechanism for applications to communicate with each other
The network is subservient to the computer
Third Wave Network
Network
Application and Data
Third Wave
OS
Data
Third Wave
OS
Data
Third Wave
OS
Data
Third Wave
OS
Data
Third Wave
OS
Third Wave
OS
Application and data exist on the network and are uncoupled from any specific machine or location
The computer is subservient to the network
A new way of doing science
> Science used to about test tubes, wet labs and big instruments
> But increasingly science is moving to networks and computers
> Science is now longer bound by bricks and mortar or geography
> NSF has announced “Cyber Infrastructure” initiative
– https://worktools.si.umich.edu/workspaces/datkins/001.nsf
> DOE SciDAC “Scientific Discovery through Advanced Computing”
– http://www.er.doe.gov/feature_articles_2001/august/SCIAC/SciDAC_announcement.htm
> Recognition that more and more science is network and computationally based
> Grids using web services will be foundation of this new research methodology
Many e-Research Projects Coming
ATLAS
Sloan Digital Sky Survey
LHC
ALMA
International Grid Testbed
Tier2 Centre ~1 TIPS
Online System
Offline Processor Farm
~20 TIPS
CERN Computer Centre
FermiLab ~4 TIPSFrance Regional Centre
Italy Regional Centre
Germany Regional Centre
InstituteInstituteInstituteInstitute ~0.25TIPS
Physicist workstations
~100 MBytes/sec
~100 MBytes/sec
~622 Mbits/sec
~1 MBytes/sec
Physicists work on analysis “channels”.
Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server
Physics data cache
~PBytes/sec
~622 Mbits/sec per channel
Tier2 Centre ~1 TIPS
Tier2 Centre ~1 TIPS
Tier2 Centre ~1 TIPS
Caltech ~1 TIPS
~622 Mbits/sec per channel
Tier 0Tier 0
Tier 1Tier 1
Tier 2Tier 2
Tier 4Tier 4
1 TIPS is approximately 25,000
SpecInt95 equivalents
ATLAS
International Grid Testbed
> First production use of international e2e lightpaths to transfer Forward Calorimeter data from CERN to Canada
> Experiments planned to directly transfer low level trigger data from CERN to Canadian computers
> 10 GbE server to server experiments> RDMA, TCP/IP offload, etc
Virtual Observatory
> http://www.us-vo.org/ > Discovery process will rely
on advanced visualization and data mining tools
> Not tied to a single brick and mortar location
> Will cross correlate existing multi-spectral databases petabytes in size
> Web services will integrate data and applications
No new telescopes or radio dishes. Just big networks interconnecting large databases
Canada Virtual Observatory Data Flows
> Flow to Terapix : 1TB/month> Flow to science centres: 0.5 TB/month> Return from Terapix: 2 TB/month> Over 5 years users will download full dataset (100
TB) ~ 30 times: 50 TB per month> Average flow 154 Mbps for five years> User network load is the largest, least predictable,
and least manageable component of the network traffic
> Accessible to students at schools
Canadian Forestry Grid
> SAFORAH (System of Agents for Forest Observation Research with Automation Hierarchies).
> SAFORAH connects five locations across the country to support the monitoring of Canada's forests
> Together, all five locations will generate data equaling 40 terabytes (TB) per month
Grids for Kids
> The ultimate goal of Grids for Kids is to allow students and eventually members of the general public to be full participants in scientific discovery and innovation.
> Will allow increasing number of computationally or networked research experiments to be seamlessly integrated with the computer capabilities of thousands of PCs located at our schools
> Some early primitive examples…
FightAIDS@Home
•Scientists at The Scripps Research Institute (TSRI) are using computational methods to identify drugs that have the right shape and interaction characteristics to fight diseases such as AIDS.
•Once such candidates are identified, they can be synthesized in a laboratory, tested according to FDA guidelines, and released as prescription drugs to benefit the public.
•Such computations require a vast number of trial dockings, testing variations in the target protein and the trial drug molecules
Folding@home
> This "virtual supercomputer" uses peer-to-peer technology to make unprecedented amounts of processing power available to medical researchers to accelerate the development of improved treatments and drugs that could potentially cure diseases.
> Rapid new discoveries in cancer research
> Two projects in Canada:– Smallpox cure at UWO– Leukemia research at Mt Sinai
Climate Prediction
> Predict future climate due to greenhouse affect> Distribute climate model to thousands of PCs worldwide> www.climateprediction.com
ALTA Cosmic Ray eScience
> Collaborative scientific research project involving the University of Alberta Center for Subatomic Research and over 50 high schools across Canada in the area of cosmic ray detection.
> Teachers and students actively contribute to the physics research while learning about an exciting area of modern science.
> Distributed computing at schools required to analyze data from sensors in near real time
Neptune/Venus Grid
> Joint US-Canadian project for undersea dark fiber network off west coast of USA and Canada
> Undersea network will connect instrumentation devices, robotic submarines, sensors, under sea cameras, etc
> All devices available to students and researchers connected to CA*net 4 and Internet 2 networks
Distributed computing and data storage devices on CA*net 4 and Internet 2 will be used to analyze and store data
Faulkes Telescope
> Provide UK schools with access to a research class telescope in Hawaii
> Provides an exciting resource for teachers to use via the Web
> To provide a real-time experience of astronomy, through live use of a telescope
> To allow students to participate in real research programs, mentored by professional astronomers
> Provides other public interest groups, such as amateurs, access to high quality astronomical data
> http://www.faulkes-telescope.com/
More Information
> http://www.canarie.ca/canet4/library/canet4design.html
> http://www.canarie.ca/canet4/obgp/index.html> http://www.canarie.ca/canet4/library/customer.html
> Thanks to the design teams at Carleton U, Ottawa U, CRC, UQAM, UoWaterloo, Montague, etc
Background Slides
RMI
Resource Management Layer
User Access Layer
Grid Application
Create service Access service
Web Server
DB
GT3 Hosting Environment
LPO FactoryService
Grid Service Interface
LPO Delegate Service J2EE Application Server
LPO Service
EJB Remote
LPO Service
EJB Home
JDBC
LPO Service
Implemen-tation
RMI
RMI
LPO Grid Service
User Controlled Service Provisioning Layer
Using Third Wave Service to manage a network
OGSI-conformant services LPO advertisement LPO query LPO termination LPO access LPO reconfiguration LPO spawning LPO concatenation End-to-end LPO establishment
OGSI-conformant services
Resource Agent
RMIService Provisioning Layer
LPO ControllerLPO Controller
RMI
TL1
Request Controller
Switch Interface