Pacific Wave, Pacific Light Rail,And

National Light Rail

Jacqueline BrownUniversity of Washington, Seattle

CANS2002, Shanghai, 22 August 2002

Pacific Wave is a service of the Pacific Northwest Gigapop.

– High speed peering point for regional and

international networks

– Complement of Abilene and CA*net3 International Transit Network services

– Open peering point for participant-managed bilateral peerings

Pacific Wave

Pacific Wave Emphasis

Features of Pacific Wave• Close to major Pacific cable landing sites

• Located in incumbent telecommunications carrier grade hotel– easy access to collocation services– easy access to fiber-meet-me-room– easy access to full U.S West Coast fiber grid

• Gigabit Ethernet is high speed, low cost, low maintenance– No PNWGP staff involvement required to setup your

peerings; no ATM PVC’s or mandated routing policy

• Switches implement PIM-SM snooping for increased multicast efficiency

• 24 x 7 Network Operations Center

• Redundant environmentals (DC, HVAC)

• PNWGP Commodity Internet Services– Engineered and delivered via four diverse tier-

one vendors– Vendors chosen after exhaustive review of

capacity, support, peering, etc.– Diverse geographic connectivity for commodity

services– 1.45Gbps throughput (under expansion)

Features of Pacific Wave (cont.)

N AT I O N A L L I G H T R A I L

• AARNet – Australian Academic & Research Network

• Abilene Network/Internet2• ATT Broadband Internet• CA*net3/CANARIE• DREN – Defense Research and Engineering Network

• ESNet – Energy Sciences Network

• Microsoft Corporation• Pacific Northwest Gigapop• Siemens Medical – Pointshare

• TANET2 – Taiwan Research Network

• TransPAC – Asia Pacific Academic Networks (APAN)

Networks at Pacific Wave: Spring 2002

RESEARCH = Network research testbeds

EXPERIMENTAL = Experimental network

infrastructure

OPERATIONAL = 24/7 operational high performance networks

Chicago 5/DEC/01 Grand Challenges in e-ScienceGrand Challenges in e-Science

Note: Aubrey Bush’s slide

Common Denominator, Advanced Services Research & Education Network

Bleeding-edge e2e services & facilities for key experiments, programs, projects, researchers & for next gen. tech., architectures, Grids, content, apparatus, etc. ‘Things that haven’t been done before; with, to, on or over networks’.

Leading-Edge ip Services for demanding apps . & middleware

Advanced Services for R&E users

Business,.Gov.& General PublicCommodity Internet

I2-Abilene, GigaPoPs; Fed. ‘NGI’ Nets. Internet-2,GigaPoPs I2-K20

ISP’s, hosting.com’s etc.

Operational High Performance Research Support Networks

CS, Comp. Science & ‘Grid’ researchers,+ major experiments (eg DTF) & net. experimenters + leading edge labs and centers, optics sys.& net. Industry R&D groups + I-WIREPACIFIC LIGHTRAIL +Internet-2 wave projects;

Research

’s,P2p fiber,

& >=10g ip …

Experimental &Developmental

Networks

LEADERS NETWORK TYPE CAPABILITIES/USERS

R&E Network Tiers NETWORK DISCONTINUITIES

12/05/01

Computer Science & Net. Researchers Academic Research, papers, lab. Experiments, spools of fiber In labs & sparse testbeds

N A T I O N A L L I G H T R A I L

An Extraordinary Opportunity for a Research &

Education Community Owned & Lit,

Fiber-Based, National-Scale

Experimental & Research, & Production ‘.edu’

Network Infrastructure

Revised 8-15-02 rj

Optical Fiber Options: Creative Approaches to

Extraordinary (but perishable) Opportunities

(to make Bandwidth a Resource instead of a Constraint!)

Research andDevelopment

Commercialization

Partnerships

Privatization

Today’s Internet

‘NGI’ efforts

Source: Ivan Moura Campos

Research Nets

Keep the Experimental Networks tier from sliding down the network development ‘Spiral’

And, Long Term

A Real Partnership & Project:

N A T I O N A L L I G H T R A I L

• Partners = CENIC & Pacific Wave (aka the ‘Pacific LightRail partners) + Distributed Terascale Facility (DTF-ETF) & I-Wire folks + UCAID + NCAR + a few others

• Rooted in a lightweight, but coordinated, and opportunistic collaboration of the NLR partners to acquire, provision and ‘operate’ optical networking assets

• Now seeking to establish national scale:– owned & lit fiber infrastructure cost-effectively providing many waves –

bulletproof dedicated, waves for separate services, projects, experiments– optical infrastructure substrate for e-science projects (proposing to a

diverse array of funding agencies)– provide appropriate hooks and support for advanced network

measurement and academic research– provide waves for various other R&E metro, regional and national

‘operational’ network etc services– A viable long term option (but full-costed 5 yr amortization for start)

– ‘AUP free’!– MetaPoPs & strategic pop’s

Optical Network Landscape

• Temporarily open windows of opportunities wrt:– Optronics kit manufacturers (some will do LAN PHY for us). – Fiber Owners Long haul & Metro – great deals available now – PoPs Co-Lo facilities – available & affordable– [But Not Wave Vendors (Instead we are seeing recent rising

prices/lessened competition, and unwillingness to: generalize occasional good pricing; to create believable/bulletproof long term price and availability protections; to provide techs like LAN PHY )]

• Existing metro, regional (&linked!) & now inter-regional/national efforts to deploy cost-effective, reasonably enduring, i.e. owned & lit fiber based multi R&E infrastructure:– Leveraging state & regional K-20 + higher ed transport activities– Leveraging ‘production’ networks (on dedicated waves) – Research & Experimental network capabilities – To/among major research sites including NSF, DoE, and university

facilities and end users

Optical network project differentiation/evolution

Distance scale (km)

U.S.Examples

Equipment

Metro(many cases)

< 60

UW-Pacific N/W Gigapop (Seattle),

USC/ISI (LA)

Dark fiber & end terminals

State/Regional(some linked)

60-500

I-WIRE (IL),CENIC ONI,I-LIGHT (IN)

Add Amps & distant ops

ExtendedRegional/National

> 500

Pacific LightRail,

TeraGrid DTF,‘Next Gen Abilene’

Add OEORegens/adm’s &

> O&M $$’s

Derived from Steve Corbato, Internet2 Feb. ‘02

P A C I F I C L I G H T R A I L

R

SEA

PDX

SAC

SNY

LA

SD

NM

DEN CHI

Abilene?

P/NWGP

S UW

CENIC ONI Tier 1 & 2

R

Peerings?

R

R

EMV

SB

ANH

SLO or

R

R

R

P/NWGP-FairbanksU. Alaska ARSC etc

R

R

ADDRESSING NETWORK DISCONTINUITIES

LONG LINES: • Lit fiber plus waves • Options for more @ discount• Could buy fiber (nationally) METRO: • Owned Fiber Rings/Segs. in SEA, SF BAY, LA, SD & SAC

PROVISIONABLE DEDICATED p2p WAVES

MetaPoPs!

ESTABLISHING KEY PIECESFOR FLEXIBLE LONG TERM LOW COST PROVISIONING!

draft 12/4/01

StanfordSLAC

SDSC

UCSD

UCSF

UCB & LBLPeerings

UCI

UCD

UCLAISIUSC

CaltechUCSB

Peerings

OHSU/OGI?

I-WIRE !!!LANL??

NCAR?!

R

R

R

NY

DC

R

10 gig waves

Leading-Edge e2e services & experimentalnetwork facilities via MetaPoPs and inter-gigapop links for research & next gen. tech., arch., grids, DTF expansion, content, sensors, apparatus …Up to 40 dedicated 10 gigabit waves with up to hundreds of dedicated gigabit links system-wide.

8/14/02

10 gbsTycom IEEAF

donation

10 gbsTycomIEEAF

Donation

MetaPoPs & Core Nodes

ADM sites

Desired Expansion PoPs/ADM

National LightRail” (NLR)

International Broadband

Metro/WAN owned Fiber ‘Rings’connecting strategic R&E endpoints.

PIT

BOS

NY

DC

LA

SUN

SEA

DEN

CHI

National LightRail: cost-effective owned lit R&E fiber fabric

ATL

CLV?

Canariefabric

Raleigh?

Initial footprint 3/03

NLR 10 gig Lambdas (startup NLR plus some CENIC & DTF-

ETF waves on same glass)

Seattle

Denver

San Diego

Tustin

Los Angeles

Santa Barbara

SunnyvaleOakland

Sacramento

Davis

Chicago

Pittsburgh

San LuisObispo

Solidad

ViaMetro

HPRTeragridHPR & DC

CalREN DCNLR

D.C.

Portland Optical Amps are configured for 40 10 gigabit waves, which sub-dividable into hundreds of one gigabit dedicated paths plus 10 gigs 4 10 gigabit waves at startup with one subdivided into 8 one gigabit paths. Inexpensive addition of waves though-out or locally Easily geographically extensible

8/14/02

• At least 4 express 10 gig waves on each DWDM span (in some cases there are as many as 8)

• 8 wdm systems/spans in the initial footprint

• Capacity up to 20 express waves and 20 add/drop waves per span

• 1 wave terminated in NLR router for a common shared IP service (with the exception of SAN)

• 1 wave terminated on NLR switches

• 1 10GE client facing port on routers

• 8 1GE client facing ports on switches for allocatable experimental 1 gig dedicated/private net services

• Client router/switch ports and extra waves connect directly at colo and/or across metro WDM – uses of other 10 gig waves under active discussion

Initial Configuration:

Lit National NLR Core Fabric Fiber

5,207 miles

11,000 miles

A.

B.

A. Starting footprint - operational by March 2003.

B. Multi-Loop core for owned fiber R&E net - June 2004?

7-28-02 ronj

Dashed lines for regional not costed

NLR Achieves: • National & Extensible Core system of Lit Fiber with Dedicated Waves

and/or bandwidth for Experimental Production and Research networks,– Initially providing no fewer than 4 Ten Gigabit Waves Nationwide; – One of which will be subdivided into dedicated 1 gigabit

circuits/facilities;– Optical Amplifiers are configured from start with 40 10gps waves;– Very low cost for bringing up each additional 10 gigabit wave (each

can be done point to point, e2e, or system-wide), or subdividing a wave;

– Max. capacity of 40 separate 10 gigabit waves (400 gb/s total), each of which is easily subdividable into at least 8 1 gigabit dedicated circuits (or > 300 total if system-wide, or many more if point to point in smaller geographies).

• Interoperates with, leverages and further enables DTF-ETF & other research & experimental network Endeavors

• Easy and Inexpensive Extension to the other critical mass geographies

• Establishes an enduring and highly flexible platform for enabling network based e-Science and for enabling NSF and other agency research & experimental net. activities.

• MetaPoP & strategic physical PoP infrastructure!

Thank you!

Jbrown@cac.washington.edu

CANS2002, Shanghai, 22 August 2002