Internet2Infrastructure Update

Rick Summerhill

Associate Director, Backbone Network Infrastructure, Internet2

CANARIE and GEANT Meeting

Ann Arbor

18 December 2003

04/08/23 2

Infrastructure Projects

1. Develop/provide advanced production infrastructure• Abilene Network• MAN LAN (Manhattan Landing)• Abilene Observatory• Engineering (IPv6/multicast, measurement, transport,

demos)

2. Enable/explore next generation optical networking• Internet’s role in NLR• FiberCo• HOPI (Hybrid Optical Packet Infrastructure – Chris will

discuss this project)

Advanced Production Infrastructure

04/08/23 4

Abilene Network – Upgrade Completed!

04/08/23 5

Abilene router node

Power

Out-of-band

Ethernet Switch

T640 router

M5 router

Power (48V DC)

Measurement Servers

SPACE!

Observatory Rack

04/08/23 6

Abilene Financial Model - 2004

Original 2003 2004OC-3c

(155 Mbps)$110k (1998)

($110k) ($110k)

OC-12c

(622 Mpbs)$320k (1998)

$270k $240k

Gig E

(1 Gbps)$325k (2001)

$325k $280k

OC-48c

(2.5 Gbps)$495k

(2000)

$430k $360k

10 Gbps

(SONET/ Ethernet)

$490k (2003)

$490k $480k

04/08/23 7

Abilene ParticipantsDecember 2003

IP-over-DWDM (OC-192c) and IP-over-SONET OC-48c Backbone47 Connectors (OC-3c 10 GigE), may drop to 39 over the next year

• 2 10 GigE connections, expected to increase next year• 6 OC-48c, probably will remain the same• 2 GigE connection, expected to increase next year• 14 OC-12c, probably will remain the same next year• 23 OC-3c, expected to decrease next year

224 participants – research universities & labs• All 50 states, District of Columbia, & Puerto Rico• Aggregation on the rise!

Expanded access• 92 sponsored participants, expected to increase• 32 State networks, expected to increase

04/08/23 8

SC2003 Bandwidth Challenge

04/08/23 9

MAN LAN Exchange Point

Internet2/NYSERnet/Indiana U partnership

Ethernet Switch• Cisco 6513 (with new fabric/backplane soon)• Located in 32 Avenue of the Americas

– same building as Abilene New York City router node

• CA*net (Canada) bringing in OC-192 to same building

Transatlantic (Abilene/SURFnet) connection• OC-192c from Amsterdam• Enabled by IEEAF and Tyco

GLIF (Global Lambda Integration Facility)• Installing Cisco 15454 switch – more on this later

04/08/23 10

Abilene Observatory

A program to provide enhanced support of computer science research over Abilene

• Create network data archive– Consists of many separate databases on a variety of servers– Forms a large correlated database– Create tools to access the database – Support from/for network researchers

• Provide data views for a real-time snapshot of Abilene• Provide server collocation for direct network measurement

and experimentation–Resources reserved for additional servers

• Power (DC), rack space (2RU), router uplink ports (GigE)–Initial deployment is PlanetLab overlay network–Additional requests from SDSC AMP project and

research team from Japan

04/08/23 11

Advanced Engineering

MPLS Experiments – Teragrid examplesLarge MTUsSecurity and the REN-ISACAdvanced Restoration TechniquesFAST TCP Experimentation

04/08/23 12

Advanced Engineering - Example

FAST TCP is an advanced, delay-based TCP flavor being developed at Caltech

• Internet2 is working with Caltech, SoX, PNW, NC-ITEC, and PSC to test FAST TCP over a production network - Abilene

• Results include:  - Congesting 2.5Gb/s OC-48 link going into SoX with FAST TCP traffic with no adverse effects on conventional traffic

– no packet loss during 1-hour test with a delay increase of only 5 ms

• Evidence that FAST TCP plays nicely with production traffic• Concurrently, a useful network performance test tool was

developed - i2perf 

Next Generation Optical Networking

04/08/23 14

One underlying hypothesis: the rise of the RONs

The fundamental nature of regional networking is changing

The GigaPoP model based on provisioned, high-capacity services steadily is being replaced – on the metro and regional scales

A model of facility-based networking built with owned assets – Regional Optical Networks (RONs) – has emerged

Notably, the importance of regional networks in the traditional three-level hierarchy of U.S. advanced networking for R&E is not diminished

04/08/23 15

RONs will influence Infrastructure Efforts

Abilene Network

FiberCo

Internet2’s role in NLR

HOPI

Quilt (Fiber project)

04/08/23 16

FiberCo

Designed to support optical initiatives• Regional• National

Not an operational entity – Will not light any fiberFiber options

• Holding company for any future initiatives• Assignment vehicle

– Regional initiatives– National initiatives (e.g., NLR)

FiberCo progress: assignments underway and anticipated• NLR • Wyoming, Indiana, and Michigan• Pursuing Texas, upstate New York, and Florida

Expect to cover Internet2 investment ($2.7M) in 2004

04/08/23 17

Hybrid Optical/Packet Infrastructure (HOPI)

Need to converge the two worlds of packet (Abilene) and optical circuit (GLIF) networking

• Concern about IP packet network scaling (lack of >10 Gbps transport) and drawbacks (security, large data flows)

• Widely recognized that the current GLIF optical networking model does not scale beyond a limited number of sites

• However, dedicated capabilities allow a vehicle for advanced testing (transport technologies beyond TCP) and very high-end requirements

• Potential end-states:– Hybrid model – with centrally and perhaps ‘user’ driven

allocation of dedicated capabilities

– Impact of hybrid experience on packet network design

04/08/23 18

HOPI – Chris Heermann

Abilene high-performance IP network• Capabilities for MPLS tunnels

10-Gbps over full NLR footprint• Expect some type of ‘TDM’ infrastructure to be provisioned

by Internet2 in collaboration with NLR– Additional capital investment required

Tie points for international collaboration• MAN LAN (NYC) and IEEAF (NYC-Amsterdam)• StarLight collaboration (Chicago)• Expect similar capabilities to emerge in Seattle (Pacific

Wave)

04/08/23 19

Internet2MAN LAN 15454

04/08/23 20

Infrastructure Prioritiesfor 2004

Drive process to develop HOPI architecture and begin testbed efforts

Accelerate RON paradigm/deployment• Quilt Fiber project• FiberCo

Expand Abilene Observatory to support a broader set of network researchers

Expand Abilene services (security & performance) and encourage connector upgrades

Play a critical role in NLR development• Governance and implementation

04/08/23 21