Internet2 UpdateR/D and Infrastructure

Guy AlmesInternet2 Project

<almes@internet2.edu>

NANOG MeetingDearborn — 9 June 1998

Outline of the Talk

Technical Working GroupsThe Challenge of Delay-

Bandwidth ProductsAbilene Project Update

Applications and Engineering

Applications

Engineering

Motivate Enables

Internet2 Working Groups - Presentation to JET

Comments on Apps and Plumbing

Advanced applications transform high-speed plumbing into value

Advanced plumbing enables advanced applications

Profligate use of bandwidth, per se, does not make an application ‘advanced’

Megalomaniac plumbing, per se, does not make the plumbing ‘advanced’

Technical Working Groups

IPv6 Measurement Multicast Network

Management Network

Storage

Quality of Service

Routing Security Topology

IPv6

Chair: Dale Finkelson, Univ Nebraska <dmf@unl.edu>

Membership: Total 12; 9 .edu, 3 .com, 1 .gov

Focus: Explore the rôle that IPv6 might play in the

Internet2 project Work with those interested in IPv6 to build

IPv6 testbeds across the Internet2 structure, including vBNS and Abilene

Measurement Chair: David Wasley, Univ California

<david.wasley@ucop.edu> Focus:

Places to measure: at campuses, at gigaPoPs, within interconnect(s)

Things to measure traffic utilization performance: delay and packet loss traffic characterization

One example measurement technology

IETF IPPM WG defining one-way delay

Take all delay to be due to: Propagation Transmission Queuing

Variation in delay suggests congestion

Multicast

Chair: vacant [Dave Meyer, Univ Oregon still serving] Nearing completion of naming a

successor Membership: Total 3; 3 .edu Focus: Make native IP multicast

scalable and operationally effective

Network Management

Chair: Mark Johnson, MCNC <mj@ncren.net>

Membership: Total 4; 3 .edu, 1 .com Focus:

Common trouble ticket system How can all our interconnects and

gigaPoPs and universities appear to be a seamless whole?

Network Storage

Chair: Micah Beck, Univ Tennessee <mbeck@utk.edu>

Membership: Total 13; 9 .edu, 4 .com Focus: Distributed Storage

Infrastructure for Internet2 Replication Physical proximity Transparency

Quality of ServiceChair: Ben Teitelbaum, Internet2

staff <ben@internet2.edu>Membership: Total 36; 17 .edu,

19 .comFocus: Multi-network IP based QoS

Relevant to advanced applications Interoperability: carriers and kit Scalable Administratable and Measurable Hosts, campus/gigaPoP/Interconnect

routers/switches

Quality of Service Sketch

• Does the approach support advanced applications?

• Are there implementations that work? Only one?

• If cloud ‘A’ and cloud ‘B’ both implement QoS, does the combined A+B catenation implement QoS?

A B

QoS, continued

Results to date: Requirements document Series of technical recommendations

First Internet2 Joint Applications/ Engineering QoS WorkshopSanta Clara, CaliforniaMay 21-22, 1998Hosted by Bay Networks

Routing

Chair: Steve Corbato, Univ Washington <corbato@cac.washington.edu>

Membership: Total 48; 32 .edu, 16 .com

Focus: Internal and External routing Critical issues

gigaPoP internal routing design Explicit routing requirement (the “fish problem”) Met at UCSD in January (21 attendees)

gigaPoP external routing recommendations Subscribers (Internet2 campuses) National interconnects (vBNS, Abilene, and NGI networks)

Security

Chair: Peter Berger, Carniege Mellon Univ <peterb@hoopoe.psc.edu>

Membership: Total 13; 13 .edu Focus:

Authentication Application to QoS Application to Digital Libraries

Topology

Chair: Paul Love, Internet2 staff <epl@internet2.edu>

Membership: Total 16; 13 .edu, 2 .com, 1 .gov

Focus: Topology of Internet2 Internal Internet2 Connections Internet2 with other Advanced

Research Networks

Summary

Internet2’s WGs focus on project’s needs

Complement IETF WGsMembership by invitation -

welcome participation by Internet2 corporate members

Large Delay-Bandwidth Products

As the product of delay and bandwidth grows: The number of unacknowledged packets grows It becomes more difficult to sustain a steady

stream of data from end to endSeveral consequences:

Need for direct physical paths Tradeoff between buffering and

variation in delay

A pessimistic result from Mathis et al.

Mathis, Semke, Mahdavi, and Ott, "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm", Computer Communication Review, July 1997.

www.psc.edu/networking/papers/model_abstract.html

BW C * packet-size / (delay * packet-loss)

Consider the implications for the international

high-performance Internet

BW packet-sizeBW 1 / delayBW 1 / packet-loss

Example: Delay

BW C / delay

delay due to distance

original raw bandwidth

Example: Delay with fatter pipe

BW C / delay

delay due to distance

more raw bandwidth

Example: Packet Loss

similar phenomenon, but … to double bandwidth, you mustcut packet loss by four

Abilene Update

UCAID Project Addresses infrastructure needs of

Internet2

Goals and Objectives

Provide high-quality, widely available Interconnect among participating gigaPoPs/universities

Connect to Internet2 members via the vBNS and to other key research/ education sites via Internet2/NGI-class federal and non-US nets

Goals and Objectives, continued

Support QoS architecture as it evolves

Support other advanced functionality as it evolves

Maximize Robustness Minimize Latency Provide Capacity to Avoid

Congestion

Evolution of Abilene with Time

Phase 1: use of operational Qwest Sonet Phase 2: use of separate wavelengths Phase 3: use of separate fibers

Allows capacity to grow with Internet2 needs

Key Attributes

IP over Sonet Benefit from Qwest OC-48 Sonet

capacity and collocation sites Benefit from Nortel OC-192 Sonet kit and

Lucent fiber Benefit from Cisco GSR 12000 routers

Architecture: Core

About 11 (up to 30) core nodes Each located at a Qwest PoP Each with a Cisco 12008 router Rack also contains measurements/ management computers

Interior lines connect core nodes OC-12 and (eventually) OC-48 Sonet IP-over-Sonet interfaces

sttl

eugn

scrm

anhm

phnx

albq

hstn nwor tlhs

atln

rlgh

wash

phil

nycm

bstnsyrc

clev

pitb

dtrtmilw chcg

mpls

kscydnvrslkc

ipls

lsvl

nsvlrcmd

Subset of Route Map of Interest to Abilene

tpka

alby

elpa

Attitude toward interior lines

Robustness: mesh plus Sonet

Latency: direct physical paths

Capacity: avoid congestion

Architecture: Access

Access node at many Qwest PoPs Qwest Sonet switches needed equipment

Access lines connect from core node to gigaPoP Local part: gigaPoP to access node Long distance part: access node to core node IP-over-Sonet or IP-over-ATM possible OC-3 and OC-12 typical

One cost-sharing implication

Long-distance part of access line is considered part of the ‘backbone’

Thus, number/location of core nodes does not affect costs borne by gigaPoP

One robustness implication

Each access line is Sonet Long-distance part (at least) will be

configured from protected Sonet ring

Thus, single access line can tolerate a break in the long-distance part of the access line

OK, so where’s the map?

Self-selection is key Each gigaPoP will determine where,

when, at what speed it connects

Detailed topology will be based on engineering considerations