1

Proposed future direction for CHEETAH

Outline What's our goal for the network:

eScience network or large-scale GP network?

Book-Ahead (BA) or Immediate-Request (IR)?

CHEETAH network evolution to support IR

Software modules that need to be implemented

Applications

CHEETAH/DRAGON/HOPI interconnectivity

Malathi VeeraraghavanUniversity of Virginia

August 9, 2006

2

eScience or GP? BA or IR?

BA IR

eScience networks

very large file transfers need high-BW and long holding time + remote viz. need to reserve timeslot for other resources

None?

general-purpose networks

circuit service to service providers - users are only administrators- router-to-router circuits

circuit/VC service for end users- host-to-host + router-to-router (E2E)- partial-path router-to-router circuits on congested links (called in by end user)

3

Reasons for choosing IR

Metcalfe's value statement SCALE, SCALE, SCALE

IR seems more natural in data world Bandwidth not as scarce as airline seats that only BA

can be supported; so IR seems like a must-have Cisco/Juniper/Sycamore control-plane

implementation + community's standardization effort is "wasted" to a large extent with BA

Shouldn't at least one testbed to study whether and, if so, how, IR bandwidth-management software built into switches can be useful

4

Co-management of bandwidthon IR and BA basis

But, BA and IR cannot coexist without bandwidth partitioning BA allows for high-BW, long-duration calls IR calls will suffer a high call blocking rate if

supported through BA scheduler

5

CHEETAH wide-area network

Raleigh PoP (MCNC)

Controlcard

GbE/10GbEcard

ORNL PoP

Controlcard

GbE/10GbEcard

SN16000

Controlcard

OC192card

OC-192

GbE/10GbEcard

End hosts

Atlanta PoP (SOX/SLR)

SN16000 SN16000

GaTech

End hosts

End hosts

ORNL

OC192card

NCSUOC192card

OC-192 (via NLR/SLR/NCREN)

via NCREN

UVa

CUNY Via Nysernet/HOPI Via Vortex/HOPI

6

What CHEETAH s/w enables now.

CHEETAHCHEETAHzelda2

Atlanta, GA

zelda4ORNL, TN

wukong

Releigh, NC

mvstu6Cville, VA

UGaUGa

UTUT

GatechGatech

dukeduke

NCSUNCSU

UNC

UNC

UVaUVa VTVT

GbE circuit

Once zelda2's GbE cardis LOCKED in a circuit headed to mvstu6,it cannot communicate with wukong or zelda4 via CHEETAH

X. FangR. Gisiger

7

Network evolution to support IR

Current CHEETAH network only supports 10 circuits per OC192 link IR mode inefficient if m, the link capacity in

channels, is small (i.e., 10) Recoup OC1-crossconnect capability of

the SN16Ks from its current 1Gbps use Has three advantages

supports higher m; better for IR GMPLS standards based signaling (GbE-SONET

signaling is Sycamore proprietary) Call setup delay: 166ms for two-hop instead of

1.5sec!

8

Network evolution options

Four options: NICs with VLAN support PLUS VLSR for SN16K

with control-plane support for VLANs 15454 with VLSR IP router with VLSR Ethernet switch with VLSR

9

NICs with VLAN support

Useful for web caching application since a cache may need to talk simultaneously to two other caches

In general useful for end-to-end circuits (whether router-to-router or host-to-host)

BUT, since the SN16K GMPLS software does not support VLAN mapping to multiples of OC1s, this solution is insufficient; hence the next slide

10

SN16K/VLSR at each PoP

Can write a VLSR for SN16K to support GMPLS control of VLAN-OCxx circuits and then use VLAN-capable NICs

Use hdb commands for flow-based Ethernet service, i.e., to map VLANs to OCxx circuits, but hdb software is unsupported

Beats the reason for our choice of SN16Ks (GA support of GMPLS control plane)

11

15454/VLSR at each PoP

Make the 15454 serve as the intermediary between Ethernet NICs in hosts and SONET based SN16Ks at cheetah PoPs

Useful for end-to-end circuits (whether router-to-router or host-to-host), e.g., webcaching.

Two problems: Needs a heterogeneous signaling procedure, Intserv

based Path message and VLAN IDs as labels to a SONET based Path message and SKULM labels

VLAN ID continuity requirement Cannot support partial-path circuits

12

IP router/VLSR at each PoP

Use OCxx SONET interfaces to connect IP router to SN16K

Connect web caches to router Have router initiate pure SONET circuit

setup Use PBR or just ordinary routing table

update to map flows to different OCxx circuits; support multiple circuits from one web cache

13

IP router/VLSR at each PoP

Can support end-to-end circuits web caching CDN servers router-to-router apps:

administrator initiated increases SNMP log monitor initiated increases

video apps at 10-15Mbps - map to one OC1 Has the potential to support PPCs (partial-

path circuits) Place router with VLSR in enterprises at edge

of GbE cheetah access link

14

Ethernet switch/VLSR at each PoP

Does not help with the problems noted in today's Gb/s circuit use of the SN16K long call setup delays: 1.5sec non-standard solution high per-circuit BW

Using an Ethernet switch/VLSR at an enterprise (e.g. CUNY) requires all VLANs sharing 1Gbps CHEETAH access link to be switched to the same exit SN16K.

Even worse, m=1 if whole 1Gb/s link used for a circuit

15

Software modules required

CVLSR for IP router Web caching & video apps CTCP code to support multiple

simultaneous flows

16

Applications

In order of preference: web caching video apps at 10-15Mbps - map to one OC1 router-to-router apps:

administrator initiated increases SNMP log monitor initiated increases

storage Contact: Paul Sheldon, Vanderbilt, has NSF MRI

to install data depots (600 TB) CDN servers

IEEE, ACM, ComSoc web sites

17

Equipment purchase IP routers with channelized SONET cards with GA

GMPLS UNI implementation at each CHEETAH PoP IP routers with GbE blades at enterprises IP router in NLR McLean (fourth CHEETAH PoP)

unless MAX/Dragon router is available - then buy GbE and OC192 blades for that router

If NC 454 transponders are unavailable, purchase transponders for DC-Raleigh NLR link - since HOPI doesn't have this

Colocation costs at NLR McLean and NLR Raleigh

18

DRAGON

How many WDM channels per link? m has to be small due to transponder costs makes it hard to support IR hence the interest in BA GMPLS in Movaz switches useful only for provisioning

but not BW sharing Heterogeneous circuit setup not advisable in IR

mode: should provision fat pipes based on aggregate usage

observation, not per small-BW VLAN Does each DRAGON PoP have an Ethernet

VLAN switch with VLSR?

19

Interest in using CHEETAH as part of the Southern leg for HOPI

HOPI is public-domain "commercial" Further HOPI uses VLAN switches -

small granularity! CAC even w/o enforcement is fine if CTCP is

the only conduit at end hosts - can enforce there

20

HOPI and web caching

Seems like a good match Rick's black cloud experiment - same as

web caching Also Rick wants the "hybrid" part to be

exercised - here's how with IP and VCs/circuits

21

What am I asking Rick?

Web caches at PoPs Permission to run router VLSR software

on Abilene routers that will need to update routing tables! Will of course demo this first thoroughly on

cheetah network

22

What am I asking Jerry?

CVLSR on HOPI switches - hop-by-hop IR mode

IR support - GMPLS peer model call setup Path message between router VLSRs (in

different domains) will carry final dest. IP address Router VLSR within cheetah (HOPI) domain finds

exit router for that domain and issues SONET circuit Path message with that exit router as destination

Web caches at DRAGON PoPs

23

Interconnect CHEETAH, DRAGON HOPI

Through IP routers of Cheetah & Abilene With our IP router/VLSR combo, setup

router-to-route SONET OC1 circuits via cheetah and router-to-router VLAN virtual circuits through DRAGON/HOPI. At routers, do PBR mapping for flows or just update routing tables for destination IP addresses (data-plane

This means packets go back to IP layer between two networks

24

Figure

HOPI + CHEETAH with routers in between

25

Web caching example

Example: web cache in CA to web-cache at ORNL VLAN from Sunnyvale Abilene router

through five HOPI nodes to McLean router McLean router to ORNL router through

SONET nodes