Projects Related to Coronet

Jennifer RexfordPrinceton University

http://www.cs.princeton.edu/~jrex

Outline• SEATTLE

– Scalable Ethernet architecture

• Router grafting (joint work with Kobus)– Seamless re-homing of links to BGP neighbors– Applications of grafting for traffic engineering

• Static multipath routing (Martin’s AT&T project)– Joint traffic engineering and fault tolerance

2

SEATTLE

3

Scalable Ethernet Architecture for Large Enterprises(joint work with Changhoon Kim and Matt Caesar)

http://www.cs.princeton.edu/~jrex/papers/seattle08.pdf

Goal: Network as One Big LAN• Shortest-path routing on flat addresses

–Shortest paths: scalability and performance–MAC addresses: self-configuration and mobility

• Scalability without hierarchical addressing–Limit dissemination and storage of host info–Sending packets on slightly longer paths

4

SH

S

S

S S

S S S S

S S S

S

H

H

H

H

HH

H

H

SEATTLE Design Decisions

5

Objective Approach Solution

1. Avoiding flooding

Never broadcast unicast traffic Network-layer

one-hop DHT2. Restraining

broadcastingBootstrap hosts

via unicast

3. Reducing routing state

Populate host infoonly when and where

it is needed

Traffic-driven resolution with caching

4. Shortest-path forwarding

Allow switches to learn topology

L2 link-state routingmaintaining only

switch-level topology

* Meanwhile, avoid modifying end hosts

Network-Layer One-hop DHT• Maintains <key, value> pairs with function F

– Consistent hash mapping a key to a switch–F is defined over the set of live switches

• One-hop DHT– Link-state routing ensures

switches know each other

• Benefits– Fast and efficient

reaction to changes– Reliability and capacity

naturally grow with size of the network

6

0 12128-1

Location Resolution

7

SwitchesEnd hostsControl messageData traffic

<key, val> = <MAC addr, location>

Host discovery

B

x

HashF(MACx) = B

Store<MACx, A>

Traffic to x

HashF(MACx ) = BTunnel

to A

Notify<MACx, A>

E

Forward directly from D to A

ATunnel to B

C

D

yOwner

User

Resolver

Publish<MACx, A>

Address Resolution

8

<key, val> = <IP addr, MAC addr>

Traffic following ARP takes a shortest pathwithout separate location resolution

B

DHash

F(IPx) = B

Store<IPx, MACx, A>

BroadcastARP requestfor IPx

HashF(IPx ) = B

Unicast reply<IPx, MACx, A>

E

AUnicastlook-up to B

C

<IPx ,MACx>

x y

Handling Network and Host Dynamics

•Network events–Switch failure/recovery

Change in <key, value> for DHT neighbor Fortunately, switch failures are not common

–Link failure/recovery Link-state routing finds new shortest paths

•Host events–Host location, MAC address, or IP address –Must update stale host-information entries

9

Handling Host Information Changes

10

Resolver

y

Host talkingwith x

< x, A >

< x, A >

< x, A >

D< x, D >

Oldlocation

New location

< x, D >

< x, D >

< x, D >

Dealing with host mobility

MAC- or IP-address change can be handled similarly

B

xA

C

E

F

Packet-Level Simulations• Large-scale packet-level simulation

–Event-driven simulation of control plane–Synthetic traffic based on LBNL traces –Campus, data center, and ISP topologies

• Main results–Much less routing state than Ethernet–Only slightly more stretch than IP routing–Low overhead for handling host mobility

11

Prototype Implementation

12

Host-info registrationand notification msgs

User/Kernel Click

XORP

OSPFDaemon

RingManager

Host InfoManager

SeattleSwitch

Link-stateadvertisements

Data Frames

Data Frames

RoutingTable

NetworkMap

ClickInterface

Throughput: 800 Mbps for 512B packets, or 1400 Mbps for 896B packets

Conclusions on SEATTLE• SEATTLE

–Self-configuring, scalable, efficient

• Enabling design decisions–One-hop DHT with link-state routing–Reactive location resolution and caching–Shortest-path forwarding

• Relevance to Coronet–Backbone as one big virtual LAN–Using Ethernet addressing

13

Router Grafting

14

Joint work with Eric Keller, Kobus van der Merwe, and Michael Schapira

http://www.cs.princeton.edu/~jrex/papers/nsdi10.pdfhttp://www.cs.princeton.edu/~jrex/papers/temigration.pdf

Today: Change is Disruptive• Planned change

–Maintenance on a link, card, or router–Re-homing customer to enable new features–Traffic engineering by changing the traffic matrix

• Several minutes of disruption–Remove link and reconfigure old router–Connect link to the new router–Establish BGP session and exchange routes

15

customerprovider

16

Router Grafting: Seamless Migration• IP: signal new path in underlying transport network• TCP: transfer TCP state, and keep IP address• BGP: copy BGP state, repeat decision process

Send state

Move link

17

Prototype Implementation• Added grafting into Quagga

– Import/export routes, new ‘inactive’ state– Routing data and decision process well separated

• Graft daemon to control process• SockMi for TCP migration

ModifiedQuagga

graftdaemon

Linux kernel 2.6.19.7

SockMi.ko

Graftable Router

HandlerComm

Linux kernel 2.6.19.7-click

click.ko

Emulatedlink migration

Quagga

Unmod.Router

Linux kernel 2.6.19.7

18

Grafting for Traffic EngineeringRather than tweaking the routing protocols…* Rehome customer to change traffic matrix

19

• Internet2 topology, and traffic data• Developed algorithms to determine links to graft

Traffic Engineering Evaluation

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.60

100000

200000

300000

400000

500000

600000

700000

800000

900000

1000000

Original Topology (optimal paths)

With Grafting

Demand Multiple

Tot

al L

ink

Usa

ge Network can handle more traffic(at same level of congestion)

Conclusions• Grafting for seamless change

–Make maintenance and upgrades seamless–Enable new management applications (e.g., TE)

• Implementing grafting–Modest modifications to the router –Leveraging programmable transport networks

• Relevance to Coronet–Flexible edge-router connectivity–Without disrupting neighboring ISPs

20

Joint Failure Recoveryand Traffic Engineering

21

Joint work with Martin Suchara, Dahai Xu, Bob Doverspike, and David Johnson

http://www.cs.princeton.edu/~jrex/papers/stamult10.pdf

Simple Network Architecture• Precomputed multipath routing

– Offline computation based on underlying topology– Multiple paths between each pair of routers

• Path-level failure detection– Edge router only learns which path(s) have failed– E.g., using end-to-end probes, like BFD– No need for network-wide flooding

• Local adaptation to path failures– Ingress router rebalances load over remaining paths– Based on pre-installed weights

22

Architecture

23

• topology design• list of shared risks• traffic demands

t

s

• fixed paths• splitting ratios

0.25

0.25

0.5

Architecture

2424

t

slink cut

path probing

• fixed paths• splitting ratios

0.5

0.5

0

State-Dependent Splitting• Custom splitting ratios

–Weights for each combination of path failures

25

0.40.4

0.2

Failure Splitting Ratios

- 0.4, 0.4, 0.2

p2 0.6, 0, 0.4

… …

configuration:

0.6

0.4

p1

p2

p3

at most 2#paths entries

Optimizing Paths and Weights• Optimization algorithms

– Computing multiple paths per pair of routers– Computing splitting ratios for each failure scenario

• Performance evaluation– On AT&T topology, traffic, and shared-risk data– Performance competitive with optimal solution– Using around 4-8 paths per pair of routers

• Benefits– Joint failure recovery and traffic engineering– Very simple network elements (nearly zero code)– Part of gradual move away from dynamic layer 3

26