The Delta Routing ProjectLow-loss Routing for Hybrid Private Networks

George Porter (UCB)

Minwen Ji, Ph.D. (SRC - HP Labs)

Outline

• Motivation/overview of corporate networks

• Problem Statement

• Architecture– Two layers: Physical and Overlay– The Delta Protocol– The Delta+TM Protocol

• Evaluation

• Conclusions

Corporate Network ConstructionNetwork Layer

• Distributed Locations connected by leased lines due to:

– Need for predictable performance

– Security

– Management and control

• Fixed initial cost, incremental additional cost due to traffic volume

• Not necessarily overprovisioned

• Reprovisioning on the timescale of days (or weeks)

• Expensive (compared to ISP connectivity

SFNYC

LON

LA

DC

DFW

ALX

SEA

HOU

OSPF

Corporate Network ConstructionOverlay Layer

• ISP Connectivity alreay at selected nodes to provide:

– Web/Email access– VPN access to at-home or

distance workers– Business services

• Per-byte, ISP much cheaper than “Intranet”

– But no QoS

• Intranet corporate network with ISP links is called a ‘Hybrid Private Network’

SFNYC

LON

LA

DC

DFW

ALX

SEA

HOU

ISP Connectivity

Problem of Congestion

• Flash traffic (video, backup, data transfer) or steady corporate growth can lead to periodic congestion

• Problem Statement:– Reduce congestion and

packet loss on the Intranet by utilizing ISP connectivity while providing good end-to-end performance

SFNYC

LON

LA

DC

DFW

ALX

SEA

HOU

Overlay / VPNTunnels

IntranetTopology

1

2

3

4

5

1

2

3

4

5

Architecture

Architecture• Overlay Layer:

- Need to forward traffic around congested portions of the Intranet

- Measurement-based path construction

- Intermediate point may be better than “last hop” selection

- Metric include measured latency and local queuing delay

- Paths are selected on order of seconds or minutes

• Physical (Intranet) Layer:

- Single-domain routing protocol (OSPF)

- Dijkstra

- Forwarding decision: which packets go to Intranet and which go to the preselected overlay paths? (per packet)

Physical Forwarding Algorithm

• Ji, Minwen. Dial-controlled Hash: Reducing Path Oscillation in Multipath Networks. Proceedings of the International Conference on Computer Communications and Networks (ICCCN). Oct 2003.

• Current Algorithm:– Prefer physical path, but if physical queue full

send to overlay layer.

Overlay Path Selection Algorithms

• Static– Lasthop– Nexthop– Random

• Dynamic– Delta

• Minimize end-to-end delay

– Delta+TM• Predict and avoid congestion by inferring global traffic matrix

Delta Path Selection

• Find path to minimize the sum of:– Local Queue delay + WAN delay + Intranet delay

• Key feature is the use of locally obtained information

Limitation of Delta Algorithm

• Since Delta uses local information, it might send traffic to an overloaded link:

congested

• Can we avoid this?

Delta+TM (Traffic Matrix)

• Key idea:– Don’t choose paths that will subject the traffic to

congestion

• Use the original Delta algorithm (minimize end-to-end delay) but throw out paths that will subject packets to congestion

• But how do we find out about remote congestion?– Given that message flooding will likely be inaccurate

and might make the problem worse

Traffic Matrix Estimation

1 2 3 4

1 0.00 4.54 1.92 1.14

2 1.04 0.00 1.57 1.78

3 2.52 3.04 0.00 4.08

4 3.64 4.30 4.13 0.00

+ TopologyInformation =

.3

.6 1.2

.03

1.3

.89• Each node measures flows

that transit through it• Long-term averages are

flooded to fill in the entries of the table that a node can’t directly measure

Evaluation

1) Simple Example2) Algorithm-antagonistic Topologies3) Large-scale Topology (PlanetLab-

based)

Linear Topology

Congestion Event

Congestion Event

Evaluation

1) Simple Example2) Algorithm-antagonistic Topologies3) Large-scale Topology (PlanetLab-

based)

Algorithm-antagonistic Topology

• Simple topology with traffic flows that should expose a weakness to each topology

Algorithm-antagonistic Topology

Evaluation

1) Simple Example2) Algorithm-antagonistic Topologies3) Large-scale Topology (PlanetLab-

based)

Planetlab as VPN-network source

• Large, distributed testbed• We modelled the Overlay

part of a fictional 43-node corporate network using traces taken over planetlab

• The Intranet link topology was obtained from 2-level clustering and eyeballing

• Traffic flows include a “measured flow” and a set of background and disruptive flows

PlanetLab (UCLA->ac.uk)

ppnonly 4,438

nexthop 2,492

lasthop 3,469

DeltaTM 3,523

Delta 4,443

Packet Losses

Conclusions

• Utilizing ISP connectivity enables balancing packet loss rate –vs- end-to-end delays

• Dynamic algorithms can adapt to a variety of wide-area conditions

• Congestion prediction can help in certain environments, however local-only decision making works well

• Certain “choke points” must be identified so that synchronization effects will not occur

• Making better use of bandwidth can lower cost of deploying distributed corporate networks