On Selfish Routing In On Selfish Routing In Internet-like Internet-like

EnvironmentsEnvironmentsLili Qiu Lili Qiu

Microsoft ResearchMicrosoft Research

March 18, 2004March 18, 2004

University of MarylandUniversity of Maryland

Today’s Internet RoutingToday’s Internet Routing

• Network in charge of routing

• Route selection affects user performance

• IP routing yields sub-optimal user performance

UMD

MSNBC

Deficiency in IP RoutingDeficiency in IP Routing

• IP routing is sub-optimal for user performance– Routing hierarchy – Policy routing– Equipment failure and transient instability– Slow reaction (if any) to network

congestion

Selfish RoutingSelfish Routing

• Selfish routing: users pick their own routes– Source routing (e.g., Nimrod)– Overlay routing (e.g., Detour, RON)

Source RoutingSource Routing

UMD

MSNBC

Overlay RoutingOverlay Routing

MSNBC

St. Louis

UMD

Salt Lake

Boston

Phoenix

Selfish RoutingSelfish Routing

• Selfish nature– End hosts or routing overlays greedily

select routes– Optimize their own performance goals– Not considering system-wide criteria

• Studies based on small scale deployment show it improves performance

• How well does selfish routing perform if everyone uses it?

Bad NewsBad News

• Selfish routing can seriously degrade performance [Roughgarden & Tardos]

S D

L0(x) = xn

L1(y) = 1

Total load: x + y = 1Mean latency: x L0(x) + y L1(y)

Worst-case ratio is unbounded - Selfish source routing

• All traffic through lower link Mean latency = 1

– Latency optimal routing• To minimize mean latency,

set x = [1/(n+1)] 1/n

Mean latency 0 as n

QuestionsQuestions

• Selfish source routing– How does selfish source routing perform?– Are Internet environments among the worst

cases?

• Selfish overlay routing– How does selfish overlay routing perform? – Does the reduced flexibility avoid the bad cases?

• Horizontal interactions– Does selfish traffic coexist well with other traffic?– Do selfish overlays coexist well with each other?

• Vertical interactions– Does selfish routing interact well with network

traffic engineering?

Our ApproachOur Approach• Game-theoretic approach with simulations

– Equilibrium behavior• Apply game theory to compute traffic equilibria• Compare with global optima and default IP routing

– Intra-domain environments• Compare against theoretical worst-case results• Realistic topologies, traffic demands, and latency

functions

• Disclaimers– Lots of simplifications & assumptions

• Necessary to limit the parameter space

– Raise more questions than what we answer• Lots of ongoing and future work

Routing SchemesRouting Schemes

• Routing on the physical network– Source routing– Latency optimal routing

• Routing on an overlay (less flexible!)– Overlay source routing– Overlay latency optimal routing

• Compliant (i.e. default) routing: OSPF– Hop count, i.e. unit weight– Optimized weights, i.e. [FRT02]– Random weights

Internet-like EnvironmentsInternet-like Environments

• Network topologies– Real tier-1 ISP, Rocketfuel, random power-law

graphs

• Logical overlay topology– Fully connected mesh (i.e. clique)

• Traffic demands– Real and synthetic traffic demands

• Link latency functions– Queuing: M/M/1, M/D/1, P/M/1, P/D/1, and BPR– Propagation: fiber length or geographical distance

• Performance metrics– User: Average latency– System: Max link utilization, network cost [FRT02]

Source Routing: Average LatencySource Routing: Average Latency

Good news: Internet-like environments are far from the worst cases for selfish source

routing

0.0E+00

5.0E+03

1.0E+04

1.5E+04

2.0E+04

2.5E+04

Ab

ove

ne

t

AT

T

EB

ON

E

Exo

du

s

Le

ve

l3

Sp

rin

t

Te

lstr

a

Tis

ca

li

Ve

rio

Po

we

rD2

Po

we

rD5

Po

we

rD1

0

Load scale factor=1

Avera

ge l

ate

ncy (

us)

selfish source routing

latency optimal routing

compliant routing (minimum network cost)

Bad NewsBad News

• Selfish routing can seriously degrade performance [Roughgarden & Tardos]

S D

L0(x) = xn

L1(y) = 1

Total load: x + y = 1Mean latency: x L0(x) + y L1(y)

Worst-case ratio is unbounded - Selfish source routing

• All traffic through lower link Mean latency = 1

– Latency optimal routing• To minimize mean latency,

set x = [1/(n+1)] 1/n

Mean latency 0 as n

1

10

100

1000

10000

Ab

ove

ne

t

AT

T

EB

ON

E

Exo

du

s

Le

ve

l3

Sp

rin

t

Te

lstr

a

Tis

ca

li

Ve

rio

Po

we

rD2

Po

we

rD5

Po

we

rD1

0

Load scale factor=1

Ne

two

rk c

os

t

selfish source routing

latency optimal routing

compliant routing (minimum network cost)

Source Routing: Network CostSource Routing: Network Cost

Bad news: Low latency comes at much higher network cost

Selfish Overlay RoutingSelfish Overlay Routing

• Similar results apply– Selfish overlay routing achieves close to

optimal average latency– Low latency comes at higher network cost

• The results apply when the overlay only covers a fraction of nodes– Scenarios tested:

• Random coverage: 20-100% nodes• Edge coverage: edge nodes only

Hop-count (load scale factor = 1)

7.6E+03

7.8E+03

8.0E+03

8.2E+03

8.4E+03

8.6E+03

8.8E+03

9.0E+03

bothcompl.

bothselfish

both latopt

selfish /compl.

selfish /latopt

latopt /compl.

Ave

rag

e la

ten

cy (

us)

overlay 1 overlay 2

Horizontal InteractionsHorizontal Interactions(Two Overlays)(Two Overlays)

Different routing schemes coexist well.

random weights (load scale factor = 1)

0.0E+00

8.0E+03

1.6E+04

2.4E+04

3.2E+04

4.0E+04

bothcompl.

bothselfish

both latopt selfish /compl.

selfish /latopt

latopt /compl.

Ave

rag

e la

ten

cy (

us)

overlay 1 overlay 2

Horizontal Interactions Horizontal Interactions (Two Overlays) (Cont.)(Two Overlays) (Cont.)

With bad weights, selfish overlay improves the performance of compliant traffic as well as its own.

Horizontal Interactions Horizontal Interactions (Many Overlays)(Many Overlays)

0100020003000400050006000700080009000

10000

0.2 0.7 1.2 1.7

Load scale factor

Ave

rag

e la

ten

cy (u

s)

one overlay per src per src-dest infinite

0

20

40

60

80

100

120

140

160

0.2 0.7 1.2 1.7

Load scale factor

Ma

xim

um

lin

k u

tiliz

ati

on

(%

)

one overlay per src per src-dest infinite

Performance degradation due to competition among overlays is

insignificant.

• An iterative process between two players– Traffic engineering: minimize network cost

• current traffic pattern new routing matrix

– Selfish overlays: minimize user latency • current routing matrix new traffic pattern

• Question: – Does system reach a state with both low

latency and low network cost?

• Short answer:– Depends on how much control the network

has

Vertical InteractionsVertical Interactions

OSPF optimizer interacts poorly with selfish overlays because it only has very coarse-grained

control.

0.0E+00

5.0E+03

1.0E+04

1.5E+04

2.0E+04

2.5E+04

0 10 20 30 40 50

Round

Ave

rag

e la

ten

cy (

us)

selfish alone TE alone

selfish + TE (OSPF)

0

20

40

60

80

100

120

140

160

180

200

0 10 20 30 40 50

RoundM

ax li

nk

uti

lizat

ion

(%

)

selfish alone TE alone

selfish + TE (OSPF)

Selfish Overlays vs. OSPF Selfish Overlays vs. OSPF OptimizerOptimizer

0

20

40

60

80

100

120

0 10 20 30 40 50

Round

Max

lin

k u

tili

zati

on

(%

)

selfish alone TE alone selfish + TE (MPLS)

0.0E+00

2.0E+03

4.0E+03

6.0E+03

8.0E+03

1.0E+04

1.2E+04

1.4E+04

0 10 20 30 40 50

Round

Av

era

ge

late

nc

y (

us

)

selfish alone TE alone selfish + TE (MPLS)

Selfish Overlays vs. MPLS Selfish Overlays vs. MPLS OptimizerOptimizer

MPLS optimizer interacts with selfish overlays much more

effectively.

SummarySummary• Contributions

– Important questions on selfish routing – Simulations that partially answer questions

• Main findings on selfish routing– Near-optimal latency in Internet-like environments

• In sharp contrast with the theoretical worst cases

– Coexists well with other overlays & regular IP traffic• Background traffic may even benefit in some cases

– Big interactions with network traffic engineering • Tension between optimizing user latency vs. network load

Other WorkOther Work

• Internet– Fault diagnosis– Web performance– Congestion control– IP telephony

• Wireless networks– Model the impact of wireless interference– Provision wireless networks– Manage wireless networks

Fault DiagnosisFault Diagnosis

• Server-based Inference of Internet Performance. IEEE INFOCOM 2003.(Joint work with V. N. Padmanabhan and H. J. Wang)

MotivationMotivation

C&W

AT&T

WebServer

Sprint

UUNet

Qwest Earthlink

AOL

It’s so slow!

Diagnosisengine

Ethernet

Why is itso slow?

Network DiagnosisNetwork Diagnosis

Diagnosisengine

Tcpdump traces

Network topology

Troublespots location

Diagnosis results:Qwest access link: 63.232.180.230->63.232.33.134Peering between Sprint and AOL: 64.45.216.154->172.139.89.74

Problem FormulationProblem Formulation

l1

l8l7l6

l2

l4 l5

l3

server

clientsp1 p2 p3 p4 p5

(1-l1)*(1-l2)*(1-l4) = (1-p1)

(1-l1)*(1-l2)*(1-l5) = (1-p2)…(1-l1)*(1-l3)*(1-l8) = (1-p5)

Challenges:• Under-constrained system

of equations• Measurement errors

Gibbs SamplingGibbs Sampling

• D– observed packet transmission and loss at the clients

– ensemble of loss rates of links in the network

• Goal– determine the posterior distribution P(|D)

• Approach– Use Markov Chain Monte Carlo with Gibbs sampling

to obtain samples from P(|D)– Draw conclusions based on the samples

Gibbs Sampling (Cont.)Gibbs Sampling (Cont.)

• Applying Gibbs sampling to fault diagnosis– 1) Initialize link loss rates arbitrarily– 2) For j = 1 : warmup

for each link i compute P(li|D, {li’}) where li is loss rate of link i, and {li’} = kI lk

– 3) For j = 1 : realSamples for each link i

compute P(li|D, {li’})– Use all the samples obtained at step 3 to

approximate P(|D)

Summary of Internet Fault DiagnosisSummary of Internet Fault Diagnosis

• Gibbs sampling yields a high coverage (over 80%), and a low false positive rate (below 5-10%)

• Two other inference techniques trade-off accuracy for speed

Model the Impact of Model the Impact of Wireless InterferenceWireless Interference

• Impact of Interference on Multihop Wireless Network Performance. ACM MOBICOM 2003. (Joint work with K. Jain, J. Padhye, and V. N. Padmanabhan)

MotivationMotivation• Multihop wireless networks

– Community networks, sensor networks, military applications

• Important to compute wireless network capacity– Capacity planning – Evaluate the efficiency of protocols

• A lot of research on capacity of multi-hop wireless networks

• Much of previous work studies asymptotic performance bounds– Gupta and Kumar 2000: O(1/sqrt(N))

• We present a framework to answer questions about capacity of specific topologies with specific traffic patterns

Community Networking ScenarioCommunity Networking Scenario

Asymptotic analysis is not useful in this case

What is the maximum possible throughput?

ChallengeChallenge

• Model the impact of wireless interference

1 Mbps 1 Mbps

1 Mbps 1 Mbps

Throughput = 1 Mbps

Throughput = 0.5 Mbps

A B

BA

C

C

Overview of Our FrameworkOverview of Our Framework1. Model the problem as a standard network

flow problem2. Represent interference among wireless

links using a conflict graph3. Derive constraints on utilization of

wireless links using independent sets in the conflict graph • Augment the linear program to obtain lower bound

on optimal throughput

4. Derive constraints on utilization of wireless links using cliques in the conflict graph• Augment the linear program to obtain upper bound

on optimal throughput

Sample Results Using Our FrameworkSample Results Using Our Framework

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 20 40 60 80 100 120

Iterations

Nor

mal

ized

Thr

ough

put

Upper Bound

Lower Bound

Houses talk to immediate neighbors, all links are capacity 1, 802.11-like MAC, Multipath routing

Sample Results Using Our Sample Results Using Our Framework (Cont.)Framework (Cont.)

Scenario Aggregate Throughput

Baseline 0.5

Double range

0.5

Two ITAPs 1

Two Radios 1

Houses talk to immediate neighbors, all links are capacity 1, 802.11-like MAC, Multipath routing

Future WorkFuture Work

• Trends– Networks become larger and more

heterogeneous

• Research problems– Internet management

• End-user based approach

– Wireless network design & management• Error-prone physical medium• Dynamic and unpredictable networks• Accessible physical medium, vulnerable to

attacks

Future Work (Cont.)Future Work (Cont.)

• Trends– Network protocols become more

complicated, e.g., various optimizations are proposed for different network layers

– Network users and providers have different and sometimes conflicting goals

• Research problems– How to optimize network performance?

• Cross different network layers• Satisfy the need of different users and network

providers

Thank you!Thank you!

Computing Traffic Equilibrium Computing Traffic Equilibrium of Selfish Routingof Selfish Routing

• Computing traffic equilibrium of source routing traffic– Use the linear approximation algorithm

• A variant of the Frank-Wolfe algorithm, which is a gradient-based line search algorithm

• Computing traffic equilibrium of overlay routing– Construct a logical overlay network– Use Jacob's relaxation algorithm on top of Sheffi's

diagonalization method for asymmetric logical networks– Use modified linear approximation algo. in symmetric case

• Computing traffic equilibrium of multiple overlays– Use a relaxation framework

• Each overlay computes its best response by fixing the other overlays’ traffic

• Merge the best response and the previous state using decreasing relaxation factors.

Overlay Routing in Inter-Overlay Routing in Inter-domaindomain

0

0.05

0.1

0.15

0.2

0.25

0.3

0 0.2 0.4 0.6 0.8 1 1.2

Load scale factor

Avera

ge la

tency (s)

selfish cooperative compliant

Selfish routing yields close to optimal latency, and better than compliant routing.

Advantages of Our ApproachAdvantages of Our Approach• “Real” numbers instead of asymptotic bounds

• Optimal bound, may not be achieved in practice

• Useful for “what if” analysis • Permits several generalizations:

• Different routing• single path or multi-path routing

• Different wireless interference models• Different antennas/radios

• directional or unidirectional, different ranges, data rates, multiple radios/channels

• Different senders• senders with limited (but constant) demand

• Different topologies• Different performance metrics

• throughput, fairness, revenue

0

2000

4000

6000

8000

10000

12000

0 0.5 1 1.5 2

Load scale factor

Av

era

ge

late

nc

y (

us

)

source routing overlay-src: opt-weight

overlay-src: hop-count overlay-src: rand-weight

0

20

40

60

80

100

120

140

160

180

200

0 0.5 1 1.5 2

Load scale factorM

ax li

nk

util

izat

ion

source routing overlay-src: opt-weight

overlay-src: hop-count overlay-src: rand-weight

Selfish Overlay Routing: Selfish Overlay Routing:

Full Overlay CoverageFull Overlay Coverage

Overlay source routing perform similarly as source routing (except for very bad weight settings)

7600

7800

8000

8200

8400

8600

8800

9000

9200

9400

0 0.5 1 1.5 2 2.5

Load scale factor

Ave

rag

e la

ten

cy (

us)

all partial

0

20

40

60

80

100

120

140

160

0 0.5 1 1.5 2 2.5

Load scale factor

Max

lin

k u

tiliz

atio

n (

%)

all partial

Selfish Overlay Routing:Selfish Overlay Routing: Partial Overlay Coverage (only edge Partial Overlay Coverage (only edge

nodes)nodes)

The effects of partial overlay coverage is insignificant in backbone topologies.

Example: Conflict GraphExample: Conflict Graph

Connectivity Graph

A B C

1

4

2

3

1 2

3 4

Conflict Graph