internet congestion control ... -...

Internet Congestion Control:Motivations

Steven LowCaltech

netlab.CALTECH.edu

Outline

Brief history of Internet congestion control Who/why cares about protocolsHow much can be improvedTech transfer

Lee Center enables impact

1969

Control milestones

1988

Tahoe

ARPANet

1974

TCP

Flow control: Prevent overwhelming receiver

Congestion control – Gen 1:Prevent overwhelming network

2006

NewTCP’s

CC – Gen 2:Serve new applications

theory based

1969

Control milestones

1988

Tahoe

ARPANet

1974

TCP



2006

NewTCP’s


By applicationgrowth

Driven by infrastructuregrowh

1969

ARPANet

Backbone milestones

1988 2006

50-56kbps, ARPANet

Backbone speed:

T1 NSFNet

1991

OC12MCI

T3, NSFNet

1996 1999

OC48vBNS

2003

OC192Abilene

HTTP

Tahoe

Napster

Network is exploding

Internet at birth (1969)

Source: http://www.computerhistory.org/exhibits/internet_history/full_size_images/1969_4-node_map.gif

December, DEC

1 November, IBM

1 October, SDS

2 September, SDS

Routers

Internet at 31

Internet growth

1969

Control milestones

1988 20061983

ARPANetcut-over to

TCP/IP

Tahoe

ARPANet

1974

TCP


TCP (1974)

Window control: ack-clocking, timeout, retransmission

For correct delivery of packet sequence over unreliable networksTo prevent overwhelming receiver (through Advertised Window in TCP header)

“We envision [HOST retransmission capability] will occasionally beinvoked to allow HOST accommodation to infrequent overdemandsfor limited buffer resources, and otherwise not used much.”

-- Cerf and Kahn, 1974

Congestion collapse

October 1986, Internet had its first congestion collapseLink LBL to UC Berkeley

400 yards, 3 hops, 32 Kbpsthroughput dropped to 40 bpsfactor of ~1000 drop!

1988, Van Jacobson proposed TCP congestioncontrol

throughput

load

Why the 1986 collapse

5,089 hosts on Internet (Nov 1986)Backbone speed: 50 – 56 kbpsControl mechanism focused only on receiver congestion, not network congestion

Large number of hosts sharing a slow (and small) network

Network became the bottleneck, as opposed to receiversBut TCP flow control only prevents overwhelming receivers

Jacobson introduced feedback control to deal with network congestion in 1988

Tahoe and its variants (1988)

Jacobson, Sigcomm 1988+ Avoid overwhelming network+ Window control mechanisms

Dynamically adjust sender window based on congestion (as well as receiver window)Loss-based AIMD

“… important considering that TCP spans a range from 800 MbpsCray channels to 1200 bps packet radio links”

-- Jacobson, 1988

1969

Control milestones

1988

Tahoe

ARPANet

1974

TCP



2006

NewTCP’s

1969 1972

Application milestones

1988

Tahoe

1971

ARPANet

NetworkMail

1973

FileTransfer

Telnet

Simple applications

1993 20051995

InternetPhone

Whitehouseonline

Internet Talk

Radio

Diverse & demanding applications

1990

Napstermusic

2004

AT&TVoIP

iTunesvideo

YouTube

Network Mail (1971)First Internet (ARPANet) application

The first network email was sent by Ray Tomlinson between these two computers at BBN that are connected by the ARPANet.

Internet applications (2006)

Telephony Music TV & home theatre

Software As A Service

Finding your way

Mail

Games

Library at your finger tip Network centric warfare

1969

Control milestones

1988

Tahoe

ARPANet

1974

TCP



2006

NewTCP’s


Summary

First 25 years of Internet (1969 – 94)Networks are several orders of magnitude smaller in size, speed, and heterogeneityApplications are much simpler

Last 14 years of Internet (1994 – 08)Networks are much bigger, faster, and heterogeneousApplications are much more diverse and demanding

Our control mechanisms have shown remarkable robustness, but also essentiallyfrozen in the last 20 years

Outline



Protocol problems

Example problem & SolutionProtocol

Link & Physical

Network

Transport

Application P: Chattiness, slowS: Application acceleration

P: TCP congestion controlS: TCP acceleration

P: Path failure & congestionS: Overlay routing optimization

FastSoft, Inc. - Confidential

Trend 1

Exploding need to deliver large contentVideo, software, games, business info


Trend 1: Online content is exploding

CAGR 2005 – 2011: 36%Google worldwide: 46PB/monthLibrary of Congress: 0.136PB


Trend 1: Video is exploding

Jan 2007


Trend 1: Multi-year growth

Only ~10% of digital content is currently online

Jan 2007


Trend 2


Infrastructure growth to support delivery need

FastSoft, Inc. - ConfidentialSource: Deutsche Bank, Jan 2007

Trend 2: Broadband penetrationGlobal broadband penetration is accelerating

11 million new subscribers/month globally83% of US home Internet access is broadband by June 2007


Trend 3


Infrastructure growth to support delivery needCentralize IT to reduce costs of management, space, power, cooling

Exacerbated by virtualization of infrastructure and personalization of content


Trend 3: CentralizationPower & cooling cost is escalating, fueling centralization

CAGR (2005-10): power & cooling 11.2%, new server spend 2.7%In 2005, 1,000 servers cost $3.8M to power & cool in 4yrs; 2% increase in electricity cost raises cost by $200K

Source: IDC Sept 2006

Worldwide Expense(US: $4.5B in 2006; EPA)


Trend 3: Centralization


Protocols becoming bottleneck


Infrastructure growth to support delivery needCentralize IT to reduce costs of management, space, power, cooling

Exacerbated by virtualization of infrastructure and personalization of content

ImplicationMore large contents over longer distanceServed from centralized data centers

http://sramanamitra.com/2007/10/21/speeding-up-the-internet-algorithms-guru-and-akamai-founder-tom-leighton-part-5/

“You could only get that sustained rate if you are delivering within100 miles, due to the way current Internet protocols work.”

Tom Leighton, MIT/Akamai, Oct 2007


Solution Approaches

Protocol problems degrade performance of long-distance transfers

Current approach: reduce distance

FastSoft approach: fix protocol problems


Outline

Brief history of Internet congestion control Who/why cares about protocolsHow much can be improved

EquilibriumDynamicsHeterogeneous protocols

Tech transfer


Duality model of TCP/AQM

TCP/AQM

Equilibrium (x*,p*) primal-dual optimal:

F determines utility function UG guarantees complementary slacknessp* are Lagrange multipliers

) ,( ) ,(

***

***

RxpGpxpRFx T

=

=

cRxxU iix≤∑

≥ subject to )( max

0


Duality model of TCP/AQM

Equilibrium (x*,p*) primal-dual optimal:

cRxxU iix≤∑

≥ subject to )( max

0

Mo & Walrand 2000:

⎪⎩

⎪⎨⎧

≠−

== −− 1 if )1(

1 if log)( 11 αα

αα

i

iii x

xxU

α = 1 : Vegas, FAST, STCP α = 1.2: HSTCPα = 2 : Renoα = : XCP (single link only) ∞


Some implications

EquilibriumAlways exists, unique if R is full rankBandwidth allocation independent of AQM or arrivalCan predict macroscopic behavior of large scale networks

Counter-intuitive throughput behaviorFair allocation is not always inefficient Increasing link capacities do not always raise aggregate throughput


⎟⎠

⎞⎜⎝

⎛−= ∑

ii ctx

cp )(1&

Traditional: integrator model

Aggregate FAST rate

)()()(

tpdtwtx

i

fii

i +−

=τ

Basic assumption


10 10.5 11 11.5 12

8

8.2

8.4

8.6

8.8

9x 10

−3

Time [sec]

Queu

e size

[sec

]

NS−2Static modelIntegrator modelJoint model

Lags true link dynamics

Traditional: integrator model


⎟⎠

⎞⎜⎝

⎛−= ∑

ii ctx

cp )(1&

New: integrate micro effects

aggregate FAST rate

))(()()(

ttwdssxtt

tτ

τ+=∫

+

Can recover all previous link models

New model


New model tracks dynamics

19.8 20 20.2 20.4 20.6 20.8 21 21.2 21.4 21.6 21.8

0.01

0.015

0.02

0.025

0.03

0.035

Time [sec]

Qu

eu

e s

ize

[se

c]

TestbedStatic link modelJoint link modelIntegrator link modelProposed model


Some implications

Interaction of paced and un-paced flows

FAST stability depends on heterogeneity of RTT’s, not their particular values

Important in applications where dynamics is persistent


Multiple equilibria

13M61Mpath 293M27Mpath 3

13M52MPath 1

eq 2eq 1

eq 1

eq 2

Tang, Wang, Hegde, Low, Telecom Systems, 2005

Dummynet experiment


13M61Mpath 293M27Mpath 3

13M52MPath 1

eq 2eq 1

Tang, Wang, Hegde, Low, Telecom Systems, 2005

eq 1

eq 2 eq 3 (unstable)

Dummynet experiment

Multiple equilibria


Heterogeneous protocols

Equilibrium: p that satisfies

∑

∑

⎩⎨⎧

>=≤

=

⎟⎠

⎞⎜⎝

⎛=

i,j ll

lji

jlil

ll

jlli

ji

ji

pcc

pxRpy

pmRfpx

0 if

)( : )(

)( )(

Duality model no longer applies !pl can no longer serve as Lagrange multiplier


Global uniqueness

CorollaryIf price mapping functions ml

j are linear and link-independent, then equilibrium is globally unique

e.g. a network of RED routers with slope inversely proportional to link capacity almost always has

TheoremIf price heterogeneity is small, then equilibrium is globally unique

0any for ]2,[

0any for ]2,[/1

/1

>∈

>∈jjLjj

l

llL

lj

l

aaam

aaam&

&


Local stability:`uniqueness’ stability

TheoremIf price heterogeneity is small, then the unique equilibrium p is locally stable

0any for ]2,[

0any for ]2,[/1

/1

>∈

>∈jjLjj

l

llL

lj

l

aaam

aaam&

&

p(t)pp δγδ )( *J=&Linearized dual algorithm:Equilibrium p is locally stable if

( ) 0 )( Re <pJλ


Local stability:`converse’

TheoremIf all equilibria p are locally stable, then it is globally unique

LpI )1( )( −=Proof idea:

For all equilibrium p:Index theorem:

L

ppI )1( )(

eq−=∑


Deployment: FAST in a box

50x delays are common over DSL speed links

02000400060008000

100001200014000160001800020000

0.1 0.5 1 5 10 20 60

File size (MB)

FTP

thro

ughp

ut (k

bps)

with Aria without Aria

32.5x28.1x21.8x17.6x6.3x3.8x1.8x

InternetFAST inside

Reno avg:35Mbps

FAST avg: 233Mbps

Throuput: LA Tokyo Throuput: San Fran MIT


Extreme loss resilience

San Francisco New YorkJune 3, 2007

Heavy packet loss in Sprint network: Aria increased throughput by 120x !

Without FASTthruput: 1Mbps

With FASTthruput: 120Mbps


Outline



Engineering

ResearchProduct

DevelopmentProduct

Maintenance

AcademicCommunity

Engineering

Academia& Govt

FinanceHumanResource

Sales &Marketing

TechnicalSupport

Admin &Operations

VentureCapitalIndustry

Market

Idea

New Products & Services

Engineering FinanceHuman

Resource

Sales &Marketing

TechnicalSupport

Admin &OperationsHigh Tech

Startup

FundingTalent

Idea

New Problems & Opportunities


Resource

Sales &Marketing

TechnicalSupport


Startup

FundingTalent

Academia& Govt

Market


Resource

Sales &Marketing

TechnicalSupport


Startup

VCIndustry

Remarks

Integration of theory, prototyping, experiment, infrastructure

Industrial partnership like ACCESS is key

Idea Technology Product CompanyEntrepreneurship takes a lot more

Closing the loop achieves more

≠ ≠ ≠

Theory Experiment

Remarks

Integration of theory, prototyping, experiment, infrastructure

Industrial partnership like ACCESS is key

Idea Technology Product CompanyEntrepreneurship takes a lot more

Closing the loop achieves more

≠ ≠ ≠

Theory ExperimentAcademic research

Technologytransfer

internet congestion control ... -...

Documents