delay-based tcp congestion control - universitetet i...

Delay-Based TCPCongestion Control

David Hayes

[email protected]

Centre for Advanced Internet Architectures (CAIA)Swinburne University of Technology

mailto:[email protected]

Outline

CAIABackground

TCP congestion controlDelay based congestion signalsSome key delay-based algorithms

LCN paper Improved coexistence and loss tolerance fordelay based TCP congestion control

Hamilton Institute’s Delay-based algorithm (HD)Shortcomings of HD,improved by CAIA HD algorithm (CHD)

back-off decision frequency and scalingtolerance to non-congestion related lossimprovements when coexisting with NewReno type flows

Other TCP workDelay-gradient based congestion controlStateless TCP

CISCO http://www.caia.swin.edu.au [email protected] 15 October, 2010 2

http://www.caia.swin.edu.au


CAIA –Centre for Advanced Internet Architectures

We are at Swinburne University of Technology, about 5 kmeast of the Melbourne CBD

CAIA is the research arm of the TelecommunicationsEngineering Academic Group (or Department)Research spans:

Congestion control (TCP, etc)Traffic classificationWireless networksCovert channels and lawful interceptionNetwork monitoring and visualisationBGPAddress space explorationGame traffic analysis





We are at Swinburne University of Technology, about 5 kmeast of the Melbourne CBDCAIA is the research arm of the TelecommunicationsEngineering Academic Group (or Department)

Research spans:






We are at Swinburne University of Technology, about 5 kmeast of the Melbourne CBDCAIA is the research arm of the TelecommunicationsEngineering Academic Group (or Department)Research spans:







Congestion control (TCP, etc)

Traffic classificationWireless networksCovert channels and lawful interceptionNetwork monitoring and visualisationBGPAddress space explorationGame traffic analysis






Congestion control (TCP, etc)Traffic classification

Wireless networksCovert channels and lawful interceptionNetwork monitoring and visualisationBGPAddress space explorationGame traffic analysis






Congestion control (TCP, etc)Traffic classificationWireless networks

Covert channels and lawful interceptionNetwork monitoring and visualisationBGPAddress space explorationGame traffic analysis






Congestion control (TCP, etc)Traffic classificationWireless networksCovert channels and lawful interception

Network monitoring and visualisationBGPAddress space explorationGame traffic analysis






Congestion control (TCP, etc)Traffic classificationWireless networksCovert channels and lawful interceptionNetwork monitoring and visualisation

BGPAddress space explorationGame traffic analysis






Congestion control (TCP, etc)Traffic classificationWireless networksCovert channels and lawful interceptionNetwork monitoring and visualisationBGP

Address space explorationGame traffic analysis






Congestion control (TCP, etc)Traffic classificationWireless networksCovert channels and lawful interceptionNetwork monitoring and visualisationBGPAddress space exploration

Game traffic analysis




CISCO supported work

Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)

Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)Heuristics to reduce BGP Update Noise (2007)FreeBSD Implementation of an SCTP friendly NAT (2007)Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)





Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)

Heuristics to reduce BGP Update Noise (2007)FreeBSD Implementation of an SCTP friendly NAT (2007)Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)





Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)Heuristics to reduce BGP Update Noise (2007)

FreeBSD Implementation of an SCTP friendly NAT (2007)Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)





Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)Heuristics to reduce BGP Update Noise (2007)FreeBSD Implementation of an SCTP friendly NAT (2007)

Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)





Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)Heuristics to reduce BGP Update Noise (2007)FreeBSD Implementation of an SCTP friendly NAT (2007)Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)

Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)





Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)Heuristics to reduce BGP Update Noise (2007)FreeBSD Implementation of an SCTP friendly NAT (2007)Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)

Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)





Exploring the efficacy of distributed statistical trafficclassification using modified open source packet filters(2010)Implementing and testing delay-based and rate-basedtransport protocols in FreeBSD (2008)Heuristics to reduce BGP Update Noise (2007)FreeBSD Implementation of an SCTP friendly NAT (2007)Anomalous Traffic Detection and Collaborative NetworkConfiguration Using 3D Multiplayer Game Engines (2006)Public Implementation and Interoperability Testing of NextGeneration TCP Stack Under FreeBSD (2005)Dynamic Self-Learning Traffic Classification Based on FlowCharacteristics (2004)




Introduction to delay and rate based TCP

Promise of low latency zero loss1 transmission

the congestion signal can be decoupled from packet loss

potential for efficient transmission on lossy paths.

Delay based intuition:

delay↑ ≡ queue↑ =⇒ indicates congestion

Rate based intuition:

Send rate > receive rate =⇒ indicates congestion

Basic questions:

How is congestion determined?and if congested, how should cwnd be adjusted

Issues:

Noise of measurementsCorrelation of measurements with congestionCompatibility with existing TCP algorithms

1congestion related





Promise of low latency zero loss1 transmissionthe congestion signal can be decoupled from packet loss


Delay based intuition:

delay↑ ≡ queue↑ =⇒ indicates congestion



Basic questions:


Issues:


1congestion related







Delay based intuition:delay↑ ≡ queue↑ =⇒ indicates congestion



Basic questions:


Issues:


1congestion related








Rate based intuition:Send rate > receive rate =⇒ indicates congestion

Basic questions:


Issues:


1congestion related









Basic questions:How is congestion determined?

and if congested, how should cwnd be adjusted

Issues:


1congestion related









Basic questions:How is congestion determined?and if congested, how should cwnd be adjusted

Issues:


1congestion related










Issues:Noise of measurements

Correlation of measurements with congestionCompatibility with existing TCP algorithms

1congestion related










Issues:Noise of measurementsCorrelation of measurements with congestion

Compatibility with existing TCP algorithms

1congestion related










Issues:Noise of measurementsCorrelation of measurements with congestionCompatibility with existing TCP algorithms

1congestion related




Background – TCP’s Congestion Window (w)In general loss-based TCP:

increases w by the maximum segment size every RTT

or 1/w for every ACK

and halves w when a packet has been lost.





increases w by the maximum segment size every RTTor 1/w for every ACK






increases w by the maximum segment size every RTTor 1/w for every ACK


wi+1 =

{wi2 lost packetwi + 1

wiotherwise

20 25 30 35 40 45 500

20

40

60

80

100

120

time (s)

cw

nd (

packets

)

NewReno




Background: Base timing measurements

drw

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

rttmin

dsw

rttmax

rtt1

daw ≈ drw

dS1





dS1

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

rttmin

dsw

rttmax

rtt1

daw ≈ drw

drw





dsw

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

rttmin

rttmax

rtt1

daw ≈ drw

drw

dS1





drw

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

rttmin

rttmax

rtt1

daw ≈ drw

dS1dsw





daw ≈ drw

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

rttmin

rttmax

rtt1

dS1dsw

drw





rtt1

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

rttmin

rttmax

dS1dsw

drw

daw ≈ drw





dS1

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

dsw

drw

daw ≈ drw

rtt1

rttmin

rttmax





Note: Queueing atFIFO network nodescan increase ordecrease theinterpacket times

daw

SS

S

SS

S

SA

A

A

AA

d ′aw

d ′′aw






daw

SS

S

SS

S

SA

A

A

AA

d ′′aw

d ′aw






daw

SS

S

SS

S

SA

A

A

AA

d ′aw

d ′′aw




Background: Base rate measurements





Ra =∑w−a1 Ai

daw

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

T1 =∑w S

rtt1

Tmax =∑w Srttmin





T1 =∑w S

rtt1

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

Tmax =∑w Srttmin

Ra =∑w−a1 Ai

daw





Tmax =∑w Srttmin

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

Ra =∑w−a1 Ai

daw

T1 =∑w S

rtt1





Ra =∑w−a1 Ai

daw

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

T1 =∑w S

rtt1

Tmax =∑w Srttmin





T1 =∑w S

rtt1

S1

S2

S3

S4

S5

A1

A2A3

A4A5

S6S7S8S9

Tmax =∑w Srttmin

Ra =∑w−a1 Ai

daw




Quick early work overview[Clark et al., 1985]&[Clark et al., 1987] NETBLT RFCs 996&998

[Jacobson, 1988]a – footnote on connectionless rate based AIMD.[Jain, 1989]b normalised delay gradient.[Wang and Crowcroft, 1992]c DUAL algorithm.[Brakmo and Peterson, 1995]d TCP Vegas.

aV. Jacobson, “Congestion avoidance and control,” in SIGCOMM ’88: Symposiumproceedings on Communications architectures and protocols. New York, NY, USA:ACM, 1988, pp. 314–329

bR. Jain, “A delay-based approach for congestion avoidance in interconnectedheterogeneous computer networks,” SIGCOMM Comput. Commun. Rev., vol. 19, no. 5,pp. 56–71, 1989

cZ. Wang and J. Crowcroft, “Eliminating periodic packet losses in the 4.3-TahoeBSD TCP congestion control algorithm,” SIGCOMM Comput. Commun. Rev., vol. 22,no. 2, pp. 9–16, Apr. 1992

dL. S. Brakmo and L. L. Peterson, “TCP Vegas: end to end congestion avoidance ona global internet,” IEEE J. Sel. Areas Commun., vol. 13, no. 8, pp. 1465–1480, Oct.1995




Quick early work overview[Clark et al., 1985]&[Clark et al., 1987] NETBLT RFCs 996&998[Jacobson, 1988]a – footnote on connectionless rate based AIMD.

[Jain, 1989]b normalised delay gradient.[Wang and Crowcroft, 1992]c DUAL algorithm.[Brakmo and Peterson, 1995]d TCP Vegas.








Quick early work overview[Clark et al., 1985]&[Clark et al., 1987] NETBLT RFCs 996&998[Jacobson, 1988]a – footnote on connectionless rate based AIMD.[Jain, 1989]b normalised delay gradient.

[Wang and Crowcroft, 1992]c DUAL algorithm.[Brakmo and Peterson, 1995]d TCP Vegas.








Quick early work overview[Clark et al., 1985]&[Clark et al., 1987] NETBLT RFCs 996&998[Jacobson, 1988]a – footnote on connectionless rate based AIMD.[Jain, 1989]b normalised delay gradient.[Wang and Crowcroft, 1992]c DUAL algorithm.

[Brakmo and Peterson, 1995]d TCP Vegas.








Quick early work overview[Clark et al., 1985]&[Clark et al., 1987] NETBLT RFCs 996&998[Jacobson, 1988]a – footnote on connectionless rate based AIMD.[Jain, 1989]b normalised delay gradient.[Wang and Crowcroft, 1992]c DUAL algorithm.[Brakmo and Peterson, 1995]d TCP Vegas.








A quick look at somekey/interesting algorithms in

the literature


Algorithms: Pkt pair flow control [Keshav, 1994]

All data is sent asback-to-back pairsAvailable send rate is:

T =size(p2)

pair dispersion

Presumes routers useround robin scheduling





All data is sent asback-to-back pairs

Available send rate is:

T =size(p2)

pair dispersion








T =size(p2)

pair dispersion


pair

disperion

network nodeSOURCE

p

2

BOTTLENECK

t

disperionpair

estimate

p1

SINK

RTT







T =size(p2)

pair dispersion


pair

disperion

SOURCE network node

1p

2

BOTTLENECK

t

disperionpair

estimate

p

SINK

RTT







T =size(p2)

pair dispersion


pair

disperion

network nodeSOURCE

p

2

BOTTLENECK

t

disperionpair

estimate

p1

SINK

RTT







T =size(p2)

pair dispersion


pair

disperion

SOURCE network node

1p

2

BOTTLENECK

t

disperionpair

estimate

p

SINK

RTT







T =size(p2)

pair dispersion


pair

disperion

network nodeSOURCE

pair

1p

2

BOTTLENECK

t

estimate

disperion

p

SINK

RTT







T =size(p2)

pair dispersion


pair

disperion

SOURCE network node BOTTLENECK

1p

2

p t

estimate

disperionpair

SINK

RTT





All data is sent asback-to-back pairsAvailable send rate is:

T =size(p2)

pair dispersion


pair

disperion

SOURCE network node BOTTLENECK

1p

2

p t

estimate

disperionpair

SINK

RTT




Algorithms: Vegas [Brakmo and Peterson, 1995]

Iconic rate based TCPDefines two rates:window adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise





Iconic rate based TCP

Defines two rates:window adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise





Iconic rate based TCPDefines two rates:

window adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise






actual =

∑S

rtt

window adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise

S1

S2

S3

S4

S5

rtt1

A1

∑S

rttmin






actual =

∑S

rtt

expected =w

rttmin

window adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise

S1

S2

S3

S4

S5

A1

rtt1

∑S

rttmin






actual =

∑S

rtt

expected =w

rttmin

and diff = expected− actual

window adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise

S1

S2

S3

S4

S5

A1

rtt1

∑S

rttmin






actual =

∑S

rtt

expected =w

rttmin

and diff = expected− actualwindow adjustment:

w ←

w − 1 diff > β

w + 1 diff < α

w otherwise

S1

S2

S3

S4

S5

A1

rtt1

∑S

rttmin




Algorithms: FAST [Wei et al., 2006]

Enhanced Vegas type algorithm

MIMD — AIMD to slow for high BDP networksUses delay as a rich (non binary) congestion indicatorCwnd is updated at regular time intervals (∆t):For MIMD, α(wt ,qi)

increase is proportional to the size of cwnd and the networkqueueing delay.





Enhanced Vegas type algorithmMIMD — AIMD to slow for high BDP networks

Uses delay as a rich (non binary) congestion indicatorCwnd is updated at regular time intervals (∆t):For MIMD, α(wt ,qi)






Enhanced Vegas type algorithmMIMD — AIMD to slow for high BDP networksUses delay as a rich (non binary) congestion indicator

Cwnd is updated at regular time intervals (∆t):For MIMD, α(wt ,qi)






Enhanced Vegas type algorithmMIMD — AIMD to slow for high BDP networksUses delay as a rich (non binary) congestion indicatorCwnd is updated at regular time intervals (∆t):

For MIMD, α(wt ,qi)







wt+∆t = min{

2wt , γ

(rttmin,i

rttiwt + α

)+ (1− γ)wt

}








wt+∆t = min{

2wt , γ

(rttmin,i

rttiwt + α

)+ (1− γ)wt

}Smoothed window increase








wt+∆t = min{

2wt , γ

(rttmin,i

rttiwt + α

)+ (1− γ)wt

}When congested, decreases in proportion to queueing

delay








wt+∆t = min{

2wt , γ

(rttmin,i

rttiwt + α

)+ (1− γ)wt

}increase is limited to 2w per ∆t








wt+∆t = min{

2wt , γ

(rttmin,i

rttiwt + α

)+ (1− γ)wt

}






Algorithms: Compound TCP [Tan et al., 2006]

In high speed high BDP networks aims to increase:

efficiencyRTT fairness and TCP fairness

In MSW Vista and 7Uses Vegas’ rates: diff = (expected− actual)rttmin

Provides NewReno+ performance throughput

The send window, winj , is calculated as:winj = min(wj + dwndj ,awndj )where wj is NewReno’s cwndand dwndj is the delay based window.and awndj is the receivers advertised window.





In high speed high BDP networks aims to increase:efficiency

RTT fairness and TCP fairness








In high speed high BDP networks aims to increase:efficiencyRTT fairness and TCP fairness









In MSW Vista and 7

Uses Vegas’ rates: diff = (expected− actual)rttmin









Provides NewReno+ performance throughputThe send window, winj , is calculated as:winj = min(wj + dwndj ,awndj )

where wj is NewReno’s cwndand dwndj is the delay based window.and awndj is the receivers advertised window.







Provides NewReno+ performance throughputThe send window, winj , is calculated as:winj = min(wj + dwndj ,awndj )where wj is NewReno’s cwnd

and dwndj is the delay based window.and awndj is the receivers advertised window.







Provides NewReno+ performance throughputThe send window, winj , is calculated as:winj = min(wj + dwndj ,awndj )where wj is NewReno’s cwndand dwndj is the delay based window.

and awndj is the receivers advertised window.







Provides NewReno+ performance throughputThe send window, winj , is calculated as:winj = min(wj + dwndj ,awndj )where wj is NewReno’s cwndand dwndj is the delay based window.and awndj is the receivers advertised window.




Algorithms: DUAL [Wang and Crowcroft, 1992]

Designed to supplement loss based congestion control

Delay based measurements provide “slow tuning” of cwndevery 2nd RTT

w ←

{βw rtt > (rttmin+rttmax)

2

w otherwise

where β = 78

Attempts to keep network buffers half fullSmaller multiplicative decreaseRelies on accurate estimates of rttmin and rttmax





Designed to supplement loss based congestion controlDelay based measurements provide “slow tuning” of cwndevery 2nd RTT

w ←


2

w otherwise

where β = 78







w ←


2

w otherwise

where β = 78

Attempts to keep network buffers half full

Smaller multiplicative decreaseRelies on accurate estimates of rttmin and rttmax






w ←


2

w otherwise

where β = 78

Attempts to keep network buffers half fullSmaller multiplicative decrease

Relies on accurate estimates of rttmin and rttmax






w ←


2

w otherwise

where β = 78





Algorithms: Others of Interest

[King et al., 2005] — TCP-Africa

Two modes: Fast delay based, and slow NewReno based.Compound TCP is based on some of Africa’s ideas

[Baiocchi et al., 2007] — YeAH-TCP

Yet Another Highspeed TCPTwo modes like AfricaProvides performance improvements on lossy paths.

A number of schemes propose traffic shaping TCP’s sendrate

[Karandikar et al., 2000] – ABR like[Wu et al., 2002] – leaky bucket[Abendroth et al., 2002] – improved leaky bucket for networkburstiness.





[King et al., 2005] — TCP-AfricaTwo modes: Fast delay based, and slow NewReno based.

Compound TCP is based on some of Africa’s ideas[Baiocchi et al., 2007] — YeAH-TCP








[King et al., 2005] — TCP-AfricaTwo modes: Fast delay based, and slow NewReno based.Compound TCP is based on some of Africa’s ideas

[Baiocchi et al., 2007] — YeAH-TCP









[Baiocchi et al., 2007] — YeAH-TCPYet Another Highspeed TCP

Two modes like AfricaProvides performance improvements on lossy paths.








[Baiocchi et al., 2007] — YeAH-TCPYet Another Highspeed TCPTwo modes like Africa

Provides performance improvements on lossy paths.A number of schemes propose traffic shaping TCP’s sendrate







[Baiocchi et al., 2007] — YeAH-TCPYet Another Highspeed TCPTwo modes like AfricaProvides performance improvements on lossy paths.










[Karandikar et al., 2000] – ABR like

[Wu et al., 2002] – leaky bucket[Abendroth et al., 2002] – improved leaky bucket for networkburstiness.








[Karandikar et al., 2000] – ABR like[Wu et al., 2002] – leaky bucket

[Abendroth et al., 2002] – improved leaky bucket for networkburstiness.




Improved coexistence andloss tolerance for

delay basedTCP congestion control

Best Paper Award LCN 2010

David Hayes and Grenville Armitage

{dahayes,garmitage}@swin.edu.au


Introduction

Delay-based congestion control can potentially provide:

low latency transmission no congestion induced packet loss.efficient TCP over lossy paths (wireless links).

CISCO http://www.caia.swin.edu.au {dahayes,garmitage}@swin.edu.au 15 October, 2010 19


Introduction

Delay-based congestion control can potentially provide:low latency transmission no congestion induced packet loss.

efficient TCP over lossy paths (wireless links).



Introduction

Delay-based congestion control can potentially provide:low latency transmission no congestion induced packet loss.efficient TCP over lossy paths (wireless links).



Introduction


Issues:

Measuring delay to infer congestionCoexistence with current loss-based TCP



Introduction


Issues:Measuring delay to infer congestion

Coexistence with current loss-based TCP



Introduction


Issues:Measuring delay to infer congestionCoexistence with current loss-based TCP



Introduction



We propose a delay based algorithm which:

improves TCP efficiency over lossy pathsimproves coexistence with loss-based TCPimplement algorithms in the FreeBSD kernel



Introduction




improves TCP efficiency over lossy paths

improves coexistence with loss-based TCPimplement algorithms in the FreeBSD kernel



Introduction




improves TCP efficiency over lossy pathsimproves coexistence with loss-based TCP

implement algorithms in the FreeBSD kernel



Introduction




improves TCP efficiency over lossy pathsimproves coexistence with loss-based TCPimplement algorithms in the FreeBSD kernel



Hamilton Delay (HD) based window updates[Budzisz et al., 2009]



Hamilton Delay (HD) based window updates[Budzisz et al., 2009]Probabilistic delay-based backoff (coexistence)




delayQueuing

probabilitybackoff

B

Per−packet

A

backoff probability

pmax

qmin qth

g(q)

qmax




delayQueuing

probabilitybackoff

B

Per−packet

A

backoff probability

pmax

qmin qth

g(q)

qmax

wi+1 =

{wi2 X < g(qi)

wi + 1wi

otherwise




delayQueuing

probabilitybackoff

B

Per−packet

A

backoff probability

pmax

qmin qmax

g(q)

qth

wi+1 =

{wi2 X < g(qi)

wi + 1wi

otherwise

Region A stable when queueing delay is low

Region B only stable when queueing delay is high




delayQueuing

probabilitybackoff

A

Per−packet

B

backoff probability

pmax

qmin qmax

g(q)

qth

wi+1 =

{wi2 X < g(qi)

wi + 1wi

otherwise

Region A stable when queueing delay is lowRegion B only stable when queueing delay is high



RTT – loss-based & delay-based congestion control

20 25 30 35 40 45 5040

60

80

100

120

140

RT

T (

s)

Time (s)

NewReno

20 25 30 35 40 45 5040

60

80

100

120

140

RT

T (

s)

Time (s)

Hamilton



Delay-based back-off decision frequency

delayQueuing

probabilitybackoff

B

Per−packet

A

backoff probability

pmax

qmin qth

g(q)

qmax

HD decision per packet:

Doesn’t scale well:

P[backoff] increases with w

CPU




delayQueuing

probabilitybackoff

B

Per−packet

A

backoff probability

pmax

qmin qth

g(q)

qmax

HD decision per packet:Doesn’t scale well:


CPU




delayQueuing

probabilitybackoff

B

Per−packet

A

backoff probability

pmax

qmin qth

g(q)

qmax



CPU

CHD decision once per RTT

Uses hr = maxr (qi)

Scales with wless CPU



Delay-based back-off decision frequencyPer−RTT

delayQueuing

probabilitybackoff

A B

backoff probability

qmax

pmax

qmin qth

g(hr)



CPU


Uses hr = maxr (qi)





delayQueuing

probabilitybackoff

A B

backoff probability

qmax

pmax

qmin qth

g(hr)



CPU


Uses hr = maxr (qi)

Scales with w

less CPU




delayQueuing

probabilitybackoff

A B

backoff probability

qmax

pmax

qmin qth

g(hr)



CPU


Uses hr = maxr (qi)




Tolerance to non-congestion related packet loss




HD (and NewReno) do not tolerate packet loss

w = w/2 when a packet is lost






CHD tolerates low level packet loss well by:

Ignoring packet loss when queueing delays are small(region A)






CHD tolerates low level packet loss well by:Ignoring packet loss when queueing delays are small(region A)






CHD tolerates low level packet loss well by:Ignoring packet loss when queueing delays are small(region A)

delayQueuing

probabilitybackoff

B

Per−RTTbackoff probability

A

qthqmin

pmax

g(hr)

qmax



Tolerance to non-congestion related losses

0 0.01 0.02 0.03 0.04 0.050

2

4

6

8

10x 10

6

Goodput (b

ps)

Probability of non−congestion related loss

NewReno

HD

CHD

1/sqrt(p)



Improving coexistence with loss-based TCP

To improve CHD’s coexistence ability

React only to packet loss in region BWe use a shadow window (s) (shadows NewReno)

On packet loss, wi+1 = max(wi ,si )2

Best explained with an example




To improve CHD’s coexistence abilityReact only to packet loss in region B

We use a shadow window (s) (shadows NewReno)


delayQueuing

probabilitybackoff

B


A

qmax

pmax

qmin qth

g(hr)





To improve CHD’s coexistence abilityReact only to packet loss in region BWe use a shadow window (s) (shadows NewReno)


delayQueuing

probabilitybackoff

B


A

qmax

pmax

qmin qth

g(hr)




Shadow window example



Shadow window examplep

ack

ets

number of round trip times

w

s = 0




ack

ets

delay based congestion


w

s = 0




ack

ets



s sync

w

s




ack

ets



lost packets sync

w

w recovery

s




ack

ets



lost packets sync

w

w recovery

s

Region B

max(wi , si)

2




ack

ets



lost packets sync

w

w recovery

without w recovery

s




ack

ets



lost packets sync

w

w recovery

without w recovery

transmissionlost

opportunity

s




ack

ets



lost packets sync

w

w recovery

without w recovery

transmissionlost

opportunity gainedtransmissionopportunity

s



Testbed for coexistence tests

(FreeBSD)

20ms

20ms

NewReno Sources(FreeBSD)

Delay CC Sources(FreeBSD)

Delay CC Sink

NewReno Sink(FreeBSD)

(FreeBSD)

Dummynet Router

We will look at HD and CHD coexisting with NewRenoFor 0 % and 1 % non-congestion related losses




(FreeBSD)

20ms

20ms



Delay CC Sink


(FreeBSD)

Dummynet Router

We will look at HD and CHD coexisting with NewReno

For 0 % and 1 % non-congestion related losses




(FreeBSD)

20ms

20ms



Delay CC Sink


(FreeBSD)

Dummynet Router

We will look at HD and CHD coexisting with NewRenoFor 0 % and 1 % non-congestion related losses



HD coexisting with NewReno (1s Av, 0 % loss)

0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

HD NewReno NewReno HD




0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

HD NewReno NewReno HD

HD does not compete well with NewReno



CHD coexisting with NewReno (1s Av, 0 % loss)

0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

CHD NewReno NewReno CHD




0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

CHD NewReno NewReno CHD

CHD reclaims some of the lost capacity from NewReno



Coexiting with NewReno on a lossy path

1 % loss5 s averages




0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

HDNewRenoNewRenoHD




0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

HDNewRenoNewRenoHD

HD and NewReno cannot efficiently utilise the availablebandwidth




0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

CHDNewRenoNewRenoCHD




0 20 40 60 80 100 120 1400

2

4

6

8

10x 10

6

Time (s)

Go

od

pu

t (b

ps)

CHDNewRenoNewRenoCHD

CHD is able to effectively use the available bandwidth(including what NewReno is unable to use)



LCN Conclusions

CHD significantly enhances HD:improves scalability with per RTT decisions

improves tolerance to non-congestion related packet lossesimproves coexistence with loss based TCP (NewReno)

Shadow windowLightly multiplexed environments

CHD and HD have been implemented in the FreeBSD kernel(caia.swin.edu.au/urp/newtcp/tools.html)


caia.swin.edu.au/urp/newtcp/tools.html


LCN Conclusions

CHD significantly enhances HD:improves scalability with per RTT decisionsimproves tolerance to non-congestion related packet losses

improves coexistence with loss based TCP (NewReno)






LCN Conclusions

CHD significantly enhances HD:improves scalability with per RTT decisionsimproves tolerance to non-congestion related packet lossesimproves coexistence with loss based TCP (NewReno)






LCN Conclusions


Shadow window

Lightly multiplexed environments





LCN Conclusions







LCN Conclusions




This work was made possible in partby a grant from the Cisco University Research Program Fund

at Community Foundation Silicon Valley




A brief look at some otherTCP work at CAIA

David Hayes and Grenville Armitage

{dahayes,garmitage}@swin.edu.au


Delay-gradient based TCP congestion control

We investigated a delay-gradient congestion signalbecause:

it does not require an accurate estimate of base RTTdelay thresholds are hard to set —need to know path’s delay characteristics

We have implemented it in FreeBSD, to be released soon.(waiting on a paper submission)





it does not require an accurate estimate of base RTT

delay thresholds are hard to set —need to know path’s delay characteristics






it does not require an accurate estimate of base RTTdelay thresholds are hard to set —need to know path’s delay characteristics




Comparing NewReno and our Delay-GradientTCP RTT dynamics as 3 sources start

NewReno0 20 40 60 80 100

40

60

80

100

120

140

RT

T (

s)

Time (s)

flow 1

flow 2

flow 3



Comparing NewReno and our Delay-GradientTCP RTT dynamics as 3 sources start

NewReno

Delay-Gradient

0 20 40 60 80 10040

60

80

100

120

140

RT

T (

s)

Time (s)

flow 1

flow 2

flow 3

0 20 40 60 80 10040

60

80

100

120

140

RT

T (

s)

Time (s)

flow 1

flow 2

flow 3



Stateless TCP

Proposed by Geoff Huston to mitigate a DNS server issuehttp://www.potaroo.net/ispcol/2009-11/stateless.pdf

Funded by APNIC and Nominet

DNSSEC may cause the answers to DNS queries to exceeda single UDP packetClients using TCP may overload serversGeoff’s idea→ stateless TCPImplemented in FreeBSD, will be released on the CAIA website in a few weeks.


http://www.potaroo.net/ispcol/2009-11/stateless.pdf



Stateless TCP


Funded by APNIC and NominetDNSSEC may cause the answers to DNS queries to exceeda single UDP packet

Clients using TCP may overload serversGeoff’s idea→ stateless TCPImplemented in FreeBSD, will be released on the CAIA website in a few weeks.





Stateless TCP


Funded by APNIC and NominetDNSSEC may cause the answers to DNS queries to exceeda single UDP packetClients using TCP may overload servers

Geoff’s idea→ stateless TCPImplemented in FreeBSD, will be released on the CAIA website in a few weeks.





Stateless TCP


Funded by APNIC and NominetDNSSEC may cause the answers to DNS queries to exceeda single UDP packetClients using TCP may overload serversGeoff’s idea→ stateless TCP

Implemented in FreeBSD, will be released on the CAIA website in a few weeks.





Stateless TCP


Funded by APNIC and NominetDNSSEC may cause the answers to DNS queries to exceeda single UDP packetClients using TCP may overload serversGeoff’s idea→ stateless TCPImplemented in FreeBSD, will be released on the CAIA website in a few weeks.





Basic stateless TCP idea

if matches hash

DNS server

IP

Data

UDP

send data up

through UDP

statelessTCP

TCP

Listen

Data



CPU time versus DNS query arrival rate

0 100 200 300 400 5000

0.5

1

1.5

2

2.5

Average requests/second

CP

U u

sage in 1

0s inte

rval

udp

tcp

stateless



Thoughts and conclusions

Delay-based TCP coexistence with loss-based TCP

current schemes coexist by behaving like NewRenoLow latency with no congestion related loss

only when there are no loss-based flows sharing path

If switches and routers could differentiate between loss anddelay based TCP, benefits would be realised sooner.

Delay-gradient as a congestion indication

works wella composite delay-based congestion indication may be better




Delay-based TCP coexistence with loss-based TCPcurrent schemes coexist by behaving like NewReno

Low latency with no congestion related loss








Delay-based TCP coexistence with loss-based TCPcurrent schemes coexist by behaving like NewRenoLow latency with no congestion related loss











Delay-gradient as a congestion indicationworks well

a composite delay-based congestion indication may be better







Delay-gradient as a congestion indicationworks wella composite delay-based congestion indication may be better



Thank you!



Thank you!

Questions?



delay-based tcp congestion control - universitetet i...

Documents