ion stoica,scott shenker, and hui zhang sigcomm’98, vancouver, august 1998 subsequently

1

Core-Stateless Fair Queueing:Core-Stateless Fair Queueing:A Scalable Architecture to A Scalable Architecture to

Approximate Fair Bandwidth Approximate Fair Bandwidth Allocations in High Speed NetworksAllocations in High Speed Networks

Ion Stoica,Scott Shenker, and Hui ZhangIon Stoica,Scott Shenker, and Hui ZhangSIGCOMM’98, Vancouver, August 1998SIGCOMM’98, Vancouver, August 1998

subsequentlysubsequentlyIEEE/ACM Transactions on Networking IEEE/ACM Transactions on Networking

11(1), 2003, pp. 33-46.11(1), 2003, pp. 33-46.

Presented by Bob KinickiPresented by Bob Kinicki

Advanced Computer Networks: CSFQ Advanced Computer Networks: CSFQ PaperPaper

22

OutlineOutline IntroductionIntroduction Core-Stateless Fair Queueing (CSFQ)Core-Stateless Fair Queueing (CSFQ)

– Fluid Model Algorithm– Packet Algorithm– Flow Arrival Rate– Link Fair Share Rate Estimation

NS SimulationsNS Simulations ConclusionsConclusions


33

IntroductionIntroduction This paper brings forward the concept of “fair” This paper brings forward the concept of “fair”

allocation.allocation. The claim is that fair allocation inherently The claim is that fair allocation inherently

requires routers to maintain requires routers to maintain statestate and perform and perform operations on a per flow basis.operations on a per flow basis.

The authors present an architecture and a set The authors present an architecture and a set of algorithms that is “approximately” fair while of algorithms that is “approximately” fair while using FIFO queueing at internal routers.using FIFO queueing at internal routers.


44

An “Island” of RoutersAn “Island” of Routers

EdgeRouter

CoreRouter

SourceDestination

Destination


55





66

Core-Stateless Fair Core-Stateless Fair QueueingQueueing

Ingress edge routers compute per-flow Ingress edge routers compute per-flow rate estimates and insert these estimates rate estimates and insert these estimates as as labelslabels into each packet header. into each packet header.

Only edge routers maintain per flow state.Only edge routers maintain per flow state.Labels are updated at each router based Labels are updated at each router based

only on aggregate information.only on aggregate information.FIFO queuing with probabilistic dropping FIFO queuing with probabilistic dropping

of packets on input is employed at the of packets on input is employed at the core routers.core routers.


77

Edge – Core Router Edge – Core Router ArchitectureArchitecture


88

Fluid Model AlgorithmFluid Model Algorithm Assume the bottleneck router has an output Assume the bottleneck router has an output

link with capacity link with capacity CC.. Assume each flow’s arrival rate, Assume each flow’s arrival rate, rrii (t)(t) , is , is

known precisely.known precisely. The main idea is that max-min fair The main idea is that max-min fair

bandwidth allocations are characterized bandwidth allocations are characterized such that all flows that are bottlenecked such that all flows that are bottlenecked by a router have the same output rate.by a router have the same output rate.

This rate is called the This rate is called the fair share rate fair share rate of the linkof the link.. Let Let α(t)α(t) be the fair share rate at time t. be the fair share rate at time t.


99

Fluid Model AlgorithmFluid Model Algorithm

If max-min bandwidth allocations are achieved, each flow receives service at a rate given by

min min (r(rii (t), (t), α(α(tt))))

Let A(t)A(t) denote the total arrival rate:

If A(t) > CA(t) > C then the fair share is the unique solution to


1010

Fluid Model AlgorithmFluid Model AlgorithmThus, the probabilistic fluid forwarding Thus, the probabilistic fluid forwarding

algorithm that achieves fair bandwidth algorithm that achieves fair bandwidth allocation is:allocation is:

Each incoming bit of flowEach incoming bit of flow ii is is dropped with probabilitydropped with probability

max (0,1-max (0,1-α(t)/rα(t)/rii(t)(t)) (2)) (2)

These dropping probabilities yield fair These dropping probabilities yield fair share arrival rates at the next hop. share arrival rates at the next hop.


1111

Packet AlgorithmPacket Algorithm

Moving from a bit-level, buffer-less fluid Moving from a bit-level, buffer-less fluid model to a packet-based, buffer model model to a packet-based, buffer model leaves two challenges:leaves two challenges:– Estimate the flow arrival rates rrii(t)(t)

– Estimate the fair share α(t)α(t)This is possible because the rate This is possible because the rate

estimator incorporates the packet size.estimator incorporates the packet size.


1212

Flow Arrival RateFlow Arrival RateAt each edge router, use exponential averaging to estimate the At each edge router, use exponential averaging to estimate the

rate of a flow. For flow rate of a flow. For flow ii, let, let

lliikk be the length of the be the length of the kkthth packet.packet.

ttiikk be the arrival time of the be the arrival time of the kkthth packet.packet.

Then the estimated rate of flow Then the estimated rate of flow i, ri, rii is updated every time a new is updated every time a new packet is received:packet is received:

rrii

new new = (1-e= (1-e-T/K-T/K)*L / T + e)*L / T + e-T/K-T/K* r* riioldold (3)(3)

wherewhere

TT == TTiikk = t= tii

kk – – ttiik-1k-1

L = lL = liikk and and KK is a constant is a constant


1313

Link Fair Rate EstimationLink Fair Rate Estimation

If we denote the estimate of the fair share by If we denote the estimate of the fair share by and the acceptance rate by , and the acceptance rate by , we havewe have

Note – if we know Note – if we know rrii(t)(t) then can be then can be determined by finding the unique solution to determined by finding the unique solution to F(x) = CF(x) = C..

However, this requires per-flow state !However, this requires per-flow state !


1414

Heuristic AlgorithmHeuristic Algorithm

The heuristic algorithm needs three aggregate state The heuristic algorithm needs three aggregate state variables:variables:

, , where is the estimated aggregate , , where is the estimated aggregate arrival rate.arrival rate.

When a packet arrives, the router computes:When a packet arrives, the router computes:

(5)(5)

and similarly computes .and similarly computes .


1515

CSFQ Algorithm CSFQ Algorithm

When a packet arrives, is updated using When a packet arrives, is updated using exponential averaging (equation 5).exponential averaging (equation 5).

If the packet is dropped, remains the same.If the packet is dropped, remains the same.

If the packet is not dropped, is updated If the packet is not dropped, is updated using exponential averaging.using exponential averaging.

At the end of an epoch (defined by At the end of an epoch (defined by KKcc ), if the ), if the

link is congested during the whole epoch, link is congested during the whole epoch, update :update :


1616

CSFQ Algorithm (cont.)CSFQ Algorithm (cont.)

If the link is not congested, is set to If the link is not congested, is set to the largest rate of any active flow.the largest rate of any active flow.

feeds into the calculation of drop feeds into the calculation of drop probability, probability, pp, for the next arriving packet , for the next arriving packet as as αα in in

p = max (0 , 1 – p = max (0 , 1 – α α / label)/ label)


1717

CSFQ Algorithm (cont.)CSFQ Algorithm (cont.)

Estimation inaccuracies may cause Estimation inaccuracies may cause to exceed link capacity. to exceed link capacity.

Thus, to limit the effect of Drop Tail Thus, to limit the effect of Drop Tail buffer overflows, every time buffer buffer overflows, every time buffer overflows is decreased by 1% in overflows is decreased by 1% in simulations.simulations.


1818

CSFQ Pseudo CodeCSFQ Pseudo Code

Figure 3Figure 3


1919

CSFQ CSFQ PseudoPseudo Code Code


2020

Label RewritingLabel Rewriting

At core routers, outgoing rate is merely At core routers, outgoing rate is merely the minimum between the incoming rate the minimum between the incoming rate and the fair rate, and the fair rate, αα . .

Hence, the packet label L can be Hence, the packet label L can be rewritten byrewritten by

L L newnew == minmin (L (L oldold , , α )α )


2121





2222

SimulationsSimulations

A major effort of the paper is to A major effort of the paper is to compare CSFQ to four algorithms via compare CSFQ to four algorithms via ns-2 simulations.ns-2 simulations.

FIFOFIFOREDREDFRED (Flow Random Early Drop)FRED (Flow Random Early Drop)DRR (Deficit Round Robin)DRR (Deficit Round Robin)


2323

FRED (Flow Random Early FRED (Flow Random Early Drop)Drop)

Maintains per flow state in router.Maintains per flow state in router. FRED preferentially drops a packet of a flow FRED preferentially drops a packet of a flow

that has either:that has either:– Had many packets dropped in the past– A queue larger than the average queue size

Main goal : FairnessMain goal : Fairness FRED-2 guarantees to each flow a minimum FRED-2 guarantees to each flow a minimum

number of buffers.number of buffers.


2424

DRR (Deficit Round Robin)DRR (Deficit Round Robin)Represents an efficient implementation of Represents an efficient implementation of

WFQ.WFQ.A sophisticated per-flow queueing A sophisticated per-flow queueing

algorithm.algorithm.Scheme assumes that when router buffer is Scheme assumes that when router buffer is

full the packet from the longest queue is full the packet from the longest queue is dropped.dropped.

Can be viewed as “best case” algorithm Can be viewed as “best case” algorithm with respect to fairness.with respect to fairness.


2525

ns-2 Simulation Detailsns-2 Simulation DetailsUse TCP, UDP, RLM and On-Off traffic Use TCP, UDP, RLM and On-Off traffic

sources in separate simulations.sources in separate simulations.Bottleneck link: 10 Mbps, 1ms latency, Bottleneck link: 10 Mbps, 1ms latency,

64KB buffer64KB bufferRED, FRED (min, max) thresholds: RED, FRED (min, max) thresholds:

(16KB, 32KB)(16KB, 32KB)KK and and KKcc = 100 ms. = 200ms. = 100 ms. = 200ms.KKαα


2626

A Single Congested LinkA Single Congested Link

First Experiment : 32 UDP CBR flowsFirst Experiment : 32 UDP CBR flows– Each UDP flow is indexed from 0 to 31 with

flow 0 sending at 0.3125 Mbps and each of the i subsequent flows sending (i+ 1) times its fair share of 0.3125 Mbps.

Second Experiment : 1 UDP CBR flow, Second Experiment : 1 UDP CBR flow, 31 TCP flows31 TCP flows– UDP flow sends at 10 Mbps– 31 TCP flows share a single 10 Mbps link.


2727

Figure 5b: 32 UDP Flows Figure 5b: 32 UDP Flows

Only CSFQ, DRRand FRED-2 cancontain UDP flows!!


2828

Figure 6b : One UDP Flow, 31 TCP Figure 6b : One UDP Flow, 31 TCP FlowsFlows

Only CSFQ andDRR can containFlow 0 – the onlyUDP flow!


2929

A Single Congested LinkA Single Congested Link

Third Experiment Set : 31 simulationsThird Experiment Set : 31 simulations– Each simulation has a different N,

N = 2 … 32.– One TCP and N-1 UDP flows with each

UDP flow sending at twice fair share rate of 10/N Mbps.


3030

Figure 6b : One TCP Flow, N-1 UDP Figure 6b : One TCP Flow, N-1 UDP FlowsFlows

Normalized fair sharethroughput for one TCP source

DRR good for lessthan 22 flows.

CSFQ better thanDRR when a largenumber of flows.

CSFQ beats FRED.


3131

Multiple Congested LinksMultiple Congested Links

Router Router KRouter Router K+1

UDPSinks

TCP/UDP-0Sink

TCP/UDP-0Source

UDPSources

1

1-10 K1-K10

10 11 20 K10K1


3232

Multiple Congested LinksMultiple Congested Links

First experiment : UDP flow 0 sends at First experiment : UDP flow 0 sends at its fair share rate, 0.909 Mbps while the its fair share rate, 0.909 Mbps while the other ten “crossing” UDP flows send at other ten “crossing” UDP flows send at 2 Mbps.2 Mbps.

Second experiment: Replace the UDP Second experiment: Replace the UDP flow with one TCP flow and leave the flow with one TCP flow and leave the ten crossing UDP flows.ten crossing UDP flows.


3333

Figure 8a : UDP sourceFigure 8a : UDP source

Fraction of UDP-0 traffic forwardedversus the number of congested links


3434

Figure 8b : TCP SourceFigure 8b : TCP Source

Fraction of TCP-0 traffic forwardedversus the number of congested links


3535

Receiver-driven Layered Receiver-driven Layered MulticastMulticast

RLM is an adaptive scheme in which RLM is an adaptive scheme in which the source sends the information the source sends the information encoded in a number of layers.encoded in a number of layers.

Each layer represents a Each layer represents a different different multicast group.multicast group.

Receivers join and leave multicast Receivers join and leave multicast groups based on packet drop rates groups based on packet drop rates experienced.experienced.


3636

Receiver-driven Layered Receiver-driven Layered MulticastMulticast

Simulation of three RLM flows and one Simulation of three RLM flows and one TCP flow with a 4 Mbps link.TCP flow with a 4 Mbps link.

Fair share for each is 1 Mbps.Fair share for each is 1 Mbps.Since router buffer set to 64 KB, Since router buffer set to 64 KB, KK, , KKcc, ,

and and are set to 250 ms.are set to 250 ms.Each RLM layer Each RLM layer II sends 2sends 2i+4i+4 Kbps with Kbps with

each receiver subscribing to the first each receiver subscribing to the first five layers. five layers.

KKαα


3737

Figure 9b : FREDFigure 9b : FRED


3838

Figure 9e : REDFigure 9e : RED


3939

Figure 9f : FIFOFigure 9f : FIFO


4040

Figure 9a : DRRFigure 9a : DRR


4141

Conference Figure : CSFQConference Figure : CSFQ

KK,== Kc Kc = = Kα Kα = = 250 ms.250 ms.


4242

Figure 9c: CSFQFigure 9c: CSFQ


4343

Figure 9d: CSFQFigure 9d: CSFQ


4444

On-Off Flow ModelOn-Off Flow Model

One approach to modeling interactive, One approach to modeling interactive, Web traffic :: OFF represents “think Web traffic :: OFF represents “think time”time”

ON and OFF drawn from exponential ON and OFF drawn from exponential distribution with means of 100 ms and distribution with means of 100 ms and 1900 ms respectively.1900 ms respectively.

During ON period source sends at 10 During ON period source sends at 10 Mbps.Mbps.


4545

Table 1 : One On-Off Flow, 19 TCP Table 1 : One On-Off Flow, 19 TCP FlowsFlows

AlgorithmAlgorithm DeliveredDelivered DroppedDropped

DRRDRR 10801080 38193819

CSFQCSFQ 10001000 38893889

FREDFRED 10641064 38253825

REDRED 28192819 20802080

FIFOFIFO 37713771 11281128

4899 packets sent!4899 packets sent!


4646

Web TrafficWeb Traffic

A second approach to modeling Web A second approach to modeling Web traffic that uses Pareto Distribution to traffic that uses Pareto Distribution to model the length of a TCP connection.model the length of a TCP connection.

In this simulation 60 TCP flows whose In this simulation 60 TCP flows whose interarrivals are exponentially interarrivals are exponentially distributed with mean 0.05 ms and distributed with mean 0.05 ms and Pareto distribution that yields a mean Pareto distribution that yields a mean connection length of 20,1 KB packets.connection length of 20,1 KB packets.


4747

Table 2 : 60 Short TCP Flows, One UDP Table 2 : 60 Short TCP Flows, One UDP FlowFlow

AlgorithmAlgorithm Mean Transfer Mean Transfer Time for TCPTime for TCP

Standard Standard DeviationDeviation

DRRDRR 2525 9999

CSFQCSFQ 6262 142142

FREDFRED 4040 174174

REDRED 592592 12741274

FIFOFIFO 840840 16951695


4848

Table 3 :Table 3 : 19 TCP Flows, One UDP 19 TCP Flows, One UDP Flow with propagation delay of 100 Flow with propagation delay of 100

ms.ms.AlgorithmAlgorithm Mean Mean

ThroughputThroughputStandard Standard DeviationDeviation

DRRDRR 60806080 6464

CSFQCSFQ 57615761 220220

FREDFRED 49744974 190190

REDRED 628628 8080

FIFOFIFO 378378 6969


4949

Packet RelabelingPacket Relabeling

RouterFlow 2

Router

Sink

Flow 1

Sources

Flow 3

Link 210 Mbps

Link 110 Mbps

10 Mbps

10 Mbps

10 Mbps


5050

Table 4 : UDP and TCP Table 4 : UDP and TCP with Packet Relabeling with Packet Relabeling

TrafficTraffic Flow 1Flow 1 Flow 2Flow 2 Flow 3Flow 3

UDPUDP 3.2673.267 3.2623.262 3.4583.458

TCPTCP 3.2323.232 3.3363.336 3.3583.358


5151

Unfriendly FlowsUnfriendly FlowsUsing TCP congestion control requires Using TCP congestion control requires

cooperation from other flows.cooperation from other flows.Three types cooperation violators:Three types cooperation violators:

– Unresponsive flows (e.g., Real Audio)– Not TCP-friendly flows (e.g., RLM)– Flows that lie to cheat.

This paper deals with unfriendly This paper deals with unfriendly flows!!flows!!


5252





5353

ConclusionsConclusionsThis paper presents Core Stateless Fair This paper presents Core Stateless Fair

Queueing and offers many simulations Queueing and offers many simulations to show how CSFQ provides better to show how CSFQ provides better fairness than RED or FIFO.fairness than RED or FIFO.

They mention issue of “large latencies”. They mention issue of “large latencies”. This is the robust versus fragile flow This is the robust versus fragile flow issue from FRED paper.issue from FRED paper.

CSFQ ‘clobbers’ UDP flows!CSFQ ‘clobbers’ UDP flows!


5454

SignificanceSignificance

First paper to use hints from the edge of First paper to use hints from the edge of the subnet.the subnet.

Deals with UDP. Many AQM algorithms Deals with UDP. Many AQM algorithms ignore UDP.ignore UDP.

Makes a reasonable attempt to look at a Makes a reasonable attempt to look at a variety of traffic types.variety of traffic types.


5555

Problems/ WeaknessesProblems/ Weaknesses

““Epoch” is related to three constants in Epoch” is related to three constants in a way that can produce different results.a way that can produce different results.

How does one set K constants for a How does one set K constants for a variety of situations.variety of situations.

No discussion of algorithm “stability”No discussion of algorithm “stability”


5656

AcknowledgmentsAcknowledgments

Figures extracted from presentation by Figures extracted from presentation by Nagaraj Shirali and Choong-Soo Lee in Nagaraj Shirali and Choong-Soo Lee in Spring 2002 and modified for Spring 2002 and modified for annotations.annotations.

ion stoica,scott shenker, and hui zhang sigcomm’98, vancouver, august 1998 subsequently

Documents