Queuing TheoryFor Dummies

Jean-Yves Le Boudec1

All You Need to Know About Queuing Theory

Queuing is essential to understand the behaviour of complex computer and communication systemsIn depth analysis of queuing systems is hardFortunately, the most important results are easy

We will study only simple concepts

2

1. Deterministic QueuingEasy but powerfulApplies to deterministic and transient analysisExample: playback buffer sizing

3

Use of Cumulative Functions

4

Solution of Playback Delay Pb

5

A(t) A’(t) D(t)

time

bits

d(0)d(0) - D d(0) + D(D1):

r(t -

d(0) +

D)

(D2): r (

t - d(

0) - D

) d(t)

A.

2. Operational Laws

Intuition:Say every customer pays one Fr per minute presentPayoff per customer = R Rate at which we receive money = NIn average λ customers per minute, N = λ R

6

Little AgainConsider a simulation where you measure R and N. You use two counters responseTimeCtr and queueLengthCtr. At end of simulation, estimate

R = responseTimeCtr / NbCustN = queueLengthCtr / T

where NbCust = number of customers served and T=simulation duration

Both responseTimeCtr=0 and queueLengthCtr=0 initiallyQ: When an arrival or departure event occurs, how are both counters updated ?A: queueLengthCtr += (tnew - told) . q(told) where q(told) is the number of customers in queue just before the event.

responseTimeCtr += (tnew - told) . q(told)thus responseTimeCtr == queueLengthCtr and thus

N = R x NbCust/T ; now NbCust/T is our estimator of

7

Other Operational Laws

8

The Interactive User Model

99

Network Laws

10

Bottleneck AnalysisApply the following two bounds

1. 2.

11

Example

(1)

(2)

17

Throughput Bounds

12

Bottlenecks

13

A

DASSA

Intuition: within one busy period: to every departure we can associate one arrival with same number of customers left behind

14

3. Single Server Queue

15

16

i.e. which are event averages (vs time averages ?)

17

18

19

2 4 6 8 10Requests per Second

0.5

1

1.5

2

2.5

Mean Response Time in seconds

Non Linearity of Response Time

20

Impact of Variability

21

0.2 0.4 0.6 0.8Utilization

2

4

6

8

10

12

14

Mean Response Time

Optimal SharingCompare the two in terms of

Response timeCapacity

22

The Processor Sharing QueueModels: processors, network links

Insensitivity: whatever the service requirements:

Egalitarian

23

PS versus FIFO

PS FIFO

24

4. A Case StudyImpact of capacity increase ?Optimal Capacity ?

25

Methodology

26

4.1. Deterministic Analysis

27

Deterministic Analysis

28

4.2 Single Queue Analysis

29

Assume no feedback loop:

4.3 Operational AnalysisA refined model, with circulating users

Apply Bottleneck Analysis ( = Operational Analysis )

30

Z/(N-1)

-Z

1/c

waiting time

31

32

5. Networks of QueuesStability

Queuing networks are frequently used modelsThe stability issue may, in general, be a hard one

Necessary condition for stability (Natural Condition)

server utilization < 1

at every queue

33

Instability Examples

34

Poisson arrivals ; jobs go through stations 1,2,1,2,1 then leaveA job arrives as type 1, then becomes 2, then 3 etcExponential, independent service times with mean mi

Priority schedulingStation 1 : 5 > 3 >1Station 2: 2 > 4

Q: What is the natural stability condition ?A: λ (m1 + m3 + m5 ) < 1

λ (m2 + m4) < 1

λ = 1m1 = m3 = m4 = 0.1 m2 = m5 = 0.6

Utilization factorsStation 1: 0.8 Station 2: 0.7

Network is unstable !

If λ (m1 + … + m5 ) < 1 network is stable; why?

35

Bramson’s Example 1: A Simple FIFO Network

Poisson arrivals; jobs go through stations A, B,B…,B, A then leaveExponential, independent service times

Steps 2 and last: mean is LOther steps: mean is S

Q: What is the natural stability condition ?A: λ ( L + S ) < 1

λ ( (J-1)S + L ) < 1 Bramson showed: may be unstable whereas natural stability condition holds

Bramson’s Example 2A FIFO Network with Arbitrarily Small Utilization Factor

m queues2 types of customersλ = 0.5 each type routing as shown, … = 7 visitsFIFOExponential service times, with mean as shown

37

L LS L LS S S S S S S

Utilization factor at every station ≤ 4 λ SNetwork is unstable for S ≤ 0.01L ≤ S8

m = floor(-2 (log L )/L)

Take Home MessageThe natural stability condition is necessary but may not be sufficient

There is a class of networks where this never happens. Product Form Queuing Networks

38

Product Form Networks

Customers have a class attributeCustomers visit stations according to Markov Routing

External arrivals, if any, are Poisson

39

2 StationsClass = step, J+3 classes

Can you reduce the number of classes ?

ChainsCustomers can switch class, but remain in the same chain

40

ν

Chains may be open or closedOpen chain = with Poisson arrivals. Customers must eventually leaveClosed chain: no arrival, no departure; number of customers is constant

Closed network has only closed chainsOpen network has only open chainsMixed network may have both

41

42

3 Stations4 classes1 open chain1 closed chain

ν

Bramson’s Example 2A FIFO Network with Arbitrarily Small Utilization Factor

43

L LS L LS S S S S S S

2 StationsMany classes2 open chainsNetwork is open

Visit Rates

44

45

2 Stations5 classes1 chainNetwork is open

Visit ratesθ1

1 = θ13 = θ1

5 = θ22 = θ2

4 = λ θs

c = 0 otherwise

46

ν

Constraints on StationsStations must belong to a restricted catalog of stationsSee Section 8.4 for full descriptionWe will give commonly used examplesExample 1: Global Processor Sharing

One serverRate of server is shared equally among all customers presentService requirements for customers of class c are drawn iid from a distribution which depends on the class (and the station)

Example 2: DelayInfinite number of serversService requirements for customers of class c are drawn iid from a distribution which depends on the class (and the station)No queuing, service time = service requirement = residence time

47

Example 3 : FIFO with B serversB serversFIFO queueingService requirements for customers of class c are drawn iid from an exponential distribution, independent of the class (but may depend on the station)

Example of Category 2 (MSCCC station): MSCCC with B serversB serversFIFO queueing with constraintsAt most one customer of each class is allowed in serviceService requirements for customers of class c are drawn iid from an exponential distribution, independent of the class (but may depend on the station)

Examples 1 and 2 are insensitive (service time can be anything)Examples 3 and 4 are not (service time must be exponential, same for all)

48

Say which network satisfies the hypotheses for product form

49

A

B (FIFO, Exp)C (Prio, Exp)

The Product Form Theorem

If a network satisfies the « Product Form » conditions given earlierThe stationary distrib of numbers of customers can be written explicitlyIt is a product of terms, where each term depends only on the stationEfficient algorithms exist to compute performance metrics for even very large networks

For PS and Delay stations, service time distribution does not matter other than through its mean (insensitivity)

The natural stability condition holds

50

51

52

ConclusionsQueuing is essential in communication and information systems

M/M/1, M/GI/1, M/G/1/PS and variants have closed forms

Bottleneck analysis and worst case analysis are usually very simple and often give good insightsQueuing networks may be very complex to analyze except if product form – be able to recognize them !

53