Chapter 8 1

Queuing Theory

Basic properties, Markovian models, Networks of queues, General service time distributions, Finite

source models, Multiserver queues

Chapter 8 2

Kendall’s Notation for Queuing Systems

A/B/X/Y/Z:

A = interarrival time distribution

B = service time distribution

G = general (i.e., not specified); M = Markovian (exponential); D = deterministic

X = number of parallel service channels

Y = limit on system pop. (in queue + in service); default is

Z = queue discipline; default is FCFS (first come first served)

Others are LCFS, random, priority

Chapter 8 3

Other NotationRandom variables:

X(t) = Number of customers in the system at time t

S = Service time of an arbitrary customer

W* = Amount of time an arbitrary customer spends in the system

WQ* = Amount of time an arbitrary customer spends waiting for service

Often we are most interested in the averages or expectations: Average number in the queue

Average number in service

Q

S

L E X t

L

L

*

*Q Q

W E W

W E W

Chapter 8 4

Little’s Formula

is the time spent in the system by the nth customer. Assume these system times are uniformly finite and letbe the customer average time spent in the system.Also, let be the time average number of jobs in the system. Then under very general conditions,

where a is the arrival rate. This is usually written L = aW.The mean number of customers in the system is proportional

to the mean time in the system!

*nw

11

lim *k

k nk nw w

1

0lim ( )

s

s sX X t dt

aX w

Chapter 8 5

Heuristic Proof of Little’s Formula

Two ways to compute

Mean time in system in (0,T):

Mean number in system in (0,T):

Then

T

A(t)

D(t)

B(t) = cum. area between curves

1 0( ) * ( )

TN

nnB T w X t dt

( ) ( ) ( ) ( )W T B T N B T A T ( ) ( )X T B T T

( ) ( )lim ( ) lim lim ( ), or

( ) aT T T

B T A TX T T W T L W

A T T

Chapter 8 6

Other Little’s Formulae

In queue:

In service:

and their implications…

Expected number of busy servers

Expected number of idle servers

Single server utilization

Single server prob. of empty system

Q a QL W S aL E S

S aL E S # servers

S aL E S 1

Chapter 8 7

Observation Times

an = Proportion of arriving customers that find n in the system

dn = Proportion of departing customers that leave behind n in the system

If customers arrive one at a time and are served one at a time then

But these proportions may not match the limiting probability of having n in the system (long run proportion of time that n are in the system)

However, if arrivals follow a Poisson process then This is known as PASTA (Poisson Arrivals See Time Averages)

n na d

n na P

lim ( ) , 0,1,...n tP P X t n n

Chapter 8 8

M/M/1 Model

Single server, Poisson arrivals (rate ), exponential service times (rate )

CTMC (birth-death) model:

Chapter 8 9

M/M/1 Steady-State Probabilities

Level-crossing equations:

Solve for Pn in terms of P0

Then use the facts that

and substitute

to get

lim ( ) , 0,1,...n tP P X t n n

1, 0,1,n nP P n

0 , 1, 2,n

nP P n

1

0 01, (1 ) if 1n

nn nP r r r

/

(1 ), 0,1, if 1nnP n

Chapter 8 10

M/M/1 Performance Measures

Steady-state expected number of customers in the system

Mean time in system

Mean time in queue

Mean number in queue

0, if 1

1nnL nP

1 1 by Little's Formula

(1 )a

LW

1

=-QW W

2

=-Q QL W

Chapter 8 11

M/M/1 Performance Measures

Distribution of time in system (FCFS)

Exponential with parameter (1-)

Reversibility: Departure process is Poisson with rate

0

0

(1 )

0 1

[ * ] [ * | arrival sees customers]

(1 ) [ * | * gamma( 1, )]

(1 ) ( ) ! 1 , 0

nn

n

n

n x l x

n l n

P W x a P W x n

P W x W n

e x l e x

Chapter 8 12

Finite Capacity: M/M/1/N Model

Single server, exponential service times (rate )

Poisson arrivals (rate ) as long as there are N in the system

CTMC (birth-death) model:

Chapter 8 13

M/M/1/N Steady-State Probabilities

Level-crossing equations:

Solve for Pn in terms of P0

Then use the facts that

to get

lim ( ) , 0,1,...n tP P X t n n

1, 0,1, , 1n nP P n N

1

0 0

11, (note: need not be < 1)

1

NN n

nn n

rP r r

r

1

1, 0,1, ,

1

n

n NP n N

Chapter 8 14

M/M/1/N Performance Measures

(In the unlikely event that , for n = 1,…, N,

Steady-state expected number of customers in the system, L, has a “messy” closed form

Mean time in system

but here, a is the rate of arrival into the system

by Little's Formulaa

LW

0 1 1 )nP P N

1a NP

1 QW W

1 Q N QL P W

Chapter 8 15

Tandem Queue

If arrivals to the first server follow a Poisson process and service times are exponential, then arrivals to the second server also follow a Poisson process and the two queues behave as independent M/M/1 systems:

P{n customers at server 1 and m customers at server 2} =

1 2

1 1 2 2

1 1n m

Chapter 8 16

Open Network of Queues

• k servers, customers arrive at server k from outside the system according to a Poisson process with rate rk, independent of the other servers

• Upon completing service at server i, customer goes to server j with probability Pij, where

For j = 1,…, k, the total arrival rate to server j is

The number of customers at each server is independent and

If j < j for all j, then P{n customers at server j}

that is, each acts like an independent M/M/1 queue!

1

k

j j i iji

r P

1

n

j j

j j

1ijjP

Chapter 8 17

Closed Queuing Network

• m customers move among k servers

• Upon completing service at server i, customer goes to server j with probability Pij, where

• Let be the stationary probabilities for the Markov chain describing the sequence of servers visited by a customer:

Then the probability distribution of the number at each server is

1ijjP

1 1

, 1k k

j i ij ji j

P

1 211

, ,..., if jn

k kj

m k m jjj j

P n n n C n m

Chapter 8 18

CQN Performance

Computation of the normalizing constant Cm to get the stationary distribution can be lengthy; but may be mostly interested in the throughput

where m(j) is the arrival rate to (and departure rate from) j.

Arrival Theorem: In the CQN with m customers, the system as seen by arrivals to server j has the same distribution as the whole system when it contains only m-1 customers.

This leads to mean value analysis to find m(j) along with

Wm(j) = the average time a customer spends at server j, and

Lm(j) = the average number of customers at server j.

1

j

m mjj

Chapter 8 19

Mean Value Analysis

Solve iteratively:

11 mm

j

L jW j

, where m m m m j mL j j W j j

1

m k

i mi

m

W i

throughput

Begin with 1

1

j

W j

Chapter 8 20

M/G/1Best combination of tractability & usefulness

• Assumption of Poisson arrivals may be reasonable based on Poisson approximation to binomial distribution

– many potential customers decide independently about arriving (arrival = “success”),

- each has small probability of arriving in any particular time interval

• Probability of arrival in a small interval is approximately proportional to the length of the interval – no bulk arrivals

• Amount of time since last arrival gives no indication of amount of time until the next arrival (exponential – memoryless)

Chapter 8 21

M/G/1Best combination of tractability & usefulness

• Exponential distribution is frequently a bad model for service times– memorylessness– large probability of very short service times with occasional very

long service times

• May not want to use one of the “standard” distributions for service times, either– in a real situation, collect data on service times and fit an empirical

distribution

• Distributions of number of customers in the system and waiting time depend on service time distribution to be specified

Chapter 8 22

M/G/1Best combination of tractability & usefulness

• Assumption of Poisson arrivals may be reasonable based on Poisson approximation to binomial distribution

– many potential customers decide independently about arriving (arrival = “success”),

- each has small probability of arriving in any particular time interval

- Distributions of number of customers in the system and waiting time depend on service time distribution

Chapter 8 23

M/G/1 PerformanceHow many customers? How much time?

S is the length of an arbitrary service time (random variable)

is the arrival rate of customers; define = E[S] and assume it is < 1.

Expected values can be found from generalizing Little’s formula from # customers in the system to amount of work in the system:

An arriving customer brings S time units of work:

The time average amount of work in the system (V)

= * the customer average amount of work remaining in the system

Chapter 8 24

Work ContentWQ* is the (random variable) waiting time in queue

Expected amount of work per customer is

If a customer’s service time

is independent of own wait in queue, get average work in system

*

0

2

*

2

S

Q

Q

E SW S x dx

E SE SW

Workremaining

S

Begin service

Enter Depart

2

*

2Q

E SV E S E W

Chapter 8 25

Mean waiting time

WQ = Customer mean waiting time = average work in the system

when a customer arrives

From PASTA, WQ = V. Therefore, (Pollaczek-Khintchine formula)

And the other measures of performance are:

2 2

2 2 1Q Q Q

E S E SW E S W W

E S

2 2

, ,2 1Q Q Q

E SL W W W E S L W

E S

Chapter 8 26

Priority Queues

Different types of customers may differ in importance.

• Type i customers arrive according to a Poisson process with rate i and require service times with distribution Gi, i = 1, 2.

• Type 1 customers have (nonpreemptive) priority: – service does not begin on a type 2 customer if there is a type 1

customer waiting.

– If a type 1 customer arrives during a type 2 service, the service is continued to completion.

What is the average wait in queue of a type i customer, i

QW

Chapter 8 27

Two customer types w/o priority

M/G/1 model with

Average work in system is

If the server is not allowed to be idle when the system is not empty, this quantity is the same for the system with priority.

1 21 2 1 2G x G x G x

2 221 1 2 2

1 1 2 2

2 21 1 2 2

1 1 2 2

2 1 2 1

2 1

E S E SE SV

E S E S E S

E S E S

E S E S

Chapter 8 28

Two customer types with priority

Let Vi be the average amount of type i work in the system

Now focus on a type 1 customer. Waiting time = amt. of type 1 work in system + amt. of type 2 work in service when this customer arrives, so

2

2

i ii ii i Q

E SV E S W

in queue in servicei

QV iSV

2 2

1 1 2 21 1 2 11 1 2 2Q S Q

E S E SW V V E S W

Chapter 8 29

Two customer types with priority

2 2

1 1 2 211 1

1 1

if 12 1Q

E S E SW E S

E S

But a type 2 customer has to wait for everyone ahead, plus any type 1 customers who arrive during the type 2 wait, so

2 21 1 2 22 2 2

1 11 1 2 2 1 1

1 1 2 2

2 1 1

if 1

Q Q Q

E S E SW V E S W W

E S E S E S

E S E S

Chapter 8 30

M/M/k Model

k identical machines in parallel, Poisson arrivals (rate ), exponential service times (rate )

CTMC (birth-death) model:

Chapter 8 31

M/M/k Steady-State Probabilities

Level-crossing equations:

Define =/k, solve for Pn in terms of P0

Then use the facts that

to get

1

1

1 , 0,1,..., 1

, , 1,...n n

n n

P n P n k

P k P n k k

( )0!

0!

, 0,1,...,

, 1, 2,...

n

k n

kn

n kk

P n kP

P n k k

1

0 01, (1 ) if 1n

nn nP r r r

1

1

0 0

( ) ( ) if 1

! (1 ) !

n kk

n

k kP

n k

is the utilizationof each server

Chapter 8 32

M/M/k Performance Measures

Steady-state expected number of customers in the system

Mean flow time

Expected waiting time

Expected number in the queue (k is expected number of busy servers)

020

( ), if 1

! (1 )

k

nn

kL nP k P

k

0

1 1 ( )= by Little's Formula

- !(1 )

kL kW P

k k

1QW W

Q QL W L k

Chapter 8 33

Erlang Loss System

M/M/k/k system: k servers and a capacity of k: an arrival who finds all servers busy does not enter the system (is lost)

Above is called Erlang’s loss formula, and it holds for M/G/k/k as well, if

0

( ), 0,1,...,

!

n

n

kP P n k

n

0

( ) ( ), 0,...,

! !

i nk

i n

k kP i k

i n

k E S