eurecom, sophia-antipolis thrasyvoulos spyropoulos / [email protected] introduction to...

Eurecom, Sophia-AntipolisThrasyvoulos Spyropoulos / [email protected]

Introduction to Continuous-Time Markov Chains and Queueing Theory

Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis

From DTMC to CTMC

DTMC1. Transitions at discrete time steps

n=0,1,2,…

2. Past doesn’t matter (Markov property) P(Xn+1|Xn,Xn-1,…X1) = P(Xn+1|Xn)

3. P(Xn+1 = j|Xn = i) = pij (stationarity)

2

0

p00 p01

1 2p10 p21

p12

… kk-1

pk,k-1

pk-1,k

k+1

pk,k-1

pk-1,k

…

CTMC1. Transitions between states can

happen any time

2. Past still doesn’t matter

3. Transition still independent of time t


CTMC: A Generic Definition

A CTMC is continuous time stochastic process {X(t), t ≥ 0}, s.t. for all s,t ≥ 0, and any i,j,x(u) P{X(t+s) = j|X(s) = i,X(u) = x(u), 0≤ u≤ s}= P{X(t+s) = j|X(s) = i} (Markov property)= P{X(t) = j| X(0) = i} (stationarity)

τi : time until CTMC leaves state i, if currently in i

Above def. implies: P{τi > t+s | τi > s} = P{τi > t} Q: What does this imply?

A: τi is exponentially distributed!

3


CTMC: Viewpoint 1

Step 1: When at state i, wait an exponentially distributed amount of time denote rate out of: vi

Step 2: When the process leaves i, it will enter j with probability pij

4

i

exp(vi)

l

k

pil

j

pik

pij


CTMC: Viewpoint 2 (more common)

When at state i: move to state j with rate λij (independently)

Q: Time to leave state i? first move to any state first phone rings

A: Exponential with rate (vi = λi1 + λi2 +…+λin = Σj λij)Q: Probability to go to state k, after leaving i?A:

5

0

3

2

λ03

1

λ02

λ01exp(λ01)

exp(λ02)

exp(λ03)

j ij

ijij λλpView 2 => View 1


A Single Server Example

6

Poisson (λ)e.g. 3 jobs/sec

service: exp(μ)e.g. 5 jobs/sec

0

λ

1 2

μ

…

λ

μ

λ

μ

Let πj = limt∞Pij(t) : limit probability of being in state j

Q: How do we determine πj ? how do we “solve” the chain?

A: Assume an equivalent DTMC “half-rigorous” argument


CTMC DTMC with time unit δ

Divide time into very small intervals δ Prob. of an arrival in the next interval δ : λδ + o(δ) Prob. of a departure in the next interval δ : μδ + o(δ) P{ii+1} = λδ (1-μδ) = λδ + o(δ) (1 arrival + 0

departure)

P{ii-1} = μδ (1- λδ) = μδ + o(δ) (0 arrival + 1 departure)

P{ii} = 1- λδ -μδ +o(δ) (no event with almost prob 1) we can ignore self-loops in the balance equations

7

0

λ

1 2

μ

…

λ

μ

λ

μ

0

λδ+o(δ)1 2

μδ+o(δ)…

λδ+o(δ)μδ+o(δ)

CTMC DTMC (time step δ 0)


Solving the Balance Equations for a CTMC Methodology

Step 1) Write balance equations

Step 2) Take limit as δ 0 For state 0 rate out = rate in

8

0

λδ+o(δ)1 2

μδ+o(δ)…

λδ+o(δ)μδ+o(δ)

ο(δ)μδ πο(δ)λδπ 10

δο(δ)μπlim δο(δ)λπlim 10δ00δ

δο(δ)μ πδο(δ)λπ 10

0110 πμλπμ πλπ

For state 1 rate out = rate in

ο(δ)μδπο(δ)λδ πο(δ)μδο(δ)λδπ 201

δο(δ)μπδο(δ)λ πδο(δ)μλπ 201

)01 πμλ(π

201 μπλ πμλπ

02200 πμλπμπλ πμλπμλ 2


Solving a CTMC – Our first Queueing Result! Lesson from previous example: can simply solve the

balance equations, rate in = rate out See book (MHB, CH.13) for generalization Can also try local balance equations Need to check for irreducibility (for πi to exist)

- What about periodicity??

Back to our queue)

Define ρ = λ\μ (utilization) (we derived earlier)

(normalize to get π0)9

Poisson (λ) service: exp(μ)

0n0n

n πρπμλπ

1

0n nπ ρ)ρπ nn 1(

Our first continuous time queueing result!!


Introduction to Queueing: M/M/1

The example so far was an M/M/1 system “M” stands for memoryless --- both arrivals and departures “1” for single server πn : number of customers in system (in service or queue)

Q: What is the probability that the server is empty?A: π0=1-ρQ: What is the expected number of customers?

A:

10


d

d 1( 1( 1(

0000

nnnn

n1-nnn ρρρ)ρρnρ)ρρ)ρnπn E[N]ρ)ρρ)ρ)ρρρρ)ρ 2

1(

E[N] 1(

1 1(

1

1

d

d 1(


M/M/1 continued

Q1: What happens when ρ 1 ? (i.e. system very busy)A: (queue goes to infinity!)

Q2: What happens to the Markov Chain when ρ ≥ 1 ?A: All states become transient πi = 0 (for all i)

Null-recurrent for ρ = 1 But this means we cannot apply balance equations (no

balance!) Could we have guessed it from the MC?? (homework!)

Q3: We know E[N]; What is the expected delay E[T]?A: Little’s Law will tell us that E[T] = E[N]/ λ

11

E[N] ρ1ρE[N] 1ρ

Eurecom, Sophia-AntipolisThrasyvoulos Spyropoulos / [email protected]

Introduction to Queueing Theory

12


Queueing Systems: Kendall’s Notation A queuing system is described by A/D/n/B/S A: the arrival process

M: for Memoryless Poisson arrivals D: Deterministic arrivals at fixed intervals G: generic distribution

D: service process M: Memoryless duration of service is exponentially

distributed D: Deterministic each service last the same fixed amount of

t G: generic distribution (e.g. uniform, pareto, gamma, etc.)

n: number of servers B: size of queue/buffer

If queue is full additional arrivals are rejected No B in notation infinite queue (e.g. M/M/1, M/G/1, G/G/4)

S: Scheduling policy FCFS (default assumed), preemptive, random, etc.

13


Queue Stability

For the system to be stable: λ < μQ: What happens if λ > μ?A: Queue size goes to infinity (and so does delay!)

Q: Can you give the intuition?A: Consider an interval [0,t] (large t)

Number in system = (arrivals in [0,t]) – (departures in [0,t]) Þ Number in system = λt – μt = (λ- μ)tÞ t -> ∞ queue size -> ∞

14



Throughput of a Queueing System

Q: Which of the two systems has a higher throughput?A: The throughput is the same for both: X = λ = 1/6 Consider a busy period of the system

Def: from time an arrival finds system empty, until it empties again

Assume A arrivals during a busy periodQ: How many departures during that busy period?A: Exactly A!

All arrived jobs should complete for the system to empty

15

Throughput X: jobs/min leaving the system

Open (stable) system: Throughput X = Input Rate


Little’s Law (or the most famous Queueing

Result)

E[N] : number of customers in the system (queue + server)

E[T]: time a customer spends in the system (queue+server)

Little’s Law: E[N] = λ E[T]

M/M/1:

(some) Intuition behind Little’s Law Customer X arrives at the end of a queue on average stays E[T] Q: how many customers have arrived while X was in the system? A: λ E[T] => on average a departure leaves E[N] = λ E[T] customers in the

system

16

Poisson (λ) service: exp(μ)M/M/1:

λμ1λ1ρ1ρλE[N]E[T]


Proof of Little’s Law

DEF: A is the area inside boxes A(t): num of arrivals by t D(t): num of departures by t

1. Sum times horizontally

2. Sum times vertically:

17

A(t)

1i iD(t)

1i i TAT(s)dsNA t

0

t TtN(s)dst TTN(s)dsT A(t)1i it0D(t)1i iA(t)1i it

0D(t)1i i

λE[T]E[N] λTNXT tA(t)A(t)TtN(s)dstD(t)D(t)T ergodicitytA(t)1i it

0D(t)1i i

area A

N(t1)=2

t1

N(t2)=3

t2


Generality of Little’s Law

Q: Did we assume anything about the number of servers?

A: No!

Q: Did we assume that scheduling is first-come first-serve (FCFS)?

A: No! In fact, in the previous example 3 finishes before 2!

Little’s Law applies to ANY system and ANY scheduling Many servers, many queues, queueing network, scheduling

policy

18


Little’s Law: Additional Results

Q: What about E[TQ]? number of jobs in queue – not server(s)

A: λE[TQ] = E[NQ] (similar proof)

Q: What about ρ (number of customers in service)?A: λE[S] = ρ ρ = λ/μ Q: What is ρ?A: System utilization -> fraction of time server(s) is busy

(time average)A: Ergodicity Prob{server is busy at a random time}

(ensemble average)

19


M/M/k/k loss system

Consider a call center with k total lines E.g. an actual call center, or switch with k circuits, or base

station with k channels

If all servers/lines busy call/job is rejected (Key Question): What is the probability of “blocking”?

20


(continuous-time) Markov Chain for M/M/k/k

21

Q: How many states do we need?A: k states

Q:What is the rate from i to i+1?A: λQ:What is the rate from i to i-1?A: kμ


Local balance equations for M/M/k/k

Q: Is the chain time-reversible (to apply local balance)? i.e. is rate from 12 = rate from 21?

A: Yes, it is so, we can apply local balance

22

0110 πμλ π μπλπ-

02

221 π2!1μλ π 2μπλπ-

0i

ii1-i πi!1μλ π iμπλπ-

k

i 0

11

i!1μλ π π i0i i

k

i 0 i!1μλk!1μλ πP i

kkblock

Erlang B


What about general service (M/G/k/k)?

Q: Job size/duration is not exponential?A: Turns out that the state probabilities πi is as in M/M/k/k

The block probability is the same and given by Erlang B

This is an important insensitivity result Result is insensitive to service distribution (only depends on

mean μ) Insensitivity results often arise when there is no queueing

23

k

i 0 i!1μλk!1μλ πP i

kkblock


M/M/k

24

Dispatch to first available server

queue if all busy

A very important model for server farms Amazon, Google, CNN, pretty much all major web

sites Cloud computing

Exponential service with rate μ

Poisson (λ)


CTMC for M/M/k

Q: How many states?A: Infinite states

Q: What is the rate from i i+1A: λQ: What is the rate from i i-1 (i ≤ k)A: iμQ: What is the rate from i i-1 (i > k)A: kμ

25

…


Local Balance Equations for M/M/k

26

…


M/M/k: Utilization and Queueing

Q: What is the expected number of busy servers?A: λ /μ- Definition: ρ = λ /kμ (utilization of a single server)

Q: What is the probability (PQ) an arriving job must queue?

A: PQ = P{arrival finds all servers busy}= P{arrival sees ≥ k jobs in system}(from PASTA property)

27

ki iπ ki

ki0k kμλπk!1μλ

kiki0

kk ρπk!kρ

i

i0kk ρπk!kρ

0

11k 0iki

00k

Q ρ)(1k!kρi!kρ πwhere ,πρ)-(1k!kρP

Erlang C:


M/M/k Queueing vs. M/M/k/k Blocking

Q: For which system (M/M/k and M/M/k/k) is the probability that all k server are busy higher?

A: For the M/M/k

Q: Why?A: In M/M/k jobs can arrive when all k servers are busy

these do not disappear but queue up more work for later In M/M/k/k these jobs are simply rejected Can confirm by comparing Erlang B and C (work the math as

HW)

28

kρρ1ρPE[N] Q

μ1λkμ1PλE[N]E[T] Q


(Back to) Comparing Channel Allocations

…or server organizations…or supermarket lines…

Q: Which is better in terms of E[T], FDM or M/M/1 (statistical multiplexing)

A: M/M/1 E[TFDM] = k/(kμ-λ) but E[TM/M/1] = 1/(kμ-λ) 29


(Back to) Comparing Channel Allocations (cont’d)

…or server organizations…or supermarket lines…

Q: What about E[TM/M/1] and E[TM/M/k]?

A:

30

ρ)k(1P]E[T ]E[T M/M/kQM/M/1M/M/k

ρ ≈ 0ρ ≈ 1k]E[T ]E[T

M/M/1M/M/k

1]E[T ]E[TM/M/1M/M/k

eurecom, sophia-antipolis thrasyvoulos spyropoulos / [email protected] introduction to...

Documents