flows and networks (158052)
DESCRIPTION
Introduction to theorie of flows in complex networks: both stochastic and deterministic apects Size 5 ECTS 32 hours of lectures : 16 R.J. Boucherie focusing on stochastic networks 16 W. Kern focusing on deterministic networks Common problem - PowerPoint PPT PresentationTRANSCRIPT
Flows and Networks (158052)
Richard BoucherieStochastische Operations Research -- TW
wwwhome.math.utwente.nl/~boucherierj/onderwijs/158052/158052.html
Introduction to theorie of flows in complex networks: both stochastic and deterministic apects
Size5 ECTS
32 hours of lectures : 16 R.J. Boucherie focusing on stochastic
networks 16 W. Kern focusing on deterministic networks
Common problemHow to optimize resource allocation so as to
maximize flow of items through the nodes of a complex network
Material: handouts / downloads
Exam: exercises / (take home) exam
References: see website
Motivation and main question
Motivation
Production / storage system
Internet http://www.warriorsofthe.net/
trailer
Road traffic
Main questions
How to allocate servers / capacity to nodes orhow to route jobs through the systemto maximize system performance, such as throughput, sojourn time, utilization
QUESTIONS ??
Aim: Optimal design of Jackson network
• Consider an open Jackson network
with transition rates
• Assume that the service rates and arrival rates
are given
• Let the costs per time unit for a job residing at queue j be
• Let the costs for routing a job from station i to station j be
• (i) Formulate the design problem (allocation of routing
probabilities) as an optimisation problem.
• (ii) Provide the solution to this problem
kk
jjj
jkjjk
pnTnq
pnTnq
pnTnq
000
00
))(,(
))(,(
))(,(
ja
jkb
0j
Flows and network: stochastic networks
Contents
1. Introduction; Markov chains
2. Birth-death processes; Poisson process, simple queue;reversibility; detailed balance
3. Output of simple queue; Tandem network; equilibrium distribution
4. Jackson networks;Partial balance
5. Sojourn time simple queue and tandem network
6. Performance measures for Jackson networks:throughput, mean sojourn time, blocking
7. Application: service rate allocation for throughput optimisationApplication: optimal routing
Today:
• Introduction / motivation course• Discrete-time Markov chain• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Today:
• Introduction / motivation course• Discrete-time Markov chain• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Markov chain
• Xn n=1,2,… stochastic process
• State space : all possible states
• Transition probability
• Markov property
• time-homogeneous
• Property
)|(),( 1 iXjXPjip nn
},...,1,0{ NS
Nijip
NjijipN
j
0,1),(
,0,0),(
0
),...,|( 1111 jXjXjXP nnnn
)|(),...,|(
11
1111
nnnn
nnnn
jXjXPjXjXjXP
Markov chain : equilibrium distribution
• n-step transition probability
• Evaluate:
• Chapman-Kolmogorov equation
• n-step transition matrix
• Initial distribution
• Distribution at time n
• Matrix form
),()|()|( 0 jiPiXjXPiXjXP nnmnm
),(),(),(1
2 jkpkipjiPN
k
),(),(),(1
jkPkiPjiP mn
N
kmn
),)((),( jiPjiPPP nn
nn
)()( 0 iqiXP ))(),..,1(( Nqqq
),()()()(1
jkPkqjXPjp n
N
knn
nnnn PNpp qp ))(),..,1((
Markov chain: classification of states
• j reachable from i if there exists a path from i to j• i and j communicate when j reachable from i and
i reachable from j • State i absorbing if p(i,i)=1• State i transient if there exists j such that j
reachable from i and i not reachable from j • Recurrent state i process returns to i infinitely
often = non transient state• State i periodic with period k>1 if k is smallest
number such that all paths from i to i have length that is multiple of k
• Aperiodic state: recurrent state that is not periodic
• Ergodic Markov chain: alle states communicate, are recurrent and aperiodic (irreducible, aperiodic)
007.0003.002.08.000001.04.05.000007.03.00000005.05.000006.04.0
P
Markov chain : equilibrium distribution
• Assume: Markov chain ergodic
• Equilibrium distribution
independent initial state
stationary distribution
• normalising
interpretation probability flux
)(),(lim)|(lim 0 jjiPiXjXP nnnn
)(),(),(1 jjiPjiP nn
),(),(),(1
1 jkpkiPjiP n
N
kn
),()()(1
jkpkjN
k
n
nPP qππ
lim
1)(1
kN
k
SjjjtXPSjjjXP
),())((
),())0((
Discrete time Markov chain: summary
• stochastic process X(t) countable or finite state space SMarkov property
time homogeneous independent tirreducible: each state in S reachable from any other state in Stransition probabilities
Assume ergodic (irreducible, aperiodic) global balance equations (equilibrium eqns)
solution that can be normalised is equilibrium distributionif equilibrium distribution exists, then it is unique and is limiting distribution
))(|)(())(,...,)(|)((
11
1111
nnnn
nnnn
jtXjtXPjtXjtXjtXP
))(|)(( jtXktXP
))(|)1((),( jtXktXPkjp
1),(
kjpSk
).()()( jkpkjSk
)())0(|)((lim kjXktXPt
Today:
• Introduction / motivation course• Discrete-time Markov chain• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Continuous time Markov chain• stochastic process X(t)
countable or finite state space S
Markov property
transition probability
irreducible: each state in S reachable from any other state in S
Chapman-Kolmogorov equation
transition rates or jump rates
))(|)(())(...,)(,)(|)(( 11
itXjstXPjtXjtXitXjstXP nn
))0(|)((),( iXjtXPjiPt
),(),(),( jkPkiPjiP stk
st
jihjiPjiq h
h
),(lim),(0
)(),(),( hohjiqjiPh
Continuous time Markov chain
• Chapman-Kolmogorov equation
transition rates or jump rates
• Kolmogorov forward equations: (REGULAR)
Global balance equations
),(),(),( jkPkiPjiP stk
st
jihjiPjiq h
h
),(lim),(0
)],()(),()([0
)],(),(),(),([),('
)],(),(),(),([
]1),()[,(),(),(),(),(
),(),(),(
kjqjjkqk
kjqjiPjkqkiPjiP
kjPjiPjkPkiP
jjPjiPjkPkiPjiPjiP
jkPkiPjiP
jk
ttjk
t
hthtjk
hthtjk
tht
htk
ht
Markov jump chain
• Hier tranparant met sprongketen, is nodig in bewijs verderop.
Continuous time Markov chain: summary
• stochastic process X(t) countable or finite state space SMarkov property
transition rates independent t
irreducible: each state in S reachable from any other state in SAssume ergodic and regular
global balance equations (equilibrium
eqns)
π is stationary distribution
solution that can be normalised is equilibrium distributionif equilibrium distribution exists, then it is unique and is limiting distribution
)],()(),()([0 kjqjjkqkjk
)())0(|)((lim kjXktXPt
))(|)(())(...,)(,)(|)(( 11
itXjstXPjtXjtXitXjstXP nn
jihjiPjiq h
h
),(lim),(0
Today:
• Introduction / motivation course• Discrete-time Markov chain• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Birth-death process• State space
• Markov chain, transition rates
• Bounded state space:
q(J,J+1)=0 then states space bounded above at J
q(I,I-1)=0 then state space bounded below at I
• Kolmogorov forward equations
• Global balance equations
otherwisejkjj
ratedeathjkjratebirthjkj
kjq
0)()(
1)(1)(
),(
ZS
)1()1()]()()[()1()1(0
)1()1,()]()()[,()1()1,(),(
jjjjjjj
jjiPjjjiPjjiPdtjidP
tttt
Example: pure birth process• Exponential interarrival times, mean 1/
• Arrival process is Poisson process• Markov chain? • Transition rates : let t0<t1<…<tn<t
• Kolmogorov forward equations for P(X(0)=0)=1
• Solution for P(X(0)=0)=1
jkjk
kjq
hohjtXjhtXPhojtXjhtXP
hohjtXjhtXPjntnXjtXjtXjhtXP
1),(
)(1))(|)(()())(|2)((
)())(|1)(())(,...,0)0(,)(|1)((
),0()0,0(
),0()1,0(),0(
jPdt
dP
jPjPdtjdP
tt
ttt
0,...,2,1,0,!)(),0( tjejtjP tj
t
Example: pure death process• Exponential holding times, mean 1/
• P(X(0)=N)=1, S={0,1,…,N}
• Markov chain? • Transition rates : let t0<t1<…<tn<t
• Kolmogorov forward equations for P(X(0)=N)=1
• Solution for P(X(0)=N)=1
jkjjkj
kjq
hohjjtXjhtXPhojtXjhtXP
hohjjtXjhtXPjntnXjtXjtXjhtXP
1),(
)(1))(|)(()())(|2)((
)())(|1)(())(,...,0)0(,)(|1)((
),0(),(
),0()1,0()1(),(
NPNdtNNdP
jPjjPjdt
jNdP
tt
ttt
0,,...,2,1,0,1),(
tNjeejN
jNPjNtjt
t
Simple queue• Poisson arrival proces rate , single server
exponential service times, mean 1/
• Assume initially empty:
P(X(0)=0)=1, S={0,1,2,…,}
• Markov chain? • Transition rates :
0,0,][
0,11
),(
)(][1))(|)(()())(|1)(()())(|1)((
jjkjjkjjk
jk
kjq
hohhjtXjhtXPhohjtXjhtXPhohjtXjhtXP
Simple queue• Poisson arrival proces rate , single server
exponential service times, mean 1/
• Kolmogorov forward equations, j>0
• Global balance equations, j>0
0,0,][
0,11
),(
jjkjjkjjk
jk
kjq
)1()0(0)1(])[()1(0
)1,()0,()0,(
)1,(])[,()1,(),(
jjj
iPiPdtidP
jiPjiPjiPdtjidP
ttt
tttt
Simple queue (ctd)
j j+1
Equilibrium distribution: <
Stationary measure; summable eq. distrib.
Proof: Insert into global balance
Detailed balance
j
jj
)/)(/1(
)/)(0()(
).1()1()1,()( jjqjjjqj
Birth-death process• State space
• Markov chain, transition rates
• Definition: Detailed balance equations
• Theorem: A distribution that satisfies detailed balance is a stationary distribution
• Theorem: Assume that
then
is the equilibrium distrubution of the birth-death prcess X.
0,)0(0,)()(
0,1)(1)(
),(
jjkjjkjj
ratedeathjjkjratebirthjkj
kjq
,...}2,1,0{ NS
).1()1()1,()( jjqjjjqj
1
1 )1,(),1()0(
rrqrrqj
rSj
Sjrrqrrqj
j
r
,)1,(),1()0()(
1
Today:
• Introduction / motivation course• Discrete-time Markov chain• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Reversibility; stationarity• Stationary process: A stochastic process is
stationary if for all t1,…,tn,
• Theorem: If the initial distribution is a stationary distribution, then the process is stationary
• Reversible process: A stochastic process is reversible if for all t1,…,tn,
NOTE: labelling of states only gives suggestion of one dimensional state space; this is not required
))(),...,(),((~))(),...,(),(( 2121 nn tXtXtXtXtXtX
1)(
jSj
))(),...,(),((~))(),...,(),(( 2121 nn tXtXtXtXtXtX
Reversibility; stationarity• Lemma: A reversible process is stationary.
• Theorem: A stationary Markov chain is reversible if and only if there exists a collection of positive numbers π(j), jS, summing to unity that satisfy the detailed balance equations
When there exists such a collection π(j), jS, it is the equilibrium distribution
• Proof
Skjjkqkkjqj ,),,()(),()(
Lemma 1.9 / Corollary 1.10:
If the transition rates of a reversible Markov process with
state space S and equilibrium distribution are
altered by changing q(j,k) to cq(j,k) for
where c>0 then the resulting Markov process is
reversible in equilibrium and has equilibrium distribution
where B is the normalizing constant.
If c=0 then the reversible Markov process
is truncated to A and the resulting Markov
process is reversible with equilibrium distribution
Truncation of reversible processes
Sjj ),(
10
ASkAj \,
ASjjBcAjjB\)(
)(
Ajkj
Ak
)()(
A
S\A
Time reversed processX(t) reversible Markov process X(-t) also, butLemma 1.11: tijdshomogeneity not inherited for
non-stationary process
Theorem 1.12 : If X(t) is a stationary Markov process with transition rates q(j,k), and equilibrium distribution π(j), jS, then the reversed processX(-t) is a stationary Markov process with transition rates
and the same equilibrium distribution
Theorem 1.13: Kelly’s lemmaLet X(t) be a stationary Markov processwith transition rates q(j,k). If we can find a collection of numbers q’(j,k) such that q’(j)=q(j), jS, and a collection of positive numbers (j), jS, summing to unity, such that
then q’(j,k) are the transition rates of the time-reversed process, and (j), jS, is the equilibrium distribution of both processes.
)(),()(),('
jjkqkkjq
Skj ,
),(')(),()( jkqkkjqj Skj ,
Kolmogorov’s criteria• Theorem 1.8:
A stationary Markov chain is reversible iff
for each finite sequence of states
Notice that
),(),()...,(),(),(),()...,(),(
122311
113221
jjqjjqjjqjjqjjqjjqjjqjjq
nnn
nnn
)0,(),(),()...,(),(),(),()...,(),(),0()0()(
112231
132211
jqjjqjjqjjqjjqjjqjjqjjqjjqjqj
nnn
nnn
Sjjj n ,...,, 21
Today:
• Introduction / motivation course• Discrete-time Markov chain• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Summary / next:
Basic queueing model; basic tools• Markov chains• Birth-death process• Simple queue• Reversibility, stationarity• Truncation• Kolmogorov’s criteria
• Nextinput / output simple queuePoisson procesPASTAOutput simple queueTandem netwerkJackson networkPartial balanceKelly/Whittle network
Exercises[R+SN] 1.1.2, 1.1.4, 1.1.5, 1.2.7, 1.2.8,
1.3.2, 1.3.3 (next time), 1.3.5, 1.3.6, 1.5.1, 1.5.2, 1.5.5, 1.6.2, 1.6.3, 1.6.4, 1.7.1, 1.7.8 (next time)
[N] 10.1,6,7,8,9,10,12,13,15