the rg-factorizations in stochastic models
DESCRIPTION
The RG-Factorizations in Stochastic Models. Dr. Quan-Lin Li Department of Industrial Engineering Tsinghua University Beijing 100084, P.R. China. Outline of this talk. Why to need the RG-factorizations How to construct the RG-factorizations How to apply the RG-factorizations - PowerPoint PPT PresentationTRANSCRIPT
The RG-Factorizations The RG-Factorizations in Stochastic Modelsin Stochastic Models
Dr. Quan-Lin LiDr. Quan-Lin Li
Department of Industrial EngineeringDepartment of Industrial Engineering
Tsinghua UniversityTsinghua UniversityBeijing 100084, P.R. ChinaBeijing 100084, P.R. China
Why to needWhy to need the RG-factorizations the RG-factorizations
How to constructHow to construct the RG-factorizationsthe RG-factorizations
How to applyHow to apply the RG-factorizations the RG-factorizations
Promising issuesPromising issues in the future in the future
Outline of this talkOutline of this talk
Why to needWhy to need From 1996 to 2000, my research focuses on quasi-From 1996 to 2000, my research focuses on quasi-
stationary distributions of stochastic modelsstationary distributions of stochastic models
Our main problem is described as follows:
Our Question: How to compute?
Discrete
is a transition probability matrix Continuous
is an infinitesimal generator 1
P
PQ
Qe
Why to needWhy to need
When the size of the matrix is finite, this computation is similar to that for the stationaryprobability vectors of the finite-state Markovchains by using systems of linear equations
-classifica
P
tion of state; solving ; is the convengence radious
0 for all the other cases
Why to needWhy to need
When the size of the matrix is infinite, this computation will become different and difficult
Need to consider the existence Need to consider the uniqueness:
There are over one quasi-stationar
P
y distributions No available expression
No effective approach
Why to needWhy to need
Our work from 1996 to 1999 was to Our work from 1996 to 1999 was to develop thedevelop the LU-block-decompositionLU-block-decomposition for for Markov chains of M/G/1 type Markov chains of M/G/1 type and GI/M/1 typeand GI/M/1 type
Our method is different from Our method is different from that used by Bean, Latouche, that used by Bean, Latouche, Taylor etc.Taylor etc.
Why to needWhy to need
1 0
2 1 0
3 2 1 0
4 3 2 1 0
1 2 3 4
0 1 2 3
0 1 2
0 1
GI/M/1 type
M/G/1 type
B BB A A
P B A A AB A A A A
B B B BB A A A
P A A AA A
Why to needWhy to needThe LU-block-decomposition is
Our Question: Such a solution is OK?
Discrete
Continuous
Our computation is given by 0
Let . Then 0
Based on this, we can give a solution
I P L U
Q L U
L U
x L xU
Why to needWhy to need For a special Markov chain, we For a special Markov chain, we
obtained two differentobtained two different LU-block-LU-block-decompositiondecompositions, which lead to two s, which lead to two different expressions different expressions
One of them is correctOne of them is correct and is the and is the same as that in the literature; while same as that in the literature; while another is wronganother is wrong
Why?
Why to needWhy to need
We analyzed many real examples We analyzed many real examples and then found the main reasonsand then found the main reasons
These computations motivate us to These computations motivate us to
extend theextend the LU-block-decompositionLU-block-decomposition to the to the RG-factorizationRG-factorization
Why to needWhy to need
For an arbitary irreducible Markov chain, the RG-factorization is given by
Discrete
Continuous
Two different LU-decompositions ar
U D L
U D L
I P I R I I G
Q I R I G
e
, ;
,
Our computation is given by
0
U D L
U D L
U D L
L I R I U I G
L I R U I I G
I R I I G
Why to needWhy to need
For this computation
0
How to take the vector
A key observation:When is -positive recurrent with = ,
U D L
U
U D
I R I I G
I Rx
I R I
P
we should use
All the other cases, we should use
U
U D
x I R
x I R I
??
Our ComparisonsOur Comparisons Utility of the Utility of the RG-factorization RG-factorization is is
related to the classification of related to the classification of state by means of state by means of the diagonal the diagonal matrixmatrix, and keep effective , and keep effective computationscomputations
Better than Better than LU-decompositionLU-decomposition
How to constructHow to construct
0,0 0,1 0,
1,0 1,1 1,
,0 ,1 ,
Consider a discrete-time Markov chain
where or
Let 0, , and 1, , . Then
N
N
N N N N
c
c
c
P P PP P P
P
P P P
N N
E n E n N
E EP E T U
E V W
How to constructHow to construct
0
We can have two types of censored Markov chains:UL-type: To the level set
which leads to the UL-type RG-factorization
LU-type: To the level set
n k
k
c
n
E
P T U W V
E
P W
1
which yields the LU-type RG-factorization
V I T U
The UL-type RG-factorizationThe UL-type RG-factorization
0,0 0,1 0,
1,0 1,1 1,
,0 ,1 ,
,
1
, ,
1, ,
We write
We define
U-measure: , 0,
R-measure: , 0 ,
G-measure: , 0
n n nn
n n nn n
n n nn n n n
nn n n
ji j i j j
ii j i i j
P
n
R I i j
G I
.j i
The UL-type RG-factorizationThe UL-type RG-factorization
0,1 0,2 0,3
1,2 1,3
2,3
For an abitrary irreducible Markov chain , the UL-typeRG-factorization is given by ,
where0
0
0
U D L
U
P
I P I R I I G
R R RR R
RR
0 1 2 3
1,0
2,0 2,1
3,0 3,1 3,2
, , , ,
00
00
D
L
diag
GG G G
G G G
The UL-type RG-factorizationThe UL-type RG-factorization
0
0
0
Important Properties for Censoring Structure: is irreducible if is irreducible; is positive recurrent if is recurrent; is transient if is transient;
is transient for all 1.k
PP
P
k
Some special casesSome special cases
0,1
1,2 1,0
2,3 2,1
1,0
2,1
0,1
1
The QBD processes0 0
0 0 ,
0 0
The M/G/1 type0
0
0
The GI/M/1 type0
0
U L
L
U
RR G
R GR G
GG
G
RR
R
,2
2,30 R
Some special casesSome special cases
0,1 0,2 0,3
1 2
1
0
1,0
2,0 1
3,0 2 1
The GI/G/1 type0
0
0
, , , ,
00
00
U
D
L
R R RR R
RR
diag
GG G G
G G G
How to applyHow to apply
0
0 0
0
0
Let . Then 0.
Observating a non-zero nonnegative solution , 0,0,0, ,
where is the stationary probability vector of
Therefore, = ,0,0,0,
U D D
U D L
U
I R I I G
x I R x I I G
x x
x
x I R
1
0 01
0
yields
,
, 1.k
k i k ii
x
R k
RemarksRemarks
Computing the stationary probability vector of the Markov chain with a huge state space or an infinite state space is decomposited into two steps:
Step one: Computing the stat
P
0
,
ionary probability vector of the censored chain with a smaller state space
Step two: Computing the R-measure for 0
by using the above iterative relations.i jR i j
Finite statesSmaller
Finite statesHuge
Infinite states
The UL-type RG-factorization
A crucial advanceA crucial advance
The LU-type RG-factorizationThe LU-type RG-factorization
, , 1 , 2
1, 1, 1 1, 2
2, 2, 1 2, 2
,
1
, ,
We write
We define
U-measure: , 0,
R-measure: , 0 ,
G-measure:
n n nn n n n n n
n n nn n n n n n n
n n nn n n n n n
nn n n
ji j i j j
P
n
I j i
R
1, , , 0 .i
i j i i jI i j G
1,0
2,0 2,1
3,0 3,1 3,2
For an abitrary irreducible Markov chain , the LU-typeRG-factorization is given by ,
where0
0 0
0
L D U
L
P
I P I I I
R G
RR R R
R R R
0 1 2 3
0,1 0,2 0,3
1,2 1,3
2,3
, , , ,
00
0
D
U
diag
G G GG G
GG
The LU-type RG-factorizationThe LU-type RG-factorization
The LU-type RG-factorizationThe LU-type RG-factorization
1 1 1 1
1 11 1
2 3
2
is transient for all 0. The matrix or of size must be invertible,
An example,
1 1 12 2 2
1 1 1 2 2 2
k
U D L
U D L
kI P Q
I P I I I
Q I I
P
G R
G R
3
2 31 1 12 2 2
Some special casesSome special cases
0,1
1,0 1,2
2,1
0,1
1,2
1,0
2,1
The QBD processes0 0
0 0 ,
0 0
The M/G/1 type0
0
0
The GI/M/1 type0
0
0
U L
L
U
GR G
R GR
GG
G
RR
R
How to applyHow to applyThe LU-type RG-factorization is different from the UL-typecase. It may be used to deal with the first passage times and the sojourn times. In addition, we provide a better example:Consider a perturbed
| 0
| 0
Markov chain . Let and be the stationary probability vectors of and , respectively.
Then
which leads to
P PP P
Pd I P
d
dd
1 1 1U D LI I I G R
Comparison for UL- and LU-typeComparison for UL- and LU-type
Systems of linear equations 0or 0
xA
Ax
Systems of linear equations ( 0)or ( 0)
xA b b
Ax b b
UL-type RG-factorization
LU-type RG-factorization
Question:
Question: Systems of linear equations
Systems of linear equations
Our work on the RG-factorizationsOur work on the RG-factorizations
The RG-factorizations
The RG-factorizations
Quasi-stationary distributions
TheoryTailed analysis
Sensitive analysis
Markov Reward processes
Markov Decision processes
Evolutionary games
Applications
Networking safetyComputer networksProduction systemsReal-time management
Promising problems (1)Promising problems (1)
For the RG-factorizations:For the RG-factorizations:1. 1. It is interesting to consider the It is interesting to consider the dd-period for the -period for the
R-, U- and G-measures. For exampleR-, U- and G-measures. For example
(1)(1) A A = = AA0 0 + + AA1 1 + + AA2 2 is irreducible and is is irreducible and is dd-period, -period,
the two matrices the two matrices RR and and GG are are dd-period-period ??(2) (2) For a Markov chain of GI/G/1 type, what For a Markov chain of GI/G/1 type, what
happen to happen to ?? Such a work is useful for tailed analysisSuch a work is useful for tailed analysis
0 0
0 0
Discrete time
,
Continuous time
0, 0
k kk k
k k
k kk k
k k
R R A G A G
R A A G
Promising problems (2)Promising problems (2)
For the RG-factorizations:For the RG-factorizations:1.1. It is interesting to consider spectral analysis It is interesting to consider spectral analysis
for the R-, U- and G-measures. for the R-, U- and G-measures.
When When A A = = AA0 0 + + AA1 1 + + AA2 2 is irreducible and is is irreducible and is infinite size, how to analyze the spectral of infinite size, how to analyze the spectral of
the two matrices the two matrices RR and and G G ??2. 2. For a Markov chain of GI/G/1 type, what For a Markov chain of GI/G/1 type, what
happen to happen to ??
Promising problems (3)Promising problems (3)
, , , ,1
, , , ,1
, ,
To construct the RG-factorization, we have formedmany useful relations such as Winner-Holp equations
i j j i j i k k k jk j
i i j i j i k k k jk j
n n n n k
R I P R I G
I G P R I G
P R I
,1
Effective algorithms are necessary to compute the R-, U- and G-measures, and then compute performancemeasures of a stochastic models.
k k nk n
G
Promising problems (4)Promising problems (4)
0
Transient Performance:Continuous-time Markov chain: or
0 exp 0 exp
Continuous-time Markov reward process: ,
t
Q Q t
d dt t Q t t Q tdt dt
t Qt t Q u du
Q f X
0
or
, , , , ,
, , ,t
t
t
u i t
R
t f X du H t x P t x X i t x
t x t x R t x Qx
Train repairable networksTrain repairable networks
GM manufacturer
GM manufacturerPart Supplier
Part Supplier
Container Park
GM manufacturer
GM manufacturer
Container Park
Part Supplier
Real-time managementReal-time management
LocalOptimization
LocalOptimization
Global Optimization
Queueing
Networks
Information Theory
Thanks for youThanks for youand and
questions ?questions ?