numerical methods for stochastic networks peter w. glynn institute for computational and...
TRANSCRIPT
![Page 1: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/1.jpg)
Numerical Methods for Stochastic Networks
Peter W. GlynnInstitute for Computational and Mathematical Engineering
Management Science and Engineering Stanford University
Based on joint work with Jose Blanchet, Henry Lam, Denis Saure, and Assaf Zeevi
Presented at Stochastic Networks Conference, Cambridge, UKMarch 23, 2010
![Page 2: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/2.jpg)
Stochastic models:
Descriptive
Prescriptive
Predictive
![Page 3: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/3.jpg)
Today’s Talk:
• an LP based algorithm for computing the stationary distribution of RBM (Saure / Zeevi)
• a Lyapunov bound for stationary expectations (Zeevi)
• Rare-event simulation for many-server queues (Blanchet / Lam)
![Page 4: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/4.jpg)
Computing Steady-State Distributions for Markov Chains
![Page 5: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/5.jpg)
One Approach
![Page 6: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/6.jpg)
![Page 7: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/7.jpg)
An LP Alternative
![Page 8: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/8.jpg)
( ) ( )n
nx K
x h x c
( ) ("tightness")inf ( )c c
n n
nx K x K
cx
h x
![Page 9: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/9.jpg)
• Where does constraint
come from?
• We assume that we can obtain a computable bound on
i.e.
• This will come from Lyapunov bounds (later).
( ) ( )n
nx K
x h x c
( )h XE( )h X c E
![Page 10: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/10.jpg)
Application to RBM
Reflected Brownian Motion (RBM):
For some stochastic models, the LP algorithm is particularly natural and powerful
![Page 11: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/11.jpg)
dR
( :1 , )ijb i j d
![Page 12: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/12.jpg)
d ( ) d d ( ) d ( )X t t B t R Y t
jY 0jX
2
1 , 1
1, =
2
d d
i ij j iji i j ii i j i
b Rx x x x
DL
![Page 13: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/13.jpg)
![Page 14: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/14.jpg)
1int ( )
int ( )
min
s/t ( )( ) ( )( ) u
1, 0, 0
| |d
k jk
dk
d
k i k i i k jkj x Fx
k k jkx
u
p f x f x
p p
R
R
L D
![Page 15: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/15.jpg)
![Page 16: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/16.jpg)
(d )( )( )d
x f x c
R L
(d )( )( ).j
j jFx f x D
j
![Page 17: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/17.jpg)
Theorem: This algorithm converges as , in the sense that
as .
,m n
n
n
j
n
![Page 18: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/18.jpg)
Numerical Results
Smoothed marginal distribution estimates for the two-dimensional diffusion. The dotted line is computed via Monte Carlo simulation, and the solid line represents the algorithm estimates based on n = 50 and m = 4, incorporating smoothness constraints.
![Page 19: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/19.jpg)
• - valued Markov process with cadlag paths
• We say if there exists such that
is a –local martingale for each
( ( ) : 0)X X t t S
( )g AD k
0( ) ( ( )) ( ( ))d
tM t g X t k X s s
xP x S
0( ( )) ( ) ( ( ))d
h
x xg X h g x k X s s E E
![Page 20: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/20.jpg)
Computing Bounds on Stationary Expectations
• Main Theorem:
Suppose is non–negative and satisfies
Then,
( )g AD
sup( )( ) .x S
Ag x
(d )( )( ) 0.S
x Ag x
![Page 21: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/21.jpg)
Diffusion Upper Bound
• Suppose is and satisfies for , where and
where . Then,
d ( ) ( ( ))d ( ( ))d ( )X t X t t X t B t
0g 2C ( )( ) ( )g x h x c Ldx R 0h
2
1 ,
1( ) ( )
2
d
i iji i ji i j
x b xx x x
L
T( ) ( ) ( )b x x x
.h c
![Page 22: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/22.jpg)
Many-Server Loss Systems
![Page 23: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/23.jpg)
Many-Server Asymptotic Regime
![Page 24: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/24.jpg)
Simplify our model (temporarily):
• using slotted time• eliminate Markov modulation• discrete service time distributions with finite
support
Consider equilibrium fraction of customers lost in the network.
![Page 25: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/25.jpg)
Key Idea:
Many server loss systems behave identically
to infinite-server systems up to the time of
the first loss.
![Page 26: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/26.jpg)
Step 1:
![Page 27: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/27.jpg)
Step 2:
![Page 28: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/28.jpg)
Step 3:
![Page 29: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/29.jpg)
Step 3:
![Page 30: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/30.jpg)
Step 3:
![Page 31: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/31.jpg)
Crude Monte Carlo : 3.7 days
I.S. : A few seconds
![Page 32: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/32.jpg)
Network Extension
• Estimate loss at a particular station
• If the most likely path to overflow a given station does not involve upstream stations, previous algorithm if efficient
• If an upstream station does hit its capacity constraint, we have “constrained Poisson statistics” that need to be sampled
![Page 33: Numerical Methods for Stochastic Networks Peter W. Glynn Institute for Computational and Mathematical Engineering Management Science and Engineering Stanford](https://reader033.vdocuments.net/reader033/viewer/2022051216/5697bfd51a28abf838cad980/html5/thumbnails/33.jpg)
Questions?