beyond heavy-traffic regimes: universal bounds …stochnet/talks/gurvich16.pdfbeyond heavy-traffic...
TRANSCRIPT
Beyond heavy-traffic regimes:Universal bounds and controls for the single-server
(M/GI/1+GI) queue
Itai GurvichNorthwestern University
Junfei HuangChinese University of Hong Kong
Stochastic Networks 2016
1 / 33
The intuitive Derivation of a Brownian Queue
1Service ∼ ⋅ , , ∞Load
• The waiting time/workload process in the M/GI/1 queue:
W (t) = W (0) +
A(t)∑i=1
si − (t − I(t))
= W (0)− (1− ρ)t − (t − I(t)) +
A(t)∑i=1
si − ρt
(M/GI/1)
W (t) = W (0)− (1− ρ)t − (t − I(t)) +√λE[s2]B(t)
(Brownian Queue)2 / 33
Brownian approximations as a model
A tractable and useful tool in the modeler’s toolbox
Pricing in queues (e.g. Kim and Randhawa, 2015)
Competition between queues (e.g. Allon and Federgruen, 2008)
Contracting in services (e.g. Akan et. al. 2011)
Inventory Optimization (e.g. Allon and Van Mieghem 2010)
Initial Model→ difficult→ Brownian approximation ?→ Accurate
3 / 33
Example: Dual Sourcing (Inventory)
Net Inventory evolution (Mexico and China renewal inputs Sc , SM ):
I(t) = I(0) + Sc(t) + SM(T sM(t))− D(t),
T sM(t) =
∫ t
01{I(u) < s}ds.
Allon and Van Mieghem: Global Dual SourcingManagement Science 56(1), pp. 110–124, © 2010 INFORMS 121
Figure 5 Comparing the Brownian Allocation to the Allocation Optimized via Simulation
0.35
0.40
Scaled cost: Brownian vs. optimal TBS
0.21
0.25
Scaled China allocation: Brownianvs. optimal TBS
0.20
0.25
0.30
0.13
0.17
Scal
edco
st
Scal
edal
loca
tion
Relative China cost cC/cM
0.150.05
0.09
0 0.2 0.4 0.6 0.8 1.0
Relative China cost cC/cM
0 0.2 0.4 0.6 0.8 1.0
Simulated cost of Brownian prescription
� = 1 � = 10
Optimization by simulation
� = 100
� = 1 � = 100� = 10
C*
Asymptotic (analytical)
� = 10
Optimization by simulation
� = 1 � = 100
Asymptotic (analytical)*�C
Yet even for � = 1, the relative error in scaled costbetween the prescription and the optimal control wasless than 7%. The main implication is that the Brown-ian prescription is a good and useful approximation ofthe optimal strategic China allocation, even for smallvolumes.
8.3. Comparing the Square-Root Allocation to theBrownian Allocation
The left panel in Figure 6 shows the optimal capaci-ties ∗
C and ∗M in the Brownian model, as well as the
square-root approximation p. To evaluate cost differ-ences, we also solved first-order condition (8) for theoptimal Mexican capacity given C = p and denoteit by M�p. The right panel shows the optimal cost �C∗
Figure 6 Comparing the Square-Root Allocation �p to the Brownian Allocation �∗C
0.6
0.7
Scaled allocation: Square root vs. Brownian
Scal
ed a
lloca
tion
Scal
ed c
ost
Scaled cost: Square root vs. Brownian
0.2
0.3
0.4
0.5
0 0.2 0.4 0.6 0.8 1.0 0
*
0.2 0.4 0.6 0.8 1.00
0.1
0.6
0.3
0.4
0.5
0.2
Relative China cost cC /cM Relative China cost cC /cM
�C
�p
*�M, p
C(�p, �M, p)
*�M
C*
and the cost C�p� M�p� when using the square-rootformulae.
One can observe that the scaled square-root allo-cation p is a reasonable approximation of the exactscaled Brownian allocation ∗
C , but the error increasesas the China cost increases. Indeed, in our numeri-cal study, the allocation difference is about 10% andbelow 28% as long as the China cost advantageexceeds 10%. Keep in mind that the relative error onthe total allocation prescription �− ∗
C
√� depends on
the volume � and will be much smaller, especially as� increases. The same comment applies to the relativecost difference. The main implication is that the sim-ple square-root formulae provide a reasonable start-ing point for the strategic China allocation.
Dow
nloa
ded
from
info
rms.
org
by [
129.
105.
199.
194]
on
06 J
une
2016
, at 0
7:21
. Fo
r pe
rson
al u
se o
nly,
all
righ
ts r
eser
ved.
Allon and Van Mieghem: Global Dual SourcingManagement Science 56(1), pp. 110–124, © 2010 INFORMS 121
Figure 5 Comparing the Brownian Allocation to the Allocation Optimized via Simulation
0.35
0.40
Scaled cost: Brownian vs. optimal TBS
0.21
0.25
Scaled China allocation: Brownianvs. optimal TBS
0.20
0.25
0.30
0.13
0.17
Scal
edco
st
Scal
edal
loca
tion
Relative China cost cC/cM
0.150.05
0.09
0 0.2 0.4 0.6 0.8 1.0
Relative China cost cC/cM
0 0.2 0.4 0.6 0.8 1.0
Simulated cost of Brownian prescription
� = 1 � = 10
Optimization by simulation
� = 100
� = 1 � = 100� = 10
C*
Asymptotic (analytical)
� = 10
Optimization by simulation
� = 1 � = 100
Asymptotic (analytical)*�C
Yet even for � = 1, the relative error in scaled costbetween the prescription and the optimal control wasless than 7%. The main implication is that the Brown-ian prescription is a good and useful approximation ofthe optimal strategic China allocation, even for smallvolumes.
8.3. Comparing the Square-Root Allocation to theBrownian Allocation
The left panel in Figure 6 shows the optimal capaci-ties ∗
C and ∗M in the Brownian model, as well as the
square-root approximation p. To evaluate cost differ-ences, we also solved first-order condition (8) for theoptimal Mexican capacity given C = p and denoteit by M�p. The right panel shows the optimal cost �C∗
Figure 6 Comparing the Square-Root Allocation �p to the Brownian Allocation �∗C
0.6
0.7
Scaled allocation: Square root vs. Brownian
Scal
ed a
lloca
tion
Scal
ed c
ost
Scaled cost: Square root vs. Brownian
0.2
0.3
0.4
0.5
0 0.2 0.4 0.6 0.8 1.0 0
*
0.2 0.4 0.6 0.8 1.00
0.1
0.6
0.3
0.4
0.5
0.2
Relative China cost cC /cM Relative China cost cC /cM
�C
�p
*�M, p
C(�p, �M, p)
*�M
C*
and the cost C�p� M�p� when using the square-rootformulae.One can observe that the scaled square-root allo-
cation p is a reasonable approximation of the exactscaled Brownian allocation ∗
C , but the error increasesas the China cost increases. Indeed, in our numeri-cal study, the allocation difference is about 10% andbelow 28% as long as the China cost advantageexceeds 10%. Keep in mind that the relative error onthe total allocation prescription �− ∗
C
√� depends on
the volume � and will be much smaller, especially as� increases. The same comment applies to the relativecost difference. The main implication is that the sim-ple square-root formulae provide a reasonable start-ing point for the strategic China allocation.
Dow
nloa
ded
from
info
rms.
org
by [
129.
105.
199.
194]
on
06 J
une
2016
, at 0
7:21
. Fo
r pe
rson
al u
se o
nly,
all
righ
ts r
eser
ved.
Gad Allon, Jan A. Van Mieghem (2010). Global Dual Sourcing: Tailored Base-Surge Allocation toNear- and Offshore Production. Management Science 56(1):110-124. 4 / 33
Example: Admission Control to a Many-Server Queue
0.2
0.4
0.6
0.8
1
1.2
1.4
Kocaga, Yasar Levent and Ward, Amy R (2010). Admission control for a multi-server queue with
abandonment. Queueing Systems, 65(3):275–323.
Asymptotic optimality for ρ(λ) = 1 +O(
1√λ
)as λ (and N)→∞
Punchline: “Universally” accurate5 / 33
Utilization (regime) assumptions and consequences
1110
Patience ∼ exp 1
Service ∼ exp 100Load 1.1
Embedding Consequence (as λ, µ→∞)
ρ(λ) = 1 +1√λ
√λW λ(·)⇒ Reflected OU (critical load)
ρ(λ) ≡ 1.1√λ(W λ(·)− µ
λθ(ρ(λ)− 1))⇒ free OU (overload)
Universal process approx in Ward and Glynn (2003), Ward (2012)6 / 33
Sensitivity of the limit to patience modeling
Patience hazard rate
Mandelbaum Avishai and Sergey Zeltyn. (2013) Data-stories about (im)patient customers in
tele-queues. Queueing Systems 75(2), 115-146
With ρ(λ) = 1− β√λ
(critical load): E[W λ] = O(1/√λ)
7 / 33
Sensitivity of the limit to patience modeling
Consider the critically loaded M/M/1 + GI:
ρ(λ) = 1− β√λ
Finite patience drawn from a distribution Fa(·).
Limit Theorems differ by model
Fλa ≡ F does not scale with λ and has fa(0) > 0:
diffusion limit has linear drift; Ward and Glynn (2005);
Fλa has hazard rate that scales with λ: Fλ
a (x) = 1− e−∫ x
0 h(√λu)du,
limit has non-linear drift; Reed and Ward (2008).
8 / 33
Sensitivity of scaling to patience modeling
Suppose ρ(λ) = 1 (critical loading).
Fa=exponential (fixed):
E[W λ] = Θ
(1√λ
), as λ→∞.
Fa(x) = x2 for x ∈ [0,1] (fixed):
E[W λ] = Θ
(1λ1/3
), as λ→∞.
Different patience dist. → different scaling needed for limits.
A result that bypasses case-by-case analysis and interpretation..
9 / 33
The M/GI/1 + GI queue
1Service ∼ ⋅ ,Load
Patience ∼ ⋅
• The virtual wait V (t) is the time an infinitely patient customer,arriving at time t , would have to wait.• The waiting time is the minimum of the virtual wait and the
customer’s patience:
W (t) = min(ν,V (t)).
• The first order (“fluid”) proxy for the stationary virtual wait is w thatsolves
µ ∧ λ = λF (w).10 / 33
Dynamics of the virtual waiting time
• V (t) is the work contained in jobs that will not abandon.
V (t) = V (0) +
A(t)∑i=1
si1{vi>ωi} − (t − I(t)).
• Satisfies the natural positivity properties,- V (t) ≥ 0, ∀t ≥ 0;- I(·) is nondecreasing with I(0) = 0;
-∫ ∞
01{V (s)>0}dI(s) = 0.
Process limits for the GI/GI/1+GI queue: Ward and Glynn (2003,2005),Reed and Ward (2008), Jennings and Reed (2012)
11 / 33
The intuitive Brownian queue
V (t) = V (0) +
A(t)∑i=1
si1{vi>ωi} − (t − I(t))
= V (0) +
∫ t
0ρFa(V (s))ds − t +
A(t)∑i=1
si1{vi>ωi} −∫ t
0ρFa(V (s))ds
+ I(t)
V (t) = V (0) +
∫ t
0ρFa(V (s))ds − t + σB(t) + I(t), σ =
√(µ ∧ λ)E[s2
1]
πV (dx) = G exp(
2∫ x
0
ρFa(u)− 1σ2 du
)dx , x ∈ [0,∞).
No scaling. The recommendation is to use πV as a proxy for πV .12 / 33
A notion of approximation accuracy
For the M/M/1 queue
V = W and W (t) is a one dimensional RBM.
If ρ < 1, W := W (∞) is expo(mean = ρ/(µ(1− ρ)))
M/M/1 : E[W k ] =k !ρ
(µ(1− ρ))k , Brownian Q : E[W k ] =k !ρk
(µ(1− ρ))k .
The approximation gap for the k th moment is
|E[W k ]− E[W k ]| =ρk !
(µ(1− ρ))k (1− ρk−1)
=k(1− ρk−1)
λρk−3(1− ρ)E[V k−1]
≤ k(k − 1)
λρk−3 E[W k−1].
For k = 1 the gap is 0 (the P-K formula for the M/GI/1 queue).13 / 33
A notion of approximation accuracy
For the M/GI/1 + GI queue, it “universally” holds
|E[W k ]− E[W k ]| ≤ CλE[W k−1]
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0
0.005
0.01
0.015
0.02
0.025
0.03
0 500 1000 1500 2000
Scaled
Error
Waitin
g Time
1.2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.002
0.004
0.006
0.008
0.01
0.012
0.9 0.95 1 1.05 1.1 1.15 1.2
Scaled
Error
Waitin
g Time
Figure: Hyper Exponential patience: Fa(x) =47
(1− e−4x ) +37
(1− e−x/2).M/M/1+GI moments using Zeltyn and Mandelbaum (2005).
14 / 33
Queue families
The M/GI/1+GI queue primitives are
p = (Arrival rate λ, Service time dist. Fs, Patience dist. Fa)
We will define Q-families parameterized by a constant H.
and prove results of the form
supp∈Q(H)
|E[W kp ]− E[W k
p ]|
E[W k−1p ]
≤ CH
λ
Universality = size the family Q(H)
Recall: scaling is sensitive to patience dist. and other primitives.
15 / 33
Queue family Q(H) = {p = (λ,Fs,Fa)}
(i) service-time moments: E[exp
(δH
s1
E[s1]
)]≤ H,
and there exists a concentration constant cp ≥ (µH)−1 such that
(ii) finite load: ρ ∈ [H−1,H], ρ ≥ 1− Hλcp
.
(iii) polynomial growth: Fa is differentiable with density fa:
fa(y) ≤ Hλc2
p
(1 +
∣∣∣∣y − wp
cp
∣∣∣∣H),
(iv) concentration:
ρFa (y)− 1 ≤ −H−1 1λcp
, for all y ≥ wp + cpH,
andρFa (y)− 1 ≥ H−1 1
λcp, for all y ≤ wp − cpH,
Notice: cp, wp vary with the primitives.16 / 33
Indeed a large family
exp(θ) patience (ρ > 1 : ρe−θw = 1):
ρFa
(w +
H√λ
)− 1 ≤ −H−1 1
λcp⇐⇒ ρe−θ
(w+ H√
λ
)− 1 ≤ −H−1 1√
λ
Facp H
ρ ≤ 1 ρ > 1
Infinite 1λ(1−ρ)
−− 1/ρ
exp(θ) 1λ(1−ρ)
∧ 1√λ
1√λ
max(θ, 2/θ, ρ, 1/ρ)
Uniform[0, α] 1λ(1−ρ)
∧ 1√λ
1√λ
max(1/α,√α/ρ, ρ, 1/ρ)
HyperExp(θ, θ) 1λ(1−ρ)
∧ 1√λ
1√λ
max(θ, 2/θ, ρ, 1/ρ)
Power(α, k ) 1λ(1−ρ)
∧ λ−1
k+1 λ− 1
k+1 ∧ 1√λ(ρ−1)
1− 1k
1∨α1∧ρ ∨
2k (k∨ρ)
(1∧α)k
Erlang(k, θ) 1λ(1−ρ)
∧ λ−1
k+1 λ− 1
k+1 ∧ 1√λ(ρ−1)
1− 1k
2k+1(ρ∨k)HE0 Γ(k)
ρ∧1 max(θk , 1θk ) ∨ U
Beta(α, β) 1λ(1−ρ)
∧ λ−1α+1 λ
− 1α+1 ∧ 1√
λ(ρ−1)1− 1
α
2α+β (ρ∨α)Γ(α+β)(ρ∧1)Γ(α)Γ(β) min(L,1)
∨ U
Table: cp and H for a family of patience distributions.
All these are in one queue family.17 / 33
The accuracy of the Brownian approximation
Theorem (Virtual waiting time)
Given H > 0 and k ∈ N, there exists a constant C1H,k > 0 such that
E[(V − w)k ]− E[(V − w)k ] = ±C1
H,k
λE[|V − w |k−1], p ∈ Q(H).
Corollary (Waiting time)
Given H and k ∈ N, there exists a constant C2H,k > 0 such that
E[W k ]− E[W k ] = ±C2
H,k
λE[W k−1], p ∈ Q(H).
For k = 1, the error is O(1/λ).
18 / 33
The accuracy of the Brownian approximation
Corollary (Queue length)
Given H, there exists a constant C2H,1 > 0 such that
E[Q] = λE[W ] = λE[W ]± C2H,1, p ∈ Q(H).
The mean-queue approximation gap is a constant.
Corollary (Abandonment)
Given H, there exists a constant CH > 0 such that
Ab = E[Fa(V )]± CH
λ2E[|V − w |2], p ∈ Q(H).
For example, if Fa = exp(θ), the Ab approximation gap is O(1/λ).19 / 33
About the tightness of the Q-family conditions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 200 400 600 800 1000 1200 1400 1600
Scaled
Error
Virtua
l Waitin
g Time
M/D/1 + GI with ρ = 1 (w = 0) and Fa = Gamma(0.5, 2).
Fa = Gamma(0.5,2) violates our conditions.20 / 33
cp captures concentration/scaling
Lemma (Concentration bounds)
There exist constants CVH,k , cV
H,k > 0 such that
cVH,k ≤
E[|V − w |k ]
cpk ≤ CVH,k , p ∈ Q(H).
There exist CH,k ,C1H,k > 0 such that for all p ∈ Q(H),
E[(V − w)k ]− E[(V − w)k ] = ±C1
H,k
λE[|V − w |k−1]
= ±CH,k
λck−1
p .
Example: Fa = exp(θ)→ cp =1√λ
and
E[(V − w)k ]− E[(V − w)k ] =±CH,k
λ
(1
√λ
k−1
).
21 / 33
cp captures concentration/scaling
Lemma (Concentration bounds)
There exist constants CVH,k , cV
H,k > 0 such that
cVH,k ≤
E[|V − w |k ]
cpk ≤ CVH,k , p ∈ Q(H).
There exist CH,k ,C1H,k > 0 such that for all p ∈ Q(H),
E[(V − w)k ]− E[(V − w)k ] = ±C1
H,k
λE[|V − w |k−1]
= ±CH,k
λck−1
p .
Example: Fa = exp(θ)→ cp =1√λ
and
E[(V − w)k ]− E[(V − w)k ] =±CH,k
λ
(1
√λ
k−1
).
21 / 33
Underlying Math: Generator comparisons (B&D)
gk (x) =(x − w)k − E[(V − w)k ]
(E[|V − w |])kso that E[gk (V )] = 0
Solve (for Ψ) (AΨ)(x) = gk (x) (Brownian Poisson Eqn)
E[(AΨ)(V )− gk (V )] = E[(AΨ)(V )− (AΨ)(V )] + E[(AΨ)(V )− gk (V )]
= E[(AΨ)(V )− (AΨ)(V )]
If E[AΨ(V )] = 0 (Glynn and Zeevi (2008)), then
E[gk (V )] =E[(V − w)k − E[(V − w)k ]]
(E[|V − w |])k= −E[(AΨ)(V )− (AΨ)(V )]
22 / 33
Generator comparison and gradient bounds
|E[(V − w)k ]− E[(V − w)k ]| ≤ (E[|V − w |])k |E[(AΨ)(V )− (AΨ)(V )]|
AΨ(x) = −Ψ(1)(x) + λFa (x)E[Ψ(x + s1)− Ψ(x)
]= −Ψ(1)(x) + λFa (x)E
[Ψ(1)(x)s1 +
12
Ψ(2)(x)s21 + ε(x , s1)
]= AΨ(x) + λFa (x)E [ε(x , s1)] .
E[ε(x , s1)] has Ψ’s derivative of order m > 2.
E[|AΨ(V )− AΨ(V )|] ≤ λE[Fa (V )E [ε(V , s1)]]show≤ CH
λE[|V − w |].
Where does V on the right-hand side come from?Show = Gradient + Apriori moment bounds (via cp drift cond.)See Braverman and Dai (2016)
23 / 33
From performance analysis to optimization
24 / 33
Two ways in which regimes arise
“Consider a sequence of queues with ρ(λ) = 1− β√λ
”
Identifying the “optimal” regime:Minimizing capacity + linear delay cost in the M/M/1 queue
µ∗(λ) := minµ
csµ+ cwλE[W (µ)] = λ+
√λcw
cs,
so that
√λ(1− ρ∗(λ)) =
√λ
1− λ
λ+√
λcwcs
→√cw
csas λ→∞
ρ∗(λ) ≈ 1−√
cw
cs
1√λ
If cw =14λ and cs = 1, then, ρ∗ = 1/2.
25 / 33
Dynamic optimization:Service-rate control in the M/G/1 queue
Arrival rate λ; Service time distribution Fs with E[s1] = 1.
Controlled service rate µ(θ) = λ(1 + θ). Holding cost hxm.
p = (λ,h).
J V ,∗p,m = inf
θ∈ΘVlim
t→∞
1tEx
[∫ t
0
(h(V (θ, s))m + (λθ(s))2
)ds].
Steps:
An unscaled Brownian Control Problem (BCP)
Universality over Q(H) = {(λ,h) : λ ≥ H−1,h ∈ (0,H)}
We will be agnostic to whether (or not) h scales down with λ
26 / 33
The Brownian control problem
V (θ, t) =V (0)− λ∫ t
0θ(s)ds +
∫ t
0λ(1 + θ(s))1{V (θ, s) = 0}ds
+
A(t)∑i=1
si − λt
V (θ, t) =V (0)− λ∫ t
0θ(s)ds + λ
∫ t
0(1 + θ(s))1{V (θ, s) = 0}ds
+√λE[s2
1]B(t).
J V ,∗p,m = inf
θ∈ΘV
limt→∞
1tEx
[∫ t
0
(h(V (θ, s))m + (λθ(s))2
)ds]
(BCP)
27 / 33
Universal optimality gap
TheoremAn optimal stationary (Brownian) policy θ∗p,m(x) exists and, for anyp ∈ Q(H) := {(λ,h) : λ ≥ H−1,h ∈ (0,H]},
J V ,∗p,m − J V
p,m(θ∗p,m) ≤ BH(λ,m)J V ,∗p,m−1
BH(λ,m)→ 0 as λ→∞. The gap is 0 if m = 2.
Recall, we found for the (uncontrolled) M/GI/1 + GI queue:
E[(V − w)k ]− E[(V − w)k ] = ±C1
H,k
λE[|V − w |k−1]
28 / 33
Universal optimality gap
TheoremAn optimal stationary (Brownian) policy θ∗p,m(x) exists and, for anyp ∈ Q(H) := {(λ,h) : λ ≥ H−1,h ∈ (0,H]},
J V ,∗p,m − J V
p,m(θ∗p,m) ≤ BH(λ,m)J V ,∗p,m−1
BH(λ,m)→ 0 as λ→∞. The gap is 0 if m = 2.
Recall, we found for the (uncontrolled) M/GI/1 + GI queue:
E[(V − w)k ]− E[(V − w)k ] = ±C1
H,k
λE[|V − w |k−1]
28 / 33
For control, too, generator comparisons
γ = minz≥0
{(Az
λΨ)(x) + (λz)2 + hxm}, (Diffusion HJB)
Ψ(0) = Ψ(1)(0) = 0 and Ψ(1)(x) ≥ 0, for all x ≥ 0,
Given (Ψ, γ) : optimal service rate z∗(x) =Ψ(1)(x)
2λ
γ = minz≥0{(Az
λΨ)(x) + (λz)2 + hxm} (M/GI/1 Bellman)
(Doshi (1978))
If for relevant values of z, AzλΨ ≈ Az
λΨ:
minz≥0{(Az
λΨ)(x) + λz + hxm} ≈ minz≥0
(AzλΨ)(x) + λz + hxm}
→ Ψ, γp,m “almost” solves the M/G/1 Bellman equation.29 / 33
From BCP to M/GI/1 optimality
Lemma
Fix (p,m) and let (Ψ, γ) be the solution the (BCPs) HJB equation.Then, for any admissible control θ for the M/G/1 queue (and anyx , t ≥ 0):
Ex
[∫ t
0
(h(V (θ, s))m + (λθ(s))2
)ds]≥ Ψ(x)− Ex
[Ψ(V (θ, t))
]+ γt
+ Ax (θ, t),
where Ax (θ, t) = Ex
[∫ t
0
(Aθ(s)λ Ψ(V (θ, s))− Aθ(s)
λ Ψ(V (θ, s)))
ds].
If θ is the BCP stationary control z∗, the inequality is replaced withequality. If m = 2, Ax (θ, t) ≡ 0 for any control θ.
30 / 33
From BCP to M/GI/1 optimality
Lemma
Fix m and let (Ψp,m, γp,m) be the (family of) solutions to the HJBequation. Then, there exist constants C1
H,m,C2H,m such that, for any
order optimal family of policies {θp,m,p ∈ Q(H)},
lim inft→∞
1tAx
p,m(θp,m, t) ≥ −C1H,mBH(λ,m)J Y ,∗
p,m−1, x ≥ 0,
and under the stationary policy θ∗p,m,
lim supt→∞
1tAx
p,m(θ∗p,m, t) ≤ C2H,mBH(λ,m)J Y ,∗
p,m−1, x ≥ 0.
31 / 33
Conclusion
• In great generality, it is fine to use the intuitive Brownian queue ofthe M/GI/1+GI.
It is universally accurate in regime and patience-scaling.
• Similar ideas are applied to static and dynamic optimization
Underlying math:Avoid scaling through Q-families.From the universal proximity of operators to the universal proximityof equation solutions (Poisson or HJB).
32 / 33
Time Dependent Expectations
0
20
40
60
80
100
120
140
0
2
4
6
8
10
12
14
0 50 100 150 200 250 300Second
mom
ent
First m
omen
t
Time
0
0.1
0.2
0.3
0.4
0.5
0.6
0 50 100 150 200 250 300
Scaled
gap
Time
Figure: Time-dependent performance for M/M/1 with µ = 1: (LHS) ρ = 0.9,(RHS) Scaled gap
33 / 33