![Page 1: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/1.jpg)
Robust Policies for Proactive ICU Transfers
Carri W. Chan1, Vineet Goyal2, Julien Grand-Clement2, Gabriel Escobar3
1 Columbia Business School, Columbia University2 IEOR Department, Columbia University3 Division of Research, Kaiser Permanente
![Page 2: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/2.jpg)
What is this talk about?
Uncertainty in transition matrices of Markov Chains.
Geometry of the uncertainty set and tractability.
Application: efficient and robust ICU admission guidelines.
1
![Page 3: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/3.jpg)
Background on Intensive Care Unit (ICU).
Critically ill patients are treated in ICU.
Unplanned transfer:
Significant impact on both patients outcomes and operation costs.
• Unplanned transfer to the ICU1:
Stays 10 days longer than average patient.
Mortality 3 times higher than average patient.
• Cost of ICU patient: up to 2.5 higher than in ward2.
• ICU admissions increased by 48.8% from 2002 to 20093.
1Escobar et al. (2013)2Mullins et al. (2013)3Mildbrandt et al. (2013)
2
![Page 4: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/4.jpg)
Background on Intensive Care Unit (ICU).
Critically ill patients are treated in ICU.
Unplanned transfer:
Significant impact on both patients outcomes and operation costs.
• Unplanned transfer to the ICU1:
Stays 10 days longer than average patient.
Mortality 3 times higher than average patient.
• Cost of ICU patient: up to 2.5 higher than in ward2.
• ICU admissions increased by 48.8% from 2002 to 20093.
1Escobar et al. (2013)2Mullins et al. (2013)3Mildbrandt et al. (2013)
2
![Page 5: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/5.jpg)
Background on Intensive Care Unit (ICU).
Critically ill patients are treated in ICU.
Unplanned transfer:
Significant impact on both patients outcomes and operation costs.
• Unplanned transfer to the ICU1:
Stays 10 days longer than average patient.
Mortality 3 times higher than average patient.
• Cost of ICU patient: up to 2.5 higher than in ward2.
• ICU admissions increased by 48.8% from 2002 to 20093.
1Escobar et al. (2013)2Mullins et al. (2013)3Mildbrandt et al. (2013)
2
![Page 6: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/6.jpg)
WARD ICU
Simplified hospital: ward vs ICU.
3
![Page 7: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/7.jpg)
WARD ICU
Suboptimal use of ICU resources (congestioned ward, ‘empty’ ICU).
4
![Page 8: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/8.jpg)
WARD ICU
Proactive transfer patients to the ICU: lower mortality.
5
![Page 9: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/9.jpg)
WARD ICU
Too many transfers ⇒ congestioned ICU: can not face unplanned events.
6
![Page 10: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/10.jpg)
Research question and literature.
Research question.
Impact of proactive transfers on hospital performances?
Robustness of predictive models to parameters mispecifications?
Related works.
ICU admission. Shmueli et al. (2004), Peck et al. (2012),
Bountourelis et al. (2012), Butcher et al. (2013), Guirgis et al.
(2013), Kim et al. (2014), Xu and Chan (2016), Hu et al. (2018).
Threshold policy. Altman et al. (2001), Shmueli et al. (2003), Kim
et al. (2015), Hu et al. (2018).
Robust MDP. Iyengar (2005), Nilim and El Ghaoui (2005), Delage
and Mannor (2010), Wiesemann et al. (2013), Goh et al. (2014),
Mannor et al. (2016), Goyal and G-C. (2018), Steimle et al. (2018).
7
![Page 11: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/11.jpg)
Research question and literature.
Research question.
Impact of proactive transfers on hospital performances?
Robustness of predictive models to parameters mispecifications?
Related works.
ICU admission. Shmueli et al. (2004), Peck et al. (2012),
Bountourelis et al. (2012), Butcher et al. (2013), Guirgis et al.
(2013), Kim et al. (2014), Xu and Chan (2016), Hu et al. (2018).
Threshold policy. Altman et al. (2001), Shmueli et al. (2003), Kim
et al. (2015), Hu et al. (2018).
Robust MDP. Iyengar (2005), Nilim and El Ghaoui (2005), Delage
and Mannor (2010), Wiesemann et al. (2013), Goh et al. (2014),
Mannor et al. (2016), Goyal and G-C. (2018), Steimle et al. (2018).
7
![Page 12: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/12.jpg)
Our results.
1. Markov Decision Process for patient dynamics.
2. Low-rank model for parameter uncertainty.
3. Optimality and Robustness of threshold policies.
4. Numerical study: worst-case analysis of hospital performances.
8
![Page 13: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/13.jpg)
Single-Patient Markov Decision Process (MDP).
Ward
Arrivals
p0, i
Severity scores transitions
ij
T0i,j
T0i,DL
T0i,RL
T0i,UT
i, j ∈ {1,...,n}
1T0
j,i
Die & Leave
Recover & Leave
Unplanned Transfer
Proactive Transfer
UT/PT: Unplanned/Proactive Transfer. DL/RL: Die/Recover and Leave.9
![Page 14: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/14.jpg)
Single-Patient Markov Decision Process (MDP).
Ward
Arrivals
p0, i
Severity scores transitions
ij
T0i,j
T0i,DL
T0i,RL
T0i,UT
i, j ∈ {1,...,n}
1T0
j,i
Die & Leave
Recover & Leave
Unplanned Transfer
Proactive Transfer
Transition matrix: T 0. Transfer policy π : { severity scores } → [0, 1].10
![Page 15: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/15.jpg)
Single-Patient Markov Decision Process (MDP).
Rewards: rPT , rUT , rRL, rDL.
How to calibrate? Natural assumption:
Ordering by outcome and use of ICU resources.
rRL ≥ rPT ≥ rUT ≥ rDL.
Goal: Find transfer policy π to maximize expected reward R(π,T 0), for
R(π,T 0) = Eπ,T0
[ ∞∑t=0
λtrst
].
11
![Page 16: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/16.jpg)
Theoretical guarantees.
Key Assumptions: for all state i ∈ {1, ..., n − 1},
Reward ( exit the ward | state i) ≥ Reward ( exit the ward | state i + 1) .
(1)
(P ( exit the ward | state i))i increases fast enough. (2)
Theorem
Let T 0 a transition matrix that satisfies Assumptions (1)-(2).
There exists a threshold policy πnom that is optimal for T 0.
Proof intuition: at every step of Value Iteration, πs is threshold.
Efficient algorithm: Value Iteration or Enumeration of threshold policies.
12
![Page 17: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/17.jpg)
Theoretical guarantees.
Key Assumptions: for all state i ∈ {1, ..., n − 1},
rRLT0i,RL + rDLT
0i,DL + rPTT
0i,PT ≥ rRLT
0i+1,RL + rDLT
0i+1,DL + rPTT
0i+1,PT ,
(1)
1 + λ · rPT1 + λ · rRL
≥
(∑nj=1 T
0i+1,j
)(∑n
j=1 T0ij
) . (2)
Theorem
Let T 0 a transition matrix that satisfies Assumptions (1)-(2).
There exists a threshold policy πnom that is optimal for T 0.
Proof intuition: at every step of Value Iteration, πs is threshold.
Efficient algorithm: Value Iteration or Enumeration of threshold policies.
13
![Page 18: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/18.jpg)
Parameter uncertainty.
In practice, T 0 is estimated from some data and is uncertain.
From the data, we obtain 95% confidence intervals:
T truei,j ∈ [T 0
i,j − αi ,T0i,j + βi ].
Example: T 02,1 = 0.3216,T true
2,1 ∈ [0.3208, 0.3232].
Simulations: deviations of ≈ 10−3 may increase mortality by 40 % !
14
![Page 19: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/19.jpg)
Robust MDP.
Robust approach: T 0 ∈ U , ‘uncertainty set’: plausible matrices.
U is computed from the confidence-intervals of T 0.
Zero-sum game:
(i) Decision-maker chooses policy π,
(ii) Nature chooses T ∈ U adversarially,
(iii) Decision-maker obtains R(π,T ).
Our goal
solve maxπ
minT∈U
R(π,T ).
Robust policy: maximizes worst-case reward over all matrices in U .
→ The robust policy guarantees a certain level of performance.
15
![Page 20: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/20.jpg)
Related works on robust MDPs.
For general U , computing the worst-case reward of a policy π is hard:
“ minT∈U
R(π,T ) ≥ γ ?” is NP-Hard (Wiesemann et al. (2013)).
Tractable for some special cases:
• (s, a)-retangularity (Iyengar (2005), Nilim and El Ghaoui (2005))
• s-rectangularity (Wiesemann et al. (2013)).
• k-rectangularity (Mannor et al. (2016)).
16
![Page 21: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/21.jpg)
Choice of uncertainty set.
r -rectangular uncertainty set4:
U = {T | T = UW>,
W = (w1, ...,wr ) ∈ W1 × ...×W r ,
W>e = e,W ≥ 0,
U ≥ 0 is fixed,Ue = 1 }.
→ Think about SVD with uncertain eigenvectors.
→ Coupled rows of T = simple transformation of uncoupled w1, ...,wr .
4Goh et al. (2014), Goyal and G-C. (2018)
17
![Page 22: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/22.jpg)
Choice of uncertainty set.
r -rectangular uncertainty set5:
U = {T | T = UW>,
W = (w1, ...,wr ) ∈ W1 × ...×W r ,
W>e = e,W ≥ 0,
U ≥ 0 is fixed,Ue = 1 }.
→ Rank r is the number of underlying variables driving the dynamics.
→ We implicetely assume T 0 = UW>0 for a feasible W0.
5Goh et al. (2014), Goyal and G-C. (2018)
18
![Page 23: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/23.jpg)
Our results (Goyal and G.-C. (2018)).
Consider an r-rectangular uncertainty set U .
Robust Value Iteration Algorithm.
We can efficiently compute an optimal robust policy.
Strong Duality and Nash Equilibrium.
maxπ
minT∈U
R(π,T ) = minT∈U
maxπ
R(π,T ).
T∗ ∈ arg minT∈U
R(π∗,T ),
π∗ ∈ arg maxπ
R(π,T∗).
Interpretability and implementability.
There exists a deterministic optimal robust policy π∗.
19
![Page 24: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/24.jpg)
Useful tools (Goyal and G.-C. (2018)).
Value vector V : for all state s,Vs(π,T ) = Eπ,T [∑∞
t=0 λtrst | s0 = s] .
Robust Maximum Principle:
1. Fix a transition matrix T 0 and let π0 be the optimal policy for T 0.
Then V (π,T 0) ≤ V (π0,T 0),∀ π.
2. Fix a policy π and let Tπ is worst-case matrix for policy π.
Then V (π,Tπ) ≤ V (π,T ),∀ T ∈ U .
3. Let π∗ an optimal robust policy.
Then V (π,Tπ) ≤ V (π∗,T ∗),∀ π.
Robust Blackwell Optimality:
Let π∗λ be the robust policy for the discount factor λ.
Then ∃ λ0 ∈ [0, 1),∀ λ ≥ λ0, π∗λ = π∗λ0.
20
![Page 25: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/25.jpg)
Back to single-patient MDP:
What is the structure of an optimal robust policy?
Numerical study:
How much change in hospital performances when parameters deviate?
21
![Page 26: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/26.jpg)
Robust transfer policies.
We make the following assumption:
Every matrix T in U satisfies Assumptions (1)-(2). (3)
Theorem
Let Assumption (3) hold.
There exists an optimal robust policy πrob that is a threshold policy.
Moreover,
threshold(πrob) ≤ threshold(πnom).
Intuition: optimal robust policy transfers more patients naive one.
Proof: relies on the robust maximum principle.
Efficient algorithm: Robust Value Iteration or Enumeration.
22
![Page 27: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/27.jpg)
Numerical study.
Kaiser Permanente dataset.
• ≈ 300, 000 hospitalizations, 10 severity scores (EDIP2).
• Patients arrive in hospital ward with severity score i ∈ { 1, ..., 10 }.• Severity score updated every six hours: i → { j ,UT ,RL,DL }.
Experimental setup:
1. Construct an hospital model for simulation.
2. Construct uncertainty set U from the data.
3. Compare nominal and worst-case performances in hospital.
23
![Page 28: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/28.jpg)
Hospital model inspired from Hu et al. (2018).
Never been to the ICU
Been to the ICU
WardICU
Arrivals
ArrivalsDischarge/death
Discharge/death
Proactive transfer
Unplanned Transfer
Nominal readmission
Nominal discharge
Demand-driven discharge
Proactive readmission
Discharge/death
Markov Chain: T 0i,j among ‘severity of illness’ scores (i , j).
24
![Page 29: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/29.jpg)
Hospital model inspired from Hu et al. (2018).
Never been to the ICU
Been to the ICU
WardICU
Arrivals
ArrivalsDischarge/death
Discharge/death
Proactive transfer
Unplanned Transfer
Nominal readmission
Nominal discharge
Demand-driven discharge
Proactive readmission
Discharge/death
Simulations: compute mortality, length-of-stay, ICU occupancy.25
![Page 30: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/30.jpg)
Parameters estimation.
a) Empirical average: T 0i,j = P ( go to score j | being in score i ).
From the data, 95% confidence intervals: T truei,j ∈ [T 0
i,j − αi ,T0i,j + βi ].
Example: T 02,1 = 0.3216,T true
2,1 ∈ [0.3208, 0.3232].
b) Hospital worst-case performances: given a policy π, we compute
Tπ ∈ arg minT∈U
R(π,T ).
Tπ is a candidate for worst-case matrix in hospital dynamics.
We compute the hospital performance of π for transition matrix Tπ.
26
![Page 31: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/31.jpg)
How to use r-rectangular U in simulations?
c) Uncertainty set: T = UW>,W ∈ W1 × ...×W r .
W is varying, but we have constraints on T ! (confidence intervals)
(i) Umin: small deviations of rank r :
maximum deviation from T will be min{ αi , βi | i ∈ [10] }.
(ii) Ustd : empirical std’s of rank r matrices in confidence intervals:
Sample T 1, ...,TN around T 0, compute W 1, ...,W N .
(iii) Usa : unrelated parameters deviations:
Usa = {T | Ti,j ∈ [T 0i,j − αi ,T
0i,j + βi ],∀ i , j ∈ [10],
Te = e,T ≥ 0 }.
27
![Page 32: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/32.jpg)
Numerical results 1: nominal performances.
68 70 72 74 76 78 80
average ICU occupancy (%)
3.5
4
4.5
5
5.5
6
6.5
7
7.5
8
In-h
ospi
tal m
orta
lity
(%)
Nom.
Hospital performances of
threshold policies.
Transition matrix: T 0.
Trade-off Mortality vs ICU
occupancy.
28
![Page 33: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/33.jpg)
Numerical results 2: random samples.
68 70 72 74 76 78 80
average ICU occupancy (%)
3.5
4
4.5
5
5.5
6
6.5
7
7.5
8
In-h
ospi
tal m
orta
lity
(%)
Nom. Rd-1 Rd-2 Rd-3 Rd-4
25 random samples around
T 0:
Mortality may increase by
6.1%.
ICU occupancy may increase
by 2.6%.
⇒ Model looks stable.
29
![Page 34: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/34.jpg)
Numerical results 3: worst-case matrices.
68 70 72 74 76 78 80
average ICU occupancy (%)
3.5
4
4.5
5
5.5
6
6.5
7
7.5
8
In-h
ospi
tal m
orta
lity
(%)
Nom. Umin Ustd Usa
Umin,Ustd: rank-r
deviations.
Usa: full-rank deviations.
1. Worst-case approach:
Mortality may increase by
40%.
ICU occupancy may
increase by 12%.
2. Different insight for Usa:
no trade-off Mortality/ICU
occupancy (for high
thresholds).
30
![Page 35: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/35.jpg)
Conclusion.
Our contributions:
For proactive transfer policies,
1. Guarantees for optimality/robustness of threshold policies.
2. Model of r-rectangular uncertainty from robust MDP.
3. Simulations for worst-case deviations.
31
![Page 36: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/36.jpg)
Main take-aways.
When using predictive models:
1. Small parameters deviations may lead to worse performances.
2. Different insights on the model:
Nominal parameters & random samples vs. worst-case analysis.
3. Important: choice of uncertainty sets: (s, a)-rec.,s-rec.,r -rec., etc.
Thanks! Any questions?
32
![Page 37: Robust Policies for Proactive ICU Transfersjg3728/slides_ICU.pdf · Critically ill patients are treated in ICU. Unplanned transfer: Signi cant impact on both patients outcomes and](https://reader034.vdocuments.net/reader034/viewer/2022050207/5f5a4c16e5ff00391545a104/html5/thumbnails/37.jpg)
Main take-aways.
When using predictive models:
1. Small parameters deviations may lead to worse performances.
2. Different insights on the model:
Nominal parameters & random samples vs. worst-case analysis.
3. Important: choice of uncertainty sets: (s, a)-rec.,s-rec.,r -rec., etc.
Thanks! Any questions?
32