dynamics of drug resistance: optimal control of an ... · dynamics of drug resistance: optimal...
TRANSCRIPT
Dynamics of Drug Resistance:Optimal control of an infectious disease
Naveed ChehraziLauren E. Cipriano
Eva A. Enns
Funding & Disclosure• Natural Sciences and Engineering Research Council of Canada (PI: Cipriano)• National Institute for Allergy and Infectious Diseases at the National Institutes of
Health [Grant K25AI118476 (PI: Enns)].
• The funding agencies had no influence on the design and conduct of the study; collection, management, analysis, and interpretation of the data; or in the preparation or review of the manuscript.
• The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Antimicrobial resistance• Treatment resistant bacteria, parasites, viruses, and fungi
• De novo resistant genes• Genes that confer resistance transferred between species and strains
• Previously easy-to-treat infections are now difficult, intensive, and expensive to treat
• Significant threat to public health• Antibiotic resistance in the US: 2 million infections and 23,000 deaths annually
• In many cases, few effective treatment options remain
Research questions• What treatment policy minimizes the cost of disease to society in the
presence of resistance?
• Should the last remaining effective treatment be withheld to preserve the drug for a potentially more serious future outbreak?
• Restricting access for general medical use • Stockpiling
Infectious disease models with resistance• Majority of literature uses detailed disease models and numerical
methods to evaluate and compare controls • i.e., vaccination vs. quarantine; prevention vs. treatment
• Few generalizable insights and sometimes contradictory results• Models of pandemic influenza with resistance have found prophylaxis,
a mix of prophylaxis and treatment, and no prophylaxis to be optimal
Modeling approach• Focus on SIS-type infectious diseases e.g., gonorrhea, H. pylori, TB• Assume that there is one remaining effective treatment
𝛽
𝑟Susceptibleto infection
1 − 𝑃(𝑡)
𝑃)(𝑡)Drug-resistant strain
Drug-susceptible strain𝑃*(𝑡)
𝛽
𝑟
Infected 𝑃(𝑡)
Infected 𝑃 𝑡 = 𝑃) 𝑡 + 𝑃* 𝑡
SIS Model
�̇�. 𝑡 = 𝛽 1 − 𝑃 𝑡 − 𝑟 𝑃. 𝑡 , 𝑡 ≥ 0
𝛽
𝑟Susceptibleto infection
1 − 𝑃(𝑡)
𝑃)(𝑡)𝛽
𝑟
Drug-resistant strain
Drug-susceptible strain𝑃*(𝑡)
1. Drug resistance doesn’t affect infectiousness (one 𝛽)
2. Infection rate is constant (not influenced by disease prevalence)
3. Drug resistance doesn’t affect virulence (one 𝑟)
4. Disease is not self-limiting: 𝑟 < 𝛽𝑃. 𝑡 =
𝑝.(𝛽 − 𝑟)𝑒 567 8
𝑝.𝛽 𝑒 567 8 − 1 + (𝛽 − 𝑟)
For an initial condition 𝑃. 0 = 𝑝.:
System of equations:
SIS Model+ Treatment
Treatment Policy: W 𝑡 ∈ [0,1]: % of the infectedpopulation that will receive treatmentEfficacy of treatment normalized to 1.
System of equations:
�̇�) 𝑡 = 𝛽 1 − 𝑃 𝑡 − 𝑟 𝑃) 𝑡�̇�* 𝑡 = 𝛽 1 − 𝑃 𝑡 −𝑊 𝑡 − 𝑟 𝑃* 𝑡
Drug “Quality”:
For 𝑡 ≥ 0,
𝑄 𝑡 =𝑃*(𝑡)
𝑃) 𝑡 + 𝑃* 𝑡
Re-write the system of equations: For 𝑡 ≥ 0,
�̇� 𝑡 = 𝛽 1 − 𝑃 𝑡 − 𝑄 𝑡 𝑊 𝑡 − 𝑟 𝑃 𝑡�̇� 𝑡 = 𝑊(𝑡)𝑄(𝑡) 𝑄 𝑡 − 1
𝛽
𝑟Susceptibleto infection
1 − 𝑃(𝑡)
𝑃)(𝑡)𝛽
𝑟 +𝑾
Drug-resistant strain
Drug-susceptible strain𝑃*(𝑡)
Problem formulation• For an initial condition, (𝑝, 𝑞), identify the treatment policy, 𝑊, that
minimizes the total cost of the disease
infD:ℝF→[),*]
H)
I(𝑐*𝑊 𝑡 𝑃 𝑡 + 𝑐K𝑃(𝑡))𝑒6L8𝑑𝑡
�̇� 𝑡 = 𝛽 1 − 𝑃 𝑡 − 𝑄 𝑡 𝑊 𝑡 − 𝑟 𝑃 𝑡 , 𝑡 ≥ 0
�̇� 𝑡 = 𝑊(𝑡)𝑄(𝑡) 𝑄 𝑡 − 1 , 𝑡 ≥ 0
S.t. Cost of treatment
Cost of illness
Discount rate
�̇� 𝑡 = 𝛽 1 − 𝑃 𝑡 − 𝑄 𝑡 𝑊 𝑡 − 𝑟 𝑃 𝑡 ,
�̇� 𝑡 = 𝑊(𝑡)𝑄(𝑡) 𝑄 𝑡 − 1 ,
s.t.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 1.00.90
0.4
0.20.1
0.3
0.50.6
0.80.9
0.7
1.0
Prev
alen
ce (p
)
Quality (q)(Prop. of infections that are susceptible)𝑡 ≥ 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 1.00.90
0.4
0.20.1
0.3
0.50.6
0.80.9
0.7
1.0
Problem formulation• For an initial condition, (𝑝, 𝑞), identify the treatment policy, 𝑊, that
minimizes the total cost of the disease
infD:ℝF→[),*]
H)
I(𝑐*𝑊 𝑡 𝑃 𝑡 + 𝑐K𝑃(𝑡))𝑒6L8𝑑𝑡
Cost of treatment
Cost of illness
�̇� 𝑡 = 𝛽 1 − 𝑃 𝑡 − 𝑄 𝑡 𝑊 𝑡 − 𝑟 𝑃 𝑡 ,
�̇� 𝑡 = 𝑊(𝑡)𝑄(𝑡) 𝑄 𝑡 − 1 ,
s.t.
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
Prev
alen
ce (p
)
Quality (q)(Prop. of infections that are susceptible)𝑡 ≥ 0
Sufficient optimality condition (HJB equation)
0 = minO∈[),*]
{𝑐*𝑤𝑝 + 𝑐K𝑝
+𝜕K𝑣∗ 𝑝, 𝑞 𝑤𝑞 𝑞 − 1 − 𝜌𝑣∗(𝑝, 𝑞)}
+𝜕*𝑣∗ 𝑝, 𝑞 𝛽 1 − 𝑝 − 𝑤𝑞 − 𝑟 𝑝0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7Proposition 1 (HJB Equation)
Let 𝑣∗: 0,1 ×[0,1] → ℝ be a bounded function that is almost everywhere differentiable on any trajectory (𝑃 𝑡; 𝑝, 𝑞,𝑊 , 𝑄(𝑡; 𝑞,𝑊)), 𝑡 ≥ 0, and that satisfies the Hamilton-Jacobi-Bellman equation:
Prev
alen
ce (p
)
Quality (q)(Prop. of infections that are susceptible)
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
Sufficient optimality condition (HJB equation)
0 = minO∈[),*]
{𝑐*𝑤𝑝 + 𝑐K𝑝
+𝜕K𝑣∗ 𝑝, 𝑞 𝑤𝑞 𝑞 − 1 − 𝜌𝑣∗(𝑝, 𝑞)}
+𝜕*𝑣∗ 𝑝, 𝑞 𝛽 1 − 𝑝 − 𝑤𝑞 − 𝑟 𝑝
Proposition 1 (HJB Equation) Let 𝑣∗: 0,1 ×[0,1] → ℝ be a bounded function that is almost everywhere differentiable on any trajectory (𝑃 𝑡; 𝑝, 𝑞,𝑊 , 𝑄(𝑡; 𝑞,𝑊)), 𝑡 ≥ 0, and that satisfies the Hamilton-Jacobi-Bellman equation:
Prev
alen
ce (p
)
Quality (q)(Prop. of infections that are susceptible)
Optimal policy• Bang-bang with a single switching time
• Treat everyone until it is not economical to treat anyone
• Withholding treatment to preserve the drug’s effectiveness is not optimal
Prev
alen
ce (p
)
Quality (q)(Prop. of infections that are susceptible)
Properties of the value function
• When Low discount rate
à Long-term focused planner
Value function is not Lipschitz continuousà Traditional numerical methods may
not be computationally stable
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
𝜌 ≤ 𝛽 − 𝑟 Prev
alen
ce (p
)Pr
eval
ence
(p)
Quality (q)(Prop. of infections that are susceptible)
Non-constant disease transmission rate
• Protective behaviours in response to high prevalence
Consider
NEW System of equations: For 𝑡 ≥ 0,
�̇� 𝑡 =𝛽 1 − 𝑃 𝑡
𝑃 𝑡− 𝑄 𝑡 𝑊 𝑡 + 𝑟 𝑃 𝑡
�̇� 𝑡 = 𝑊(𝑡)𝑄(𝑡) 𝑄 𝑡 − 1
𝐵 𝑝 = 𝛽𝑝6*.\
B(p)
𝑟Susceptibleto infection
1 − 𝑃(𝑡)
𝑃)(𝑡)
B(p)
𝑟 +𝑾
Drug-resistant strain
Drug-susceptible strain𝑃*(𝑡)
Optimal policy with a non-constant disease transmission rate
• Bang-bang with a singular arc
• Treat everyone until the boundary,then treat a fraction of the population(stay on the boundary)
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
Optimal policy with a non-constant disease transmission rate
• Bang-bang with a singular arc
• Treat no one until it is economical totreat a fraction of the population
• Withholding treatment to preserve the drug’s effectiveness may be optimal
(e.g., strategic stockpile of antivirals)
0
0.4
0.20.1
0.3
0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.80.7 0.9
0.60.7
Extensively resistant gonorrhea
• Prevalence: 0.27% (general pop’n)• Resistance to ceftriaxone &
azithromycin: 0.3%
• Current cost: $1.50/person• Expected value of optimal policy:
$28.19/person
Prev
alen
ce (%
)
Quality (%)(% of infections that are susceptible)
9.5 years5-20 years
12 years6-25 years30 years
14-62 years
Contribution and Key findings• SIS infectious disease with constant disease transmission rate
• Closed form solution to optimal control problem with two continuous state variables• Analysts need to check value function properties before applying traditional numerical
methods• Optimal policy: Bang-bang with a single switching time
àWithholding treatment to preserve the drug’s effectiveness is not optimal• Applied model to problem of extensively resistant gonorrhea to identify value of research
• SIS infectious disease with non-constant disease transmission rate • Dramatically changes the form of the optimal policy
à Withholding treatment to preserve the drug’s effectiveness may be optimal
• Approximating with a constant rate may indicate a suboptimal policy
Limitations & Future Research• SIS model
• Expand to consider R and V state (natural or vaccine immunity)
• Limited to the last drug• Expand to consider the optimal use of a set of n drugs
• Do not consider the effect of drug misuse on resistance pressure• i.e., antibiotic use for viral infections
• Do not consider innovation• How much should be invested in new antimicrobial research?• How to balance investment in stewardship vs. research?
