resource pack: models for health decision science€¦ · 1 this resource pack was developed the...

31
1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use. This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported chds.hsph.harvard.edu Resource Pack: Models for Health Decision Science Citation: Resource Pack: Models for Health Decision Science. Center for Heath Decision Science, Harvard T.H. Chan School of Public Health 2018. http://repository.chds.hsph.harvard.edu/repository/collection/resource-pack- models-for-health-decision-science Overview This resource pack, curated by the Center for Health Decision Science, provides broad exposure to mathematical models used in health decision science (e.g., microsimulation, dynamic transmission, agent-based, etc.). Resources include overviews, guidelines, tutorials, and applications relevant to a broad range of clinical and public health topics. A decision analytic approach relies on the use of a mathematical model to formally structure the components of the decision over time. Models are particularly useful when multiple data sources need to be synthesized in an internally consistent and epidemiologically plausible way. Models can extend empiric information by extrapolating outcomes beyond the time horizon of a single study, can be used to evaluate a number of different strategies which would be unable to accommodate in a clinical trial, and can be adapted to specific settings so that alternatives may be evaluated in the context in which they will be applied. Models vary in how they take into account a range of important attributes including time, chance, population, timing of events, and interaction within the population. The choice of will depend on the nature of the decision problem and its salient features, the natural history of the disease, and the data available to parameterize the model.

Upload: others

Post on 03-Jun-2020

12 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

1

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Resource Pack: Models for Health Decision Science Citation: Resource Pack: Models for Health Decision Science. Center for Heath Decision Science, Harvard T.H. Chan School of Public Health 2018. http://repository.chds.hsph.harvard.edu/repository/collection/resource-pack-models-for-health-decision-science

Overview

This resource pack, curated by the Center for Health Decision Science, provides broad exposure to mathematical models used in health decision science (e.g., microsimulation, dynamic transmission, agent-based, etc.). Resources include overviews, guidelines, tutorials, and applications relevant to a broad range of clinical and public health topics.

A decision analytic approach relies on the use of a mathematical model to formally structure the components of the decision over time. Models are particularly useful when multiple data sources need to be synthesized in an internally consistent and epidemiologically plausible way. Models can extend empiric information by extrapolating outcomes beyond the time horizon of a single study, can be used to evaluate a number of different strategies which would be unable to accommodate in a clinical trial, and can be adapted to specific settings so that alternatives may be evaluated in the context in which they will be applied.

Models vary in how they take into account a range of important attributes including time, chance, population, timing of events, and interaction within the population. The choice of will depend on the nature of the decision problem and its salient features, the natural history of the disease, and the data available to parameterize the model.

Page 2: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

2

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Selected Resources – At a Glance MODELS FOR HEALTH DECISIONS Modeling to Inform Strategies to Improve Population Health Report. How Modeling Can Inform Strategies to Improve Population Health: Workshop Summary. The National Academies Press 2016. https://doi.org/10.17226/21807

Decision Models in Clinical Preventive Services Recommendations Guidelines. Owens DK, Whitlock E, Henderson J et al. Use of Decision Models in the Development of Evidence-Based Clinical Preventive Services Recommendations: Methods of the U.S. Preventive Services Task Force. Annals of Internal Medicine 2016; 165 (7): 501-508. https://www.uspreventiveservicestaskforce.org/Page/Name/use-of-decision-models-in-the-development-of-evidence-based-clinical-preventive-services-recommendations

Decision and Simulation Modeling Alongside Systematic Reviews Report. Kuntz K, Sainfort F, Butler M et al. Chapter 1: Decision and Simulation Modeling Alongside Systematic Reviews in Kuntz K, Sainfort F, Butler M et al. Decision and Simulation Modeling in Systematic Reviews. US Agency for Healthcare Research and Quality 2013: 1-14. https://effectivehealthcare.ahrq.gov/topics/methods-decision-simulation-modeling/research

Adding Decision Models to Systematic Reviews Review. Sainfort F, Kuntz KM, Gregory S et al. Adding Decision Models to Systematic Reviews: Informing a Framework for Deciding When and How to Do So. Value in Health 2013; 16 (1): 133-139. https://doi.org/10.1016/j.jval.2012.09.009

Modeling for Health Care and Other Policy Decisions: Uses, Roles and Validity Review. Weinstein MC, Toy EL, Sandberg EA, Neumann PJ, Kuntz KM, Hammitt JK et al. Modeling for Health Care and Other Policy Decisions: Uses, Roles, and Validity. Value in Health 2001; 4 (5): 348-361. https://doi.org/10.1046/j.1524-4733.2001.45061.x TYPES OF MODELS FOR HEALTH DECISIONS

Using Data-Driven Agent-Based Models to Forecast Emerging Infectious Diseases Article. Venkatramanan S, Lewis B, Chen J et al. Using Data-Driven Agent-Based Models for Forecasting Emerging Infectious Diseases. Epidemics 2017. https://doi.org/10.1016/j.epidem.2017.02.010

Combining Microsimulation and Agent-Based Modeling Article. Bae JW, Paik E, Kim K et al. Combining Microsimulation and Agent-Based Model for Micro-Level Population Dynamics. Procedia Computer Science 2016; 80: 507-517. https://doi.org/10.1016/j.procs.2016.05.331

Choosing an Epidemiological Model Structure for Economic Evaluation Review. Briggs ADM, Wolstenholme J, Blakely T et al. Choosing an Epidemiological Model Structure for the Economic Evaluation of Non-Communicable Disease Public Health Interventions. Population Health Metrics 2016; 14: 17. https://dx.doi.org/10.1186%2Fs12963-016-0085-1

Agent-Based Models and Microsimulation Review. Heard D, Dent G, Schifeling T et al. Agent-Based Models and Microsimulation. Annual Review of Statistics and Its Application 2015; 2: 259-272. https://doi.org/10.1146/annurev-statistics-010814-020218 Not open access.

Dynamic Microsimulation Models for Health Outcomes: a Review Review. Rutter CM, Zaslavsky A, Feuer E. Dynamic Microsimulation Models for Health Outcomes: a Review. Medical Decision Making 2011; 31 (1): 10-18. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3404886/pdf/nihms-393222.pdf

Page 3: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

3

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Decision Support for Infectious Disease Control Review. Manheim D, Chamberlin M, Osoba O et al. Improving Decision Support for Infectious Disease Prevention and Control: Aligning Models and Other Tools with Policymakers' Needs. RAND Corporation 2016. https://www.rand.org/pubs/research_reports/RR1576.html

Markov Modeling and Discrete Event Simulation in Health Care: a Systematic Comparison Review. Standfield L, Comans T, Scuffham P. Markov Modeling and Discrete Event Simulation in Health Care: a Systematic Comparison. International Journal of Technology Assessment in Health Care 2014; 30 (2): 165-172. https://doi.org/10.1017/S0266462314000117 Not open access.

Cost-Effectiveness of Vaccination: Review of Modelling Approaches Review. Kim SY, Goldie SJ. Cost-Effectiveness Analyses of Vaccination Programmes: A Focused Review of Modelling Approaches. PharmacoEconomics 2008; 26 (3): 191-215. https://doi.org/10.2165/00019053-200826030-00004 Not open access.

Economic Evaluations with Agent-Based Modelling: An Introduction Tutorial/Primer. Chhatwal J, He T. Economic Evaluations with Agent-Based Modelling: An Introduction. PharmacoEconomics 2015; 33 (5): 423-433. http://dx.doi.org/10.1007/s40273-015-0254-2 TUTORIALS AND PRIMERS

Bayesian Methods for Calibrating Health Policy Models: A Tutorial Tutorial/Primer. Menzies NA, Soeteman D, Pandya A et al. Bayesian Methods for Calibrating Health Policy Models: A Tutorial. PharmacoEconomics 2017; 35 (6): 613-624. http://dx.doi.org/10.1007/s40273-017-0494-4

Calibration of Complex Models through Bayesian Evidence Synthesis: A Tutorial Tutorial/Primer. Jackson C, Jit M, Sharples L et al. Calibration of Complex Models through Bayesian Evidence Synthesis: A Demonstration and Tutorial. Medical Decision Making 2015; 35 (2): 148-161. https://doi.org/10.1177/0272989X13493143 MODELING GUIDELINES

Decision and Simulation Modeling in Systematic Reviews Guidelines. Kuntz K, Sainfort F, Butler M et al. Decision and Simulation Modeling in Systematic Reviews. Methods Research Report. Agency for Health Care Research and Quality 2013. https://ahrq-ehc-application.s3.amazonaws.com/media/pdf/methods-decision-simulation-modeling_research.pdf

Modeling Good Research Practices - Overview: A Report of the ISPOR-SMDM Modeling Task Force-1 Guidelines. Caro JJ, Briggs AH, Siebert U et al. Modeling Good Research Practices - Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Value in Health 2012; 15: 796-803. https://www.ispor.org/workpaper/Modeling-Good-Research-Practices-Overview.asp

Conceptualizing a Model: A Report of the ISPOR-SMDM Modeling Task Force-2 Guidelines. Roberts M, Russell LB, Paltiel AD et al. Conceptualizing a Model: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2. Value in Health 2012; 15: 804-811. https://www.ispor.org/workpaper/Conceptualizing-A-Model.asp

State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Task Force-3 Guidelines. Siebert U, Alagoz O, Bayoumi AM et al. State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value Health 2012; 15: 812-820. https://www.ispor.org/workpaper/State-Transition-Modeling.asp

Page 4: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

4

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Task Force-4 Guidelines. Karnon J, Stahl JE, Brennan A et al. Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Value in Health 2012; 15: 821-827. https://www.ispor.org/workpaper/Modeling-Using-Discrete-Event-Simulation.asp

Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Task Force-5 Guidelines. Pitman RJ, Fisman D, Zaric GS et al. Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-5. Value in Health 2012; 15: 828-834. https://www.ispor.org/workpaper/Dynamic-Transmission-Modeling.asp

Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Task Force-6 Guidelines. Briggs AH, Weinstein MC, Fenwick E et al. Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value in Health 2012; 15: 835-842. https://www.ispor.org/workpaper/Model-Parameter-Estimation-and-Uncertainty-Analysis.asp

Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Task Force-7 Guidelines. Eddy DM, Hollingworth W, Caro JJ et al. Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value in Health 2012; 15: 843-850. https://doi.org/10.1177/0272989X12454579

ISPOR Task Force Report: Good Practice for Decision Analytic Modeling in Health-Care Report. Weinstein MC, O’Brien B, Hornberger J et al. Principles of Good Practice for Decision Analytic Modeling in Health-Care Evaluation: Report of the ISPOR Task Force on Good Research Practices--Modeling Studies. Value in Health 2003; 6 (1): 9-17. https://www.ispor.org/workpaper/healthscience/TFModeling.asp CALIBRATION AND VALIDATION

Validation and Calibration of Structural Models that Combine Information from Multiple Sources Review. Dahabreh IJ, Wong JB, Trikalinos TA. Validation and Calibration of Structural Models that Combine Information from Multiple Sources. Expert Review of Pharmacoeconomics and Outcomes Research 2017; 17 (1): 27-37. https://doi.org/10.1080/14737167.2017.1277143 Not open access.

Likelihood Approach for Calibration of Stochastic Epidemic Models Article. Zimmer C, Yaesoubi R, Cohen T. A Likelihood Approach for Real-Time Calibration of Stochastic Compartmental Epidemic Models. PLoS Computational Biology 2017; 13 (1): e1005257. https://doi.org/10.1371/journal.pcbi.1005257

Calibration of Complex Models through Bayesian Evidence Synthesis: A Tutorial Tutorial/Primer. Jackson C, Jit M, Sharples L et al. Calibration of Complex Models through Bayesian Evidence Synthesis: A Demonstration and Tutorial. Medical Decision Making 2015; 35 (2): 148-161. https://doi.org/10.1177/0272989X13493143

Bayesian Methods for Calibrating Health Policy Models: A Tutorial Tutorial/Primer. Menzies NA, Soeteman D, Pandya A et al. Bayesian Methods for Calibrating Health Policy Models: A Tutorial. PharmacoEconomics 2017; 35 (6): 613-624. http://dx.doi.org/10.1007/s40273-017-0494-4

Validating a Cardiovascular Disease Microsimulation Model Article. Pandya A, Sy S, Cho S, Alam S, Weinstein MC, Gaziano TA. Validation of a Cardiovascular Disease Policy Microsimulation Model Using Both Survival and Receiver Operating Characteristic Curves. Medical Decision Making 2017; 37 (7): 802-814. https://doi.org/10.1177/0272989X17706081 Not open access.

Page 5: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

5

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Validation and Calibration of a Simulation Model of Pediatric HIV Infection Article. Ciaranello AL, Morris B, Walensky R et al. Validation and Calibration of a Computer Simulation Model of Pediatric HIV Infection. PLoS One 2013; 8 (12): e83389. https://doi.org/10.1371/journal.pone.0083389

Calibrating Models in Economic Evaluation Article. Vanni T, Karnon J, Madan J et al. Calibrating Models in Economic Evaluation: A Seven-Step Approach. PharmacoEconomics 2011; 29 (1): 35-49. https://doi.org/10.2165/11584600-000000000-00000

Empirically Evaluating Decision-Analytic Models Article. Goldhaber-Fiebert JD, Stout NK, Goldie SJ. Empirically Evaluating Decision-Analytic Models. Value in Health 2010; 13 (5): 667-674. https://doi.org/10.1111/j.1524-4733.2010.00698.x

Validation of Population-Based Disease Simulation Models: A Review Review. Kopec JA, Finès P, Manuel DG et al. Validation of Population-Based Disease Simulation Models: A Review of Concepts and Methods. BMC Public Health 2010; 10: 710. https://doi.org/10.1186/1471-2458-10-710 EXAMPLES BY DISEASE AREA Empirically Calibrated Model of Hepatitis C Virus Infection in the United States Article. Salomon JA, Weinstein MC, Hammitt JK, Goldie SJ. Empirically Calibrated Model of Hepatitis C Virus Infection in the United States. American Journal of Epidemiology 2002; 156 (8): 761-773. https://www.ncbi.nlm.nih.gov/pubmed/12370165 Not open access.

Exploring Model Uncertainty in Economic Evaluation of Health Interventions: Rotavirus Vaccination in Vietnam Article. Kim SY, Goldie SJ, Salomon JA. Exploring Model Uncertainty in Economic Evaluation of Health Interventions: The Example of Rotavirus Vaccination in Vietnam. Medical Decision Making 2010; 30 (5): E1-E28. https://doi.org/10.1177/0272989X10375579 Not open access.

Eleven Mathematical Models of TB Article. Houben RMGJ, Menzies NA, Sumner T et al. Feasibility of Achieving the 2025 WHO Global Tuberculosis Targets in South Africa, China, and India: A Combined Analysis of 11 Mathematical Models. The Lancet Global Health 2016; 4 (11): e806-e815. http://dx.doi.org/10.1016/S2214-109X(16)30199-1

Assessing the Performance of a Computer-Based Policy Model of HIV and AIDS Article. Rydzak CE, Cotich KL, Sax PE et al. Assessing the Performance of a Computer-Based Policy Model of HIV and AIDS. PLoS ONE 2010; 5 (9): e12647. https://doi.org/10.1371/journal.pone.0012647

The Rise and Fall of HIV in High-Prevalence Countries: A Challenge for Mathematical Modeling Article. Nagelkerke NJD, Arora P, Jha P et al. The Rise and Fall of HIV in High-Prevalence Countries: A Challenge for Mathematical Modeling. PLoS Computational Biology 2014; 10 (3). https://doi.org/10.1371/journal.pcbi.1003459

Reduced Burden of Childhood Diarrheal Diseases through Increased Access to Water and Sanitation in India: A Modeling Analysis Article. Nandi A, Megiddo I, Ashok A et al. Reduced Burden of Childhood Diarrheal Diseases through Increased Access to Water and Sanitation in India: A Modeling Analysis. Social Science and Medicine 2017; 180: 181-192. http://www.sciencedirect.com/science/article/pii/S0277953616304853

Modeling Human Papillomavirus and Cervical Cancer in the United States for Analyses of Screening and Vaccination Article. Goldhaber-Fiebert JD, Stout NK, Ortendahl J et al. Modeling Human Papillomavirus and Cervical Cancer in the United States for Analyses of Screening and Vaccination. Population Health Metrics 2007; 5: 11. https://doi.org/10.1186/1478-7954-5-11

Page 6: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

6

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Modeling Cervical Cancer Prevention in Developed Countries Review. Kim JJ, Brisson M, Edmunds WJ, Goldie SJ. Modeling Cervical Cancer Prevention in Developed Countries. Vaccine 2008; 26 (Suppl 1): K76-K86. https://doi.org/10.1016/j.vaccine.2008.06.009 Not open access.

Health and Economic Implications of HPV Vaccination in the United States Article. Kim JJ, Goldie SJ. Health and Economic Implications of HPV Vaccination in the United States. NEJM 2008; 359 (8): 821-832. http://www.nejm.org/doi/full/10.1056/NEJMsa0707052#t=article

Cost Effectiveness Analysis of Including Boys in a HPV Vaccination Programme in the U.S. Article. Kim JJ, Goldie SJ. Cost Effectiveness Analysis of Including Boys in a Human Papillomavirus Vaccination Programme in the United States. BMJ 2009; 339: b3884. https://doi.org/10.1136/bmj.b3884

Including Boys in an HPV Vaccination Programme: A CEA in a Low-Resource Setting Article. Kim JJ, Andres-Beck B, Goldie SJ. The Value of Including Boys in an HPV Vaccination Programme: A Cost-Effectiveness Analysis in a Low-Resource Setting. British Journal of Cancer 2007; 97 (9): 1322-1328. https://www.nature.com/bjc/journal/v97/n9/full/6604023a.html

Health and Economic Impact of HPV 16/18 Vaccination and Cervical Cancer Screening in Eastern Africa Article. Campos NG, Kim JJ, Castle PE, Ortendahl JD, O'Shea M, Diaz M, Goldie SJ. Health and Economic Impact of HPV 16/18 Vaccination and Cervical Cancer Screening in Eastern Africa. International Journal of Cancer 2012; 130 (11): 2672-2684. http://onlinelibrary.wiley.com/doi/10.1002/ijc.26269/full

HPV Vaccine Introduction in LMIC's: Guidance on the Use of Cost-Effectiveness Models Guidelines. Jit M, Demarteau N, Elbasha E et al. Human Papillomavirus Vaccine Introduction in Low-Income and Middle-Income Countries: Guidance on the Use of Cost-Effectiveness Models. BMC Medicine 2011; 9 (1): 54. https://doi.org/10.1186/1741-7015-9-54

Modeling Preventative Strategies Against Human Papillomavirus-related Disease in Developed Countries Review. Canfell K, Chesson H, Kulasingam SL, Berkhof J, Diaz M, Kim JJ. Modeling Preventative Strategies against Human Papillomavirus-Related Disease in Developed Countries. Vaccine 2012; 30 (Suppl 5): F157-F167. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3783354

Unifying Screening Processes within the PROSPR Consortium: A Conceptual Model for Breast, Cervical, and Colorectal Cancer Screening Article. Beaber EF, Kim JJ, Schapira MM et al. Unifying Screening Processes within the PROSPR Consortium: A Conceptual Model for Breast, Cervical, and Colorectal Cancer Screening. Journal of the National Cancer Institute 2015; 107 (6): djv120. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4838064

Cancer Models and Real-World Data: Better Together Article. Kim J, Tosteson AN, Zauber AG et al. Cancer Models and Real-World Data: Better Together. Journal of the National Cancer Institute 2015; 108 (2). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4907359

Model-Based Analyses to Compare Health and Economic Outcomes of Cancer Control: Inclusion of Disparities Article. Goldie SJ, Daniels N. Model-Based Analyses to Compare Health and Economic Outcomes of Cancer Control: Inclusion of Disparities. Journal of the National Cancer Institute 2011; 103 (18): 1373-1386. https://doi.org/10.1093/jnci/djr303

Modeling to Improve Policy Decisions in the Americas: Noncommunicable Diseases Report. Legetic B, Cechini M, eds. Applying Modeling to Improve Health and Economic Policy Decisions in the Americas: The Case of Noncommunicable Diseases. Pan American Health Organization, Organisation for Economic Co-Operation and Development 2015. http://iris.paho.org/xmlui/handle/123456789/7700

Page 7: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

7

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Development of an Empirically Calibrated Model of Gastric Cancer in Two High-Risk Countries Article. Yeh JM, Kuntz KM, Ezzati M et al. Development of an Empirically Calibrated Model of Gastric Cancer in Two High-Risk Countries. Cancer Epidemiology Biomarkers & Prevention 2008; 17 (5): 1179-1187. http://cebp.aacrjournals.org/cgi/pmidlookup?view=long&pmid=18483340

Contribution of H. Pylori and Smoking to US Incidence of Gastric Adenocarcinoma: A Microsimulation Model Article. Yeh JM, Hur C, Schrag D, Kuntz KM, Ezzati M, Stout N, Ward Z, Goldie SJ. Contribution of H. Pylori and Smoking Trends to US Incidence of Intestinal-Type Noncardia Gastric Adenocarcinoma: A Microsimulation Model. PLoS Medicine 2013; 10 (5): e1001451. https://doi.org/10.1371/journal.pmed.1001451

Simulation Models of Obesity: A Review of the Literature Article. Levy DT, Mabry PL, Wang YC et al. Simulation Models of Obesity: A Review of the Literature and Implications for Research and Policy. Obesity Reviews 2011; 12 (5): 378-394. https://dx.doi.org/10.1111%2Fj.1467-789X.2010.00804.x Not open access.

Modeling the Risks and Benefits of Depression Treatment for Children and Young Adults Article. Soeteman DI, Miller M, Kim JJ. Modeling the Risks and Benefits of Depression Treatment for Children and Young Adults. Value in Health 2012; 15 (5): 724-729. http://dx.doi.org/10.1016/j.jval.2012.03.1390

Page 8: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

8

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Annotated Bibliography MODELS FOR HEALTH DECISIONS

Modeling to Inform Strategies to Improve Population Health Report. How Modeling Can Inform Strategies to Improve Population Health: Workshop Summary. The National Academies Press 2016. https://doi.org/10.17226/21807 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2398 This workshop report summarizes a workshop convened by the Institute of Medicine to explore the potential uses of simulation and other types of modeling for improving health. Participants worked to identify how modeling could inform population health decision making (selecting and refining potential strategies, ranging from interventions to investments) based on lessons learned from models that have been, or have not been, used successfully, opportunities and barriers to incorporating models into decision making, and data needs and opportunities to leverage existing data and to collect new data for modeling.

The workshop’s second panel featured three case studies to illustrate some of the ways in which models can be used to inform health policy. In each case, the models were nonlinear, dynamic, and interactive, and they cross multiple disciplines. The cases included (1) tobacco models; (2) EPA’s use of models to set clear air standards; and (3) communities that have used models to engage in regional health reform efforts. Decision Models in Clinical Preventive Services Recommendations Guidelines. Owens DK, Whitlock E, Henderson J et al. Use of Decision Models in the Development of Evidence-Based Clinical Preventive Services Recommendations: Methods of the U.S. Preventive Services Task Force. Annals of Internal Medicine 2016; 165 (7): 501-508. https://www.uspreventiveservicestaskforce.org/Page/Name/use-of-decision-models-in-the-development-of-evidence-based-clinical-preventive-services-recommendations CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2404 The U.S. Preventive Services Task Force (USPSTF) develops evidence-based recommendations about preventive care based on comprehensive systematic reviews of the best available evidence. Decision models provide a complementary, quantitative approach to support the USPSTF as it deliberates about the evidence and develops recommendations for clinical and policy use. This article describes the rationale for using modeling, an approach to selecting topics for modeling, and how modeling may inform recommendations about clinical preventive services. Decision and Simulation Modeling Alongside Systematic Reviews Report. Kuntz K, Sainfort F, Butler M et al. Chapter 1: Decision and Simulation Modeling Alongside Systematic Reviews in Kuntz K, Sainfort F, Butler M et al. Decision and Simulation Modeling in Systematic Reviews. US Agency for Healthcare Research and Quality 2013: 1-14. https://effectivehealthcare.ahrq.gov/topics/methods-decision-simulation-modeling/research CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3000 This chapter is part of a report entitled, Decision and Simulation Modeling in Systematic Reviews, that seeks to provide guidance for determining when incorporating a decision-analytic model alongside a systemic review would be of added value for decision making purposes.

The chapter discusses the role of decision analysis and decision-analytic models in health care, specifically within the context of the current emphasis on evidence-based medicine and the proliferation of systematic reviews. It describes the types of model available along with their limitations and propose when and how they can add value to the results of a systematic review.

The other 5 chapters in the report are:

Page 9: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

9

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

• Overview of Decision Models Used in Research which provides a “scan” of the past 5 years of medical literature on the use of models in studies that compare intervention strategies using multiple sources of data.

• Use of Modeling in Systematic Reviews: The EPC Perspective which highlights the extent to which Evidence-based Practice Centers (EPC) have incorporated models into data and presents results from key informant interviews with EPC members.

• Suggested Framework for Deciding When a Modeling Effort Should Be Added to a Systematic Review which presents a framework for deciding when a decision model can inform decision making alongside a systematic review.

• Potential Modeling Resources which discusses possible approaches and some of the challenges when undertaking modeling efforts.

• Best Practices for Decision and Simulation Modeling which combines the literature on modeling best practices with information from a focus group with modeling experts, and presents lessons learned about conducting a modeling exercise alongside a systematic review.

Adding Decision Models to Systematic Reviews Review. Sainfort F, Kuntz KM, Gregory S et al. Adding Decision Models to Systematic Reviews: Informing a Framework for Deciding When and How to Do So. Value in Health 2013; 16 (1): 133-139. https://doi.org/10.1016/j.jval.2012.09.009 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3009 Clarity is lacking about when and how to conduct modeling studies in tandem with systematic reviews, as well as about how to evaluate and present model results. This article reviews the use of decision models, specifically alongside systematic reviews, to synthesize evidence.

The authors collected data through 1) review and analysis of evidence reports that used decision models; 2) review and synthesis of current best practices for the development of decision models; 3) interviews of Evidence-Based Practice Center directors and selected staff, United States Preventive Services Task Force members, and decision modelers who developed models used by the United States Preventive Services Task Force; and 4) a focus group of expert modelers.

The results show that models are well suited to address gaps in the literature, better suited for certain types of research questions, and essential for determining the value of information relating to future research. Opinions differ regarding whether model outputs constitute evidence, but interviewees expressed concern over the lack of standards and directions in grading and reporting such “evidence.” Interviews of stakeholders and modelers revealed the importance of communication and presentation of model results as well as the importance of model literacy and involvement of stakeholders. Modeling for Health Care and Other Policy Decisions: Uses, Roles and Validity Review. Weinstein MC, Toy EL, Sandberg EA, Neumann PJ, Kuntz KM, Hammitt JK et al. Modeling for Health Care and Other Policy Decisions: Uses, Roles, and Validity. Value in Health 2001; 4 (5): 348-361. https://doi.org/10.1046/j.1524-4733.2001.45061.x CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3002 This is a review article of the role of modeling approaches to guide decision making in health care and other domains. The role of models to support recommendations on the cost-effective use of medical technologies and pharmaceuticals is controversial. At the heart of the controversy is the degree to which experimental or other empirical evidence should be required prior to model use.

The authors argue that the controversy stems in part from a misconception that the role of models is to establish truth rather than to guide clinical and policy decisions. In other domains of public policy that involve human life and health, such as environmental protection and defense strategy, models are generally accepted as decision aids, and many models have been formally incorporated into regulatory processes and governmental decision making.

Page 10: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

10

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

The authors formulate an analytical framework for evaluating the role of models as aids to decision making. Implications for the implementation of Section 114 of the Food and Drug Administration Modernization Act (FDAMA) are derived from this framework. TYPES OF MODELS FOR HEALTH DECISIONS

Using Data-Driven Agent-Based Models to Forecast Emerging Infectious Diseases Article. Venkatramanan S, Lewis B, Chen J et al. Using Data-Driven Agent-Based Models for Forecasting Emerging Infectious Diseases. Epidemics 2017. https://doi.org/10.1016/j.epidem.2017.02.010 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2996 This paper describes an agent-based model framework developed to forecast the 2014-15 Ebola epidemic, which was subsequently used in the Ebola forecasting challenge. Producing timely and reliable forecasts for an epidemic of an emerging infectious disease is a challenge. Epidemiologists and policy makers have to deal with poor data quality, limited understanding of the disease dynamics, a rapidly changing social environment and the uncertainty around the effects of various interventions in place.

In this setting, detailed computational models provide a comprehensive framework for integrating diverse data sources into a well-defined model of disease dynamics and social behavior, potentially leading to better understanding and actions. The authors describe the various components of their model, the calibration process and summarize the forecast performance across different scenarios. They conclude by highlighting how such a data-driven approach can be refined and adapted for future epidemics. Combining Microsimulation and Agent-Based Modeling Article. Bae JW, Paik E, Kim K et al. Combining Microsimulation and Agent-Based Model for Micro-Level Population Dynamics. Procedia Computer Science 2016; 80: 507-517. https://doi.org/10.1016/j.procs.2016.05.331 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2995 This paper proposes a hybrid model structure combining microsimulation and agent-based modeling to simulate population dynamics. Microsimulation describes the population dynamics at the individual level, and actions conducted by the individuals are generated by stochastic processes. An emerging method is the agent-based model, which focuses on the interactions among individuals and expects to see unexpected situations created from the interactions.

In the proposed hybrid model, the microsimulation model takes a role to depict how an individual chooses its behavior based on a stochastic process parameterized by real data; the agent-based model incorporates interactions among individuals considering their own states and rules. The case study introduces a Korean population dynamics model developed by the proposed approach, and its simulation results show the population changes triggered by a variance of behavior and interaction factors. Choosing an Epidemiological Model Structure for Economic Evaluation Review. Briggs ADM, Wolstenholme J, Blakely T et al. Choosing an Epidemiological Model Structure for the Economic Evaluation of Non-Communicable Disease Public Health Interventions. Population Health Metrics 2016; 14: 17. https://dx.doi.org/10.1186%2Fs12963-016-0085-1 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3003 This review presents a taxonomy of epidemiological model structures and applies it to the economic evaluation of public health interventions for non-communicable diseases. Growing pressures on health services and on social care have led to a greater need for prevention of chronic diseases. In order for decision makers to make informed judgements about how to best spend finite public health resources, they must be able to quantify the anticipated costs, benefits, and opportunity costs of each prevention option available.

Page 11: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

11

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Through a discussion of the pros and cons of model structures and examples of their application to public health interventions, the authors suggest that individual-level models may be better than population-level models for estimating the effects of population heterogeneity. Furthermore, model structures allowing for interactions between populations, their environment, and time are often better suited to complex multifaceted interventions. Other influences on the choice of model structure include time and available resources, and the availability and relevance of previously developed models. Agent-Based Models and Microsimulation Review. Heard D, Dent G, Schifeling T et al. Agent-Based Models and Microsimulation. Annual Review of Statistics and Its Application 2015; 2: 259-272. https://doi.org/10.1146/annurev-statistics-010814-020218 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3005 This article reviews the principles and applications of agent-based models (ABMs). ABMs are computational models used to simulate the actions and interactions of “agents” within a system. Usually, each agent has a set of rules for how he or she responds to the environment and to other agents. These models are used to gain insight into the emergent behavior of complex systems with many agents, in which the emergent behavior depends upon the micro-level behavior of the individuals. ABMs are widely used in many fields, and this article reviews some of those applications. However, as relatively little work has been done on statistical inference for such models, this article also points out some of those gaps and recent strategies to address them. Dynamic Microsimulation Models for Health Outcomes: a Review Review. Rutter CM, Zaslavsky A, Feuer E. Dynamic Microsimulation Models for Health Outcomes: a Review. Medical Decision Making 2011; 31 (1): 10-18. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3404886/pdf/nihms-393222.pdf CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3001 This review article presents an overview of microsimulation modeling, focusing on the development and application of these models for health policy questions. Microsimulation models for health outcomes simulate individual event histories associated with key components of a disease process; these simulated life histories can be aggregated to estimate population-level effects of treatment on disease outcomes and the comparative effectiveness of treatments.

The authors argue that methodological improvements in modeling approaches have been slowed by the lack of communication among modelers. In addition, there are few resources to guide individuals who may wish to use microsimulation projections to inform decisions. The authors discuss goals, overall components of the models, methods for selecting parameters to reproduce observed or expected results (calibration), methods for model checking (validation), and issues related to reporting and interpreting MSM findings (sensitivity analyses, reporting of variability, and model transparency). Decision Support for Infectious Disease Control Review. Manheim D, Chamberlin M, Osoba O et al. Improving Decision Support for Infectious Disease Prevention and Control: Aligning Models and Other Tools with Policymakers' Needs. RAND Corporation 2016. https://www.rand.org/pubs/research_reports/RR1576.html CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2370 This report from RAND reviews decision-support tools, including models and nonmodeling approaches, that are relevant to infectious disease prevention, detection, and response and aligns these tools with real-world policy questions that the tools can help address. This overview is designed to help modelers and other technical experts understand the questions that policymakers will raise and the decisions they must make.The report also presents policymakers with the capabilities and limitations of the different tools that may inform their decisions.

Page 12: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

12

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

This report describes the characteristics, requirements, uses, applicability, and limitations of three classes of theory-based models (population, microsimulation, agent-based simulation) and two classes of statistical models (regression-based and machine-learning), as well as several complementary nonmodeling decision-support approaches. The report then aligns all of these tools and approaches with a set of real-world policy questions.

Finally, based on a review of published literature, an assessment of the different models and nonmodeling approaches, and recent experiences (such as the 2009 influenza pandemic), the authors recommend nine best practices for using modeling and decision-support tools to inform policymaking.

This research was conducted within the Forces and Resources Policy Center of the National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community. Markov Modeling and Discrete Event Simulation in Health Care: a Systematic Comparison Review. Standfield L, Comans T, Scuffham P. Markov Modeling and Discrete Event Simulation in Health Care: a Systematic Comparison. International Journal of Technology Assessment in Health Care 2014; 30 (2): 165-172. https://doi.org/10.1017/S0266462314000117 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3004 This review assesses whether the use of Markov modeling (MM) or discrete event simulation (DES) for cost-effectiveness analysis (CEA) may alter healthcare resource allocation decisions. A systematic literature search and review of empirical and non-empirical studies comparing MM and DES techniques used in the CEA of healthcare technologies was conducted.

The primary advantages described for DES over MM were the ability to model queuing for limited resources, capture individual patient histories, accommodate complexity and uncertainty, represent time flexibly, model competing risks, and accommodate multiple events simultaneously. The disadvantages of DES over MM were the potential for model overspecification, increased data requirements, specialized expensive software, and increased model development, validation, and computational time.

The authors conclude that where individual patient history is an important driver of future events, an individual patient simulation technique like DES may be preferred over MM. Where supply shortages, subsequent queuing, and diversion of patients through other pathways in the healthcare system are likely to be drivers of cost-effectiveness, DES modeling methods may provide decision makers with more accurate information on which to base resource allocation decisions. Cost-Effectiveness of Vaccination: Review of Modelling Approaches Review. Kim SY, Goldie SJ. Cost-Effectiveness Analyses of Vaccination Programmes: A Focused Review of Modelling Approaches. PharmacoEconomics 2008; 26 (3): 191-215. https://doi.org/10.2165/00019053-200826030-00004 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2396 This review examines the modelling approaches used in cost-effectiveness analyses (CEAs) of vaccination programmes. After overviewing the key attributes of models used in CEAs, a framework for categorising theoretical models is presented. Categories are based on three main attributes: static/dynamic; stochastic/deterministic; and aggregate/individual based. Economic Evaluations with Agent-Based Modelling: An Introduction Tutorial/Primer. Chhatwal J, He T. Economic Evaluations with Agent-Based Modelling: An Introduction. PharmacoEconomics 2015; 33 (5): 423-433. http://dx.doi.org/10.1007/s40273-015-0254-2

Page 13: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

13

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2416 This tutorial presents the basic concepts and important features of agent-based models, and provides a case study of a simple cost effectiveness analysis of screening for an infectious disease. Agent-based modelling (ABM) is a relatively new technique, which overcomes some of the limitations of other methods commonly used for economic evaluations. These limitations include linearity, homogeneity and stationarity. Agents in ABMs are autonomous entities, who interact with each other and with the environment. ABMs provide an inductive or ‘bottom-up’ approach, i.e. individual-level behaviours define system-level components. ABMs have a unique property to capture emergence phenomena that otherwise cannot be predicted by the combination of individual-level interactions. The authors provide their model, developed using an open-source software program, NetLogo, and discuss software, resources, challenges and future research opportunities. TUTORIALS AND PRIMERS

Bayesian Methods for Calibrating Health Policy Models: A Tutorial Tutorial/Primer. Menzies NA, Soeteman D, Pandya A et al. Bayesian Methods for Calibrating Health Policy Models: A Tutorial. PharmacoEconomics 2017; 35 (6): 613-624. http://dx.doi.org/10.1007/s40273-017-0494-4 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2412 This article provides a tutorial on Bayesian approaches for model calibration. It describes the theoretical basis for Bayesian calibration approaches as well as pragmatic considerations that arise in the tasks of creating calibration targets, estimating the posterior distribution, and obtaining results to inform the policy decision. These considerations, as well as the specific steps for implementing the calibration, are described in the context of an extended worked example about the policy choice to provide (or not provide) treatment for a hypothetical infectious disease. Calibration of Complex Models through Bayesian Evidence Synthesis: A Tutorial Tutorial/Primer. Jackson C, Jit M, Sharples L et al. Calibration of Complex Models through Bayesian Evidence Synthesis: A Demonstration and Tutorial. Medical Decision Making 2015; 35 (2): 148-161. https://doi.org/10.1177/0272989X13493143 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2417 This tutorial demonstrates how to implement a Bayesian synthesis of diverse sources of evidence to calibrate the parameters of a complex model. To illustrate these methods, the authors demonstrate how a previously developed Markov model for the progression of human papillomavirus (HPV-16) infection was rebuilt in a Bayesian framework. Transition probabilities between states of disease severity are inferred indirectly from cross-sectional observations of prevalence of HPV-16 and HPV-16–related disease by age, cervical cancer incidence, and other published information. The authors derive a Bayesian posterior distribution, in which scenarios are implicitly weighted according to how well they are supported by the data. MODELING GUIDELINES

Decision and Simulation Modeling in Systematic Reviews Guidelines. Kuntz K, Sainfort F, Butler M et al. Decision and Simulation Modeling in Systematic Reviews. Methods Research Report. Agency for Health Care Research and Quality 2013. https://ahrq-ehc-application.s3.amazonaws.com/media/pdf/methods-decision-simulation-modeling_research.pdf CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2397 The purpose of this review is to provide guidance for determining when incorporating a decision-analytic model alongside a systemic review would be of added value for decision making purposes. The purpose of systematic reviews is to synthesize the current scientific literature on a particular topic in the form of evidence reports and technology assessments to assist public and private organizations in developing strategies that improve the quality of health care and decision making. However, there is often not enough evidence to fully address the questions that are relevant for

Page 14: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

14

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

decision makers. Decision models may provide added value alongside systematic reviews by adding a formal structure, which can be informed by the evidence. Modeling Good Research Practices - Overview: A Report of the ISPOR-SMDM Modeling Task Force-1 Guidelines. Caro JJ, Briggs AH, Siebert U et al. Modeling Good Research Practices - Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Value in Health 2012; 15: 796-803. https://www.ispor.org/workpaper/Modeling-Good-Research-Practices-Overview.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2423 This paper provides an overview of the work of the joint Task Force between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM), provides the overarching recommendations, and discusses future work that is needed. The audience for these papers includes anyone who build models, stakeholders who utilize their results, and those concerned with the use of models to support decision making.

This article is part 1 of a series of seven papers presenting the updated recommendations for best practices in conceptualizing models; implementing state–transition approaches, discrete event simulations, or dynamic transmission models; dealing with uncertainty; and validating and reporting models transparently.

The other articles include:

• Conceptualizing a Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2 • State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 • Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices

Task Force-4 • Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

5 • Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-6 • Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task

Force-7 Conceptualizing a Model: A Report of the ISPOR-SMDM Modeling Task Force-2 Guidelines. Roberts M, Russell LB, Paltiel AD et al. Conceptualizing a Model: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2. Value in Health 2012; 15: 804-811. https://www.ispor.org/workpaper/Conceptualizing-A-Model.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2420 This article provides a series of consensus-based best practices regarding the process of model conceptualization, specifically when models are used to inform medical decisions and health-related resource allocation. The authors divide the conceptualization process into two distinct components: the conceptualization of the problem, which converts knowledge of the health care process or decision into a representation of the problem, followed by the conceptualization of the model itself, which matches the attributes and characteristics of a particular modeling type to the needs of the problem being represented.

Recommendations are made regarding the structure of the model, the statement of the problem, the perspective and target population, the interventions and outcomes represented as well as the specific characteristics of the problem that might be most easily represented.

This paper is one of a 7-part series of articles on modeling good research practices based on a collaboration between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

Page 15: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

15

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

The other articles include:

• Modeling Good Research Practices – Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1

• State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 • Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices

Task Force-4 • Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

5 • Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-6 • Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task

Force-7 State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Task Force-3 Guidelines. Siebert U, Alagoz O, Bayoumi AM et al. State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value Health 2012; 15: 812-820. https://www.ispor.org/workpaper/State-Transition-Modeling.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2419 State-transition modeling includes both Markov model cohort simulation as well as individual-based (first-order Monte Carlo) microsimulation. These models have been used in many different populations and diseases, and their applications range from personalized health care strategies to public health programs. Most frequently, state-transition models are used in the evaluation of risk factor interventions, screening, diagnostic procedures, treatment strategies, and disease management programs.

Recommendations are made on choice of model type (cohort vs. individual-level model), model structure, model parameters, analysis, reporting, and communication with specific examples from the literature. These recommendations for state-transition modeling are directed both to modelers and to users of modeling results such as clinicians, clinical guideline developers, manufacturers, or policymakers.

This paper is one of a 7-part series of articles on modeling good research practices based on a collaboration between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

The other articles include:

• Modeling Good Research Practices – Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1

• Conceptualizing a Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2 • Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices

Task Force-4 • Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

5 • Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-6 • Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task

Force-7

Page 16: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

16

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Task Force-4 Guidelines. Karnon J, Stahl JE, Brennan A et al. Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Value in Health 2012; 15: 821-827. https://www.ispor.org/workpaper/Modeling-Using-Discrete-Event-Simulation.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2422 This paper reports on consensus-based guidelines on the application of DES in a health care setting, covering the range of issues to which DES can be applied. Discrete event simulation (DES) is a form of computer-based modeling that provides an intuitive and flexible approach to representing complex systems. The article works through the different stages of the modeling process: structural development, parameter estimation, model implementation, model analysis, and representation and reporting.

Recommendations are made for each stage of the modeling process. Descriptions and discussion of issues that are of particular relevance to the application of DES in a health care setting are also provided.

This paper is one of a 7-part series of articles on modeling good research practices based on a collaboration between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

The other articles include:

• Modeling Good Research Practices – Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1

• Conceptualizing a Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2 • State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 • Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

5 • Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-6 • Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task

Force-7 Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Task Force-5 Guidelines. Pitman RJ, Fisman D, Zaric GS et al. Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-5. Value in Health 2012; 15: 828-834. https://www.ispor.org/workpaper/Dynamic-Transmission-Modeling.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2421 This paper reports the consensus-based guidelines on dynamic transmission modeling in health care. The transmissible nature of communicable diseases is what sets them apart from other diseases modeled by health economists. The probability of a susceptible individual becoming infected at any one point in time (the force of infection) is related to the number of infectious individuals in the population, will change over time, and will feed back into the future force of infection. These nonlinear interactions produce transmission dynamics that require specific consideration when modeling an intervention that has an impact on the transmission of a pathogen.

Recommendations are provided for designing, building and best use of transmission models.

This paper is one of a 7-part series of articles on modeling good research practices based on a collaboration between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

Page 17: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

17

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

The other articles include:

• Modeling Good Research Practices – Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1

• Conceptualizing a Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2 • State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 • Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices

Task Force-4 • Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-6 • Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task

Force-7 Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Task Force-6 Guidelines. Briggs AH, Weinstein MC, Fenwick E et al. Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value in Health 2012; 15: 835-842. https://www.ispor.org/workpaper/Model-Parameter-Estimation-and-Uncertainty-Analysis.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2424 This paper discusses methods for the reporting of uncertainty, both in terms of deterministic sensitivity analysis techniques and probabilistic methods. Stochastic (first-order) uncertainty is distinguished from both parameter (second-order) uncertainty and from heterogeneity, with structural uncertainty relating to the model itself forming another level of uncertainty. The article points out that the estimation of point estimates and uncertainty in parameters is part of a single process and explores the link between parameter uncertainty through to decision uncertainty and the relationship to value-of-information analysis.

Recommendations are provided on best choices for reporting, including expected value of perfect information, which is argued to be the most appropriate presentational technique, alongside cost-effectiveness acceptability curves, for representing decision uncertainty from probabilistic analysis.

This paper is one of a 7-part series of articles on modeling good research practices based on a collaboration between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

The other articles include:

• Modeling Good Research Practices – Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1

• Conceptualizing a Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2 • State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 • Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices

Task Force-4 • Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

5 • Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task

Force-7

Page 18: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

18

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Task Force-7 Guidelines. Eddy DM, Hollingworth W, Caro JJ et al. Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value in Health 2012; 15: 843-850. https://doi.org/10.1177/0272989X12454579 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2411 This paper discusses how to improve trust in the use of health care models through validation and transparency. Validation involves face validity (wherein experts evaluate model structure, data sources, assumptions, and results), verification or internal validity (check accuracy of coding), cross validity (comparison of results with other models analyzing same problem), external validity (comparing model results to real-world results), and predictive validity (comparing model results with prospectively observed events).

Recommendations are provided for nontechnical description (model type, intended applications, funding sources, structure, inputs, outputs, data sources, validation methods, results, and limitations) as well as technical documentation, which should be written in sufficient detail to enable a reader with necessary expertise to evaluate the model and potentially reproduce it.

This paper is one of a 7-part series of articles on modeling good research practices based on a collaboration between the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

The other articles include:

• Modeling Good Research Practices – Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1

• Conceptualizing a Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2 • State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 • Modeling Using Discrete Event Simulation: A Report of the ISPOR-SMDM Modeling Good Research Practices

Task Force-4 • Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

5 • Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-6 ISPOR Task Force Report: Good Practice for Decision Analytic Modeling in Health-Care Report. Weinstein MC, O’Brien B, Hornberger J et al. Principles of Good Practice for Decision Analytic Modeling in Health-Care Evaluation: Report of the ISPOR Task Force on Good Research Practices--Modeling Studies. Value in Health 2003; 6 (1): 9-17. https://www.ispor.org/workpaper/healthscience/TFModeling.asp CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2993 This report describes the consensus of a task force convened to provide modelers with guidelines for conducting and reporting modeling studies. While published more than a decade ago, it remains a clearly written resource for thinking about how to accurately describe the components of models and their quality.

Criteria for assessing the quality of models fell into three areas: model structure, data used as inputs to models, and model validation. Several major themes cut across these areas. Models and their results should be represented as aids to decision making, not as statements of scientific fact; therefore, it is inappropriate to demand that models be validated prospectively before use.

However, model assumptions regarding causal structure and parameter estimates should be continually assessed against data, and models should be revised accordingly. Structural assumptions and parameter estimates should be

Page 19: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

19

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

reported clearly and explicitly, and opportunities for users to appreciate the conditional relationship between inputs and outputs should be provided through sensitivity analyses. CALIBRATION AND VALIDATION

Validation and Calibration of Structural Models that Combine Information from Multiple Sources Review. Dahabreh IJ, Wong JB, Trikalinos TA. Validation and Calibration of Structural Models that Combine Information from Multiple Sources. Expert Review of Pharmacoeconomics and Outcomes Research 2017; 17 (1): 27-37. https://doi.org/10.1080/14737167.2017.1277143 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2990 This is a review of calibration and validation methods in mathematical modeling. Such models that attempt to capture structural relationships between their components and combine information from multiple sources are increasingly used in medicine. The authors provide an overview of methods for model validation and calibration and survey studies comparing alternative approaches. Model validation entails a confrontation of models with data, background knowledge, and other models, and can inform judgments about model credibility.

Calibration involves selecting parameter values to improve the agreement of model outputs with data. When the goal of modeling is quantitative inference on the effects of interventions or forecasting, calibration can be viewed as estimation. This view clarifies issues related to parameter identifiability and facilitates formal model validation and the examination of consistency among different sources of information. In contrast, when the goal of modeling is the generation of qualitative insights about the modeled phenomenon, calibration is a rather informal process for selecting inputs that result in model behavior that roughly reproduces select aspects of the modeled phenomenon and cannot be equated to an estimation procedure.

Current empirical research on validation and calibration methods consists primarily of methodological appraisals or case-studies of alternative techniques and cannot address the numerous complex and multifaceted methodological decisions that modelers must make. Further research is needed on different approaches for developing and validating complex models that combine evidence from multiple sources. Likelihood Approach for Calibration of Stochastic Epidemic Models Article. Zimmer C, Yaesoubi R, Cohen T. A Likelihood Approach for Real-Time Calibration of Stochastic Compartmental Epidemic Models. PLoS Computational Biology 2017; 13 (1): e1005257. https://doi.org/10.1371/journal.pcbi.1005257 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2549 Stochastic transmission dynamic models are especially useful for studying the early emergence of novel pathogens given the importance of chance events when the number of infectious individuals is small. However, methods for parameter estimation and prediction for these types of stochastic models remain limited. This paper describes a calibration and prediction framework for stochastic compartmental transmission models of epidemics.

The proposed method applies a linear noise approximation to describe the size of the fluctuations, and uses each new surveillance observation to update the belief about the true epidemic state. Using simulated outbreaks of a novel viral pathogen, the authors evaluate the accuracy of this method (multiple shooting for stochastic systems or MSS) for real-time parameter estimation and prediction during epidemics. They assume that weekly counts for the number of new diagnosed cases are available and serve as an imperfect proxy of incidence.

The authors compare the performance of MSS to three state-of-the-art benchmark methods: 1) a likelihood approximation with an assumption of independent Poisson observations; 2) a particle filtering method; and 3) an ensemble Kalman filter method. They find that MSS significantly outperforms each of these three benchmark methods in the majority of epidemic scenarios tested.

Page 20: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

20

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Calibration of Complex Models through Bayesian Evidence Synthesis: A Tutorial Tutorial/Primer. Jackson C, Jit M, Sharples L et al. Calibration of Complex Models through Bayesian Evidence Synthesis: A Demonstration and Tutorial. Medical Decision Making 2015; 35 (2): 148-161. https://doi.org/10.1177/0272989X13493143 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2417 This tutorial demonstrates how to implement a Bayesian synthesis of diverse sources of evidence to calibrate the parameters of a complex model. To illustrate these methods, the authors demonstrate how a previously developed Markov model for the progression of human papillomavirus (HPV-16) infection was rebuilt in a Bayesian framework. Transition probabilities between states of disease severity are inferred indirectly from cross-sectional observations of prevalence of HPV-16 and HPV-16–related disease by age, cervical cancer incidence, and other published information. The authors derive a Bayesian posterior distribution, in which scenarios are implicitly weighted according to how well they are supported by the data. Bayesian Methods for Calibrating Health Policy Models: A Tutorial Tutorial/Primer. Menzies NA, Soeteman D, Pandya A et al. Bayesian Methods for Calibrating Health Policy Models: A Tutorial. PharmacoEconomics 2017; 35 (6): 613-624. http://dx.doi.org/10.1007/s40273-017-0494-4 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2412 This article provides a tutorial on Bayesian approaches for model calibration. It describes the theoretical basis for Bayesian calibration approaches as well as pragmatic considerations that arise in the tasks of creating calibration targets, estimating the posterior distribution, and obtaining results to inform the policy decision. These considerations, as well as the specific steps for implementing the calibration, are described in the context of an extended worked example about the policy choice to provide (or not provide) treatment for a hypothetical infectious disease. Validating a Cardiovascular Disease Microsimulation Model Article. Pandya A, Sy S, Cho S, Alam S, Weinstein MC, Gaziano TA. Validation of a Cardiovascular Disease Policy Microsimulation Model Using Both Survival and Receiver Operating Characteristic Curves. Medical Decision Making 2017; 37 (7): 802-814. https://doi.org/10.1177/0272989X17706081 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2918 This paper describes examines a cardiovascular disease model used to evaluate prevention and treatment. The authors perform calibration and a validation process in which simulated results were compared to observed all-cause and CVD-specific mortality data from the National Health and Nutrition Examination Survey using survival curves and ROC curves. Validation and Calibration of a Simulation Model of Pediatric HIV Infection Article. Ciaranello AL, Morris B, Walensky R et al. Validation and Calibration of a Computer Simulation Model of Pediatric HIV Infection. PLoS One 2013; 8 (12): e83389. https://doi.org/10.1371/journal.pone.0083389 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2832 The authors developed a microsimulation model of pediatric HIV disease progression using the Cost-Effectiveness of Preventing AIDS Complications (CEPAC) model framework. This CEPAC-Pediatric model was then validated by varying CD4 data and comparing the corresponding model-generated survival curves to empirical survival curves obtained from the International Epidemiologic Database to Evaluate AIDS (IeDEA). The model was calibrated to other African countries by systematically varying immunologic and HIV mortality-related input parameters. In the calibration analyses, the model-generated survival curves were compared against UNAIDS data.

The findings indicated that the model-generated survival curves fit the IeDEA data well (survival at 16 months was 91.2% and 91.1%, respectively). The calibration analyses showed that increases in IeDEA-derived mortality risks were necessary to fit the UNAIDS survival data.

Page 21: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

21

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Based on these results, the authors conclude that the CEPAC-Pediatric model is internally valid and that the increases in modeled mortality risks that were required to match the UNAIDS data highlight the importance of pre-enrollment mortality in many pediatric cohort studies. Calibrating Models in Economic Evaluation Article. Vanni T, Karnon J, Madan J et al. Calibrating Models in Economic Evaluation: A Seven-Step Approach. PharmacoEconomics 2011; 29 (1): 35-49. https://doi.org/10.2165/11584600-000000000-00000 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2593 This article provides guidance on the theoretical underpinnings of different calibration methods used for mathematical models for economic evaluations. The calibration process is divided into seven steps and different potential methods at each step are discussed, focusing on the particular features of disease models in economic evaluation. The seven steps are (i) Which parameters should be varied in the calibration process? (ii) Which calibration targets should be used? (iii) What measure of goodness of fit should be used? (iv) What parameter search strategy should be used? (v) What determines acceptable goodness-of-fit parameter sets (convergence criteria)? (vi) What determines the termination of the calibration process (stopping rule)? (vii) How should the model calibration results and economic parameters be integrated?

Models based on scientific knowledge of disease use simplifying assumptions, and contain input parameters with varying levels of uncertainty. Calibration is one tool for estimating uncertain parameters and more accurately defining model uncertainty. Calibration involves the comparison of model outputs with empirical data, leading to the identification of model parameter values that achieve a good fit. The lack of standards in calibrating disease models in economic evaluation can undermine the credibility of calibration methods. In order to avoid public scepticism regarding calibration, the authors present a unified approach to the problem and report the various methods used. Empirically Evaluating Decision-Analytic Models Article. Goldhaber-Fiebert JD, Stout NK, Goldie SJ. Empirically Evaluating Decision-Analytic Models. Value in Health 2010; 13 (5): 667-674. https://doi.org/10.1111/j.1524-4733.2010.00698.x CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3026 To augment model credibility, evaluation via comparison to independent, empirical studies is recommended. The authors developed a structured reporting format for model evaluation and conducted a structured literature review to characterize current model evaluation recommendations and practices.

As an illustration, they applied the reporting format to evaluate a microsimulation of human papillomavirus and cervical cancer. The model's outputs and uncertainty ranges were compared with multiple outcomes from a study of long-term progression from high-grade precancer (cervical intraepithelial neoplasia [CIN]) to cancer. Outcomes included 5 to 30-year cumulative cancer risk among women with and without appropriate CIN treatment. Consistency was measured by model ranges overlapping study confidence intervals.

The structured reporting format included: matching baseline characteristics and follow-up, reporting model and study uncertainty, and stating metrics of consistency for model and study results. Structured searches yielded 2963 articles with 67 meeting inclusion criteria and found variation in how current model evaluations are reported. Evaluation of the cervical cancer microsimulation, reported using the proposed format, showed a modeled cumulative risk of invasive cancer for inadequately treated women of 39.6% (30.9-49.7) at 30 years, compared with the study: 37.5% (28.4-48.3). For appropriately treated women, modeled risks were 1.0% (0.7-1.3) at 30 years, study: 1.5% (0.4-3.3). Validation of Population-Based Disease Simulation Models: A Review Review. Kopec JA, Finès P, Manuel DG et al. Validation of Population-Based Disease Simulation Models: A Review of Concepts and Methods. BMC Public Health 2010; 10: 710. https://doi.org/10.1186/1471-2458-10-710 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2991

Page 22: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

22

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

This article develops a framework for validating population-based chronic disease simulation models, and reviews the principles and methods for such models. While computer simulation models are used increasingly to support public health research and policy, questions about their quality persist.

Based on the review, the authors formulated a set of recommendations for gathering evidence of model credibility. They find that evidence of model credibility derives from examining: 1) the process of model development, 2) the performance of a model, and 3) the quality of decisions based on the model. Many important issues in model validation are insufficiently addressed by current guidelines.

These issues include a detailed evaluation of different data sources, graphical representation of models, computer programming, model calibration, between-model comparisons, sensitivity analysis, and predictive validity. The role of external data in model validation depends on the purpose of the model (e.g., decision analysis versus prediction). EXAMPLES BY DISEASE AREA

Empirically Calibrated Model of Hepatitis C Virus Infection in the United States Article. Salomon JA, Weinstein MC, Hammitt JK, Goldie SJ. Empirically Calibrated Model of Hepatitis C Virus Infection in the United States. American Journal of Epidemiology 2002; 156 (8): 761-773. https://www.ncbi.nlm.nih.gov/pubmed/12370165 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3020 This article presents an epidemiologic model of hepatitis C in the United States. The authors used empirical calibration of model parameters to gain insights into uncertainty in the natural history of hepatitis C and to improve future projections.

Using sampled values from plausible ranges for parameters that were identified using a systematic review of the literature multiple model simulations were conducted and model predictions for each set of sampled values were compared with epidemiologic data on infection prevalence and mortality from liver cancer. Goodness-of-fit criteria were used to identify parameter values that were consistent with these data.

Results indicated that rates of progression to advanced liver disease may be lower than previously assumed. The authors also found a wide range of plausible assumptions about heterogeneity beyond what could be explained by age and sex that were all consistent with observed epidemiologic trends. Exploring Model Uncertainty in Economic Evaluation of Health Interventions: Rotavirus Vaccination in Vietnam Article. Kim SY, Goldie SJ, Salomon JA. Exploring Model Uncertainty in Economic Evaluation of Health Interventions: The Example of Rotavirus Vaccination in Vietnam. Medical Decision Making 2010; 30 (5): E1-E28. https://doi.org/10.1177/0272989X10375579 Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3025 Motivated by observed discrepancies between 2 published studies on the cost-effectiveness of rotavirus vaccination in Vietnam, the authors' objectives were to illustrate a specific, systematic approach to assessing model (structure and process) uncertainty and to quantify explicitly the contributions of different sources of variation in the outputs of different studies that share the same research question.

On the basis of a series of working definitions of key model elements, the authors developed 5 alternative computer simulation (state-transition) models of rotavirus disease. They examined how epidemiological outcomes and cost-effectiveness ratios associated with rotavirus vaccination would change as elements of model structure and modeling process were progressively modified. They also explicitly decomposed the relative contributions of different modeling elements to differences in the cost-effectiveness results between the 2 previous analyses motivating the present study.

Page 23: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

23

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

The findings suggest that within the category of a static, deterministic, aggregate-level model, different choices in model structure and process lead to relatively modest differences in the estimated cost-effectiveness of rotavirus vaccination, but that intermediate epidemiologic outcomes vary more substantially depending on the choice of model structure.

The authors caution against generalizing the quantitative results in this study beyond the present example but suggest that the approach presented here may serve as a template for other examinations of model uncertainty. Eleven Mathematical Models of TB Article. Houben RMGJ, Menzies NA, Sumner T et al. Feasibility of Achieving the 2025 WHO Global Tuberculosis Targets in South Africa, China, and India: A Combined Analysis of 11 Mathematical Models. The Lancet Global Health 2016; 4 (11): e806-e815. http://dx.doi.org/10.1016/S2214-109X(16)30199-1 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2915 To assess the feasibility of goals to reduce TB incidence and mortality, 11 independent models were empirically calibrated and used to simulate prevention, diagnosis, and treatment strategies in China, India, South Africa. While drivers of between-model differences were identified, public health findings were robust. Assessing the Performance of a Computer-Based Policy Model of HIV and AIDS Article. Rydzak CE, Cotich KL, Sax PE et al. Assessing the Performance of a Computer-Based Policy Model of HIV and AIDS. PLoS ONE 2010; 5 (9): e12647. https://doi.org/10.1371/journal.pone.0012647 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3027 Description of calibration processes serve to enhance the transparency of disease-specific models. This paper reports on the process of adapting a computer-based simulation model of HIV disease to the Women's Interagency HIV Study (WIHS) cohort and assesses model performance and to address policy questions in the U.S. relevant to HIV-infected women when using the study data. Calibration targets included 24-month survival curves stratified by treatment status and CD4 cell count.

The authors found that assumptions around chronic HIV-associated mortality following an opportunistic infection in untreated women, effectiveness of HAART and the ability of HAART to prevent complications were the most influential. Once the authors found good-fitting parameter sets, projected rates of treatment regimen switching using the calibrated cohort-specific model closely approximated independent analyses published using data from the WIHS. The Rise and Fall of HIV in High-Prevalence Countries: A Challenge for Mathematical Modeling Article. Nagelkerke NJD, Arora P, Jha P et al. The Rise and Fall of HIV in High-Prevalence Countries: A Challenge for Mathematical Modeling. PLoS Computational Biology 2014; 10 (3). https://doi.org/10.1371/journal.pcbi.1003459 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2624 Several countries with generalized, high-prevalence HIV epidemics, mostly in sub-Saharan Africa, have experienced rapid declines in transmission. These HIV epidemics, often with rapid onsets, have generally been attributed to a combination of factors related to high-risk sexual behavior. The subsequent declines in these countries began prior to widespread therapy or implementation of any other major biomedical prevention. This change has been construed as evidence of behavior change, often on the basis of mathematical models, but direct evidence for behavior changes that would explain these declines is limited.

In this paper, the authors look at the structure of current models and argue that the common “fixed risk per sexual contact" assumption favors the conclusion of substantial behavior changes. They argue that this assumption ignores reported non-linearities between exposure and risk. Taking this into account, they propose that some of the decline in HIV transmission may be part of the natural dynamics of the epidemic, and that several factors that have traditionally been ignored by modelers for lack of precise quantitative estimates may well hold the key to understanding epidemiologic trends.

Page 24: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

24

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Reduced Burden of Childhood Diarrheal Diseases through Increased Access to Water and Sanitation in India: A Modeling Analysis Article. Nandi A, Megiddo I, Ashok A et al. Reduced Burden of Childhood Diarrheal Diseases through Increased Access to Water and Sanitation in India: A Modeling Analysis. Social Science and Medicine 2017; 180: 181-192. http://www.sciencedirect.com/science/article/pii/S0277953616304853 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2985 Each year, more than 300,000 children in India under the age of five years die from diarrheal diseases. Clean piped water and improved sanitation are known to be effective in reducing the mortality and morbidity burden of diarrhea but are not yet available to close to half of the Indian population.

This analysis estimates the health benefits (reduced cases of diarrheal incidence and deaths averted) and economic benefits (measured by out-of-pocket treatment expenditure averted and value of insurance gained) of scaling up the coverage of piped water and improved sanitation among Indian households to a near-universal 95% level. The authors use IndiaSim, a previously validated, agent-based microsimulation platform to model disease progression and individual demographic and healthcare-seeking behavior in India, and use an iterative, stochastic procedure to simulate health and economic outcomes over time.

They found that scaling up access to piped water and improved sanitation could avert 43,352 diarrheal episodes and 68 diarrheal deaths per 100,000 under 5 children per year, compared with the baseline. They estimated a savings of (in 2013 US$) $357,788 in out-of-pocket diarrhea treatment expenditure, and $1646 in incremental value of insurance per 100,000 under 5 children per year over baseline. Based on the distribution of benefits, they concluded scaling up access to piped water and improved sanitation could lead to large and equitable reductions in the burden of childhood diarrheal diseases in India. Modeling Human Papillomavirus and Cervical Cancer in the United States for Analyses of Screening and Vaccination Article. Goldhaber-Fiebert JD, Stout NK, Ortendahl J et al. Modeling Human Papillomavirus and Cervical Cancer in the United States for Analyses of Screening and Vaccination. Population Health Metrics 2007; 5: 11. https://doi.org/10.1186/1478-7954-5-11 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3022 This paper discusses a model of human papillomavirus (HPV) and cervical cancer that incorporates uncertainty about the natural history of disease that was used to provide quantitative insight into U.S. policy choices for cervical cancer prevention. The authors developed a stochastic microsimulation of cervical cancer that distinguishes different HPV types by their incidence, clearance, persistence, and progression. For each set of sampled input parameters, likelihood-based goodness-of-fit (GOF) scores were computed based on comparisons between model-predicted outcomes and calibration targets that included age-specific prevalence of HPV by type and cervical intraepithelial neoplasia (CIN), HPV type distribution within CIN and cancer, and age-specific cancer incidence.

Approximately 200 good-fitting parameter sets were identified from 1,000,000 simulated sets and the authors used 50 good-fitting parameter sets to assess the external consistency and face validity of the model through comparison of screening outcomes to independent data not used during calibration. Modeled screening outcomes were found to be externally consistent with results from multiple independent data sources. Based on these 50 good-fitting parameter sets, the expected reductions in lifetime risk of cancer with annual or biennial screening were 76% (range 69-82%) and 69% (60-77%) and from vaccination alone was 75% (range 60-88%). The uncertainty was reduced when vaccination was combined with every-5-year screening to 89% (range 83-95%). Modeling Cervical Cancer Prevention in Developed Countries Review. Kim JJ, Brisson M, Edmunds WJ, Goldie SJ. Modeling Cervical Cancer Prevention in Developed Countries. Vaccine 2008; 26 (Suppl 1): K76-K86. https://doi.org/10.1016/j.vaccine.2008.06.009 Not open access.

Page 25: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

25

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3024 Decision-analytic models are increasingly developed to simulate disease burden and interventions in different settings in order to evaluate the benefits and cost-effectiveness of primary and secondary interventions. This article is a review of mathematical models that have been used to evaluate HPV vaccination in the context of developed countries with existing screening programs.

Despite variations in model assumptions and uncertainty in existing data, pre-adolescent vaccination of females in the setting of current screening practices has been consistently shown to be attractive, provided there is complete lifelong vaccine protection and high vaccination coverage. There was far more uncertainty found in regard to catch-up vaccination programs, benefits of including boys, and outcomes other than cervical cancer with conflicting conclusions reported. This review serves to highlight points of consensus, areas of divergence and to provide insight into critical decisions related to cervical cancer prevention.

Health and Economic Implications of HPV Vaccination in the United States Article. Kim JJ, Goldie SJ. Health and Economic Implications of HPV Vaccination in the United States. NEJM 2008; 359 (8): 821-832. http://www.nejm.org/doi/full/10.1056/NEJMsa0707052#t=article CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3047 This article reports on a study using models of HPV-16 and HPV-18 transmission and cervical carcinogenesis to compare the health and economic outcomes of vaccinating preadolescent girls in the US (at 12 years of age), and vaccinating older girls and women in catch-up programs (to 18, 21, or 26 years of age). The study also examined the health benefits of averting other HPV-16-related and HPV-18-related cancers, the prevention of HPV-6-related and HPV-11-related genital warts and juvenile-onset recurrent respiratory papillomatosis by means of the quadrivalent vaccine, the duration of immunity, and future screening practices.

On the assumption that the vaccine provided lifelong immunity, the study authors found that the cost-effectiveness ratio of vaccination of 12-year-old girls was $43,600 per quality-adjusted life-year (QALY) gained, as compared with the current screening practice. Under baseline assumptions, the cost-effectiveness ratio for extending a temporary catch-up program for girls to 18 years of age was $97,300 per QALY; the cost of extending vaccination of girls and women to the age of 21 years was $120,400 per QALY, and the cost for extension to the age of 26 years was $152,700 per QALY. They report that the results were sensitive to the duration of vaccine-induced immunity; if immunity waned after 10 years, the cost of vaccination of preadolescent girls exceeded $140,000 per QALY, and catch-up strategies were less cost-effective than screening alone.

The cost-effectiveness ratios for vaccination strategies were more favorable if the benefits of averting other health conditions were included or if screening was delayed and performed at less frequent intervals and with more sensitive tests. They conclude that the cost-effectiveness of HPV vaccination will depend on the duration of vaccine immunity and will be optimized by achieving high coverage in preadolescent girls, targeting initial catch-up efforts to women up to 18 or 21 years of age, and revising screening policies. Cost Effectiveness Analysis of Including Boys in a HPV Vaccination Programme in the U.S. Article. Kim JJ, Goldie SJ. Cost Effectiveness Analysis of Including Boys in a Human Papillomavirus Vaccination Programme in the United States. BMJ 2009; 339: b3884. https://doi.org/10.1136/bmj.b3884 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3046 This article reports on a societal-perspective cost effectiveness analysis of including preadolescent boys in a routine human papillomavirus (HPV) vaccination programme for preadolescent girls. The analysis included girls and boys aged 12 years; interventions included HPV vaccination of girls alone and of girls and boys in the context of screening for cervical cancer.

The authors found that with 75% vaccination coverage and an assumption of complete, lifelong vaccine efficacy, routine HPV vaccination of 12 year old girls was consistently less than $50,000 per QALY gained compared with screening alone.

Page 26: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

26

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Including preadolescent boys in a routine vaccination programme for preadolescent girls resulted in higher costs and benefits and generally had cost effectiveness ratios that exceeded $100,000 per QALY across a range of HPV related outcomes, scenarios for cervical cancer screening, and assumptions of vaccine efficacy and duration. Vaccinating both girls and boys fell below a willingness to pay threshold of $100,000 per QALY only under scenarios of high, lifelong vaccine efficacy against all HPV related diseases (including other non-cervical cancers and genital warts), or scenarios of lower efficacy with lower coverage or lower vaccine costs.

The authors conclude that including boys in an HPV vaccination programme generally exceeds conventional thresholds of good value for money, even under favourable conditions of vaccine protection and health benefits. Including Boys in an HPV Vaccination Programme: A CEA in a Low-Resource Setting Article. Kim JJ, Andres-Beck B, Goldie SJ. The Value of Including Boys in an HPV Vaccination Programme: A Cost-Effectiveness Analysis in a Low-Resource Setting. British Journal of Cancer 2007; 97 (9): 1322-1328. https://www.nature.com/bjc/journal/v97/n9/full/6604023a.html CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3039 This paper looks at the cost-effectiveness of including boys vs girls alone in a pre-adolescent vaccination programme against human papillomavirus (HPV) types 16 and 18 in Brazil. Using demographic, epidemiological, and cancer data from Brazil, the authors developed a dynamic transmission model of HPV infection between males and females. Model-projected reductions in HPV incidence under different vaccination scenarios were applied to a stochastic model of cervical carcinogenesis to project lifetime costs and benefits.

They found that at 90% coverage, vaccinating girls alone reduced cancer risk by 63%; including boys at this coverage level provided only 4% further cancer reduction; at a cost per-vaccinated individual of USD 50, vaccinating girls alone was alone was <USD 200 per year of life saved (YLS), while including boys ranged from USD 810-18,650 per YLS depending on coverage. For all coverage levels, the authors concluded that increasing coverage in girls was more effective and less costly than including boys in the vaccination programme. Health and Economic Impact of HPV 16/18 Vaccination and Cervical Cancer Screening in Eastern Africa Article. Campos NG, Kim JJ, Castle PE, Ortendahl JD, O'Shea M, Diaz M, Goldie SJ. Health and Economic Impact of HPV 16/18 Vaccination and Cervical Cancer Screening in Eastern Africa. International Journal of Cancer 2012; 130 (11): 2672-2684. http://onlinelibrary.wiley.com/doi/10.1002/ijc.26269/full CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3037 In this article the authors use epidemiologic data from Kenya, Mozambique, Tanzania, Uganda, and Zimbabwe to develop models of HPV-related infection and disease. For each country, they assessed HPV vaccination of girls before age 12 followed by screening with HPV DNA testing once, twice, or three times per lifetime (at ages 35, 40, 45). For women over age 30, they assessed only screening (with HPV DNA testing up to three times per lifetime or VIA at age 35). Assuming no waning immunity, mean reduction in lifetime cancer risk associated with vaccination ranged from 36 to 45%, and vaccination followed by screening once per lifetime at age 35 with HPV DNA testing ranged from 43 to 51%.

For both younger and older women, the most effective screening strategy was HPV DNA testing three times per lifetime. Provided the cost per vaccinated girl was less than I$10 (I$2 per dose), vaccination had an incremental cost-effectiveness ratio [I$ (international dollars)/year of life saved (YLS)] less than the country-specific per capita GDP, a commonly cited heuristic for "very cost-effective" interventions. If the cost per vaccinated girl was between I$10 (I$2 per dose) and I$25 (I$5 per dose), vaccination followed by HPV DNA testing would save the most lives and would be considered good value for public health dollars.

The authors suggest that these results should be used to catalyze design and evaluation of HPV vaccine delivery and screening programs, and contribute to a dialogue on financing HPV vaccination in poor countries.

Page 27: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

27

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

HPV Vaccine Introduction in LMIC's: Guidance on the Use of Cost-Effectiveness Models Guidelines. Jit M, Demarteau N, Elbasha E et al. Human Papillomavirus Vaccine Introduction in Low-Income and Middle-Income Countries: Guidance on the Use of Cost-Effectiveness Models. BMC Medicine 2011; 9 (1): 54. https://doi.org/10.1186/1741-7015-9-54 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3033 This article is a literature review of HPV vaccination models suitable for low-income and middle-income country use to provide information about the feasibility of using such models in a developing country setting. The authors evaluated models in terms of their capacity, requirements, limitations and comparability.

Their literature review identified six HPV vaccination models suitable for low-income and middle-income country use and representative of the literature in terms of provenance and model structure. Each model was adapted by its developers using standardised data sets representative of two hypothetical developing countries (a low-income country with no screening and a middle-income country with limited screening). The authors compared model predictions before and after vaccination of adolescent girls in terms of HPV prevalence and cervical cancer incidence, as was the incremental cost-effectiveness ratio of vaccination under different scenarios.

The authors found that none of the models perfectly reproduced the standardised data set provided to the model developers. However, they agreed that large decreases in type 16/18 HPV prevalence and cervical cancer incidence are likely to occur following vaccination. The most influential factors affecting cost effectiveness were the discount rate, duration of vaccine protection, vaccine price and HPV prevalence. Demographic change, access to treatment and data resolution were found to be key issues to consider for models in developing countries. Modeling Preventative Strategies Against Human Papillomavirus-related Disease in Developed Countries Review. Canfell K, Chesson H, Kulasingam SL, Berkhof J, Diaz M, Kim JJ. Modeling Preventative Strategies against Human Papillomavirus-Related Disease in Developed Countries. Vaccine 2012; 30 (Suppl 5): F157-F167. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3783354 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3032 This review article is part of a special supplement on “Comprehensive Control of HPV Infections and Related Diseases.” At the time of its writing, prophylactic vaccination against human papillomavirus (HPV) in pre-adolescent females had been introduced in most developed countries, supported by modeled evaluations that had almost universally found vaccination of pre-adolescent females to be cost-effective. Vaccination of pre-adolescent males had been shown to be cost-effective at a cost per vaccinated individual of ~US$400-500 if vaccination coverage in females could not be increased above ~50%; but increasing coverage in females appeared to be a better return on investment.

The authors report that comparative evaluation of the quadrivalent (HPV16,18,6,11) and bivalent (HPV16,18) vaccines centered around the potential trade-off between protection against anogenital warts and vaccine-specific levels of cross-protection against infections not targeted by the vaccines. They propose that future evaluations will also need to consider the cost-effectiveness of a next generation nonavalent vaccine designed to protect against ~90% of cervical cancers and that the timing of the effect of vaccination on cervical screening programs will be country-specific, and will depend on vaccination catch-up age range and coverage and the age at which screening starts.

They conclude that comprehensive evaluation of new approaches to screening would need to consider the population-level effects of vaccination over time, and that future evaluations of screening would also need to focus on the effects of disparities in screening and vaccination uptake, the potential effects of vaccination on screening participation, and the effects of imperfect compliance with screening recommendations.

Page 28: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

28

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Unifying Screening Processes within the PROSPR Consortium: A Conceptual Model for Breast, Cervical, and Colorectal Cancer Screening Article. Beaber EF, Kim JJ, Schapira MM et al. Unifying Screening Processes within the PROSPR Consortium: A Conceptual Model for Breast, Cervical, and Colorectal Cancer Screening. Journal of the National Cancer Institute 2015; 107 (6): djv120. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4838064 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3031 General frameworks of the cancer screening process are available, but none directly compare the process in detail across different organ sites. This limits the ability of medical and public health professionals to develop and evaluate coordinated screening programs that apply resources and population management strategies available for one cancer site to other sites.

This paper presents a conceptual model that incorporates a single screening episode for breast, cervical, and colorectal cancers into a unified framework based on clinical guidelines and protocols. The model covers four types of care in the screening process: risk assessment, detection, diagnosis, and treatment. Interfaces between different provider teams (eg, primary care and specialty care), including communication and transfer of responsibility, may occur when transitioning between types of care.

The model highlights across each organ site similarities and differences in steps, interfaces, and transitions in the screening process and documents the conclusion of a screening episode. This model was developed within the National Cancer Institute–funded consortium Population-based Research Optimizing Screening through Personalized Regimens (PROSPR). Cancer Models and Real-World Data: Better Together Article. Kim J, Tosteson AN, Zauber AG et al. Cancer Models and Real-World Data: Better Together. Journal of the National Cancer Institute 2015; 108 (2). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4907359 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3030 Decision-analytic models synthesize available data on disease burden and intervention effectiveness to project estimates of the long-term consequences of care. While models have been influential in informing US cancer screening guidelines under ideal conditions, incorporating detailed data on real-world screening practice has been limited given the complexity of screening processes and behaviors throughout diverse health delivery systems in the United States.

The authors describe the synergies that exist between decision-analytic models and health care utilization data that are increasingly accessible through research networks that assemble data from the growing number of electronic medical record systems. They present opportunities to enrich cancer screening models by grounding analyses in real-world data with the goals of projecting the harms and benefits of current screening practices, evaluating the value of existing and new technologies, and identifying the weakest links in the cancer screening process where efforts for improvement may be most productively focused.

The example of the National Cancer Institute–funded consortium Population- based Research Optimizing Screening through Personalized Regimens (PROSPR) is provided as an example. This is a collaboration to harmonize and analyze screening process and outcomes data on breast, colorectal, and cervical cancers across seven research centers. Model-Based Analyses to Compare Health and Economic Outcomes of Cancer Control: Inclusion of Disparities Article. Goldie SJ, Daniels N. Model-Based Analyses to Compare Health and Economic Outcomes of Cancer Control: Inclusion of Disparities. Journal of the National Cancer Institute 2011; 103 (18): 1373-1386. https://doi.org/10.1093/jnci/djr303 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3028 In order to identify strategies that improve both population health and ensure its equitable distribution, the authors developed a typology of cancer disparities that considers types of inequalities among black, white, and Hispanic

Page 29: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

29

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

populations across different cancers. This paper reports on the typology using an existing disease simulation model of cervical cancer that was calibrated to clinical, epidemiological, and cost data in the United States and presents characteristics important for policy discussions. The typology proposed may be useful when trade-offs between fairness and cost-effectiveness are unavoidable.

Using average reductions in cancer incidence overall and for sub-categories of black, white, and Hispanic women under different prevention strategies the authors estimated average costs and life expectancy per woman, and cost-effectiveness ratios and found that strategies may provide greater aggregate health benefit may also widen disparities. For example combining human papillomavirus vaccination with current cervical cancer screening patterns resulted in an average cancer incidence reduction of 69% but the reduction in whites was higher (71.6%) than either black (68.3%) or Hispanic women (63.9%) Strategies that employ targeted risk-based screening and new screening algorithms, with or without vaccination were able to reduce those disparities while providing excellent value, with the most effective strategy having a cost-effectiveness ratio of $28,200 per year of life saved when compared with the same strategy without vaccination. Modeling to Improve Policy Decisions in the Americas: Noncommunicable Diseases Report. Legetic B, Cechini M, eds. Applying Modeling to Improve Health and Economic Policy Decisions in the Americas: The Case of Noncommunicable Diseases. Pan American Health Organization, Organisation for Economic Co-Operation and Development 2015. http://iris.paho.org/xmlui/handle/123456789/7700 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2607 In the Region of the Americas, noncommunicable diseases (NCDs) are a clear threat not only to human health, but also to a country’s economic development and growth. The evidence on both of these counts is compelling. In 2012, cardiovascular disease, diabetes, cancers, chronic respiratory conditions including asthma, and other NCDs were the cause of 4.5 million deaths in the Americas. Of that total number, 1.5 million of them were premature, occurring among people aged 30-69 years.

The financial impact of NCDs in the Americas is just as dismaying, with chronic diseases posing a growing threat to many nations’ economic stability. According to a 2007 Lancet article, without intensified NCD prevention efforts, countries around the world could expect their gross domestic product (GDP) to decline by billions of dollars. Over the 2006-2015 period, from just three chronic illnesses—heart disease, stroke, and diabetes—the countries of Argentina, Brazil, Colombia, and Mexico together could face a cumulative combined GDP loss of US$ 13.5 billion.

The text presents different economic models and illustrate their application to NCDs in the Region of the Americas. It aims to stimulate the use of economic modeling as a tool to support the decision-making process for NCD interventions, and to encourage investment in cost-effective strategies for healthy living and NCD prevention in the Region, with the ultimate goal being to help policymakers find the best strategies for cost-effective and evidence-based NCD interventions. Development of an Empirically Calibrated Model of Gastric Cancer in Two High-Risk Countries Article. Yeh JM, Kuntz KM, Ezzati M et al. Development of an Empirically Calibrated Model of Gastric Cancer in Two High-Risk Countries. Cancer Epidemiology Biomarkers & Prevention 2008; 17 (5): 1179-1187. http://cebp.aacrjournals.org/cgi/pmidlookup?view=long&pmid=18483340 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3023 This paper presents an empirically calibrated mathematical model of gastric cancer and H. pylori in China and Colombia to provide qualitative insight into the cost-effectiveness of gastric cancer prevention strategies. Despite studies that have established the relationship between Helicobacter pylori and gastric cancer and H. pylori treatment reducing cancer incidence among individuals without preexisting precancerous lesions, screening for H. pylori is still being debated.

Page 30: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

30

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

The authors synthesized available data to develop a natural history model of noncardia intestinal gastric adenocarcinomas with health states such as normal gastric mucosa, chronic nonatrophic gastritis, gastric atrophy, intestinal metaplasia, dysplasia, and gastric cancer that were all stratified by H. pylori status. A likelihood-based empirical calibration approach was used to identify good-fitting parameter sets consistent with epidemiologic data. A range of likely outcomes associated with H. pylori screening that incorporated parameter uncertainty were reflected in the results. Contribution of H. Pylori and Smoking to US Incidence of Gastric Adenocarcinoma: A Microsimulation Model Article. Yeh JM, Hur C, Schrag D, Kuntz KM, Ezzati M, Stout N, Ward Z, Goldie SJ. Contribution of H. Pylori and Smoking Trends to US Incidence of Intestinal-Type Noncardia Gastric Adenocarcinoma: A Microsimulation Model. PLoS Medicine 2013; 10 (5): e1001451. https://doi.org/10.1371/journal.pmed.1001451 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/3036 Although gastric cancer has declined dramatically in the US, the disease remains the second leading cause of cancer mortality worldwide. This analysis estimates the contribution of risk factor trends on past and future intestinal-type noncardia gastric adenocarcinoma (NCGA) incidence.

The authors developed a population-based microsimulation model of intestinal-type NCGA and calibrated it to U.S. epidemiologic data on precancerous lesions and cancer. The model explicitly incorporated the impact of Helicobacter pylori and smoking on disease natural history, for which birth cohort-specific trends were derived from the National Health and Nutrition Examination Survey (NHANES) and National Health Interview Survey (NHIS).

Between 1978 and 2008, the model estimated that intestinal-type NCGA incidence declined 60% from 11.0 to 4.4 per 100,000 men, <3% discrepancy from national statistics. H. pylori and smoking trends combined accounted for 47% of the observed decline. With no tobacco control, incidence would have declined only 56%, suggesting that lower smoking initiation and higher cessation rates observed after the 1960s accelerated the relative decline in cancer incidence by 7%. With continued risk factor trends, incidence is projected to decline an additional 47% between 2008 and 2040, the majority of which will be attributable to H. pylori and smoking (81%).

The authors concluded that trends in modifiable risk factors explain a significant proportion of the decline of intestinal-type NCGA incidence in the US, and are projected to continue. Although past tobacco control efforts have hastened the decline, full benefits will take decades to be realized, and further discouragement of smoking and reduction of H. pylori should be priorities for gastric cancer control efforts. Simulation Models of Obesity: A Review of the Literature Article. Levy DT, Mabry PL, Wang YC et al. Simulation Models of Obesity: A Review of the Literature and Implications for Research and Policy. Obesity Reviews 2011; 12 (5): 378-394. https://dx.doi.org/10.1111%2Fj.1467-789X.2010.00804.x Not open access. CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2817 Simulation models combine information from a variety of sources to provide a useful tool for examining how the effects of obesity unfold over time and impact population health. They can aid in the understanding of the complex interaction of the drivers of diet and activity and their relation to health outcomes.

This paper provides an overview of different types of simulation models used to evaluate the potential impact of policies to address the obesity epidemic. The authors discuss the strengths and limitations of different types of models, and review existing obesity models.

The authors categorize existing models according to their focus: health and economic outcomes, trends in obesity as a function of past trends, physiologically-based behavioral models, environmental contributors to obesity, and policy interventions.

Page 31: Resource Pack: Models for Health Decision Science€¦ · 1 This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health

31

This resource pack was developed the Center for Health Decision Science at the Harvard T.H. Chan School of Public Health. All materials produced by the Center for Health Decision Science are free and publicly accessible for educational use.

This resource is licensed Creative Commons Attribution-Non Commercial-NoDerivs3.0Unported

chds.hsph.harvard.edu

Modeling the Risks and Benefits of Depression Treatment for Children and Young Adults Article. Soeteman DI, Miller M, Kim JJ. Modeling the Risks and Benefits of Depression Treatment for Children and Young Adults. Value in Health 2012; 15 (5): 724-729. http://dx.doi.org/10.1016/j.jval.2012.03.1390 CHDS repository link: http://repository.chds.hsph.harvard.edu/repository/2841 This article, published in Value in Health, presents a discrete event simulation model to quantify the trade-offs with respect to clinical benefits and the risk of fatal and non-fatal suicidal behavior of alternative treatment strategies for a U.S. pediatric population with major depressive disorder. The authors evaluate treatment strategies including: selective serotonin reuptake inhibitors (SSRIs), cognitive behavioral therapy (CBT), and a combination of both.

The results show that the use of SSRIs is associated with the highest number of suicide-related events, while CBT is associated with the lowest number. Moreover, the strategy with the highest number of symptom-free weeks depends on assumptions made regarding treatment efficacy beyond the 36 weeks for which clinical data is available.

Based on these findings the authors conclude that CBT is favorable with respect to suicide deaths and attempts over combination treatment or SSRIs alone and that any clinical benefits of SSRIs, either alone or in combination with CBT, must be weighed against the expected increase in suicides.