Introduction to decision analysis modeling Alice Zwerling, Postdoctoral fellow, JHSPH McGill TB Research Methods Course July 7, 2015 Overview of Decision Analysis models Understand why we use decision analysis models & what these models can and cannot do Examples of decision analysis and simple decision trees Understand key decision analysis concepts and terminology 2 Decision Analysis Models Some decisions are easier than others 3 Decision Analysis 4 Provides a systematic approach to decision-making under conditions of uncertainty Is employed across many diverse fields: healthcare, business, economics, law, engineering.. Helps decision makers think clearly about many elements of complex decisions Decision Analysis Modeling Requires defining events in terms of their logical and temporal significance Disaggregating a complex problem into smaller steps or events simpler to understand 5 What is a model? Representation of real world phenomena, object or behaviours Theoretical construct Simplification of real world Model needs to be simple enough to be understood but complex enough to capture essential elements of problem or question to be addressed Many simplifying assumptions are needed 6 Decision Trees 7 Branching structure used to represent different kinds of events including decisions and uncertainties, and the associated outcomes or alternatives Decision Trees 8 Decision Node Chance Node Terminal Node (Outcomes or pay-offs) Time Branches (alternative pathways) A B Time (sequence of events) moves from left to right Decision Trees 9 Pathway probabilities (probability of going down each branch) p =0.1 p =0.9 User defined probabilities are entered at each chance node, need to add to 1 Decision Trees 10 Outcomes: Costs & Effectiveness (DALYs/QALYs or cases 0/1) Decide what sequences/events are important to include/ model 11 12 Vassall et al PlosMed, 2011 Steps in Decision Analysis Define the problem Define alternatives to be assessed Identify clinical outcomes of interest Build the tree structure Assign probabilities to all chance events Assign costs and outcomes Estimate expected average values of each strategy (Folding back) Assess uncertainty and variability (sensitivity analyses) 13 A simple problem Evaluate a new treatment 14 Patients with TB diagnosis Standard Tx New Tx Cure Failure Cure Failure Create structure A simple problem Evaluate a new treatment 15 Patients with TB diagnosis Standard Tx New Tx Cure p=0.6 Failure p=0.4 Cure p=0.8 Failure p= 0.2 Assign probabilities A simple problem Evaluate a new treatment 16 Patients with TB diagnosis Standard Tx New Tx Cure p=0.6 Failure p=0.4 Cure p=0.8 Failure p= 0.2 $100 Saves 10 life years $100 Saves 1 life year $500 Saves 10 life years $500 Saves 1 life year Assign values and outcomes Calculations 17 Patients with TB diagnosis Standard Tx New Tx Cure p=0.6 Failure p=0.4 Cure p=0.8 Failure p= 0.2 $100 Saves 10 life years $200 Saves 1 life year $500 Saves 10 life years $600 Saves 1 life year OutcomeStandard TreatmentNew Treatment Expected cost(0.6 x $100) + (0.4 x $200) =$60 + $80 = $140 (0.8 x $500) + (0.2 x $600) =$400 + $120 = $520 Expected life years saved(0.6 x 10) + (0.4 x 1) = 6.4(0.8 x 10) + (0.2 x 1 ) = 8.1 Calculations 18 Incremental cost: $520 - $140 = $380 Incremental life years saved: 8.1 6.4 = 1.7 Cost per life year saved = $380/1.7 = $ Expected OutcomesStandard TreatmentNew Treatment Expected cost(0.6 x $100) + (0.4 x $200) =$60 + $80 = $140 (0.8 x $500) + (0.2 x $600) =$400 + $120 = $520 Expected life years saved(0.6 x 10) + (0.4 x 1) = 6.4(0.8 x 10) + (0.2 x 1 ) = 8.1 Decision analysis Final model outcomes are calculated based on the probability of entering into a particular node and the price tag or effectiveness measure associated with that node Individuals move through the decision trees for a specified amount of time Costs and rewards accrue over the simulation At end of simulation get a tally of specified outcomes (eg. TB related costs per person, number of TB cases, number of TB deaths, etc for each intervention considered (outcomes) 20 Comparing scenarios!!! 21 22 Brief note on sensitivity analyses Systematically asking how would decision results change if parameters/probabilities were different Determines how robust model result may be Vary probabilities and/or outcomes across possible ranges Can produce tornado diagrams 23 Decision Analysis Models Not designed to capture transmission Static, not dynamic models Typically deterministic (can be stochastic) 24 Decision Analysis Models STATIC models: (as opposed to dynamic transmission models) The annual risk of infection (ARI) is not sensitive to the changing number of infectious cases in the population Does not account for ongoing transmission in a population Typically population based, deterministic, static models 25 Population vs. Individual models Population based Follows populations Divides population into mutually exclusive groups Homogeneity within groups, but can be subdivided Individual level factors are averaged together, model shows changes in average characteristics of whole population Individual based Follows individuals Characteristics of each individual are tracked through time Can explore complex relationships, social/spatial interactions, heterogeneity May include approaches such as agent-based models, or queue model 26 Decision analysis are most commonly population based, but can track individuals Population based 25% of population has TB HIV co-infection 45% of cohort is female Cohort comprised of persons on average 45 years old Individual based Each individual tracked Pt 1 of 10,000 simulated pts Is a female 45 years old, HIV+ 27 Deterministic models All parameters are fixed, no random element Model predictions are fixed, same answer with every run Describes what happens on average to whole population More frequent in literature Stochastic models Incorporate chance into the model Results vary with every run Important in small populations or where chance might play a role Require many simulations (more computing power) 28 Decision analysis can use either approach, but are most commonly deterministic models Easy to learn & user friendly Intuitive, visual representation of problem Can take advantage of average data (e.g. meta-analyses) Low cost and faster (relative to empirical studies) Can consider hypothetical situations or specific populations/scenarios for whom a trial is not ethical or feasible Can capture more complex pathways Can be used to generalize/extrapolate trial findings (time/popn) Integrated costing capability and can be easily modified for cost-effectiveness (common form of modeling methods in economic evaluation) Extensive and sophisticated sensitivity analysis Decision Analysis: Advantages Decision analysis: Disadvantages Decision analysis describes what happens to a cohort of selected individuals By design does not incorporate popn outside of cohort Population level impact not accounted for Transmission effects not typically incorporated in decision analysis models The annual risk of infection is not sensitive to changing number of infectious cases in the population Can be partially addressed using Markov models and introducing assumptions around transmission parameters 30 Decision Analysis Models CAN. Compare relative impact and cost of two different well defined interventions Understand problems in a logical and transparent fashion Identify weakness in our conceptualization of problem Make assumptions explicit 31 Summary: Models are not so good for Predicting the future Giving precise estimates Working magic with bad/limited data Can only work to level of complexity that we understand/ have data to support Models rely on assumptions, may be limited by poor data (can test these) Struggle to capture heterogeneity/phenomenon that we are not aware of or do not understand Summary Different types of models are available Choice depends on : The research question The data that we have to work with The assumptions that we are willing to make How quickly we need the results The expertise of the modelling team 34 Reproduced from xkcd.com