medical decision making - mcgill university · 292 •medical decision making/july–aug 2002...

http://mdm.sagepub.comMedical Decision Making

DOI: 10.1177/027298902400448867 2002; 22; 290 Med Decis Making

Andrew H. Briggs, Ron Goeree, Gord Blackhouse and Bernie J. O'Brien Gastroesophageal Reflux Disease

Probabilistic Analysis of Cost-Effectiveness Models: Choosing between Treatment Strategies for

http://mdm.sagepub.com/cgi/content/abstract/22/4/290 The online version of this article can be found at:

Published by:

http://www.sagepublications.com

On behalf of:

Society for Medical Decision Making

can be found at:Medical Decision Making Additional services and information for

http://mdm.sagepub.com/cgi/alerts Email Alerts:

http://mdm.sagepub.com/subscriptions Subscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

© 2002 Society for Medical Decision Making. All rights reserved. Not for commercial use or unauthorized distribution. at DALHOUSIE UNIV on March 7, 2007 http://mdm.sagepub.comDownloaded from

http://www.gwu.edu/~smdm

http://mdm.sagepub.com/cgi/alerts

http://mdm.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://mdm.sagepub.com

BRIGGS, GOEREE, BLACKHOUSE, O’BRIENPROBABILISTIC COST-EFFECTIVENESS: GERD MANAGEMENTCLINICAL APPLICATIONS CLINICAL APPLICATIONS

Probabilistic Analysis of Cost-EffectivenessModels: Choosing between Treatment Strategies

for Gastroesophageal Reflux Disease

Andrew H. Briggs, DPhil, Ron Goeree, MA, Gord Blackhouse, MBA, Bernie J. O’Brien, PhD

When choosing between mutually exclusive treatment op-tions, it is common to construct a cost-effectiveness frontieron the cost-effectiveness plane that represents efficientpoints from among the treatment choices. Treatment optionsinternal to the frontier are considered inefficient and are ex-cluded either by strict dominance or by appealing to the prin-ciple of extended dominance. However, when uncertainty isconsidered, options excluded under the baseline analysismay formpart of the cost-effectiveness frontier. By adopting aBayesian approach, where distributions for model parame-ters are specified, uncertainty in the decision concerning

which treatment option should be implemented is addresseddirectly. The approach is illustrated using an example from arecently published cost-effectiveness analysis of differentpossible treatment strategies for gastroesophageal reflux dis-ease. It is argued that probabilistic analyses should be en-couragedbecause theyhavepotential to quantify the strengthof evidence in favor of particular treatment choices. Keywords: economic evaluation; probabilistic sensitivity analy-sis; Bayesian methods; uncertainty; simulation. (Med DecisMaking 2002;22:290–308)

It is now increasingly common for economic evalua-tions to be conducted alongside clinical trials. Re-

cent research attention has been focused on how tohandle uncertainty in these so-called stochastic cost-effectiveness analyses where patient-level data areavailable on the costs and effects of treatment options.1–3

However, the majority of economic evaluations stillemploy a decision analytic modeling framework tosynthesize data from a number of sources.4 Such cost-effectiveness models are often described as determinis-tic analyses. Although the limitations of simpleunivariate sensitivity analysis are well known, this re-mains the most popular technique for handling uncer-tainty in cost-effectiveness models.

Probabilistic sensitivity analysis is an alternative ap-proach that involves specifying distributions for inputparameters in the model and employing Monte Carlosimulation to sample from these distributions, allow-ing the joint effect of parameter uncertainty to be as-sessed.5,6 A number of commentators have suggestedthat probabilistic sensitivity analysis methods be usedto handle uncertainty in cost-effectiveness models,1,7

including the US panel on cost-effectiveness analysis.8

Despite these recommendations, few probabilistic

analyses of cost-effectiveness models have been under-taken. The relative paucity of probabilistic analysesmay be due to the increased complexity of the ap-

290 • MEDICAL DECISION MAKING/JULY–AUG 2002

Received 7 May 2001 from the Health Economics Research Centre,University of Oxford, Institute of Health Sciences, Headington, Oxford,United Kingdom (AHB); and the Centre for Evaluation of Medicines, StJoseph’s Hospital and Department of Clinical Epidemiology andBiostatistics, McMaster University, Hamilton, Ontario, Canada (AHB,RG,GB, BJO). Financial support for this studywas provided in part by agrant from a Joint UK Medical Research Council/South East RegionTraining Fellowship (AHB) and the Centre for Evaluation of Medicines(CEM) at McMaster University. The funding agreement ensured the au-thors’ independence in designing the study, interpreting the data, andwriting and publishing the report. This article was prepared while AHBwas visiting the CEM at McMaster University, and an earlier version ofthis article appeared as a working paper in the Centre for Health Eco-nomics and Policy Analysis series and was presented at the 22nd An-nual Meeting of the Society for Medical Decision Making and at theHarvard Center for Risk Analysis. We are grateful to participants and to2 anonymous referees for comments. Remaining errors are our own re-sponsibility. Revision accepted for publication 11 April 2002.

Address correspondence and reprint requests to Dr. Briggs, HealthEconomics Research Centre, University of Oxford, Institute of HealthSciences, Headington, Oxford OX3 7LF, United Kingdom; e-mail:[email protected].



proach and a lack of clarity concerning which distribu-tions for input parameters are appropriate.

The aim of this article is to demonstrate the use ofprobabilistic sensitivity analysis to handle uncertaintyin a cost-effectiveness decision problem relating to al-ternative treatment options for gastroesophageal refluxdisease (GERD). We argue that adopting a Bayesian ap-proach to uncertainty offers both technical and concep-tual advantages over traditional sensitivity analyses. Inparticular, a Bayesian approach allows a more intuitiveinterpretation of probability—we show how the studyquestion of whether a treatment is cost-effective can beanswered directly in the form of a probability that theintervention is cost-effective. Furthermore, we demon-strate this approach in the case of multiple treatmentoptions for GERD, rather than the standard 2-treatmentapproach that is the norm in the majority of analyses.

The article is structured as follows. In the next sec-tion we give a brief introduction to the decision prob-lem, the structure of the model, and the results of thepreviously published cost-effectiveness model—where uncertainty was handled through standard de-terministic sensitivity analysis methods. The sectionthat follows demonstrates how the model can be madeprobabilistic by specifying distributions for the inputparameters following standard principles of Bayesianmethods. The assumptions and calculations involvedin specifying these distributions are discussed in de-tail. Results of the probabilistic analysis are then pre-sented on the cost-effectiveness plane and summarizedthrough the use of cost-effectiveness acceptabilitycurves. These results, and the general probabilistic ap-proach to cost-effectiveness modeling, are discussed inthe final section of the article.

TREATMENT STRATEGIES FOR GERD

In this section, a model for assessing the cost-effectiveness of 6 management strategies for the treat-ment of GERD is briefly outlined. Full details of themodel were presented in detail in a previous publica-tion.9 First, the structure and assumptions concerningthe decision model are discussed. Second, the resultsof the deterministic analysis are presented. Finally, thelimitations of the originally reported univariate sensi-tivity analysis are highlighted.

A Model for Assessing the Cost-Effectivenessof GERD Treatment

GERD is a common condition that results from re-gurgitation of acid from the stomach into the esopha-gus. The most frequent symptom of GERD is heartburn,

and the majority of patients with GERD requirepharmacotherapy to reduce acid secretion. Currently,the choice of first-line antisecretory therapy is betweenthe H2-receptor antagonists (H2RAs), such as ranitidineand cimetidine, and proton pump inhibitors (PPIs),such as omeprazole. Although they have higher acqui-sition costs, PPIs have been found to be more effica-cious than H2RAs in terms of both the rate and speed ofhealing.10,11

The objective of the original study was to compare,over a 1-year period, the expected costs and outcomesof alternative drug treatment strategies for the manage-ment of patients with erosive esophagitis confirmed byendoscopy, but without complications such as Barrett’sesophagus or stricture. Outcomes are quantified interms of GERD recurrence and weeks per year withoutGERD as indicated by data from clinical trials on heal-ing and recurrence of esophagitis.

Treatment Strategies and Model Structure

Six strategies involving different combinations offirst-line agents and change of therapy conditional onfailure to heal or recurrence of GERD were modeled.

Strategy A: Intermittent PPI. Acute treatment with a PPIfor 8 weeks and then no further treatment with pre-scription medication until recurrence.

Strategy B: Maintenance PPI. Acute treatment with a PPIfor 8 weeks then continuous maintenance treatmentwith a PPI (same dose).

Strategy C: Maintenance H2RA. Acute treatment with anH2RA for 8 weeks and then continuous maintenancetreatment with an H2RA (same dose).

Strategy D: Step-down maintenance prokinetic agent (PA).Acute treatment with a PA for 12 weeks and then con-tinuous maintenance treatment with a lower dose ofPA.

Strategy E: Step-down maintenance H2RA. Acute treat-ment with a PPI for 8 weeks and then continuous main-tenance treatment with an H2RA.

Strategy F: Step-down maintenance PPI. Acute treatmentwith a PPI for 8 weeks and then continuous mainte-nance treatment with a lower dose PPI.

Treatment options A to F represent clinical strate-gies rather than single-drug treatments for the manage-ment of erosive esophagitis where the physician is as-sumed to increase the dose of a drug or switch toanother drug if the patient fails to respond to the first-line treatment. The logic of these assumptions with re-gard to stepping up dosage or switching can be found inTable 1. The structure of the decision tree that was de-veloped is shown in Figure 1 and is based on the treat-ment strategies and step-up switching algorithms in Ta-

CLINICAL APPLICATIONS 291

PROBABILISTIC COST-EFFECTIVENESS: GERD MANAGEMENT



ble 1. The model is recursive in two 6-month periods;hence, probabilities of recurrence in the period to 12months are conditional upon recurrence or non-recurrence in the period from 0 to 6 months.

Treatment Outcomes

For GERD, the most commonly used formulation ofoutcome for economic evaluation has been eitheresophagitis-free or symptom-free time in a period offollow-up. The advantage of such a measure is that itcombines 2 important aspects of efficacy: (1) the speedwith which esophagitis is healed and (2) the likelihoodof esophagitis recurring. In this analysis, the primaryoutcome measure is GERD-free time during the 12-month period of the model, defined as the time whenthe esophagitis is healed. A meta-analysis of healingand recurrence studies published to November 1997was undertaken to estimate healing and recurrenceprobabilities together with associated GERD-free time.Full details of this analysis are given in the originalstudy.9

Resource Use and Unit Costs

Generic prices were used for drugs when a genericequivalent was available, employing the “best avail-able price” from the Ontario Drug Benefit (ODB)program12 together with a 10% pharmacy markupcharge. A dispensing fee of Can$4.11 was used (i.e.,ODB program fee of Can$6.11 less a Can$2.00 patientcopayment).

Cost estimates for physician fees were taken fromthe physician fee schedule for Ontario,13 and procedurecosts, such as endoscopy, were estimated from a hospi-tal participating in the Ontario Case Costing Project inSouthwestern Ontario.14

To estimate the costs associated with the manage-ment of patients with symptoms of GERD recurrence,information on clinical practice patterns and resourceutilization was obtained by convening an expert physi-cian panel and using a modified Delphi technique.15

Estimated resource utilization was then combinedwith unit cost information to give the average cost asso-


BRIGGS, GOEREE, BLACKHOUSE, O’BRIEN

Strategy A: Intermittent PPI Strategy B: Maintenance PPI

Healing Maintenance 1ST Recurrence Healing Maintenance 1ST Recurrence

PPI No therapy PPI to heal PPI PPI DD PPI to heal unhealed unhealed

DD PPI PPI DD PPI to heal DD PPI PPI DD PPI to heal

Strategy C: Maintenance H2RA Strategy D: Step-Down Maintenance PA


H2RA H2RA DD H2RA PA LD PA PA to heal unhealed unhealed

PPI H2RA PPI to heal PPI LD PA PPI to heal

unhealed unhealedDD PPI PPI DD PPI to heal DD PPI PPI DD PPI to heal

Strategy E: Step-Down Maintenance H2RA Strategy F: Step-Down Maintenance PPI


PPI H2RA PPI to heal PPI LD PPI PPI to heal unhealed unhealed

DD PPI PPI DD PPI to heal DD PPI PPI DD PPI to heal

Note: PPI = proton pump inhibitor (e.g., omeprazole 20 mg OD).DD PPI = double dose proton pump inhibitor (e.g., omeprazole 40 mg OD).LD PPI = low dose proton pump inhibitor (e.g., omeprazole 10 mg OD).H2 RA = H2 receptor antagonists (e.g., ranitidine 150 mg BID).DD H2 RA = double dose H2 receptor antagonists (e.g., ranitidine 300 mg BID).PA = prokinetic agent (e.g., cisapride 10 mg QID).LD PA - low dose prokinetic agent (e.g., cisapride 10 mg BID).Reprinted with permission from Goeree et al. (1999), Adis International.

Table 1. Step-Up and Switching Algorithms Conditional on Healing Failure or Recurrence



ciated with each recurrence under each managementstrategy.

Results of the DeterministicCost-Effectiveness Analysis

The decision tree model outlined above was evalu-ated to estimate the expected costs and the expectedweeks without GERD in the 12-month period of themodel. The analysts of the original study took the con-ventional approach to examining the cost-effectivenessof the alternative strategies.16–18 First, it was determinedwhether any strategies were simply dominated byother strategies having both lower costs and greatertherapeutic effects. Second, it was determined whetherany strategies were dominated through the principlesof extended dominance (i.e., whether linear combina-tions of other strategies can produce greater benefit atlower cost).19 Finally, among nondominated treatmentoptions, incremental cost-effectiveness ratios were cal-culated by comparing each option to the next morecostly and more effective intervention. This process

produces an “efficiency frontier” of increasingly morecostly and more effective strategies. The results of theanalysis are presented on the cost-effectiveness (CE)plane in Figure 2, which also shows the efficiencyfrontier.

The figure clearly shows that step-down mainte-nance PA (strategy D) is dominated by maintenanceH2RA (strategy C), intermittent PPI (strategy A), andstep-down maintenance H2RA (strategy E). The effi-ciency frontier is given by the lines joining strategies C(the origin), A, E, and B. Strategy F is internal to thisfrontier, indicating that it also can be ruled out throughthe principle of extended dominance (i.e., a linear com-bination of strategies E and B would strongly dominateF). The slope of the frontier at any point reflects incre-mental cost-effectiveness—the additional cost atwhich additional effects can be purchased.

Limitations of Conventional Sensitivity Analysis

Parameter estimates employed in the model are notknown with certainty; therefore, it is important to ex-



EndoscopicallyProven ErosiveEsophagitis

Healed No Prescription Therapy

Strategy A: PPI

Strategy B: PPI

Strategy C: H2RA

Strategy D: PA

Step up therapy (Table 1)

Healed Maintenance PPI

Not Healed


Healed Maintenance H2RA

Not Healed

Healed Maintenance Low Dose PA


Not Healed

Recurrence(step up therapy --

see Table 1)

Recurrence(step up therapy --

see Table 1)

No Recurrence No Recurrence

Step up therapy (Table 1)0-6 MONTHS 6-12 MONTHS

Strategy E: PPI

Strategy F: PPI

Healed

Healed

Not Healed

Not HealedStep up therapy (Table 1)


Maintenance H2RA

Maintenance Low Dose PPI

Not Healed

Figure 1. Decision tree for the management of erosive esophagitis. PPI = proton pump inhibitor; H2RA = H2-receptor antagonist; PA =prokinetic agent. Reprinted with permission from Goeree et al. (1999), Adis International.



plore the implications of parameter uncertainty for theresults of the analysis. In particular, given the impor-tance of excluding dominated interventions from theanalysis before calculating the frontier, the extent towhich particular strategies are part of (or can be ex-cluded from) the frontier should be assessed.

In the original analysis, the authors examined the ef-fect of a number of the parameters in their model usingconventional sensitivity analysis techniques. Theyshowed how the uncertainty in the parameter valueschosen for the baseline analysis might affect the fron-tier. The sensitivity analysis that the authors presentedwas much less arbitrary than that of many reportedcost-effectiveness analyses because the outcome rangeschosen were the 95% confidence intervals (CIs) fromthe reported meta-analyses of healing and recurrencerates. The authors reported that

there were marked differences in expected costs, recur-rences and weeks with GERD when using the lowerand upper CIs for both healing and recurrence rates.However, there were no changes in the relative rankingof strategies for either costs or outcomes. The basic con-clusions of the base-case analysis were not altered byusing the lower or upper 95% CIs for healing or recur-rence rates.9(pp689,691)

Despite this convincing argument, we might still beconcerned that the full effects of uncertainty are moreimportant than the authors suggest. It is well known7,8

that conventional univariate sensitivity analysis,whereby individual parameters are varied while main-

taining all remaining parameters at their baselinevalue, is likely to underestimate uncertainty because,in reality, parameters will not vary in isolation.

A BAYESIAN APPROACH TOPROBABILISTIC SENSITIVITY ANALYSIS

In this section, the general appeal of adopting aBayesian approach to probabilistic analysis of cost-effectiveness models is presented. In the past, the use ofprobabilistic sensitivity analysis has been employedwithout reference to the statistical framework of the ap-proach.5,6 We argue that probabilistic sensitivity analy-sis (and indeed decision analysis itself) is naturallyBayesian and adopting this approach offers both tech-nical and conceptual advantages. Particular emphasisis given to the choice of distributions for the differenttypes of parameters commonly encountered in cost-effectiveness models, and it is argued that Bayesianmethods provide the solution to which distributionsshould be used for parameters estimated from differenttypes of data.

Probabilistic Analysis

Probabilistic sensitivity analysis involves specify-ing distributions for model parameters to represent un-certainty in their estimation and employing MonteCarlo simulation to select values at random from thosedistributions.5,6 In this way, probabilistic models allowthe effects of joint uncertainty across all the parameters



D

C

A

E

B

F

600

700

800

900

1000

1100

1200

38.00 39.00 40.00 41.00 42.00 43.00 44.00 45.00 46.00 47.00 48.00

Weeks free of GERD

Str

ateg

y co

st

2RAD: Step-E: Step- 2RAF: Step-

$10/GFW

$264

/GFW

$36/GFW

D

C

A

E

B

F

2

A: Intermittent PPIB: Maintenance PPIC: Maintenance H

down maintenance PAdown maintenance Hdown maintenance PPI

Figure 2. Baseline cost-effectiveness results on the cost-effectiveness plane showing the “efficient frontier.” PPI = proton pump inhibitor;H2RA = H2-receptor antagonist; PA = prokinetic agent; GERD = gastroesophageal reflux disease; GFW = GERD-free week.



of the model to be considered. Note that for standardfrequentist analyses (such as practiced in almost allclinical trials), parameters to be estimated from the dataare considered to have true values that do not vary.Probabilities attached to CIs relate to the long-run cov-erage probabilities of the intervals if the same experi-ment were to be repeated many times.

By contrast, in probabilistic modeling, parametersare considered random variables, which can take arange of values described by the specified distribution.Although these distributions will represent “degrees ofbelief” in the parameters of interest, it does not neces-sarily follow that the analysis will become automati-cally “subjective” (the great fear of many of those whoobject to Bayesian methodology). When data are lack-ing and it becomes necessary to engage experts to pro-vide information on prior distributions, then a numberof experts should be consulted in order that the distri-butions reflect uncertainty between experts rather thanrepresenting the subjective beliefs of a single expert.Eddy et al.20,21 outlined just such an approach to syn-thesizing data based on Bayesian methods that theytermed the “confidence profile” technique.

Choosing Distributions for the Parameters

Parameters in decision models represent summaryvalues related to the average experience across a popu-lation of (potential) patients. Therefore, the relevantuncertainty to capture in the formation of a distributionfor the parameter is 2nd-order uncertainty related tothe sampling distribution of the parameter, not the vari-ability in the values observed in a particular population(1st-order uncertainty; see Stinnett and Paltiel3 for fur-ther discussion). Although an assumption of normalityfor parameters is widely used in statistics, it is worth re-membering that the assumption is based onasymptotics (the central limit theorem) and that thenormal distribution has no bounds on values it cantake. In practice, parameters of the model will have log-ical limitations on the values they can take. In this sec-tion, we discuss 4 different types of parameters com-monly employed in cost-effectiveness models:probabilities, resource items, unit costs, and relativerisks. For each, we discuss the nature of the data in-forming parameter estimates, the logical bounds on theparameter, and the way in which Bayesian methodscan help to select distributions for parameters.

Probability Parameters

Probabilities for cost-effectiveness models are oftenbased on the observed proportions of the event of inter-est (e.g., the number of successfully treated cases). At

an individual level, a treated patient is classed as eithera success or a failure; therefore, the data can be consid-ered as independent Bernoulli trials leading to a bino-mial form of the data likelihood. With such data, it isnatural to use the proportion of successful patients asthe estimate of the corresponding probability in themodel. However, in considering the distribution of thatprobability, note that the binomial distribution is adiscrete distribution related to the sample size of thestudy generating the data, whereas it makes sense tomodel the distribution of probability in the model ascontinuous.

Standard frequentist methods for estimating a CI fora proportion involve calculating the binomial estimateof variance and assuming a normal sampling distribu-tion in order to generate the interval

( )p p p n p p p n− × − + × −196 1 196 1. ( ) , . ( )

(where p is the proportion and n is the sample size).22

Although this method gives a good approximation tothe true CI when p is not close to 0 or 1, the assumptionof normality is not appropriate for probabilistic sensi-tivity analysis. This is because the probability is knownto be bounded on the interval 0-1 whereas the normaldistribution will (eventually) generate values outsidethis interval in a Monte Carlo simulation because it isunbounded.

One solution to this is to make the transformation tothe logistic scale, which is unbounded, assume nor-mality on this scale, then transform back to the original0-1 probability scale, as described by Doubilet et al.6 intheir early contribution on the use of probabilisticmethods for clinical decision making. Although thisprovides a solution to the problem of bounding, it turnsout to be a reasonably sophisticated problem to choosethe parameters for the normal distribution on the logitscale in order to match the desired mean and standarddeviation on the probability scale (see the derivationsin Appendix A of Doubilet et al.6 for details).

Fortunately, Bayesian methods provide a straight-forward method for moving from the discrete binomiallikelihood to the continuous uncertainty concerningthe probability parameter. The beta distribution is acontinuous distribution on the interval 0-1 and is con-jugate to the binomial distribution. This means that if itis possible to represent prior belief using a beta distri-bution, then the integration of that prior belief with thebinomial data has a closed form, with the result that theposterior distribution of the probability will also followa beta distribution. Fortunately, by varying the 2 pa-rameters of the beta distribution, a wide variety of pos-sible shapes to the distribution over the interval can be





obtained: skewed, symmetric, uniform, near normal,and even U-shaped. One parameterization of the betadistribution, beta(r,n), has a similar interpretation to rsuccesses from n trials. A 2nd common parameter-ization is beta(α,β), whereα= r andα+β=n. The meanand variance of beta(r,n) distribution are given by

meanrn

sdr n rn n

= = −+

( )( )

.2 1

Furthermore, if we can specify a prior distribution asbeta(r ′, n ′ – r ′), then following an observation of r suc-cesses and n – r failures in n trials, the application ofBayes’s theorem yields the result that the posterior dis-tribution is beta(r ′+ r,n′ – r ′+n – r). Where no prior in-formation exists as to the probability, it is appropriateto use an “uninformative” or reference prior. Althoughbeta(1,1), which yields a uniform distribution over theinterval 0-1, seems an intuitively obvious choice ofprior, in fact what constitutes uninformative in thiscontext is not as straightforward as it appears,23 andbeta(0.5,0.5), which yields a U-shaped distribution, isalso a popular choice for a reference prior. Because un-informative priors will be dominated by the data, theissue of which uninformative prior to employ is un-likely to be of practical importance when data are avail-able to update that prior.

Resource Item Parameters

All economic analyses are concerned with the use ofresources. The numbers of resource items that a patientuses can be considered a count variable. The Poissondistribution with parameter λ (which gives both themean and variance of the distribution) is often used tomodel count data. If we are interested in the distribu-tion of the mean resource use for a group of patients, wecould use the Poisson estimate of variance to obtain astandard error for the mean resource use, relying on thecentral limit theorem to give a normal sampling distri-bution. However, this may be problematic for smallersamples due to a nonnegligible probability that the nor-mal distribution could take a value less than 0 when itis clear that mean resource use cannot be negative.

Again, the Bayesian approach provides a solution.The gamma distribution is conjugate to the Poisson dis-tribution, is constrained to be positive, and is fully con-tinuous. Therefore, the gamma distribution for themean resource use can be specified without fear of gen-erating inconsistent values in a probabilistic analysis.

Unit Cost Parameters

Unit costs are applied to resource volumes in orderto evaluate all resource use on a common (monetary)

scale. Note that the unit of analysis for such costs is dif-ferent from that for other parameters—unit costs aretypically calculated across a broad group of patients.The unit cost of a surgical procedure or stay in a partic-ular ward will typically be given at the level of the hos-pital (or similar provider unit). By contrast, the unitcost of a drug or device may be set provincially or na-tionally and may not vary at all within the context of acountry-specific cost-effectiveness analysis. Further-more, the unit cost of a resource item is strictly continu-ous, unlike the data on resource use considered above.Because unit costs are constrained to be positive, agamma distribution could be used to represent uncer-tainty in these costs. However, it is less clear that unitcosts will be highly variable than the resource itemsthey are employed to value, which means that a normaldistribution may be safely employed. It is perhaps tell-ing that most economic analyses conducted alongsideclinical trials treat unit costs as fixed rather thanstochastic.

Relative Risk Parameters

It is very common for economic models to includerelative risks as parameters. This mirrors the fact thatrelative risk is often the primary outcome in clinical tri-als. Methods for calculating CIs for relative risk esti-mated in such trials assume that the central limit theo-rem will lead to the natural logarithm of relative risk(which is additive) being normally distributed suchthat CIs can be determined in the standard way. A CI fora relative risk is then obtained by exponentiating theCIs on the log scale. This standard approach to CI esti-mation clearly suggests an equivalent approach tospecifying a log-normally distributed parameter for rel-ative risk to be used in a probabilistic sensitivity analy-sis. Furthermore, because the normal distribution isself-conjugate (a normal prior and a normal data likeli-hood generate a normal posterior distribution), the ap-plication of Bayes theorem on a normally distributedparameter is especially straightforward.

A PROBABILISTIC ANALYSIS OFTREATMENT STRATEGIES FOR GERD

In this section, we describe how a probabilistic sen-sitivity analysis of this decision problem was under-taken in order to more fully account for uncertainty inthe choice of treatment strategy for GERD. Within themodel, there are 3 main categories of parameters:model probabilities relating to the healing and recur-rence rates of GERD symptoms, parameters relating tothe level of resource consumption by patients withGERD symptoms, and unit costs of those resources.





Each of these parameter categories are discussed in de-tail below, and a full list of the parameters of the modelis given in Table 2. Note that the outcome variables inthe model—weeks free of GERD—are completely de-termined by the healing and recurrence rates and,therefore, are endogenous variables in the model.9

Parameter Distributions

Distributions for the Healingand Recurrence Probabilities

All patients begin the model with GERD. Followingfirst-line therapy, there is a probability that their GERDwill have healed. Once GERD has healed, there is thenthe probability that it will recur. The healing and recur-rence probabilities were estimated from the literature.Consider that at an individual level, a patient withGERD has either been healed or has experienced a re-currence. Therefore, the clinical investigation of heal-ing and recurrence can be considered as leading to a bi-nomial form of the data likelihood as described above;hence, a beta distribution was chosen to represent un-certainty in the healing/recurrence parameters.

The original study went to some lengths to present arigorous meta-analysis of healing and recurrence prob-abilities. This method resulted in estimates of constanthazards for healing probabilities and estimates of pro-portions of patients recurring in 2 periods—0 to 6months following healing and 6 to 12 months followinghealing—together with associated estimates of stan-dard error. Due to the random-effects assumption, thisis more conservative than the Bayesian updating ap-proach described above, which would be equivalent tosimply pooling all the studies directly. Therefore, betadistributions were fitted by method of moments23: themean and standard errors from the meta-analysis wereequated to the estimates of mean and standard error ofthe beta distribution given, and these equations werethen solved to give the appropriate beta distributionparameters (see the appendix for this derivation). De-tails of the distributions for the healing hazards and therecurrence probabilities fitted by this method are givenin Table 3.

Distributions for Resource Use Assumptions

In contrast to the rigorous meta-analytic approachemployed to summarize the wealth of information onthe healing and recurrence rates associated with differ-ent drug interventions for GERD, the information on re-source use, particularly the level of investigations re-ceived by patients following a recurrence, wasextremely sparse. Although in the original study a

Delphi panel of experts was convened in order to esti-mate the likely experience of patients, the purpose ofthe panel was to forge consensus, and no informationon the variance of estimates that emerged prior to con-sensus of the experts remains. Therefore, the assump-tions concerning the distributions of estimated re-source use parameters are much more arbitrary.

For the estimated number of visits to general practi-tioners and for endoscopic investigation, a gamma dis-tribution was assumed. This is because the number ofvisits is constrained to be positive and the gamma dis-tribution is only defined for positive values. Again amethod-of-moments approach to fitting was employedsuch that the mean of the gamma distribution wasequal to the point estimates of the visits generated bythe expert panel and assuming that the standard errorwas half that value (i.e., assuming the coefficient ofvariation was 0.5; see the appendix for the derivation).

For the proportions of patients receiving the variousinvestigative procedures, it was assumed that the ex-pert panel had related its estimates to a hypothetical co-hort of 100 patients. Therefore, a beta distribution wasagain employed as if the event rates given by the expertpanel were per 100. Given the considerable experienceof the panel with GERD treatment, it is likely that thisapproach is conservative.

It is assumed that variation in medication use is neg-ligible, such that all patients obtain their prescriptionsand all prescriptions accord with the treatment strate-gies under evaluation. The chosen distributions for theresource use parameters are presented in Table 4.

Unit Costs of Resources

The 2nd component of uncertainty in cost is the po-tential uncertainty in the unit cost estimates employedto value resource items. We do not believe that it is ap-propriate to handle uncertainty in drug pricesprobabilistically, since, at the point of the evaluation,drug prices are determined by the manufacturers.Hence, although drug prices might be considered vari-able (because they are under the control of the manu-facturer and may change over time), they are not uncer-tain. Of course, there may be some uncertaintyconcerning which drugs it is appropriate to prescribe,but that is part of the decision problem and is best han-dled outside of the probabilistic component of the anal-ysis. Therefore, drug prices were not varied in thisanalysis.

A separate issue relates to the use of scheduled infor-mation of the cost of resource items in Ontario, Canada,where the original study was carried out. Althoughthere is certainly an issue concerning whether sched-uled reimbursement values for resources reflect the







Table 2. Parameters in the Model

Name Value Description

Healing hazardshPA 0.045 Hazard for healing on PAhH2 0.068 Hazard for healing on H2RAshDDH2 0.091 Hazard for healing on double-dose

H2RAshPPI 0.215 Hazard for healing on PPIshDDPPI 0.209 Hazard for healing on double-dose

PPIsCorresponding transition probabilities

pPA 0.42 Probability of healing on PA(at 12 weeks)

pH2 0.42 Probability of healing on H2RAs (at8 weeks)

pDDH2 0.52 Probability of healing on double-dose H2RAs (at 8 weeks)

pPPI 0.82 Probability of healing on PPIs (at8 weeks)

pDDPPI 0.81 Probability of healing on double-dose PPIs (at 8 weeks)

Recurrence variablesp06PL 0.67 Probability of recurrence on placebo

(0-6 months)p06PA 0.23 Probability of recurrence on PA

(0-6 months)p06H2 0.38 Probability of recurrence on H2RAs

(0-6 months)p06PPI 0.12 Probability of recurrence on PPIs

(0-6 months)p06LDPPI 0.25 Probability of recurrence on low-

dose PPIs (0-6 months)p06SU 0.12 Probability of recurrence after

surgery (0-6 months)p612PL 0.23 Probability of recurrence on placebo

(0-6 months)p612PA 0.13 Probability of recurrence on PA

(6-12 months)p612H2 0.18 Probability of recurrence on H2RAs

(6-12 months)p612PPI 0.08 Probability of recurrence on PPIs

(6-12 months)p612LDPPI 0.13 Probability of recurrence on

low-dose PPIs (6-12 months)Symptom-week variables

SWPA 2.280 Symptom weeks on PASWH2 1.521 Symptom weeks on H2RAsSWDDH2 1.816 Symptom weeks on double dose of

H2RAsSWPPI 2.386 Symptom weeks on PPIsSWDDPPI 2.383 Symptom weeks on double dose of

PPIsSWSU 0.539 Symptom weeks after surgery

Unit cost variablesFee $4.11 Dispensing feeH2RA 0.44 150 mg H2RACIS 0.61 10 mg prokinetic agentPPI 2.42 20 mg PPILDPPI 1.93 10 mg PPIGPGA $48.20 General practitioner general

assessmentGPRA $28.10 General practitioner reassessmentGPMA $16.25 General practitioner minor

assessmentGERA $38.65 Gastroenterologist reassessmentGEPA $23.10 Gastroenterologist partial

assessmentUGIE $118.22 Upper gastrointestinal endoscopyUGIS $141.43 Upper gastrointestinal seriesCST $84.81 Cardiac stress testECG $42.77 ElectrocardiogramBS $135.10 Barium swallowNF $2462.60 Laparoscopic fundal plication

Resource use variablesnGPR1 2.5 Visits to general practitioner (1st

recurrence)nGER1 1.5 Visits to gastroenterologist (1st

recurrence)nBSR1 0.1 Percentage getting a barium swallow

(1st recurrence)nCSTR1 0.0 Percentage getting cardiac stress test

(1st recurrence)nECGR1 0.0 Percentage getting electrocardiogram

(1st recurrence)nUGIER1 0.1 Percentage getting upper gastro-

intestinal endoscopy (1st recurrence)nUGISR1 0.1 Percentage getting upper gastro-

intestinal series (1st recurrence)nGPR2 2.5 Visits to general practitioner (2nd

recurrence)nGER2 1.5 Visits to gastroenterologist (2nd

recurrence)nBSR2 Percentage getting a barium swallow

(2nd recurrence)nCSTR2 0.0 Percentage getting cardiac stress test

(2nd recurrence)nECGR2 0.0 Percentage getting electrocardiogram

(2nd recurrence)nUGIER2 0.6 Percentage getting upper gastro-

intestinal endoscopy(2nd recurrence)

nUGISR2 0.1 Percentage getting upper gastro-intestinal series (2nd recurrence)

Note: PPI = proton pump inhibitor; H2RA = H2-receptor antagonist; PA =prokinetic agent.

Table 2 Continued

Name Value Description





Recurrence probability

Drug dose Healing hazard 0-6 months 6-12 months

None (placebo)

(Note different

scale from rest of

0.00 0.20 0.40 0.60 0.80 1.00

mean = .67se = 0.06

0.00 0.10 0.20 0.30 0.40 0.50

mean = .23se = 0.05

PA

0.00 0.06 0.12 0.18 0.24 0.30

mean = .045se = 0.006

0.00 0.10 0.20 0.30 0.40 0.50

mean = .230se = 0.020

0.00 0.06 0.12 0.18 0.24 0.30

mean = .13se = 0.02

Regular dose

H2RA

0.00 0.06 0.12 0.18 0.24 0.30

mean = 0.068se = 0.006

0.00 0.10 0.20 0.30 0.40 0.50

mean = .38se = 0.05

0.00 0.06 0.12 0.18 0.24 0.30

mean = .18se = 0.04

Double dose

H2RA

0.00 0.06 0.12 0.18 0.24 0.30

mean = 0.091se = 0.012

Low-dose PPI

0.00 0.10 0.20 0.30 0.40 0.50

mean = .25se = 0.04

0.00 0.06 0.12 0.18 0.24 0.30

mean = .13se = 0.02

Regular dose

PPI

0.00 0.06 0.12 0.18 0.24 0.30

mean = 0.215se = 0.009

0.00 0.10 0.20 0.30 0.40 0.50

mean = .12se = 0.03

0.00 0.06 0.12 0.18 0.24 0.3

mean = .08se = 0.01

Double-dose

PPI

0.00 0.06 0.12 0.18 0.24 0.30

mean = 0.209se = 0.018

column!)

Table 3. Distributions for Healing Hazards and Recurrence Probabilities

Note: PA = prokinetic agent; H2RA = H2-receptor antagonist; PPI = proton pump inhibitor.





Resource item First recurrence Second recurrence

Visits to GP(Gamma distribution)

0.00 2.00 4.00 6.00 8.00 10.00

mean = 2.5se = 1.6

0.00 2.00 4.00 6.00 8.00 10.00

mean = 2.5se = 1.6

Visits to gastroenterologist(Gamma distribution)

0.00 2.00 4.00 6.00 8.00 10.00

mean = 1.5se = 0.6

0.00 2.00 4.00 6.00 8.00 10.00

mean = 1.5se = 0.6

Proportion getting a bariumswallow

0.00 0.04 0.08 0.12 0.16 0.20

mean = .07se = 0.03

Proportion getting cardiac stresstest

0.00 0.04 0.08 0.12 0.16 0.20

mean = .035se = 0.018

0.00 0.04 0.08 0.12 0.16 0.20

mean = .01se = 0.01

Proportion getting ECG

0.00 0.04 0.08 0.12 0.16 0.20

mean = .035se = 0.018

0.00 0.04 0.08 0.12 0.16 0.20

mean = .01se = 0.01

Proportion getting upper GIendoscopy

0.00 0.04 0.08 0.12 0.16 0.20

mean = .10se = 0.03

0.00 0.20 0.40 0.60 0.80 1.00

mean = .63se = 0.05

Proportion getting upper GIseries

0.00 0.04 0.08 0.12 0.16 0.20

mean = .07se = 0.03

0.00 0.04 0.08 0.12 0.16 0.20

mean = .08se = 0.03

Table 4. Distributions for Resource Use Parameters

Note: GP = general practitioner; ECG = electrocardiogram; GI = gastrointestinal.



true opportunity cost of those resources, it is not clearhow such uncertainty could be represented in thismodel. Therefore, all unit costs were taken as being de-terministic and were not ascribed distributions in thisanalysis. Although there are some problems with thisapproach,24 this is not conceptually different from theapproach taken in stochastic cost-effectiveness analy-sis alongside clinical trials where it is typical for unitcosts to be treated as fixed.

Results of the Probabilistic Analysis

Having specified distributions for all the relevantparameters of the model, the probabilistic analysis wasundertaken by randomly sampling from each of the pa-rameter distributions and calculating the expectedcosts and expected weeks free of GERD for that combi-nation of parameter values. This process formed a sin-gle replication of the model results, and a total of10,000 replications were performed in order to exam-ine the distribution of the resulting cost and outcomesfor each strategy. The results of these 10,000 replica-tions from the model are presented on the cost-effectiveness plane in Figure 3 together with the base-line estimate of the efficient frontier.

It is clear that for each of the individual replications,an efficient frontier could be calculated together withthe incremental cost-effectiveness ratios for treatmentson the frontier. In particular, Figure 3 shows how it maynot be possible to rule out strategy F, the strategy based

on step-down maintenance PPI, since it potentiallyforms part of the frontier in many replications. Note,however, that it is not possible to gain a clear view fromFigure 3 as to how often strategy F forms part of thefrontier: this is because there can be substantialcovariance between the simulations plotted in the fig-ure. Table 5 summarizes the proportion of the 10,000simulations in which each strategy forms part of thecost-effectiveness frontier. It is immediately clear fromthis table (as it was from Figure 3) that strategy D can beruled out. By contrast, it turns out that strategy F formspart of the frontier in 27% of simulations, and it is notclear how this result should be interpreted.

If the shadow price for a week free of GERD symp-toms (the maximum willingness to pay or ‘ceiling ra-tio’) were known, it would be possible to choose be-tween all of the treatment strategies, not just identifythose that form the efficient frontier. Therefore, condi-



F

B

E

A

C

D

$600

$700

$800

$900

$1,000

$1,100

$1,200

38.00 39.00 40.00 41.00 42.00 43.00 44.00 45.00 46.00 47.00 48.00

Weeks free of GERD

Str

ateg

y co

st

A: Intermittent PPIB: Maintenance PPIC: Maintenance H2RAD: Step-down maintenance PAE: Step-down maintenance H2RAF: Step-down maintenance PPI

Figure 3. Results of 10,000MonteCarlo simulation evaluations of the gastroesophageal reflux disease (GERD)model presented on the cost-effectiveness plane. PPI = proton pump inhibitor; H2RA = H2-receptor antagonist; PA = prokinetic agent.

Table 5. Percentage of Times Each Strategy FormedPart of the Cost-Effectiveness Efficiency Frontier

Strategy Percentage of Times on the Frontier

A 72B 100C 76D 0E 98F 27



tional upon knowing the ceiling ratio, there is only 1treatment of choice from the 6 strategies under evalua-tion, and the proportion of times that an intervention isthe treatment of choice from the 10,000 replications ofthe model gives the strength of evidence in favor of thattreatment. Although it is possible to identify the effi-cient frontier, calculate the incremental cost-effective-ness ratios, and choose 1 strategy from the 6 availablefor each of the 10,000 replications, a much morestraightforward approach exists.

The net benefit framework has been argued to offermany advantages for handling uncertainty in cost-effectiveness analysis25 and overcomes the particularproblem associated with negative incremental cost-effectiveness ratios.26 A further property is thatwhereas average cost-effectiveness ratios have nomeaningful interpretation, average net benefits havethe useful property that the incremental net benefit be-tween any 2 treatments can be calculated from the dif-ference between their individual average net benefits.25

Therefore, the treatment of choice from the 6 strategiesunder evaluation will be the treatment with the greatestaverage net benefit. This must be the case, as only thattreatment will have a positive incremental net benefitwhen compared to any other treatment alternative. Theproportion of times a strategy has the highest net bene-fit among the 10,000 replications of the model gives thestrength of evidence in favor of that strategy being cost-effective. This ability of the net benefit framework to

handle multiple mutually exclusive treatment optionsis a very strong advantage of the approach, as pointedout by the authors in their original article.25

Of course, in reality the shadow price of a week freeof GERD symptoms is not known. However, by plottingout, for all possible values of the ceiling ratio (Rc), theproportion of times the intervention has the greatest netbenefit, much can be learned concerning the implica-tions of the estimated uncertainty for the treatment de-cision. Figure 4 shows the result of just such an exercisefor the probabilistic evaluation of the GERD model pre-sented in Figure 3. These curves are conceptually thesame as the use of acceptability curves to summarizeuncertainty on the cost-effectiveness plane in a2-treatment decision problem.2

As expected, strategy D does not feature in Figure 4,indicating that it is never a contender for cost-effectiveness. Strategy F does feature, although it neverachieves more than 13% of simulations, suggesting it iscost-effective, even at the most favorable ceiling ratio(about $260 per day free of GERD symptoms). In thesesituations, Stinnett and Mullahy25 suggested that welook to the principle of stochastic dominance in orderto rule out interventions. Because average net benefitsare separable (in contrast to cost-effectiveness ratios), ifit can be shown that the cumulative distribution of onetreatment option is always greater than that of anotheroption, the original treatment is said to dominate. Thismeans that a utility-maximizing decision maker



A

BC E

F0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 10 100 1000

Ceiling Ratio (Rc)

Pro

bab

ility

co

st-e

ffec

tive

Figure 4. Acceptability curves for the choice of treatment strategy (a log scale is employed to better illustrate the low values).



should always prefer the 1st strategy to the 2nd no mat-ter what their risk preference. This condition is knownas 1st-order stochastic dominance. A slightly weakerform is 2nd-order stochastic dominance, which allowsthe cumulative density functions over net benefit tocross, but holds that a treatment dominates another ifthe area under its cumulative density function is al-ways greater than that under the alternative strategy. InFigures 5 and 6, we take the most favorable value of theceiling ratio from the perspective of strategy F and plot

the cumulative density functions of strategies B, E, andF. It turns out that we cannot quite rule out strategy Funder 1st-order stochastic dominance, but strategy F isvery clearly ruled out under the 2nd-order condition.

DISCUSSION

Probabilistic sensitivity analysis methods have beenproposed for decision analysis and cost-effectivenessmodels for some time. Despite this, the vast majority of



0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

$10,500 $10,700 $10,900 $11,100 $11,300 $11,500

Net-benefit

Cu

mu

lati

ve d

ensi

ty

Strategy B

Strategy E

Strategy F

Figure 5. Cumulative density functions over net benefit (Rc = $260 per day free of gastroesophageal reflux disease symptoms).

A

BC E

F0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 10 100 1000

Ceiling Ratio (Rc)

Pro

bab

ility

co

st-e

ffec

tive

C BEA

36 264

Figure 6. Decision making using management strategy “most likely” to be cost-effective.



published economic evaluations do not make use ofprobabilistic methods. Although a number of authorshave begun to emphasize the advantages of adopting aBayesian perspective in probabilistic modeling,27–29

many probabilistic analyses do not adopt a Bayesianapproach, and for such studies the choice of distribu-tions to represent uncertainty in parameters may ap-pear rather ad hoc.30–36 Furthermore, the Bayesian ap-proach may afford a more intuitive interpretation of theprobabilistic sensitivity results that has been high-lighted as lacking in a review of the use of medical deci-sion analysis models.37

Adopting a Bayesian approach to the probabilisticanalysis of the model of GERD management allows theintuitive interpretation often afforded to acceptabilitycurves as showing the probability that the interventionis cost-effective. Note that the curves all sum to 1 on thevertical axis (this clearly must be the case, since only 1strategy is chosen for each value of the ceiling ratio andfor each replication of the model). It is immediately ap-parent from Figures 3 and 4 that strategy D is alwaysdominated. The acceptability curves in Figure 4 clearlyshow that initial concern that strategy F might formpart of the frontier and might therefore be a cost-effectiveness choice in some situations was unwar-ranted. In fact, the conditions necessary for strategy F to

be considered the most cost-effective option rarelyarose in the simulations. Appealing to the principle ofstochastic dominance (2nd-order) shows strategy F tobe clearly rejected even under the most favorable as-sumptions concerning the ceiling ratio (Figs. 5, 6). Thisillustrates how by employing probabilistic methodsthat directly quantify the strength of evidence in favorof alternative treatment options, it is possible to be con-fident about excluding particular strategies, in contrastto the traditional sensitivity analysis approaches.

Although this sort of presentation of the choice be-tween mutually exclusive treatments in the face ofmany options is a natural extension of the use of cost-effectiveness acceptability curves in the 2-treatmentcase, the issue arises of how exactly decision makersshould use this information to choose between the re-maining strategies that form part of the frontier. One ap-proach (as illustrated in Fig. 7) would be to say that, forany given value of the ceiling ratio, the optimal deci-sion would be to choose the strategy that is most likelyto be cost-effective. But, of course, this decision rulegives the exact same treatment recommendations as thebaseline estimates in Figure 2, where uncertainty wasnot considered.

The conventional approach to statistical decisionmaking is based on the adoption of a 5% type I error



A

BC E

F

A+E

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 10 100 1000

Ceiling Ratio (Rc)

Pro

bab

ility

co

st-e

ffec

tive

C BA+E

26 480

Figure 7. Decisionmaking assumingmore expensive strategies have to be shown to be significantlymore cost-effective than the current strat-egy (at conventional 5% level).



rate. We might therefore adopt a decision rule that amore effective and more expensive treatment strategyshould replace the currently provided treatment only ifit can be shown to be significantly more cost-effective.This approach to decision making is illustrated in Fig-ure 7 and gives markedly different cutoff points for de-cision maker’s ceiling ratios for the different strategiesto be considered cost-effective. For example, the mosteffective therapy, maintenance PPI, would only be con-sidered the appropriate treatment option if decisionmakers had an underlying willingness to pay of $480per week free of GERD symptoms, assuming that F isnot part of the choice set.

Note that under a 5% significance decision rule, nei-ther strategy A nor E would be considered significantlycost-effective to be a clear decision choice. However, itis clear that strategy C is not cost-effective (at the 5% er-ror rate) for a shadow price greater than $26 per weekfree of GERD symptoms. Recall that strategy A involveshealing with PPI without maintenance and strategy Einvolves healing with PPI and then maintenance usingH2RAs. The choice between these strategies is betweenno maintenance and maintenance with H2RAs follow-ing healing. If the shadow price of a week free of GERDsymptoms is between $26 and $480 per week free ofGERD symptoms, then we know that strategies B, C, D,and F are not cost-effective. We cannot distinguishstrategies A and E at conventional significance levels,so for purposes of decision making it should be clearthat the strategy to heal with PPI should be adopted, butthat it is unlikely to be important whether, subsequentto healing, no maintenance or maintenance withH2RAs is undertaken. This is illustrated in Figure 7 bycombining the 2 strategies in 1 acceptability curve.

It is important to recognize, however, the arbitrarynature of the conventional (and frequentist) decisionrule. Consider whether instead of placing the “burdenof proof” for cost-effectiveness on more expensive andmore effective strategies, it is the cheaper but less effec-tive strategies that would be used only if they wereshown to be significantly cost-effective. Although notshown in Figure 7, it should be clear from the above ex-position that this change in the burden of proof wouldresult in a new set of threshold values such that strategyB would be the treatment of choice unless the shadowprice were below $170 per week free of GERD symp-

toms; between $17.50 and $170 per week free of GERDsymptoms, either strategy A or E would be consideredcost-effective; and below a shadow price of $17.50, ei-ther A or C would be the strategy of choice.

Of course the arbitrary nature of such decision mak-ing under uncertainty emphasizes the inadequacies ofsuch a simple decision rule—Claxton38 argued that sig-nificance testing of this sort is irrelevant. Instead, hesuggested that decision making be concerned funda-mentally with expected values. That is not to say thatthe decisions should be made on the basis of the base-line point estimates as presented in Figure 2 withoutreference to uncertainty in obtaining those estimates.Rather, the expected returns to obtaining further infor-mation should be assessed in order to determinewhether it is worth commissioning more research toobtain improved estimates of the decision parameters.Such an approach would require estimates of the lossfunction associated with incorrect decision making,the size of the population relevant to the decision, thelifetime of the technologies associated with each man-agement strategy, and the returns to sampling. Each isitself subject to a great deal of uncertainty, and themethods for incorporating all this information into asingle overall analysis are currently under develop-ment. We have not attempted such an analysis in thisarticle, although we recognize that it is a possible ex-tension to the work presented here and that such ananalysis would be considered a fully Bayesian decisionmodel.

In summary, probabilistic modeling of deterministicmodels is a practical solution to the problems of con-ventional sensitivity analysis. Adopting a Bayesian ap-proach encourages analysts and users to think carefullyabout the state of evidence relating to the parameters ofthe model. Single-parameter specifications arestraightforward to apply in a Bayesian framework andprovide a simple way to update parameter distribu-tions as new data become available. The use of accept-ability curves to present information on the probabilityof multiple treatment options is a natural extension ofthe 2-alternative case usually presented in the litera-ture. Much current research interest is focused on ex-pected value of information methods. It is clear thatsuch methods will have to be predicated on a well-specified probabilistic model.





APPENDIX

The model that formed the basis of this article is availablefor download from the World Wide Web at www.ihs.ox.ac.uk/herc/downloads.

Two formats are available: a spreadsheet model (MicrosoftExcel 2000) and a TreeAge DATA™ model (version 4.0). Bothmodels use the same parameters and naming conventions re-ported in this article and both are fully probabilistic. Full de-tails of the modeling assumptions and parameters are givenin the original (deterministic) article,9 and readers are ad-vised to familiarize themselves with this article to aid under-standing of the model structure.

Note that the model implemented in TreeAge DATA™ iscomplete and can be run as either a deterministic model or aprobabilistic model. An evaluation copy of DATA is availablefrom www.treeage.com.

The spreadsheet model implemented in Excel has beenstripped of the simulation results to reduce the file size and tofacilitate speed of downloading. When opening the file, thesimulation results can be generated by pressing the “RunMonte Carlo Simulation” button on the 1st page, which em-ploys a visual basic macro to record simulation results. Notethat no calculations are embodied in this macro—the macrosimply implements a loop to repeatedly copy the results ofcalculations into the spreadsheet.

The purpose of this appendix is to describe in more detailthe fitting of distributions in the probabilistic modeling exer-cise and the sampling from those distributions in relation tothe 2 software platforms employed to illustrate the model.

Beta Distribution

As described in the article, obtaining parameters for abeta(r,n) distribution is straightforward when modeling aprobability from binomial data because the parameters havean interpretation as r = events and n = total sample size. Thisis the default parameterization of the beta distribution inDATA; however, this only works with integer values of r andn. For the method-of-moments fitting described in this arti-cle, it is necessary to choose the “real numbered parameters”version of the beta distribution: beta(α, β), where α = r andα + β = n.

Method-of-moments fitting involves equating the meansand variances observed in the data to the expressions for themean and variance of the distribution and solving for the pa-rameters of that distribution. Therefore, for observed mean,µ,and standard error, s, we formulate the expressions

µ = = −+

rn

sr n rn n

22 1( )( )

and rearrange to give

ns

r n

= − −

=

µ µ

µ

( )112 .

So, for example, the instantaneous hazard for the prokineticagent in the model was estimated to be 0.045 with a standarderror of 0.006. Using the above expressions generates parame-ters for the beta distribution of approximatelyn= 1194 and r=54, or α = 54 and β = 1140 (these are approximate due torounding—the models use full accuracy).

Gamma Distribution

Parameters involving numbers of visits to health care pro-fessionals (primary care physicians and gastroenterologists)were modeled using the gamma distribution, which has 2 pa-rameters, α and β. The same method-of-moments approachwas used to obtain these parameters. The mean and varianceof the gamma(α, β) distribution are given by

mean = =αβ

αβ

, var 2 .

Setting these equal to the corresponding estimates, and thenrearranging, gives expressions for the parameters

α µ β µ= =2

2 2s s, .

For example, visits to the general practitioner after a recur-rence of GERD are estimated to occur with a mean 2.5 visitsand a standard error of 1.6, which is modeled in the DATAmodel as gamma(4,1.6).

Note, however, that the parameterization of the gammadistribution in Excel is slightly different from that commonlyfound in other packages. Excel uses exactly the same distribu-tion but has parameterized β as 1/β, which means that in Ex-cel the appropriate version of the expression for the parame-ters is

α µ βµ

= =2

2

2

ss

, .

For the same example of general practitioner visits after re-currence of GERD, the same distribution modeled in Excel isgamma(4,0.625).

Generating Random Drawsfrom the Specified Distribution

In DATA, there is no need to be concerned about drawingrandom values from the specified distribution because, hav-ing specified the parameters, the software takes care of simu-lation. In Excel (and many other software packages), there isno direct simulation function. However, it is straightforwardto use the distribution functions provided together with therandom number generator function to obtain random draws.The upper panel of Figure A1 shows a cumulative distribu-tion function (cdf) for a (standard normal) distribution—the





values of the distribution are shown on the horizontal axis,and the vertical axis shows the proportion of the probabilitydensity function (pdf) falling below that value. The verticalaccess therefore has a scale of 0 to 1, and the cdf can bethought of as a function that maps the 0-to-1 scale onto thescale of the pdf. We can use the cdf together with a uniformrandom variate to obtain a random draw from the pdf simplyby generating a random value between 0 and 1 and readingacross from the vertical access to the curve to obtain a valuefrom the pdf (see Fig. A1). Repeating this process a large num-ber of times generates random values from the pdf shown atthe bottom of Figure A1.

In Excel, the required functions for the cdfs of distribu-tions have an INV suffix (because we are using the inverse ofthe cdf) and each function has 3 arguments—for example,

BETAINV(p,α, β) and GAMMAINV(p,α, β)–and the functionreturns a value,q. The 1st argument represents the proportionof the pdf up to the value q, and the 2 remaining argumentsare the parameters of the distribution function. The functionRAND() in Excel generates a uniform random variate on theinterval 0 to 1. Hence, to obtain a random draw from the betaand gamma distributions specified in the preceding sections,we use the BETAINV(RAND(),54,1140) and GAMMAINV(RAND(),4,0.625) in Excel.

Note that a 0-1 switch is used in the Excel model to allow the user to choosebetween a probabilistic and a deterministic version of the model. Thisswitch is located in cell B3 of the <Parameters> worksheet. Setting this cellto 1 allows the user to observe simulated values in real time—pressing the<F9> key draws another set of random values for the parameters.

REFERENCES

1. O’Brien BJ, Drummond MF, Labelle RJ, Willan A. In search ofpower and significance: issues in the design and analysis of stochas-tic cost-effectiveness studies in health care. Med Care. 1994;32:150–63.

2. van Hout BA, Al MJ, Gordon GS, Rutten FF. Costs, effects andC/E-ratios alongside a clinical trial. Health Econ. 1994;3:309–19.

3. Stinnett AA, Paltiel AD. Estimating CE ratios under second-order uncertainty: the mean ratio versus the ratio of means. MedDecis Making. 1997;17:483–9.

4. Briggs AH, Gray AM. Handling uncertainty when performingeconomic evaluation of health care interventions. Health TechnolAssess. 1999;3(2).5. Critchfield GC, Willard KE, Connelly DP. Probabilistic sensitiv-

ity analysis methods for general decision models. ComputersBiomed Res. 1986;19:254–65.6. Doubilet P, Begg CB, Weinstein MC, et al. Probabilistic sensitivity

analysis using Monte Carlo simulation: a practical approach. MedDecis Making. 1985;5:157–77.7. Briggs AH. Handling uncertainty in cost-effectiveness models.

PharmacoEcon. 2000;17:479–500.



0

0.2

0.4

0.6

0.8

1

-3 -2 -1 0 1 2 3

-3 -2 -1 0 1 2 3

0

1

Uni

form

ran

dom

var

iate

Cumulative distribution function

Probability distribution function

0

0.2

0.4

0.6

0.8

1

-3 -2 -1 0 1 2 3

-3 -2 -1 0 1 2 3

0

1

0

1

Figure A1. Random draws from a parametric distribution.



8. Manning WG, Fryback DG, Weinstein MC. Reflecting uncer-tainty in cost-effectiveness analysis. In: Gold MR, Siegel JE, RussellLB, Weinstein MC, eds. Cost-Effectiveness in Health and Medicine.New York: Oxford University Press, 1996. p. 247–75.9. Goeree R, O’Brien B, Hunt R, Blackhouse G, Willan A, Watson J.

Economic evaluation of long term management strategies for erosiveoesophagitis. PharmacoEcon. 1999;16:679–97.10. Chiba N, de Gara CJ, Berget D. Rapidity of healing in GERD [ab-stract]. Gastroenterology. 1993;104:A53.11. Chiba N, Wilkinson J, Hunt RH. Symptom relief in erosive GERD[abstract]. Am J Gastroenterol. 1993;88:A1486.12. Ministry of Health. Drug Benefit Formulary: Comparative DrugIndex no. 35. Toronto: Publications Ontario, 1996.13. Ontario Ministry of Health. Schedule of Benefits: Physician Ser-vices under the Health Insurance Act. Toronto: Publications Ontario,1992.14. Ontario Hospital Association. Ontario Case Costing Project: On-tario Guide to Case Costing. Toronto: Publications Ontario, 1995.15. Park RE, Fink A, Brook RH. Physician ratings of appropriate in-dications for six medical and surgical procedures. Am J PublicHealth. 1986;76:766–72.16. Weinstein MC. From cost-effectiveness ratios to resource alloca-tion: where to draw the line? In: Sloan FA, ed. Valuing Health Care.Cambridge, UK: Cambridge University Press, 1995. p. 77–98.17. Gold MR, Siegel JE, Russell LB, et al. Cost-Effectiveness in Healthand Medicine. New York: Oxford University Press, 1996.18. Drummond MF, O’Brien B, Stoddart GL, et al. Methods for theEconomic Evaluation of Health Care Programmes. 2nd ed. Oxford,UK: Oxford University Press, 1997.19. Cantor SB. Cost-effectiveness analysis, extended dominance,and ethics: a quantitative assessment. Med Decis Making.1994;14:259–65.20. Eddy DM, Hasselblad V, Shachter R. A Bayesian method for syn-thesizing evidence: the confidence profile method. Int J Technol As-sess Health Care. 1990;6(1):31–55.21. Eddy DM, Hasselblad V, Shachter R. An introduction to aBayesian method for meta-analysis: the confidence profile method.Med Decis Making. 1990;10(1):15–23.22. Altman DG. Practical Statistics for Medical Research. London:Chapman & Hall, 1991.23. Pratt JW, Raiffa H, Schlaifer R. Introduction to Statistical Deci-sion Theory. Cambridge, MA: MIT Press, 1995.24. Rittenhouse BE, Dulisse B, Stinnett AA. At what price signifi-cance? The effect of price estimates on statistical inference in eco-nomic evaluation. Health Econ. 1999;8:213–9.

25. Stinnett AA, Mullahy J. Net health benefits: a new framework forthe analysis of uncertainty in cost-effectiveness analysis. Med DecisMaking. 1998;18(2 suppl):S65–S80.26. Stinnett AA, Mullahy J. The negative side of cost-effectivenessanalysis [letter; comment]. JAMA. 1997;277:1931–2.27. Parmigiani G, Samsa GP, Ancukiewicz M, Lipscomb J,Hasselblad V, Matchar DB. Assessing uncertainty in cost-effective-ness analyses: application to a complex decision model. Med DecisMaking. 1997;17:390–401.28. Sendi PP, Craig BA, Meier G, Pfluger D, Gafni A, Opravil M,Battegay M, Bucher HC. Cost-effectiveness of azithromycin for pre-ventingMycobacteriumavium complex infection in HIV-positive pa-tients in the era of highly active antiretroviral therapy: the Swiss HIVCohort Study. J Antimicrob Chemother. 1999;44:811–7.29. Fryback DG, Chinnis JO, Ulvila JW. Bayesian cost-effectivenessanalysis: an example using the GUSTO trial. Int J Technol AssessHealth Care. 2001;17(1):83–97.30. Berwick DM, Cretin S, Keeler E. Cholesterol, children, and heartdisease: an analysis of alternatives. Pediatrics. 1981;68:721–30.31. Hornberger JC. The hemodialysis prescription and cost effective-ness. J Am Soc Nephrol. 1993;4:1021–7.32. Gabriel SE, Campion ME, O’Fallon WM. A cost-utility analysisof misoprostol prophylaxis for rheumatoid arthritis patients receiv-ing nonsteroidal antiinflammatory drugs. Arthritis Rheum.1994;37:333–41.33. Pharoah PD, Hollingworth W. Cost effectiveness of lowering cho-lesterol concentration with statins in patients with and without pre-existing coronary heart disease: life table method applied to healthauthority population. Br Med J. 1996;312:1443–8.34. Fiscella K, Franks P. Cost-effectiveness of the transdermal nico-tine patch as an adjunct to physicians’ smoking cessation counseling[see comments]. JAMA. 1996;275:1247–51.35. Oh PI, Maerov P, Pritchard D, Knowles SR, Einarson TR, ShearNH. A cost-utility analysis of second-line antibiotics in the treatmentof acute otitis media in children. Clin Ther. 1996;18:160–82.36. Pasta DJ, Taylor JL, Henning JM. Probabilistic sensitivity analysisincorporating the bootstrap: an example comparing treatments forthe eradication of Helicobacter pylori. Med Decis Making.1999;19:353–63.37. Sonnenberg FA, Roberts MS, Tsevat J, Wong JB, Barry M, KentDL. Toward a peer review process for medical decision analysis mod-els. Med Care. 1994;32(7 suppl):JS52–JS64.38. Claxton K. The irrelevance of inference: a decision-making ap-proach to the stochastic evaluation of health care technologies. JHealth Econ. 1999;18:341–64.





medical decision making - mcgill university · 292 •medical decision making/july–aug 2002...

Documents