cost-effectiveness models to inform trial design: calculating the expected value of sample...
TRANSCRIPT
![Page 1: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/1.jpg)
Cost-effectiveness models to inform trial design:
Calculating the expected value of sample information
Alan Brennanand
J Chilcott, S Kharroubi, A O’Hagan
![Page 2: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/2.jpg)
Overview • Principles of economic viability• 2 level Monte-Carlo algorithm & Mathematics• Calculating EVSI (Bayesian Updating) case studies
– Normal, Beta, Gamma distributions– Others – WinBUGS, and approximations.
• Illustrative and real example• Implications • Future Research
![Page 3: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/3.jpg)
Example:- Economic viability of a proposed oil reservoir
• Some information suggesting there is oil• Could do further sample drilling to “size” the oil reservoir• Decision = “Go / No go” • Criterion = expected profit (net present value or NPV)
Is the sampling worthwhile? … that depends on …• Costs of collecting the data• Current uncertainty in reservoir size Expected gain from sampling = (P big reservoir*Big profits)+(P small reservoir*Big loss)–(Sample cost)
![Page 4: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/4.jpg)
Analogies• Drug Development Project
– Go / No go decisions– Trial supports consideration of next decision (Phases to launch)– Criterion = Expected profit (NPV)– Correct decision
profit if good drug, avoided financial loss if not a good drug
• NICE / NCCHTA decision– Approval or not– Is additional research required before decision can be made– Criterion = Cost per QALY…. i.e. net health benefits– Correct decision better health (efficiently) if good drug,
avoided poor health investment if not a good drug
![Page 5: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/5.jpg)
Principles• Strategy options with uncertainty about their performance• Decision to make• Sampling is worthwhile if
Expected gain from sampling - expected cost of sampling > 0
• Expected gain from sampling = Function (Probability of changing the decision|sample, .
amount of gain made / loss avoided)
• Applies to all decisions
![Page 6: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/6.jpg)
Algortihm
![Page 7: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/7.jpg)
2 Level EVSI - Research Design4, 5
0)Decision model, threshold, priors for uncertain parameters1) Simulate data collection: • sample parameter(s) of interest once ~ prior • decide on sample size (ni) (1st level)• sample a mean value for the simulated data | parameter of interest 2) combine prior + simulated data --> simulated posterior 3) now simulate 1000 times
parameters of interest ~ simulated posteriorunknown parameters ~ prior uncertainty (2nd level)
4) calculate best strategy = highest mean net benefit 5) Loop 1 to 4 say 1,000 times Calculate average net benefits 6) EVSI parameter set = (5) - (mean net benefit | current information)
![Page 8: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/8.jpg)
Mathematics
![Page 9: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/9.jpg)
2 Level EVSI - Mathematics 4, 5
Mathematical Formulation: EVSI for Parameters = the parameters for the model (uncertain currently).d = set of possible decisions or strategies.NB(d, ) = the net benefit for decision d, and parameters
Step 1: no further information (the value of the baseline decision) Given current information chose decision giving maximum expected net benefit. Expected net benefit (no further info) = (1)
i = the parameters of interest for partial EVPI -i = the other parameters (those not of interest, i.e. remaining uncertainty)
4 Brennan et al Poster SMDM 2002
5 Brennan et al Poster SMDM 2002
)NB(d,max Ed
![Page 10: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/10.jpg)
2 Level EVSI - Mathematics 4, 5
Step 6: Sample Information on iExpected Net benefit, sample on i = (6)
Step 7: Expected Value of Sample Information on i(6) – (1) Partial EVSI =
(7)
This is a 2 level simulation due to 2 expectations 4 Brennan et al Poster
SMDM 2002
5 Brennan et al Poster
SMDM 2002
)NB(d,max|)NB(d,max EEEd
id
X i
id
X iEE |)NB(d,max
![Page 11: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/11.jpg)
Bayesian Updating
NormalBeta
Gamma
![Page 12: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/12.jpg)
Normal Distribution
![Page 13: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/13.jpg)
Normal Distribution 0= prior mean for the parameter
0= prior uncertainty in the mean (standard deviation) = precision of the prior mean
2pop = patient level uncertainty from a sample ( needed for Bayesian update
formula) = sample mean (further data collection from more patients /
clinical trial study entrants). = precision of the sample mean .
= sample variance 4 Brennan et al Poster
SMDM 20025 Brennan et al Poster
SMDM 2002
200 /1 I
X
2/1 sI
npop /22
![Page 14: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/14.jpg)
Normal Distribution= implied posterior mean
(the Bayesian update of the mean following the sample information)
= implied posterior standard deviation
(the Bayesian update of the std dev following the sample information)
s
s
0
001
2022
0
221
/
/
n
n
pop
pop
![Page 15: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/15.jpg)
Normal Distribution - Implications• Implied posterior variance will always be smaller than the
prior variance because the denominator of the adjustment term is always larger than the numerator.
• If the sample size is very small then the adjustment term will almost be equal to 1 and posterior variance is almost identical to the prior variance.
• If the sample size is very large, the numerator of the adjustment term tends to zero, the denominator tends to the prior variance and so, posterior variance tends towards zero.
![Page 16: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/16.jpg)
Normal DistributionNormal Bayesian Update (n=50)
0
50
100
150
200
250
300
0% 20% 40% 60% 80% 100%
Response Rate (T0)
Fre
qu
ency
(10
00 s
amp
les)
Prior
Posterior aftersample 1
Posterior aftersample 2
Posterior aftersample 7
![Page 17: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/17.jpg)
Beta Distribution
![Page 18: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/18.jpg)
Beta / Binomial Distribution • e.g. % responders
• Suppose prior for % of responders is ~ Beta (a,b)
• If we obtain a further n cases, of which y are successful responders then
• Posterior ~ Beta (a+y,b+n-y)
![Page 19: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/19.jpg)
Gamma Distribution
![Page 20: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/20.jpg)
Gamma / Poisson Distribution e.g. no. of side effects a patient experiences in a year
• Suppose prior for mean number of side effects per person is ~ Gamma (a,b)
• If we obtain a further n samples, (y1, y2, … yn) from a Poisson distribution then
• Posterior for mean number of side effects per person ~ Gamma (a+ yi , b+n)
![Page 21: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/21.jpg)
Bayesian Updating
Other Distributions
![Page 22: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/22.jpg)
Other Distributions Beta Laplace
Inverted Beta LogisticCauchy 1 LognormalCauchy 2 NormalChi ParetoChi² 1 PowerChi² 2 RayleighErlang r-Distr.Expon. UniformFisher StudentGamma TriangularInverted Gamma WeibullGumbel
![Page 23: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/23.jpg)
Bayesian Updating without a Formula • WinBUGS
• Put in prior distribution
• Put in data (e.g. sample of patients or parameter)
• Use MCMC to generate posterior (‘000s of iterations)
• Use posterior in model to generate new decision
• Loop round and put in a next data sample
• Other approximation methods (talk to Samer!)
![Page 24: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/24.jpg)
Illustrative Model
![Page 25: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/25.jpg)
First (Illustrative) Model • 2 treatments – T1 versus T0
• Criterion = Cost per QALY < £10,000
• Uncertainty in ……
• % responders to T1 and T0
• Utility gain of a responder
• Long term duration of response
• Other cost parameters
![Page 26: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/26.jpg)
Illustrative Model d e
Number of Patients in the UK Uncertainty in Parameter MeansThreshold cost per QALY
Mean Model T0 T1 T0 T1Increment
(T1 over T0)Cost of drug 1 1 1,000£ 1,499£ 499£ % admissions 2% 2% 10% 11% 0%Days in Hospital 1.00 1.00 5.01 7.05 2.05
Cost per Day 200 200 589£ 589£ -£
% Responding 10% 10% 88% 70% -18%Utility change if respond 0.1000 0.0500 0.3119 0.2120 0.0998- Duration of response (years) 0.5 1.0 3.1 4.1 1.0
% Side effects 10% 5% 31% 24% -7%Change in utility if side effect 0.02 0.02 -0.06 -0.14 -0.08 Duration of side effect (years) 0.20 0.20 0.70 0.65 0.05-
Total Cost 1,308£ 1,937£ 629£
Total QALY 0.8394 0.5948 -0.2446
Cost per QALY 1,558£ 3,256£ -£2,570
Net Benefit of T1 versus T0 £7,086 £4,011 -£3,075
Sampled ValuesStandard Deviations
Illustrative Model a b c
Number of Patients in the UK 1,000 Threshold cost per QALY £10,000
Mean Model T0 T1Increment
(T1 over T0)Cost of drug 1,000£ 1,500£ 500£ % admissions 10% 8% -2%Days in Hospital 5.20 6.10 0.90
Cost per Day 400£ 400£ -£
% Responding 70% 80% 10%Utility change if respond 0.3000 0.3000 - Duration of response (years) 3.0 3.0 -
% Side effects 25% 20% -5%Change in utility if side effect -0.10 -0.10 0.00Duration of side effect (years) 0.50 0.50 -
Total Cost 1,208£ 1,695£ 487£
Total QALY 0.6175 0.7100 0.0925
Cost per QALY 1,956£ 2,388£ £5,267
Net Benefit of T1 versus T0 4,967£ 5,405£ £437.80
-£2,000
-£1,500
-£1,000
-£500
£0
£500
£1,000
£1,500
£2,000
-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Inc QALY
Inc
Co
st
![Page 27: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/27.jpg)
Illustrative Model d e
Number of Patients in the UK Uncertainty in Parameter MeansThreshold cost per QALY
Mean Model T0 T1 T0 T1Increment
(T1 over T0)Cost of drug 1 1 1,000£ 1,499£ 499£ % admissions 2% 2% 10% 11% 0%Days in Hospital 1.00 1.00 5.01 7.05 2.05
Cost per Day 200 200 589£ 589£ -£
% Responding 10% 10% 88% 70% -18%Utility change if respond 0.1000 0.0500 0.3119 0.2120 0.0998- Duration of response (years) 0.5 1.0 3.1 4.1 1.0
% Side effects 10% 5% 31% 24% -7%Change in utility if side effect 0.02 0.02 -0.06 -0.14 -0.08 Duration of side effect (years) 0.20 0.20 0.70 0.65 0.05-
Total Cost 1,308£ 1,937£ 629£
Total QALY 0.8394 0.5948 -0.2446
Cost per QALY 1,558£ 3,256£ -£2,570
Net Benefit of T1 versus T0 £7,086 £4,011 -£3,075
Sampled ValuesStandard Deviations
Illustrative Model a b c
Number of Patients in the UK 1,000 Threshold cost per QALY £10,000
Mean Model T0 T1Increment
(T1 over T0)Cost of drug 1,000£ 1,500£ 500£ % admissions 10% 8% -2%Days in Hospital 5.20 6.10 0.90
Cost per Day 400£ 400£ -£
% Responding 70% 80% 10%Utility change if respond 0.3000 0.3000 - Duration of response (years) 3.0 3.0 -
% Side effects 25% 20% -5%Change in utility if side effect -0.10 -0.10 0.00Duration of side effect (years) 0.50 0.50 -
Total Cost 1,208£ 1,695£ 487£
Total QALY 0.6175 0.7100 0.0925
Cost per QALY 1,956£ 2,388£ £5,267
Net Benefit of T1 versus T0 4,967£ 5,405£ £437.80
-£2,000
-£1,500
-£1,000
-£500
£0
£500
£1,000
£1,500
£2,000
-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Inc QALY
Inc
Co
st
![Page 28: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/28.jpg)
Illustrative Model Results • Baseline strategy = T1
• Cost per QALY = £5,267
• Overall EVPI = £1,351 per person
![Page 29: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/29.jpg)
£1,319
£867 £878
£716
£330
£-
£200
£400
£600
£800
£1,000
£1,200
£1,400
Val
ue
of
Info
rmat
ion
All Six Durations Trial +Utility
UtilityStudy
Trial on %Response
EVSI (n=50)
EVSI for Parameter Subsets
![Page 30: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/30.jpg)
EVI for % Responders to T0
0
50
100
150
200
250
0 50 100 150 200
Sample Size (n)
Va
lue
of
Info
rma
tio
n
EVSI (n)
EVPI
![Page 31: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/31.jpg)
Illustrative data collection cost = £100k fixed plus £500 marginal
Expected Net Benefit of Sampling
-200
-100
0
100
200
300
0 50 100 150 200
Sample Size (n)
Val
ue
of
Info
rmat
ion EVSI
Cost ofSampling
ENBS
Expected Net Benefit of Sampling
![Page 32: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/32.jpg)
Real Example
![Page 33: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/33.jpg)
Second Example • Pharmaco-genetic Test to predict response
• Rheumatoid Arthritis
• Up to 20 strategies of sequenced treatments
• U.S. - 2 year costs and benefits perspective
• Criterion = Cost per additional year in response
• Range of thresholds ($10,000 to $30,000)
• Real uncertainty (modelled by Beta’s)
![Page 34: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/34.jpg)
“Biologics”Anakinra ($12,697), Etanercept ($18,850), Infliximab ($24,112)*
Is Response Genetic?
91 patients, 150mg Anakinra, 24 week RCT1,2, gene = IL-1A +4845Positive response = reduction of at least 50% in swollen joints
1 Camp et al. American Human
Genetics Conf abstract 1088, 1999
2 Bresnihan
Arthritis & Rheumatism, 1998
0%
20%
40%
60%
80%
100%
Placebo Anakinra Gene+ve
Gene -ve
% a
ch
iev
ing
"S
wo
llen
50
"
0 .0 %
1 0 .0 %
2 0 .0 %
3 0 .0 %
4 0 .0 %
5 0 .0 %
6 0 .0 %
7 0 .0 %
8 0 .0 %
9 0 .0 %
1 0 0 .0 %
*Costs include monitoringAnakinra 100mgEtanercept 25mg eowInfliximab 3mg/kg 8 weekly 50% 50%100%
![Page 35: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/35.jpg)
Respond ?
Gene +ve? Yes …..
Yes Anakinra
No …..
PGt
Respond?
Yes …..
No Etanercept
No …..
Respond?
Yes
Anakinra
No
Before 0 - 6 months
Before 0 - 6 months
A Pharmaco-Genetic Strategy
Strategy 1
Strategy 2
![Page 36: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/36.jpg)
Partial EVSI: PGt Research onlyEVSI for PGt Research only
(for threshold = $20,000 per responder year gained)
$0.0$5.0
$10.0$15.0$20.0$25.0$30.0
10 20 50 100
200
500
1000
2000
5000
Perfe
ct
Sample Size
EV
SI
$m
Caveat: Small No.of Simulations on 1st Level
![Page 37: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/37.jpg)
Doing Fewer Calculations?
![Page 38: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/38.jpg)
Properties of the EVSI curve • Fixed at zero if no sample is collected
• Bounded above by EVPI
• Monotonic
• Diminishing return
• Suggests perhaps exponential form?
• Tried with 2 examples – fitted curve is exponential function of the square root of n
![Page 39: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/39.jpg)
EVI for % Responders to T0
0
50
100
150
200
250
0 50 100 150 200
Sample Size (n)
Va
lue
of
Info
rma
tio
n
EVSI (n)
EVPI
Exponential fit
EVSI = EVPI * [1- EXP( -0.3813 * sqrt(n) ]
Fitting an Exponential Curve to EVSI:Illustrative Model - % response to T0
![Page 40: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/40.jpg)
Fitting an Exponential Curve to EVSI:Pharmaco-genetic Test response
Pharmaco-genetic Test Response
0
5
10
15
20
25
30
0 100 200 300 400 500
Proposed Sample Size (n)
Val
ue
of
Info
rmat
ion
EVSI
Exponential Fit
EVPI
EVSI = EVPI * [1- EXP( -0.3118 * sqrt(n) ]
![Page 41: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/41.jpg)
Unresolved Question • Does the following formula always provide a good fit?
• EVSI (n) = EVPI * [1 – exp -a*sqrt(n) ]
• The 2 examples are Normal and Beta
• Is it provable by theory?
![Page 42: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/42.jpg)
Discussion Issues
Phase III trials Future Research Agenda
![Page 43: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/43.jpg)
Discussion Issues – Phase III trials
• Based on proving a clinical DELTA
• Implication is that if clinical DELTA is shown then adoption will follow i.e. it is a proxy for economic viability
• Often FDA requires placebo control (lower sample size), which implies DELTA versus competitors is unproven
• Could consider economic DELTA …….
![Page 44: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/44.jpg)
Discussion Issues – Phase III trials Early “societal” economic models provide a tool for assessing:
1. What would be an economic DELTA?
2. Implied sample needed in efficacy trial for cost-effectiveness
3. What other information is needed to prove cost-effectiveness?
4. Will proposed clinical DELTA be enough for decision makers
Similar commercial economic models could link • proposed data collection with • probability of re-imbursement and hence with • expected profit (NPV)
![Page 45: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/45.jpg)
Discussion Issues – Problems & Development Agenda
1. Technical - Bayesian Updating for other distributions
2. Partnership and case studies - to develop Bayesian tools for researchers who currently use frequentist only sample size calculation
3. Methods for complexity in Bayesian updating - e.g. the new trial will have slightly different patient group to the previous trial (meta-analysis and adjustment)
![Page 46: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/46.jpg)
Conclusions• Can now do EVSI calculations from a societal
perspective using the 2 level Monte-Carlo algorithm
• Bayesian Updating works for case studies– Normal, Beta, Gamma distributions– Others need – WinBUGS, and/or approximations.
• Future Research Issues– Bayesian Technical – Collaborative Issues with Frequentist Sample Size
![Page 47: Cost-effectiveness models to inform trial design: Calculating the expected value of sample information Alan Brennan and J Chilcott, S Kharroubi, A O’Hagan](https://reader035.vdocuments.net/reader035/viewer/2022062518/56649cb85503460f9497e99a/html5/thumbnails/47.jpg)
Thankyou