bayesian trial designs: drug case study donald a. berry [email protected] donald a. berry...
TRANSCRIPT
![Page 1: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/1.jpg)
Bayesian Trial Designs: Drug Case Study
Bayesian Trial Designs: Drug Case Study
Donald A. [email protected]
Donald A. [email protected]
BERRY
STATISTICAL INNOVATION
CONSULTANTS
![Page 2: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/2.jpg)
22
OutlineOutline
Some history
Why Bayes?
Adaptive designs
Case study
Some history
Why Bayes?
Adaptive designs
Case study
![Page 3: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/3.jpg)
![Page 4: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/4.jpg)
44
2004 JHU/FDA Workshop:“Can Bayesian Approaches to
Studying New Treatments Improve Regulatory Decision-Making?”
2004 JHU/FDA Workshop:“Can Bayesian Approaches to
Studying New Treatments Improve Regulatory Decision-Making?”
www.prous.com/bayesian2004
www.cfsan.fda.gov/~frf/ bayesdl.html
www.prous.com/bayesian2004
www.cfsan.fda.gov/~frf/ bayesdl.html
![Page 5: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/5.jpg)
55
Upcoming in 2005Upcoming in 2005
Special issue of Clinical Trials
“Bayesian Clinical Trials”Nature Reviews Drug Discovery
Special issue of Clinical Trials
“Bayesian Clinical Trials”Nature Reviews Drug Discovery
![Page 6: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/6.jpg)
66
Selected history of Bayesian trialsSelected history of Bayesian trials
Medical devices (30+)
200+ at M.D. Anderson (Phase I, II, I/II)
Cancer & Leukemia Group B
Pharma ASTIN (Pfizer) Pravigard PAC (BMS) Other
Decision analysis (go to phase III?)
Medical devices (30+)
200+ at M.D. Anderson (Phase I, II, I/II)
Cancer & Leukemia Group B
Pharma ASTIN (Pfizer) Pravigard PAC (BMS) Other
Decision analysis (go to phase III?)
![Page 7: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/7.jpg)
77
Why Bayes?Why Bayes?
On-line learning (ideal for adapting)
Predictive probabilities (including modeling outcome relationships)
Synthesis (via hierarchical modeling, for example)
On-line learning (ideal for adapting)
Predictive probabilities (including modeling outcome relationships)
Synthesis (via hierarchical modeling, for example)
![Page 8: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/8.jpg)
88
PREDICTIVE PROBABILITIES
Critical component of experimental design
In monitoring trials
![Page 9: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/9.jpg)
99
Herceptin in neoadjuvant BCHerceptin in neoadjuvant BC Endpoint: tumor response Balanced randomized, H & C Sample size planned: 164 Interim results after n = 34:
Control: 4/16 = 25% Herceptin: 12/18 = 67%
Not unexpected (prior?) Predictive probab of stat sig: 95% DMC stopped the trial ASCO and JCO—reactions …
Endpoint: tumor response Balanced randomized, H & C Sample size planned: 164 Interim results after n = 34:
Control: 4/16 = 25% Herceptin: 12/18 = 67%
Not unexpected (prior?) Predictive probab of stat sig: 95% DMC stopped the trial ASCO and JCO—reactions …
![Page 10: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/10.jpg)
1010
ADAPTIVE DESIGNS: Approach and Methodology
ADAPTIVE DESIGNS: Approach and MethodologyLook at the accumulating dataUpdate probabilitiesFind predictive probabilitiesUse backward inductionSimulate to find false positive
rate and statistical power
Look at the accumulating dataUpdate probabilitiesFind predictive probabilitiesUse backward inductionSimulate to find false positive
rate and statistical power
![Page 11: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/11.jpg)
1111
Adaptive strategiesAdaptive strategies
Stop early (or late!) Futility Success
Change dosesAdd arms (e.g., combos)Drop armsSeamless phases
Stop early (or late!) Futility Success
Change dosesAdd arms (e.g., combos)Drop armsSeamless phases
![Page 12: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/12.jpg)
1212
GoalsGoals
Learn faster: More efficient trials
More efficient drug/device development
Better treatment of patients in clinical trials
Learn faster: More efficient trials
More efficient drug/device development
Better treatment of patients in clinical trials
![Page 13: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/13.jpg)
1313
Troxacitabine (T) in acute myeloid leukemia (AML) combined with cytarabine (A) or idarubicin (I)
Adaptive randomization to:IA vs TA vs TI
Max n = 75 End point: Time to CR (< 50 days)
Troxacitabine (T) in acute myeloid leukemia (AML) combined with cytarabine (A) or idarubicin (I)
Adaptive randomization to:IA vs TA vs TI
Max n = 75 End point: Time to CR (< 50 days)
ADAPTIVE RANDOMIZATIONGiles, et al JCO (2003)
ADAPTIVE RANDOMIZATIONGiles, et al JCO (2003)
![Page 14: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/14.jpg)
1414
Adaptive RandomizationAdaptive Randomization
Assign 1/3 to IA (standard) throughout (until only 2 arms)
Adaptive to TA and TI based on current probability > IA
Results
Assign 1/3 to IA (standard) throughout (until only 2 arms)
Adaptive to TA and TI based on current probability > IA
Results
![Page 15: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/15.jpg)
1515
Patient Prob IA Prob TA Prob TI Arm CR<501 0.33 0.33 0.33 TI not2 0.33 0.34 0.32 IA CR3 0.33 0.35 0.32 TI not4 0.33 0.37 0.30 IA not5 0.33 0.38 0.28 IA not6 0.33 0.39 0.28 IA CR7 0.33 0.39 0.27 IA not8 0.33 0.44 0.23 TI not9 0.33 0.47 0.20 TI not
10 0.33 0.43 0.24 TA CR11 0.33 0.50 0.17 TA not12 0.33 0.50 0.17 TA not13 0.33 0.47 0.20 TA not14 0.33 0.57 0.10 TI not15 0.33 0.57 0.10 TA CR16 0.33 0.56 0.11 IA not17 0.33 0.56 0.11 TA CR
![Page 16: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/16.jpg)
1616
Patient Prob IA Prob TA Prob TI Arm CR<5018 0.33 0.55 0.11 TA not19 0.33 0.54 0.13 TA not20 0.33 0.53 0.14 IA CR21 0.33 0.49 0.18 IA CR22 0.33 0.46 0.21 IA CR23 0.33 0.58 0.09 IA CR24 0.33 0.59 0.07 IA CR25 0.87 0.13 0 IA not26 0.87 0.13 0 TA not27 0.96 0.04 0 TA not28 0.96 0.04 0 IA CR29 0.96 0.04 0 IA not30 0.96 0.04 0 IA CR31 0.96 0.04 0 IA not32 0.96 0.04 0 TA not33 0.96 0.04 0 IA not34 0.96 0.04 0 IA CR
Compare n = 75
DropTI
![Page 17: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/17.jpg)
1717
Summary of resultsSummary of results
CR < 50 days: IA: 10/18 = 56% TA: 3/11 =
27% TI: 0/5 = 0%
Criticisms . . .
CR < 50 days: IA: 10/18 = 56% TA: 3/11 =
27% TI: 0/5 = 0%
Criticisms . . .
![Page 18: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/18.jpg)
1818
Consequences of Bayesian Adaptive Approach
Consequences of Bayesian Adaptive Approach
Fundamental change in way we do medical research
More rapid progressWe’ll get the dose right!Better treatment of patients . . . at less cost
Fundamental change in way we do medical research
More rapid progressWe’ll get the dose right!Better treatment of patients . . . at less cost
![Page 19: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/19.jpg)
1919
CASE STUDY: PHASE III TRIALCASE STUDY: PHASE III TRIAL
Dichotomous endpoint Q = P(pE > pS|data)
Min n = 150; Max n = 600 1:1 randomize 1st 50, then assign
to arm E with probability Q Except that 0.2 ≤ P(assign E) ≤ 0.8
Dichotomous endpoint Q = P(pE > pS|data)
Min n = 150; Max n = 600 1:1 randomize 1st 50, then assign
to arm E with probability Q Except that 0.2 ≤ P(assign E) ≤ 0.8
Small company!
![Page 20: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/20.jpg)
2020
Recommendation to DSMB toRecommendation to DSMB to
Stop for superiority if Q ≥ 0.99
Stop accrual for futility if P(pE – pS < 0.10|data) > PF
PF depends on current n . . .
Stop for superiority if Q ≥ 0.99
Stop accrual for futility if P(pE – pS < 0.10|data) > PF
PF depends on current n . . .
![Page 21: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/21.jpg)
2121
0.0
0.2
0.4
0.6
0.8
1.0
0 100 200 300 400 500 600
n
Futility stopping boundary
0.75
0.95
PF
![Page 22: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/22.jpg)
2222
Common prior density for pE & pS
Common prior density for pE & pS
Independent
Reasonably non-informative
Mean = 0.30
SD = 0.20
Independent
Reasonably non-informative
Mean = 0.30
SD = 0.20
![Page 23: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/23.jpg)
2323
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1
p
Beta(1.275, 2.975)
density
![Page 24: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/24.jpg)
2424
UpdatingUpdating
After 20 patients on each arm
8/20 responses on arm S
12/20 responses on arm E
After 20 patients on each arm
8/20 responses on arm S
12/20 responses on arm E
![Page 25: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/25.jpg)
2525
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1
p
Beta(9.275,
14.975)
Beta(13.275,
10.975)
Q = 0.79
![Page 26: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/26.jpg)
2626
AssumptionsAssumptions
Accrual: 10/month
50-day delay to assess response
Accrual: 10/month
50-day delay to assess response
![Page 27: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/27.jpg)
2727
Need to stratify. But how?Need to stratify. But how?
Suppose probability assign to experimental arm is 30%, with these data . . .
Suppose probability assign to experimental arm is 30%, with these data . . .
![Page 28: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/28.jpg)
2828
Proportions of Patients onExperimental Arm by Strata
Stratum 1Stratum 2
Small Big
Small 6/20 (30%) 10/20 (50%)
Big 6/10 (60%) 2/10 (20%)
Probability of Being Assigned toExperimental Arm for Above Example
Stratum 1Stratum 2
Small Big
Small 37% 24%
Big 19% 44%
![Page 29: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/29.jpg)
2929
One simulation; pS = 0.30, pE = 0.45
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 6 12 18 24 Months
Proportion Exp
Probability Exp is better 178/243
= 73%
FinalStd 12/38 19/60 20/65Exp 38/83 82/167 87/178
Superiority boundary
![Page 30: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/30.jpg)
3030
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.00 6 12 18 24 Months
Proportion Exp
Probability Exp is better
87/155 = 56%
Probability futility
9 mos. End FinalStd 8/39 15/57 18/68Exp 11/42 32/81 22/87
One simulation; pE = pS = 0.30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.00 6 12 18 24 Months
Proportion Exp
Probability Exp is better
87/155 = 56%
Futility boundary
![Page 31: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/31.jpg)
3131
Operating characteristicsOperating characteristics
True ORR Mean # of patients (%)Std Exp
Probselect
exp Std Exp Total
Meanlength(mos)
Probmax n
0.3 0.2 <0.001 95 (62.1) 58 (37.9) 153 15 <0.0010.3 0.3 0.05 87 (43.1) 115 (56.9) 202 20 0.0030.3 0.4 0.59 87 (30.4) 199 (69.6) 286 29 0.050.3 0.45 0.88 79 (30.7) 178 (69.3) 257 26 0.020.3 0.5 0.98 59 (29.5) 141 (70.5) 200 20 0.0030.3 0.6 1.0 47 (30.1) 109 (69.9) 156 16 <0.001
![Page 32: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/32.jpg)
3232
FDA: Why do this? What’s the advantage?
FDA: Why do this? What’s the advantage?
Enthusiasm of patients & investigators
Comparison with standard design . . .
Enthusiasm of patients & investigators
Comparison with standard design . . .
![Page 33: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/33.jpg)
3333
Adaptive vs tailored balanced design w/same false-positive rate & power
(Mean number patients by arm)
Adaptive vs tailored balanced design w/same false-positive rate & power
(Mean number patients by arm)
ORR
Arm
pS = 0.20pE = 0.35
pS = 0.30pE = 0.45
pS = 0.40pE = 0.55
Std Exp Std Exp Std Exp
Adaptive 68 168 79 178 74 180
Balanced 171 171 203 203 216 216
Savings 103 3 124 25 142 36
![Page 34: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/34.jpg)
3434
FDA:FDA:
Use flat priors
Error size to 0.025
Other null hypotheses
We fixed all … & willing to modify as necessary
Use flat priors
Error size to 0.025
Other null hypotheses
We fixed all … & willing to modify as necessary
![Page 35: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/35.jpg)
3535
The rest of the story …The rest of the story …
PIs on board
CRO in place
IRBs approved
FDA nixed!
PIs on board
CRO in place
IRBs approved
FDA nixed!
![Page 36: Bayesian Trial Designs: Drug Case Study Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org](https://reader035.vdocuments.net/reader035/viewer/2022062417/5514bc79550346b0338b4671/html5/thumbnails/36.jpg)
3636
OutlineOutline
Some history
Why Bayes?
Adaptive designs
Case study
Some history
Why Bayes?
Adaptive designs
Case study