adapting for success: the genesis of adaptive designs
DESCRIPTION
Adapting for Success: The Genesis of Adaptive Designs. Andy Grieve SVP Clinical Trials Methodology, Innovation Centre, Aptiv Solutions. Outline. Basic Principles of Adaptive Designs Why adaptive trials? Differences Between Early / Late Phase Adaptive Designs - PowerPoint PPT PresentationTRANSCRIPT
Adapting for Success: The Genesis of Adaptive Designs
Andy GrieveSVP Clinical Trials Methodology, Innovation Centre, Aptiv Solutions.
• 1
Outline
• Basic Principles of Adaptive Designs
• Why adaptive trials?
• Differences Between Early / Late Phase Adaptive
Designs
• Categories, benefits and regulatory feedback
• Practical Issues
2
Adaptive Ideas Are Not New
Biometrika, 1933
Ethical Design – concentrating on delivering the best treatment to the most patients Thall and Wathen (Eur J Cancer, 2007)
3
4
Basic Principles of Adaptive Designs
Terminology
Flexible
Multi-Stage
Response-Driven
Dynamic
Sequential
Self-Designing
Bayesian
Adap
tive
D
esig
ns
5
Adaptive Design: Definition
An Adaptive Trial uses accumulating data to decide how to modify aspects of the study without undermining the validity and integrity of the trial. (PhRMA)
Validity4 providing correct statistical
inference:4 adjusted p-values, estimates,
confidence intervals4 providing convincing results to a
broader scientific community4 minimizing statistical bias
Integrity Pre-planning based on intended
adaptations
maintaining confidentiality of data
assuring consistency between different stages of the study
minimizing operational bias
6
General Structure – Dragalin (DIJ, 2006)
• An adaptive design requires the trial to be conducted in several stages with access to the accumulated data
• An adaptive design may have one or more rules:
– Allocation Rule: how subjects will be allocated to available arms– Sampling Rule: how many subjects will be sampled at next stage– Stopping Rule: when to stop the trial (for efficacy, harm, futility)– Decision Rule: the terminal decision rule and interim decisions pertaining to
design change not covered by the previous three rules
• At any stage the following stages can be redesigned taking account of all available data
7
Aspects of the Study Modifiable
• Number of Subjects• Study Duration• Endpoint Selection• Treatment Duration• Patient Population• Number of Treatments• Number of Interim Analyses• Hypotheses
8
The Role Of Adaptive Trials in the Drug Development Process
NewProduct
Standard Development Process
Phase 1 Phase 2 Phase 3
Adaptive Development Process
Option to:• Verify trial assumptions• Explore additional doses• Stop for futility early
Option to:• Select best dose• Submit application early• Stop for futility
9
Long periods of information “blackout”No opportunity to:
- verify trial assumptions- adjust dosing- make minor adjustments to trial design- stop for futility
Types of Adaptive Design: Learn
• Single ascending dose escalation designs • Up-and-Down and CRM to find MTD• Establish Proof-of-Mechanism or Proof-of-Target Modulation
First-in Human
• Two-stage adaptive approach in patients• 1st stage – to identify MTD• 2nd stage – to select dose and exposure levels (necessary cond.)
MAD and PoC
• Start with the highest feasible tolerated dose and placebo• If a pre-specified futility condition is satisfied => stop • Otherwise, open enrollment to lower doses
PoC and ADRS
• SAD or MAD combined with Biomarker-based Efficacy• To identify the Optimal Safe Dose
Seamless Phase I/II Design
• Finding a target dose (MED, EDp)• Response Adaptive Allocation• Covariate Adjusted Response Adaptive Allocation
Adaptive Dose Ranging Design
CRM: Continual Reassessment Method; MTD: Maximum Tolerated Dose; MAD: Multiple Ascending Dose; SAD: Single Ascending Dose; MED: Minimum Effective Dose; EDp: Dose achieving 100p% of maximum effect
10
Types of Adaptive Design: Confirm
• Sample size adjustment based on blinded or unblinded data:• Using nuisance parameter estimate• Using treatment effect estimate
Sample Size Reassessment
• Early stopping for efficacy, futility, harm or safety• Adjusting the number and/or timing of interim analyses• Increasing the maximum sample size
Adaptive Group Sequential Design
• Design combining the objectives of Phase II dose ranging study and confirmatory Phase III trial in a single protocol
• Dose selection at the interim analysis
Seamless Phase II/III Design
• Placebo run-in; Active control run-in; Dose titration• Adaptively enrich the population at the interim analysis• Enrich based on biomarker or clinical endpoint response
Population Enrichment Design
• Marker by Treatment Design• Targeted Design• Marker x TRT Design with Response adaptive allocation within strata
Drugs with Companion Diagnostics
11
Failure in Pharmaceutical Drug Development
Infectious D
iseases
Endocrine
AutoImmune
Respiratory
Neurology CV
Oncology
0%20%40%60%80%
Phase II Phase III
Therapeutic Area
Succ
ess R
ate
12
Arthriti
s & Pain CV
CNS
Infectious D
iseases
Oncology
Opthalmology
Metabolic Dise
ases
Urology
Women's Health
0%20%40%60%80%
Phase II Phase III
Therapeutic Area
Succ
ess R
ate
Kola and Landis (2004)Nature Reviews Drug Discovery
Biomed Tracker February 2011
Why are Adaptive Clinical Trials Essential for Development of New Products?
• Pharmaceutical industry facing a major pipeline challenge
– Fewer approvals, escalating development costs, tougher regulatory environment, re-imbursement hurdles, expiring patents for existing blockbusters
• Failure rate in Phase III estimated at 50%
• Traditional development paradigm is not sustainable
• Innovative designs are key priorities for improving R&D productivity and increasing the probability of success at Phase III.
13
15
The Need for Sample Size Re-Estimation in Late Phase Studies
Impact of key parameters on trial success
Delta: the treatment effect – measures the difference between drug group and placebo groupSigma: the standard deviation - measures the dispersion of patient responses around the meanPower (90%): probability of successError (5%): probability of success although no effect (aka type-1 error or false positive)Sample size needed, depends on all the above
16
SSSS SS
observed
design
Impact of Incorrect Assumptions: treatment effect and standard deviation
3.0
2.0
1.0
0.03.02.01.00.0
+ Primary Endpoint met - Primary Endpoint not met
Good news
observed
design
Data from 39 studies (phase 2 and 3) performed in a 2 year period
correct assumption
17
Uncertainties and Adaptive Insurance Solutions
• Uncertainty about treatment effect or variability of dataEarly stopping for futilitySample size re-estimation
• Uncertainty about dose arm to take forwardDose response adaptiveDose selection (seamless 2/3)
• Uncertainty about (sub)populationPopulation enrichment
• Drug simply doesn’t work
Early stopping for futility
18
Regulatory Environment
• EMA released (2007) the “Reflection paper” on adaptive designs in confirmatory trials– The emphasis is on control of Type I error rate– Consistency of treatment effect before and after adaptation
• PMDA started adaptive discussions and workshops with PhRMA since 2007– Wants to discuss pros and cons of adaptive designs– In summary (*), they will consider adaptive trials when there is a
compelling medical need unanswered otherwise.• Oncology (group sequential trials are more common)• Orphan drugs, rare disease indications• Severity of disease or difficulty of trial conduct• CNS with uncertainty about subjective endpoints and past trial results
(*) Ando et al. (2011). Adaptive Clinical Trials for new drug applications in Japan, European Neuropsycopharmacology,
19
Categories and Benefits
Adaptive Design adaptation Benefits FDA feedback
Phase 1 CRM*Dose escalation
Choice of Next Dose Better selection of MTD, less safety/efficacy issues
Phase 2 Adaptive dose finding
Change of randomization fraction
Better dose selection, less phase 3 failure, more $value at end of phase 2 Combine objectives of 2 trials in 1
Sample size re-estimation + Futility(Un-) Blinded
Increase sample sizeEarly stopping
Save underpowered trials, less phase 3 failure, verify trial assumptions and correct = buy an insurance policy, re-allocate resources faster if futile
Phase 2/3 combined
Drop dose Save time and patients, select the optimal dose
Population enrichment in phase 3
Change inclusion criteria, focus on subgroup
Win on the responder subgroup if drug not efficacious on the whole population
FDA
FDA
FDA
*CRM: Continual Re-assessment Method
FDAEMA
CbC
CbC
*CbC: Case by Case
CbC
CbC
20
21
Phase I Dose-Escalation Designs in Oncology
22
The Background (Oncology)
• Given several doses of a new compound, determine an acceptable dose for treating patients in future trials
• Assumptions– Definition of Dose Limiting Toxicity (DLT)– Definition of Maximum Tolerated Dose (MTD)
• Prob ( DLT | MTD) = *– Prob (Response) with dose A)– Prob (Toxicity) with dose B)
• These conflict : A) is good; B) is bad
Adaptive Dose-Escalation in Phase I Studies
Algorithmic Model-Based
Non-Bayesian
3+3 (Storer, 1989)• No dose-response assumptions
needed.• No explicit control of the target MTD-
rate.
Bayesian
mTPI (Ji et al, 2013)• Heuristic approach for determining a
dose with acceptable toxicity close to the MTD
• Overdose-control based on posterior probabilities of toxicity ranges
• Bayesian methodology for determining acceptance of dose levels
CRM (O’Quigley et al, 1990)• Dose selection based on the posterior
distribution of the dose-response.• No explicit over-dose controlBLRM (Neuenschwander et al (2008)• Dose selection is based on a two-
parameteric logistic model.• Overdose-control based on posterior
probabilities of toxicity ranges
23
Enter 3 patients
< 1/3 DLTs 1 /3 and < 2/3 DLTs > 2/3 DLTs
Add 3 patients
Escalate to Dose Level i + 1 Dose Level (i-1) is MTD
2/6 DLTs< 2/6 DLTs
Dose Level i
The 3+3 design has been used in approximately 95% of the published phase I oncology trials over the last two decades (Tourneau, 2009)
Schematic of a 3+3 Design
Candidate Dose Response Shapes
• tendency of the 3+3 designs to underestimate the target dose
• increases the chance of failure in terms of efficacy later
• Other characteristics - # number of patients with DLT’s, # patients dosed above the MTD …..
Probabilities of Selecting Appropriate Doses
Underdosing Target toxicity Overdosing0
20
40
60
80
100
MTD-selection - Scenario 1
BLOC mTPI CRM 3+3
Underdosing Target toxicity Overdosing0
20
40
60
80
MTD-selection - Scenario 2
BLOC mTPI CRM 3+3
Underdosing Target toxicity Overdosing0
20
40
60
80
100
MTD-selection - Scenario 3
BLOC mTPI CRM 3+3
27
Current MRC Developmental Pathway Funding Scheme (DPFS) Statisticians
“Good Cop” “Bad Cop”
It would be a good idea if the applicants considered a model-based approach
The applicants should use a model-based approach
28
Phase II / III Seamless Adaptive Design in Kidney Transplant Patients : Orphan Condition
Phase 2b Dose Selection Design Circa 1993
■ More Efficient● wider range of doses, smaller numbers of patients per group● followed by one large parallel group study focusing on the doses
showing promise in exploratory study.
2.5 mgs 10 mgsPlacebo 80mgs 120 mgs 160 mgsDose40 mgs
Redu
ction
from
Bas
elin
e25
20
15
10
5
0
29
0 0.5 1 1.5Dose
Clinician 1
Clinician 2
Clinician 3
Designing a Dose-Response Trial
31
Phase 2b Dose Response/Finding/Selection Designs2005-2010 / 2011-2013
2 3 4 5 >60
10
20
30
40
50
60
Number of Active Doses
Perc
enta
ge o
f Stu
dies
2005-2010
2011-2013
Seamless Phase II/III Designs
• Seamless design– A clinical trial design which combines into a single trial objectives which are
traditionally addressed in separate trials (operationally seamless)
• Adaptive Seamless design – A seamless trial in which the final analysis will use data from patients
enrolled before and after the adaptation(inferentially seamless)
• Primary objective – combine “dose selection” and “confirmation” into a single trial
• Key Benefits: Efficiency; faster and more informed decision-making• Key Challenges: Effective and Efficient Implementation
32
Efficiency of Adaptive Seamless Phase II/III Designs
33
Development Timeline
Inferentially Seamless Phase II/III trials
Dose B
Dose A
Placebo
Dose B
PlaceboPhase II
Operationally Seamless Phase II/III trial
Dose B
Dose A
Placebo
Dose B
PlaceboPhase II
Confirmatory AnalysisConfirmatory Analysis
Confirmatory Analysis
Interim analysis: Trigger for phase III
Dose C
Confirmatory Analysis
Dose C
Separate Phase II and phase III trials
Dose B
Dose A
Placebo
Dose B
PlaceboPhase II End of Phase III
Dose C
Confirmatory AnalysisI.
II.
III.
33
33
Phase II/III Seamless Design – checklist for feasibility
• The hypotheses are pre-defined and will not change• There is positive data from proof-of-concept studies• The remaining uncertainty primarily concerns dose• The primary endpoint for confirmation is pre-specified and will
be measured on all patients• Patient population will stay the same in both phases• The marketing formulation is available• There is sufficient animal data to allow longer drug exposure:
Phase II decision may be based on a biomarker believed to be predictive of the clinical endpoint for confirmation
34
Example: Seamless Phase II/III study in an Orphan Condition (on-going)
• Two-stage group sequential design with O’Brien & Fleming boundaries
• Dunnett intersection test (dose selection methodology)• Three doses of a drug with pre-specified effect sizes• Primary endpoint: Short-term response (0 or 1) (7 days)
Assumptions • One-sided type I error 0.025• Power 80%• Placebo rate 45% (to be reduced by treatment)
35
Comparison
• An inferentially seamless adaptive Phase II/III design with stopping after stage 1 only for efficacy using an O’Brien and Fleming boundary. The interim is assumed to be conducted at an information rate of 1/3rd
• An operationally seamless Phase II/III study with the same decision rule, and sample size (in stage 1), as the inferentially seamless design followed by a phase III study powered at 80%
36
0 200 400 600 800 1000 1200 14000
200
400
600
800
1000
1200
1400
ASN Inferentially Seamless Phase II/III Design
ASN
Ope
ratio
nally
Sea
mle
ss P
hase
II/II
I De
sign
Seamless Phase II/IIIIncreased Sample Size for Operationally Seamless Design
30% increase
37
Seamless Phase II/III study in an Orphan Condition
38
• Inferentially seamless design chosen• Design approved by EMA and FDA• Other case studies have shown similar savings• Orphan status would have been a potential
reason for PMDA to accept the design
39
Practical Issues in Execution of Adaptive Designs
Endpoints in Adaptive Trials
Indication Endpoint
Migraine Pain Control 2 hours
Kidney Transplantation Delayed Graft Function 7 days
Neuropathic Pain Pain Score 6 weeks
Acute Stroke Neurological Score 13 weeks
Oncology Months / Years
Delayed Response in Group Sequential TestsLisa Hampson & Chris Jennison (JRSSB, 2013)
• Develop designs which maximise the use of the pipeline data to increase the test’s power
Pipeline Data
Modelling Approach Basic Principle – Ian Marschner
enrolment times
parti
cipa
nt
start end of accrual interim analysis end of study
first measurement second measurement third measurement
6
5
4
3
2
1
Link to Missing Data
• Little & Rubin (2002) – “Analysis of Missing Data”– the concept of “monotone missing data”– for K measurements Y1,Y2,…,YK
– Yj+1,Yj+2,…,YK are missing when Yj is missing
• The advantage of this structure is that the likelihood can be partitioned allowing simplified estimation
“Simple” Illustration of Efficiency Gains
• Assumptions– Normally distributed data, – Two treatments– n patients per treatment have data on short-term and long-term
endpoints– m patients per treatment have data on short-term endpoint only– Variances equal and known, correlation known
“Simple” Illustration of Efficiency Gains
• Assumptions– Normally distributed data, – Two treatments– n patients per treatment have data on short-term and long-term
endpoints– m patients per treatment have data on short-term endpoint only– Variances equal and known, correlation known
Nyy
~.12
.11
m
Nx2
11.1 ,~
m
Nx2
21.2 ,~
1
1,~
2
12
11
.12
.11
nN
yy
1
1,~
2
22
21
.22
.21
nN
yy
Short-term
Long-term
Short-term
Long-term
Treatment 1
Complete data :
Short-term only:
Treatment 2Complete data :
Short-term only:
“Simple” Illustration of Efficiency Gains
• Interested in estimating long-term treatment effect : 12 - 22
nV
yy2
22121
.22.122212
2)ˆˆ(
ˆˆ
Using only long-term data
)(22)ˆˆ(
ˆˆ
222
22122
.21.1121.22.122212
mnnm
nV
yyxxmn
myy
Using all data (long-term and short-term)
cf. Galbraith & MarschnerSection 2.3 generalized to k measurements
Efficiency = 100V1/ V2 ; m=an
a
100
150
200
250
300
350
400
0 0.2 0.4 0.6 0.8 1
Effici
ency
(%)
108
6
4
2
1
½¼
Data from Copenhagen Stroke Database (Jorgensen et al, 1995):Mean Response by Length of stay in the Acute Stroke Unit (0-12 Weeks)
12 1212
12 12 12 12 12 12 12 12 12 12
11
1111 11 11 11 11 11 11 11 11 11
1010
1010 10 10 10 10 10 10 10
99
99 9 9
9 9 9 9
8 88
88 8 8 8
01
1
2
22
3
3 3 3
7
7 77 7 7 7 7
6
66
66 6 6
4
44 4 4
5
55 5
5 5
8
25
30
35
40
45
50
55
60
0 2 4 6 8 10 12 14
Week (Post Stroke)
Mea
n S
cand
inav
ian
Stro
ke S
core