Challenges of Non-Inferiority Trial Designs

R. Sridhara, Ph.D.

12/6/06 Pediatric Advisory Committe 2

Terminology Superiority = Drug T better than Placebo (P) or

T better than Active Control (C) C may or may not have known efficacy

Non-inferiority (NI) = T not much less effective than C ‘Was Not Different’ or ‘Similar’

C must have known efficacy

12/6/06 Pediatric Advisory Committe 3

Clinical Trials to Demonstrate Efficacy

Superiority Claim First choice Direct evidence

Non-inferiority Claim Last choice Indirect evidence Interpretation could be misleading

12/6/06 Pediatric Advisory Committe 4

Non-inferiority Claim Implies:

1. The effect of T and C are close2. T and C may be both equally

beneficial orT and C may be both equally not beneficial

12/6/06 Pediatric Advisory Committe 5

Assumptions for NI consideration

C has an effect (compared to placebo, P) Therefore, C can not be another

experimental therapy in NI trial Can reliably estimate C effect size * C effect size compared to P Control (C) effect in the future

study population will be same as in the historical population.

12/6/06 Pediatric Advisory Committe 6

Non-inferiority Trial Design Considerations

Endpoint - Overall Survival - Progression-free Survival - Response Rate

Control Effect

% Retention

12/6/06 Pediatric Advisory Committe 7

Non-inferiority or Superiority

Challenges with Non-inferiority Trial How well we know the effect of the

control How much of the control effect can we

afford to give up At least ‘’% retention of the control

effect necessary to make sure that the new treatment is significantly better than placebo

12/6/06 Pediatric Advisory Committe 8

Superiority trial designs with OS as the primary endpoint

Null Hypothesis, H0: HR(T/C) = 1 vs.

Alternative, H1: HR (T/C) < 1 if T is better than C

12/6/06 Pediatric Advisory Committe 9

Non-inferiority trial designs with OS as the primary endpoint

H0: HR(T/C) ≥ M vs. H1: HR (T/C) = 1 if the two are expected to be

similar, Or H1: HR(T/C) = 0.95, for example, if the new

treatment (T) is expected to be slightly betterM is the margin determined based on the active

control (C) effect size estimated from historical trials and percentage of the effect to be retained

HR > 1 implies, T is worse than C

12/6/06 Pediatric Advisory Committe 10

Percent Retention and Estimated Active Control Effect Size

Non-inferiority ≈ “not much less effective” ‘X’ is the effect size of the active control;

example: Point estimate of HR (C/P) = 0.5 (or HR(P/C) = 2.0) implies an estimate of active control effect size was a 50% reduction in the risk of event (ex: death)

Percent retention is a percentage of ‘X’ that is retained; example: a 50% retention of the 50% effect size is 25% effect size, i.e., the putative HR(T/P) = 0.75 (or HR(P/T) = 1.33)

12/6/06 Pediatric Advisory Committe 11

Estimates of True Control Effect

Lower 95% C.L.

( << 0.025)

Lower % C.L.

( = 0.025)

Point Estimate

( >> 0.025)

12/6/06 Pediatric Advisory Committe 12

Methods for estimating active control effect size: No method is ideal and no particular method is endorsed by the Agency

All methods assume that the control effect has not changed over time

Some methods do not consider the variation between and within studies

Other methods incorporate variation between and within studies

12/6/06 Pediatric Advisory Committe 13

Hypothetical Example of a NI Trial with Overall Survival Endpoint

Suppose HR(P/C) = 2.0 (95% CI:

1.9, 2.1)

Then, Lower 70% Confidence limit = 1.97

12/6/06 Pediatric Advisory Committe 14

Hypothetical example: Estimates of True Control Effect

Lower 95% C.L. of HR = 1.9

Lower 70% C.L. of HR: 1.97

Point Estimate

HR(P/C) = 2.0

12/6/06 Pediatric Advisory Committe 15

Hypothetical Example of a NI Trial with Overall Survival Endpoint

Suppose we want to retain 50% of 1.97, i.e., HR(P/T) = 1.49, and we can accrue 100 patients per unit time

If H1: HR(T/C) = 1, then N = 407 events or 1045 patients will be required

If H1: HR(T/C) = 0.95, then N = 290 events or 836 patients

12/6/06 Pediatric Advisory Committe 16

Sample Size Depends On:

How good the historical data is: One randomized study – large

confidence interval – more uncertainty about the effect size

Many randomized studies – smaller confidence interval – unless there were fundamental differences in the conduct of the studies: population, dose, etc.

12/6/06 Pediatric Advisory Committe 17

Summary - 1

Superiority trials provide direct evidence New drug can be compared to placebo or

control (established or not) Non-inferiority trials provide indirect

evidence New drug must be compared to

established control Interpretation can be misleading

12/6/06 Pediatric Advisory Committe 18

Summary - 2

For NI trial consideration, active control effect must be well characterized Able to estimate effect size Control effect is same now as before

NI trials are generally large Sample sizes for a NI trial is dependent on

the estimate of control effect, population, % retention and the alternative hypothesis

12/6/06 Pediatric Advisory Committe 19

Summary - 3

In considering NI trial, any potential loss of efficacy must be weighed against risk-benefit ratio

Failed superiority does not imply non-inferiority