issues that plague non- inferiority trials past and future ralph b. dagostino, sr. boston university...

ISSUES THAT PLAGUE NON-INFERIORITY TRIALS

PAST AND FUTURE

RALPH B. D’AGOSTINO, SR.

BOSTON UNIVERSITY

HARVARD CLINICAL RESEARCH INSTITUTE

OBJECTIVES

1. REVIEW ISSUES: PAST, PRESENT AND FUTURE IN NON-INFERIORITY (NI) STUDIES

2. PRESENT/ DISCUSS EXAMPLES

3. MAKE SOME COMMENTS FOR IMPROVEMENTS

4. PRESENT A PERSONAL VIEW

OUTLINE

1. Early Objectives and Issues

2. Approaches to Non-inferiority Trials

3. Examples (Here are some Problems)

4. Non-Inferiority AND/OR Superiority

5. All is Non-Inferiority

6. Intent-to-Treat vs. Per Protocol

7. New Major Issues

EARLY OBJECTIVES AND ISSUES: EQUIVALENCY

• American Dental Association (ADA 1980s)• CREST equivalent to COLGATE?• Ho: A-B>= 10% or A-B<= 10%• What does the 10% mean?

– DFMS or DFMT for 2 years, 3 years?

• Study done on differences and ratio used as descriptive measure of “effect”– 5.0 vs 5.4 becomes (5.4-5.0)/5.0 = .4/5.= 8%

EARLY OBJECTIVES

• M = 10% CAME FROM NOWHERE, BUT WE KNEW WHAT IT WAS, That is, 10%

• TREATMENT DIFFERENCES CONCERNED DIFFERENCES (RATIOS) BETWEEN ACTIVE TREATMENTS

• WE WERE LOST BUT WE BELIEVED WE HAD A “SENSE” ABOUT IT

APPROACHES TO NI TESTS

• MUST DO BETTER THAN PLACEBO

• But you cannot use a Placebo (P)

1. Putative Placebo Approach

2. Test Treatment (T) vs Positive Control (C) directly with given Margin M (Assay Sensitivity approach)

APPROACH 1 (Putative Placebo) Stellar Example from the Past

• CAPRIE Study. Hasselblad and Kong (2001) present this as their major example for using meta-analyses for dealing with estimating assay sensitivity (T vs. P)

• Want T vs. C, C vs. P, T vs. P

21

CAPRIE STUDY (cont)

• Can we obtain effect of Clopidogrel vs. Aspirin

• Yes, if we can locate Asprin vs. Placebo

• Do we believe what we get?

For Aspirin vs. PlaceboAntiplatelet Trialists’ Collaboration

Meta-Analysis• Meta-analysis of all published and

unpublished unconfounded randomized trials available March 1990

• Trials identified by literature search, trial registry and inquiry of investigators and pharmaceutical manufacturers

• Clear definitions of endpoints

• Well defined statistical methodology

APPROACH

• T vs. C (from Caprie trial)

• C vs. P (from Meta-analysis)

• Obtain T vs. P (from multiplication)

• (T/C) (C/P) = (T/P)

Clopidogrel Vs. Synthetic Placebo Control Odds Ratios and 95% Confidence Intervals Overall Patient Population

Endpoint

All Strokes, MIs,or Vascular Deaths p < 0.000001

All Strokes, MIsor Death from p < 0.000001Any Cause

Vascular p < 0.0016Deaths

All Cause p < 0.0045Deaths 0.4 0.6 0.8 1.0 1.2 1.4 1.6 First Drug Better Second Drug Better

CAPRIE: Clopidogrel Vs. Aspirin

Meta-Analysis: Aspirin Vs. Placebo

Estimated: Clopidogrel Vs. Placebo

• Meta-analysis studies contain very old studies (only up to 1990), many prior to all of the elaborate medical interventions (procedures) now routinely provided

• Many of the studies did not have MI, IS or vascular death as their outcomes (the meta-analysis went back to original investigators who in turn, had to generate data). Ever try to get data on something you did not collect?

• None of the studies used for Clopidogrel with aspirin comparison had PAD as an entry criteria (PAD represented 1/3 of Clopidogrel Study)

EFFECT SIZE: Relative Risk Reduction by Qualifying Condition (ASA vs Clopidogrel)

IS n = 6431

MI n = 6302

PAD n = 6452

Total n =19185

30 20 10 0 10 20

Clopidogrel Better Aspirin Better

Problems With Historical Controls

• Biases– Time Biases

• Change in recognition or diagnosis of disease

• Changing disease process

• Change in usual therapy

(Myocardial Infarctions MI, Dx, Tx)

– Selection Biases• Patients/Health care systems

• Are we really seeing the same patients in historical studies as those in active control trial?

Problems With Meta-AnalysesSo What Is Sponsor to DO?

If we plan to use placebo controlled trials, what should we require of the historical placebo trials?

• Same Disease/Conditions?• Same Population• Same Dose and Administration Levels of

Active Control C? • Same Outcomes? • Combine “All” or “Some (good)” Placebo

Controlled Studies

Still Other Problems With Meta-Analyses

• What if previous studies had multiple arms? How to put correctly into meta-analysis?

• What if none of the individual studies achieved significance?

• What are we to believe from meta-analyses?• Do we believe the p-levels of the meta-analysis? (I

do not think we should.)

APPROACH 2NON-INFERIORITY STUDIESACTIVE CONTROL STUDIES

NON-INFERIORITY TEST H0: T-C >= M vs. H1: T-C < M

(Say data are event rates)

T is new treatment

C is positive control

M IS NON-INFERIORITY MARGIN

NON-INFERIORITY STUDIES

APPROACH 2

• SELECT A VALUE OF M THAT MAKES SENSE

• WANT ASSURANCE THAT ASSAY SENSITIVITY IS PRESENT (Placebo is working)

• WANT CONSISTENCY WITH PAST

NON-INFERIORITY STUDIES Statistical Approach

1. Need Active Control C vs. Placebo P data from Historical data (C vs. P)

2. Need to test effectiveness of T vs. C3. Need estimate of fraction of C-P preserved by T

(e.g., (T-P)/(C-P) = M) M=0.5 (C-P)METHODS EXIST THAT ALLOW TEST TO

BUILD IN NEW AND HISTORICAL DATA (STATISTICS IN MEDICINE, 2002)

WHAT IS NEEDED FOR 2

• CONFIDENCE INTERVAL IS OFTEN USED. WANT M=1.11 (SAY) OUTSIDE UPPER LIMIT OF CONFIDENCE INTERVAL (M is relative risk)

• FDA ODAC 8/04 (non-small cell lung cancer)

1.0 1.11= M

SOME REALITIES

• Sounds nice

• What happens

Anti-infective Product No placebo data

• Historical data is not Placebo, but C

• VRE (vancomycin resistant enterococcal)

High dose Low dose

• MITT 60.0 % (N=65) vs. 46.2 % (N=52)

• Bacteremic

55.6 (N=18) vs. 25.0 (N=16)

• What is M? One trial OK? Any superiority?

ANOTHER EXAMPLERespiratory Distress

• Respiratory Distress Syndrome in Premature Infants– Treatments

• New Drug

• Comparator

– Outcome• Survival at 28 Day

Respiratory Distress (cont)

• Survanta versus Sham (two studies one positive, other negative) All Cause mortality

• Study 1: 8% vs. 23% Study2: 17% vs. 14%

• What is M? .23-.08? .180-.125?

CONSISTENCY Example Control rate different

from historical • Historical Data says C=0.5 and P=0.6

• Want T<=0.55

• P-C=0.10, M=0.5(0.10) = 0.05

• (T-C)/C = 0.05/0.50 = 10%

• Data is C=0.30 and T=0.33, T-C=0.03

• (T-C)/C = 0.03/0.30 = 10%

• IS STUDY A SUCCESS? USE RATIOS?

ANSWER TO CONSISTENCY

• There was consistency

• Differences related to birth weight

Non-Inferiority and Inferiority at the same time

• Sponsor falls apart

00 MM

Non-Inferiority and Superiority

• Sponsor jumps for joy (Sequential test)

00 MM

Switching trial design (Cardiac Stent Trials)

• (1) New drug coated stents, we can do non-inferiority study with margin set (15%)

• (2) We can do superiority study with non-coated stent as control

• With first option we have to worry about evaluating Ms, Effect size and CREEP

• With superiority trial “clean” results

Respiratory Distress

• Compare new surfaxin to another “not so great” one, but still used in practice

Switching from Superiority to non-Inferiority

• HOW CAN WE SWITCH FROM A SUPERIORITY TEST TO NON-INFERIORITY ?

• This is a question thrown at me constantly

Assessing Efficacy Non-Inferiority and Safety Superiority• Carotid artery Magnetic Resonance Imaging agent• Imaging Agents

– Agent N (New) Agent C (Comparator)

• Non-inferiority” Outcome– Endpoint: agent’s ability to classify correctly patients

with > 25% stenosis (sensitivity)

– Sensitivity of Comparator is .80 or 80%

– Non-inferiority margin M set to 0.10

Assessing Efficacy Non-Inferiority and Safety Superiority (Cont’d)

• There is a specific adverse event that is hypothesized to occur less often with New than with Comparator– Do we want to make the specific adverse event

rate an additional primary endpoint? WHY NOT?

Non US STUDIES

• Forced off shore (ethical and other reasons)

The BLOB EFFECT

• Everything is suddenly Non-Inferiority

ALLHAT STUDY

• COMPARISON OF ANTI-HYPERTENSIVE MEDICATIONS (MULTIPLE ARMS)

• NOT A NON-INFERIORITY STUDY

Safety Studies

• Safety studies have become carefully designed and executed studies

• Should they be non-inferiority studies?

SAFETY STUDIES (PHASE 4)

HISTORICAL APPROACH: NEW RATE > OLDH01: T-C <= 0 vs. H11: T-C > 0

H02: RR=T/C <= 1 vs. H12: RR=T/C > 1

STUDY POWERED TO REJECT T/C >1.5 (SAY)

SHIFT IS TO MAKING THESE NON-INFERIORITY STUDIES

• H0: T-C >= M vs. H1: T-C < MH0: RR=T/C >= M vs. H1: RR=T/C < M

Safety Studies

• OLD

• NEW11

MM

SAFETY STUDY TO NON-INFERIORITY STUDY

(QT LONGATION)• Safety issue: drug may cause QT problem• Ho: A/B = 1.0 vs H1: R = A/B > 1.0• Study powered for R > 1.0• When interest in risk fades can we suddenly

say this should be a non-inferiority study?• Ho:R >= 1.5 vs. H1:R < 1.5 was not

original objective• If we do not reject Ho is that enough?

Form of Interest and Sample Size

• Ho: p1-p2 >= M

• Ho: p1-p2>=Rp2

• Ho: p1/p2 >= R

• Best Choice does depend on p2 (control rates)

Intent-to-Treat vs. Per-Protocol• In superiority trials, the primary analysis is

often on intent-to-treat (ITT) population

• Per Protocol (PP) “bigger” differences of treatments

• In non-inferiority should we use PP?

Intent-to-Treat vs. Per-Protocol (Cont’d)

• PP as primary not always accepted– “the ITT analysis is as important as the PP analysis”

– “need to reconcile differences between ITT and PP analysis”

– Perform “sensitivity” analyses. Results should be similar in both populations (ROBUSTNESS).

– The Committee on Proprietary Medicinal Products draft Points to Consider: “…similar conclusions from both the ITT and PP are required in a noninferiority trial”.

• We ask sponsor to do both (ITT and PP) and expect to achiev the sam significant result on both.

• What is the true alpha associated with this?

NEW MAJOR ISSUES

• Missing Data

• Noncompliance

• Interim Analysis

• OUR USUAL LOGIC INCREASES CHANCE OF ACCEPTANCE OF non-inferiority

MORE NEW ISSUES

• Multiple endpoints

• Multiple groups

• Repeated Measures

WHERE ARE WE?

• NON-INFERIORITY TRIALS HAVE MADE A BIG IMPACT

• They have brought many new problems and challenges with them