non-inferiority - miceapps · primary (latent) tuberculosis standard tmt = 9 months isoniazid...

1

Non-inferiority RCTs

..& other ‘new breeds’ of RCTs

Non-inferiority RCTs

..& other ‘new breeds’ of RCTs

Craig Mellis2019

Learning Objectives

1. Recognise the ‘new’ RCT designs

2. Appreciate the roles of ‘new’ RCTs

3. How to appraise a Non-inferiority RCT

Controversy…-> Parachutes & Evidence Based Medicine

“..effectiveness parachutes not proved with RCTs..”

“..did not find any RCTs.”

Superiority, parallel group RCT

Superiority, parallel group RCT

Audience participation -> ‘Needs’ assessment

Q..How familiar are you with the following …

… “________” (type of) RCT..?

3 possible answers..

1. I’ve never heard of it

How familiar are you with the following …

… “________” (type of) RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

How familiar are you with the following …

… “________” (type of) RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding - & could explain to others

Q1. How familiar are you with…

… “Cluster” RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding & could explain to others

METHODS:

Design: Cluster, randomized, controlled, open trialPopulation:

n = 911 children aged 0 to 3 yearsN = 24 Day Care Centres

Intervention: Hand washing & education

Comparator:Usual care (Control)

Outcome: Rate of respiratory tract infections; F/Up= 8 months

Cluster RCT

ie, Randomisation of ‘GROUPS’eg, Schools, DCCs, Hospitals, ICUs.

Aim: Practical, & reduce ‘contamination’

But.. Loss of stats powerRequires ‘cluster analysis’

Q2. How familiar are you with…

… “N-of-1” RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding & could explain to others

“N-of-1” RCT

Repeated Randomised

Crossover trials

in

INDIVIDUAL patients

Intervention

Placebo (control)

Not new!.. NEJM, 1986, Guyatt & Sackett et al

Which patients are SUITABLE for N-of-1 RCT..?

Indications:

*Treatment -> rapid response & minimal carryover effect

*Chronic / frequently recurring symptoms..

eg, Back pain; Cough; Wheeze; Insomnia etc

* Patient / Clinician -> Highly motivated..!

Which patients are UNSUITABLE for N-of-1 RCT..?

Contraindications:

*Treatment -> slow onset / substantial carryover/ ‘cure’

*Rapidly progressive condition

* Patient / Clinician -> Not motivated..!

88 pageGuide..!

“N-of-1” RCT

Heirarchy of Evidence: Therapy

Q3. How familiar are you with…

… “Proof-of-Concept” RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding & could explain to others

Q4. How familiar are you with …

… “Pragmatic” RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding & could explain to others

continuum

‘Pragmatic’ RCTrials ‘Proof of Concept’ RCTs

Spectrum of RCT designs

continu

Pragmatic RCTrials ‘Proof of Concept’ RCTs

‘Real world’

High External Validity

Simple, practical design

Diverse settings

‘Explanatory’ RCTs

High Internal Validity

Sophisticated design

Controlled environment

Narrow range patients

Range of RCT designs

continuum

‘Pragmatic’ RCTrials ‘Proof of Concept’ RCTs

‘Real world’ RCT

High External Validity

Simple, practical design

Diverse settings (GP/eHR)

Broader range patients

‘Explanatory’ RCT

High Internal Validity

Sophisticated design

Controlled environment

Narrow range patients

Range of RCT designs

Example: ‘Real world effectiveness’

Parallel, multicenter, pragmatic RCT...

Setting: 53 primary care practices in the United Kingdom.

Patients:N=300, age 12 to 80 years with doctor diagnosis of asthma

(requiring initiation of asthma-controller therapy).

Randomised:Open-label -> LTRA (Montelukast) or Inhaled glucocorticoid

(Beclo; Bud; or FP).Duration: 24 months

Outcome:Primary outcome measure: Asthma Q-of-L Score.

Conclusion…for 1st line asthma Tmt..

“ ..LTRAs are as effective as ICSteroids” (re Asthma Related Quality of Life)

Months

Q5. How familiar are you with …

… “Equivalence” RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding & could explain to others

Q6. How familiar are you with…

… “Non-inferiority” RCT..?

Have….

1. Never heard of it

2. Heard term - ? some idea what it means

3. Clear understanding & could explain to others

Aim -> Two Active Treatments =

“ Not TOO Different..! ”

re clinically important outcomes

EQUIVALENCE

EQUIVALENCE

Tmt A vs Tmt B ..“Not TOO Different..”

OLD NEW

Aim -> ‘New’ treatment is WORSE…!

but… Not “TOO MUCH” worse..!( ie, ‘acceptably worse’ )

..than current, standard treatment.

NON-INFERIORITY

Aim -> ‘New’ treatment is WORSE…!

but… Not “TOO MUCH” worse..!( ie, ‘acceptably worse’ )

..than current, standard treatment.

AND.. ‘New’ Tmt has other advantages

NON-INFERIORITY

Tmt Bnew “Not MUCH worse..” than Tmt Aold

NON-INFERIORITY

OLD

NEW

Non-inferiority RCTs

• New Drug vs Old (‘Standard Tmt’)

• Assumes New is ‘similar’.. But LESS Effective

“Trade-off” re advantages ‘New’ ..

eg …cheaper / safer..

…more acceptable / more convenient

…less hassles

Non-inferiority RCTs

• New Tmt vs Old (‘Standard’) Tmt’

• New Tmt has ‘similar’ efficacy..

• Sponsored by Pharma ( of NEW Tmt..! )

Child with Primary (‘latent’) Tuberculosis

Primary (‘latent’) Tuberculosis

‘Standard’ Tmt = 9 months Isoniazid (INAH)( ‘chemoprophylaxis’ )

Primary ( ‘latent’ ) Tuberculosis

* ‘New’ approach = 4 mo Rifampin

* Advantage of shorter course

-> better Adherence -> better Efficacy

BUT…RIFAMPIN -> Adverse Events

SAFETY: 4 months Rifampin vs 9 months Isoniazid.

Funded by Canadian Institutes of Health Research

“..Non-inferiority, open-label RCT.”

Primary Outcome: Adverse effect -> Cessation Tmt

Secondary Outcome: Any Adverse effect

* ASSUMPTIONS..

1. Similar Efficacy ( INAH vs RIF )

2. Better Adherence with Rifampin..

3. BUT .. Adverse Events with Rifampin

But..? How much ‘trade off’ acceptable -> ie, more side effects with Rifampin

-> for better adherence..

From literature rate AEs on Isoniazid = 6%

..Expect higher rate AEs with Rifampin

Question for you..

5

4

Rate AEs on Isoniazid = 6%

Q: How much of an INCREASE in Adverse Effects ‘acceptable’ with RIF (*RD)…?

A. 2% ( ie, rate AEs with RIF -> 8% )B. 5% ( -> 11% ) C. 10% ( -> 16% )

*RD = Risk Difference (Absolute RD)

Rate AE Isoniazid = 6%

The authors chose an additional 5% (RD)…

A. 2% B. 5% ( rate AEs with Rif <11% ) C. 10%

ie, ‘Non-inferiority Boundary’ = 5% difference

ie, Maximum ‘acceptable’ increase side effects on Rifampin.

If CI95% < 11% (RIF) = Non-inferior wrt AE ( vs 6% INAH)

If ‘superiority’ RCT..

Risk Difference ( Adverse Events )

RIF better RIF worse (Less side effects) (More side effects)

0%

Non-inferiority boundary (‘Trade Off’)...

Absolute Risk Difference (ARD)

RIF better RIF worse (Less side effects) (More side effects)

0% 5%

Non-inferiority boundary (‘Trade Off’)...

Absolute Risk Difference (ARD)

RIF better RIF worse (Less side effects) (More side effects)

0% 5%

Zoneof Non-inferior

?Possible Result #1 = ISONIAZID is ‘superior’( wrt ‘Safety’)

0% 5%

Risk Difference

RIF WorseRIF Better

CI95%

?..Result #2 = RIF ‘superior’ wrt ‘Safety’ ( Unlikely..! )

0% 5%

Risk Difference

RIF WorseRIF Better

?..Result #3 = ‘Underpowered’

0% 5%

Risk Difference

RIF WorseRIF Better

?..Result #4 = RIF ‘Non-inferior’..!

0% 5%

Risk Difference

RIF WorseRIF Better

RESULTS:

Severe Adverse Effects -> Stop Treatment

-> Rifampin = 0% vs Isoniazid = 0%..!!

Secondary Outcome: -> Minor AEs = 8.1% (RIF) vs 8.5% (INAH)

Risk Diff = -0.3% ( CI95%: -3.3% to 2.7% )

RESULTS ( 20 OM : Minor AEs)..RD = -0.3% ( CI95%: -3.3% to 2.7% )RIF = ‘Non-inferior’ ( ie, <5% RD )

0% 5%

Risk DifferenceRIF WorseRIF Better

Adherence:Rifampin = 86.5% vs Isoniazid = 77.1%

Difference (adjusted) = 13.6% better on RIF(CI955= 7.9 to 19.3); p<0.05

Checklist (Crit Appraisal): Non-inferiority RCT

Critical Appraisal: Non-inferiority RCTs

Step 1. RCT Valid.?

* Prognostic balance at Start ?Concealed Randomization (n = 829);

Risk (prognostic) factors balanced (table 1)

* Prognostic balance Maintained ?Open label (*Adjudicators of AEs = Blinded)

Min Loss Follow-Up (2%) ; ‘Modified’ ITT (?)

Appraisal of Non-inferiority RCTs

Step 1. RCT Valid..

Step 2. Non-inferiority established..?

Did investigators guard against

Unwarranted conclusion of non-inferiority?

Quality criteria wrt Non-inferiority…

1. Non-inferiority boundary defined & justified.?

YES…

Tho, used adult rates AEs (9 mo Isoniazid)

Quality criteria wrt Non-inferiority…

1. Non-inferiority boundary defined & justifiedYES. Adult rates AE (on 9 mo Isoniazid)

2. Sample size calc took boundary into account..?

YES…

80% power detect non-inferiority boundary

Quality criteria wrt Non-inferiority…

2. Sample size calc took boundary into accountYES. 80% power non-inferiority boundary

3. Both ‘ITT’ & ‘per-protocol’ analyses..?

?..YES / ..? NO

“Mod ITT”(?) -> ‘Per-protocol’-> same (?!)

( ‘justified’: Min loss F/Up & Good adherence )

Quality criteria wrt Non-inferiority…

3. Both ITT & per-protocol analyses?..YES ..Per-protocol-> same as “Mod ITT”

4. Was the effect of ‘Standard’ Tmt retained?

NO..!…AE rate on Isoniazid very low (0%)..!Expected rate AEs = 6%

No obvious explanation (?child vs adult)Dosages for both were ‘standard’

Quality criteria wrt Non-inferiority…

4. Was the effect of ‘Standard’ Tmt retained?NO…AE rate on Isoniazid very low (0%)..!No obvious explanation (?child vs adult)

5. Results include CI95%..?

YES.

CI95% = −3.3% to 2.7% ( RD = -0.3% )

ie, < Non-inferiority boundary (ie, <5%)

Question for you..

7

6

Q. What is the effect on SAFETY of longer duration Tmt with Isoniazid (ie, 9 m vs 4 mo Rif) ?

? ..Promotes -> ‘Non-inferiority’ of RIF…

Yes No Unsure

Q. Effect on SAFETY of Longer Duration Tmt with Isoniazid (9 vs 4 mo Rif) ?

YES -> Likely MORE AEs with 9 mo Isoniazid

-> ‘Non-inferior’ wrt AEs with 4 mo RIF..!

…“to account for the longer treatment period & more follow-up

visits in isoniazid group, we estimated the average percentage of visits in which minor symptoms (such as stomach upset, poor appetite, or fatigue were reported).”

Question for you..

7

9

Q. Are you convinced by author’s conclusion of 'Non-inferiority' ..?

0/10‘TotallyUNCONVINCED’

10/10‘Totally

CONVINCED’

..difficultes in design, conduct, analysis & interpretation..!

Syst Review: ‘Quality of Non-inferiority RCTs’JAMA (2006);295:1147–1151

n=162 N.I. RCTs.. Using 4 Quality criteria:1. Non-inferiority boundary defined & justified 2. Sample size calculation took boundary into account3. Both ITT & per-protocol analyses reported4. Results include CI95%

* Only 20.3% (of 162) fulfilled all 4 criteria..!

* And… 12.1% of high quality trials -> -> misleading conclusions..!

*ie, Medical journals with impact factor >10

Esp.. Lancet & New England Journal of Medicine (>75%) (also JAMA, BMJ, & Annals of Internal Medicine)

Published between January 2010 -> May 2015

n = 168 N.I. trials included.

“Non-inferiority trials: Systematic review of reporting in *MAJOR general medical journals.”

Rehal S, et al. BMJ Open (2016)

Results: n = 168 N.I. trials

79% (132/168) trials concluded ‘Non-inferiority’…!

Only 46% (77/168) -> Justification of ‘N.I. margin’.

“Non-inferiority trials: Systematic review of reporting in major medical journals.”

Rehal S, et al. BMJ Open (2016)

Results: n = 168 N.I. trials

35% (65/168) -> conducted & report only one analysis-> most commonly ITT analysis.

59% (99/168) ignored “missing data” ..!

Conclusion (in high impact Med Js):“Reporting of non-inferiority trials is POOR…!”

“Non-inferiority trials: Systematic review of reporting in major medical journals.”

Rehal S, et al. BMJ Open 2016;6:e012594.

Be suspicious of Non-inferiority RCTs..!

No Journal, paper, or author is above suspicion of ‘SPIN’

.. How to ‘FAKE’ -> Non-inferiority..!( esp..wrt ‘Safety’ )

1. Population..

Exclude high risk patients

Enroll non-adherant patients

Enroll currently tolerating ‘unsafe’ drug

2. Intervention / Comparator

Low dose of ‘unsafe’ intervention

Protective co-interventions

Shorter duration of ‘unsafe’ drug

.. How to ‘FAKE’ -> Non-inferiority..!(esp..wrt ‘Safety’)

..How to ‘FAKE’ -> Non-inferiority..!

3. Outcome..

Unjustified ‘Non-inferiority Boundary’

Less important 10 outcome measure

Surrogate outcome measure

..How to ‘FAKE’ -> Non-inferiority..!

4. Study design / execution

Poor (or un-measured) adherence

Large 'Loss to Follow-up’

Small sample size / Low event rates

Learning Objectives

1. Recognise the ‘new’ RCT designs

2. Appreciate the roles of ‘new’ RCTs

3. How to appraise a Non-inferiority RCT

-> Take care with…claims of ‘Non-inferiority’

Take Home Message