superiority, non-inferiority, equivalence studies - what is the difference?

Superiority-Equivalence-Non-Inferiority Trials: What does it all mean?

Gerald Gartlehner

Is the new treatment better than the established one ?

If not, is it equally effective and preferable for some other reason

What decision-makers want to know

Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems

Gerald Gartlehner

Overview


Gerald Gartlehner

Definitions and concepts Challenges of non-inferiority trials What does this mean for systematic reviews? Language considerations for comparative

effectiveness reviews

Proving Efficacy

1. Showing superiority of one (new) treatment over another (placebo or active)

2. Showing equivalence or non-inferiority of a new intervention relative to an already existing efficacious treatment


Gerald Gartlehner

The interpretation of non-inferiority and equivalence can be confusing


Gerald Gartlehner

Ranking position: 72

Ranking position: 34


Gerald Gartlehner

Superiority game: the winner takes it all….


Gerald Gartlehner

Equivalence game – equally good or clearly better

Equivalence margin: tied or less than 1 goal difference


Gerald Gartlehner

Non-inferiority game– at least not substantially worse….

Non-inferiority margin: can‘t lose with more than 1 goal difference


Gerald Gartlehner

Definitions

Superiority trialObjective: To determine a clinically relevant difference between two interventions

Equivalence trialObjective: To determine whether a (new) intervention is neither worse nor better than another (established) intervention

Non-inferiority trialObjective: To determine whether a (new) intervention is not inferior to another (established) intervention


Gerald Gartlehner

Equivalence - Non-Inferiority

The naïve approach: If a head-to-head trial shows no statistically significant

difference, two interventions are “equivalent” Problem – underpowered studies or high variance will

create “equivalent” treatments

The statistical approach: Define a margin of non-inferiority or equivalence If 95% confidence interval of the difference DOES NOT

cross the margin, the new intervention is non-inferior or equivalent


Gerald Gartlehner

01020

Arthroscopy vs. sham arthoscopy in patients with knee osteoarthritis

Mosley et al. New England J Med, 2002;347:81-88

Knee Specific Pain Scale: difference in points

10 20

MID

favors arthroscopyfavors sham


Gerald Gartlehner

Non-inferiority margin (d)

The limit of acceptable inferiority:Minimal important differenceClinical judgementStatistical considerations


Gerald Gartlehner

012

Standard vs. placebo2 points (1.5 to 2.5)

Maximimum non-inferiority margin: dmax = 1.5Fractional preservation of treatment effect: f = 0.5d = dmax x (1 – f)d = 1.5 x ( 1 – 0.5) = 0.75

New vs. standard

Determining the non-inferiority margin statistically

Peculiar issues of non-inferiority trials: the backwardness

Null hypothesis and alternative hypothesis are reversed

Type I and type II errors are reversedPer-protocol analyses can be more important than

ITT analysis (ITT analyses are biased towards finding no difference)

P-value is one-sided (0.025)

Assay Sensitivity and Constancy Assumption

The ability of a trial to distinguish effective from ineffective treatments (depends on the effect size the trial wants to detect).


Gerald Gartlehner

Assay Sensitivity

Assumptions in non-inferiority trials:

The efficacy of the active control was preserved in the non-inferiority study (i.e. that it had assay sensitivity).

If it was not, equivalence or non-inferiority conclusions are meaningless (The non-inferior drug could have no effect at all).

Constancy Assumption

Active comparator must be well established and have predictable and consistent treatment effects

Participants must be similar to those in trial establishing efficacy

Outcomes must be similar to those in trials establishing efficacy

012

Biocreep

Effective treatment

Still clinically relevant?


Gerald Gartlehner

Key Points for critical appraisal

Was a non-inferiority margin defined based on clinical considerations and statistical reasoning?

Was it established a priori?Was the study powered based on the non-inferiority

margin?Was an ITT and a per-protocol analysis conducted?Was the trial design (e.g. eligibility criteria) consistent

with placebo controlled trials of the established treatment?

What does this mean for systematic reviews ?

For meta-analyses – data can be used just as from any superiority trial

For qualitative assessments – language considerations


Gerald Gartlehner

Language considerations

“..there was no statistically significant difference between A and B..”

“..studies failed to show a difference..”Can mean:1) The evidence shows equivalence2) The evidence is inconclusive (because confidence

intervals are wide-lack of precision)

AHRQ guidance: Assessing equivalence and non-inferiority [draft]


Gerald Gartlehner

Language considerations

Better:“..treatments A and B had similar mortality rates..”

Expressing non-inferiority:

“…treatment A is at least as effective as treatment B for [Outcome or study objective]…”

AHRQ guidance: Assessing equivalence and non-inferiority [draft]Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems

Gerald Gartlehner

Do we need to establish equivalence margins for CERs ?

superiority, non-inferiority, equivalence studies - what is the difference?

Health & Medicine