learning programs to accelerate the biopharma transition network meta-analysis what is a network...

Learning Programs to Accelerate the BioPharma Transition

Network Meta-analysis

• What is a network meta-analysis?

• GRADE approach to confidence in estimates

• Determining credibility of NMA


Comparing Multiple Treatments:Introduction to Network Meta-Analyses

• Many disease areas where many alternatives exist

• Clinicians/patients need to know about relative merits

• Impractical to test each comparator directly

• Simultaneous comparison of multiple treatments

• “Network meta-analysis”, “mixed treatment comparisons”, “adjusted indirect comparisons”


Conventional meta-analysisPooled Estimate assumption

Patients Interventions Outcomes Methodology

Assumes effect similar across

“Homogeneity assumption”

Single best estimate of treatment effect


Relative Risk with 95% CI for Vitamin D Non-vertebral Fractures

Chapuy et al, (2002) 0.85 (0.64, 1.13)

Pooled Random Effect Model 0.82 (0.69 to 0.98)p= 0.05 for heterogeneity, I2=53%

Chapuy et al, (1994) 0.79 (0.69, 0.92)

Lips et al, (1996) 1.10 (0.87, 1.39)

Dawson-Hughes et al, (1997) 0.46 (0.24, 0.88)

Pfeifer et al, (2000) 0.48 (0.13, 1.78)

Meyer et al, (2002) 0.92 (0.68, 1.24)

Trivedi et al, (2003) 0.67 (0.46, 0.99)

Favors Vitamin D Favors Control

Relative Risk 95% CI

'

'

'

'

'

'

'

'

0.1 1


Alendronate (A) Risedronate (B)

Placebo (C)

Interested in A versus B available data A vs C, B vs C

Less confidence than direct?Why?

Indirect Comparisons


Vulnerability of Indirect comparison

• Effect modifiers– Patients– Optimal interventions– Comparator– Cointerventions– Outcome measures– Risk of bias


Combine direct and indirect comparisons- additional assumption mediators same in direct and indirect- “consistency” or “coherence” assumption


Comparison with

TreatmentsNRT

.5 1 2.75 1.33

Odds ratio

4

Antidepressants

NRT + NRT

1.54(1.02-2.31)

.5 1 2.75 1.33

Odds ratio

4 .5 1 2.75 1.33

Odds ratio

4

Varenicline

Antidepressants

Varenicline1.70

(1.41-2.04)

NRT + antidepressant1.62

(1.00-2.63)

1.28(0.82-1.99)

.5 1 2.75 1.33

Odds ratio

NRT + antidepressant

1.34 (0.71-2.56)

Comparison with

TreatmentsNRT

.5 1 2.75 1.33

Odds ratio

4

Antidepressants

NRT + NRT

1.54(1.02-2.31)

.5 1 2.75 1.33

Odds ratio

4 .5 1 2.75 1.33

Odds ratio

4

Varenicline

Antidepressants

Varenicline1.70

(1.41-2.04)


(1.00-2.63)

1.28(0.82-1.99)

.5 1 2.75 1.33

Odds ratio


1.34 (0.71-2.56)

Network Meta-analysis Case Study: Which Approach to Nicotine Addiction Works Best


Network Meta-analysis Case Study

Combines effect estimates from direct and indirect comparisons

Placebo

Varenicline

Antidepressants + NRT

Nicotine replacement

treatment (NRT)

Antidepressants


Comparison with

TreatmentsNRT

.5 1 2.75 1.33

Odds ratio

4

Antidepressants

NRT + NRT

1.54(1.02-2.31)

.5 1 2.75 1.33

Odds ratio

4 .5 1 2.75 1.33

Odds ratio

4

Varenicline

Antidepressants

Varenicline1.70

(1.41-2.04)


(1.00-2.63)

1.28(0.82-1.99)

.5 1 2.75 1.33

Odds ratio


1.34 (0.71-2.56)

Comparison with

TreatmentsNRT

.5 1 2.75 1.33

Odds ratio

4

Antidepressants

NRT + NRT

1.54(1.02-2.31)

.5 1 2.75 1.33

Odds ratio

4 .5 1 2.75 1.33

Odds ratio

4

Varenicline

Antidepressants

Varenicline1.70

(1.41-2.04)


(1.00-2.63)

1.28(0.82-1.99)

.5 1 2.75 1.33

Odds ratio


1.34 (0.71-2.56)

Comparison with

TreatmentsNRT

1.01 (0.88- 1.15)

.5 1 2.75 1.33

Odds ratio

4

Antidepressants

NRT + NRT 1.35 (1.04- 1.75)

1.34(1.00- 1.78)

.5 1 2.75 1.33

Odds ratio

4 .5 1 2.75 1.33

Odds ratio

4

Varenicline

Antidepressants

Varenicline 1.17 (0.98- 1.39)

1.17 (0.98- 1.39)

1.16(0.97- 1.38)

1.16(0.97- 1.38)

NRT + antidepressant 1.30(0.98- 1.73)

1.30(0.98- 1.73)

1.29 (0.96- 1.74)

1.29 (0.96- 1.74)

1.12 (0.81- 1.55)

.5 1 2.75 1.33

Odds ratio

1.16 (0.85- 1.58)


1.04 (0.71- 1.52)


Antide-pressants

NRT

Direct Comparison

control

buspirone

rimonabant

varenicline

antidepressants +NRT

clonidine

NRT+NRT

1.63, I2=0% 4 comparisons




1.28, I2=0%3 comparisons

4.851 comparison




1.281 comparison

1.121 comparison



1.5 1 1.5 2 2.5NRT superior Antidepressants

superior

Direct evidence (3 trials)

1.34 (0.71, 2.56)I-squared=43.7%


Antide-pressants

NRT

Indirect Comparison 1

control

buspirone

rimonabant

varenicline


clonidine

NRT+NRT






4.851 comparison




1.281 comparison

1.121 comparison



1.5 1 1.5 2 2.5NRTsuperior Antidepressants

superior


Indirect evidence

1.34 (0.71, 2.56)I-squared=43.7%

1.01 (0.81,1.27)


Antide-pressants

NRT

Indirect Comparison 2

control

buspirone

rimonabant

varenicline


clonidine

NRT+NRT






4.851 comparison




1.281 comparison

1.121 comparison





1.5 1 1.5 2 2.5NRTsuperior Antidepressants

superior


Indirect evidence

1.34 (0.71, 2.56)I-squared=43.7%

0.85 (0.38, 1.92)


5 Paths to Indirectly CompareAntidepressants vs NRT

placebo andnonplacebo control

buspirone

rimonabant

vareniclineantidepressants

NRT


clonidine

NRT+NRT

4 comparisons

2 comparisons

2 comparisons

5 comparisons29 comparisons

67 comparisons

3 comparisons

1 comparison

5 comparisons

6 comparisons

3 comparisons

3 comparisons

1 comparison

1 comparison

1

2

3

4

5


5 Paths to Indirectly Compare Antidepressants vs NRT

placebo andnonplacebo control

buspirone

rimonabant

vareniclineantidepressants

NRT


clonidine

NRT+NRT

4 comparisons

2 comparisons

2 comparisons

5 comparisons29 comparisons

67 comparisons

3 comparisons

1 comparison

5 comparisons

6 comparisons

3 comparisons

3 comparisons

1 comparison

1 comparison

1

2

3

4

5

1.01 (0.81, 1.27)

0.85 (0.38, 1.92)

0.89 (0.29, 2.77)

1.56 (0.54, 4.49)

1.31 (0.25, 6.76)


Antide-pressants

NRT

Comparative effectiveness of NRT vs. Antidepressants on prolonged abstinence (≥6 months)

control

buspirone

rimonabant

varenicline

clonidine

NRT+NRT






4.851 comparison




1.281 comparison

1.121 comparison




Indirect evidence

Direct evidence

I-squared = 43.7%

1.34 (0.71, 2.56)

1.5 1 1.5 2 2.5

3 trials pooled

NRT superior Antidepressants superior

0.89 (0.29, 2.77)

1.56 (0.54, 4.49)

0.85 (0.38, 1.92)

1.31 (0.25, 6.76)

1.01 (0.81, 1.27)

1.5 1 1.5 2 2.5

Antidepressants superior

NRT superior

1

2

3

4

5

Path


Antide-pressants

NRT


control

buspirone

rimonabant

varenicline

clonidine

NRT+NRT






4.851 comparison




1.281 comparison

1.121 comparison




Indirect evidence

Direct evidence

I-squared = 43.7%

1.34 (0.71, 2.56)

1.5 1 1.5 2 2.5

3 trials pooled

NRT superior Antidepressants superior

0.89 (0.29, 2.77)

1.56 (0.54, 4.49)

0.85 (0.38, 1.92)

1.31 (0.25, 6.76)

1.01 (0.81, 1.27)

1.5 1 1.5 2 2.5


NRT superior

1

2

3

4

5

Path

0.98 (95% 0.85-1.13)


Antide-pressants

NRT


control

buspirone

rimonabant

varenicline

clonidine

NRT+NRT






4.851 comparison




1.281 comparison

1.121 comparison




Indirect evidence

Direct evidence

I-squared = 43.7%

1.34 (0.71, 2.56)

1.5 1 1.5 2 2.5

3 trials pooled


0.89 (0.29, 2.77)

1.56 (0.54, 4.49)

0.85 (0.38, 1.92)

1.31 (0.25, 6.76)

1.01 (0.81, 1.27)

1.5 1 1.5 2 2.5


NRT superior

1

2

3

4

5

Path

0.98 (95% 0.85-1.13)

pooled estimate1.01

(95% 0.88-1.15)

NRT superior

Alendronate n=5,084 Raloxifene

n=10,975

Zoledronate n=4,954

Denosumab n=3,933

Vitamin D n=12,469

Calcium n=3,896

Ibandronate n=1,912

Placebo n=41,548

Teriparatide (PTH) n=1,093

Risedronate

n=6,850

Hip Fractures# of trials =40# of participants =139,647# of hip fracture =2,567

Vitamin D and Calcium n=46,933

Confidence in Estimates

Step 1: Present direct and indirect estimate for each comparison of the evidence network

Step 2: Rate the confidence in the direct and indirect estimate

Comparison Direct evidence OR

(95% confidence

interval)

Direct evidence

confidence in estimates

Indirect evidence OR (95% credible

interval)

Indirect evidence


NetworkOR (95% credible interval)

Network confidence in

estimates

Alendronate vs. Raloxifene

0.49 (0.04 to 5.45)

0.53(0.30 to 0.90)

ExampleApproach


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

LOW

0.53(0.30 to 0.90)

MODERATE

Step 3: Present the NMA estimate

Step 4: Rate the confidence in the NMA estimate


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

LOW

0.53(0.30 to 0.90)

MODERATE

0.51 (0.29-0.87)


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

LOW

0.53(0.30 to 0.90)

MODERATE

0.51 (0.29-0.87)

MODERATE

Starting level of indirect

• Intutitively lower confidence– Previous GRADE guidance start at moderate

• NMA enthusiasts argue no different– New guidance: Start at high– Only rate down for lack of similarity

(intransivity)


n=10,975

Zoledronate n=4,954

Denosumab n=3,933

Vitamin D n=12,469

Calcium n=3,896

Ibandronate n=1,912

Placebo n=41,548


Risedronate

n=6,850




n=10,975

Zoledronate n=4,954

Denosumab n=3,933

Vitamin D n=12,469

Calcium n=3,896

Ibandronate n=1,912

Placebo n=41,548


Risedronate

n=6,850



Dominant link

Confidence in indirect estimate

• Lowest of direct estimates

• A versus B comparison of interest

• Through C dominant link

• A versus C high; B versus C moderate or low

• Confidence based on B versus C

Step 1: Present direct and indirect estimate for each comparison of the evidence network

Step 2: Rate the confidence in the direct and indirect estimate


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

0.53(0.30 to 0.90)

ExampleApproach


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

LOW

0.53(0.30 to 0.90)

MODERATE

Step 3: Present the NMA estimate

Step 4: Rate the confidence in the NMA estimate


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

LOW

0.53(0.30 to 0.90)

MODERATE

0.51 (0.20-1.32)


(95% confidence

interval)

Direct evidence



interval)

Indirect evidence




estimates


0.49 (0.04 to 5.45)

LOW

0.53(0.30 to 0.90)

MODERATE

0.51 (0.20-1.32)

MODERATE

Confidence in NMA estimate

• If only direct or indirect use that

• If both use higher

Confidence in NMA estimate

• Direct and indirect very different

• Terminology differs

• Ours (for now)– Heterogeneity for direct– Transitivity for indirect– Incoherence for direct versus indirect

Judging incoherence

• Difference in point estimates

• Extent of overlap in confidence intervals

• P-value for test of incoherence

P-value for test of incoherence 0.02


n=10,975

Zoledronate n=4,954

Denosumab n=3,933

Vitamin D n=12,469

Calcium n=3,896

Ibandronate n=1,912

Placebo n=41,548


Risedronate

n=6,850



Credibility of the Process of NMA

• Usual criteria for systematic review– Explicit sensible questions– Search exhaustive– Selection and assessment reproducible– Present results ready for clinical application

• Address confidence in effect estimates

Presentation of Results

• Often presented with rankings

• Potentially very misleading– Small difference between ranks– Everything low or very low confidence– First ranked lower than others

Comparison Direct evidence OR (95%

confidence interval)

Direct evidence



interval)

Indirect evidence confidence in

estimates

NetworkOR (95% credible

interval)


estimates

Teriparatide vs. Placebo --- --- 0.42 (0.10-1.82) very low 3, 6 0.42 (0.10-1.82) very low

Zoledronate vs. Placebo --- --- 0.50 (0.33-0.74) high 0.50 (0.34-0.73) high

Risedronate vs. Placebo 0.17 (0.05 to 0.59) moderate 1 0.54 (0.36-0.75) low 6 0.48 (0.31-0.66) moderate

Conclusion

• NMA will play important role in EBCP

• Needs usual criteria for SR/MA credibility

• If includes confidence in estimates interpretable

• If no confidence very difficult to interpret– Probably shouldn’t bother

learning programs to accelerate the biopharma transition network meta-analysis what is a network...

Documents

biopharma transition

learning programs

mixed treatment comparisons

coherence assumption

credibility of nma slide

network metaanalyses

indirect consistency

lips et