Introduction to Meta-analysis

What does a statistical significant

or non-significant result mean?

Type I & II error

Statistical power

• Many (most) clinical trials do not have enough

statistical power to detect clinically important

effects.

Research synthesis

• Karl Pearson synthesized data on efficacy of typhoid

vaccination (1904)

– “Many of the groups… are far too small to allow of any

definite opinion being formed at all, having regard to the

size of the probable error involved.”

• "It is surely a great criticism of our profession that we have

not organised a critical summary, by specialty or subspecialty,

adapted periodically, of all relevant randomized controlled

trials”-- Archibald Cochrane

Research synthesis

• Meta-analysis is a statistical technique for

combining the findings from independent

studies.

• Meta-analysis is most often used to assess the

clinical effectiveness of healthcare

interventions; it does this by combining data

from two or more randomised control trials.

Hierarchy of evidence

• Systematic reviews and meta-analyses of randomized controlled trials

• Randomized controlled trials

• Non-randomized intervention studies

• Cohort studies

• Case control studies

• Case series

• Case report

• Expert opinion

Meta-analyses indexed in PubMed

Search: meta-analysis(Publication Type)

Types of reviews

• Traditional, narrative review

• Overview

• Systematic review

• Meta-analysis

• Pooled analysis

Issues with traditional, narrative reviews

• Subjective, prone to bias and error

• Inadequate literature search strategy

• Unclear inclusion/exclusion criteria

• No description of the methods used by the review

• Lack of consistent methodology to draw consistent qualitative or

quantitative conclusions

– Replication or validation of review conclusions issues

– Conflicting conclusions based on the same evidence

Mammography Screening

• Gotzsche & Olsen [Nordic Cochrane Centre] concluded that

“screening for breast cancer with mammography is unjustified”

(Gotzsche & Olsen. Lancet 2000;355:129 – 34. & Lancet

2001;358:1340-42) RR =1.00 (95% CI 0.96, 1.05)

• The US Preventive Services Task Force reviewed the same set of trials (2001):

– The USPSTF recommends screening mammography, with or without clinical breast examination, every 1-2 years for women aged 40 and older.” RR=0.77 (95 percent CI, 0.67-0.89)

Meta-analyses Steps

• Formulate question & write a protocol

• Retrieve study articles

• Assess study quality: inclusion/exclusion

• Extract data

• Analyze data

• Conclusions and reports

P - Patient population

I - Intervention or exposure

C –Comparison group

O - Outcome compared

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997;127:380-7

Formulate question

Issues with meta-analyses

• Meta-analysis should be part of a systematic review

• Poor quality studies should be excluded

• Heterogeneity of studies should be addressed

• Be aware of biases

– Publication bias

– Time lag bias

– Duplicate publication bias

– Language bias

– Outcome reporting bias

Heterogeneity

• Heterogeneity index

–Cochrane's Q and I2 statistic

• Non-homogeneous studies should not be

pooled and analyzed together.

Meta-analyses Evaluation

• The quality of meta-analyses are highly

variable

• Cochrane reviews, on average, may be better

reported than journal reviews

• The quality of meta-analyses cannot be taken

for granted

Appraising Meta-analyses

Examples sources of references

• http://media.wix.com/ugd/dded87_a02ff2e3445f495

2992d5a96ca562576.pdf

• Users’ Guides to the Medical Literature 3rd ed.

(2015)

Effect Size

• The effect size

– the “dependent variable” of meta-analyses

– standardizes findings across studies

• Different meta-analyses may use different effect size

indices

– Central Tendency Research

– Pre-Post Contrasts

– Group Contrasts

– Association Between Variables

– Regression analyses

Summarizing studies

• A weighted average according to the amount of information that each trail contains is more reasonable

• Weight by:

– Size (sample size (n))

– Event rate

– Homogeneity (inverse of the variance)

– Quality

– Other factors…

A two-stage process

• A summary statistic for each study

• A summary (pooled) treatment effect

estimate as a weighted average of the

treatment effects estimated in the individual

studies.

What to look for in a systematic review?

• A clearly defined question

• Comprehensive literature search strategy

• Explicit, reproducible inclusion/exclusion criteria

• Quality assessment of primary studies

• Reproducible data extraction procedure

• Data analysis and reporting – Exploration of heterogeneity, publication bias, etc.

• Limitations of evidence

• Data Interpretation

• Implications for patient care and future research

Conclusions

• Rosiglitazone was associated with a significant

increase in the risk of myocardial infarction

and with an increase in the risk of death from

cardiovascular causes that had borderline

significance.

The End