department of o utcomes r esearch
DESCRIPTION
Department of O UTCOMES R ESEARCH. Clinical Research Design. Systematic Reviews and Meta-analysis. Daniel I. Sessler, M.D. Michael Cudahy Professor and Chair Department of O UTCOMES R ESEARCH The Cleveland Clinic. Literature Reviews. Reviews are important - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/1.jpg)
Department of OUTCOMES RESEARCH
![Page 2: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/2.jpg)
Daniel I. Sessler, M.D.Michael Cudahy Professor and ChairDepartment of OUTCOMES RESEARCH
The Cleveland Clinic
Clinical Research Design
Systematic Reviewsand
Meta-analysis
![Page 3: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/3.jpg)
Literature Reviews
Reviews are important•Often too much primary literature•Clinicians cannot critically review all literature
Classical reviews•Informed synthesis by authors
– Most helpful when authors are experts and active investigators
•Excellent perspective– Integrates historical development with future directions
•Typically restricted to best relevant articles•Most suitable for reviewing an entire field•Subject to author(s) bias
![Page 4: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/4.jpg)
Useful for specific interventions & outcomes•Specific, important, and sensible question essential•Equally effective for complications and therapeutic outcomes
Standardized search of all relevant work•Documented and reproducible selection process•Tabular presentation, often stratified by
– Research approach– Study quality– Population– Outcome
Synthesis can be•Qualitative, based on authors’ expertise (and bias)•Quantitative: meta-analysis
Systematic Reviews
![Page 5: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/5.jpg)
Meta-analysis
Statistical summary of systematic review•Effect size and significance•Patient level (patient pooling) or study level (aggregate stats)
– Individual patient data preferable, but rarely available
Usually used for randomized trials•Can be used for observational studies— with great caution
Studies must evaluate similar treatment & outcomes
Suitable for various types of data•Dichotomous, continuous, risk difference, relative risk, etc.
Generalizability good; internal validity variable
![Page 6: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/6.jpg)
Data-acquisition
Individual studies are unit of analysis•Summary statistics are the data elements
Consider studies to be like patients in a trial•Rigorous a priori inclusion and exclusion criteria
Specify search strategies and sources of studies•Which databases? Search terms?•Language restrictions?•Randomized trials only?•Primary outcomes only?•Published versus unpublished?
Specify adjudication methods
![Page 7: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/7.jpg)
Sample Data-extraction Form
Population
Comparison•Treatment•Active vs. placebo
Outcome(s)
Measures of quality
Surprisingly difficult•Adjudication critical
![Page 8: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/8.jpg)
Evaluating Study Quality
No good way•Many design errors non-obvious or subtle
Various scoring systems used; points for•Legitimate randomization•Concealed allocation•Blinded outcome evaluation•Drop-outs and reasons described
Standard-of-care: report quality of included studies
![Page 9: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/9.jpg)
Reporting Search Results
![Page 10: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/10.jpg)
Major Sources of ErrorGarbage in, garbage out
•Meta-analysis never better than underlying studies•Cannot correct for methodologic errors or bias
Reporting bias•Changed or omitted primary outcomes•Significant findings 2.2-4.7 X more likely to be complete (Dwan 2008)
Subtle (or not) treatment & measurement differences
Publication bias•Large trials are almost always published•Positive studies usually published even if under-powered•Small negative studies less likely than others to be published
– Censoring by authors or corporate sponsors– Appropriate editorial decision, but unpublished studies disappear– Meta-analysis depends on knowing about all relevant results
![Page 11: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/11.jpg)
Funnel Plots
SE of Log(OR
)
Log(OR)
![Page 12: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/12.jpg)
Heterogeneity
Data: variation in study results exceeding chance
Biology: true differences related to methodology•Differences in populations: age, gender, ethnicity, etc.•Differences in drug dose (or drug within a class)•Unappreciated patient factors
Tests: chi square, etc.
Analysis strategies•Minor heterogeneity
– Report amount– Combined analysis may be sensible
•Treat serious heterogeneity as an interaction– Stratify analysis as for other effect modifiers
![Page 13: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/13.jpg)
Analysis Strategies
Fixed-effects model•Assumes all trials share same underlying treatment effect
– Treats each trial as random samples from one large trial – Differences in results due to chance alone
•Similar to Mantel-Haenszel•Often over-estimates significance
Random-effects model•Assumes each study estimates a unique treatment effect
– That is, may truly differ from other included studies– Allows heterogeneity, and is required for heterogeneous data
•Weights smaller studies more heavily•Generally provides similar effect estimate with lower precision
– More conservative; probably should always be used
![Page 14: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/14.jpg)
Forest Plots
Log weighted mean effect ≈ sum of {log (effect)/variance)} for individual studies, divided by sum of 1/variance
![Page 15: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/15.jpg)
How Good are Meta-analyses?
“Large” defined by n≥1,000
“Large” defined by powerGenerally, pretty good. But not perfect.
Cappelleri, JAMA 1996
![Page 16: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/16.jpg)
Meta-analyses Increasingly Common
Most published as part of systematic reviews
Increasingly included in trial reports•Objective comparison of current to previous results
Grant applications•Summarize knowledge•Support equipoise •Need for proposed trial•Complications unlikely
Blood loss with low-dose perioperative aspirin
![Page 17: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/17.jpg)
Cochrane Collaboration
International non-profit, 1993
Repository for meta-analyses
Standardized reporting•QUORUM (1999)•PRISMA (2009)
Provides free software
Evidence-based med movement•David Sackett•Gordon Guyatt•Tom Chalmers
Archie Cochrane
![Page 18: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/18.jpg)
Summary
Systematic reviews•More objective than “expert” reviews•May lack expert perspective and subtlety•Meta-analysis is quantification of systemic review
Subject to major errors•Any problems with underlying studies remain•Publication and reporting bias can be substantial•Heterogeneity can complicate analysis
Conduct and report per guidelines
Useful summary of available literature•Especially when many similar studies are available
![Page 19: Department of O UTCOMES R ESEARCH](https://reader036.vdocuments.net/reader036/viewer/2022062805/56814df9550346895dbb67f2/html5/thumbnails/19.jpg)
Department of OUTCOMES RESEARCH