September 19, 2012

SYSTEMATIC REVIEWS

It is necessary, while formulating the problems of which in our advance we are to find the solutions, to call into council the views of those of our predecessors who have declared an opinion on the subject, in order that we may profit by whatever is sound in their suggestions and avoid their errors.

Aristotle, De Anima, Book 1, chapter 2

Definitions

•Literature review•A survey of previously published material

•Systematic review (aka research synthesis)•Synthesis of previously published material to answer a research question

•Meta-analysis•Quantitative procedures to combine the results of published research studies

Systematic Review

•Key differences from literature review: • Integration of primary research studies• Synthesis of those studies• Seek to answer a research question•Quantitative analysis (usually)

Importance of Systematic Review

•Since 1970s systematic review has become its own type of research•Citations to systematic reviews over time:

ExamplesProblem: Hantavirus in the United States

•Literature review•History of hantavirus in the United States

•Systematic review•Are pet owners at lower risk for hantavirus infection?

•Meta-analysis•Quantitative methods to analyze risk of hantavirus infection in pet owners vs non pet owners

Stages of Systematic Review

1. Define the Problem2. Literature Search3. Data Evaluation4. Data Analysis5. Interpretation of Results6. Presentation of Results

Why conduct a systematic review?

•Synthesize knowledge•Critically analyze a problem•Create generalizations• Identify and analyze central theories• Identify limitations•Resolve conflicts• Identify central issues for future research

Terms to Consider

•Bias•Grey literature•Heterogeneity•Representativeness•Utility

Bias

“if the sample of studies retrieved for review is biased, the validity of the results, no matter how systematic and thorough in other respects, is threatened”

A bias prevents neutral review of information:

• Personal bias (due to author’s views)• Publication bias (reporting of positive results)• Language bias (excluding studies by language)• Availability bias (readily available study results)• Cost bias (inclusion of free or low cost studies)

Grey Literature

• Literature that is hard to retrieve• Results of research not published• Literature produced by government, academia, etc that with other goal than commercial publishing• Will not be retrieved by traditional database search

Heterogeneity

•The extent to which observed effects differ from one another (the variability of the effect sizes in various studies in your systematic review)•Systematic review pools data from many research studies•These studies are inherently different•Are the included studies similar enough to yield meaningful results when pooled?

Tests of Heterogeneity

•Cochran’s Q•Weighted sum of squared differences in study effects•Distributed as chi-squared statistic• Informs of presence vs absence of heterogeneity

• I2

• Estimate of percent variability across studies•Helps quantify degree of heterogeneity• As I2 increases, more heterogeneity exists

Representativeness

• How well included studies represent all studies• Can we draw meaningful inferences using studies included in a review?

Limitations:• Types of studies that have been conducted• Search methods – exhaustive?• Accessibility of conducted studies• Publication or reporting bias

Utility

“fitness for some purpose or worth to some end”

–Merriam Webster Dictionary

• Considers ultimate goal of the review• What will the review add to the evidence

database?• What information to include• Audience