Systematic Reviews & Knowledge Syntheses:

What a Librarian Needs to Know

Lorie Kloda, MLIS, AHIP, PhD

Assessment Librarian

GLIS-671

February 11, 2015

Systematic Reviews

2

“…reviews of a clearly formulated question that use explicit methods to identify, select, and critically appraise relevant research and to collect and analyse data from the studies that are included in the review.”

Cochrane Collaboration

Knowledge Synthesis

3

“the contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic. A synthesis must be reproducible and transparent in its methods...”

Canadian Institutes of Health Research http://www.cihr-irsc.gc.ca/e/41382.html

Systematic Review Nonsystematic Review

Question • Usually narrow question • Usually broad question

Search Strategy • Explicitly stated

• May be performed in duplicate

• Not stated

Study

Identification

• Criteria explicitly stated

• May be performed in duplicate

• Not stated

Methods • Methods & outcomes of interest

explicitly stated

• May include study quality

assessment, sensitivity analyses

• Not stated

Presentation of

Results

• Typically by study characteristics

so that equivalent components are

compared

• Typically by study

Conclusions • Typically confined to what the data

could infer

• May include personal approaches,

opinions not supported by data

From: Wong R. (2003). Systematic reviews and the Cochrane Collaboration. Oncology

Rounds, 5(10). Retrieved from www.oncologyrounds.ca

Knowledge Syntheses

5

Systematic review

Meta analysis

Scoping review

Evidence mapping

Mixed methods synthesis

Meta-synthesis approaches

Realist synthesis approaches

Rapid evidence assessment

Why Conduct a Knowledge Synthesis?

6

Growth of Systematic Review Publications

7 Bastian, H., Glasziou, P., & Chalmers, I. (2010). Seventy-five trials and eleven systematic reviews a day: How will we ever keep up? PLOS Medicine, 7(9), e1000326. DOI: 10.1371/journal.pmed.1000326

Growth of Syntheses in Graduate Theses

a

Cobus-Kuo, L., Gore, G., & Kloda, L. A. (2014). Research syntheses in graduate research: A scoping review. Canadian Health Libraries Association Annual Conference, June 16-20, 2014, Montreal, QC.

Growth of Syntheses in Theses per 1000

Syntheses in Graduate Theses by Discipline

Cochrane Reviews

11

Midwife-led continuity models versus other

models of care for childbearing women

Oral immunotherapy for milk allergy

Standards

12

For conducting systematic reviews (or research syntheses):

Cochrane Handbook

Institute of Medicine – Standards for Systematic Reviews

Agency for Health Quality Research – Methods Guide for Effectiveness and Comparative Effectiveness Reviews

For reporting systematic reviews & protocols

PRISMA Statement – Preferred Reporting Items for Systematic Reviews and Meta-Analyses (& PRISMA-P, PRISMA-C)

Sources for Finding Systematic Reviews

• Cochrane Library– Cochrane Database of Systematic Reviews (includes Methodology

Reviews)– Database of Abstracts of Reviews of Effects (DARE)– Health Technology Assessments– Other databases of original studies (clinical trials, methods,

economic evaluations)

• Database for Promoting Health Effectiveness (DoPHER)• Health databases

– AMED– CINAHL– EMBASE– MEDLINE– PsycINFO

13

Steps in a Systematic Review

14

1. Define the (clinical) question

2. Identify all relevant research (published and unpublished)

3. Select studies for inclusion

4. Assess the quality of each study

5. Synthesize the findings (meta-analysis or meta-synthesis, if possible)

6. Interpret the findings and present an unbiased summary

McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association, 93(1), 74-80.

Lorie Kloda, McGill University

Mary Ellen Macdonald, McGill University

Rosa Caporicci, McGill University

Susan Cadell, Wilfrid Laurier University

Stephen Liben, Montreal Children’s Hospital

Canadian Health Libraries Association Annual Conference, May 29, 2011, Calgary, Alberta

Experiences of Family

Members after a Child Dies:Scoping the Literature in

Interdisciplinary Research

16

Steps in a Comprehensive Literature Search

17

Database /trial registry searches

Grey literature search

Hand searches

Conference proceedings

Major journals

Bibliographies

Citation searches

“Related citations”

Snowball searches (Reference harvesting)

Review papers

All included studies

Contact researchers

Initial

searches

Supplemental

searches

18

“Conducting a comprehensive,

objective and reproducible search

for studies can be the most time

consuming and challenging task in

preparing a systematic review.”

Higgins, J. P. T., & Green, S. (eds.) (2009). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2. The Cochrane Collaboration. Available from www.cochrane-handbook.org.

Selection of Sources

19

Databases Several required (+ clinical trial registries) Unpublished trials contribute ~20% of the weight in

meta-analysisFries, J. F., & Krishnan, E. (2004). Equipoise, design bias, and randomized controlled trials: The

elusive ethics of drug development. Arthritis Research Therapy, 6, R250-R255.

Grey literature “unpublished,” fugitive,” “in-house,” “non-commercial” Definition/distinction not important; as long as it’s

relevant, it should be included Reduces publication bias

Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No.: MR000010. doi: 10.1002/14651858.MR000010.pub3

20

Search Strategy Development

21

Keywords (natural language) Truncation, synonyms, alternative spelling

Subject Headings Explode

Limitations Languages, date, publication type

Boolean (logical operators),

Field searching Title, abstract

Hedges (optimal search strategies; filters)

Peer reviewMcGowan, J., Sampson, M. & Lefebvre, C. (2010). An evidence based checklist for the peer review of

electronic search strategies (PRESS EBC). Evidence Based Library and Information Practice 5 (1),149-154.

22

23

“Terms within social sciences are often

ambiguous, poorly defined and constantly

changing. Unfortunately, the use of

controlled vocabularies and indexing is not

applied across the social sciences

databases with the same rigour as in

medical databases.”

Papaioannou, D., Sutton, A., Carroll, C., Booth, A., & Wong, R. (2010). Literature

searching for social science systematic reviews: Consideration of a range

of search techniques. Health Information and Libraries Journal, 27(2),114-

122.

Record Keeping

24

PRISMA Statement checklist includes:• Databases with dates of coverage, date last searched,

platform/provider

• Who developed and conducted the search

• Supplementary methods: hand searches, citation searches, snowball searches, contacting known researchers

• Full electronic search strategy for at least one database, such that it can be repeated

• Use of hedges or any peer reviewed search strategies

• Additional limitationsMoher. D., Liberati, A., Tetzlaff, J., Altman, D.G., & The PRISMA Group. (2009). Preferred Reporting Items for

Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Medicine 6(6), e1000097. doi:10.1371/journal.pmed1000097

25

26

27

Citation Searching

28

Supplements subject search

Across disciplines

Can be used for books or book chapters

“Safety net” – can confirm a comprehensive strategy

3 citation databases offer overlapping content: Web of Science (“Science Citation Index”) – most multisciplinary

Scopus

Google Scholar – can lead to grey lit

Search all 3 if possible, using a few key referencesKloda, L.A. (2007). Evidence Summary: Use Google Scholar, Scopus and Web of Science for comprehensive citation tracking. Evidence Based Library and Information Practice 2 (3),87-90.

Managing References

29

Citation Software (e.g., EndNote, RefWorks)

• Exporting from databases (filters)

• Removes duplicates

• Search for full-text articles online

• Groups (folders)

• Annotations

• Citations / references in manuscript

Systematic Review Software(e.g., SRdistiller, EPPI-Reviewer)

Librarians and Syntheses

30

What roles can librarians play in supporting systematic reviews?

To fulfill these roles, what skills and knowledge do librarians need?

How would you go about acquiring these skills?

Systematic Reviews: Library Guide

31

http://wikisites.mcgill.ca/systematicreview

Health Librarians’ Competencies

32

Systematic Reviews with Librarian (Co-)Authors

33

Agencies Requiring or Recommending a Librarian

34

Canadian Institutes of Health Research (CIHR) Knowledge Synthesis Grant

Agency for Healthcare Research and Quality (AHRQ)

Cochrane Collaboration Programme Grant

Role of Librarians & Syntheses in the Literature

35

PubMed from 1995 to 2015…

Contact

lorie.kloda@mcgill.ca

slideshare.net/lkloda