Literature Reviews & Systematic Reviews

Dr. Valerie IrvineCo-Director, TIE Research LabUniversity of Victoria, Canadahttp://tie.uvic.cavirvine@uvic.caTwitter: _valeriei

Today’s Session

• Assumption: • Students have prior knowledge of the

literature review process• Brief review of literature review provided• Overview of systematic reviews

Types of Reviews

• Traditional narrative literature reviews• Systematic reviews• Meta-analysis

Reference: Kathy Hornby, DMD, MLS, CertMINF (2005). Systematic Reviews session.

Common Elements

• Retrospective• Subject to bias and error• Should address an important question• Identifies gaps in current knowledge• Provides information on the outcome of

interest

Literature Review

• Helps round out the problem• Identifies what has been done• Identifies potential methodology and

procedures for your study• Helps develop theoretical structure• Helps stimulate inductive reasoning

5

Literature Review - Steps

• Write problem statement• Consult secondary sources

• Look for recent reviews of basic problem• Determine descriptors• Search primary sources

• Start with current literature and move backward

6

Lit Review – Search Engines

• DATABASES• PsychLit, PsychInfo, Web of Science, Eric,

etc.• Depending on your discipline area

• INTERNET• Google, etc.

7

Lit Review - After the Search

• List may be huge and difficult to organize• Create a working bibliography• Use the abstract to help delimit your

search• Computer software tools to help

• RefWorks, EndNote, etc.

8

Reading Research

• Weight by• Peer-reviewed journals• Book chapters• Published abstracts/proceedings• Dissertation• Internet sources/non-refereed material

9

Impact Factor

• Represents the average number of times all articles in a journal are cited during one year

• Not a perfect system because biased with:• small fields of study• large numbers of journal in a field of study• important work that is not cited in scientific

journals10

Impact Factor cont.

• Better to try to roughly classify journals in a given area by A, B, C

• Consider rejection rate• You are the final judge of the merit of the

paper

11

Critical Reading

• Develop a table of findings to help look for similarities and differences between studies

• Next slide shows components to record….

12

Components of Chart

• Statement of problem

• Participant characteristics

• Instruments• Procedures• Variables (IV &

DV)

• Treatment (if an experimental study)

• Design and analyses• Findings• Questions raised for

future studies

13

Systematic Reviews

• A systematic review is a literature review focused on a single question which tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question. http://en.wikipedia.org/wiki/Systematic_review

14

Systematic Reviews• Systematic reviews are generally regarded as

the highest level of evidence

• An understanding of systematic reviews and how to implement them in practice is becoming mandatory for all professionals involved in the delivery of health care. http://en.wikipedia.org/wiki/Systematic_review

15

Systematic Reviews

• Scientific investigations• Primary study is the “subject”• Strategies to limit bias and error• Comprehensive search• Explicit reproducible criteria• Data synthesis and interpretation of

results16

Differences Between Narrative & Systematic ReviewsFeature Narrative Systematic

Question Broad Focused

Sources/Search Not specified, potentially biased

Comprehensive sources & explicit search strategy

Selection Not specified, potentially biased

Criterion-based selection, uniformly applied

Appraisal Variable Rigorous critical appraisal

Synthesis Often qualitative Quantitative (but not a statistical synthesis)

Inferences Sometimes evidence-based Usually evidence-based

17

Cook, DJ, Mulrow, CD, Haynes, RB (1997). Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions. Annals of Internal Medicine, 126(5), 376-80.

Weaknesses in Systematic Reviews• Heterogeneity of the studies

• Designs, populations, interventions, outcomes• Uncertainty of whether all relevant studies are

included• Searching difficulties, publication bias (for

significance)• Uncertainty as to whether to include

unpublished results?• Quality assessment issues – assign weights? 18

STEPS…

19

The question

• The question must be focused• Includes

• Population, problem, setting• Intervention• Outcome

20

Survey the review literature

• Check to see if a review has already been done on your question

• Re-define and focus your question, if necessary

• Helps to understand the literature in the area prior to conducting the systematic review

21

Develop the protocol

• The protocol is your research plan and should include elements as defined by the collaboration group (campbell or cochrane)

• Sample:• http://www.campbellcollaboration.org/

systematic_reviews/index.php?page=3

22

Locating information

• Start with the question• Decide on sources• Develop search strategies• Keep records

23

Managing information

• Use bibliographic software• Consider systematic review software

• DistillerSR• EGOR• Etc.

24

Sources• Databases of primary

research articles• Hand searching core

subject journals• Checking cited

references• Conference

proceedings

• Contacting researchers & experts for recommendations

• Dissertations• Gray Literature• Web of Science

(citation tracking)• Internet

25

Gray Literature Def’n & Sources

• http://en.wikipedia.org/wiki/Gray_literature• Technical reports, white papers, etc.• OpenSIGLE system for Information on Grey

Literature in Europe• Self-archiving (see John Willinsky)

26

Rationale for Thoroughness

• Missing studies can affect the results• Publication bias

• Tendency for studies with positive results to get published

• Tendency for studies with no significant difference or negative results to get rejected

27

Government Priorities

• “evidence-based decision-making”

28

Tour Protocols & Steps• Campbell Collaboration (Education focus)

http://www.campbellcollaboration.org/http://www.campbellcollaboration.org/resources/resource_center.php

• Cochrane Collaboration (Health focus)• http://en.wikipedia.org/wiki/Cochrane_Collaboration• http://www.cochrane.org/

• Note the “podcasts” as audio summaries of reviews

29

Sample Reviews

• See Campbell Collaboration Website• Click on Review Library

http://www.campbellcollaboration.org/library.php

• Select sample on cyberbullying

30

Questions?

31