Systematic reviews - how to check you're doing all the right things,

and how the Medical Library can help

Isla KuhnMedical Librarian, Cambridge University Library IPH 8th July 2015

CAMBRIDGE UNIVERSITY LIBRARYMEDICAL LIBRARY

Why we need systematic reviews

Adapted from Am J Clin Nutr January 2013 vol. 97 no. 1 127-134 http://ajcn.nutrition.org/content/97/1/127.full.pdf+html And https://twitter.com/HowardNWhite/status/614290048410714112

How are Systematic Reviews different?

Traditional reviews

•Individual opinions based on haphazardly selected data rather than comprehensive systematic assessment

•Inconsistent, prone to error and unconvincing

Systematic reviews

•Prepared as methodically and as carefully as a piece of primary research

•Describe how trials were identified, selected and evaluated

•Checked and verified

• Rapid review: narrow, quick search and assessment of very specific question

• Scoping review : assessment of potential size/scope of available literature

• Integrative review: includes qualitative/quantitative/ theory

• Meta ‐ analysis: quantitative synthesis

• Umbrella review: review of systematic reviews

Question Scoping search

Full search

Title and abstract

screening

Full text screening

Additional searching

Included references

agreed

Data extraction

Synthesis

Writing up

Reference management

Protocol

Update search

Full text retrieval

Formulating a question

• Participants

• Interventions (Exposure)

• Comparisons

• Outcomes

• [Time or Type of study design?]

• But could use:

• PHICCO, SPICE, PIECE, WWH, ProPheT

Why is the question important?

• Keystone of a systematic review protocol

• Prompts you to considier inclusion and exclusion criteria

• Helps build up a search strategy

• Gets you to think about which data to extract, and what quality criteria are important

• Allows authors to decide on their analysis now

Make your question FINER

Feasible Interesting Novel Ethical Relevant• adequate # of

studies• Adequate

technical expertise

• Affordable in time and money

• Manageable in scope

• Who will be interested in the answer?

• Confirms / refutes previous findings

• Extends previous findings

• Provides new findings

• Time and money spend on project an appropriate use?

• To advance scientific knowledge

• To clinical /public health policy

• To future research directions

Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing clinical research. 3rd ed. Lippincott Williams and Wilkins; 2007

Bias

Statistically significant ‘positive’ results are:

•more likely to be published

publication bias

•more likely to be published rapidly

time lag bias

•more likely to be published in English

language bias

•more likely to be cited by others

citation bias

Publication bias | an example

• Systematic review of reboxetine, a third generation anti-depressant

• 13 trials, published and unpublished data

• (searched: Bibliographic databases (Medline, Embase, PsycINFO, BIOSIS, and Cochrane Library), clinical trial registries, trial results databases, and regulatory authority websites)

• 74% of patient data previously unpublished

• Reboxetine is “overall an ineffective and potentially harmful antidepressant”

• Contradicts findings of previous reviews which considered only published data

Eyding et al, BMJ 2010 http://www.bmj.com/content/341/bmj.c4737

What about your bias? Impact on the search strategy

• Very specific characteristics

• Duration of breastfeeding and intelligence

• Judgements

• Poor family functioning, harsh parenting and bullying behaviour

• Looking for evidence to reinforce

• Does eating broccoli help in the prevention of cancer?

Open Data and Data Management

• BMJ Tamiflu campgaign http://www.bmj.com/tamiflu

• AllTrials http://www.alltrials.net/

• Research Data Management at Cambridge http://www.data.cam.ac.uk/

Trials registers

• UK Clinical Trials Gatewayhttp://www.ukctg.nihr.ac.uk/

• Research Councils Gateway to Researchhttp://gtr.rcuk.ac.uk/

• ClinicalTrials.govwww.clinicaltrials.gov

• International Clinical Trials Registry Platform (WHO) http://www.who.int/ictrp/en/

• And many more at: https://delicious.com/ilk21/york_session

www.data.cam.ac.uk

• Creating your data

• Organising your data

• Accessing your data

• Looking after and sharing your data

Question Scoping search

Full search

Title and abstract

screening

Full text screening

Additional searching

Included references

agreed

Data extraction

Synthesis

Writing up

Reference management

Protocol

Update search

Full text retrieval

Tools that might help

• http://systematicreviewtools.com/

• http://prisma.thetacollaborative.ca/

• RIGOR – the spreadsheet to end all spreadsheets

• http://guides.library.tamu.edu/content.php?pid=188082&sid=2666018

Search Strategies

#1 Accidents/ #2 exp Accidents, Traffic/ #3 exp Accident Prevention/ #4 exp Motorcycles/ #5 motor-cycl* or motorcycl* or motor-bik* or motorbik* or scooter* or moped* or moto

or motocycl* or motocicl* #6 trauma* or injur* or fatal* or accident* or crash* or prevent* or collide* or collision* #7 #1 or #2 or #3 or #4 or #5 or #6 #8 head near protect* #9 head near shield* #10 helmet* #11 exp Head Protective Devices/ #12 #8 or #9 or #10 or #11 #13 #7 and #12 

http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD004333.pub3/abstract

Search Filters

• InterTASC

• SIGN

• Pubmed clinical filters

From topic to search: same strategy, different words

• http://www.sign.ac.uk/methodology/filters.html Observational Study Filters

Medline (via OVID)1 Epidemiologic studies/ 2 Exp case control studies/ 3 Exp cohort studies/ 4 Case control.tw. 5 (cohort adj (study or studies)).tw. 6 Cohort analy$.tw. 7 (Follow up adj (study or studies)).tw. 8 (observational adj (study or studies)).tw. 9 Longitudinal.tw. 10 Retrospective.tw. 11 Cross sectional.tw. 12 Cross-sectional studies/ 13 Or/1-12

Embase (via OVID)1 Clinical study/ 2 Case control study 3 Family study/ 4 Longitudinal study/ 5 Retrospective study/ 6 Prospective study/ 7 Randomized controlled trials/ 8 6 not 7 9 Cohort analysis/ 10 (Cohort adj (study or studies)).mp. 11 (Case control adj (study or studies)).tw. 12 (follow up adj (study or studies)).tw. 13 (observational adj (study or studies)).tw. 14 (epidemiologic$ adj (study or studies)).tw. 15 (cross sectional adj (study or studies)).tw. 16 Or/1-5,8-15

From topic to search: same strategy, different words

• http://www.sign.ac.uk/methodology/filters.html Observational Study Filters

Medline (via OVID)1 Epidemiologic studies/ 2 Exp case control studies/ 3 Exp cohort studies/ 4 Case control.tw. 5 (cohort adj (study or studies)).tw. 6 Cohort analy$.tw. 7 (Follow up adj (study or studies)).tw. 8 (observational adj (study or studies)).tw. 9 Longitudinal.tw. 10 Retrospective.tw. 11 Cross sectional.tw. 12 Cross-sectional studies/ 13 Or/1-12

Medline (via Pubmed)#1 “Epidemiologic studies”[Mesh]#2 “case control studies”[Mesh]#3 “cohort studies”[Mesh]#4 Case control[Text Word] #5 (cohort stud*))[Text Word] #6 Cohort analy*[Text Word]#7 (Follow up AND (study or studies))[Text Word] #8 (observational AND (study or studies))[Text Word]#9 Longitudinal[Text Word]#10 Retrospective[Text Word]#11 Cross sectional[Text Word]#12 “Cross-sectional studies”[Mesh]#13 #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7 OR #8 OR #9 OR #10 OR #11 OR #12

Syntax – can you code your search?

OvidSP EBSCOHost PubMedBoolean AND, OR, NOT

Thesaurus Exp “…”/ DE “..+”MH “ ..+”

“…”[MeSH]

Title/Abstract ().ti,ab. TI () or AB () [tiab]

Proximity () ADJn () (() Nn ()) (() AND ())

Optimising your strategy

• Test your strategy

• Thesaurus NOT (title/abstract)

• (Title/abstract) NOT (Thesaurus)

• What terms are you missing?

Too “many” or too “few”

Increase Sensitivity

-Drop less important elements-Add extra words (thesaurus NOT ti/ab)-Generalise specific elements (take one thesaurus term “up”)-Explode thesaurus terms-Broaden/remove proximity (change ADJ/N to AND)-Truncation *

Increase Specificity

-Remove general terms – make the thesaurus terms more specific -Do not explode (or do so selectively)-Add element-Narrow/add proximity (change AND to ADJ/N)-Remove truncation or lengthen wordstem-Add filters (eg for study design)

But only if you can afford to potentially miss relevant papers

Test your strategy

Strategy for Self-management interventions for epilepsy in people with learning disabilities

(evaluat*[Title/Abstract] OR validat*[Title/Abstract] OR feasib*[Title/Abstract] OR apprais*[Title/Abstract] OR judg*[Title/Abstract] OR measur*[Title/Abstract])

AND  (knowledge[Title/Abstract]) OR ("self care"[Title/Abstract] OR selfcare[Title/Abstract])) OR (self-assessment[Title/Abstract] OR "self assessment"[Title/Abstract] OR self-exam*[Title/Abstract] OR "self examin*"[Title/Abstract])) OR educat*[Title/Abstract]) OR "disease management"[Title/Abstract]) OR attitude[Title/Abstract]))) OR (("Risk Assessment"[MeSH Terms]) OR ((((((("Health Education"[Mesh]) OR "Health Knowledge, Attitudes, Practice"[Mesh]) OR "Attitude to Health"[Mesh]) OR "Self Care"[Mesh]) OR "Disease Management"[Mesh]) OR "Self-Assessment"[Mesh]) OR "Self-Examination"[Mesh])

AND ( (learning disorder*[Title/Abstract]) OR learning disab*[Title/Abstract]) OR intellectual disab*[Title/Abstract]) OR retard*[Title/Abstract]) OR cognitive impair*[Title/Abstract])) OR ("Learning Disorders"[Mesh]) OR "Intellectual Disability"[Mesh])

AND ("Seizures"[Mesh]) OR seizure*[Title/Abstract]) OR "Epilepsy"[Mesh]) OR epilep*[Title/Abstract])

Why doesn’t it capture this Cochrane review?

S14 S1 AND S7 AND S10 AND S13S13 S11 OR S12S12 (MH "Seizures+") OR (MH "Epilepsy+")S11 AB ( seizure* or epilep* ) OR TI ( seizure* or epilep* )S10 S8 OR S9

S9(MH "Learning Disorders+") OR (MH "Intellectual Disability+") OR (MH "Mentally Disabled Persons")

S8AB ( learning disorder* or learning disab* or intellectual disab* or retard* or cognitive impair* ) OR TI ( learning disorder* or learning disab* or intellectual disab* or retard* or cognitive impair* )

S7 S2 OR S3 OR S4 OR S5 OR S6S6 AB behav* N3 outcome* OR TI behav* N3 outcome*S5 AB ( (self-manage* or (self N3 manage*)) ) OR TI ( (self-manage* or (self N3 manage*)) )S4 TI manage* N3 condition* OR AB manage* N3 condition*

S3

(MH "Risk Assessment+") OR (MH "Health Education+") OR (MH "Health Knowledge, Attitudes, Practice") OR (MH "Attitude to Health+") OR (MH "Self Care+") OR (MH "Disease Management+") OR (MH "Self-Assessment") OR (MH "Self-Examination+")

S2

TI ( knowledge or "self care" or selfcare or self-assessment or "self assessment" or self-exam* or "self examin*" ) OR AB ( knowledge or "self care" or selfcare or self-assessment or "self assessment" or self-exam* or "self examin*" )

S1TI ( evaluat* or validat* or feasib* or apprais* or judg* or measur* ) OR AB ( evaluat* or validat* or feasib* or apprais* or judg* or measur* )

Reporting

• PRISMA (formerly QUOROM) preferred reporting items for systematic reviews and meta-analyses

• EQUATOR: Enhancing the Quality and Transparency of Health Research

• MOOSE (Meta-analysis Of Observational Studies in Epidemiology)

• JAMA Instruction for Authors- Systematic Review abstract

Other useful resources

• What is a sytsematic review?

University of Manchester

• Improving efficiency and confidence in systematic literature searching

Wichor Bramer, Erasmus MC, Rotterdam

• Systematic Reviews – LibGuide

• Margaret Foster, Texas A&M University

• Systematic reviews – a “how to guide”

• Isla Kuhn, University of Cambridge Medical Library

The Library can help with….

• Developing your strategy• Identifying key databases• Adapting your strategy for each

database• Reference management• Retrieving all full text• Data management• Open access publishing

Help!

Isla Kuhn

Medical Librarian

University of Cambridge Medical Library

Box 111

Addenbrooke’s Hospital

Thank you.

email: ilk21@cam.ac.uk

twitter: @ilk21

phone: (01223) (3)36750

web: http://library.medschl.cam.ac.uk