Supporting authors of

systematic reviews.

Margaret Anderson

Trials Search Co-ordinator m.anderson@qub.ac.uk

Cochrane Developmental, Psychosocial and Learning Problems Group

Queen’s University Belfast

Systematic reviews in the social sciences

0

100

200

300

400

500

600

700

10

1 861

214

614

IBSS records with "systematic review" in title or abstract.

No of IBSS records

Systematic review

“A review of a clearly formulated question that uses

systematic and explicit methods to identify, select, and

critically appraise relevant research, and to collect and

analyse data from the studies that are included in the

review. Statistical methods (meta-analysis) may or may not

be used to analyse and summarise the results of the

included studies.”

Cochrane community website http://community.cochrane.org/glossary/5#letters accessed 28 January 2015

Cochrane worldwide

• More than 22,400 active Cochrane authors worldwide

attached to one or more

• 53 Review Groups

• Publish protocols and reviews in Cochrane Database of

Systematic Reviews (> 8,500)

• Well defined subject scope

• Editorial base

• Typically a Co-editor, Managing Editor, Trials Search Co-

ordinator

• Editorial board

• Subject , methods and statistical expertise

• External peer reviewers

Cochrane Developmental, Psychosocial

and Learning Problems Group

• Funded by HSC Northern Ireland

• Based at Queen’s University Belfast (since 2010)

• Editorial base• Co-editor

• Managing Editor

• Assistant Managing Editor

• Trials Search Co-ordinator

• 992 authors

• 174 reviews

• 32 protocols

• 17 registered titles

Review Life cycle (18 months to 2 years)

Title registered and accepted by Cochrane group

Write the protocol

Feedback and editorial approval

Publish the protocol in CDSR

Run the searches

Write the review

Feedback and editorial approval

Publish the review in CDSR

Update the searches

Incorporate new data

Feedback and editorial approval

Publish review update in CDSR

Overview of review process: TSC input

• Review Proposal stage ( Title Registration)

• Protocol stage

• First draft – liaise with authors search strategy

• Check subsequent drafts for revisions

• Run searches or check review teams progress

• Deliver records and summary of search

• Review stage – check all search related sections

Overview of review process: TSC input

• Top-up searches

“Rerun or update searches for all relevant databases

within 12 months before publication of the review or

review update, and screen the results for potentially

eligible studies”*

C37 Methodological Standards for the Conduct of Cochrane Intervention

Reviews http://editorial-unit.cochrane.org/mecir

Title registration

• Editorial discussion

• Check The Cochrane Library for similar

protocols or reviews

• Author team

• Who is developing the strategy and running searches?

• Search expertise?

• library access?

• advice from an information professional?

Guidance and standards

• MECIR Project http://editorial-unit.cochrane.org/mecir

• Methodological standards for the conduct of new Cochrane reviews

• Methodological standards for the reporting of new Cochrane intervention reviews

• Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0[updated March 2011]

• http://handbook.cochrane.org/

• Handbook for DTA Reviews (work in progress)

• http://srdta.cochrane.org/handbook-dta-reviews

Protocol development

“Searches for studies should be as extensive as possible…” *

• Comprehensive list of sources (CDPLPG average is 12)

• Include mandatory and highly desirable sources**

• Published and unpublished sources

• Reference lists and personal contacts

yet also need to

• Avoid duplication

• Ensure list is realistic

• Access

• Review Team’s capacity

* C24 to C31 Methodological Standards for the Conduct of Cochrane Intervention Reviews

http://editorial-unit.cochrane.org/mecir

Protocol: developing a search strategy

• Work from the protocol (not just the title)

• Is the intervention clear?

• What is the population (condition, age group)?

• Study types?

• RE-read the protocol!• It might change after editorial feedback

Protocol: developing a search strategy

• Engaging with the authors

• Try to provide encouraging feedback

• Ask them to suggest search terms (older

terms, regional terms)

• Check understanding (mine and authors)

• Seek clarification from authors

• Test drive the strategy

• Use references from other reviews

• Ask authors for likely eligible studies

Features of the core search strategy

• published as part of the Protocol

• structured using main review concepts

• population AND intervention and RCT filter

• Controlled vocabulary

• Comprehensive free-text terms

• Balance sensitivity and precision

Item C32 Methodological Standards for the Conduct of Cochrane Intervention

Reviews http://editorial-unit.cochrane.org/mecir

Cochrane Handbook Sections 6.4.2; 6.4.4; 6.4.7

Developing the core strategy

• Choose core database

• Sort out the structure

(Concept 1 AND Concept 2 AND Concept 3)

• Find controlled thesaurus terms

• Begin with the most obvious term and examine

broader and narrower terms .

• Work backwards from a known study

• Look at strategies from other reviews

• Test free text words and phrases one at a time

• Use a separate search line for each one

• What happens if …

Preparing to run the searches

• Agree who is doing what and record the decision

• Discuss reference manager software

• Agree when to start (searches have a “shelf life”)**

• Agree who will receive the records and co-ordinate

screening

C37 Methodological Standards for the Conduct of Cochrane Intervention Reviews

http://editorial-unit.cochrane.org/mecir

Cochrane Handbook Sections 6.6.1

Preparing to run the searches

• Customise the core strategy for other databases

• Check appropriate syntax and controlled

vocabulary

C33 Methodological Standards for the Conduct of Cochrane Intervention Reviews

http://editorial-unit.cochrane.org/mecir

Cochrane Handbook Sections 6.6.1

Managing expectations: new CDPLPG reviews 2010-

2012

Maximum= 31879

Minimum =27

Number of reviews= 40

Documenting the searches

• Record enough detail so search can be

replicated*

• Database name, provider and dates of

coverage

• Full name and URLs of websites

• Save exact search strategy for each source

• Collate search strategies as you go

*C36 Methodological Standards for the Conduct of Cochrane Intervention Reviews

http://editorial-unit.cochrane.org/mecir

Cochrane Handbook Sections 6.6.1

Managing the records

• Downloading records

• Label downloaded records with database, search date and number of records)

• Importing into reference manager

• Global edit to add database name

• Make separate RIS files for each source

• Keep a master copy of all records!

• Deduplication

• Keep deduplication settings conservative

• Move duplicates to a separate file

• Advise authors to expect more duplicates

Reporting the searches

• Abstract (word limited)

• Search date

• the earliest date of the most recent set of searches

• must be within 12 months (and preferably 6

months) of the publication date

• List of databases

• Indicative -name the main ones then “and 5 others”)

R6 Methodological Standards for the reporting of new Cochrane Intervention Reviews

http://editorial-unit.cochrane.org/mecir

Reporting the searches

• Search methods

• List all sources

• Include provider/dates of coverage

• Issue/version number (e.g. CENTRAL 2014

Issue 12)

• Full name and URL of websites

• Latest searches

• Date of latest searches

R34 R35 Methodological Standards for the Conduct of Cochrane

Intervention Reviews http://editorial-unit.cochrane.org/mecir

PRISMA study flow diagram

Liberati A, Altman DG, Tetzlaff J, Mulrow C, et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta-

Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med 6(7): e1000100.

doi:10.1371/journal.pmed.1000100

http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.1000100

Reporting the searches

• Study flow diagram (PRISMA)

• Ensure numbers add up

• Do numbers of records and studies tally?

• Check numbers of included and excluded studies

• Check studies used in any meta-analysis

• Appendix with search strategies

• One for each source

Updates

• Generally expect to update every 2 years

• Monitor downloads

• Monitor citations (journals, policy documents clinical

guidelines (NICE, WHO)

• Be aware of plans to update guidelines

• Monitor Altmetrics*

• Update process begins with searches

• Chance to improve/update search strategy

*http://exchanges.wiley.com/authors/faqs---altmetrics_340.html

Information specialist input

• Awareness of sources

• Advise on coverage

• Access

• Limitations

• Searching expertise

• Exploiting thesaurus terms

• Adapting searches to different databases

• Downloading and managing records

• Reporting

• Documenting searches

• Referencing

• Monitoring impact

Other systematic reviews are available…

• Journals

• Not-for-profit

organisations

• Commercial

organisations

• Research

calls/commission

s