Appraising Randomized Clinical Trials and Systematic Reviews

October 12, 2012

Mary H. Palmer, PhD, RN, C, FAAN, AGSF

University of North Carolina at Chapel Hill

Objectives

• Discuss systematic reviews in the context of the pyramid of evidence

• Describe the importance of the CONSORT statement for randomized controlled trials

• Appraise select randomized controlled trials• Appraise select systematic literature reviews • Describe the process of synthesis and

evaluation

Pyramid of Evidence

Appraising Systematic Reviews

Systematic reviews are a compilation of similar studies that address a specific clinical question.

Systematic reviews are not the same as a literature review or narrative review

The process the was used to conduce the review should be explicit

Look for a detailed description of the databases accessed, the search strategies, and the search terms

Clinicians should be able to clearly see which studies were included and which were excluded and how selected studies were assessed

Systematic Reviews• A specific research question is answered.• Detailed inclusion and exclusion criteria are

explicated.• Elaborate and thorough search strategies.• Standardized review protocols that use trained

reviewers (rather than PIs)• Standardized abstracting process for capturing

details• Pre-established quality criteria to rate the value

of the individual studies.

Systematic Reviews

• How is bias eliminated?– Databases accessed– Search strategies used– Search terms used– Years searched – Language used – Unpublished studies included (publication bias)

Systematic Reviews

Randomized controlled trialsPRISMA

Meta-analyses and observational and studiesMOOSE

Meta-Analysis

• Meta-analysis is a systematic review, not all systematic reviews are meta-analysis.

• Quality of studies included is critical. • Meta-analyses pool data from individual

studies– Increased sample size – Pooled statistical results – Definitive conclusions possible

Meta-Analysis (cont’d) Remember, the report should inform clinicians about how

data were extracted from the individual studies Summary Meta-analysis: the statistical approach to synthesizing the

results of two or more studies A relatively new methodology that has become a

hallmark of EBP

As with all methodologies, applicability must be considered

www.consort-statement.org

CONSORThttp://www.consort-statement.org/consort-statement/translations/

CONSORT Translations

•The CONSORT website will announce and host translated CONSORT documents that we are aware of. If you are interested in translating a CONSORT document into another language, please see the CONSORT translation policy.

•CONSORT 2010 Translations•Chinese* •CONSORT Statement PDFCONSORT Explanation & Elaboration PDFChecklist PDF / MS WordFlow Diagram PDF / MS Word

Appraising Randomized Controlled Trials (RCTs)

These studies can accurately establish cause and effect and can inform the efficacy and effectiveness of interventions

Questions related to validity of RCTs Were the subjects randomly assigned to the

experimental and control groups? Was random assignment concealed from the

individuals who were first enrolling subjects into the study?

Were the subjects and providers kept blind to study group?

Appraising the Validity of RCTs

Were reasons given to explain why subjects did not complete the study?

Were the follow-up assessments long enough to fully study the effects of the intervention?

Were the subjects analyzed in the group to which they were randomly assigned?

Was the control group appropriate? Were instruments valid and reliable? Were the subjects in each of the groups similar on

demographic and baseline clinical variables?

Evaluation and Synthesis of Quantitative Studies

Once studies have been selected, they should be melded together into a synthesis upon which to base practice and standards

Individual studies should be evaluated using an evaluation table - see Table 5-8 in your textbook

Synthesis is NOT reporting the findings of study after study; it is combining, contrasting, and interpreting a body of evidence to reach a conclusion

Question

Tell whether the following statement is true or false. Meta-analysis results in evidence that is applicable to a

larger patient population than individual studies.

Answer

False Rationale: In meta-analyses, combining the results of

several studies produces a larger sample size and thus greater power to accurately determine the magnitude of the effect. This does not, however, increase the size of the relevant patient population.

Homework

Find a systematic review or a randomized controlled trial related to your PICOT question.•Write a critical appraisal and send it and a pdf of your article to:

mhpalmer@email.unc.edu

Deadline: October 22, 2012