v i r g i n i a c o m m o n w e a l t h u n i v e r s i t y using study design to determine the...
TRANSCRIPT
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Using Study Design to Determine the Level of
EvidenceCritical Appraisal Skills depend on
understanding the relationship between study design and levels of evidence
Al Best, PhDPerkinson 3100B
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Evidence-Based Dentistry
Derek Richards "Is it worth reading this paper?" Evidence-based Dentistry (2000) 2, 50-52. Integration of:
Best Evidence Available
Clinical Judgment Patient Values/Circumstances
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Critical Appraisal Skills
Are the results of the study valid? What are the results? Will the results help locally?
Best Evidence Available
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Review
Yeudall and Gordon– EBD Methodology / Practice– PICO PECO questions
Types of Research and Study Design– Levels of Evidence
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
1. ASK an answerable question
2. SEARCH for the evidence
3. EVALUATE the evidence
4. APPLY the evidence
5. EVALUATE the outcome
ASK
ACQUIRE
EBD Methodology / Practice
APPRAISE
APPLY
ASSESS
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Figure The evidence pyramid displays the quality of evidence according to type of study. “Quality” refers to the likelihood of predicting what would occur in one’s own practice (and the least probability of bias). Guidelines explicitly based on this evidence pyramid offer clinicians a simplified mechanism for obtaining and potentially using the knowledge identified in this pyramid. Conversely, the lowest level of evidence, with the least likelihood of predicting what would occur in one’s practice (and the highest probability of bias), can provide useful background information (such as laboratory and animal studies, cross-sectional epidemiologic studies, and expert opinion or narrative reviews). “Filtered information” is so-called secondary research. These reports systematically search for, critically appraise, distill and present the results of primary research, called here “unfiltered information.” Adapted from Harvey Cushing/John Hay Whitney Medical Library, Yale University.
Niederman, R. et al. JADA 2011;142:4:364-367
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Niederman, JADA 2011
“a significant evolution from – intuition-based care, – to experience-based care, – to evidence-based care.
The intuition, experience and now evidence on which we base our care are simply steps on the evidence pyramid…”
pubmed/21454838 or JADA
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Niederman, JADA 2011
“Those who are proponents of experience-based care … place themselves and their patients at risk, clinically and legally.
Clinically, it is unlikely that the majority of clinicians are implementing the current best evidence.”
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
"The Toothpuller", ascribed to Caravaggio (1571-1610).
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
My goals for you
Be able to answer four questions: Based on the study design, what is the
level of evidence? How were threats to validity addressed? Based on the goals of the study, How do
you describe the results? To justify the conclusions, were
comparisons done appropriately?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Example
A 63 year old male patient has a heart murmur caused by an aortic valve prolapse and is scheduled for tooth cleaning.
Do you prescribe an antibiotic to be taken before dental treatment?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Google: does tooth cleaning cause bacteremia?
Bacteremia can have several consequences. The immune response to the bacteria can cause sepsis and septic shock, which has a relatively high mortality rate.
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Meta Cognition
“a significant evolution from – intuition-based care, – to experience-based care, – to evidence-based care.
The intuition, experience and now evidence on which we base our care are simply steps on the evidence pyramid…”
Niederman, R. et al. JADA 2011;142:4:364-367
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Example
A 64 year old male patient has a heart murmur caused by an aortic valve prolapse and is scheduled for tooth cleaning.
Do you prescribe an antibiotic to be taken before dental treatment?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
A 63 year old male patient has a heart murmur caused by an aortic valve prolapse
and is scheduled for tooth cleaning.Do you prescribe an antibiotic to be taken
before dental treatment?PICO / PECO
–Population / Participants / Problem
–Intervention / Exposure–Comparator / Control–Outcome
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Appraise:Forner, JClinPerio2006
pubmed/16677328
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Appraise: van der Meer, Arch Inter Med. 1992
pubmed/1520053
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Appraise: van der Meer, Lancet 1992
pubmed/1346008
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Appraise: Lockhart, Circulation 2008
pubmed/18541739
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Appraise: Friend, USA Today
The most common strain of bacteria in dental plaque can cause blood clots that induce heart attacks when they escape into the bloodstream, researchers said Monday.
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
One glossary: http://ktclearinghouse.ca/cebm/glossary/
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Which Study Design?
Focus on– Selection of participants– Interventions compared– Flow of time– Outcomes measured
Also– Purpose– Advantages & Disadvantages
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Case Report, Case Series
Characteristics: The description of interesting observations. No controls.
Purpose: to describe. Generate hypotheses.
Advantages: Simple; easy. Disadvantages: Not conclusive. Bias: Findings due to chance?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Cross-sectional Study AKA: a survey, a poll Characteristics: data collected
on subjects at one point in time. Often looks at prevalence.
Purpose: What is happening right now?
Advantages: Short duration. Disadvantages: Short duration.
Only looks at “here and now,” not across time. Bias: The inclusion/exclusion of subjects is critical.
Representativeness: The sample is representative of what?
Source: Figure 2-2, Dawson & Trapp. Basic and Clinical Biostatistics
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Longitudinal Study Types (usually) Retrospective Case-Control (usually) Prospective Cohort Study (occasionally) Retrospective Cohort Study
Source: Figure 2-5, Dawson-Trapp
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Case-Control Study Characteristics: Begins with an
outcome of interest, then look back to detect risk factors.
Purpose: to explain outcomes by evaluating previous events. What happened?
Advantage: Good for rare diseases or for those that develop over a long time period.– Quick and easy.
Disadvantage: The largest number of possible biases and errors. Strongly depends on high-quality historical records.
Bias: The major problem is the selection of the control group.
Source: Figure 2-1, Dawson Trapp
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Prospective Cohort Study Characteristics: Follow a
cohort—a group of individuals with something in common—across time. Note that some are exposed to the risk factor.
Purpose: What will happen?
Advantage: Studies the normal course of events.
Disadvantage: Takes time Bias: Generally less subject to bias than a case-
control study.Source: Figure 2-3, Dawson Trapp
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Retrospective Cohort Study AKA: historical cohort
study Characteristics: Attempt
to follow a cohort back intime. Try to determine the risk factor.
Purpose: How did disease happen?
Advantage: Studies the normal course of events.
Disadvantage (major): depends on near perfect records.
Bias: Perhaps as biased as a case-control study.Source: Figure 2-4, Dawson Trapp
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Experimental Studies
In vitro (eg, cellular assays) Animal models Humans (AKA Clinical Trials)
Note: Studies with concurrent controls are highest quality. Historic controls are problematic.
We will talk about random allocation and blinding/masking next time.
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Randomized Control Trial (RCT) Characteristics: Specific
inclusion of subjects, controlled exposure to risk factor or intervention,defined outcome measure(s)
Purpose: Does this cause that?
Advantages: Masked interventions, Blind assessment, Gold standard
Disadvantages: Difficult, expensive, time consuming
Bias: Are subjects who consent typical?Source: Figure 2-6, Dawson-Trapp
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Cross-over Trial Characteristic: All subjects get both
treatments
Source: Figure 2-7 Dawson Trapp
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Systematic Reviews
Narrative review Meta-Analysis Practice Guidelines
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
DecidingEvidence-Based GuidelineSystematic Review
Yes
No
Exposure or intervention controlled by the investigator
Yes
No
Subjects followed over time
Yes
No
Subjects selected according to outcome
Yes
No
Guideline or Systematic Review
Experiment, maybe a RCT
Cohort Study
Case-Control Studyor Case Series
Opinion, otherDescriptive or AnalyticPICOPECO
PO
Cross sectional / Survey
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Study Design ≈ Level of EvidenceGuideline or
Systematic Review
Experiment, maybe a RCT
Cohort Study
Case-Control Studyor Case Series
Opinion, other
Prospective CohortRetrospective Cohort
E-B Guideline
Systematic ReviewRCT
Experimental trial
Case-Control Study Case Series
Narrative, Opinion
Animal, In Vitro
E-B Guideline
Cross sectional / Survey
Article Synops
es
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
My goals for you
Be able to answer four questions: Based on the study design, what is the
level of evidence? How were threats to validity addressed? Based on the goals of the study, How do
you describe the results? To justify the conclusions, were
comparisons done appropriately?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
A mother asks how often her family should come in for a dental checkup for the prevention of disease.
How Often?
PICO / PECO–P–I E–C–O
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Mettes D: Insufficient evidence to support or refute the need for 6-monthly dental check-ups EBD
2005(6):62-63 Results: “Only one study (with 188 participants) was included in this review and was assessed as having a high risk of bias. This study provided limited data for dental caries outcomes (dmfs/DMFS increment) and economic cost outcomes …
Conclusions: “There is insufficient evidence from randomised controlled trials (RCT) to draw any conclusions regarding the potential beneficial and harmful effects of altering the recall interval between dental check-ups. …
pubmed/16184154 2013 Cochrane review: same conclusions
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
In-class Exercise
Cross-sectional Survey
ProspectiveCohort
Case-Control
RCTRetrospective
Cohort
TODAY
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
A mother asks how often her family should come in for a dental
checkup for the prevention of disease.
How Often?PICO–P children–I 6-month recall–C 12-month recall–O Mean dmft/DMFT
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Groups
4-5 people One reporter Take 5 minutes
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Case-Control Study
Selection of participants InterventionMeasurement variables
–What is measured/recorded?–When?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Cross-sectional Survey
Selection of participants InterventionMeasurement variables
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Prospective Cohort
Selection of participants InterventionMeasurement variables
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
Randomized Control Trial
Selection of participants InterventionMeasurement variables
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
My goals for you
Be able to answer four questions: Based on the study design, what is the
level of evidence? How were threats to validity addressed? Based on the goals of the study, How do
you describe the results? To justify the conclusions, were
comparisons done appropriately?
V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y
“After much research and deliberation, we feel we cannot ignore what a parent or conspiracy theorist feels “in their gut”. There are just too many anecdotes and too many people buying untested alternative health products to ignore this any longer,” explained spokesman Dr. Harold Rami.Homeopaths, Naturopaths, Chiropractors and anti-vaxxers the world over are celebrating this as a huge victory.“Even though my son was showing signs of autism before he got vaccinated, I know in my gut and in my heart that it was still the vaccines that caused it,” said mother and anti-vaccine advocate Cheryl Jones.“This is a big win for us,” said Naturopath and homeopathy dispenser, Paul Theroult. “I have seen it many times. I sell my patients a homeopathic remedy – for say the common cold- and then bam, they are cured within one to two weeks. There is no science backing up my claim that the homepathic pill cured their cold, but in my gut I just know it did.”
“I just know” replaces systematic reviews at top of evidence
pyramid
thespudd.com