clinical problem solving an introduction to evidence-based physical therapy basics

123
Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Upload: colin-gregory-robinson

Post on 28-Dec-2015

218 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Page 2: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Lecture overview

• Objectives • EBM skills for practicing physical therapy

– Asking– Acquiring– Assessing – Applying

Page 3: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Objectives

• Define evidence-based physical therapy(EBM) • Explain why we use EBM

– Compare with expert-based physical therapy– How are we misled by:

• Surrogate outcomes • Personal observation • Pathophysiologic reasoning

• Describe the tools of EBP• Construct a well-built clinical question

Page 4: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

What is Evidence-Based physical therapy(EBM)?

“Using the best available evidence for making decisions about health care”

Page 5: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

What is important to read in the medical literature

Things that 1. Have patient oriented outcomes 2. Answer a patient-care question3. Might change your practice4. Are on a topic you have been following5. People are talking about and you want to know more6. You find interesting POEM or DOE

• Patient-oriented evidence that matters vs disease-oriented evidence

Page 6: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Intro”: EBM (I)

“Evidence-based physical therapy(EBM) requires the integration of the best research evidence with our clinical expertise and our patient’s unique values and circumstances”

EBM, 2006, Straus et al

Page 7: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Why learn EBM / EBP?

Page 8: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Good education teaches us to become both producers

of knowledge &

discerning consumers of what other people claim to know.”

Page 9: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Helps you find the truth in face of pharmaceutical marketing

Cal Ripkin, Jr. is not hypertensive and is not taking

PRINIVIL

Cal Ripkin, Jr. is not hypertensive and is not taking

PRINIVIL

Its always in the fine print.

Page 10: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Value of Learning EBM:

Short-Term Trial • A controlled trial of teaching critical appraisal of clinical

literature conducted among medical students• Experimental group of students worked with clinical tutors

who had– Taken course in clinical appraisal– Evaluated specialty-specific articles on diagnostic tests and

treatments

• Control group of students worked with usual clinical tutorsBennett et al. JAMA. 1987;257:2451-2454.

Page 11: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Value of Learning EBM: Short-Term Trial (cont)

• Students in experimental group made greater number of correct diagnostic and treatment decisions and were better able to justify their decisions

• Students in control group were more likely to make incorrect decisions after their tutorial than before it– Students in the control group had become more accepting of

recommendations from authority figures

Bennett et al. JAMA. 1987;257:2451-2454.

Page 12: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

The Patient

• Patient is a 27-year-old woman with severe right lower quadrant pain. – initial peri-umbilical pain x 2 days migrating yesterday to

current site.

• Loss of appetite. No vomiting, diarrhea; no bowel movement

• no known infectious exposure/

suspicious ingestions, or recent travel

Page 13: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Standard medical practice for hot, moist diseases

Page 14: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Louis’ Study of Bloodletting

Day of 1st bleeding

Averages

Number of bleedings

Duration of illness

Page 15: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Pierre Louis (1787-1872)Inventor of the “numeric method” and the “method of bservation”

Discovered in 1828 that patients who were bled did worse than those who weren’t

Page 16: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Many advances in physical therapywith uncontrolled use

• PCN for life-threatening disease

• Insulin for type I diabetes

• Treatment of malignant hypertension

Page 17: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Traditional Guides to Medical Practice

• Pathophysiology and pharmacology– Foundation of medical practice– Do what “makes sense”

• Expert opinion– In training: learning at the bedside from the master clinician– In practice: lectures and seminars with thought leaders

• Clinical experience– Successes, outcomes, and adverse events

in our own practice

Page 18: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

1498 subjects with suppressible arrhythmias post-MI

RANDOMIZED

7.7% 3.0%

Treatment Placebo

Mortality

Cardiac Arrhythmia Suppression Study

Page 19: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Problems With the Traditional Approach• Physiology may not predict clinical response

– Beta-adrenergic blockade in heart failure– Encainide for post-MI arrhythmia– Estrogen replacement for cardioprotection

• Expert opinion– Only as good as the expert– May be affected by biases and conflicts of interest

• Clinical experience– Dramatic clinical experiences may unduly influence our practice

patterns– May not take account of recent medical literature

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-BasedClinical Practice. Chicago, IL: American Medical Association; 2001.

Page 20: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Paradigms of physical therapy

Expert Based Evidence BasedPathophysiological reasoning Clinical Studies

Personal observation Best evidence available

Expert based guidelines Evidence based guidelines

Page 21: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Seven alternatives to EBMHumorous approach from BMJ

Basis Marker Unit

Evidence RCT Odds ratio

Eminence White hair Optical dentistry

Vehemence Level of stridency Decibels

Eloquence Smoothness Adhesion score

Providence Religious fervor International units of piety

Diffidence Level of gloom Sighs

Nervousness Litigation phobia level

Bank Balance

Confidence* Bravado No sweat

BMJ 1999;319:1618-1618

Page 22: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Integrates Evidence With

• Clinical expertise– Experience – Judgment

• Patient values and preferences• Quality of life • Costs • Other important factors

Page 23: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Best Available Evidence

Clinical Expertise

Patient Values and Preferences

Quality of Life

Costs

Page 24: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Focus: Treatment & Diagnosis

Page 25: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

The Patient

• Patient is a 27-year-old woman with severe right lower quadrant pain.

– initial peri-umbilical pain x 2 days migrating yesterday to current site.

• Loss of appetite.

• No vomiting, diarrhea; no bowel movement

• no known infectious exposure/ suspicious ingestions, or recent travel

Page 26: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Patient exam

VS BP 120/78 P 16 RR 12 T 98.8

Chest CTA. CV RRR s M/R/G

ABD: NML exam x decreased bowel tones and definite right lower quadrant pain, specifically at McBurney’s point.

no heptomegaly nor splenomegaly (enlarged liver or spleen). She has no rebound pain or involuntary

Page 27: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

The Five “A’s”

1. Ask the right question

2. Acquire the evidence

3. Appraise the evidence

4. Apply the evidence

5. Assess its impact

Page 28: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Concern:

• Case discussion: 27 year old woman with right lower quadrant (RLQ) abdominal pain

• Background information available from textbooks-– What typically presents as RLQ pain

– What is the clinical course of the different diagnoses

– Specifically, what is typical presentation of appendicitis

• Foreground information– How good is a CT scan for appendicitis?

Page 29: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Formulating the Question

Page 30: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Formulating the Question• An ideal question:

– Focused enough to be answerable

– Pertinent to clinical scenario

– Framed as Population receiving an Intervention (test or treatment) [as Compared to other test/treatment or placebo] associated with Outcome (disease or improvement)

Page 31: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

PICOS

P roblem/population

I ntervention

C omparison

O utcome

S tudy design

Page 32: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Examples of tough questions

• Should I screen men for prostate cancer?

• Who is a good candidate for hormone replacement therapy?

• Are angiotensin receptor blockers now first-line for hypertension?

Page 33: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Examples of better questions• Would a PSA test reduce mortality in a 65 year-old

asymptomatic man? • What is the reduction in fracture risk associated

with hormone replacement therapy? • Is losartan more effective than atenolol at

preventing cardiovascular events in middle-aged hypertensive diabetic women?

Page 34: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

PICOS

PICOS for confirmatory diagnosis of appendicitis

P: 27 year old woman with symptoms suggestive of appendicitis

I: CT Scan

C: Ultrasound

O: Accurate diagnosis without undue delay

S: ??

Page 35: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Important Outcomes• Patient Oriented Outcomes:

outcomes patients actually care about – Death (overall or disease-specific)– Heart attacks, strokes, amputations, bed sores, broken hips, renal

failure, etc.– Ability to perform activities of daily living

Versus• Disease oriented outcomes:

– Biochemical, physiologic, pharmacologic, or laboratory measures

Page 36: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Comparing DOE and POE

Shaughnessy AF, Slawson DC. Getting the Most from Review Articles: A Guide for Readers and Writers. American Family Physician 1997 (May 1);55:2155-60.

ExampleDisease-Oriented

Evidence

Patient-Oriented Evidence that

Matters CommentAntiarrhythmic Therapy

Drug X PVCs on ECG

Drug X increases mortality

POE contradicts DOE

Type 2 Diabetes Aggressive Tx with insulin or oral agentscan keep BS low

Aggressive Tx does not reduce mortality or prevent most complications

POE contradicts standard teaching

ProstateScreening

PSA screening detects prostate cancer early

Does PSA screening mortality?

DOE exists, but POE is unknown

Page 37: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Background versus foreground information

• Case discussion: 27 year old woman with right lower quadrant (RLQ) abdominal pain

• Background information available from textbooks-– What typically presents as RLQ pain

– What is the clinical course of the different diagnoses

– Specifically, what is typical presentation of appendicitis

• Foreground information– How good is a CT scan for appendicitis?

Page 38: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Steps of EBM-5 A’s

• Ask• Acquire• Appraise• Apply• Assess

Page 39: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Finding Evidence”: Sources (I)

• Primary research database (articles)– PubMed (aka MEDLINE), Pyschlit, CCTR

• Secondary research databases (synthesis)– Cochrane Library, Clinical Evidence, InfoPOEMS,

UpToDate

• Tertiary resources (meta search engines, databases of databases)– TRIP+ (Translating Research Into Practice),

PrimeEvidence

Page 40: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Finding Evidence”: Sources • PubMed

– 16 million peer reviewed biomedical articles indexed (note can use PubMed limits to search on particular populations, study types, etc.)

• Cochrane Library– ~3000 clinical systematic reviews (gold standard database)

• Clinical Evidence– ~2500 tsystematic reviews of treatment classified by likelihood of benefit

• InfoPOEMS (www.infopoems.com)– ~3000 regularly updated entries, Patient Oriented Evidence the Matters

(POEM), 100+ journals monitored• UpToDate

– 70,000 pages, evidence based clinical information resource, ~3000 authors, 350+ journals monitored, peer reviewed

• TRIP+– Meta-search of 55 sites of evidence based information

Page 41: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Finding Evidence”: Searching1. Convert clinical question to searchable question (e.g. PICOS)

2. Choose the database you want to search (e.g. PubMed)

3. Apply filters to restrict your search (e.g. PubMed limits linked to

PICOS such as gender, age, study type limits)

4. Assess result (e.g. using systematic review worksheet)

5. Decide if you have enough information to make a decision

6. If not then refine steps 1-3 until you either have an answer or decide

there isn’t enough evidence to make an evidence based decision

Page 42: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Appraising the Evidence

Page 43: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assess the Evidence• Is the study valid?

– Validity is defined as relative freedom from bias and confounding factors• What are the results?

– What is the outcome and how was it measured?– What is the magnitude of the effect?– Are the results statistically significant?

• Do the results apply to my patient?– Does my patient resemble those in the study?– Were all outcomes relevant to my patient evaluated?– Are there other factors (eg, cost, availability) that limit applicability to my

patient?

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001

Page 44: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Objectives

• Understand difference between observational and experimental studies

• For 2 major study designs (randomized controlled trial and cohort study) describe– How the study is designed– Advantages and disadvantages of design– How to assess validity– How to assess results– How to assess applicability

Page 45: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Experimental vs Observational Studies

• In experimental studies, the investigator controls subjects’ exposure to intervention

– Example: randomized controlled trial (RCT)

• In observational studies, investigator does not control the exposure; it occurs naturally or is initiated by patients or their physicians

– Examples: cohort study, case-control study

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 46: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

RCTs

Generally held to be the optimal methodology for determining benefit or harm

Eligible Patients

Randomization

Treatment

Control Outcome

Outcome

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 47: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

RCTs: Advantages

• Treatment and control groups are likely to have similar distribution of known and unknown prognostic factors (potential confounders)

• Outcomes are determined prospectively in a standardized, systematic fashion

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 48: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

RCTs: Disadvantages

• Costly to perform• Size limitations make detection of rare events

difficult (eg, adverse medication effects) • Eligibility restrictions may reduce applicability to

real patients• Cannot be ethically performed if exposure is

expected to cause harm (eg, smoking)

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 49: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

RCTs: Disadvantages• Costly to perform• Size limitations make detection of rare events

difficult (eg, adverse medication effects) • Eligibility restrictions may reduce applicability to

real patients• Cannot be ethically performed if exposure is

expected to cause harm (eg, smoking)

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 50: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Validity of RCTs• Was randomization concealed?• Were patients analyzed in groups to which they were

randomized?• Were patients in treatment & control groups similar with

respect to prognostic factors?• Were patients, clinicians, outcome assessors, and data

analysts aware of allocation?• Were groups treated equally?• Was follow-up complete?Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice . Chicago, IL: American

Medical Association; 2001.

Page 51: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Results of an RCT• Magnitude of result: How large was the treatment effect?

– Relative risk and odds ratio– Absolute risk reduction and number needed to treat (NNT)

• Statistical significance– P value– Confidence interval: How precise was estimate of treatment

effect?• Clinical significance

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 52: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Results of an RCT• Magnitude of result: How large was the treatment effect?

– Relative risk and odds ratio

– Absolute risk reduction and number needed to treat (NNT)

• Statistical significance– P value

– Confidence interval: How precise was estimate of treatment effect?

• Clinical significance Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago,

IL: American Medical Association; 2001.

Page 53: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Calculating the Risk Ratio and Number Needed to Treat (NNT)

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Treatment(n = 1000)

Control(n = 1000)

100 have the outcome

120 have the outcome

Risk = 0.1, or 10%(100/1000)

Risk = 0.12, or 12%(120/1000)

Risk ratio = 0.1/0.12 = 0.83, or 83%

Absolute risk reduction = 0.12 - 0.1 = 0.02, or 2%

NNT = 1/0.02 = 50

Page 54: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Applicability of an RCT

• Were the study patients similar to my patient?– Eligibility criteria– “Table 1” data (baseline characteristics)

• Were all clinically important outcomes considered?• Are the likely treatment benefits worth the potential

harm and costs?Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago,

IL: American Medical Association; 2001.

Page 55: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Cohort Studies

• Similar to RCTs, except that assignment to intervention is not random

Eligible Patients

Choice or Happenstance

Exposed

Not Exposed

Outcome

Outcome

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 56: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Cohort Studies: Advantages• Outcomes are determined prospectively in a

standardized, systematic fashion• Often includes a larger, more diverse population

than those eligible for or included in RCTs• Can be used to assess effects of harmful exposures

(eg, smoking)

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001

Page 57: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Cohort Studies: Disadvantages

• Costly to perform• Size limitations make detecting rare events

difficult• Exposure and control groups are likely to differ in

factors that may affect outcomes• Control of confounding through statistical analysis

may be inadequateGuyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 58: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Validity of a Cohort Study

• Were the exposed and control groups similar in all known determinants of outcome? – Did the analysis adjust for potential differences?

• Were the outcomes measured in the same way in the groups being compared?

• Was follow-up sufficiently complete?

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 59: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Results of a Cohort Study

• How strong is the association between exposure and outcome?– Risk ratio or odds ratio– Absolute risk increase or number needed to harm (NNH)

• Statistical significance– P value– Confidence interval: How precise was estimate of risk?

• Clinical significance

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 60: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Assessing the Applicability of a Cohort Study

• Were the study patients similar to the patient under consideration in my practice?

• Should I attempt to stop the exposure?

Guyatt et al. Users' Guides A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001 to the Medical Literature:.

Page 61: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Case-Control Studies

• In contrast to RCTs and cohort studies, participants are selected based on the presence of the outcome rather than the exposure

• Exposure status is determined retrospectively

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 62: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Case-Control Studies: Design

Select Subjects: Cases (diseased)

Controls(nondiseased)

Exposed Not ExposedObserve: Exposed Not Exposed

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 63: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Case-Control Studies: Advantages

• Much more efficient for investigation of rare outcomes

• Take less time to perform than RCTs or cohort studies

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 64: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Case-Control Studies: Disadvantages

• Retrospective assessment of exposure may be inadequate (recall bias)

• Can be performed only after outcomes have occurred (ie, after damage has already occurred)

• Selection of appropriate controls may be difficult• Control of confounding through statistical analysis may

be inadequate

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.

Page 65: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Steps of EBM-5 A’s

• Ask• Acquire• Appraise• Apply• Assess

Page 66: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Applying EBM

Page 67: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Clinical Expertise

Patient Values and Preferences

Quality of Life

Costs

Best Available Evidence

Page 68: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Integrates Evidence With

• Clinical expertise– Experience – Judgment

• Patient values and preferences• Quality of life • Costs • Other important factors

Page 69: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Integrates Evidence With

• Clinical expertise– Experience – Judgment

• Patient values and preferences• Quality of life • Costs • Other important factors

Page 70: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy Review” -VALIDITY• Clearly focused question? • Randomization• Blinding- subjects, providers, investigators• Groups similar at start and treated the same throughout?• Followed in randomized groups and accounted for at end? (intention to treat)• Enough subjects to minimize chance differences?

REUSLTS AND PRECISION1. What are results? How presented?2. Certainty & precision? (95% CI’s)

APPLICABILITY1. Can the results be applied to my patient?2. All important outcomes addressed?3. Should there by change in policy?

Page 71: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Intention to treat

• Subjects are analyzed in the groups they were randomized to.– Maintains randomization– Better reflects real world outcomes– Measures efficacy (“Will this work?”)– Detects issues about intervention other than

effectiveness “In the best possible circumstances, do they work?”

Page 72: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Bias

• Randomization helps lessen patient bias

– Self-selection

• Blinding helps lessen patient and investigator bias

Page 73: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: What Are the Results ?

• RR OR RRR

• ARR

• NNT / NNH

• P value/ CI

• Clinically significant?

Page 74: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: What Are the Results ?

• RR OR RRR

• ARR

• NNT / NNH

• P value/ CI

• Clinically significant?

Page 75: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Expressing Results

Risk = outcome event rate

= number having event number receiving the intervention

Relative risk = risk in intervention group(RR) risk in control group

Relative risk reduction (RRR) = 1 - RR

Page 76: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Expressing Results

Absolute risk reduction (ARR) = difference in risk (control –

intervention)

Page 77: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Expressing Results

Number-needed-to-treat (NNT) = 1/ARR

in words: the number of patients who need to be treated to prevent one outcome event from occurring in specified time

Page 78: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Example

• An oncology trial testing a new treatment with 4-year follow-up for mortality

• experimental treatment: 30%• control group: 50%

• What are the RR, RRR, ARR, NNT?

Page 79: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Example

RR = risk of death in experiment/control groups

= 30%/50% = 0.6 or 60%

RRR = 1 - RR = 1-0.6 = 0.4 or 40%

ARR = risk of death in control – experimental groups

= .50 -.30 = 0.2 or 20%

NNT =1/ARR = 1 ÷ 0.2

= 5 patients treated with the experimental therapy to prevent one death at 4 years

Page 80: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics
Page 81: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Relative Versus Absolute Benefits

• Consider – July 3, 2002: Worldcom stock rose 120%

(relative increase)– The stock rose from: $0.10 $0.22 (absolute

increase)

Page 82: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Relative risk reduction versus absolute risk reduction

• Baseline risk 10/100 5/100 RRR = 50% ARR = 5% NNT = 20

• Baseline risk 1/100 0.5/100RRR = 50% ARR = 0.5% NNT = 200

• Baseline risk 0.1/100 0.05/100RRR = 50% ARR = 0.05% NNT = 2000

Page 83: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: RRR Lipid Trials 4S WOSCOPS CARE AFCAPS

for acute myocardial infarction

RRR 27 31 25 40(%)NNT 19 42 40 435(5 year)

Page 84: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: 95% Confidence Interval• Any statistic only an estimate of the “true value” of that

statistic. • Confidence Interval (CI) gives range within which that “true

value” probably lies. • 95% CI - if we repeated the experiment with similar

populations an infinite number of times, the results would fall within the CI 95% of the time. 95% certain that the “true value” will fall within the 95% CI range.

• CI gives us an idea of the precision of the result, since the narrower the CI is, the more certain we can be that the experimental value is close to the “true value”.

• And, generally, the larger the sample size, the narrower the CI

Page 85: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

• CI =idea of significance, e.g.– If the 95% CI for the ARR includes 0, no difference between the

experimental and control groups.

– If the 95% CI for the RRR or Odds Ratio includes 1, – no difference between the experimental and control groups.

• Similar to P values (e.g., P<0.05) =statistically significant

• CI gives a sense of the size of the differences found in the study.

• e.g., research study - 50% of patients treated with Drug A are cured, compared with 45% of patients treated with Drug B.

• ARR I s thus 5%.• Statistical analysis P<0.001, statistically significant.• But if 95% CI of ARR is 0% to 10%, indicates result is not clinically

significant (includes “0%” - “no difference”).

Page 86: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Therapy”: Statistical Significance Clinical Significance

• Are the results clinically important?

– Duration of pharyngitis: 8.1 days to 7.4 days

– Weight: 279 lbs to 266 lbs after 3 months

– Survival increased from 4.5 mos to 5.2 mos with 100% mortality at

12 months

– Claudication: Increase in walking distance by 34 ft.

Page 87: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis Review” - WorksheetVALIDITY• Clearly focused question? • Appropriate reference standard?• Reference standard & test applied to all subjects? (verification bias)• Did results of standard influence interpreting test results? (review bias)• Disease status reported and varied? (spectrum bias)• Test method reported with sufficient detail to be replicated?

REUSLTS AND PRECISION1. What are results?2. Certainty & precision? (95% CI’s)

APPLICABILITY1. Can the results be applied to my patient?2. Are local resources (equipment, expertise, cost) sufficient to apply these

results?

Page 88: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: 2X2 table Diagnostic test characteristics

• Sensitivity

• Specificity

• Predictive Value

• Likelihood Ratios

Page 89: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

DIAGNOSTIC TEST

D I S E A S EPresent Absent TOTALS

Test positive

True

Positive

False

Positive

All positive

Test negative

False

Negative

True

Negative

All negative

TOTALSAll with

disease

All without disease

Entire population

T

E

S

T

Page 90: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

(+) (-)

(+)a b

(-)c d

D I S E A S E

T

E

S

T

Page 91: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Pt has diseaseDz (+)

Dz (-)

Test(+)a b

Test(-)c d

Page 92: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Pt has diseaseDz (+)

Dz (-)

Test(+)a b

Test(-)c d

Page 93: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Dz (+) Pt does not have disease Dz (-)

Test(+)a b

Test(-)c d

Page 94: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Pt has diseaseDz (+)

Dz (-)

Test(+)a b

Test(-)c d

Page 95: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Pt has diseaseDz (+)

Dz (-)

Test(+)a b

Test(-)c d

TP

FN

Page 96: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Sensitivity

Sensitivity is proportion of people with disease who have a positive test

Dz (+) Dz (-)

Test(+)a b

Test(-)c d

Sensitivity = (a/a+c) =(TP/TP+FN)

TP

FN

Page 97: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Dz (+) Pt does not have disease Dz (-)

Test(+)a b

Test(-)c d

Page 98: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Dz (+) Pt does not have disease Dz (-)

Test(+)a b

Test(-)c d

Page 99: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Dz (+) Pt does not have disease Dz (-)

Test(+)a b

Test(-)c d

FP

TN

Page 100: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Specificity

Specificity is proportion of people without disease who have negative test

Dz (+) Dz (-)

Test(+)a b

Test(-)c d

Specificity = (d/b+d) =(TN/FP+TN)

FP

TN

Page 101: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Tradeoffs of sensitivity & specificity labeling diabetes

Blood sugar

• 70• 100• 130• 160• 200

Sensitivity Specificity

98.6% 8.8%88.6% 69.8%64.3% 96.9%47.1% 99.8%27.1% 100%

Page 102: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Choosing a test

• SnNout-

A sensitive test, if negative, rules out a disease

• SpPin-

A specific test, if positive, rules in a disease

Page 103: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Sensitivity & Specificity

• Useful for picking a test (test properties)– Screening- prefer sensitive test– Diagnosis – prefer specific test

• Less help in making diagnosis

Page 104: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

• In patients, what you know are their test results- you are trying to determine whether they actually have the disease.

• Positive Predictive Value :

– Of all who tested positive for a disease, the proportion that actually has it

• Negative Predictive Value :

– Of all who tested negative for a disease, the proportion that actually does not have it

Page 105: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

(+) (-)

(+)a b

(-)c d

D I S E A S E

T

E

S

T

Page 106: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Positive Predictive Value

Proportion of people with a positive test who have a disease

PPV = a/a+b=

TP/TP+FP

Dz (+) Dz (-)

Test(+) TPa

FPb

Test(-)c d = true positives

over all positives

Page 107: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Negative Predictive Value

Proportion of people with a negative test who don’t have a disease

NPV= d/d+c=

TN/TN+FN = true negatives

over all positives

Dz (+) Dz (-)

Test(+)a

FPb

Test(-) FNc

TNd

Page 108: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

PPV dependent on prevalence, even when using the same test

• Example: Prevalence of a particular disease in a population is 50%. Sensitivity= 90% Specificity= 95%

PPV: = a/a+b

= 95%

Dz (+) Dz (-)

Test(+)a b

Test(-)c d

100 100 200

90

10 95

5

Page 109: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: PPV & prevalence

PPV dependent on prevalence using same test

• Example: Prevalence of a particular disease in a population is 5%. Same test, same sensitivity & specificity– Sensitivity= 90% Specificity= 95%

PPV: = a/a+b

= 47%

Dz (+) Dz (-)

Test(+)a b

Test(-)c d

10 190 200

9

1 180

10

Page 110: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: PPV & NPV

• Useful for diagnosis– Probability of disease after (+ ) or (–) test

• Drawbacks:– Sensitive to prevalence of disease– Prevalence of disease in general population may not be

the same as that of patients you see in clinic/ER.– Not all test results can be categorized as “+” or “-”.

For these reasons, some consider PPV & NPV “Old School”.

Page 111: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Likelihood Ratios

• Likelihood Ratio is how much more likely is it that someone with this finding has the disease, compared to someone who doesn’t. It does NOT vary with prevalence.

Technically, the + LR is how much more likely someone is to get any positive test result if they have disease, compared to someone who doesn’t.

Page 112: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: Likelihood ratio

LR = SENSITIVITY 1 - SPECIFICITY

Page 113: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What do all the numbers mean?

The Likelihood Ratio is a diagnostic weight;

It tells you by how much a given diagnostic test result will raise or lower the probability of having the disorder.

Pretest Probability: the chance that the pt has disease, prior to ordering any tests. This is often an estimation based on clinical experience

Post-test Probability: the chance that the pt has disease, given the results of the test

Page 114: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

What do all the numbers mean?

A LR of 1.0 means the post-test probability is exactly the same as the pretest probability.

A LR >1.0 increases the probability of having the disorder.

A LR<1.0 decreases the probability of having the disorder.

Page 115: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

“Diagnosis”: What are the Results?

Likelihood ratios >10 or <0.1 generate large changes from pre- to post-test probability and are generally considered significant.

Strong evidence to rule in/rule out a diagnosis.

Likelihood ratios of 5-10 and 0.1-0.2 generate moderate changes in probability.

Moderate evidence to rule in/rule out a diagnosis.

Likelihood ratios of 2-5 and 0.2-0.5 generate small changes.

Minimal evidence to rule in/rule out a diagnosis

Likelihood ratios 0.5-2 usually have little effect

Page 116: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Figure 1a: Likelihood Ratio Nomogram

Page 117: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

LRs = Diagnostic Weights

-45%0.1

-30%0.2

-15%0.5Values between 0 and 1 DECREASE probability of disease

01.0

+15%2

+30%5

+45%10Values greater than 1 INCREASE probability of disease

Change in probability

Likelihood Ratio

From Steve McGee, Evidence Based Physical Diagnosis

Page 118: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Table 2a: Likelihood Ratios of Tests for the Diagnosis of Appendicitis

Page 119: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Likelihood ratio

• DVT– Homan’s sign +LR 1.5– Doppler + LR 39

• ANEMIA– Conjunctival rim pallor +LR 16.7

Page 120: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Summary Diagnostic Test

D I S E A S EPresent Absent TOTALS

Positive True

Positive

False

Positive

ALL Positives

Negative False

Negative

True

Negative

ALL Negatives

TOTALSAll with

disease

All without disease

Entire population

T

E

S

T

Sensitivity Specificity

PPV

NPV

Page 121: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Common pitfalls

• Results reported as relative risk – (ex. Migraines, CVA & OC)

• Results that came from recalculating the data after trial was done – (Post-hoc analysis) (ex. Hot study)

• Over-interpreting results• Relying on just one study (ex. Mg for heart dz),

a poor study, or wrong type of study (ex. HRT)• Confusing statistical significance with clinical

significance (ex. Drugs for BPH)• Not looking at CI’s • Not considering who funded study

Page 122: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Steps of EBM-5 A’s

• Ask• Acquire• Appraise• Apply• Assess

Page 123: Clinical Problem Solving an introduction to Evidence-Based physical therapy basics

Core of EBP

“Supposing is goodbut finding out

is better.”

Mark Twain