teaching evidence-based medicine

TeachingEvidence-Based Medicine

Gary S Gronseth, MD, FAANProfessor of NeurologyUniversity of Kansas

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from principles

• Rate Evidence on a Hierarchy– Understand two sources of error– Love the 2 x 2 table

• Emphasize Evidence never enough– Apply to your patient– Incorporate patient values

A case…• A 58 year-old right-handed man suddenly

developed problems speaking, right lower facial weakness and right hand clumsiness. His symptoms slowly resolved over a week.

• He had a history of controlled HTN and no other risk factors.

• Head MRI: small left frontal infarct.• EKG: sinus rhythm. • MRA: no cranial artery stenosis. • Echocardiogram: PFO

The Physician’s Dilemma

To Close or Not to Close

Even if the answer is unknown, a decision must be made!

Clinical Reasoning

Close PFO?

“Where I trained”

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable

Clinical Reasoning

Close PFO?

Deceitful

“Closure of PFO in patients with cryptogenic ischemic stroke is

the standard of care in the community.”

“The consequences of a second stroke are potentially devastating. PFO closure

is mandatory.”

Fallacious• Irrelevant• Rhetoric• Psychological appeal• Emotion-Driven• Persuasion

Patient

Intervention

Co-intervention

Outcome

Determining relevance:Define the question

For patients with cryptogenicstroke and PFO

does PFO closurevs no PFO closure

reduce the risk of the next stroke

Determining relevance:Define the question

Popular Appeal

“Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community.”

Begging the Question

“The consequences of a second stroke are

potentially devastating. PFO closure is

mandatory.”

Irrelevant Outcomes

I’ll be sued.

I’ll be reimbursed

Deceitful

Fallacious

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable

Deceit

Close PFO?

Fallacy

Deceitful

Fallacious

Reasoned

Reasoned

•Relevant•Logical appeal•Data-Driven •Truth

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from

principles

Principles

Close PFO?

Evidence

Judgment

Principles

Decision

Deductive InferenceFrom Principles

• The left side of the brain controls the right side of the body

• My patient can’t control the right side of his body

• My patient has a problem with the left side of his brain

Principles

Use a Parachute?

Principles

Close PFO?

PFO

• Fibrous adhesions fail to seal the atrial septum after birth

• Persistence of a potential shunt between the right and left atria of the heart

PFO might allow paradoxical embolism

• Small emboli normally filtered by lung without clinically important consequence

• In patients with PFO, emboli can travel to the brain and cause ischemic stroke

• Closing the PFO will prevent future strokes

Principles

Close PFO?

Reasoned•Relevant•Reason•Logical appeal•Data-Driven •Truth

Deduction(Principles)

Principles

Close PFO?

Evidence

• Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke

• Jane has a stroke and PFO. We should treat her with closure to prevent another stroke.

• John had a stroke and PFO and was treated with closure, he didn’t have another stroke.

• Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke.

Analogy and Inductive Inference

Evidence

What happened to

patients?

Principles

Close PFO?

Evidence

Reasoned•Relevant•Reason•Logical appeal•Data-Driven •Truth

Induction (Evidence)

Deduction(Principles)

Principles

Close PFO?

Evidence

Judgment

Best Guess

Opinion

Hypothesis

Reasoned•Relevant•Reason•Logical appeal•Data-Driven •Truth

Intuition(Judgment)

Induction (Evidence)

Deduction(Principles)

Distinguishing Opinion from Principles• Is there equipoise?

– Do reasonable people disagree?– Would an IRB approve a trial?– Is there an ongoing trial?

• Evidence separates judgment from principles

Principles

Close PFO?

Evidence

Judgment

Theory

Scientific Method

Experiment

Hypothesis

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from

principles

Principles

Close PFO?

Evidence

Judgment

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion

from principles• Rate Evidence on a Hierarchy

Strong

Weak

• Jane has a stroke and PFO. We should treat her with closure to prevent another stroke.

• John had a stroke and PFO and was treated with closure, he didn’t have another stroke.

Strong

Weak

• Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke.

• Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke

Inferences from Evidence…

Are not valid or invalid

Are never certain

Strong

Weak

• Jane has a stroke and PFO. We should treat her with closure to prevent another stroke.

• John had a stroke and PFO and was treated with closure, he didn’t have another stroke.

Strong

Weak

• Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke.

• Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke

Informally recalled cases

Why is this a weak inference?

Inferences from informally recalled cases can mislead

• Too few cases• Selective recall: remember those

– That are more recent– With extreme results– That support our pre-conceptions

Experts are not immune to these limitations

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from principles

• Rate Evidence on a Hierarchy– Understand two sources of error

• Often too few cases

• Selective recall: remember those– That are more recent– With extreme results– That support our pre-

conceptions

Two Sources of Error

Systematic

Bias

Random

Chance

Find More Cases

Retrospective Observational Sudy 2002 to 2010

Of all Stroke and PFO Cases: 319

Rats…I’m going to have to

start counting these cases

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from principles

• Rate Evidence on a Hierarchy– Understand two sources of error– Love the 2 x 2 table

Relationships between variables

PFO Closure and Stroke

2 X 2 TableOutcome

Treatment No stroke

Stroke All

Closure 167

No Closure 152

Total 239 80 319

Expected if No RelationshipOutcome

Treatment No stroke

Stroke All

Closure 125 42 167

No Closure 114 38 152

Total 239 80 319

Expected if no RealtionshipOutcome

Treatment No stroke

Stroke All

Closure 75% 25% 100%

No Closure 75% 25% 100%

Total 75% 25% 100%

“Actual”Outcome

Treatment No stroke

Stroke All

Closure 150 17 167

No Closure 89 63 152

Total 239 80 319

“Actual”Outcome

Treatment No stroke

Stroke All

Closure 90% 10% 100%

No Closure 59% 41% 100%

Total 75% 25% 100%

2 X 2 TableOutcome

Treatment No stroke

Stroke All

Closure a b 167

No Closure c d 152

Total 239 80 319

Relative Risk stroke =b/(a+b)

d/(c+d)

Risk difference stroke = b/(a+b) - d/(c+d)

Measures of AssociationOutcome

Treatment No stroke

Stroke All

Closure a b 167

No Closure c d 152

Total 239 80 319

Measure of AssociationRelative Risk Stroke Outcome

Outcome

Treatment No stroke

Stroke

Closure 90% 10%

No Closure 59% 41% RR Stroke10/41 = 0.24

Cryptogenic stroke patients receiving Closure were 0.24 times less likely to have stroke.Therefore, I should offer my patients with

stroke and PFO Closure.

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from

principles• Rate Evidence on a

Hierarchy– Understand two sources of

error– Love the 2 x 2 table

Random (Sampling) Error--Incorrect result from bad luck

Equally likely to be too high or too lowStatistical power/precision

--Measured by:

P-values (p < 0.001)

Confidence intervals RR 0.24: (95% confidence intervals 0.15 to 0.40)

Systematic Error

Incorrect results from poor study design or execution

More likely to be too high or too low

Risk of Bias Measured:

Semi-quantitatively

Class of Evidence

0.5 0.75 1 1.25 1.50.25

TruthMeasured

Our Study

+Cl

-Cl

Stroke No Stroke

Patients not receiving Closure were more often older, diabetic and hypertensive

Sometimes had to “guess” the outcome from the record.

Major Sources of Bias

+Cl

-Cl

Poor Good

Confounding

Misclassification

Lower Risk of BiasThe Randomized Masked Trial

+Cl

-Cl

Poor Good

R

Randomized Masked Trial

SingleCase Report

What is the risk of Bias?

Strong

Weak

Find the best evidenceSearch online databasesMEDLINE

There is insufficient evidence to

support or refute the benefit or

lack of harm of PFO closure.

Conclusion

Strong

Weak

Despite the weak evidence, a

decision must be made.

Decide

Strong

Weak

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from

principles• Rate Evidence on a Hierarchy

– Understand two sources of error– Love the 2 x 2 table

• Emphasize Evidence never enough– Apply to your patient– Incorporate patient values Principles

Close PFO?

Evidence

Judgment

Induction is never certain

• Often the evidence is weak• Even when strong, the Evidence never

perfectly applies to your patient

• Explicitly consider how well the evidence applies to your patient

Incorporating patient values

Benefits Risks

Uncertainty

Know what is not Known

• If you fail to acknowledge the uncertainty and tell the patient we know that the PFO should or should not be closed…

• You have failed to distinguish opinion from principles. Principles

Close PFO

Evidence

Judgment

To Teach EBM…• Explicitly Reason

– Exclude the unreasonable– Distinguish opinion from

principles• Rate Evidence on a Hierarchy

– Understand two sources of error– Love the 2 x 2 table

• Emphasize Evidence never enough– Apply to your patient– Incorporate patient values Principles

Decision

Evidence

Judgment