1

Course: EPIB 679-001 Clinical Epidemiology

Date: May 8 to June 28:35 – 11:40

Session 9: Evidence-Based Medicine

Dr. J. Brophy

Mid term

Their analysis

Statin lives 30 days

Non statin 15 days

Proper analysis

Statin lives 31 days

Non statin 105 days

Mid term

• Misclassification of exposure• Residual confounding (big differences)• Differential censoring (16 months vs. 24)• 7 year follow-up – calendar time bias• Benefit only on mortality – not MI

Mid term

• No mention if CPH model assumptions hold?• 217 deaths, non-fatal MI or strokes in the

statin group. Later we are told there are 167 fatalities and 68 non fatal MIs in this group. Conclusion there were – 18 strokes!!!

• 33% less new cancers

Causality

• Strength of association• Consistency• Specificity• Temporality• Biologic gradient• Plausibility• Coherence• Experimental evidence• Analogy

2

Scientific Inference

• Deduction (good for mathematics, difficult for empirical sciences)

• Induction (start with observations then make general inferences)

• Conjecture & refutationism (Karl Popper)• Bayes

Survival analysis

Kaplan-Meier

Product-limit method

Kaplan-Meier Life-Tables

• Order survival times: t1 <t2 <…tm – n observations – N failures– m<N failures – Number at risk denoted by ni and number of

deaths at ti is di

Data Layout

cN

…

c2

c1

Number censored (ci)

N

…

2

1

Order

dNnNtN

………

d2n2t2

d1n1t1

Number of deaths (di)

Number at risk (ni)

Survival time (ti)

Example

0.040.67213[58,60)

….….….….….….

0.690.8621073[4,5)

0.800.945583[2,4)

0.850.85215100[1,2)

1.01.000100[0,1)

S(n-d)/nNo. censored

No. deaths (d)

No at risk(n)

Interval

Survival = 1 before 1st failure, by definition

[0=include 01)=exclude 1

3

Estimating Survival

• Kaplan-Meier estimator of survival by time is:

NB: These curves are step-functions

Log-rank test used to compare survival curves

Log Rank Test Log Rank Test

Log Rank Test Cox PH Model

4

Cox Model Cox Model

Hazard Function Data

Pocock, S. J BMJ 2006;332:1256-1258

How to perform the simplest statistical test of significance

5

Vigor (one more time)

• Hazard ratio (HR) =2.37, 95% CI 1.39 to 4.00• 45 and 19 in rofexocib and naproxen groups

respectively • z = (45 − 19)/√45+19= 3.25, P = 0.0012• More exact log rank test (P = 0.0016)• Ratio of events 45/19 = 2.37 = HR

Summary

Is more always better?

• More individual testing• More facilities leading to more medical

contacts • Not necessarily better, be careful of lead and

length time bias may give false notion of more disease and improved outcomes

• Theoretical construct of “Law of diminishing returns”

RITA -2

• Stable patients randomized to medical Tx vs. PTCA

• PTCA less angina in the most severe patients• But overall increased mortality & MI

New York vs. Texas

• Risk adjusted outcomes in patients following MI• Texans had 1.5 X angiograms• After 2 years FU, had low exercise tolerance

and higher mortality

Hadler N. The last well patient. 2004

A well person

“somebody who is not yet completely worked up”

6

More not always better Life expectancy

a77.2a74.5a79.9United States1

78.475.581.3Germany78.576.280.7United Kingdom1

79.475.882.9France

a79.7a77.2a82.1Canada

a80.4a77.8a83.0Switzerland81.878.485.3Japan

2003TotalMenWomen

Life expectancy at birthyears

Infant mortality

8.4a7.0United States15.84.2Germany6.35.3United Kingdom16.53.9France6.8a5.4Canada5.54.3Switzerland4.33.0Japan

19932003

Infant mortalityper 1 000 live births

Changing thresholds

• 65M Americans have HBP• 59M have Pre- HBP• $700,000 from industry to ASH• All 7 guideline writers large COI• Develop new def’n whereby 50% would

become class 1 HBP• Probably drug sales but• No evidence for better outcomes• Chol, diabetes, obesity

PLoS Medicine | www.plosmedicine.org May 2005

Getting what you want

Avoid what you don’t want

7

Avoid what you don’t want Avoid what you don’t want

Copyright ©2003 BMJ Publishing Group Ltd.

Aldrich, R. et al. BMJ 2003;327:1283-1285

Framework for including effects of socioeconomic position in guidelines

BMJ 2003;327;1424-1427

Power of stories

I was finally forced to leave the wreckage due to prohibitiveand deadly smoke. The first person I encountered wasa mother of a 22 month old boy—the same mother I hadcomforted and reassured right after the engine exploded.She was trying to return to the burning wreckage to findhim, and I blocked her path, telling her she could notreturn. And when she insisted, I told her that helpers wouldfind him.Sylvia Tsao then looked up at me and said, “You told meto put my baby on the floor, and I did, and he’s gone.

Guidelines AAP

In 2001, the AAP endorsed a proposed newrequirement that children under 2 years of age ride in infant safety seats on aeroplanes, rather than being allowed to travel free on a parent’s lap.

The evidence

• The FAA estimated that about five aeroplanecrash deaths could be prevented over 10 years by adoption

• Because of additional cost of an aeroplaneticket for a child is likely to lead some families to drive rather than to fly, with estimated increase of about 87 road deaths

• Cost / death prevented was $6.4 million for each $1 cost of the ticket for the child, or $1.3 billion if the ticket cost $200.

8

Cognitive biases in perception of benefit and harm

Cognitive biases in perception of benefit and harm

Other paradigms than EBM

• Conventional wisdom• Expert opinion• Narrative review• Power of stories

Levit SD, Dubner SJ. Freakonomics

Conventional wisdom

• We associate truth with convenience, with what most closely accords with self-interest and personal well-being or promises to best avoid awkward effort or unwelcome dislocation of life.

Evidence – Based Medicine

The High Priest The New Priests

The new paradigm (religion)

Why Evidence-Based Medicine?

Good intentions and plausible theories are no substitute for reliable evidence from empirical research about the effects of healthcare interventions

9

D Sackett BMJ 1996;312:71-72

EBM - Definition

The conscientious, explicit, and judicious use of current best evidence in making decisions aboutthe care of individual patients.

Means integrating individual clinical expertise with the best available external clinical evidence from systematic research

neither alone is enough

Evidence based medicine: what it is and what it isn't

• Is integrating the best current external evidenceand individual clinical expertise

• Is dynamic, evolving and generally helpful• Is not “cookbook medicine” nor a religion

D Sackett BMJ 1996;312:71-72

Evidence-Based Medicine

“Without clinical expertise, practice risks becoming tyrannised by evidence”

The Key

Avoid turning evidence-based medicine into evidence bound medicine

BMJ 318:1999;1618

EBM Not Perfect But What About the Alternatives …. Clinical (Policy) Decision Making

• Evidence based medicine– Evaluating the evidence– Efficacy– Safety

• Economic analysis– $ / QALY– Budget impact

• Ethics / values

10

Evidence-Based Medicine - elements

• Clinical decisions based on the best scientific evidence

• Best evidence means using best epidemiological and biostatistical methods (study design, analysis, interpretation)

• Science is meant to be cumulative as results of a particular research study cannot be interpreted with any confidence unless they have been synthesised

• But researchers usually don’t cumulate scientifically

“Standard EBM”Steps in EBM process

1. Formulate an answerable question2. Track down the best evidence 3. Critically appraise the evidence4. Integrate with clinical expertise and patient values5. Monitoring your performance

Its practice requires:

• Asking• Acquiring• Appraising• Applying• Assessing

What is the intervention?

• If our learners are interested in the ‘using’ mode, the intervention should focus on formulation of questions, searching for preappraised evidence and applying that evidence

• If the learners are interested in the ‘doing’ mode, they should receive training in all 5 skills

• The intervention should match the clinical setting, available time and other circumstances

What are the relevant outcomes?

• Attitudes• Knowledge • Skills• Behaviours• Clinical outcomes

EBM & Patient Values

• Patient values are the beliefs that patients bring to discussions about treatment options

• Evidence based information is important for both clinicians and patients in making decisions about treatment options, but it is only part of the picture

• Patient values are a filter through which patients view the evidence that they are presented with

• Patients should be encouraged to express their values

11

Therapy

• Ask a question• Acquire some articles• Appraise the evidence• Apply the findings• Assess your performance

General Approach

• Is the study valid?• What are the results? • Will results help me treat my patients?

Is the study valid?

• Proper randomization • Blinded assesment of outcomes• No loss to follow-up• Intention to treat analysis• Systematic review of all pertinent studies

• Is it important? How large & precise is the treatment effect?– NNT for what

over how longwith what precision

• Chief emphasis on overall results (beware subgroup / post hoc analysis)

• Limitations of p values• Confidence intervals• Ideally Bayesian analysis

What are the results?

Will results help me treat my patients?

• Generalizability (comparable patients / practice patterns)

• Important meaningful endpoints (composite)• Ask if it makes sense (synthesis with cumulative

previous experience)• Cost (feasibility)

1 RCT

P value <.05

Evidence based medicine=====

P values and evidence based medicine

12

Information versus wisdom Top successes in teaching EBM

• Teaching EBM succeeds:– When it centers around real clinical decisions– When it focuses on learners’ actual learning needs– When it balances passive with active learning– When it connects new knowledge to old– When it involves everyone on the team

Top successes

• Teaching EBM succeeds:– When it matches and takes advantage of, the clinical setting,

available time, and other circumstances– When it balances preparedness with opportunism– When it makes explicit how to make judgments, whether

about the evidence itself or how to integrate evidence with other knowledge, clinical expertise and patient preferences

– When it builds learners’ lifelong learning abilities

Top 10 mistakes when teaching EBM

• Teaching EBM fails:– When learning how to do research is emphasised over how

to use it– When learning how to do statistics is emphasised over how

to interpret them– When teaching EBM is limited to finding flaws in published

research– When teaching portrays EBM as substituting research

evidence for, rather than adding it to clinical expertise, patient values and circumstances

Top 10 mistakes when teaching EBM

• Teaching EBM fails:– When teaching with or about evidence is disconnected from

the team’s learning needs about the patient’s illness or their own clinical skills

– When teaching occurs at the speed of the teacher’s speech or mouse clicks rather than the pace of the learner’s understanding

– When the teacher strives for full educational closure by the end of each session rather than leaving plenty to think about and learn between sessions

How to Practice EBM

• Look for “seal of approval” (eg Cochrane)• Basic understanding of study hierarchy (RCT

& meta-analyses)• Awareness of potential biases• Basic understanding of probability

(biostatistics)

13

Practitioners’expertise

Evidencefrom

research

Preferences,values & rights

Availableresources

Practice Decisions

Practice Decisions Principles of good practice

• All new research studies should be:

• designed in the light of scientifically-defensible syntheses of relevant existing research evidence

• reported using the new evidence to update these research syntheses (thus making clear what contribution the new study has made to the total evidence)

Moving in the right direction - BMJ Question

• If academia cannot be relied on to ensure that new research projects begin and end with syntheses of the results of other relevant studies, who will protect the public from the adverse consequences of current scientific indiscipline?