evidence based medicine a new approach to clinical care and research

EVIDENCE BASED MEDICINEEVIDENCE BASED MEDICINEA new approach to clinical care and researchA new approach to clinical care and research

OBJECTIVES OF THE SESSION

• Recognize the concepts and principles of EBM.

• Identify the important of EBM as an essential part of clinical practice.

• Discuss the Skills needed for EBM practice.

• Recall the five steps approach to EBM practice.

• Identify the application of EBM in clinical practice.

• Discuss the barriers to practice EBM

• Provide some examples of EBM practice

Pause for Thought For three minutes

• Why this session is important?

• What is EBM

• What are the • Benefits ??

First alone then 2-3 in group

a test…1st ?

A.A. Training, clinical experience and consultation Training, clinical experience and consultation with other professionalswith other professionals

B.B. Convincing evidence (non-experimental) from Convincing evidence (non-experimental) from articles, case reports, product literature, etc.articles, case reports, product literature, etc.

C.C. Preferences of the patientPreferences of the patient

D.D. Active search of Randomized Controlled Trials, Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis ReportsSystematic Reviews, Meta-Analysis Reports

WHAT IS THE BASIS OF WHAT IS THE BASIS OF YOURYOUR MEDICAL PRACTICE?MEDICAL PRACTICE?

(Check all that apply)

A.A. Training, clinical experience and consultation Training, clinical experience and consultation with other professionalswith other professionals

B.B. Convincing evidence (non-experimental) from Convincing evidence (non-experimental) from articles, case reports, product literature, etc.articles, case reports, product literature, etc.


D.D. Active search of Randomized Controlled Trials, Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis ReportsSystematic Reviews, Meta-Analysis Reports


EXCELLLENT!EXCELLLENT!

BUT… Past knowledge and practice BUT… Past knowledge and practice might be outdated or inadequatemight be outdated or inadequate

Graduate Medical School Practiced Physician

A.A. Training, clinical experience and consultation with Training, clinical experience and consultation with other professionalsother professionals

B.B. Convincing evidence (non-experimental) from articles, Convincing evidence (non-experimental) from articles, case reports, product literature, etc.case reports, product literature, etc.


D.D. Active search of Randomized Controlled Trials, Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reportsSystematic Reviews, Meta-Analysis reports


FANTASTIC!FANTASTIC!

BUT… This evidence may be biased, outdated, BUT… This evidence may be biased, outdated, incorrect, or not applicable to your patientincorrect, or not applicable to your patient

ARTICLES ADVERTISEMENTS

JOURNALS (1987 to present)

A.A. Training, clinical experience and consultation with other Training, clinical experience and consultation with other professionalsprofessionals

B.B. Convincing evidence (non-experimental) from articles, Convincing evidence (non-experimental) from articles, case reports, product literature, etc.case reports, product literature, etc.


D.D. Active search of Randomized Controlled Trials, Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reportsSystematic Reviews, Meta-Analysis reports


WONDERFUL! WONDERFUL!

Mutual Respect + Shared Goals = Better Cooperation and Compliance

The patient should be involved in The patient should be involved in all important decisionsall important decisions

But this is But this is NOTNOT always an easy task! always an easy task!

And conflicts And conflicts WILLWILL occur! occur!

But doctor, I DO want to have

children!

No salt?Lose weight?

Forget it! Just give me a pill!

I WON’T take that medicine… The side effects

are INTOLERABLE!

And conflicts And conflicts WILLWILL occur! occur!

Education about current alternatives and risks is often Education about current alternatives and risks is often needed… needed… for both the Patient for both the Patient andand the Doctor! the Doctor!

But doctor, I DO want to have

children!

No salt?Lose weight?

Forget it! Just give me a pill!

I WON’T take that medicine… The side effects

are INTOLERABLE!

I’ll discuss those risks with my husband.

Yes, I’d like to try that new medication!

Wow… I never knew that high blood pressure could

be so dangerous at my age!

Education about current alternatives and risks is often Education about current alternatives and risks is often needed… needed… for both the Patient for both the Patient andand the Doctor! the Doctor!

The patient’s preferences MUST be considered!The patient’s preferences MUST be considered!

An important rule in Evidence Based Medicine…An important rule in Evidence Based Medicine…

It It STARTSSTARTS with the patient and with the patient and ENDSENDS with the patient. with the patient.

A. Training, clinical experience and consultation with other professionals

B. Convincing evidence (non-experimental) from articles, case reports, product literature, etc.

C. Preferences of the patient

D. Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reports


WOW!!! SUPERB!!!

In the practice of Evidence Based Medicine, In the practice of Evidence Based Medicine, it is the physician’s it is the physician’s dutyduty to find the best and to find the best and

most current information and apply it most current information and apply it judiciously for the benefit of the patient.judiciously for the benefit of the patient.

But… A practice based exclusively on science and math is effective only if your patients are robots or clones!

Don’t forget to allow for individual human differencesand personal preferences!






If you checked all 4 items…If you checked all 4 items…





You You areare practicing practicing EVIDENCE BASED EVIDENCE BASED

MEDICINE!MEDICINE!

CONGRATULATIONS!

EVIDENCE BASED MEDICINEEVIDENCE BASED MEDICINEA new approach to clinical care and researchA new approach to clinical care and research

1. Definition of EBM

2. Basic Steps

3. Trials, Studies and Reports

4. Pros, Cons and Limitations

5. EBM Library

6. Advanced EBM

““What What isis Evidence Based Medicine?” Evidence Based Medicine?”

““And where did it come from?”And where did it come from?”

A BRIEF HISTORYA BRIEF HISTORY

1980’s: McMasters University in Ontario, Canada1980’s: McMasters University in Ontario, Canada

Dr. David Sackett and colleagues proposed Evidence Dr. David Sackett and colleagues proposed Evidence Based Medicine (EBM) as a new way of teaching, learning Based Medicine (EBM) as a new way of teaching, learning and practicing medicine.and practicing medicine.

Dr. Sackett defines EBM as:Dr. Sackett defines EBM as:

“…“…The conscientious, explicit, and judicious use The conscientious, explicit, and judicious use of current best evidence in making decisions of current best evidence in making decisions about the care of individual patients.”about the care of individual patients.”

• "Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values."

Sackett, D. L. (2000). Evidence-based medicine: How to practice and teach EBM(2nd ed.). Edinburgh; New York: Churchill Livingstone.

• Clinical expertise: the clinician’s cumulated experience, education, and clinical skills

• Patient values: The patient brings to the encounter his or her own personal and unique concerns, expectations, and values.

• Best Research Evidence: usually found in clinically relevant research that has been conducted using sound methodology

Evidence Based Medicine

It is a change in the way physicians practice medicine, teach and learn, and handle research.

Clinical practice: Based on the best current evidence(not necessarily on how it’s always been done)

Patient Care: Compassionate, patient-oriented(less authoritarian)

Learning & Teaching: Problem-based, problem-solvingmore investigative, less know-it-all-by-yesterday

Research: More stringent approach, better proof criteria(more demanding of proof, less room for error)

PATIENT

PHYSICIANINFORMATION

Questionor

Problem

THREE MAJOR COMPONENTS of EBM

PATIENTValues, Concerns Preferences,

ExpectationsLife predicament

PHYSICIANTraining & Experience

Current ExpertiseContinued learningDemand for proof

INFORMATIONClinically relevant

Proven by researchBest up-to-date evidence

EBM

THE ADDED DETAILS

“Isn’t this the way we have always

practiced medicine?”

“Aren’t these just the same old ingredients

tossed into a new recipe?”

When am I supposed to find the time to do that?

The basic steps of EBM

THE FIVE BASIC STEPS OF EBM

1. Clinical Question

Patient-focused, problem-oriented

2. Find Best Evidence

Literary Search

3. Critical Appraisal

Evaluate evidence for quality and usefulness

4. Apply the Evidence

Implement useful findings in clinical practice

5. Evaluate

The information, intervention, and EBM process





Literary Search


Evaluate evidence for quality and usefulness(validity and relevance)


Implement useful findings in clinical practice Making a decision, by integrating the evidence

with your clinical expertise and the patient’s values.

5. Evaluate


The Clinical QuestionThe Clinical Question

The FIRST stepThe FIRST stepThe HARDEST stepThe HARDEST step

The The MOST IMPORTANTMOST IMPORTANT step! step!

FACT: We all have informational needs!

That is not a problem!

Problems ariseProblems arise

• if we fail to recognize those needsif we fail to recognize those needs

• if we fail to bridge the information gapif we fail to bridge the information gap

• if we fail to ask the right questionsif we fail to ask the right questions

And also for others And also for others aroundaround you! you!

Lee, exactly how much time did you spend on that big

project?

Hmmm… Is he about to give me a

BONUS?

Or is he about to FIRE me?

It will make life It will make life easier for you...easier for you...

Asking good questions Asking good questions is a is a skillskill t to be learned.

A GOOD QUESTION…A GOOD QUESTION…

• Is focused and relevantIs focused and relevant

• Provides clear Provides clear communication communication

• Clarifies your goal or needClarifies your goal or need

• Will reduce the amount of Will reduce the amount of time needed to obtain the time needed to obtain the answeranswer

Lee, can you give me an accounting of the extra time you spent on that project so that I can charge it back to

the client?

Oh sure! I’ll have it on your desk by

tomorrow!

Foreground

Questions

Background

Questions

Novice Expert

Asking Questions

The Question

• Background– Anatomy and Physiology– Pathophysiology– Pharmacology and Toxicology– Differential diagnosis– Diagnostic testing– Treatment– Textbooks, reviews, lectures, experts

The Clinical Question

• Foreground– Detailed information– Patient focus– Evidence-based process

• Be specificBe specific Identify the problem, clarifiy the clinical issue

• Be answerableBe answerable through the literature

• Contain multiple aspectsContain multiple aspects (patient, options, comparisons, etc)

WHEN PRACTICING EBM, WHEN PRACTICING EBM, a good question must also:a good question must also:

It should It should NOTNOT involve a involve a question of Personal Preference question of Personal Preference

or Local Concern.or Local Concern.

EBM QUESTIONEBM QUESTION: : Should include multiple factorsShould include multiple factors

(Examples)(Examples)

PP PATIENTPATIENT type of patient or populationEx: 47 yr male w/DM2 and cellulitis toe, 25 yr female w/DVT and chest pain

EE EXPOSUREEXPOSURE environmental, personal, biologicalEx: TB, tobacco, drug, diet, pregnancy or menopause, MRSA, allergy

II INTERVENTIONINTERVENTION clinical interventionEx: medication, procedure, test, surgery, radiation, drug, vaccine

CC COMPARISONCOMPARISON compare alternative treatmentEx: other prior, new or existing therapy

OO OUTCOMEOUTCOME clinical outcome of interestEx: Reduced death rate in 5 yrs, decreased infections, fewer hospitalizations

A healthy adult presents to the clinic inquiring about the aspirin that it might prevent heart attack ?

Scenario and Question

The Question

“In an asymptomatic adult and no risk factors, would the use of aspirin reduce the incidence of cardiovascular events?

Aspirin and Primary Prevention

1. Patient population.

2. Intervention.

3. Comparison intervention.

4. Outcomes.

Asymptomatic adults with no risk factors

AspirinAspirin

PlaceboPlacebo

Incidence of CV eventsIncidence of CV events

““In asymptomatic adults no risk factors, would the In asymptomatic adults no risk factors, would the use of aspirin reduce the incidence of cardiovascular use of aspirin reduce the incidence of cardiovascular events?events?

Scenario

A 32-year-old man, single, teacher in primary school, known to have IBS for last 3 years with no response to conventional medication. I decided to search for effect of TCA in patients with IBS.

Scenario and Questions (Cont’d)

Use of TCA in IBS

1. Patient population.

2. Intervention.

3. Comparison intervention.

4. Outcomes.

Middle age adults with IBS

Using of TCAUsing of TCA

dietary fibers, bulking dietary fibers, bulking agents and mebeverneagents and mebeverne

Relieving of symptomsRelieving of symptoms

““In middle age adults with IBS, would the use In middle age adults with IBS, would the use of TCA reduce the pain and improve symptoms?of TCA reduce the pain and improve symptoms?

FRAMING THE QUESTION (Example: PICO)FRAMING THE QUESTION (Example: PICO)

ELEMENTELEMENT PROMPTS THE QUESTIONPROMPTS THE QUESTION::

PatientPatient How would I describe a group of patients similar to mine?How would I describe a group of patients similar to mine?InterventionIntervention What main action am I considering?What main action am I considering?ComparisonComparison What is/are the other options?What is/are the other options?OutcomeOutcome What do I (or the patient) want to happen (or not happen)?What do I (or the patient) want to happen (or not happen)?

Example:Example:

P: P: In kids under age 12 with poorly controlled asthma on metered In kids under age 12 with poorly controlled asthma on metered dose dose inhaled steroids…inhaled steroids…

I: I: would the addition of salmetrol to the current therapywould the addition of salmetrol to the current therapyC:C: compared to increasing the dose of current steroidcompared to increasing the dose of current steroidO:O: lead to better control of symptoms without increasing side effects?lead to better control of symptoms without increasing side effects?

CATEGORY OF QUESTIONCATEGORY OF QUESTION

MAJOR CATEGORIESMAJOR CATEGORIES

1.1. DiagnosisDiagnosis2.2. PrognosisPrognosis3.3. Therapy/ TreatmentTherapy/ Treatment PICO4.4. Harm (iatrogenic, other)Harm (iatrogenic, other) PEO

MISCELLANEOUS• Quality of care• Health economics• Office Management• Etc.

THE PATIENT’S QUESTIONSTHE PATIENT’S QUESTIONS MustMust be considered! be considered!

Often QUALITATIVEOften QUALITATIVE (not based on measureable outcomes)Feelings, ideas, experiences, preferences, concerns, fears, beliefs, ethnicity

Usually based on LIMITED BACKGROUNDUsually based on LIMITED BACKGROUND Perception of problemSelf-diagnosisTreatment wanted or needed Alternatives (read, heard, considered, tried)What is the patient hoping to avoid?What benefits does the patient want or need most? Etc.

QUANTITATIVEQUANTITATIVE: “: “Solid EvidenceSolid Evidence””

• Measurable answer or responseMeasurable answer or response

• Necessary for scientific studyNecessary for scientific study

• Necessary for the practice of EBMNecessary for the practice of EBM

QUALITATIVEQUALITATIVE: “: “Quality of LifeQuality of Life””

• ““Fuzzy” data - Impact on daily life, work, family, etc. Fuzzy” data - Impact on daily life, work, family, etc.

• May be very important and influential to decisions – May be very important and influential to decisions – especially for the patientespecially for the patient

• Creates added challenge or twist to practice of EBMCreates added challenge or twist to practice of EBM

QUANTITATIVE vs QUALITATIVE QUESTIONSQUANTITATIVE vs QUALITATIVE QUESTIONS





Literary Search





5. Evaluate


Some examples:

Questions from our clinics• What to do with IBS patients?• Management of premenopousal women. • How to deal with psychosomatic cases ?• Guidelines for shifting patient from one drug to

another ?• Proper management of IBS ?• Assessment of ED ?• Assessment of prostatic enlargement ? • Assessment of alcohol intake.

Find the Best EvidenceFind the Best Evidence

“The Literary Search”“The Literary Search”

HINT: If your desk looks like this, it’s probably the LAST place you should start looking!

Find the Best EvidenceFind the Best Evidence

“The Literary Search”“The Literary Search”

The BEST EVIDENCE isThe BEST EVIDENCE is::

ExternalExternal - from outside resources (researchers, experts)

CurrentCurrent – not out of date, most recent

High QualityHigh Quality - accurate, precise, effective, safe

Patient focusedPatient focused - applicable and appropriate for your individual patient

FIVE STEPS TO FINDING THE BEST EVIDENCE

1. IDENTIFY NEEDS: What type of information is needed?

2. IDENTIFY RESOURCES: Types, Availability, Timeliness,Costs?

3. SEARCH & RETRIEVE: Use efficient strategies

4. REVIEW : Check quality and usefulness of info

5. INTERPRET: Help patient understand info, application

WHAT TYPE OF INFORMATION IS NEEDED?WHAT TYPE OF INFORMATION IS NEEDED?

WHAT CATEGORY IS THE QUESTION?WHAT CATEGORY IS THE QUESTION?

• DiagnosisDiagnosis• PrognosisPrognosis• TherapyTherapy• HarmHarm

WHAT STUDY DESIGN FITS IT BEST?WHAT STUDY DESIGN FITS IT BEST?There are MANY study designs!There are MANY study designs!

EXPERIMENTAL TRIALSEXPERIMENTAL TRIALS (Answers questions of diagnosis or treatment)Randomized Controlled Trials (RCTs)Controlled studiesBlinded vs OpenETC.

OBSERVATIONAL STUDIESOBSERVATIONAL STUDIESDescriptive reportsRetrospective studiesCohort studiesCase ControlETC.

EXAMPLE

Randomized Controlled Trials (RCT)

“Gold Standard” of research

Ideal experimental design - Best design for TREATMENT questions

Must identify objective of treatment (Ex: cure, prevent complication, palliation, reassurance)

Still not always the right intervention for individual patient at that particular time and place

What type of evidence best addresses the question, problem or issue?

CLINICAL PRACTICECLINICAL PRACTICE APPROPRIATE DESIGN FOR CLINICAL RESEARCHAPPROPRIATE DESIGN FOR CLINICAL RESEARCH

Diagnosis, Dx testing Cross-sectional study – not randomized trial

Prognosis Follow-up studies of patients evaluated at same early point of illness

Therapy, treatment RCT or Systematic review of multiple RCTs must be used Avoid non-experimental approaches to avoid false conclusions

about efficacy

Exceptions: When treatment may be successful in an otherwise fatal

conditionWhen no studies are available (rare conditions, new

treatments, etc.)

Harm RCT, Cohort, Case-control

OTHER INFORMATIONALOTHER INFORMATIONALExplore hypothesis Qualitative researchHistory-taking Case control studyIndividual trial & error n of 1 trialFollowing clinical course Cohort studyRecordkeeping Systematic registry-based (computer supported) researchQuality of Care research Individual peer review, Process Evaluation

MISCELLANEOUSMISCELLANEOUS Basic Science, Genetics, Immunology, etc.

WHAT FORM OF INFORMATION?WHAT FORM OF INFORMATION?

Case report

Controlled Trial

Systematic review

Meta-analysis

Clinical guidelines

etc.

LITERARY SEARCH: NEXT STEPLITERARY SEARCH: NEXT STEPIDENTIFY YOUR RESOURCESIDENTIFY YOUR RESOURCES

ColleaguesColleaguesConsultation, DiscussionConsultation, Discussion(Caution: Response may be an outdated “This is what we (Caution: Response may be an outdated “This is what we do”)do”)

Paper resourcesPaper resourcesbooks, reports, journalsbooks, reports, journals

Electronic databasesElectronic databases

Health Literature ServicesHealth Literature Services specialized librarians, staffspecialized librarians, staff

Review services, Abstract Services, etc.Review services, Abstract Services, etc.

SEARCH AND RETREIVE THE BEST EVIDENCESEARCH AND RETREIVE THE BEST EVIDENCE

Learn and Practice various SEARCH STRATEGIESLearn and Practice various SEARCH STRATEGIES::• To find useful information quicklyTo find useful information quickly• To eliminate irrelevant, inappropriate or weak informationTo eliminate irrelevant, inappropriate or weak information

Try to develop the habit of learning as you go; Try to develop the habit of learning as you go; Not just in lengthy formal sessions!Not just in lengthy formal sessions!

LITERARY SEARCH STRATEGYLITERARY SEARCH STRATEGY

ASK FOR HELP!ASK FOR HELP!

SPECIALIZED PERSONNELSPECIALIZED PERSONNEL• track down information, textbooks, articles, track down information, textbooks, articles,

guidelinesguidelines• may provide electronic search support or trainingmay provide electronic search support or training

EXAMPLESEXAMPLES• Medical Librarians• Medical Informatics Specialists• Specially trained staff member

LITERARY RESOURCESLITERARY RESOURCES

• TEXTBOOKS (caution – most obsolete!)TEXTBOOKS (caution – most obsolete!)• TraditionalTraditional• Evidence BasedEvidence Based

• JOURNALS (may be outdated)JOURNALS (may be outdated)

• REVIEW ARTICLES (summaries, abstracts)REVIEW ARTICLES (summaries, abstracts)

• SYSTEMATIC REVIEWS (prepared in systematic, rigorous SYSTEMATIC REVIEWS (prepared in systematic, rigorous manner) manner) Ex: Cochrane CollectionEx: Cochrane Collection

• META-ANALYSISMETA-ANALYSIS

• CLINICAL PRACTICE GUIDELINESCLINICAL PRACTICE GUIDELINESSummarized and easily digestible informationSummarized and easily digestible information

ELECTRONIC RESOURCES, DATABASES, INTERNETELECTRONIC RESOURCES, DATABASES, INTERNET

Bibliographic DatabaseBibliographic DatabaseExample: Medline, PubMedExample: Medline, PubMed

Medical Information Services: Medscape, HDCNMedical Information Services: Medscape, HDCN

Review ServicesReview ServicesSubjective Subjective Systematic ReviewsSystematic ReviewsMeta-analysis Meta-analysis

Examples: Examples: • Cochrane, Cochrane, • Best Evidence, Best Evidence, • Up to DateUp to Date

MORE GREAT INTERNET RESOURCESMORE GREAT INTERNET RESOURCES

Websites Websites cyberNephrology, National Kidney Foundation. NIDDK, cyberNephrology, National Kidney Foundation. NIDDK, American Heart Association, American Cancer Society. American Heart Association, American Cancer Society. National Institutes of Health, etcNational Institutes of Health, etc

Listserve Discussion GroupsListserve Discussion GroupsCyberNephrology, C-span, etc.CyberNephrology, C-span, etc.

Specialty Electronic DatabasesSpecialty Electronic DatabasesPsyclitPsyclitCancerLitCancerLitCINAHL CINAHL (allied health and nursing journals)(allied health and nursing journals)EtcEtc

OTHER RESOURCESOTHER RESOURCES

TapesTapes

VideosVideos

CD-ROMsCD-ROMs

Specialty seminarsSpecialty seminars

Product information and comparisonsProduct information and comparisons

A closer look at some Internet Resources…

MEDLINEMEDLINE

WHAT IS IT?WHAT IS IT?

Searchable database of medical information compiled by National Library of Searchable database of medical information compiled by National Library of Medicine in US 1966-presentMedicine in US 1966-present

Catalogs articles from approx 4000 world journals (of estimated 12-15k total)Catalogs articles from approx 4000 world journals (of estimated 12-15k total)

SEARCH METHODSSEARCH METHODS

Any word or words (title, abstract, content, author name, institution, etc.)Any word or words (title, abstract, content, author name, institution, etc.)

Medical Subject Heading (MeSH) termsMedical Subject Heading (MeSH) terms

A restricted thesaurus of medical titlesA restricted thesaurus of medical titles

Articles categorized by most specific possible MeSH headingArticles categorized by most specific possible MeSH heading

COST: FREE!COST: FREE!

Or may subscribe to companies with specialized search strategies:

• Ovid Technologies (ovid)

• Silver Platter Information (WinSPIRS)

BENEFITSBENEFITS

Free

Vast database

LIMITATIONSLIMITATIONS

Not all articles are indexed on Medline (only 1/3 of approx 10 million!)

Much material listed and described on Medline can only be accessed through journal article

MEDLINE: ELECTRONIC SEARCH STRATEGIES

Search through “Clinical Queries” service of PubMed http://www.ncbi.nlm.nih.gov/clinical.html

Medical Subject Headings (MeSH)

Search filtersSearch by a text word can supplement a MeSH searchBoolean search: “and”, “not”, etc.

To increase sensitivity• use “explode” command• avoid using subheadings

Online Tutorial is available!Online Tutorial is available!

http://www.ncbi.nlm.nih.gov/clinical.html

COCHRANE LIBRARY

Cochrane Database of Systematic Reviews -systematically compiled reviews of intervention

Cochrane Controlled Trials Register-citations of controlled trials identified anywhere in the world

Cochrane Review Methodology Database-methodological papers relating to systematic reviews

Etc.

BEST EVIDENCE

Electronic version of two publications:

• Evidence Based Medicine

• American College of Physicians Journal Club

Covers broad topics of information





Literary Search





5. Evaluate


CRITICAL APPRAISAL CRITICAL APPRAISAL

Interpreting the evidenceInterpreting the evidence

• How to read a paperHow to read a paper

• How to do the mathHow to do the math


IMPORTANT!IMPORTANT!

You do You do NOTNOT have to become a researcher, have to become a researcher, epidemiologist, or statistician to practice EBM.epidemiologist, or statistician to practice EBM.

Focus on how to Focus on how to USEUSE research reports – research reports – not on how to generate them!not on how to generate them!

HOWEVER…HOWEVER…

You must have a solid understanding of You must have a solid understanding of

basic research principlesbasic research principles and and

study designsstudy designs in order to understand in order to understand

and interpret the evidence!and interpret the evidence!


TYPES OF STUDIES AND REPORTS

Randomized Controlled Trial - “The Gold Standard”Systematic reviewMeta-analysisRetroactive vs ProspectiveIncidence PrevalenceCase ControlCohort (Follow-up)Cross-sectionalEcologicLongitudinalExperimentalBlinded vs OpenQualitative Screening

DETOUR

BASIC RESEARCH PRINCIPLES

STUDY DESIGNS

THE THE TIMETIME FACTOR FACTORWhen was the study done?When was the study done?

In what time direction is it headed?In what time direction is it headed?What was its duration?What was its duration?

RETROSPECTIVE PROSPECTIVEPROSPECTIVE

THE THE TIMETIME FACTOR FACTOR

When was the study done?When was the study done?

What year?What year?

What technology? (ie: test, drug, equipment, procedure)What technology? (ie: test, drug, equipment, procedure)

Any associated social factor or historical event?Any associated social factor or historical event?

THE THE TIMETIME FACTOR FACTOR

What was the Study Duration?What was the Study Duration?

Was it an appropriate length of time for the intended goal?

Limited time study or ongoing?

Was study completed? Stopped early?

““LOOKING BACKLOOKING BACK””Historical Review or Historical Review or

InvestigationInvestigation

““LOOKING FORWARDLOOKING FORWARD””Future ResultsFuture Results

The Great UnknownThe Great Unknown

PRESENTPAST FUTURE

In what direction is it headed?In what direction is it headed?


PROPRO•May provide good May provide good

direction for future studydirection for future study““Hind Sight is 20/20”Hind Sight is 20/20”

CON:CON: •Prone to BiasProne to Bias

•A“Fishing Expedition” for A“Fishing Expedition” for positive resultspositive results

PROPRO•Lower risk of bias Lower risk of bias

CON:CON: May get faulty results based May get faulty results based

on incomplete data or on incomplete data or insignificant subgroups insignificant subgroups

(Example of Error: Untreated (Example of Error: Untreated hypertension unlikely to cause hypertension unlikely to cause

cardiac event in child, so treatment cardiac event in child, so treatment is unnecessary below age 18yrs)is unnecessary below age 18yrs) PRESENT

In what direction is it headed?In what direction is it headed?


“Was there a similar comparison group?”

Experimental Intervention

No comparison group

All subjects receive Experimental Intervention


NO EVENT

OUTCOME EVENT

““Trial and Error?Trial and Error?”” or ““Before & After?Before & After?””

UNCONTROLLED STUDIES

PROBLEMSPOSITIVE OUTCOME MAY BE DUE TO:

•Other factors

•Natural course of disease (some get better, some don’t!)

•Spontaneous change of health

•Placebo Effect

•Hawthorne Effect

NEGATIVE OUTCOMEMay be due to study treatment.Could be disastrous!

BENEFITSCan answer some questions about:

•likelihood of response

•adverse effect, etc.

VERY PATIENT-SPECIFIC!

MAY BE ONLY OPTIONRare conditionsPreviously unknown conditions

““Trial and ErrorTrial and Error”” ““Before & AfterBefore & After””

UNCONTROLLED STUDIESGenerally NOT accepted: Potentially Dangerous and FlawedProne to BIAS!

“Traditional Study Method”May produce strong results

SMALLPOXVACCINATION

SMALLPOX VACCINE1. 1796: Edward Jenner inoculates 8yr-old James Phipps with cowpox virus

from a milkmaid’s hands.

Child develops illness, recovers.

2. Two weeks later, inoculates same child with smallpox virus.

Child survives, no illness.

(Centuries later, smallpox eradicated!)

n=1

GOOD!Resistant to Cowpox and Smallpox

(NO DISEASE OUTCOME)

James Phipps, age 8 years

Example#1Example#1

UNCONTROLLED TRIALS: “TRIAL AND ERROR”

Drinks culture of H.pylori

HELICOBACTER PYLORI - GASTRIC ULCERS1982: Australian microbiologist Barry J. Marshall presents evidence showing a possible infectious cause for gastric ulcers. Suggests they may be treatable with antibiotics.

Findings are met with disinterest and disbelief by medical community. Lacks support for further study.

5 years later: Prepares a broth of live organisms isolated from a gastric ulcer patient and drinks it. Becomes violently ill, develops severe acute gastritis.

1990’s Antibiotics are used routinely to cure some gastric ulcers!

Example #2Example #2 NO OUTCOME

SEVERE GASTRITIS

n=1UNCONTROLLED TRIALS: “TRIAL AND ERROR”

Dr. MarshallMicrobiologist

UNCONTROLLED TRIAL


PresentFUTURE

May represent the ONLY treatment option for a new or rare

disease

DIED

RECOVERED


Control Group

STRONGLY PREFERRED! Reduces BIAS. Provides stronger results.

Nothing

Placebo

Observation only

ExperimentalIntervention

Control group may receive…

Only the TEST group receives the Experimental Intervention

CONTROLLED STUDY

Other

IMPORTANTAll other differences

should be minimized or eliminated to reduce

potential BIAS

Gold Standard Treatment


Control Group

RANDOMIZED RANDOMIZED CONTROLLEDCONTROLLED TRIAL (RCT) TRIAL (RCT)

““The Gold The Gold Standard”Standard”

1944 TUBERCULOSIS TREATMENT: Streptomycin vs Bedrest1944 TUBERCULOSIS TREATMENT: Streptomycin vs Bedrest

Streptomycin(n=50)

Bedrest (n=50)

THE FIRST RANDOMIZED CONTROLLED TRIALBy Sir Austin Bradford Hill

(BLINDED)


Control Group

OPEN vs BLINDED STUDIESOPEN vs BLINDED STUDIES

OPENOPEN

BLINDED TRIAL

BLINDED TRIAL

BLINDEDBLINDED

OPEN vs BLINDED STUDIESOPEN vs BLINDED STUDIES

BLINDING

SINGLE BLINDED: Pt unaware of what group s/he is in

DOUBLE BLINDED: Pt and MD unaware

OPEN LABEL:Everyone is aware

RANDOMIZED vs NON-RANDOMIZED TRIALSRANDOMIZED vs NON-RANDOMIZED TRIALS


Control Group

How is this group divided?

NON-RANDOMIZEDNON-RANDOMIZED


Control Group

Assigned to Assigned to groups, usually groups, usually

by the by the researcherresearcher

Potential for RESEARCHER BIAS!Potential for RESEARCHER BIAS!

RANDOMIZEDRANDOMIZED


Control Group

Random method of Random method of assignment usedassignment used

Maximizes “sameness,” Eliminates BIAS!Maximizes “sameness,” Eliminates BIAS!

RANDOMIZEDRANDOMIZED CONTROLLED TRIAL (RCT) CONTROLLED TRIAL (RCT) (EXPERIMENTAL TRIAL)(EXPERIMENTAL TRIAL)


Control Group

PresentFUTURE

““The Gold Standard”The Gold Standard”

Intervention

A

ONE GROUP, MULTIPLE TESTS

CROSSOVER TRIALS

Intervention

BIntervention

B

Intervention

A

ASSESS OUTCOMES #1

ASSESS OUTCOMES #2

COMPARE OUTCOMES

(Best if participants are blinded)

Intervention

A

CROSSOVER TRIALS

Intervention

BIntervention

B

Intervention

A

ASSESS OUTCOMES #1

ASSESS OUTCOMES #2

Fewer participants needed than a RCT!

Lower costsAll are in experimental group

PROS & CONSPROS & CONS

Intervention

A

CROSSOVER TRIALS

Intervention

BIntervention

B

Intervention

A

ASSESS OUTCOMES #1

ASSESS OUTCOMES #2

MUST HAVE SHORT CARRYOVER EFFECTMUST HAVE SHORT CARRYOVER EFFECTMUST HAVE SHORT WASHOUT EFFECTMUST HAVE SHORT WASHOUT EFFECT

(OR WAIT A SUITABLY LONG WASHOUT TIME!)(OR WAIT A SUITABLY LONG WASHOUT TIME!)

PROS & CONSPROS & CONS

CASE CONTROL

Present

RISK FACTOR?

(PAST)

(“A LOOK BACK”)

CASE CONTROL

Present

NEVER SMOKED

(PAST)

(“A LOOK BACK”)

SMOKER

LUNG CANCER

HEALTHY

RISK FACTOR

CASE CONTROL

Present

NORMAL WEIGHT

(“A LOOK BACK”)

OBESITY

DM TYPE II

NON-DIABETIC

RISK FACTOR

COHORT

IS RISK FACTOR

PRESENT?

Future Outcome

“FOLLOWUP DESIGN”

(Exclude those with outcome

already!)

COHORT

IS RISK FACTOR

PRESENT?

Future Outcome“FOLLOWUP DESIGN”

Present

TO INVESTIGATE ETIOLOGY OR HYPOTHETICAL CAUSE

OF DISEASE/OUTCOME

COHORT

Present

RISK FACTORHgb <9

EXAMPLE

DIALYSIS PATIENTS

Measures future outcome for dialysis pts w/o treatment of anemia

CROSS SECTIONAL DESIGN

? Cause ? Risk factors

A look back

CROSS SECTIONAL DESIGN

INFANT DEATHSSIDS DEATHS

OTHER CAUSES

RISK = SLEEP PRONE

NO CONTROL

OVER CONTROL

GROUP

VARIATION IN

TREATMENT OR METHOD

NON-SIMILAR CONDITIONS

SocialPersonal

Comorbid conditionsOther treatments

Etc.

Not usually accepted by

medical journals(accepted in

popular press, not reviewed)

CURRENT GROUP OF PATIENTS

Problems of looking back

RANDOMIZED & CONTROLLED TRIAL (RCT)RANDOMIZED & CONTROLLED TRIAL (RCT)


Control Group

PROSPECTIVEPROSPECTIVE

MAY BE MAY BE BLINDEDBLINDED

START WITH YOUR TARGET POPULATION


Set CRITERIA forSet CRITERIA forINCLUSION / EXCLUSIONINCLUSION / EXCLUSION

This will determine:This will determine:

ELIGIBILITYELIGIBILITY at the start at the start

VALIDITYVALIDITY at the end at the end

ELIMINATE THOSE WHO DO NOT MEET THE CRITERIAELIMINATE THOSE WHO DO NOT MEET THE CRITERIA

NEXT: GATHER A SAMPLE GROUP

THE SAMPLE GROUP WILLTHE SAMPLE GROUP WILL::

•Represent the target populationRepresent the target population•Meet the criteria for inclusion / exclusionMeet the criteria for inclusion / exclusion

SIDE NOTES…SIDE NOTES…

Study should be approved by an Study should be approved by an Ethics CommitteeEthics Committee

Informed consent should be Informed consent should be obtained from study participantsobtained from study participants

SAMPLE GROUP SAMPLE GROUP MAYMAY BE SUBDIVIDED FURTHER BE SUBDIVIDED FURTHER

STRATIFICATIONSTRATIFICATIONDivide into subgroups based onDivide into subgroups based on important similar characteristicsimportant similar characteristics

RANDOMIZATIONRANDOMIZATIONDivide into sub-groups based on Divide into sub-groups based on unknown confoundersunknown confounders

STRATIFICATIONSTRATIFICATION

““important similar characteristics”important similar characteristics”

Examples:• Male or Female• Age • Stage of illness • Prior illness or treatment • Hospital vs Office groups• Comorbid condition• Etc.

EXAMPLE OFSTRATIFICATIONSTRATIFICATION

FEMALE

MALE

RANDOMIZATIONRANDOMIZATION

““unknown confounders”unknown confounders”

Examples:Examples:• Postal codePostal code• Month of birthMonth of birth• Random numberRandom number• Etc.Etc.

EXAMPLE OF RANDOMIZATIONRANDOMIZATION

DX IN JANUARY-JUNE

DX IN JULY-DECEMBER


Control Group

Next… Divide your sample group(s) into Divide your sample group(s) into STUDY GROUPSSTUDY GROUPS

““Test Group”Test Group”

““Baseline Group”Baseline Group”


Control Group

Next… Divide your sample group(s) into Divide your sample group(s) into STUDY GROUPSSTUDY GROUPS

Receives Experimental Receives Experimental InterventionIntervention

““Baseline GroupBaseline Group””• NothingNothing• ObservationObservation• ““Same” miscellaneous Same” miscellaneous

intervention (non-intervention (non-experimental)experimental)

• PlaceboPlacebo• ““Gold Standard” therapy - Gold Standard” therapy -

especially if unethical to do especially if unethical to do otherwise!otherwise!

““Test GroupTest Group””

ASSIGN PATIENTS TO STUDY GROUPS


Control Group

Use caution against bias!

Sample Group Study Groups


Control Group

STUDY INVESTIGATOR

usually assigns patients to study groups.

usually has a personal preference for the treatment or patient

might unconsciously “work harder” to make the study work with non-preferred candidates

= POTENTIAL FOR BIAS


Control Group

Use randomseparation

and assignment!

RANDOMIZED CONTROLLED TRIAL (RCT)RANDOMIZED CONTROLLED TRIAL (RCT)


Control Group

RANDOMIZED RANDOMIZED CONTROLLEDCONTROLLED TRIAL (RCT)TRIAL (RCT)



Control Group

FUTUREFUTUREPresentProceed with study


Control Group


EXPERIMENTAL EVENT RATE (EER)

CONTROL EVENT RATE

(CER)


Control Group

FUTUREPresent


EXPERIMENTAL EVENT RATE (EER)

CONTROL EVENT RATE

(CER)

““The Gold The Gold Standard”Standard”

Disadvantages of RCTExpensive

large # pts needed

Prolonged recruitment and follow-up time needed

Funding difficult to obtain except w/support of pharmaceutical companies (problematic!)

RETURN FROM DETOUR





Literary Search





5. Evaluate


HEIERARCHY OF EVIDENCEHEIERARCHY OF EVIDENCE (value of study design to maximize wt, minimize bias)(value of study design to maximize wt, minimize bias)1. Systematic Review of all relevant RCTs

2. At least one properly designed RCT

3. Trials and case studies

4. Well-designed Controlled Trial without Randomization

5. Well designed Cohort or Case Control Studies, preferably from >1 centre or group

6. Multiple Time series with or without intervention

7. (Exception: Dramatic results in uncontrolled trials, such as introduction of PCN in the 1940s)

8. Opinions of respected authorities, based on

9. Clinical expertise

10. Descriptive studies

11. Reports of Expert Committees





Literary Search





5. Evaluate


INFORMATIONAdequate resources?Ease or Difficulty of finding and getting desired information?Costs?

INTERVENTIONPatient response or acceptance?Ease or Difficulty of Application?Clinical outcomes?

EBM PROCESSEFFECT ON PRACTICEWill this particular experience change our thinking or practice?

SELF EVALUATIONHow did we do? (Question, Search, Appraise, Apply)How could we improve our own EBM performance?

Evaluate

EBM: PROS, CONS and LIMITATIONS

PROS

Clinicians update knowledge base routinely

Improved understanding of research methods

Physician becomes more critical in use of data

Increased confidence in management decisions

Increased computer literacy, data search technology

Better reading habits

Provides framework for group problem solving, team generated practice

Transforms weakness or paucity of knowledge into positive change

OK to be uncertain

OK to be skeptical

OK to be flexible

Integrates medical education, research and clinical expertise

Can be learned by non-clinicians – other HCWs, patient groups, purchasers, etc.

Allows us to keep up with our better-educated patients!

Increased contribution of junior MDs

Increased patient benefit

Better communication with patients re: rationale of management decisions

Promotes better and more appropriate use of limited resources

May reduce costs or medical care or practice by eliminating outdated or unnecessary factors

Can be learned at any stage of physician’s career

CONSTime consuming

Information overload

Time to learn and practice

Time may be needed for team conferencing, planning and review

Takes $$$ to establish resource infrastructure – library, office, etc.

computers, peripherals

Internet costs

Programs, software information, CD-ROMS

Subscription costs – online and paper resources

May increase cost of care (but hopefully offset by elimination of unnecessary medical interventions, tests, journals, etc. – plus save time in getting proper intervention)

Online references made to unavailable journals or references

Exposes gaps in the evidence (but provides ideas for researchers!)

Requires computer skills (but can be done with minimal computer literacy and skill)

May expose your current practice as obsolete or dangerous (loss of authority and respect)

LIMITATIONS

Lack of evidence (shortage of studies)

Difficulty applying evidence to care of a particular patient

Barriers to the practice of high quality medicineLack of skills (search, appraise, etc.) (foster development of new skills!)

Lack of time to learn and practice EBM (promotes lifelong learning thru better focus)

Lack of physician resources for instant access to evidence (EBM has worldwide applicability)

RESTRICTED AVAILABILITY OF LAB TESTS

NON-TEXTBOOK CASEco morbidity, additional risk factors

AFFORDABILITY (MD & PT)“I can’t afford to practice EBM.”

Language barriers – available evidence may be unreadable, should be included

Physician attitude: Can be the greatest limitation!

“It decreases the importance of my clinical expertise”(that’s a necessary component!)

“It only applies to those involved in research.” (promotes cooperation among multiple physicians)

“It ignores patient values and preferences.”

“It’s just another cookbook approach to medicine.”

“It’s a poorly disguised way to cut medical costs.” (cost of care may actually increase)

“It’s a way to ration care and resources.” (Provides better utilization of avail resources)

DISAGREEMENT

Pt’s comfort, choice, acceptance, values preferencesVs MD’s recommendations

DOES RISK OR SIDE EFFECTS OF TREATMENT OUTWEIGHT THE BENEFITS?

The unanswered question…

“DOES EBM REALLY MAKE A DIFFERENCE?”

Effect of practicing EBM on patient outcome is actually unknown – no studies done

EBM good based on population studies: (ie: Pts who rec’d ___ generally fare better than those who don’t)

EBM IN DEVELOPING COUNTRIES

LIMITED RESOURCESMay help to eliminate unnecessary or poor quality screening tests (ie: resting EKG to screen for CAD = high false negative and false positive rates)

LIMITED DRUG REGULATIONApproval for drug marketing easy - promotes insurgence of new drugs for questionable indications, limited effectiveness, false claims, inflated prices based on ad response (include “more expensive is better”)


LIMITED CAPACITY FOR CMEDrug companies - may sponsor meetings that are little more than captive marketing sessions or biased education sessions (drug education vs promo)

Result may be push for more expensive, less effective treatments (ie push for CCB’s over BB’s) - calc channel blockers over Beta Blockers


LIMITED ACCESS TO LITERATURE DATABASES

Desktop computer with CD ROM reader and modem ($900)Electricity1 yr subscription to MedLine on CD ROM (?500)Internet connection $25/mt

Convince administrators of expense: Publicly cite how searches help with lectures, research and patient care management decisions

Get equipment from drug companies (usually strings attached)


LIMITED ACCESS TO ADEQUATE LIBRARY FACIILITIES

ALMOST INEVITABLE IN DEVELOPING COUNTRIESIdentify resources via search, but then unable to retrieve articles!

A top EBM practitioner (Philippines) recommends: 1. Top 3 medical libraries in your country2. Multinational drug company libraries3. Friends and colleagues - including in other countries


QUESTIONABLE APPLICABILITY OF ARTICLES RETRIEVED

Article describes a treatment that worked in one country, but seems impossible in yours

Check…• Are there pathophysiologic differences?• Will patient differences diminish the treatment response?• Patient compliance issues?• Provider compliance issues?• Co-morbid conditions which will alter the benefits or risks?


OBSTACLES TO TEACHING OR LEARNING EBM

Your Hospital or Institution does not reimburse for time spent on Continuing Medical Education programs

The standard 5-day workshop would be far too costly to provide or attend!

Need to learn the basics - computer skills, etc.

TRY THESE!Combine efforts to learn more and practice EBM with handful of colleagues (small group learning)

Ask about basis for information provided by drug reps, medical supply companies, etc. It will prompt them to provide you with on the spot teaching and better information, too!

EBM LIBRARY

BASIC REQUIREMENTSConvenient – easy access at point of contact with patient if

possible

Current – Up to date information

Electronic Database – Should be included• Online• CD-ROM

ELECTRONIC DATABASES

Evidence-Based Medicine Reviews (EBMR) – from Ovid (ovid.com)- combines Cochrane, Best evidence, Evidence Based Mental health, EB Nursing, Cancerlit, healthstar, AIDSline, Medline, and journal links (Described by one EBM specialist as “the best”)

Cochrane Library – “Gold Standard” for systematic reviews

Best Evidence

Medline – world’s largest, free resource – over 10 million references

PERSONNELMedical LibrarianInformatics Specialist

“We can learn a great deal about current best information sources from librarians and other experts in medical informatics, and should seek hands-on training from them as an essential part of our clinical training.” (ch 2 p29-30 – Blue circled 2)

PRINTED RESOURCES

TEXTBOOKSmost obsolete!Some updated yearly, plus heavy references and scientific evidence for support

Clinical Evidence (BMJ Publishing Group & ACP – 1999-present)

Evidence-Based On Call (http//cebm.jr.ox.uk/eboc/eboc.html)

Up To Date (General medicine, CD format, Medline abstracts used for evidence)

Scientific American Medicine – limited references from Medline, Harrisons

JOURNALSTraditional Journals subject to author submissionsspecialists need to read and evaluatemay subscribe to services that send articles of interest to your specialtytimely, instant information at time of publicationEx: NEJM, Clinical Nephrology, etc.

Evidence Based journals selects best studies from multiple journals of interest, summarizes best evidenceGood for use by generalists Lag time from original publication: 3-6 monthsEx: Evidence Based Medicine, Evidence Based Nursing, Evidence Based CV Medicine, etc.

SPECIAL RESOURCESWHO Blue trunkHinari

PATIENT RESOURCESMedical treatments www.nlm.nih.gov/medlineplusMedical guidelines www.guideline.gov

http://www.nlm.nih.gov/medlineplus

http://www.guideline.gov/

EVIDENCE BASED MEDICINEEVIDENCE BASED MEDICINE

evidence based medicine a new approach to clinical care and research

Documents

clinical practice

medical practice

clinical experience

important of ebm

clinical care

systematic reviews

case reports

metaanalysis reportswhat