oregon epcoregon epc evidence-based practice centers created in 1997; now 13 centers produce...
TRANSCRIPT
OREGON EPC
Evidence-based Practice Centers
• Created in 1997; now 13 centers• Produce
– “evidence reports” – systematic reviews– technology assessments– “rapid reviews”– meta-analyses and cost analyses– analysis of large databases
• Work with public and private sector partners
OREGON EPC
Evidence-based Medicine
Mark Helfand, MDDirector
Oregon Evidence-based Practice Center
OREGON EPC
What is the kind and strength of the
evidence you are relying on to make a recommendation?
The Question:
OREGON EPC
What does evidence-based mean?
• A comprehensive, systematic, open minded review of all the evidence
• The evidence determines the conclusion, not vice versa
• Not, the citation of papers supporting a preformed conclusion (and trashing of those that don’t)
• Not, the use of evidence when it is ‘positive’ but judgement when it isn’t
OREGON EPC
Systematic literature reviews
• Are systematic to remove bias in finding and reviewing the literature.
OREGON EPC
Systematic literature reviews
• Are systematic to remove bias in finding and reviewing the literature.– Experts may interpret the data (and
their own experience) differently.
OREGON EPC
How sure are we?Expert estimates of breast
implant rupture rates
0% 0.2% 0.5% 1% 1% 1% 1.5% 2% 3% 3% 4%
5% 5% 5% 5% 5% 5% 5% 5% 6% 6% 6% 8%
10% 10% 10% 10% 13% 13% 15% 15% 18%
20% 20% 20% 25% 25% 25% 30% 30% 40%
50% 50% 50% 62% 70% 73% 75% 75% 75%
75% 80% 80% 80% 80% 80% 80% 100%
Source: Dr. David Eddy
OREGON EPC
Experts estimates of the effect of colon cancer screening on
chance of dying
0% 25% 50% 75% 100%
Source: Dr. David Eddy
OREGON EPC
Experts’ estimates of probability of acute retention in men with
BPH
0
5
10
15
20
25
30
35
0% 20% 40% 60% 80% 100%
Number ofRespondants
Source: Dr. David Eddy
OREGON EPC
Systematic literature reviews
• Are systematic to remove bias in finding and reviewing the literature.– Studies with disappointing results
may get less attention
OREGON EPC
Trial Number Groups RESULTS
114 302 40mg bid 80mg bid
Total improvement at all doses compared with PLACEBO
115 419 20 mg bid60 mg bid 100mg bid
Total improvement at all doses compared with PLACEBO
106 139 20 mg bid, 60mg bid
Borderline improvement at 60 mg dose compared with PLACEBO
104 153* 20 mg bid, 40 mg bid
No improvement compared with placebo at either dose.
303 (32 wks)
294 20 mg bid, 60 mg bid 80mg bid
Lower relapse rate (31% to 36%) vs. PLACEBO (57%)
*Excludes 5 mg bid group
OREGON EPC
Trial 114
OREGON EPC
Systematic literature reviews
• Are systematic to remove bias in finding and reviewing the literature.– Experts may underplay controversy
or select only supportive evidence
OREGON EPC
Simpson et al, 2004
OREGON EPCSimpson et al, 2004
OREGON EPC
OREGON EPC
In a double-blind study vs risperidone…GEODON sustained control of positive symptoms at 1 year
1
OREGON EPC
In a double-blind study vs risperidone…GEODON sustained control of positive symptoms at 1 year
1
OREGON EPC
Systematic literature reviews
• Are systematic to remove bias in finding and reviewing the literature.– Experts may underplay controversy or
select only supportive evidence
• Emphasize the best evidence
OREGON EPC
The best evidence
• Reflects patients’ concerns– By addressing health outcomes
patients, their caregivers, and families care about
OREGON EPC
The best evidence
• Reflects patients’ concerns– By addressing health outcomes
patients, their caregivers, and families care about
•Help you feel similar to other people•Help you feel less lonely and removed from others•Help you feel more hopeful and happy•Allow you to think and express yourself more clearly
OREGON EPC
Selecting questions
• Researchers often use their own curiosity or research interest as the basis for selecting questions.
• They often use “standard” scales and measures instead of seeking a deeper understand of the patient’s well-being and quality of life.
OREGON EPC
Selecting questions
• Our premise is that important questions arise from practice, and from life. “Experts in practice”--and patients--select the populations, interventions, and outcome measures of interest.
OREGON EPC
The best evidence
• Reflects patients’ concerns– By addressing health outcomes
patients, their caregivers, and families care about
– By using simple measures of benefit and risk
OREGON EPC
Example
Relative benefit
PRESS RELEASE
“half as many patients treated with DRUG A experienced dry mouth.”
Absolute benefit
DATA FROM STUDY
1/100 vs. 2/100 or1 in 10 vs. 2 in 10
972
OREGON EPC
• Define the strengths and limits of the evidence.
• Clarify what is based on evidence and what is based on other grounds.
• Do not necessarily tell you what to do when the evidence is limited. Other factors, such as equity, clinical judgment, values, and preferences play a role in using the evidence.
Why use systematic literature reviews?
OREGON EPC
+
= Evidence-based decision-
making
+
OREGON EPC
An evidence-based decision process
• Makes use of an independent, systematic review of the evidence
• Employs rules for linking evidence to recommendations
• Produce explicit, defensible recommendations
OREGON EPC
Oregon ApproachWhat are we after?
• Systematic drug-class reviews should address questions that reflect clinicians’ and patients’ concerns.
• Decision-makers should begin to wrestle with the idea of what is good evidence.
• Manufacturers should gain market share if they produce good evidence of superiority over other drugs in a class.
• Patients, caregivers, payers (and NAMI) should demand better evidence about outcomes that matter !
OREGON EPC
OREGON EPC
Drug Class Review on
Atypical Antipsychotics
OREGON EPC
Included Drugs
Clozapine not posted
risperidone (1993) not posted
olanzapine (1996) not posted
quetiapine (1997) not posted
ziprasidone (2001) posted
aripiprazole (2002) posted
OREGON EPC
Eligible Outcomes
OREGON EPC
Results• 196 studies included overall
– 33 head-to-head – 24 placebo-controlled– 58 active controlled– 63 observational studies– 18 systematic reviews
• 427 study publications excluded
OREGON EPC
SchizophreniaHead to Head Trials
• 3 Effectiveness Trials– 12 month pragmatic trial of olanzapine,
risperidone or continuing typical AP– One 12-month switching study of olanzapine
& risperidone– InterSept trial of clozapine and olanzapine
to prevent suicidality found clozapine superior
• 30 Efficacy Trials
OREGON EPC
Head to head trials in outpatients
OREGON EPC
Summary: Benefits
•Clozapine, olanzapine and risperidone had similar efficacy with two exceptions
–Clozapine > olanzapine in suicidality/suicide prevention
– Olanzapine > risperidone in reducing rates of relapse
•Aripiprazole, quetiapine, and ziprasidone: Evidence too limited to say
OREGON EPC
•Weight gain•Greater risk for olanzapine than risperidone •Results mixed in long-term observational studies
•Diabetes mellitus•Risk greater for olanzapine than risperidone, but studies had mixed results•Risk with clozapine relative to others not clear •Limited evidence on quetiapine
•Other long-term safety•No conclusions about comparative safety can be made
Summary: Harms
OREGON EPC
Other harms
• Movement disorders• Somnolence• Hyperprolactinemia/sexual
dysfunction• Long QT interval• Bone marrow problems
OREGON EPC
Outpatient studies
Better head-to-head comparisons of antipsychotics are needed to discern the relative efficacy and safety profiles of these compounds.
OREGON EPC
What we can do together
1. select and refine questions that puts patients’ and caregivers’ concerns center stage
2. Rely on unbiased reviews to inform patients, families, and clinicians
3. Promote an evidence-based process, not just systematic reviews.
4. Promote higher standards for evidence about treatments for mental illnesses
OREGON EPC
Observational Studies: Long-term Safety
• 48 studies, 6 months in duration• primarily schizophrenia patients• 8 head-to-head cohort studies• 10 AAP versus typical AP cohort studies• 29 descriptive epidemiologic studies
• 1 case-control study • Death: Rates ranged from 0.1% to 3.3% for
clozapine, quetiapine and risperidone (7 uncontrolled studies)
OREGON EPC
Criticism
• “By adhering to rigorous rules of inclusion, the process maximizes the validity of assessing proven treatment efficacy (strength), while it ignores or discards other germane but less statistically rigorous evidence of real-world effectiveness and cost-effectiveness (weakness).
OREGON EPC
Our response
• We agree controlled trials ignore important aspects of effectiveness…
OREGON EPC
Limitations of RCTs
• There aren’t enough of them.
• They test interventions that may or may not fit easily into practice.
• They often don’t tell you about important subgroups.
• They may not extend for a long time.
OREGON EPC
More limitations of RCTs
• Design features are poorly adapted to the purpose of assessing average effectiveness– Populations
• run-in periods• Exclusions
– Comparators and comparisons– Outcome measures
• Followup period
– Feasibility• Implementation costs• Maintenance costs
OREGON EPC
Most common problems with head-to-head trials
• Doses of the different drugs aren’t equivalent.
• Strategies for using the drugs aren’t realistic.
• Usually, focus on efficacy or harms but not on both
• Do not address all important outcomes
OREGON EPC
RCTs & harms
• Design features are poorly adapted to the purpose of assessing harms– run-in periods– exclusions of susceptible people
• Reporting is poor•unreported•Selectively reported•Misleadingly reported•Lack of severity data
OREGON EPC
Applicability: How to bias an efficacy study and still
get a “good-quality” rating
• select compliant patients• dilute the control group
interventions• measure only certain outcomes• cheat
– selective use of cut-off dates– what are the norms?
OREGON EPC
• We agree controlled trials ignore important aspects of effectiveness…
• and agree on what information we’d like to have.
OREGON EPC
Quality of the evidenceat 4 levels
1.Type of study.2.Quality of each study
based on study design.3.Overall quality of the
evidence for a key question.
OREGON EPC
1. Types of studies
• case reports, case series• animal studies• studies of etiology• prospective cohort studies• “open-label” controlled or
uncontrolled studies• randomized trials
OREGON EPC
2. Quality of individual studies
• quality (“good,” “fair,” or “poor”) for each type of study design
• Use of random allocation• Concealed allocation• Double-blind method• Exclusions after randomization
• applicability
OREGON EPC
• Initial assembly of comparable groups
• Maintenance of comparable groups• Minimal loss to follow-up• Measurements: equal, reliable, valid• Clear definition of interventions• All important outcomes considered• Intention-to-treat analysis OHSU EPC
Internal Validity Criteria for RCTs & cohort studies
OREGON EPC
3. Evidence at each linkage
• Aggregate internal validity: Are there any studies with good design (for the question) that were also well-conducted? Is the “best evidence” of good internal validity?
• Consistency/coherence: Do studies conflict in their findings? Is there a body of supporting evidence so that the “best evidence” makes sense?
OREGON EPC
3. Rating each link in the AF
• Quality and consistency of studies– large numbers of patients– consistent results across studies
• Applicability of studies– patient populations, interventions,
outcomes like those of interest to the organization
– “real life” evidence not just “efficacy”– attention to harms
OREGON EPC
• Define the strengths and limits of the evidence.
• Clarify what is based on evidence and what is based on other grounds.
• Do not necessarily tell you what to do when the evidence is limited. Other factors, such as equity, clinical judgment, values, and preferences play a role in using the evidence.
Systematic literature reviews
OREGON EPC
What Does it Mean for Decisions
to be “Evidence-Based”?
• Decisions are based on “best evidence”
• Best evidence:– Is unbiased– Is appropriate for decision at hand– Includes all germane evidence
Luce
OREGON EPC
An evidence-based decision process
• Makes use of an independent, systematic review of the evidence
Employs rules for linking evidence to recommendations
Produce explicit, defensible recommendations
OREGON EPC
Strength of recommendations
Estimate of Net Benefit (Benefit Minus Harms)
Quality of Overall Evidence Substa
ntial Moderate Small Zero/
Negative Good A B C D Fair B B C D Poor I – Insufficient Evidence
OREGON EPC
Strength of recommendations
Estimate of Net Benefit (Benefit Minus Harms)
Quality of Overall Evidence Substa
ntial Moderate Small Zero/
Negative Good A B C D Fair B B C D Poor I – Insufficient Evidence
OREGON EPC
What is evidence-based medicine?
“Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values.”
David Sackett
OREGON EPC
What is evidence-based medicine?
• Where there is evidence of benefit and value, do it
• Where there is evidence of no benefit, harm, or poor value, don’t do it.
• When there is insufficient evidence to know for sure, be conservative
David Eddy
OREGON EPC
OREGON EPC
Evidence-based Practice Centers
• Created in 1997; now 13 centers• Produce
– “evidence reports” – systematic reviews– technology assessments– “rapid reviews”– meta-analyses and cost analyses– analysis of large databases
• Work with public and private sector partners
OREGON EPC
Oregon Evidence-based Practice Center
• USPSTF• Drug class reviews for states• Food claims for FDA• Various other topics
– HBOT for cerebral palsy– Rehabilitation for traumatic brain injury– Treating actinic keratoses– Telemedicine– VBAC– Osteoporosis diagnosis and treatment– Preventing youth violence
OREGON EPC
Oregon Evidence-based Practice Center
EVIDENCE REPORTS FOR DRUG CLASSES:
http://www.ohsu.edu/drugeffectiveness/reports/
USPSTF RECOMMENDATIONS:
http://www.ahrq.gov/clinic/uspstfix.htm
OREGON EPC
Criticism 3. EBM hurts minorities and vulnerable
populations-- “each drug is unique”-- “each patient is unique”-- “doctors should be able to choose
any drug for any patient”
OREGON EPC
Other study designs could be helpful, after the following questions are answered:
• Will our users find them credible enough to use them?
• Can it be identified, introduced into the review in a systematic way?
• Can we tell a good outcomes study from a poor one?
• Can we tell a good economic study from a poor one?
• Can users incorporate it into decisions in a meaningful way?
OREGON EPC
Most common problems with observational studies of adverse
events• Incomplete ascertainment• Few data on severity of the event• Don’t report on efficacy (to
examine trade-offs)• Confounding, bias
OREGON EPC
Level 1: “Would you have this done for yourself or for someone else in your immediate family?”
Influenced by one’s personal experience with the disease and capacity to deal with risk.
Affects few people.Level II: “What would I recommend to my
patient/client?”Physician making a recommendation for his/her
patients. Influenced by prior experience, but the scientific evidence may play a greater role.
Affects possibly hundreds of people.Level III: “What would I recommend to the nation, the
world?”Across-the-board recommendations for a
population. Must be based on rigorous assessment of the
scientific evidence.Affects hundreds of thousands, even millions of
people.
OREGON EPC
1998—First FDA application 2001—FDA approval for schizophrenia2004—Approval in acute maniaAugust, 2004—Warning hyperglycemia and diabetes April, 2005—Warning on “off-label” use in elderly (olanzapine), Abilify (aripiprazole), Risperdal (risperidone), and Seroquel (quetiapine).
June, 2005—Lilly settles Zyprexa suits