Using Evidence to Guide Drug Therapy Decisions

Dean Haxby, Pharm.D.

Associate Professor of Pharmacy

Oregon State University, College of Pharmacy

To receive 1.5 AMA PRA Category 1 Credits™, you must review this progam and pass the CME

quiz at the end.

Release Date: January 2009 Expiration Date: January 2012

Attachments

• The attachments tab contains documents that supplement the presentation.

• The slides are available as an attachment to print out to use as a handout.

• Another attachment provides a listing of additional resources

Program Funding

This work was made possible by a grant from the state Attorney General Consumer and Prescriber Education Program which is funded by the multi-state settlement of consumer fraud claims regarding the marketing of the prescription drug Neurontin.

Continuing Education Sponsors

Continuing Medical Education for the following activity titled “Using Evidence to Guide Drug

Therapy Decisions”, is jointly sponsored by The University of Texas Southwestern Medical Center

and the Federation of State Medical Board’s Research and Education Foundation.

Program Speaker/Author: Dean Haxby, PharmD,

Course Director: Barbara S. Schneidman, MD, MPHFederation of State Medical Boards Research and Education Foundation, SecretaryFederation of State Medical Boards , Interim President and Chief Executive Officer

Program Directors: David Pass, MDDirector, Health Resources Commission, Oregon Office for Health Policy and ResearchDean Haxby, PharmDAssociate Professor of Pharmacy Practice, Oregon State University College of Pharmacy Daniel Hartung, PharmD, MPHAssistant Professor of Pharmacy Practice, Oregon State University College of Pharmacy

Target Audience: This educational activity is intended for health professionals who are involved with medication prescribing, and those that are involved with committees involved with medication use policies.

Educational Objectives: Upon completion of this activity, participants should be able to: Describe the advantages and limitations of evidence-based medicine (EBM), and its role in improving prescribing decisions; Describe the steps in the EBM process; Identify strategies to search for evidence-based medical literature, including useful web sites; Review key factors that should be appraised with systematic reviews, clinical practice guidelines, and individual RCTs; Describe the influence different methods of presenting study results can have on decisions; Given the results of a study calculate ARR and NNT; Discuss how EBM principles can be applied to evaluating new drugs.

CME Information

CME PoliciesAccreditation: This activity has been planned and implemented in accordance with the Essential Areas & Policies of the Accreditation Council for Continuing Medical Education through the joint sponsorship of The University of Texas Southwestern Medical Center and the Federation of State Medical Boards Research and Education Foundation. The University of Texas Southwestern Medical Center is accredited by the ACCME to provide continuing medical education for physicians.

Credit Designation: The University of Texas Southwestern Medical Center designates this educational activity for a maximum of 1.5 AMA PRA Category 1 Credits™. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Conflict of Interest: It is the policy of UT Southwestern Medical Center that participants in CME activities should be made aware of any affiliation or financial interest that may affect the authors presentation. Each author has completed and signed a conflict of interest statement. The faculty members’ relationships will be disclosed in the course material.

Discussion of Off-Label Use: Because this course is meant to educate physicians with what is currently in use and what may be available in the future, “off-label” use may be discussed. Authors have been requested to inform the audience when off-label use is discussed.

DISCLOSURE TO PARTICIPANTS

It is the policy of the CME Office at The University of Texas Southwestern Medical Center to ensure balance, independence, objectivity, and scientific rigor in all directly or jointly sponsored educational activities. Program directors and authors have completed and signed a conflict of interest statement disclosing a financial or other relationship with a commercial interest related directly or indirectly to the program.

Information and opinion offered by the authors represent their viewpoints. Conclusions drawn by the audience should be derived from careful consideration of all available scientific information. Products may be discussed in treatment outside current approved labeling.

FINANCIAL RELATIONSHIP DISCLOSURE

Faculty Type of Relationship/Name of Commercial Interest(s)

David Pass, M.D. NoneDean Haxby, Pharm.D Employment/CareOregonDaniel Hartung, Pharm.D., MPH NoneBarbara S. Schneidman, MD, MPH None

Learning Objectives

• Describe the advantages and limitations of evidence-based medicine (EBM), and its role in improving prescribing decisions

• Describe the steps in the EBM process • Identify strategies to search for evidence-based

medical literature, including useful web sites• Review key factors that should be appraised with

systematic reviews, clinical practice guidelines, and individual RCTs

Learning Objectives

• Describe the influence different methods of presenting study results can have on decisions

• Given the results of a study, calculate ARR and NNT

• Discuss how EBM principles can be applied to evaluating new drugs.

Problems With Drug Therapy• Over the past 15 years, drug costs have

escalated at a higher rate than any other area of health care

• The US has 5% of the worlds population, but has 50% of drug consumption

• The US ranks very low in measures of health care quality

• Overuse, underuse and inappropriate use of medications are an important quality issue

• Many of these problems are preventable

Limitations of the FDA Drug Approval Process

• FDA review determines if a drug is effective and has acceptable safety.

• It does not determine place in therapy.• Value not assessed.• Often there are many unanswered questions at the

time a drug is approved.– Limited numbers and types of patients– Limited duration of studies– May not know outcomes of greatest interest

• Much of the FDA funding comes from industry

What is Evidence-based Medicine?

• A process to identify and use the best available scientific evidence to guide decision making– Patient care– Policy

• The scientific evidence is integrated with clinical judgment and the patients unique circumstances to provide the best clinical decisions

• The result is predictable improvement in patient outcomes

David Eddy. Formulary 2002;37:525-30

EBM Drug Therapy Decisions

• When there is evidence of benefit and value, do it.

• When there is evidence of no benefit, harm or poor value, don’t do it.

• When there is insufficient evidence to know for sure, be conservative.

Decision Making in Clinical Practice

• Unfortunately, many decisions are made based on unreliable “evidence”

• Personal observation or anecdotal experience – Tends to overestimate efficacy

• Reasoning based on pharmacology, pharmacokinetics or theory instead of “patient oriented evidence that matters”

• Use of observational studies or case series to draw cause and effect conclusions

• Expert opinion plays a heavy role in medical decision making– Opinions vary– Unbiased expert opinion using EBM is very useful– The consensus approach based on uncontrolled clinical experience

risks widespread application of useless or even harmful treatment

Advantages of Using an EBM Approach

• Guides prescribing decisions to get predictable improvements in patient outcomes.

• Helps sort through the marketing, opinions, and theory to get an accurate assessment of the benefits and risks of various treatments.

• Can help identify and target opportunities to improve drug therapy.

• Assists with decisions about use of limited resources.

• Provides incentive to conduct useful research.

Limitations of EBM

• Studies may not answer the question you are trying to answer– Evidence may be lacking or of low quality– Surrogate endpoints

• The body of evidence continues to change• It can take a lot of resources to conduct clinical

trials or create original high quality evidence reviews

• It does not make clinical decisions or value judgments about cost versus benefit– But it can assist you in doing so

Steps in the EBM Process• Formulate the question(s) to be answered• Gather Evidence

– Decide on type of literature– Conduct search

• Critical appraisal to identify the best evidence • Evidence summary

– Evaluating the strength of the evidence– Magnitude benefits and risks (ARR,NNT, NNH)

• Form recommendations/conclusions and apply the evidence

Step 1: Formulate Clinical Question(s)

• An important and sometimes difficult step

• Well constructed questions guide the process

• Helps define what you really want to know

Clinical Questions• Ask for information about managing patients • Contain several important components

(PICOS)– 1. Population and/or clinical problem/disorder– 2. Intervention– 3. Comparison intervention (placebo, gold standard)– 4. Outcomes

• helps define what is most important– 5. Setting

Clinical Question Example• For cigarette smokers, does varenicline differ

in efficacy for promoting long term smoking cessation than bupropion SR or NRT in the primary care setting?

• Population: cigarette smokers• Intervention: varenicline• Comparator: bupropion, NRT, placebo• Outcome: smoking cessation at one year• Setting: Primary Care

Step 2: Gathering Evidence

• When evaluating drug therapy, several types of literature are typically most useful– Randomized controlled trials – Systematic reviews of RCTs– Evidence-based CPGs

• The goal is to locate the best available evidence to answer your questions

Evidence Hierarchy

Lab studies and Animal Research

Case Reports

Case Control Studies

Systemati

cReviewsRCTs

Cohort Studies

Case Series

Editorials and Opinions

Strengths of Randomized Controlled Trials (RCT)

• The foundation for evidence-based evaluation of drug therapy

• If high quality, can establish benefits and harms of drug therapy

• Effective randomization minimizes risk of unexpected factors influencing results (confounding)

• Fortunately, RCTs are required for FDA approval

Limitations of RCTs

• Experimental design and inclusion-exclusion criteria may not reflect general practice and make it difficult to apply or generalize results (external validity)

• Outcomes trials in chronic diseases may take years

• Expensive• It takes time and effort to review individual

studies once they are published

Systematic Reviews (SRs)

• Usually a good starting place when gathering evidence about drug therapy

• A SR is an evidenced-based review of the medical literature up to a certain time point

• Predefined, detailed methods are used to search, screen, critically evaluate, and summarize the medical research to answer specific questions

• SRs can be qualitative or quantitative (meta-analyses)• Meta-analyses combine results of individual studies, but not

all meta-analyses are SRs • High quality SRs save time and can provide a clearer picture

of the body of evidence than individual studies

Traditional Reviews

• Typically lack the rigor of a systematic review

• Can be more of an opinion piece rather than a careful unbiased review of the literature.

• Selective use of literature can be used to support the authors point of view.

• The medical literature is full of non-systematic reviews.

Clinical Practice Guidelines

• Usually broader in scope than SR; goal - influence practice• Are especially useful if they are well developed and free from

bias– Political, financial or other factors can introduce bias

• Over 2500 guidelines are available • Vary in quality (evidence vs. consensus opinion) and

recommendations can vary, so need to carefully evaluate• High quality guidelines have two major components

– A systematic review of the evidence– Specific recommendations with explicit links to the evidence

and graded for strength of evidence

Searching for Evidence

• This program will focus on a few basic search strategies or useful sources to locate the following:– RCT’s– Systematic reviews– Clinical practice guidelines– Additional sources of information useful for

evaluating new drugs

Search Criteria

• Decide on the type of literature best suited to answer your question

• The clinical question components can help guide the search (PICOS)– Select search terms– Screening to make sure the literature is

addressing your question

Searching for Evidence: PubMed

• A very user friendly approach to Medline• Can access online through various medical libraries• Can also go to pubmed.gov• Enter your search terms based on your clinical question

– Example: varenicline smoking cessation • Click on limits tab and scroll down to “type of article”

– limit to types of literature you want (RCT, Meta-analysis and/or guidelines)

– other limits can also help target your search• Can adjust search based on results

PubMed: Searching for Systematic Reviews

• On the PubMed side bar on the left of the screen, click on “Clinical Queries”

• Scroll down to “Find Systematic Reviews”

• Enter your search terms based on your clinical question– This search retrieves systematic reviews, meta-

analyses and other EBM literature

High Quality Systematic Reviews • Drug Effectiveness Review Project (DERP)

– Contains systematic reviews of various drug classes– Developed by the OHSU Evidence-based Practice Center– http://www.ohsu.edu/ohsuedu/research/policycenter/DERP/

about/final-products.cfm– http://www.oregonrx.gov

• Canadian Agency for Drugs & Technologies in Health– www.cadth.ca/

• Therapeutics Initiative– www.ti.ubc.ca

High Quality Systematic Reviews

• Agency for Health Care Research and Quality Effective Healthcare Program

– http://effectivehealthcare.ahrq.gov

• Cochrane Collaboration (subscription)

– Can include unpublished data

– http://www.cochrane.org

• Center for Reviews and Dissemination (CRD)

– http://www.crd.york.ac.uk/crdweb

– Database of abstracts of reviews of effects (DARE)

Clinical Practice Guidelines

• National Guidelines Clearinghouse– www.guideline.gov

• New Zealand Guidelines Group– www.nzgg.org.nz/

• National Institute for Health and Clinical Excellence (NICE)– www.nice.org.uk/

• Veterans Health Administration– www.oqp.med.va.gov/cpg/cpg.htm– www.pbm.va.gov (monographs, drug use criteria)

FDA Website: fda.gov

• An important place to look for data regarding new drugs, but can be a difficult site to navigate

• The advisory panel transcripts can contain a wealth of information including information on studies not published, and analyses by fda staff

• Information about unresolved issues and post marketing requirements may be available

• Drug approval letter and product labeling

Information from Manufacturers• Package Insert

– A great starting point for a new drug review– Will indicate how many studies were used in the

review and approval of the drug

• AMCP Dossier Format– Can be a source of information on unpublished

studies– Contains an economic analysis

• http://clinicalstudyresults.org

Step 3: Critical Appraisal

• Once you have identified literature that meets your criteria, it needs to be critically appraised to decide if it is worth using.

• Critical appraisal is an important step in the EBM process.• This step helps identify the best evidence to answer our

question.• Much of the medical literature is not useful or reliable, and

in some cases may even be misleading.• With critical appraisal, we determine confidence in the

results (internal validity) and if the results are applicable to our practice (external validity).

Additional Programs on Critical Appraisal

• In this program, a very brief introduction to critical appraisal will be provided.

• For a more in-depth review, please see the following programs that are available from the same website you accessed this program:– “Critical Appraisal: Randomized Controlled Trials for

Drug Therapy ”

– “Critical Appraisal: Systematic Reviews and Clinical Practice Guidelines for Drug Therapy”

Evaluating Individual RCTs

• Internal validity reflects the confidence that the results are due to the intervention

• Focus on the methods and results sections

• Good studies provide a valid estimate of the efficacy of an intervention and are usually easier to review

• Form your own conclusions based on the above

Items to Evaluate for Internal Validity of RCTs

• Adequate sample size with statistical power

• Randomization methods appropriate

• Comparable groups at baseline

• Equal co-treatment• Compliance• Low dropout rate

• Adequate length of follow-up

• Blind assessment• Equal assessment• Intention-to-treat

analysis• Post-hoc analysis

Applicability (External Validity) of RCTs

• How relevant is the study?– Were clinically meaningful endpoints studied?– What is the magnitude of the treatment effect and is it

clinically significant?

• How generalizable are the results?– How comparable were the patients studied to your practice?– Does the study reflect real world practice?

• If internal validity unacceptable, the applicability is irrelevant

Pharmaceutical Industry Sponsored Studies

• Generally have acceptable internal validity

• Applicability is more often the issue– Active controls may not be gold standard– Surrogate outcomes often involved– Dosing regimens not comparable– Population may not reflect population of most

interest

• Publication bias

Items to Assess with Systematic Reviews

• Evaluate for bias, methods and generalizability

• Is there a clear, clinically relevant question that addresses what you are looking for, that was defined before hand?

• Were study eligibility criteria defined and appropriate?

• Was the search detailed and exhaustive?

• Were the studies critically appraised by more than one person?

• Were methods for data synthesis described?

• Were conclusions clear and reflect evidence?

• Was funding disclosed and potential conflict of interest minimized?

DARE

• Database of Abstracts of Reviews of Effects

• A useful resource for readers of SRs

• Provides concise summaries of published SRs of healthcare interventions

• Assesses strengths and weaknesses

• http://www.crd.york.ac.uk/crdweb and click on “DARE” tab

Critically Evaluate Clinical Practice Guidelines (CPG)

• Studies of guidelines have found that methodological problems, or potential conflicts of interest, are common

• Objectives, key questions and applicable population clearly defined

• Stakeholders involvement (relevant disciplines, patient views, target user pilot)

• Editorial independence from funding, methods to manage conflict of interest, bias

CPG Rigor of Methods

• Systematic methods used to search for evidence• Criteria for selecting evidence described• Appropriate and clearly defined methods for

formulating recommendations• Benefits, risks and costs considered • Explicit link between supporting evidence and

recommendations (grades of recommendations)• Guideline externally reviewed by unbiased experts

Step 4: Evidence SummaryRate Confidence in Strength of Evidence

High : Large, well-designed randomized controlled trials, rigorous systematic

reviews Small, or not-so-well designed controlled clinical trials Non-randomized prospective cohort studies Non-randomized case-control studies

Low : Case series

Evidence Summary: Quantify Results

RR: % treatment group % control groupRRR: 1-RR X 100%ARR: % difference between control and treatment

NNT: 100% ARR

RR= Relative Risk RRR = Relative Risk ReductionARR = Absolute Risk Reduction/absolute response rate NNT = Number Needed to Treat

Limitations of Relative Risk

• Example 1: risk of death is 1% Tx and 2% P– RRR is a 50% reduction– ARR is 1%– NNT= 100

• Example 2: risk of death: 25% Tx vs 50%P– RRR is a 50% decrease– ARR is 25%– NNT= 4

• Same RRR, but different clinical implications

Calculating ARR and NNT

• Useful if a statistically significant difference exists

– the confidence interval helps determine precision

– Important to link to the treatment duration

• Can only do this if specific responders are reported

• Can’t do calculations if results are presented as mean changes for the population.

• Examples:

– Mean change in BP vs. % reaching target BP

– Mean change in HgA1c vs. % below HgA1c of 7

Example Calculation

• 95% of patients with AOM are improved after 10days with gorillacillin vs. 85% on placebo (p<.05)

• What is the:– ARR = ?– NNT = ?

Example: Amitiza for IBS-C

• “patients receiving Amitiza were nearly twice as likely to acheive a statistically significant overall response compared to those given placebo” (from the monthly prescribing reference)

• Study 1: 13.8% respond to Amitiza vs 7.8% P• Study 2: 12.1% Amitiza vs. 5.7% P• Response = responded 2 out of 3 months• ARR = about 6% will respond 2 out of 3 months• NNT= For every 16 patients treated for 12 weeks, one

will respond two out of three months

Eddy DM.Formulary 2002;37:525-30

Step 5: Form Conclusions and Apply the Evidence

• When there is evidence of benefit and value, do it.

• When there is evidence of no benefit, harm or poor value, don’t do it.

• When there is insufficient evidence to know for sure, be conservative.

Do it Examples

• Thiazides for hypertension

• Aspirin to prevent CVD events

• ACEI in CHF

• Statins in secondary prevention

• Warfarin in atrial fibrillation

• Beta-blockers post-MI

Don’t do it Examples

• Estrogen/progestin to prevent CVD in late postmenopausal women

• Alpha-blockers in hypertension

• Calcium blockers with systolic ventricular dysfunction

• Zelnorm, Vioxx

• Antibiotics for acute URI

Eddy DM. Formulary 2002;37:525-30

What be conservative means

• With new drugs, burden of proof is on the manufacturer

• If we use a new treatment without adequate evidence, there is little incentive to do the research

• For old drugs, we should not actively promote without good evidence.

Summary: EBM Process• Formulate the questions to be answered• Gather Evidence

– Decide on type of literature– Conduct search

• Critical appraisal to identify the best evidence • Evidence summary

– Evaluating confidence in the strength of the evidence– Quantitative benefits and risks (ARR,NNT, NNH)

• Form conclusions and apply the evidence

Conclusions

• Using an evidence-based process results in prescribing decisions that lead to predictable improvements in patient care.

• There are numerous sources for high quality systematic reviews and clinical practice guidelines that provide a sound framework for drug therapy decisions.

• Prescribing agents that have the best evidence for benefits vs. harms, and that provide good value can help address some of the problems associated with prescription drug use.

Thank you

This work was made possible by a grant from the state Attorney General Consumer and Prescriber Education Program which is funded by the multi-state settlement of consumer fraud claims regarding the marketing of the prescription drug Neurontin.

CME Instructions

• Please complete the survey, quiz and program evaluation questions.

• Click the finish button.

• Fill out the form and send via fax, e-mail, or regular mail to the address on the form to receive CME credit.

PROPERTIES

On passing, 'Finish' button: Goes to URLOn failing, 'Finish' button: Goes to Next SlideAllow user to leave quiz: After user has completed quizUser may view slides after quiz: At any timeUser may attempt quiz: Unlimited times