CAT 2: CAT 2: TherapyTherapyMaribeth Chitkara, MDMaribeth Chitkara, MD

Rachel Boykan, MDRachel Boykan, MDStony Brook Long Island ChildrenStony Brook Long Island Children’’s s

HospitalHospital

Where Do We Get Where Do We Get Evidence?Evidence?

Your patientsYour patientsColleaguesColleaguesPublished Published

anecdotal casesanecdotal casesCase seriesCase series

Cohort studiesCohort studiesRandomized Randomized

control trialscontrol trialsSystematic Systematic

reviewreviewMeta-analysesMeta-analyses

ScenarioScenario

4 month old infant w/ RSV bronchiolitis4 month old infant w/ RSV bronchiolitis

Hospitalized for 2.5 days for O2 and IVFHospitalized for 2.5 days for O2 and IVF

Family h/o asthmaFamily h/o asthma

Parents wonder if thereParents wonder if there’’s s ““anything out anything out therethere”” to prevent the infant from developing to prevent the infant from developing asthmaasthma

Answerable Clinical QuestionAnswerable Clinical QuestionPICOPICO

Patient – Infant w/ RSV bronchiolitisPatient – Infant w/ RSV bronchiolitis

Intervention – Given a course of Intervention – Given a course of montelukastmontelukast

Comparison – Compared with infants Comparison – Compared with infants without montelukastwithout montelukast

Outcome – Decrease the subsequent Outcome – Decrease the subsequent asthma illnesses?asthma illnesses?

The SearchThe Search

Pubmed search MeSH databasePubmed search MeSH database

Terms Terms ““Respiratory Syncytial VirusRespiratory Syncytial Virus”” and and ““montelukastmontelukast””

You find a double-blind, placebo-You find a double-blind, placebo-controlled study of infants hospitalized controlled study of infants hospitalized with RSV bronchiolitis evaluating with RSV bronchiolitis evaluating efficacy of montelukast in reducing efficacy of montelukast in reducing subsequent asthma exacerbationssubsequent asthma exacerbations

Bisgaard J for the Study Group on Montelukast and RSV: A Bisgaard J for the Study Group on Montelukast and RSV: A Randomized Trial of Montelukast in RSV Postbronchiolitis. Am J of Randomized Trial of Montelukast in RSV Postbronchiolitis. Am J of Respir Crit Care Med 2003; 167:379-383.Respir Crit Care Med 2003; 167:379-383.

Is the study VALID?Is the study VALID?

What are the RESULTS?What are the RESULTS?

Can I APPLY the results to my patient?Can I APPLY the results to my patient?

ValidityValidity

Randomized trial?Randomized trial?

Was Randomization concealed?Was Randomization concealed?

Follow up long enough?Follow up long enough?

Patients accounted for at end of trial?Patients accounted for at end of trial?

Were patients, health care workers, data Were patients, health care workers, data collectors, and/or data analysts blinded to collectors, and/or data analysts blinded to treatment?treatment?

ValidityValidity

Were both groups similar at start of Were both groups similar at start of trial?trial?

Aside from the treatment itself, were Aside from the treatment itself, were both groups treated equally?both groups treated equally?

Did authors utilize an “Intention to Did authors utilize an “Intention to Treat” analysis?Treat” analysis?

Intention to TreatIntention to Treat

The principle of attributing all patients to The principle of attributing all patients to the group to which they were randomizedthe group to which they were randomized

Preserves the value of randomizationPreserves the value of randomization

Considers the “worst case scenario”Considers the “worst case scenario” Scenario 1: Lost experimental subjects Scenario 1: Lost experimental subjects

fare like control subjects, lost control fare like control subjects, lost control subjects fare like experimental subjectssubjects fare like experimental subjects

Scenario 2: Lost experimental subjects do Scenario 2: Lost experimental subjects do poorly, lost control subjects do wellpoorly, lost control subjects do well

Scenario 3: Lost subjects in both groups Scenario 3: Lost subjects in both groups do poorly do poorly

Per Protocol AnalysesPer Protocol Analyses

Only patients who completed research Only patients who completed research protocol in entirety are included in final protocol in entirety are included in final data analysisdata analysis

Excludes all non-compliant patientsExcludes all non-compliant patients

Leaves behind those who may be Leaves behind those who may be destined to have a better outcomedestined to have a better outcome

Has the potential to severely bias the Has the potential to severely bias the findings in the study or negate the findings in the study or negate the value of randomizationvalue of randomization

Therapy – ResultsTherapy – Results(per protocol)(per protocol)

Outcome Present

Outcome Absent

Totals

New Drug/tx

A B A + B

Placebo

C D C + D

Totals A + C B + D A + B + C + D

Control Event Rate (CER)Control Event Rate (CER) C / C + DC / C + D

Experimental Event Rate Experimental Event Rate (EER)(EER) A / A + BA / A + B

Absolute Risk Reduction Absolute Risk Reduction (ARR)(ARR) CER - EERCER - EER

Relative Risk (RR)Relative Risk (RR) EER / CEREER / CER

Relative Risk Reduction Relative Risk Reduction (RRR)(RRR) RRR = [1-RR] x 100RRR = [1-RR] x 100

Number Needed to Treat Number Needed to Treat (NNT)(NNT) 1 / ARR1 / ARR

ScenarioScenario

RAD

No RAD

Total

Lost to Follow UP

Montelukast

4 57 61 4

Placebo 10 45 55 10

Total 14 102 116

CER = 10/55= 0.18CER = 10/55= 0.18 EER = 4/61 = 0.07EER = 4/61 = 0.07 ARR = 0.18-0.07 = ARR = 0.18-0.07 =

0.110.11 RR = 0.07/0.18 = RR = 0.07/0.18 =

0.380.38 RRR = [1-0.38] x RRR = [1-0.38] x

100=62%100=62% NNT = 1/0.11= 9NNT = 1/0.11= 9

Bisgaard J for the Study Group on Montelukast and RSV: A Randomized Trial of Montelukast in RSV Postbronchiolitis. Am J of Respir Crit Care Med 2003; 167:379-383.

Per Protocol vs ITTPer Protocol vs ITT

Per protocolPer protocol ARR = 0.11ARR = 0.11 NNT = 9NNT = 9

ITT (Scenario 1)ITT (Scenario 1) ARR = 0.9ARR = 0.9 NNT = NNT = 1111

ITT (Scenario 2)ITT (Scenario 2) ARR = 0.3ARR = 0.3 NNT = NNT = 3232

ITT (Scenario 3)ITT (Scenario 3) ARR = 0.18ARR = 0.18 NNT = NNT = 55

Number Needed to Treat Number Needed to Treat (NNT)(NNT)

Your personal treatment thresholdYour personal treatment threshold

I would be willing to treat…I would be willing to treat…3 patients to see benefit in one3 patients to see benefit in one5 patients to see benefit in one5 patients to see benefit in one10 patients to see benefit in one10 patients to see benefit in one100 patients to see benefit in one100 patients to see benefit in one

ResultsResults

How precise was the estimate of treatment How precise was the estimate of treatment effect?effect?

Calculated results are point estimatesCalculated results are point estimates

True result is somewhere within the 95% True result is somewhere within the 95% CONFIDENCE INTERVAL (CI)CONFIDENCE INTERVAL (CI)

Tighter the CI, the more likely the calculated Tighter the CI, the more likely the calculated numbers are near the truthnumbers are near the truth

Larger sample sizeLarger sample sizelarger the number of larger the number of outcome eventsoutcome eventsgreater confidence that the greater confidence that the true risk is close to what we have observedtrue risk is close to what we have observed

95% Confidence Intervals 95% Confidence Intervals for the for the ARRARR

If the value “O” lies in your 95% CI for your If the value “O” lies in your 95% CI for your ARR, your result is NOT significant…there ARR, your result is NOT significant…there may be no difference between treatment may be no difference between treatment and placeboand placebo

If your 95% CI includes a negative number, If your 95% CI includes a negative number, your patient may get worse with your patient may get worse with treatment…STOPtreatment…STOP

ApplicabilityApplicability

Are the results applicable to my patientAre the results applicable to my patient?? Is our patient so different from those in the Is our patient so different from those in the

study that its results cannot apply?study that its results cannot apply? Is the treatment feasible in our setting?Is the treatment feasible in our setting? What are our patientWhat are our patient’’s potential benefits and s potential benefits and

harms from the therapy?harms from the therapy? What are our patientWhat are our patient’’s values and s values and

expectations for both the outcome we are expectations for both the outcome we are trying to prevent, and the treatment we are trying to prevent, and the treatment we are offering?offering?

Applicability-ScenarioApplicability-Scenario

Do the parents in the scenario have a Do the parents in the scenario have a personal experience with terrible asthma personal experience with terrible asthma and would be willing to try anything?and would be willing to try anything?

Is this therapy too expensive for them to Is this therapy too expensive for them to consider?consider?

Does the NNT of 9 influence your or the Does the NNT of 9 influence your or the parents’ decision about this medication?parents’ decision about this medication?

What if the NNT were 2?What if the NNT were 2?

Applicability- Bottom Applicability- Bottom LineLine

Is this research highly valid, the best there Is this research highly valid, the best there is, and should influence practice?is, and should influence practice?

Are there some issues with validity, but the Are there some issues with validity, but the best we have for now and thus should not best we have for now and thus should not dismiss altogether?dismiss altogether?

Is it so flawed that we should not consider Is it so flawed that we should not consider this new treatment , or practice without this new treatment , or practice without evidence beyond experience?evidence beyond experience?

Practice Case Practice Case ReferencesReferences

Case 1: Case 1: Cohen HA, Varsano I, Kahan E, Sarrell EM, Uziel Y. Effectiveness of an Herbal Preparation Containing Echinacea, Propolis, and Vitamin C in Preventing Respiratory Tract Infections in Children. Arch Pediatr Adolesc Med, 2004; 158:217-221.

Case 2: Freedman SB, Adler M, Seshadri R, Powell EC. Oral Ondansetram for Gastroenteritis in a Pediatric Emergency Department. N Engl J Med, 2006; 354:1698-1705.

Case 3: Bauchner H, Vinci R, Bak S, Pearson C, Corwin M. Parents and Procedures: A Randomized Controlled Trial. Pediatrics, 1996; 98:861-867.

EBM EBM References/ResourcesReferences/Resources

Guyatt, G., Rennie, D., Meade, M.O., & Cook, D.J. (2008) Users' guides to the medical literature: A manual for evidence-based clinical practice (2nd ed.). New York: McGraw-Hill Medical.

Online EBM Calculator: Online EBM Calculator: http://araw.mede.uic.edu/~alansz/tools.htmlhttp://araw.mede.uic.edu/~alansz/tools.html