Integrating Health Outcomes Integrating Health Outcomes into the Publication Planinto the Publication Plan
Frank J. Rodino, PA, MHSFrank J. Rodino, PA, MHSChurchill CommunicationsChurchill CommunicationsCh hill Cli i l R hCh hill Cli i l R hChurchill Clinical ResearchChurchill Clinical Research
December 9, 2009December 9, 2009
Lecture ObjectivesLecture ObjectivesLecture ObjectivesLecture Objectives
P id b i i f h l h i h P id b i i f h l h i h •• Provide basic overview of health outcomes, with an Provide basic overview of health outcomes, with an emphasis on “Real World” research emphasis on “Real World” research
•• Explore key differences between RCT and health Explore key differences between RCT and health Explore key differences between RCT and health Explore key differences between RCT and health outcomes reportingoutcomes reporting
•• Highlight some helpful resources for health outcomes Highlight some helpful resources for health outcomes bli hibli hipublishingpublishing
Glossary of AbbreviationsGlossary of AbbreviationsGlossary of AbbreviationsGlossary of Abbreviations
ARRA A i R d R i t t A tARRA A i R d R i t t A t•• ARRA: American Recovery and Reinvestment ActARRA: American Recovery and Reinvestment Act•• CER: Comparative Effectiveness ResearchCER: Comparative Effectiveness Research•• HIPAA: Health Information Portability and Accountability ActHIPAA: Health Information Portability and Accountability ActHIPAA: Health Information Portability and Accountability ActHIPAA: Health Information Portability and Accountability Act•• IOM: Institutes of MedicineIOM: Institutes of Medicine•• OR: Outcomes ResearchOR: Outcomes Research•• PE: PharmacoeconomicsPE: Pharmacoeconomics•• QOL: Quality of LifeQOL: Quality of Life•• QUOROM: Quality of Reporting of MetaQUOROM: Quality of Reporting of Meta--analysisanalysis•• QUOROM: Quality of Reporting of MetaQUOROM: Quality of Reporting of Meta--analysisanalysis•• RCT: Randomized Clinical TrialRCT: Randomized Clinical Trial•• RW: Real WorldRW: Real World•• STROBE: Strengthening the Reporting of Observational Studies STROBE: Strengthening the Reporting of Observational Studies
in Epidemiologyin Epidemiology
Health Outcomes ResearchHealth Outcomes ResearchOverview Overview
4
ISPORISPORISPORISPOR
I i l S i f Ph i d I i l S i f Ph i d •• International Society of Pharmacoeconomics and International Society of Pharmacoeconomics and Outcomes ResearchOutcomes Research
•• Global organization focusing on PE and ORGlobal organization focusing on PE and ORGlobal organization focusing on PE and ORGlobal organization focusing on PE and OR•• Excellent resources on all aspects of PE and ORExcellent resources on all aspects of PE and OR•• Numerous policy statements on designing, interpreting, Numerous policy statements on designing, interpreting,
and publishing outcomes researchand publishing outcomes research•• Value in HealthValue in Health, official, indexed, peer, official, indexed, peer--reviewed journal reviewed journal
of ISPORof ISPORof ISPORof ISPOR•• www.ispor.orgwww.ispor.org
5
ISPOR Definition of h
ISPOR Definition of hOutcomes ResearchOutcomes Research
Th i f h i (h l h i ) d The science of pharmacoeconomics (health economics) and outcomes research (the scientific discipline that evaluates the effect of healthcare interventions on patient well-being, p g,including clinical, economic, and patient-reported outcomes) that facilitates the translation of this research into useful information for healthcare decision-makers to into useful information for healthcare decision-makers to ensure that society allocates scarce healthcare resources wisely, fairly, and efficiently.
6
From the ISPOR Mission Statement; www.ispor.org
Growing Importance of l h h
Growing Importance of l h hHealth Outcomes Research Health Outcomes Research
P i li i l i h h i illP i li i l i h h i ill•• Patients are living longer with chronic illnessesPatients are living longer with chronic illnesses•• Few largeFew large--scale, headscale, head--toto--head studies of therapeutic head studies of therapeutic
interventionsinterventionsinterventionsinterventions•• Scant evidence for offScant evidence for off--label treatmentslabel treatments•• Wide variability in treatment patternsWide variability in treatment patterns•• We spend the most (15% of GDP) but don’t live the We spend the most (15% of GDP) but don’t live the
longest (78 yrs)longest (78 yrs)
7
Randomized Clinical TrialsRandomized Clinical TrialsRandomized Clinical TrialsRandomized Clinical Trials
R d i d li i l i l i f d i i R d i d li i l i l i f d i i •• Randomized clinical trials are important for determining Randomized clinical trials are important for determining efficacy and safety, and registrationefficacy and safety, and registration
•• But they areBut they areBut they areBut they are–– ExpensiveExpensive–– Limited, controlled experimentsLimited, controlled experiments
Oft l b i f l ti f th l lOft l b i f l ti f th l l–– Often vs. placebo or a previous formulation of the same moleculeOften vs. placebo or a previous formulation of the same molecule–– Not always generalizable to larger populationsNot always generalizable to larger populations
•• There is a need to follow RCTs with studies of how the There is a need to follow RCTs with studies of how the product is used in the real worldproduct is used in the real world
8
Who Benefits from h
Who Benefits from hOutcomes ResearchOutcomes Research
H l h hH l h h•• Health care researchersHealth care researchers•• Health technology assessorsHealth technology assessors•• Government health regulatorsGovernment health regulators•• Government health regulatorsGovernment health regulators•• Payers of health carePayers of health care•• Health care providersHealth care providersHealth care providersHealth care providers•• PatientsPatients•• Producers of therapeutic interventionsProducers of therapeutic interventions
Efficacy vs. EffectivenessEfficacy vs. EffectivenessEfficacy vs. EffectivenessEfficacy vs. Effectiveness
EffiEffi•• EfficacyEfficacy–– What an intervention can or can’t doWhat an intervention can or can’t do
Good for patientGood for patient–– Good for patientGood for patient•• EffectivenessEffectiveness
–– Evaluates interventions in real worldEvaluates interventions in real worldEvaluates interventions in real worldEvaluates interventions in real world–– Good for populationGood for population
10
Comparative Effectiveness ResearchComparative Effectiveness ResearchComparative Effectiveness ResearchComparative Effectiveness Research
C i f h id d C i f h id d •• Comparison of 2 or more agents that are considered true Comparison of 2 or more agents that are considered true alternativesalternatives
•• Evaluates outcomes of interventions derived from actual Evaluates outcomes of interventions derived from actual Evaluates outcomes of interventions derived from actual Evaluates outcomes of interventions derived from actual practicepractice
•• American Recovery and Reinvestment Act (ARRA) is American Recovery and Reinvestment Act (ARRA) is i i $i i $investing $1.1B to support CERinvesting $1.1B to support CER
•• Approximately 100 highApproximately 100 high--priority research projectspriority research projects•• Goal is to spark innovation and provide information to Goal is to spark innovation and provide information to •• Goal is to spark innovation and provide information to Goal is to spark innovation and provide information to
assist clinicians in making sound evidenceassist clinicians in making sound evidence--based decisionsbased decisions
11
Lee EH, Nash DB, ISMPP 2009Lee EH, Nash DB, ISMPP 2009
Pharmacoeconomic ModelingPharmacoeconomic ModelingPharmacoeconomic ModelingPharmacoeconomic Modeling
S h i id h l h d S h i id h l h d •• Synthesizes evidence on health consequences and costs Synthesizes evidence on health consequences and costs from many different sources includingfrom many different sources including–– Clinical trialsClinical trialsClinical trialsClinical trials–– Observational studiesObservational studies–– Insurance claims databasesInsurance claims databases–– Case registriesCase registries–– Public health statisticsPublic health statistics–– Preference surveysPreference surveys
12
Weinstein MC, Value Health 2003Weinstein MC, Value Health 2003
Pharmacoeconomic EvaluationsPharmacoeconomic EvaluationsPharmacoeconomic EvaluationsPharmacoeconomic Evaluations
T f l iT f l i•• Types of evaluationsTypes of evaluations–– Cost consequenceCost consequence
Cost utilityCost utility–– Cost utilityCost utility–– Cost benefitCost benefit
13
Weinstein MC, Value Health 2003Weinstein MC, Value Health 2003
Real World OutcomesReal World OutcomesReal World OutcomesReal World Outcomes
ClinicalOutcomes
Economic Patient Reported EconomicOutcomes Outcomes
Quality of Life
Real World Data BenefitsReal World Data BenefitsReal World Data BenefitsReal World Data Benefits
A diff i i i A diff i i i A different perspective on interventions, eg, A different perspective on interventions, eg, •• Estimates of effectiveness vs. efficacy in a variety of Estimates of effectiveness vs. efficacy in a variety of
settingssettingssettingssettings•• Comparison of multiple interventions for optimal choicesComparison of multiple interventions for optimal choices•• Evolving riskEvolving risk--benefit profiles of new interventionsbenefit profiles of new interventions•• Clinical outcomes in a diversity of study populationsClinical outcomes in a diversity of study populations•• Results on a broader range of outcomes than those for Results on a broader range of outcomes than those for
RCTRCTRCTRCT•• Data on resource use for cost evaluationsData on resource use for cost evaluations
15Garrison Jr. LP, Value Health 2007Garrison Jr. LP, Value Health 2007
Real World Data Benefits, contd.Real World Data Benefits, contd.Real World Data Benefits, contd.Real World Data Benefits, contd.
D d i d li i li i l iD d i d li i li i l i•• Data on dosing and compliance in clinical practiceData on dosing and compliance in clinical practice•• Useful in situations where RCT is impossibleUseful in situations where RCT is impossible•• Substantiation of RCT dataSubstantiation of RCT data•• Substantiation of RCT dataSubstantiation of RCT data•• Provide data to support reimbursement in urgent Provide data to support reimbursement in urgent
situationssituations•• Source of interim evidence for preliminary decisionsSource of interim evidence for preliminary decisions•• Net clinical and economic impact of payment policiesNet clinical and economic impact of payment policies
Garrison Jr LP Value Health 2007Garrison Jr LP Value Health 2007
16
Garrison Jr. LP, Value Health 2007Garrison Jr. LP, Value Health 2007
Real World Data CategoriesReal World Data CategoriesReal World Data CategoriesReal World Data Categories
S l di i l i i RCTS l di i l i i RCT•• Supplements to traditional registration RCTsSupplements to traditional registration RCTs•• Large simple trials (practical clinical trials)Large simple trials (practical clinical trials)•• RegistriesRegistries•• RegistriesRegistries•• Administrative dataAdministrative data•• Health surveysHealth surveysHealth surveysHealth surveys•• Electronic medical records and chart reviewsElectronic medical records and chart reviews
17
Garrison Jr. LP, Value Health 2007Garrison Jr. LP, Value Health 2007
Evidence HierarchyEvidence HierarchyEvidence HierarchyEvidence Hierarchy
Magnitude of Net BenefitMagnitude of Net BenefitMagnitude of Net BenefitMagnitude of Net Benefit
Wh i h b fi i f i l Wh i h b fi i f i l •• What is the net benefit in support of a particular What is the net benefit in support of a particular technology?technology?
•• Magnitude of net benefit must be taken into account Magnitude of net benefit must be taken into account Magnitude of net benefit must be taken into account Magnitude of net benefit must be taken into account when assessing real world evidencewhen assessing real world evidence
19
Limits of Real World DataLimits of Real World DataLimits of Real World DataLimits of Real World Data
NN d i dd i d•• NonNon--randomizedrandomized•• Potential for bias and confoundingPotential for bias and confounding•• Not likely to meet methodological rigor of an RCTNot likely to meet methodological rigor of an RCT•• Not likely to meet methodological rigor of an RCTNot likely to meet methodological rigor of an RCT•• Good research practices are needed and should be Good research practices are needed and should be
followed from study design through publicationfollowed from study design through publication
Garrison Jr LP Value Health 2007Garrison Jr LP Value Health 2007
20
Garrison Jr. LP, Value Health 2007Garrison Jr. LP, Value Health 2007
Health Outcomes and Health Outcomes and Publication PlanningPublication Planning
21
Growth of Outcomes PublishingGrowth of Outcomes PublishingGrowth of Outcomes PublishingGrowth of Outcomes Publishing
dli i i
0006000
Medline Citations (Human/Clinical/Meta‐analysis/Guidelines)
2000300040005000
010002000
2000 2001 2002 2003 2004 2005 2006 2007 20082000 2001 2002 2003 2004 2005 2006 2007 2008
Indexed Publications Focusing l h
Indexed Publications Focusing l hon Health Outcomeson Health Outcomes
Journal Circulation ImpactHealth and Quality of Life Outcomes 37,041 3.2Q y ,Pharmacoeconomics 9,980 2.808Value in Health 4,000 3.009
• Other journals have OR sections
23
CONSORT, QUOROM, d
CONSORT, QUOROM, dSTROBE, and STARDSTROBE, and STARD
CONSORT (1996 2001)CONSORT (1996 2001)•• CONSORT (1996, 2001)CONSORT (1996, 2001)–– CONCONsolidated solidated SStandards tandards OOf f RReporting eporting TTrialsrials–– Improve the reporting of a randomized clinical trialImprove the reporting of a randomized clinical trial
•• QUOROM (1999)QUOROM (1999)–– QUQUality ality OOf f RReporting eporting OOf f MMetaeta--analysesanalyses–– Improve the quality of reporting metaImprove the quality of reporting meta--analyses of randomized, analyses of randomized,
ll d li i l i lll d li i l i lcontrolled clinical trialscontrolled clinical trials•• STROBE (2007)STROBE (2007)
–– SStrengthening trengthening TThe he RReporting of eporting of OBOBservational Studies in servational Studies in EEpidemiologypidemiology–– Improve the quality of reporting of observational studiesImprove the quality of reporting of observational studies
•• STARD (2003)STARD (2003)–– STASTAndards for the ndards for the RReporting of eporting of DDiagnostic accuracy studiesiagnostic accuracy studies–– Improve reporting of studies of diagnostic accuracyImprove reporting of studies of diagnostic accuracy
24
CONSORT, QUOROM, d
CONSORT, QUOROM, dand STROBEand STROBE
Ch kli t CONSORT QUOROM STROBEChecklist CONSORT QUOROM STROBE
# of Items 22 21 22
Title, Abstract, I t d ti
Similar Similar SimilarIntroduction
Methods RandomizationBlinding
SearchingSelectionValidity assessment
Study type•Cohort•Case-controly
Data abstractionQuantitative data synthesis
•Cross-sectionalPotential for bias
Results Recruitment Quantitative data DescriptiveResults RecruitmentAdverse events
Quantitative data synthesis
Descriptiveanalysis
Discussion Interpretation Describe biases in review process
“Cautious” interpretationLi i iLimitations
25
Structural ConsiderationsStructural ConsiderationsStructural ConsiderationsStructural Considerations
S f ffi AND ff iS f ffi AND ff i•• Safety, efficacy AND effectivenessSafety, efficacy AND effectiveness•• Incorporate health outcomes research earlier in the drug Incorporate health outcomes research earlier in the drug
development processdevelopment processdevelopment processdevelopment process–– Protocol designProtocol design–– AnalysisAnalysis
E t i t diE t i t di–– Extension studiesExtension studies
•• Add a health outcomes expert(s) to your publication Add a health outcomes expert(s) to your publication planning teamplanning teamp gp g
26
Planning ConsiderationsPlanning ConsiderationsPlanning ConsiderationsPlanning Considerations
H l h h ld b l f H l h h ld b l f •• Health outcomes should be a core element of a Health outcomes should be a core element of a publication planpublication plan–– Most congresses/journals now have sections for outcomes Most congresses/journals now have sections for outcomes g jg j
researchresearch
•• A discussion of OR should be on the agenda of every pub A discussion of OR should be on the agenda of every pub planning meetingplanning meetingplanning meetingplanning meeting
•• OR should be part of every gap analysisOR should be part of every gap analysis•• Timing is keyTiming is keyg yg y
27
Reporting ConsiderationsReporting ConsiderationsReporting ConsiderationsReporting Considerations
OR f li OR f li d i d d d i d d •• OR often relies on nonOR often relies on non--randomized data randomized data –– Not likely to meet methodological rigor of an RCTNot likely to meet methodological rigor of an RCT
•• Detailed reporting is required forDetailed reporting is required forDetailed reporting is required forDetailed reporting is required for–– Study designStudy design–– MethodsMethods–– Treatment effectsTreatment effects–– Bias and confoundingBias and confounding
GeneralizabilityGeneralizability–– GeneralizabilityGeneralizability•• Additional details may be included in appendixAdditional details may be included in appendix
28
HIPAA, IRBs, and h h
HIPAA, IRBs, and h hOther Thorny IssuesOther Thorny Issues
NN i i l b i l i i i l b i l i •• NonNon--interventional, observational, or retrospective interventional, observational, or retrospective studies are not exempt from HIPAA rulesstudies are not exempt from HIPAA rules
•• Was the study reviewed, or exempted from review, by an Was the study reviewed, or exempted from review, by an Was the study reviewed, or exempted from review, by an Was the study reviewed, or exempted from review, by an IRB?IRB?
•• FDAAA currently applies to RCT. Will it apply to OR in FDAAA currently applies to RCT. Will it apply to OR in h fh fthe future?the future?
29
SummarySummarySummarySummary
OR i hOR i h•• OR is researchOR is research–– Requires sound methodologies, privacy protection, unbiased data Requires sound methodologies, privacy protection, unbiased data
sources, and thorough analysissources, and thorough analysisOR fill OR fill •• OR fills gapsOR fills gaps–– Helps to address questions and concerns that go beyond the scope of Helps to address questions and concerns that go beyond the scope of
registration studiesregistration studiesOR id tifi OR id tifi •• OR identifies gapsOR identifies gaps–– Helps to identify newer, more effective approaches and interventionsHelps to identify newer, more effective approaches and interventions
•• OR is publishableOR is publishable–– Growing number of guidelines and journals/venues dedicated to ORGrowing number of guidelines and journals/venues dedicated to OR
•• OR is the futureOR is the future–– CER will require continuing followCER will require continuing follow--up to remain relevantup to remain relevant
30
BibliographyBibliographyBibliographyBibliography
•• Begg C Cho M Eastwood S Horton R Moher D Olkin I et al Improving the quality of reporting of randomized Begg C Cho M Eastwood S Horton R Moher D Olkin I et al Improving the quality of reporting of randomized •• Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. controlled trials. The CONSORT statement. JAMA.JAMA. 1996;276:6371996;276:637--99
•• Berger ML, Berger ML, MamdaniMamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: The defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task force reportISPOR good research practices for retrospective database analysis task force report——Part I. Part I. Value HealthValue Health 2009;122009;12
•• Cox E Martin BC Van Cox E Martin BC Van StaaStaa T Garbe E T Garbe E SiebertSiebert U Johnson ML Good research practices for comparative U Johnson ML Good research practices for comparative •• Cox E, Martin BC, Van Cox E, Martin BC, Van StaaStaa T, Garbe E, T, Garbe E, SiebertSiebert U, Johnson ML, Good research practices for comparative U, Johnson ML, Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of noneffectiveness research: approaches to mitigate bias and confounding in the design of non--randomized studies of randomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task forceanalysis task force––Part II. Part II. Value HealthValue Health 2009;122009;12
•• Evidence Report/Technology Assessment No. 47. Systems to rate the strength of scientific evidence. (AHRQ Evidence Report/Technology Assessment No. 47. Systems to rate the strength of scientific evidence. (AHRQ Publication No 02Publication No 02--E016)E016)Publication No. 02Publication No. 02--E016)E016)
•• Garrison Jr. LP, Neumann PJ, Erickson P, et al. Using realGarrison Jr. LP, Neumann PJ, Erickson P, et al. Using real--world data for coverage and payment decisions: The world data for coverage and payment decisions: The ISPOR realISPOR real--world data task force report. world data task force report. Value Health Value Health 2007;10:3262007;10:326--3535
•• McGhan WF, Al M, McGhan WF, Al M, DoshiDoshi JA, et al. The ISPOR good practices for quality improvement of costJA, et al. The ISPOR good practices for quality improvement of cost--effectiveness effectiveness research task force report. research task force report. Value Health.Value Health. 2009;122009;12M h D C k DJ E t d S Olki I M h D C k DJ E t d S Olki I R iR i D D StSt DF I i th lit f t f tDF I i th lit f t f t l f l f •• Moher D, Cook DJ, Eastwood S, Olkin I, Moher D, Cook DJ, Eastwood S, Olkin I, RennieRennie D, D, StroupStroup DF. Improving the quality of reports of metaDF. Improving the quality of reports of meta--analyses of analyses of randomisedrandomised controlled trials: the QUOROM statement. Quality of Reporting of Metacontrolled trials: the QUOROM statement. Quality of Reporting of Meta--analyses. analyses. Lancet.Lancet. 1999; 1999; 354:1896354:1896--900.900.
•• Moher D, Moher D, SchulzSchulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallelreports of parallel--group group randomisedrandomised trials. trials. Lancet.Lancet. 2001;357:11912001;357:1191--44
El E Alt DG E M t l Th St th i th R ti f Ob ti l St di i E id i l El E Alt DG E M t l Th St th i th R ti f Ob ti l St di i E id i l
31
•• von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med.Ann Intern Med. 2007;147:5732007;147:573––77
Thank YouThank YouThank YouThank You
32