ahrq quality indicators 101: background and introduction to the ahrq qis john bott, contractor,...

AHRQ Quality Indicators 101: AHRQ Quality Indicators 101: Background and Introduction to the Background and Introduction to the

AHRQ QIsAHRQ QIs

John Bott, Contractor, Agency for Healthcare Research and QualityJohn Bott, Contractor, Agency for Healthcare Research and Quality

April 28th,April 28th, 20112011

AHRQ – Agency within DHHSAHRQ – Agency within DHHS

2

OverviewOverview

1.1. OriginsOrigins

2.2. Current modulesCurrent modules

3.3. Advantages & challengesAdvantages & challenges

4.4. Recent improvementsRecent improvements

5.5. Uses of the AHRQ QIsUses of the AHRQ QIs

6.6. Tools: public reporting & quality improvementTools: public reporting & quality improvement

7.7. National Quality Forum EndorsementNational Quality Forum Endorsement

8.8. Additional activitiesAdditional activities

9.9. QuestionsQuestions

3

AHRQ QIs & the Healthcare Cost AHRQ QIs & the Healthcare Cost & Utilization Project (HCUP)& Utilization Project (HCUP)

HCUP: Partnership among States, industry & HCUP: Partnership among States, industry & AHRQAHRQ

Uniform database for cross-State studies; Uniform database for cross-State studies; includes clinical, demographic, and resource includes clinical, demographic, and resource use informationuse information

Represents all inpatient discharge data from Represents all inpatient discharge data from 44 participating States — represents 44 participating States — represents approximately 95% of all dischargesapproximately 95% of all discharges

4

The AHRQ QIs utilize HCUP in measure The AHRQ QIs utilize HCUP in measure development & maintenancedevelopment & maintenance

So what is HCUP?So what is HCUP?

AHRQ Quality Indicators (QIs)AHRQ Quality Indicators (QIs)

Developed through contract with UCSF-Stanford Developed through contract with UCSF-Stanford Evidence-based Practice Center & UC DavisEvidence-based Practice Center & UC Davis

Use existing hospital discharge data, based on Use existing hospital discharge data, based on readily available data elementsreadily available data elements

Incorporate a range of severity adjustment Incorporate a range of severity adjustment methods, including APR-DRGs* & co-morbidity methods, including APR-DRGs* & co-morbidity

groupingsgroupings Current modules: Prevention, Inpatient, Patient Current modules: Prevention, Inpatient, Patient

Safety, Pediatric & NeonatalSafety, Pediatric & Neonatal * 3M All Patient Refined - Diagnosis Related Groups * 3M All Patient Refined - Diagnosis Related Groups 5

Example of indicator evaluationExample of indicator evaluation

PANEL EVALUATION

FURTHEREMPIRICAL ANALYSES

REFINED DEF.

FURTHER REVIEW?

FINAL DEFINITION

INITIAL EMPRICAL ANALYSES

AND DEFINITION

LITERATURE REVIEW

USER DATA

6

Structure of AHRQ QIsStructure of AHRQ QIs

Measure definitions based on a number of data elements, e.g.:Measure definitions based on a number of data elements, e.g.:– ICD-9-CM* diagnosis & procedure codes ICD-9-CM* diagnosis & procedure codes – Medicare Diagnostic Related Groups (DRGs), Major Diagnostic Medicare Diagnostic Related Groups (DRGs), Major Diagnostic

Categories (MDC), sex, age, procedure dates, admission type, Categories (MDC), sex, age, procedure dates, admission type, admission source, discharge disposition, discharge quarter, point admission source, discharge disposition, discharge quarter, point of origin, present on admissionof origin, present on admission

Numerator: Number of cases with the outcome of interest (e.g., Numerator: Number of cases with the outcome of interest (e.g., postoperative sepsis, avoidable asthma hospitalization asthma, death)postoperative sepsis, avoidable asthma hospitalization asthma, death)

Denominator: Population at risk (e.g., pneumonia patients, elective Denominator: Population at risk (e.g., pneumonia patients, elective surgical patients, county population from census data)surgical patients, county population from census data)

Observed rate: The numerator / denominatorObserved rate: The numerator / denominator

Volume counts for selected proceduresVolume counts for selected procedures

Counts of admissions at an area level for certain types of admissionsCounts of admissions at an area level for certain types of admissions

** International Classification of Diseases, Ninth Revision, Clinical Modification International Classification of Diseases, Ninth Revision, Clinical Modification 7

OverviewOverview

1.1. OriginsOrigins

2.2. Current ModulesCurrent Modules

3.3. Advantages & ChallengesAdvantages & Challenges

4.4. Recent ImprovementsRecent Improvements




8.8. Additional ActivitiesAdditional Activities


8

Inpatient QIsInpatient QIs

Mortality,Mortality,Utilization,Utilization,

VolumeVolume

AHRQ Quality IndicatorsAHRQ Quality Indicators

Prevention QIsPrevention QIs(Area Level)(Area Level)Avoidable Avoidable

Hospitalizations /Hospitalizations /Other Avoidable Other Avoidable

ConditionsConditions

Patient Safety Patient Safety QIsQIs

Complications,Complications,Unexpected DeathUnexpected Death

Pediatric QIsPediatric QIs

Neonatal Neonatal QIsQIs

9

OverviewOverview

1.1. OriginsOrigins









10

AdvantagesAdvantages

Public AccessPublic Access– All development documentation and details All development documentation and details

on each indicator available on Web site on each indicator available on Web site – Software available to download at no costSoftware available to download at no cost

Documentation & software at: Documentation & software at: www.qualityindicators.ahrq.gov

– Standardized indicator definitionsStandardized indicator definitions– Can be used with any administrative data, Can be used with any administrative data,

e.g. HCUP, MEDPAR*, State data sets, e.g. HCUP, MEDPAR*, State data sets, payer data, hospital internal datapayer data, hospital internal data

– Hospitals can replicate resultsHospitals can replicate results

* * Medicare Provider Analysis & Review (Medicare administrative inpatient data)11

http://www.qualityindicators.ahrq.gov/

Advantages Advantages (cont.)(cont.)

ScopeScope– Over 90 individual measuresOver 90 individual measures

– Each measure can be stratified by other variables Each measure can be stratified by other variables including patient race, age, sex, provider, including patient race, age, sex, provider, geographic regiongeographic region

– Include priority populations & areas, e.g.: Include priority populations & areas, e.g.: Child health, women’s health (pregnancy & child-birth), Child health, women’s health (pregnancy & child-birth),

diabetes, hypertension, ischemic heart disease, stroke, diabetes, hypertension, ischemic heart disease, stroke, asthma, patient safety, preventive careasthma, patient safety, preventive care

– Focus on acute care, but crosses over to Focus on acute care, but crosses over to community & outpatient care delivery settingscommunity & outpatient care delivery settings

12

AdvantagesAdvantages (cont.)(cont.)

Harmonization of measuresHarmonization of measures Indicator maintenance and updatesIndicator maintenance and updates Tools & technical assistanceTools & technical assistance National benchmarks:National benchmarks:

– National Healthcare Quality ReportNational Healthcare Quality Report

– National Healthcare Disparities ReportNational Healthcare Disparities Report

– HCUPnetHCUPnet

13

Current limitations & challengesCurrent limitations & challenges

Outcomes data less actionable than processesOutcomes data less actionable than processes

Limited clinical detailLimited clinical detail

Risk adjustment challenges Risk adjustment challenges

Accuracy hinges on accuracy of documentation Accuracy hinges on accuracy of documentation

& coding& coding

Data potentially subject to gamingData potentially subject to gaming

Time lag of the data Time lag of the data

14

OverviewOverview

1.1. OriginsOrigins









15

Recent improvementsRecent improvements

Composite measuresComposite measures– Developed composite measures for the IQIs, PSIs, PQIs & PDIsDeveloped composite measures for the IQIs, PSIs, PQIs & PDIs

Risk adjustment based on administrative dataRisk adjustment based on administrative data– Additional risk adjustment methods for AHRQ QIsAdditional risk adjustment methods for AHRQ QIs

Updated literature reviewsUpdated literature reviews– Completed IQIs, PDIs & PSIsCompleted IQIs, PDIs & PSIs

Reporting templateReporting template– Tested & refinedTested & refined

National Quality Forum review & endorsement of a National Quality Forum review & endorsement of a number of the QIsnumber of the QIs

Use of present on admission & point of origin dataUse of present on admission & point of origin data

16

OverviewOverview

1.1. OriginsOrigins









17

General uses of the AHRQ QIsGeneral uses of the AHRQ QIs

Hospital quality improvement effortsHospital quality improvement efforts

– Individual hospitals & health care systems, such as:Individual hospitals & health care systems, such as: Banner Health (a multi-hospital system in AZ)Banner Health (a multi-hospital system in AZ) Norton Healthcare (a multi-hospital system in KY)Norton Healthcare (a multi-hospital system in KY) Baycare Health System (a multi-hospital system in FL)Baycare Health System (a multi-hospital system in FL) Ministry Health Care (a multi-hospital system in WI)Ministry Health Care (a multi-hospital system in WI)

– Hospital association member based reports, such as:Hospital association member based reports, such as: University Healthsystem ConsortiumUniversity Healthsystem Consortium Dallas - Fort Worth Hospital CouncilDallas - Fort Worth Hospital Council Premier (note: Premier participating in CMS pay for Premier (note: Premier participating in CMS pay for

performance demonstration, which includes AHRQ QIs)performance demonstration, which includes AHRQ QIs)18


Aggregate reporting: National, state, regional Aggregate reporting: National, state, regional – National Healthcare Quality / Disparities ReportsNational Healthcare Quality / Disparities Reports– Commonwealth Fund’s Health Performance InitiativeCommonwealth Fund’s Health Performance Initiative

ResearchResearch– Tracking quality of care for populations over time & across areasTracking quality of care for populations over time & across areas– Tracking disparities in care over time and across areasTracking disparities in care over time and across areas– Comparing quality between different types of hospitals or Comparing quality between different types of hospitals or

hospital systems (e.g., size, volume, teaching status, ownership, hospital systems (e.g., size, volume, teaching status, ownership, accreditation, critical access status)accreditation, critical access status)

– Evaluating impact of interventions to reduce costs or improve Evaluating impact of interventions to reduce costs or improve quality (e.g., resident work hours reform, electronic health quality (e.g., resident work hours reform, electronic health information systems, hospital mergers & consolidations)information systems, hospital mergers & consolidations)

19


Value based purchasing / pay for performance (P4P)Value based purchasing / pay for performance (P4P)– CMS - Premier DemoCMS - Premier Demo– Anthem of VirginiaAnthem of Virginia– The Alliance (Wisconsin)The Alliance (Wisconsin)

Hospital level public reportingHospital level public reporting– Currently: Statewide public reporting Currently: Statewide public reporting (upcoming slide)(upcoming slide)– Upcoming: CMS Hospital Compare, including Veterans Affairs Upcoming: CMS Hospital Compare, including Veterans Affairs

medical centers medical centers (upcoming slide)(upcoming slide)

Hospital profiling: Public reporting & P4PHospital profiling: Public reporting & P4P– Blue Cross / Blue Shield of IllinoisBlue Cross / Blue Shield of Illinois

20

Current & future uses of AHRQ QIs in public reports

State level State level National levelNational levelAHRQ QIs used in public reports AHRQ QIs used in public reports FY11 IPPS rule *:FY11 IPPS rule *:reports in 25 states – two-thirds 6 individual AHRQ QIsreports in 25 states – two-thirds 6 individual AHRQ QIsof U.S. population of U.S. population & 2 AHRQ QI composites& 2 AHRQ QI composites slated for Hospital Compareslated for Hospital Compare

FY12 IPPS rule *: FY12 IPPS rule *: 2 additional individual2 additional individual AHRQ QIs to be added toAHRQ QIs to be added to Hospital CompareHospital Compare

* * Inpatient Prospective Payment System rule

OverviewOverview

1.1. OriginsOrigins2.2. Current ModulesCurrent Modules3.3. Advantages & ChallengesAdvantages & Challenges4.4. Recent ImprovementsRecent Improvements5.5. Uses of the AHRQ QIsUses of the AHRQ QIs6.6. Tools: public reporting & quality improvementTools: public reporting & quality improvement7.7. National Quality Forum EndorsementNational Quality Forum Endorsement8.8. Additional ActivitiesAdditional Activities9.9. Future VisionFuture Vision10.10. QuestionsQuestions

22

AHRQ tools for public reporting AHRQ tools for public reporting the AHRQ QIs: Recent & currentthe AHRQ QIs: Recent & current

Reporting templates (or model reports)Reporting templates (or model reports)

– Tool for reporting sponsors to use the best evidence on Tool for reporting sponsors to use the best evidence on

public reportspublic reports

Reporting guidanceReporting guidance

– Report on the appropriate uses of the AHRQ QIs based on Report on the appropriate uses of the AHRQ QIs based on

the evidence to datethe evidence to date

Quality Indicators Learning InstituteQuality Indicators Learning Institute

– Web conference series to disseminate technical Web conference series to disseminate technical

information of various aspects of public reportinginformation of various aspects of public reporting

23

AHRQ tools for public reporting AHRQ tools for public reporting the AHRQ QIs the AHRQ QIs (cont.)(cont.)

MONAHRQ softwareMONAHRQ software– Software to input hospital administrative claims data & Software to input hospital administrative claims data &

produce a website to publicly report performance in the produce a website to publicly report performance in the AHRQ QIs at the hospital & community levelAHRQ QIs at the hospital & community level

- - MONAHRQ MONAHRQ Web site at: Web site at: www.monahrq.ahrq.gov

MONAHRQ Learning NetworkMONAHRQ Learning Network– Provide information & technical assistance to users & Provide information & technical assistance to users &

potential users of MONAHRQpotential users of MONAHRQ

24

http://www.monahrq.ahrq.gov/

AHRQ tools for quality improvement AHRQ tools for quality improvement using the AHRQ QIs: Upcomingusing the AHRQ QIs: Upcoming

Quality improvement toolkit under development Quality improvement toolkit under development for hospital use to make improvement related to for hospital use to make improvement related to the AHRQ IQIs & PSIs the AHRQ IQIs & PSIs

Some specifics of the toolkit:Some specifics of the toolkit:– Methods to evaluate the data for identifying Methods to evaluate the data for identifying

opportunities for improvement opportunities for improvement

– Strategies for implementing interventions (or Strategies for implementing interventions (or evidence-based best practices)evidence-based best practices)

– Methods to measure progressMethods to measure progress

Available late - 2011Available late - 201125

OverviewOverview

1.1. OriginsOrigins









26

National Quality Forum National Quality Forum Endorsement: OverviewEndorsement: Overview

Currently, NQF endorsement in regard to:Currently, NQF endorsement in regard to: 48 of the AHRQ QIs48 of the AHRQ QIs

– Inpatient Quality Indicators (IQIs): 13Inpatient Quality Indicators (IQIs): 13

– Patient Safety Indicators (PSIs): 10Patient Safety Indicators (PSIs): 10

– Prevention Quality Indicators (PQIs): 14Prevention Quality Indicators (PQIs): 14

– Pediatric Quality Indicators (PDIs): 10Pediatric Quality Indicators (PDIs): 10

– Neonatal Quality Indicators (NQIs): 1Neonatal Quality Indicators (NQIs): 1

3 AHRQ QI composites3 AHRQ QI composites– Mortality for selected conditionsMortality for selected conditions

– Patient safety for selected indicatorsPatient safety for selected indicators

– Pediatric patient safety for selected indicatorsPediatric patient safety for selected indicators

27

OverviewOverview

1.1. OriginsOrigins









28

Highlights of additional activitiesHighlights of additional activities

AHRQ funded HCUP Partner projects to add more AHRQ funded HCUP Partner projects to add more clinical data to statewide administrative databases (e.g. clinical data to statewide administrative databases (e.g. present on admission, lab values)present on admission, lab values)

AHRQ proposed coding changes & clarifications to AHRQ proposed coding changes & clarifications to ICD-9 to enhance accuracy & use of some indicatorsICD-9 to enhance accuracy & use of some indicators – Retinopathy of prematurityRetinopathy of prematurity– Necrotizing enterocolitisNecrotizing enterocolitis– Disruption of postoperative woundDisruption of postoperative wound– Deep vein thrombosisDeep vein thrombosis– Transfusion reactionTransfusion reaction

Working with other measure developers to align Working with other measure developers to align specifications of similar measuresspecifications of similar measures– The Joint Commission The Joint Commission – Centers for Medicare & Medicaid ServicesCenters for Medicare & Medicaid Services– National Perinatal Information CenterNational Perinatal Information Center– ResearchersResearchers

29

Organisation for Economic Cooperation Organisation for Economic Cooperation and Development (OECD)and Development (OECD)

The OECD Health Care Quality Indicators Project The OECD Health Care Quality Indicators Project includes a patient safety componentincludes a patient safety component

The project conducted a pilot with seven countries to The project conducted a pilot with seven countries to translate a number of PSIs to the WHO’s version of translate a number of PSIs to the WHO’s version of ICD 10ICD 10

A paper on the pilot was recently published in the A paper on the pilot was recently published in the International Journal of Quality Health Care International Journal of Quality Health Care – volume 21, number 4, pages 272 – 278volume 21, number 4, pages 272 – 278

The pilot has recently expanded The pilot has recently expanded – Currently 18 countries are participatingCurrently 18 countries are participating

30

Validation pilotValidation pilot

Pilot Objectives:Pilot Objectives:– Gather evidence on the scientific Gather evidence on the scientific

acceptability of the PSIsacceptability of the PSIs Medical record reviews, data analysis, clinical Medical record reviews, data analysis, clinical

panels, evidence reviews panels, evidence reviews

– Consolidate the evidence base Consolidate the evidence base

– Improve guidance on the interpretation and Improve guidance on the interpretation and use of the datause of the data

– Evaluate potential refinements to the Evaluate potential refinements to the specificationsspecifications

31

Validation pilot, phases 1 to 4Validation pilot, phases 1 to 4

Phase 1 (Concluded)Phase 1 (Concluded)– Focus on estimating false positive rate for 5 PSIs (# 6, 7, 12, 13, Focus on estimating false positive rate for 5 PSIs (# 6, 7, 12, 13,

15)15)– 6 articles published or soon to be published6 articles published or soon to be published

Phase 2 & 3 (Currently in data analysis)Phase 2 & 3 (Currently in data analysis)– Focus on estimating false positive rate for 2 other PSIs (# 9, 10)Focus on estimating false positive rate for 2 other PSIs (# 9, 10)– Will also estimate false negative rate (sensitivity) for these 2 PSIs Will also estimate false negative rate (sensitivity) for these 2 PSIs

and up to 6 more PSIs (# 5-7, 11, 14, 15)and up to 6 more PSIs (# 5-7, 11, 14, 15) Phase 4 (Ongoing)Phase 4 (Ongoing)

– Collaboration with University HealthSystem Consortium on 3 PSIs Collaboration with University HealthSystem Consortium on 3 PSIs (# 3, 11, 12)(# 3, 11, 12) 2 articles published (PSI 11, 12) and 1 in preparation (PSI 3)2 articles published (PSI 11, 12) and 1 in preparation (PSI 3)

– Collaboration with Veterans Health Administration on 10 PSIs (# 3, Collaboration with Veterans Health Administration on 10 PSIs (# 3, 5, 6, 8, 9, 11-15)5, 6, 8, 9, 11-15) 3 articles published or soon to be published (PSI 6, 11-13, 15) 3 articles published or soon to be published (PSI 6, 11-13, 15)

– Collaboration with National Perinatal Information Center on PSI 17Collaboration with National Perinatal Information Center on PSI 1732

Current Measurement WorkCurrent Measurement Work

Measure & Beta Measurement pipelineMeasure & Beta Measurement pipeline

- Readmission measures- Readmission measures

- Health care associated infections- Health care associated infections

- Emergency Preparedness measures- Emergency Preparedness measures

- Care coordination measure development- Care coordination measure development

- Emergency dept. PSI measure development- Emergency dept. PSI measure development

- Emergency dept. PQI measure development- Emergency dept. PQI measure development

- Medicaid Home & Community Based Services population measures- Medicaid Home & Community Based Services population measures

Exploration of Measure Enhancement pipelineExploration of Measure Enhancement pipeline

- Integration of lab values- Integration of lab values

- Further integration of present on admission- Further integration of present on admission

- Assessment of PQIs for pay for performance- Assessment of PQIs for pay for performance

33

For more information…For more information…

Web site: Web site: http://qualityindicators.ahrq.gov

– QI documentation and software are availableQI documentation and software are available

– Sign up for AHRQ QI listservSign up for AHRQ QI listserv

Support E-mail: Support E-mail: [email protected]

Support Phone: (888) 512-6090 (voicemail) Support Phone: (888) 512-6090 (voicemail)

Staff: Staff: [email protected]

[email protected]

34

http://qualityindicators.ahrq.gov/

mailto:[email protected]



ahrq quality indicators 101: background and introduction to the ahrq qis john bott, contractor,...

Documents