![Page 1: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/1.jpg)
Applying PerformanceIndicators to Health Systems
Improvement
Sheila LeathermanCenter for Health Care Policy and Evaluation, Minneapolis, MNNuffield Trust, London, EnglandSchool of Public Health, University of North Carolina
![Page 2: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/2.jpg)
Outline of Remarks
• A conceptual framework• The role of public reporting• Use of performance indicators for change• The way forward: “what to do” considerations
![Page 3: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/3.jpg)
Performance Measurement andReporting Cycle
• Establish goals• Adopt specific measures/indicators• Performance analysis
• Reporting of data (publicly and confidentially)
• Systematic implementation/improvement interventions
• Monitoring and feedback
![Page 4: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/4.jpg)
Organizing and Integrating Performance
PolicyFormulation &Infrastructure
Performance Monitoring Macromanagement
Operations ManagementGovernance
Clinical Service ProvisionIndividual Accountability
Regional
National
Institutional
Individual
![Page 5: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/5.jpg)
Organizing and Integrating Performance
PolicyFormulation &Infrastructure
Performance Monitoring Macromanagement
Operations ManagementGovernance
Clinical Service ProvisionIndividual Accountability
Unite
d Ki
ngdo
m United States
![Page 6: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/6.jpg)
Performance DomainsIndividual and Population Level
– Effectiveness– Efficiency– Equity– Responsiveness– Appropriateness– Safety
Quality
![Page 7: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/7.jpg)
Quality
“the degree to which health services for individuals and populations increase the likelihoodof desired health outcomes and are consistent with
current professional knowledge”
IOM Definition
![Page 8: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/8.jpg)
Applying Performance Indicators toImprove Health Systems
• Systemic standardized measurementmeasurement ofperformance
• Public reportingreporting of data• Use of performance data to catalyze positive
changechange
![Page 9: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/9.jpg)
Performance Reporting: Why?
• Performance monitoring for regulation• “The Information Age”• Media coverage• Public confidence eroding• Accountability a growing movement
![Page 10: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/10.jpg)
Concerns Regarding Quality
Physician Perceptions (1999-2000)• 5 country survey (Australia, NZ, UK, Canada, and
USA)• % saying ability to provide quality care worsened
over 5 years– Australia 38%– Canada 50%– New Zealand 53%– United Kingdom 46%– United States 57%
![Page 11: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/11.jpg)
Concerns Regarding Quality
Nurses Perceptions (1998-1999)• Five country survey (Canada, Germany,
Scotland, England, and USA)• 17 - 44% reported quality had deteriorated in
last year
![Page 12: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/12.jpg)
Concerns Regarding Quality
Public Perception (1998)• Five country survey (Australia, Canada, New
Zealand, United Kingdom and USA)• Overwhelmingly stated that health care
systems needed “fundamental change orcomplete overhaul”
![Page 13: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/13.jpg)
Theoretical Purposes: Public Reporting
• Regulation• Purchasing or commissioning decisions• Facilitation of consumer selection/choice• Provider/systems behavior change• Accountability
![Page 14: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/14.jpg)
Adapted from Emanuel and EmanuelAnnals of Internal Medicine, Jan 15, 1996
Concept Methodsof accountability
Professional Patient receives Licensure, Certificationservices from professional Malpractice suit
Economic Consumer of health care Choice and “exit”commodity in regulated market
Political Citizen receiving “Voice” andpublic good provided by government pressuregovernment for reforms
Accountability: ModelsAccountability: Models
![Page 15: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/15.jpg)
Performance Reporting
• National Quality Reports• “Report Cards”• League Tables• Provider profiling• Popular press: mass media• Commercial initiatives
![Page 16: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/16.jpg)
Current Status
• Measurement and public reporting inevitable• Inadequate evaluation research - what works?• Challenge: How to move ahead responsibly?
![Page 17: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/17.jpg)
Purposes for Public Disclosure
Regulation Purchasing Facilitation ofconsumer choice
Provider/ Systemsbehavior change
Accountability
Public
Providers
Purchasers
Policymakers
![Page 18: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/18.jpg)
Evidence of Effectiveness ofPerformance Reporting: USA
• Public• Provider• Purchaser/payers• Policymakers
![Page 19: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/19.jpg)
The PublicEvidence from the USA
• Performance data used minimally• Most data designed for other purposes• Not easily comprehended or actionable• Not salient (ex: CABG mortality rates)• Unmotivated-believe individual care is good
![Page 20: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/20.jpg)
The ProvidersEvidence from the USA
• Institutions (hospitals, systems) do pay attentionand use:– To improve appropriateness of care– To identify poor performers– To alter processes responsive to complaints
• Individual providers less responsive to data
![Page 21: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/21.jpg)
Case Study: New YorkReporting of Performance Data
• Publicly reported risk-adjusted mortality pastCABG
• New York had the lowest risk-adjustedmortality rate in the USA after 4 years.
• First 3 years mortality rate fell 41%• Rate of decline 2x national rate of decline in
5 years
![Page 22: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/22.jpg)
Case Study: New York
• Improvement driven through actions taken byhospital staff– Changes in leadership– Curtailment of operating privileges– Intensive peer review
• Consumer or market force: minimal action
BUT …. WAS PUBLIC DISCLOSURE THE DRIVER?
![Page 23: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/23.jpg)
Purchasers/Payers/CommissionersEvidence from the USA
• Little evidence of performance to exercise “market clout”• Two large studies (15,000 employers nation wide)
– Data used minimally– Price still main selection factor– Data suffers as not designed for buyer decision-makers
• Reliance on purchasers and payers to use performance datanot a reliable strategy
![Page 24: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/24.jpg)
Policymakers
• Some evidence that policymakers do usecomparative performance indicators
• New national initiatives in Canada, Australia,United Kingdom and United States for nationalperformance reporting
![Page 25: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/25.jpg)
The Way Forward: Considerations
• Performance reporting has unrealized potential• Public reporting has risks
– Manipulation of data– Tunnel vision– Unintended effects on access– (Further) erode patient confidence– Commercialization– Jeopardize professional QI efforts
• Public reporting is one tactic in overall strategy
![Page 26: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/26.jpg)
Knowing is not enough, we must apply…..
Knowing is not enough, we must apply…..
Goethe
![Page 27: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/27.jpg)
Apply Indicators to Interventionsfor Change
• External oversight• Patient engagement/empowering consumers• Regulations• Knowledge/skill enhancement of providers• Incentives
![Page 28: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/28.jpg)
External Oversight
• External review/inspection• Accreditation, licensing and certification
• Setting performance targets
![Page 29: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/29.jpg)
External Oversights:Setting Performance Targets
WHY?WHY?• To make policy priorities explicit• Define responsibilities/expectations• Facilitate accountability• Focus resources
![Page 30: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/30.jpg)
External Oversights: Setting Performance Targets
Case Study: Safety/adverse events in the NHS• Priority; quality of care problem• Resource problem; outstanding claims for alleged
clinical negligence of £3.9 billion (5.6 million US$)• Government White Paper established policy in
Spring 2001• Identified 4 areas measurable targets/reporting
(40% reduction of prescribing errors by 2005)
![Page 31: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/31.jpg)
Knowledge/Skill Enhancement
• Peer review and data feedback• Use of guidelines and protocols
![Page 32: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/32.jpg)
Knowledge/Skill Enhancement
•Both WILL and SKILL problems–Impossibility to assimilate new knowledge–Numbers of articles published from RCTs
•1960 1,000 annually•1990 10,000 annually
–15-20 year time lapse: research >>> practice
![Page 33: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/33.jpg)
Knowledge/Skill Enhancement
• Evidence that multiple interventions needed:– Explicit performance indicators agreed– Publish guidelines/protocols and indicators embedded– Peer review
• Adherence to “gold standard”• Peer practice comparisons
– Public reporting– Computer assisted decision-support– Incentives
![Page 34: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/34.jpg)
Patient Engagement/EmpoweringConsumers
• Two applications of performanceindicators at level of individual– Role of consumer of services– Role of patient
• Co-producer of health
![Page 35: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/35.jpg)
Patient Engagement/ EmpoweringConsumers
Still relatively little use of published data.
Patient use of information; evidence•Use of health information- better health outcomes•Shared decision making - choose less risky procedures•Informed of errors - less litigation/claims filed
![Page 36: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/36.jpg)
Incentives
• Performance targets set• Standardized measurement and reporting• “Pay-for-performance”
– Example UK and Australia:Financial incentives for immunization
• Non-financial incentives– Example UK:
“Earned autonomy” and “traffic lighting”
![Page 37: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/37.jpg)
What is Needed for Capacity Building ?•“Will” to address problems•Articulated national policy•Priority setting
•Performance monitoring capability•“Essential infrastructure”
–New organizations–Legal network–IT
•Knowledge aids (protocols, DSS)
•Incentives
![Page 38: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/38.jpg)
Organizing and Integrating Performance
Professionalism
Patient engagement
Incentives
IT Support
Knowledge aids
Design measures
Design reporting formats
National priority setting
National performance framework
Legal framework
PolicyFormulation &Infrastructure
Performance Monitoring Macromanagement
Operations ManagementGovernance
Clinical Service ProvisionIndividual Accountability
![Page 39: Applying Performance Indicators to Health Systems …Applying Performance Indicators to Health Systems Improvement Sheila Leatherman Center for Health Care Policy and Evaluation, Minneapolis,](https://reader035.vdocuments.net/reader035/viewer/2022062506/5f0b096f7e708231d42e89e4/html5/thumbnails/39.jpg)
Knowing is not enough, we must apply
Willing is not enough, we must do.
Knowing is not enough, we must apply
Willing is not enough, we must do.
Goethe