1
Measuring & Managing Health Services: The Balanced Scorecard
David Peters, Director, Future Health David Peters, Director, Future Health Systems Research Consortium Systems Research Consortium
Sept 28, 2006Sept 28, 2006
2
“What gets measured gets managed. What gets managed gets done.”
Tom PetersTom Peters
3
Would This Measurement Describe Your Child?
4
Why Have a Balanced Scorecard?
Health sector is complex, has many components
Need an efficient way to assess multiple objectives
Overloaded with different types of reports
Stakeholders demand vigilance Poor measurement can lead to crisis
5
Balanced Scorecard – Original Definition
“A multi-dimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking objectives, initiatives, and measures to an organization’s strategy.”
Kaplan & Norton, 1996
6
Scorecard Domains
Which dimensions should we Which dimensions should we measure?measure?
7
Balanced Scorecard: General Domains
Customer Results Internal Processes Staff and Organizational Growth Financial Results Vision & Strategy
8
Qualities of Good Scorecard Measures
Valid Reliable Balanced Easily Understood Intermediable Agreed Upon Limited
Specific Measurable Achievable
Targets Relevant Timely
9
Presenting Scorecards: Performance Dashboards
Simple to see & interpret Multi-dimensional measures of
critical indicators of overall performance
Allows for the early detection of problems and successes
Can be benchmarked against other data
10
Afghanistan National Health Services Performance Assessment (NHSPA)
11
Developing the Balanced Scorecard
Frontline providers, NGOs, MOPH, donors toagree on: Purpose of BSC Domains to measure Unit of analysis Process & frequency of review/decisions Principles for benchmarking Short-listing indicators based on face
validity, importance, reliability
12
Afghanistan Balanced Scorecard Domains
1. Patient & Community Results2. Staff Results3. Capacity for Service Provision4. Service Provision Results5. Financial System Results6. Overall Vision Results
13
Measuring perceptions
Many survey questions sought perceptions. But how to ask?
Likert scale couldn’t be understood
Money?
ExcellentGoodFairPoor
The Naanogram
14
2004 NHSPA
2005 NHSPA
Provinces 33 30
Facilities 617 628
Basic Health Centers 323 368
Comprehensive HCs 243 217
District Hospitals 51 43
Observations 5719 5863
Exit Interviews 5597 5863
Health worker Interviews 1553 1458
CHW Interviews 167 306
Afghanistan Sample for Balanced Scorecard
Afghanistan NHSPA Provincial Balanced Scorecard Measure
National Median
Lower Benchmark
Upper Benchmark Badakhshan Badghis Baghlan Balkh Bamyan Farah
A. Patients & Community
1 Overall Patient Satisfaction % 83.1 66.4 90.9 86.4 76.9 90.9 84.7 97.9 82.82 Patient Perception of Quality Index % 76.0 66.2 83.9 77.6 66.2 82.2 80.0 84.4 69.43 Written Shura-e-sehie activities in community % 34.2 18.1 66.5 35.6 0.0 34.2 17.7 34.5 73.2
B. Staff
4 Health Worker Satisfaction Index % 63.5 56.1 67.9 63.5 57.6 67.9 68.3 61.4 54.45 Salary payments current % 76.7 52.4 92.0 54.9 91.8 45.8 53.3 91.4 97.7
C. Capacity for Service Provision
6 Equipment Functionality Index* % 65.7 61.3 90.0 69.6 62.2 57.5 67.3 75.8 66.37 Drug Availability Index % 71.1 53.3 81.8 52.9 50.1 72.8 56.1 85.6 9.88 Family Planning Availability Index % 61.4 43.4 80.3 54.2 57.9 70.4 64.9 82.7 0.09 Laboratory Functionality Index (Hospitals & CHCs) % 18.3 5.6 31.7 31.7 3.8 15.2 0.0 37.0 0.0
10 Staffing Index -- Meeting minimum staff guidelines % 39.3 10.1 54.0 38.0 22.4 42.7 45.8 53.0 57.111 Provider Knowledge Score % 53.5 44.8 62.3 48.6 41.6 49.3 54.0 69.0 45.512 Staff received training in last year % 39.0 30.1 56.3 68.9 50.9 39.0 52.4 35.5 37.213 HMIS Use Index % 67.7 49.6 80.7 60.9 62.7 40.0 72.9 67.7 72.414 Clinical Guidelines Index % 34.8 22.5 51.0 18.3 25.5 29.9 16.4 41.9 59.515 Infrastructure Index % 55.0 49.3 63.2 63.2 49.7 50.0 58.3 57.9 76.716 Patient Record Index % 65.6 56.1 92.5 51.5 98.5 80.7 97.3 64.5 97.117 Facilities having TB register % 15.8 8.3 26.6 32.5 27.0 16.1 16.4 0.0 4.3
D. Service Provision
18 Patient History and Physical Exam Index % 70.6 55.1 83.5 54.2 71.7 55.1 85.4 83.6 52.019 Patient Counseling Index % 29.6 23.3 48.9 23.3 40.4 29.3 55.3 33.2 16.020 Proper sharps disposal % 62.2 34.1 85.0 64.4 34.1 76.9 75.1 85.0 67.821 Average new outpatient visits per month (BHC>750 visits) % 22.2 6.7 57.1 27.3 10.0 27.3 71.4 22.2 0.022 Time spent with patient (> 9 minutes) % 18.0 3.5 31.2 21.0 30.7 1.2 27.3 12.8 18.023 BPHS facilities providing antenatal care % 62.0 28.9 82.8 28.9 49.4 49.7 67.2 88.1 82.824 Delivery care according to BPHS % 25.4 10.5 39.3 38.0 36.2 10.5 39.3 38.0 57.1
E. Financial Systems
25 Facilities with user fee guidelines % 90.6 80.3 100.0 94.8 95.6 95.9 28.9 86.1 100.026 Facilities with exemptions for poor patients % 84.7 64.4 100.0 68.5 54.6 69.3 84.3 95.6 93.9
F. Overall Vision
27 Females as % of new outpatients % 55.2 46.5 59.7 46.9 45.9 56.0 55.1 55.2 59.028 Outpatient visit concentration index CI (-1 to 1) -0.010 0.041 -0.055 0.021 0.024 -0.038 0.025 -0.076 -0.03629 Patient satisfaction concentration index CI (-1 to 1) 0.002 0.020 -0.018 -0.019 0.000 0.003 -0.007 -0.005 0.020
Composite Scores
30 Upper Benchmarks Achieved % 17.2 10.3 30.8 17.2 6.9 6.9 20.7 34.5 31.031 Lower Benchmarks Achieved % 82.8 75.9 89.7 86.2 79.3 86.2 86.2 96.6 72.4
16
Mean Score(29 Provinces)
2004 2005 Difference
1. Overall patient satisfaction 81.3 84.1 2.8
2. Patient perceptions of quality
74.1 76.5 2.4
3. Written Shura-e-Sehie Activities 39.9 52.4 12.5
Patients & Community Perspectives
17
Mean Score(29 Provinces)
2004 2005 Difference
4. Health worker satisfaction 61.9 64.0 2.1
5. Salary payments current 68.9 83.9 15.0
Staff Perspectives
20
Mean Score(29 Provinces)
2004 2005 Difference
18. Patient history and physical exam
69.5 74.7 5.1
19. Patient counseling
36.7 38.1 1.4
20. Proper sharps disposal
58.5 48.9 -9.6
21. New outpatient visits (>750/month)
27.6 42.8 15.2
Service Provision
23
Mean Score(29 Provinces)
2004 2005 Difference
27. Females as % of new outpatients 53.7 56.3 2.6
28. Outpatient visit concentration index
-0.008 -0.021 -0.013
29. Pt. satisfaction concentration index
-0.001 0.004 0.005
Overall Vision
24
Identifying Exceptional Provincial Performance
Top Performers
Change 2004-05
Wardak 18.0
Balkh 16.0
Ghor 15.5
Faryab 12.0
Saripul 11.1
Bottom Performers
Change 2004-05
Herat -2.2
Paktika -3.0
Nangarhar -4.7
Nimroz -5.5
Nuristan -13.7
25
Balanced Scorecard: Improvements Progress has been made in most areas
Improvements in 22 out of 29 indicators
Areas of largest improvement include: Shura-e-Sehie activity Salary payments current Laboratory functionality Drug availability Meeting minimum staffing guidelines
26
Balanced Scorecard: Problems Improvement not consistent across all
provinces or aspects of service delivery
Areas of concern:Time spent with patientsFacility infrastructure Sharps disposal User fee guidelines Patient records
27
Balanced Scorecard on Contract Performance: Cumulative Ratings
2004 2005 Difference
Performance Based Contracts (WB)
52.3 64.4 12.1
Contracts with Technical Assistance (USAID)
53.4 63.6 10.2
Ministry of Health Strengthening (WB)
53.7 61.8 8.1
Contracting Without Performance (EU)
54.2 57.7 3.5
Usual Management 51.5 48.5 -3.0
28
Some Final Thoughts
29
Successful Scorecards
Focus on significant, success-determining measures of organization’s total performance
Based on timely information Predictive (leading) indicators more
useful than trailing indicators
30
“You can’t fatten a cow by weighing it”