the challenge of designing ‘good enough’ performance measures & results framework targets...
TRANSCRIPT
![Page 1: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/1.jpg)
The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework
Targets & Performance ManagementWorld Bank’s Public Sector Performance
Global Expert Team 9 May
Gwyn Bevan
![Page 2: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/2.jpg)
Models of delivery of public services
Knights / Knaves
Econs / Humans
Prospect theory
Trust & Altruism (T&A)
Knights Humans Reward failure?
Hierarchy & Targets (H&T)
Knights & Knaves
Econs Sack staff?
Public Ranking (PR) Knights Humans Reputation?
Choice & Competition (C&C)
Knights & Knaves
Econs Winners & losers?
![Page 3: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/3.jpg)
UK’s natural experiment: before & after devolution (1999)
NHSs Schools
Before After Before After
EnglandT&A
H&T
PR
C&C
PRWales T&A C&C
PR
C&C
![Page 4: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/4.jpg)
H&T+PR: ‘naming & shaming’ spotlight + prospect theory
Staff in zero-rated hospital ‘devastating …hit right
down to workforce -- bad reports usually hit senior management upwards
nurses demand changing rooms .. because being accosted in streets’
Source: Mannion et al (2005)
‘dirty dozen’ zero-rated CEOs(English NHS)
![Page 5: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/5.jpg)
Hospital waiting times: T&A H&T+PR
Numbers waiting elective admissions (England) (‘000s)
H&T+PR
![Page 6: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/6.jpg)
Hospital waiting times: T&A
0
10
20
30
40
50
60
70
1997 1999 2001 2003 2005
England Wales
No/‘000 waiting > 3 months GP 1st outpatient appointment
![Page 7: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/7.jpg)
Hospital waiting times. Natural experiment: T&A vs H&T+PR
051015202530354045
1997 1999 2001 2003 2005
England Wales
No/‘000 waiting > 3 months GP 1st outpatient appointment
H&T+PR
![Page 8: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/8.jpg)
Ambulances:T&A
45
50
55
60
65
70
75
80
1999 2001 2003 2005 2007
England Wales
Source: Bevan & Hamblin (2009)
% responses life-threatening emergencies < 8 minutes Target
E & W
![Page 9: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/9.jpg)
Ambulances. Natural experiment: T&A vs H&T+PR
45
50
55
60
65
70
75
80
1999 2001 2003 2005 2007
England Wales
Source: Bevan & Hamblin (2009)
% responses life-threatening emergencies < 8 minutes H&T+P
RTarget E
Target W
![Page 10: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/10.jpg)
35
45
55
65
1994 1996 1998 2000 2002 2004 2006 2008
England Wales
% pupils achieving > 5 good grades at 16
Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years
Schools: C&C+PR
![Page 11: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/11.jpg)
35
45
55
65
1994 1996 1998 2000 2002 2004 2006 2008
England Wales
Schools. Natural experiment: C&C+PR vs C&C
Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years
Wales stops PR
% pupils achieving > 5 good grades at 16
![Page 12: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/12.jpg)
H&T & PR: design
Residual domain R
Gages on P
No measures on P
Tin openers on P
Selected domain P
![Page 13: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/13.jpg)
H&T & PR : problems of gaming
Selected domain T: incentives apply to measures M[T]
Excluded domain N: unimportant or cannot be measured
Holmstrom & Milgram (1991)
![Page 14: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/14.jpg)
H&T & PR: types of gaming
N? Worsen
performance? M[T]?
Hit the target & miss the point
Misreporting & cheating
CollusionTarget M[T]
Domain N
Domain T
![Page 15: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/15.jpg)
For discussion
Theory of contingency mix of models
T&A performance measurement no gaming ineffective
PR + H&T powerful gaming Design to counter gaming
C&C choice attractive in principle Characteristics influence effectiveness
![Page 16: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/16.jpg)
For discussion C&C: Requirements
Universities Schools Hospitals
Money follows choice Supply-side flexibility Information Transaction costs Management freedom
![Page 17: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/17.jpg)
COFEPRIS: Commission for Protection against Sanitary Risks
9,000 registrations (value $1bn) required 150 registered in 2010
![Page 18: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/18.jpg)
Elements for transformation of failing organisations
English NHS COFEPRIS
Spotlight effect
Naming & shaming in media
Inform media of failure
‘Broken windows’
Waiting times Registrations
Prospect theory
Sack ‘failing’ CEOs Sack underperforming middle managers
Rewards Name elite organisations
Promote young able managers
![Page 19: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance](https://reader035.vdocuments.net/reader035/viewer/2022081421/56649f0c5503460f94c1f96f/html5/thumbnails/19.jpg)
COFEPRIS: Sanitary registrations
Source: Mikel Arriola Peñalosa (2012) Federal Commissioner for the Protection against Sanitary Risks