measuring public sector productivity prelims - ipa · measuring public sector productivity: lessons...

Measuring Public SectorProductivity: Lessons fromInternational Experience

Richard Boyle

CPMR Discussion Paper35

Measuring Public SectorProductivity: Lessons fromInternational Experience

First published in 2006by the Institute of Public Administration57-61 Lansdowne RoadDublin 4Irelandin association withThe Committee for Public Management Research

www.ipa.ie

2006 with the Institute of Public Administration

All rights reserved. No part of this publication may bereproduced or transmitted in any form or by any means,electronic or mechanical, including photocopying, recordingor any information storage and retrieval system, withoutpermission in writing from the publisher.

British Library Cataloguing in Publication DataA catalogue record for this book is available from the BritishLibrary.

ISBN-13: 978-1-904541-49-3ISBN-10: 1-904541-49-6ISSN: 1393-6190

Cover design by Creative Inputs, DublinTypeset by the Institute of Public AdministrationPrinted by ColourBooks Ltd, Dublin

CONTENTS

Foreword vii

Executive Summary ix

Chapter 1: Introduction 11.1 Background and context of the study 11.2 Report Structure 2

Chapter 2: Some definitions and challenges 42.1 Introduction 42.2 Challenges to public sector productivity measurement 6

Chapter 3: Cross-national comparisons of public sectorproductivity and performance 83.1 Introduction 83.2 European Central Bank international comparison of

public sector efficiency 83.3 Netherlands Social, Cultural and Planning Office

study of public sector performance 113.4 World Bank governance indicators 143.5 OECD Management in Government: Comparative

Country Data project 153.6 Conclusions 16

Chapter 4: National and sectoral productivitymeasurement initiatives 174.1 Introduction 174.2 National public sector productivity measurement

initiatives 174.3 Sectoral productivity measurement 224.4 Conclusions 27

Chapter 5: Organisation-level and bottom upproductivity measurement 295.1 Introduction 295.2 Measuring organisational productivity in Denmark 29

v

5.3 Bottom up productivity measurement 305.4 Conclusions 32

Chapter 6: Conclusions and recommendations aframework for the development of public sectorproductivity measurement in Ireland 346.1 Introduction 346.2 Creating a framework for developing public sector

productivity measurement in Ireland 346.3 Conclusion 38

Annex 1 Listing of sources and concepts included inthe World Bank Government Effectiveness indicator 40

Annex 2 Examples of output indicators in the centralgovernment in Finland 44

Annex 3 The UK Office of the Deputy Prime Ministerbasket of cost effectiveness indicators 46

Notes 47

References 48

vi

This paper is the thirty-fifth in a series undertaken by theCommittee for Public Management Research. TheCommittee is developing a comprehensive programme ofresearch designed to serve the needs of the futuredevelopments of the Irish public service. Committeemembers come from the following eight departments:Finance; Environment, Heritage and Local Government;Health and Children; Taoiseach; Transport;Communications, Marine and Natural Resources; Socialand Family Affairs; Office of the Revenue Commissionersand also from Trinity College Dublin, University CollegeDublin and the Institute of Public Administration.

This series aims to prompt discussion and debate ontopical issues of particular interest or concern. The papersmay outline experience, both national and international, indealing with a particular issue. Or they may be moreconceptual in nature, prompting the development of newideas on public management issues. They are not intendedto set out any official position on the topic under scrutiny.Rather, the intention is to identify current thinking andbest practice.

We would very much welcome comments on this paperand on public management research more generally. Toensure that the discussion papers and wider researchprogramme of the Committee for Public ManagementResearch are relevant to managers and staff, we need tohear from you. What do you think of the issues beingraised? Are there other topics you would like to seeresearched?

Research into the problems, solutions and successes ofpublic management processes and the way organisationscan best adapt in a changing environment has much tocontribute to good management, and is a vital element inthe public service renewal process. The Committee forPublic Management Research intends to provide a service to

vii

FOREWORD

people working in public organisations by enhancing theknowledge base on public management issues.

Jim Duffy, ChairCommittee for Public Management ResearchDepartment of Finance

For further information or to pass on any comments pleasecontact:

Pat HicksonSecretaryCommittee for Public Management ResearchDepartment of FinanceLansdowne HouseLansdowne RoadDublin 4

Phone: (+353) 1 676 7571; Fax: (+353) 1 668 2182E-mail: [email protected]

General information on the activities of the Committee forPublic Management Research, including this paper andothers in the series, can be found on its website:www.cpmr.gov.ie; information on Institute of PublicAdministration research in progress can be found atwww.ipa.ie.

viii

This is a study of international experience in measuringpublic service productivity. The research informs possibleapproaches to public sector productivity measurement forthe Irish public service. The study focuses on three mainaspects of productivity measurement: attempts to developcomparative, cross-national assessments of public sectorefficiency and performance; national and sectoral publicsector productivity measurement initiatives; and a moremicro-level examination of productivity measurement,looking at organisation-based and bottom up initiatives tomeasure public sector productivity.

Comparative cross-national assessments of publicsector productivity and performanceThere has been a growth in recent years in internationalcomparative studies of public sector performance. Some ofthese explicitly include productivity measurements; othersfocus more generally on broad performance issues. Theseinternational studies provide scope for a comparativeassessment of how Ireland is performing, particularly ifstudies are repeated over time, allowing trends to beestablished.

However, the studies themselves warn of the danger ofputting too much faith in drawing comparisons, givenqualifications about the type and reliability of data used togenerate the indicators in the studies. It is clear thatfindings are of a tentative nature, and that improvementsare needed if such studies are to provide a sound evidencebase. In the context of improving the evidence base, theOECDs Management in Government: Comparative CountryData project, started in 2006, is a significant initiative.

National and sectoral public sector productivitymeasurement initiativesIn recent years, various countries at both national andsectoral levels have engaged in productivity measurement

ix

EXECUTIVE SUMMARY

initiatives. Brief reviews of progress in the UK, Finland,Sweden and Australia are examined here. This is followedby illustrative examples of productivity measurement inthree sectors: health, education and local government.

The evidence from national and sectoral studies ofpublic sector productivity measurement is that despiteefforts going back to the 1980s, the productivity measuresbeing produced need to be interpreted cautiously. There isalso the danger that over-simplistic use of the measurescan lead to perverse consequences. Supporting evidence isneeded to corroborate the findings of productivitymeasures.

A further general point emerging from the casesexamined is the lead role being taken by national statisticsoffices in public sector productivity measurementinitiatives. The involvement of the national statistics officesis required because of a Eurostat directive on developingoutput measures for the national accounts. Nationalstatistics offices also play a lead role in providing qualityassurance and guarantees about data reliability andvalidity.

Where similar institutions are providing similarservices, it is possible to develop comparative productivitymeasurements, as examples from the Australian states andfrom local government show. Using techniques such asfrontier analysis it is possible to identify relatively efficientand relatively inefficient organisations. The same cautionsas to data reliability and interpretations as raised above,however, still apply.

Organisation-based and bottom up initiatives on publicsector performance measurementOrganisation level productivity measurement is likely to bea feasible and useful tool for those organisations that haveclear, identifiable outputs that can be linked to inputs used.These measures do not necessarily need to cover the wholeorganisation, and may be indicators of productivity fordiscrete parts of the organisation.

Bottom up/service user measurements of performance,such as the time and cost associated with setting up a newbusiness, are being developed in a number of places. Whilethey are not productivity measurements in the strict sense

x

(as they focus on the outputs and broad performance ofpublic sector organisations rather than linking this data toinputs in a direct manner) they do help provide a picture ofwhat value is being delivered by public services in return forthe expenditure supports provided. As such, they have apotentially important role to play in productivitymeasurement when interpreted in a broad sense. Bottomup measures can also be a helpful source of information toprovide triangulation data for more conventionalproductivity studies.

Developing a framework for public sector productivitymeasurement in Ireland

Information on public sector productivity in Ireland islimited. So as to develop a broad range of measures ofproductivity and not rely on single data sources, aframework for the development of productivitymeasurement is outlined in the table below. Thisframework proposes that action be taken at a number oflevels cross-national, national and sectoral, andorganisation-based and bottom up to develop informationon public sector productivity in Ireland. In this way, adiversity of approaches to productivity measurement can beused to provide a broad picture of productivitydevelopments. The framework draws from lessons learnedfrom the international experience outlined in this study.

xi

A framework for the development of public sectorproductivity measurement in Ireland

xii

Productivity initiative

Action required

Cross-national comparative studies

Track Irelands comparative

performance in periodic studies of public sector performance and efficiency such as the World Bank and European Central Bank studies.

Actively participate in and encourage the OECD Management in Government: Comparative Country Data initiative.

National and sectoral initiatives

The Central Statistics Office should

take a lead role in the development of the measurement of government output and subsequent productivity studies.

Annual output statements being developed by government departments should inform productivity studies.

The health and education sectors should be priorities for productivity studies.

Relevant state bodies and academic institutions should be encouraged to undertake research into public sector productivity measurement.

Benchmarking of comparable organisations should take place.

Organisation-based and bottom up initiatives

Organisation-based measures of

productivity should be developed, using annual output statements as a basis for this work.

Central agencies should sponsor a number of service user based studies of the efficiency of public service provision across a range of sectors, repeated periodically.

Benchmarking with comparable organisations should be encouraged.

1.1 Background and context of the studyThe pay awards recommended by the Public ServiceBenchmarking Body and implemented as part of the socialpartnership arrangements have generated significant publicdiscussion about the productivity gains realised in returnfor pay increases in the public sector. The subject of publicsector productivity is of growing concern to many interestedparties. As Afonso, Schuknecht and Tanzi (2006) note inrelation to public sector efficiency in European andemerging markets:

Health, education and similar activities absorb a largeshare of the government payroll and the personnel whowork for government If mostly higher salaries absorbadditional resources allocated to these activities and thehigher salaries are not accompanied by higherproductivity of the public employees, the higher publicspending can be unproductive and produce littleadditional benefits to the students or patients.

But public sector productivity is notoriously difficult tomeasure, not only in Ireland but also internationally. Muchproductivity data for the public sector is of questionablevalidity and/or reliability. Assessing the productivity ofpolicy-oriented organisations has proved particularlychallenging.

A number of international studies have been carried outthat address the issue of public sector productivity. Someof these studies examine whole of government productivityand make comparisons between countries (for example seeSocial and Cultural Planning Office, 2004). Some studiesfocus on sectors (health, education etc) and may becountry-based or aim to examine cross-national trends (for

1

1

Introduction

MEASURING PUBLIC SECTOR PRODUCTIVITY

example, see Atkinson, 2005). Other studies aim to trackproductivity changes over time in public administration inindividual countries (such as productivity studiesundertaken in Sweden, Ministry of Finance, 1997).

There are also initiatives underway at present that willfurther the examination of public sector productivity. AEurostat directive on price and volume measurement ofgovernment output is generating work in European memberstates on improving output measures in the public serviceand the development of productivity measures (Eurostat,2001)1. The OECD public sector management andperformance division are undertaking a project onManagement in Government: Comparative Country Data thataims to develop comparable data and indicators of goodgovernment and efficient public services. The OECDproposes to take 3-5 years to develop the database, with aninitial report at the end of 2006. The database will includeinformation on government inputs and outputs and hencefacilitate productivity assessment.

Given the current state of play with regard to publicservice productivity measurement, a phased approach hasbeen adopted by CPMR for this research topic. In this firstphase, a detailed examination of previous internationalexperience in assessing public service productivity has beenundertaken. This research will inform possible approachesto be developed for the Irish public service into the future.As such, it is intended to inform both future CPMR workand the work of others interested in this area.

1.2 Report structureChapter 2 deals with some of the definitions of productivityand the challenges associated with productivitymeasurement. Chapter 3 examines attempts to developcomparative, cross-national assessments of public sectorefficiency and performance. In Chapter 4, national publicsector productivity initiatives in a number of countries areoutlined, together with sectoral studies (health, educationetc) within countries. Chapter 5 is a more micro-levelexamination of productivity measurement, looking atorganisation-based and bottom up initiatives to measure

2

INTRODUCTION 3

public sector productivity. Finally, in Chapter 6, thelessons learned from international experience are drawntogether to help develop a framework for the development ofpublic sector productivity measurement in Ireland.

2.1 IntroductionProductivity is generally defined as a measure of theamount of output generated per unit of input. In manycountries, including Ireland, public sector productivity hasbeen assumed to be zero in the national accounts. Theoutput of the government sector has been measured as ofvalue equal to the total value of inputs. This output=inputconvention has increasingly come under scrutiny in recentyears and is no longer accepted practice from 20062. Thechallenge is to devise alternative estimates based on outputmeasurement, in a public sector context where there isprovision of collective services, and where there is nomarket transaction in services provided to individuals inmost instances.

If it is accepted that in reality changes in outputs arenot likely to be directly equivalent to changes in inputs,Pollitt and Bouckaert (2004) note that public sectorproductivity may increase for a variety of reasons:

where resources (inputs) decrease and outputs increase where resources remain the same and outputs increase where resources increase but outputs increase by an

even larger amount where outputs remain static but resources decrease where outputs decrease but inputs decrease by an even

larger amount.

However, this definition of productivity as beingconcerned with the relationship between inputs andoutputs does not cover issues that many people have inmind when they talk about public sector productivity. Amore general interpretation of productivity encompassesbroader concerns about the outcomes achieved by the

Some definitions and challenges

2

4

SOME DEFINITIONS AND CHALLENGES 5

public sector. In common parlance, when many people talkabout public sector productivity, they have in mind thegeneral question of what value they are receiving frompublic services in return for the application of public funds.

Putnam (1993) rejects the idea of including outcomes inproductivity measurement. His argument is that to focuson outcomes (changes in health rather than patientstreated; changes in educational status rather than numbersof lessons taught) includes changes over which thegovernment has no control:

To include social outcomes in an assessment ofgovernment performance is to commit theMassachusetts Miracle Fallacy: only a modest part ofthe praise for the affluence of New England in the 1980s(and a similarly modest portion of the blame for thesubsequent recession) was realistically attributable tostate government, despite 1988 presidential campaignrhetoric to the contrary.

Notwithstanding the problems with assessingproductivity using an outcomes focus as indicated above, inthis paper, both the input/output measurement andbroader assessments of public sector productivity,including a focus on outcomes, are included in thediscussion on productivity. It is accepted that for nationalaccounts purposes, and when attributing changes inproductivity to the public sector, strict definitions areneeded. But the broader interpretation of productivity asincluding a concern with outcomes, while having statisticaland measurement limitations, nevertheless has resonancewith the general public and may raise interesting questionseven if it does not provide definitive answers.

Both the narrow, economic definition of productivityconcerned with the input/output ratio and the broaderproductivity definition concerned with the input/outcomeratio can be considered sub-sets of public sectorperformance measurement. Performance measurement isnot confined to issues of productivity, and may examine


inputs, outputs and outcomes without necessarily beingconcerned with the measurement of the ratio betweenthem.

2.2 Challenges to public sector productivitymeasurement

Even when the definition of public sector productivity isconfined to the relationship between inputs and outputs,there are challenges in measurement, both of inputs andoutputs.

Inputs are made up of three elements: labour,procurement of goods and services and capitalconsumption (Atkinson Review, 2005). Measurement ofeach of these elements may pose particular challenges inpractice. For example, with regard to labour, shouldnumber of hours worked (differentiated by skill) be usedinstead of the number of people employed? With regard tothe measurement of outputs, the European Commission(2004) identify three important issues: how to defineoutput; how to define aggregate output over a range ofdifferent products; and how to incorporate exogenousconditions, such as the general health condition of apatient.

A further challenge with regard to output measurementfor productivity purposes is how to incorporate changes inthe quality of outputs. The importance of this point isillustrated by Pritchard (2002a), of the Office of NationalStatistics in the UK, who states that: the measurementprocess must reflect the fact that 100 units of good qualitythis year represent more output than 100 units of a lesserquality last year. Pritchard (2002b) goes on to furtherillustrate the importance and challenge of monitoringquality changes in outputs:

Our approach starts with the idea that producingsomething of a higher quality is equivalent to producinga higher volume of output: the quality is an attribute ofthe output (and not of the inputs). In the market sector,where goods and services have prices, there are several

6

SOME DEFINITIONS AND CHALLENGES 7

options for measuring the amount which improvedquality adds to the volume of production: they all relatein some way to price. But government services usuallyhave no price. Nevertheless, the rate of change ofquality may well be significant: and it is also an issue ofpublic concern and debate, always in the spotlight. It isimperative that we try to bring quality change into ourmeasure.

Yet another challenge with regard to measurementrelates to possible time lags between the inputs andoutputs. Money spent on public sector inputs may not havean impact in terms of improved outputs for some time (insome cases years) after the initial expenditure.

There are considerable technical and other challengesassociated with measuring public sector productivity,however defined. This makes comparability of trends, overtime and across sectors and countries, particularlyproblematic. In any discussion on measuring public sectorproductivity, these challenges must be borne in mind andfactored into interpretations of findings.

3.1 IntroductionA small number of studies have been carried out thatcompare administrative efficiency and performance interna-tionally at the aggregate level (Van de Walle, 2005). Themain studies examined here are: a European Central Bankinternational comparison of public sector efficiency (Afonso,Schuknecht and Tanzi, 2003 and 2006); a report of theSocial and Cultural Planning Office in the Netherlands onpublic sector performance (Social and Cultural PlanningOffice, 2004); and a World Bank study of governanceindicators (Kaufmann, Kraay and Mastruzzi, 2005). Inaddition, the work of the OECD in developing a project onManagement in Government: Comparative Country Data,referred to in section 1.1, is discussed.3

3.2 European Central Bank international comparison ofpublic sector efficiency

Afonso, Schuknecht and Tanzi (2003) examine theperformance and efficiency of the public sectors of twenty-three industrialised OECD countries. They developmeasures of both public sector performance (which theydefine as the outcome of public sector activities) andefficiency (which they define as the outcome relative to theresources employed).

With regard to public sector performance, they defineseven sub-indicators of public performance (see Figure 3.1).The first four examine administrative, education, healthand public infrastructure outcomes. They term theseopportunity indicators, concerning the role of governmentin providing opportunities and a level playing field in themarket process. The other three sub-indicators try tocapture the traditional Musgravian tasks for government ofallocation, distribution and stabilisation (Musgrave, 1959),measuring income distribution, economic stability and

Cross-national comparisons of publicsector productivity and performance

3

8

CROSS NATIONAL COMPARISONS 9

economic performance as a measure of allocative efficiency.Each sub-indicator is measured as a composite of a numberof indices, as outlined in Figure 3.1.

Figure 3.1 Total public sector performance indicator

Source: Afonso, Schuknecht and Tanzi, 2003

Opportunity indicators Standard Musgravian indicators

Corruption

Quality of

judiciary

Red tape

Shadow

economy

Secondary

school

Quality

communication

and transport

infrastrucure

Infant

mortality

Education

achievement

Life

expectancy

Income share of

40% poorest

households

Stability of GDP

growth

(coefficient of

variation)

Inflation (10

years average)

GDP per capita

(PPP)

GDP growth (10

years average)

Unemployment

(10 years

average)

Distribution

Stability

Administrative

Education

Health

Public

infrastructure

Economic

performance

Total public sector

performance


With regard to public sector efficiency, publicexpenditure, expressed as a share of GDP, is used to reflectthe opportunity costs of achieving public sectorperformance. In addition to total public spending, theyexamine average spending on goods and services, transfers,functional spending on education and health, and publicinvestment. In order to arrive at efficiency indicators,public spending is normalised across countries.

The study finds that the difference in public sectorperformance overall is moderate across the samplecountries. Countries with small public sectors (publicspending less than 40 per cent of GDP) on average reportthe highest scores, especially for administrative andeconomic performance. Countries with large public sectors(public spending over 50 per cent Of GDP) show more equalincome distribution. Regarding public sector efficiency,countries with small public sectors display considerablyhigher indicators of efficiency than countries with medium-sized or big public sectors. However, the authors cautionthat the results must be seen as indicative and need to beinterpreted with great care.

This latter point about caution is well made. Taking theindicator of administrative performance as an illustration(and one particularly appropriate to this study), Van deWalle (2005) notes that contrasting this indicator withgovernment goods and services expenditure to develop ameasure of efficiency fails to recognise that the goods andservices category in the national accounts is a crudeapproximation of what is spent on the publicadministration and judiciary. With regard to the fourfactors that make up the administrative indicator corruption, red tape, quality of the judiciary and extent ofthe shadow economy Van de Walle (2005) further notesthat:

Despite this study being one of the only ones attemptingto compare public sector efficiency internationally, theindicators used for administrative performance aredefective. First, only four subfactors are used,neglecting many factors relevant for administrative

10


performance. Second, three out of four subfactors areof a purely subjective nature, as they are based on theexecutive surveys that are used for the World EconomicForum's World Competitiveness Yearbook, a surveywhich is based on rather small samples in somecountries. Third, for one specific subfactor (confidencein the administration of justice), the authors rely on theWorld Competitiveness Yearbook, while much bettersurvey material is available for measuring thissubfactor.

But the study, despite these caveats, represents aninteresting approach to cross-national assessment of publicsector productivity and performance. The authors repeatedthe study to analyse public sector efficiency in the newmember states of the European Union and emergingmarkets in Asia (Afonso, Schuknecht and Tanzi, 2006).

3.3 Netherlands Social, Cultural and Planning Officestudy of public sector performance

As part of the Dutch presidency of the European Union inthe second half of 2004, the Dutch Ministry of the Interiorand Kingdom Relations asked the Social and CulturalPlanning Office of the Netherlands to investigate publicperformance in the EU member states and four major non-EU Anglo-Saxon countries (Social, Cultural and PlanningOffice, 2004). The report covers four main areas: education,health care, law and order, and public administration. Italso assesses the overall performance of the public sector.

Education. Achievement and attainment criteria are usedas criteria for effectiveness of the educational system.Achievement indicators are based on internationalcomparative achievement tests, measuring reading andmathematical skills and scientific literacy among fifteenyear olds. Attainment indicators used are the proportion ofthird level graduates in the population and the proportionof early school leavers. A single measure is used to combineachievement and attainment indicators. This measure is


set against the costs of education (per capita expenditure).

Health care. Life expectancy, infant mortality, proportionof healthy life years and general feeling of good health areused as indicators and combined to produce an index ofhealth status to measure the effectiveness of nationalsystems. Per capita expenditure on health care is used todetermine cost-effectiveness.

Law and order. The number of convictions expressed as aproportion of the number of recorded offences is used togive a picture of the way the criminal justice systemfunctions. Labour productivity is determined by thequotient of the number of convicted suspects and staffnumbers of the police, public prosecutions department andcriminal courts. In the prisons service, the number ofprisoners per prison guard measures labour productivity.

Public administration. In the report, four indicators ofgovernment quality are used:

the level of the bureaucracy: does bureaucracy hinderbusiness decisions?

the level of transparency: is transparency of governmentpolicy satisfactory?

the level of effectiveness: are government decisionseffectively implemented?

the level of corruption: do bribing and corruption existin the economy?

The first three of these indicators are measured using asurvey among representatives of the business communityin a range of countries (IMD, 2003). The corruptionindicator is measured using the Transparency Internationalcomposite index of corruption.

In a review of the work of the Social, Cultural andPlanning Office, Kuhry, Pommer and de Kam (2006) notethat:

12


By the very nature of its products shaping policy in awide variety of areas, law making, maintaining publicorder, managing the government apparatus nonatural performance indicators are available for publicadministration as such. The functioning of governmentadministrations can therefore only be measured byusing subjective indicators. Such indicators reflectmainly trust and confidence in the civil service.

Overall performance of the public sector. The scores onvarious government functions are combined in one overallindex of public sector performance. The combined scorerepresents four main dimensions of performance:stabilisation and growth of the economy, distribution ofwelfare, allocation of public services, and quality of publicadministration. The overall performance is related to theresources absorbed by producers active in the publicsector. Roughly speaking, the study finds little connectionbetween public sector performance and the level of publicand private spending. But using the global efficiencymeasure, Kuhry, Pommer and de Kam (2006) find:

By this measure, Finland is the most efficient inproducing public services of high quality at moderatelyhigh costs, while in terms of efficient production Ireland scores slightly above average at low costs. Justbehind these leaders we find Sweden, Denmark,Austria, Luxembourg and the Netherlands; the firstthree countries post relatively high spending levels,while the last two have fairly average spending.Australia, Canada, Spain and the Czech Republiccombine an average performance score with fairly lowgovernment spending, while others (particularlyGermany, Belgium and France) occupy fairly averagepositions in both respects. The US and the UnitedKingdom perform fairly poorly at relatively low spendinglevels.


One of the most striking outcomes is that the sameclusters of countries repeatedly emerge in analyses ofpublic sector performance, regardless of the policy areaand characteristics reviewed. Again and again,Northern European countries, Western Europeancountries, Southern European countries, CentralEuropean countries and Anglo-Saxon countries aredemonstrated to form fairly consistent clusters.

As with the European Central Bank study, the authorsurge caution with regard to the interpretation of the results.

3.4 World Bank governance indicatorsSince 1996, the World Bank has been developinggovernance indicators as part of its work in promoting goodgovernance. Governance indicators are produced for justover 200 countries every two years. Kaufmann, Kraay andMastruzzi (2005) note that the governance indicators usedmeasure six dimensions of governance: voice andaccountability; political instability and violence;government effectiveness; regulatory quality; rule of law;and control of corruption. The indicators are based onseveral hundred individual variables measuring perceptionsof governance, drawn from thirty-seven separate datasources constructed by thirty-one different organisations.

Most relevant from the perspective of this study is thegovernment effectiveness indicator. It aims to measure thecompetence of the bureaucracy and the quality of publicservice delivery. A broad range of sources is used, includingthe Economist Intelligence Unit, World Economic Forum,Bertelsmann Foundation and Institute for ManagementDevelopment (a full listing of sources and conceptsmeasured is given in Annex 1). By taking this broad,encompassing approach, Van de Walle (2005) notes that:it covers a broad range of related concepts: red tape,quality of public schools, government stability, bureaucratsexpertise, policy consistency, ability to deliver basicinfrastructure. Point estimates of the dimensions are

14


presented as well as margins of error for each country andperiod. Kaufmann, Kraay and Mastruzzi (2005) suggestthat: An advantage of our measures of governance is thatwe are able to be explicit about the accompanying margin oferror, whereas these are most often left implicit withobjective measures of governance.

Van de Walle (2005) provides a helpful critique of theWorld Bank governance indicators:

The World Bank Governance Indicators dataset is one ofthe most complete datasets to assess the quality ofgovernance. Many of its composing indicators, however,are of a subjective nature, and therefore do notnecessarily present us with a correct picture. Thenumber of data sources employed by the World Bank,however, softens this criticism. But then again, thislarge number of data sources also leads to a very broadrange of concepts covered, which make it difficult todetermine what exactly is measured by governmentefficiency.

3.5 OECD Management in Government: ComparativeCountry Data project

The Public Governance Committee of the OECD hasmandated the Public Governance and TerritorialDevelopment Directorate to assess the feasibility ofdeveloping comparable data and indicators of goodgovernment and efficient public services. This project,entitled Management in Government: Comparative CountryData, aims to provide good empirical data and indicators ofgood government. The intention is to move, on a phasedbasis, to the production of a publication provisionallyentitled Government at a Glance, which will mirror theOECD's Education at a Glance publication and showcomparative cross-national data on an annual basis.

An initial assessment of available data has beenundertaken, alongside a detailed literature review (OECD,2005). The focus is on several types of measures: inputs,processes, outputs, outcomes and antecedents or


constraints that put government efficiency in context. Theintention is to produce a first working paper towards theend of 2006, mostly concerned with inputs and processes,as these are the most readily available data. Dataconcerning outputs and outcomes are seen as more difficultto gather, but the intention is to gradually improve coveragein these areas.

3.6 ConclusionsThere has been a growth in recent years in internationalcomparative studies of public sector performance. Some ofthese explicitly include productivity measurements; othersfocus more generally on broad performance issues. Theseinternational studies provide scope for a comparativeassessment of how Ireland is performing, particularly ifstudies are repeated over time, allowing trends to beestablished.

However, the studies themselves warn of the danger ofputting too much faith in drawing comparisons, givenqualifications about the type and reliability of data used togenerate the indicators used in the studies. It is clear thatfindings are of a tentative nature, and that improvementsare needed if such studies are to provide a sound evidencebase.

In the context of improving the evidence base, theOECDs Management in Government: Comparative CountryData project is a significant initiative. It is important thatIreland support this project so as to help develop aninformation base from which comparative performance andproductivity data can be developed over time.

16

4.1 IntroductionIn recent years, various countries at both national andsectoral levels have engaged in productivity measurementinitiatives. In this chapter, national initiatives areexamined in the first instance, with brief reviews of progressin the UK, Finland, Sweden and Australia. This is followedby illustrative examples of productivity measurement inthree sectors: health, education and local government.

4.2 National public sector productivity measurementinitiatives

4.2.1 Measuring public sector productivity in the UKSince 1988, the Office for National Statistics has beenprogressively moving away from the output=input approachto productivity, and incorporating direct measures of thevolume of government output in the national accounts. By2005, these direct output estimates accounted for twothirds of general government final consumption. In thecontext of this focus on output measurement, the UKgovernment commissioned Sir Tony Atkinson to undertakea review of the measurement of government output in thenational accounts. This review (Atkinson, 2005) provides acomprehensive overview of developments and recommenda-tions for future progress.

The Atkinson review outlines a number of principlescovering the measurement of outputs, inputs andproductivity. One particularly significant point is that thereview strongly recommends that, in principle, measures ofoutput growth should take account of quality change. Also,and specifically with regard to productivity, the reviewstates:

17

4

National and sectoral productivitymeasurement initiatives


Outputs divided by inputs provides a measure ofproductivity change. However, the move from the(output=input) convention to direct measurement ofgovernment output should be carefully interpreted. It isa definite advance in the sense that government outputis no longer simply assumed to equal measured inputs,but the move should not be seen as solving at a strokethe complex problem of measuring governmentproductivity. The statistic obtained by dividing outputsby inputs may no longer be equal to 1 by definition, butno single number, however carefully constructed, canfully capture the performance of complex public serviceswith multiple objectives. Productivity change should beinterpreted in the light of a range of other information the triangulation principle.

The UK government accepted the findings and recom-mendations of the Atkinson review, and the Office forNational Statistics (ONS) is taking the lead role in takingforward the recommendations. To this end, the ONS hasset up the UK Centre for the Measurement of GovernmentActivity (UKCeMGA). The UKCeMGA issued its first annualreport in 2006, outlining progress and in particular drawingstrands of research on productivity analysis together in keyareas such as health and education (Office for NationalStatistics, 2006a).

4.2.2 Measuring public sector productivity in FinlandFinland, along with the UK, is widely regarded as aEuropean leader in public sector productivitymeasurement. A project was established to measure publicsector productivity in Finland in 1995, located in StatisticsFinland, the national statistics office. The aim of the projectis: to develop a measurement and monitoring system forgovernment sector production and productivity by using anoutput indicator method to measure the volume of output(Niemi, 1998). In 1997, the scope of the project wasexpanded to include the measurement of the productivity oflocal government services.

18

NATIONAL AND SECTORAL PRODUCTIVITY MEASUREMENT INITIATIVES 19

Under the terms of the project, for central governmentservices the final output and the output indicators arespecified by the agencies themselves. Examples of outputindicators are given in Annex 2. The agencies for whichinput and output data are gathered cover about 80 per centof the compensation of employees in central government.Initial results show growth rates of output and productivityvarying extensively. For local government, measurementscover educational, cultural and social services. To givesome examples of output indicators, teaching hours areused as the output indicator for educational services,numbers of library visitors for libraries and number of beddays, customers or visits for social services (Niemi, 1998).

4.2.3 Measuring public sector productivity in SwedenSweden has been measuring public sector productivitysince the mid-1980s. An Expert Group on Public Finances( a subcommittee under the Ministry of Finance) establisheda steering group to conduct the work. The steering groupwas supported by Statistics Sweden, the national statisticsoffice (Ministry of Finance, 1997).

Particular focus is given to the development of outputindicators for services. These include items such as thenumber of admitted patients for in-patient medical care, thenumber of learning hours for education, traffic volume asmeasured by vehicle kilometres for public roads, number offlying hours for the air force (Ministry of Finance, 1997).Attempts are made to adjust the quantity of outputs forquality variations where data are available. A number oflessons are drawn based on the experience of producingannual productivity measures over a period of time:

Productivity varies greatly from year to year. To assessany given year, a time series of several years informationis needed.

Most agencies have several categories of output.Attributing the same weight to all categories may yielddeceptive results. In the enforcement service, forexample, counting cases dealt with regardless of


category gives a 2 per cent decrease in output from1981 to 1992. If, however, different weights areassigned to different categories of output, to take intoaccount differences in composition of the cases, theresult is a 20 per cent increase in output.

To measure the productivity of an agency as a wholemeans overheads must be taken into account.

Results differ depending on the kind of price indexchosen (Ministry of Finance, 1997).

Findings from the studies as summarised by Pollitt andBouckaert (2004) suggest that:

A close examination of specific cases suggests thatpublic management reforms can help to increaseproductivity, especially when carried out in conjunctionwith budget cuts or increases in demand for a service,which were not paralleled by any significant increase inresource inputs, but that management reforms in theabsence of downward pressure on inputs are notnecessarily terribly effective in improving productivity.

4.2.4 Measuring public sector productivity inAustralia

In Australia, in 1993 the Council of AustralianGovernments established the Review of Government ServiceProvision to provide information on the effectiveness andefficiency of government services (Australian ProductivityCommission, 2006). The review is conducted annually, andoverseen by a steering committee of senior representativesfrom the central agencies of all the state governments, withthe assistance of a secretariat provided by the ProductivityCommission. Performance information is provided onfourteen service areas covering six main governmentfunctions: education; justice; emergency management;health; community services; and housing.

The report includes performance comparisons acrossjurisdictions for the services using a common method. The

20


general performance framework used is set out in Figure4.1. It can be seen that both outputs and outcomes aremeasured, as well as efficiency, effectiveness and equity.With regard to efficiency, the report's focus is on technicalefficiency:

Technical efficiency indicators measure how wellservices use their resources (inputs) to produce outputsfor the purpose of achieving desired outcomes.Government funding per unit of output delivered istypically used as an indicator of technical efficiency forexample, recurrent funding per annual curriculum hourfor vocational education and training.

Where there are shortcomings in the data, otherindicators of efficiency are used (including partialproductivity ratios such as staff level per student ingovernment schools, staff per prisoner in correctiveservices and administrative costs as a proportion oftotal expenditure in services for people with a disability)(Australian Productivity Commission, 2006).

Figure 4.1 A general framework and examples ofperformance indicators

Source: Adapted from the Australian Productivity Commission,2006

Cost effectiveness indicators

Techinical efficiency indicators

Inputs per output unit

Efficiency

Quality indicatorsQualityPerformance

Program effectiveness

indicators

Appropriateness indicators

AppropriatenessEffectiveness

Access indicatorsAccessObjectives

Equity of income indicators

Equity of access indicators

AccessEquity

Outputs Outcomes


4.3 Sectoral productivity measurementIn this section, a small number of productivity studies arecited to give illustrative examples of the type of sectoralproductivity information being produced in three sectors:health, education and local government.

4.3.1 Health sector productivity measurementThe UK Office for National Statistics (2006b) has publisheda major review of health service productivity. Usingavailable data, the Office for National Statistics producedthree different estimates of NHS productivity (see Figure 4.2a, b, and c). The first estimate is based on current nationalaccounts estimates of output. Using this measure, NHSproductivity is estimated to have fallen during the period1995 to 2004 by an average of between 0.6 and 1.3 per centper year.

The second estimate is based on the principle outlinedin the Atkinson Review (Atkinson, 2005) that output shouldbe adjusted to take account of quality change. A number ofquality indicators are used, including: survival rates; healtheffects; an adjustment for life expectancy for survival ratesand health expectancy; waiting times; improvements inprimary medical care; longer term survival rates formyocardial infarction; and patient experience. On thisbasis, productivity is estimated to have either increased byan average of 0.2 per cent per year, or has fallen by anaverage of 0.5 per cent per year.

The third estimate is also based on a recommendationoutlined in the Atkinson review, that the value of NHSoutput should be adjusted by rising real earnings in theeconomy to reflect the fact that health becomes increasinglyvaluable in a growing and increasingly productiveeconomy.4 On this basis, NHS productivity is estimated tohave increased by an average of between 0.9 and 1.6 percent per year.

22


Figure 4.2 NHS Productivity Estimates

(a) NHS productivity, excluding quality change for NHS output,1995 to 2004

88

90

92

94

96

98

100

102

1999 2000 2001 2002 2003 2004

Output without quality;inputs: drugs deflatedby cost of all items;capital consumption,direct labour method

Output without quality;inputs: drugs deflatedby Paasche PriceIndex; capitalservices, indirectlabour method

(b) NHS productivity including quality change in NHS output but no allowance for increasing value of health, 1999 to 2004

95

96

97

98

99

100

101

102

1999 2000 2001 2002 2003 2004

Output with qualitybut not value ofhealth; inputs: drugsdeflated by cost of allitems; capitalconsumption, directlabour method

Output with qualitybut not value ofhealth; inputs: drugsdeflated by PaaschePrice Index; capitalservices, indirectlabour method

United Kingdom Index 1999=100



Source: Adapted from the Office for National Statistics, 2006

These estimates of productivity are further testedagainst wider corroborative evidence:

since 1991/92 the average length of stay in hospitalhas been falling steadily (apart from a small risebetween 1999/00 and 2000/01); and there has been asteady increase in the rate for elective day casetreatments. This suggests a shift towards more costeffective treatment and would be consistent with aproductivity increase from NHS resources. At the sametime, emergency readmission rates have increased veryslightly over the period. If this requires additional NHSresources, this could dampen down productivity (Officefor National Statistics, 2006b).

This process of checking productivity estimates againstother corroborative evidence is known as triangulation. Itis important in a context where It is unlikely that a singlenumber for productivity will ever capture all the costs and

24

(c) NHS productivity including quality change in NHS output and allowance for increasing value of health, 1999 to 2004

96

98

100

102

104

106

108

110

1999 2000 2001 2002 2003 2004

Output with quality andvalue of health; inputs:drugs deflated by costof all items; capitalconsumption, directlabour method

Output with quality andvalue of health; inputs:drugs deflated byPaasche Price Index;capital service, indirectlabour method



benefits of the NHS (Office for National Statistics, 2006b).

4.3.2 Education sector productivity measurementEducation is one of the services measured by the review ofgovernment services in Australia. The framework modeloutlined in section 4.2.4 is used to develop a set ofperformance indicators for schools, as set out in Figure 4.3.

Figure 4.3 Performance indicators for schools

Goals andObjectives

Performance

Equity

Effectiveness

Efficiency

Literacy - reading

Numeracy

Literacy - writing

Science

Information andcommunication

technology

Estimated completion

Destination

Civics and citizenship

Enterprise education

Other social outcomes

Other areas to beidentified

Access andequity

measures forparticipationand retention

Participation

Apparentretention

Studentlearning

Inputs peroutput unit

Governmentrecurrent

expenditure perstudent

Staffexpenditure per

student

Notional usercost of capitalper student

Student-to-staffratio

VET in schools

Equity and effectivenessmeasures for all outcomes

Outputs Outcomes


Source: Adapted from Banks, 2005Equity indicators measure how special needs groupscompare in terms of participation and retention rates.Effectiveness is measured in terms of learning outcomeswith regard to reading, writing and numeracy. Efficiency ismeasured in terms of government expenditure per student,staff expenditure per student, and student to staff ratios(Banks, 2005).

The different states are compared and contrasted interms of performance against the agreed indicators.Comparing the unit costs of providing a particular serviceacross jurisdictions is seen as a way of helping states toidentify if they have scope for improvements in theirefficiency.

4.3.3 Local government productivity measurementIn the UK, changes in local government performance areassessed using a sample of sixty-three indicators includingBest Value performance indicators, indicators from theSocial Services Performance Assessment Framework, andindicators from the Department for Education and Skills(Martin and Bovaird, 2005). This grouping of indicators isused by the Office of the Deputy Prime Minister as a basketof indicators showing the cost-effectiveness of localauthorities. The indicators are arranged by service area(see Annex 3).

The basket of indicators suggests that overallperformance has improved by 12.5 per cent between200/01 and 2003/04. There are significant variationsbetween authorities. There are also large variationsbetween services, with particularly large improvements inwaste management and culture. Martin and Bovaird (2005)suggest:

It seems the greatest improvements have been achievedin services where there has been a combination ofincreased funding, a strong focus on improvementtargets set at national level and scope for significant re-engineering of service delivery.

26


Applying the principle of triangulation noted in section4.3.1, Martin and Bovaird (2005) compare these resultswith other indicators of local government performance fromthe perspective of service providers and service users. Withregard to service provider perceptions, a survey of 1,500officers indicates that a large majority of respondentsbelieve that services have improved over a three-yearperiod. Eighty-four per cent believe that value for moneyhas improved.

Surveys of public perception, however, reveal a differentpicture. User satisfaction surveys conducted as part of theBest Value initiative indicate a significant decline in publicsatisfaction with overall local authority performancebetween 2001 and 2003. This information is confirmed byan analysis of other survey data, including MORI nationalsurveys. MORI surveys suggest that overall the proportionof respondents who believe their authorities provide goodvalue for money declined from 49 per cent to 37 per centbetween 1997 and 2002. Users of services consistently rateservices more highly than non-users.

When similar services are provided by entities such aslocal authorities, there is also scope for comparativeproductivity analysis (as in the case of Australian statesoutlined in section 4.2.4). For example, Haubrich,Gutierrez and McLean (2006) use an econometric analysistechnique called panel data analysis to try to identifyrelatively efficient and inefficient authorities.

4.4 ConclusionsThe evidence from national and sectoral studies of publicsector productivity measurement is that productivitymeasurement is still in its early stages. Despite effortsgoing back to the 1980s, the productivity measures beingproduced need to be interpreted cautiously. There is alsothe danger that over-simplistic use of the measures couldlead to perverse consequences. For example, the number ofvehicle kilometres is an output measure for public roads inSweden. Using this measure, it is possible to increase


productivity by increasing vehicle kilometres, but this islikely to run counter to transport policy aimed at movingpeople from cars to public transport and cutting down onunnecessary travel: the policy goal may be to reduce vehiclekilometres.

It is clear that no single figure of productivity can beused for public sector activities, unless there is clear andwidespread agreement that it is an appropriate measure.The Atkinson (2005) recommendation that a range ofsupporting information is needed to measure productivitychange the triangulation principle is one that should beapplied generally.

A further general point emerging from the casesexamined here is the lead role being taken by nationalstatistics offices in public sector productivity measurementinitiatives. The involvement of the national statistics officesis required because of the Eurostat directive on the need todevelop output measures for the national accounts,referenced in section 1.1. National statistics offices alsoplay a lead role in providing quality assurance andguarantees about data reliability and validity.

Where similar institutions are providing similarservices, it is possible to develop comparative productivitymeasurements, as the Australian states and localgovernment examples examined show. Using techniquessuch as frontier analysis it is possible to identify relativelyefficient and relatively inefficient organisations.5 The samecautions as to data reliability and interpretations as raisedabove, however, still apply.

28

5.1 IntroductionSo far, the productivity measures examined have beensectoral or national in nature, and often driven from a topdown perspective. It is important to note that, at a moremicro level, productivity measurement in the public sectorcan also take place at the level of the organisation and froma bottom up or service user perspective.

5.2 Measuring organisational productivity in DenmarkSince 2004, all ministerial departments in Denmark arerequired to draw up efficiency strategies. Performancecontracts are used to set targets and explicit requirementsfor productivity improvements are included. Previous yearsresults are reviewed in annual reports. The DanishImmigration Service and Statistics Denmark provide twoillustrative examples of productivity measurement (FinanceMinistry, 2005).

Up to 2004, the Danish Immigration Service applied ameasure of departmental productivity that assessed thenumber of cases dealt with per full time equivalent post.The measure comprised both the direct case specificresource usage and the total additional resource usage.From the performance contract of 2005 onwards, thismeasure has been refined to include only the direct casespecific resource usage, termed case processingproductivity. The case categories spontaneous asylumrequest and application for family reunion both show aconsiderable drop in estimated productivity between 2001and 2004. In other case categories, productivity isestimated as constant or increasing. Part of the drop inproductivity is explained by new legal requirements and achanged composition of applicants.

29

5

Organisation-level and bottom upproductivity measurement


Since 1997, Statistics Denmark has been developingproductivity measurements to assess its work. The numberof statistics produced is used as the output basis forproductivity measurements. Labour productivity isestimated to have grown by 3.1 per cent per annumbetween 2002 and 2004, with total factor productivitygrowing by 1.2 per cent per annum over the same period.The productivity measures do not take into account anumber of factors that can impact on productivity, such aschanges in quality or the content of a particular statisticover time. As a consequence, Statistics Denmark excludesfrom the productivity calculations statistics that areinfluenced significantly by such changes. From 2005, anew time recording system has been introduced, which isexpected to make it possible to correct the productivitymeasures for some of the factors that have an unintendedimpact.

5.3 Bottom up productivity measurementAs noted by de Walle (2005) with regard to good governanceperformance indicators: Another attempt at using objectiveindicators starts from a bottom-up approach: by collectingobjective performance indicators on specific services. Inthis case, aspects of performance are assessed from aservice user perspective, to see how efficiently a service isprovided.

The World Bank uses this approach in some contexts,in particular with regard to the assessment of the effects ofregulation. For example, the steps and costs associatedwith starting a business are assessed and the resultscompared over time and across countries. All genericprocedures that are officially required for an entrepreneurto start up and operate an industrial or commercialbusiness are recorded. These include obtaining allnecessary licenses and permits and completing anyrequired notifications, verifications or inscriptions withrelevant authorities. Local incorporation lawyers andgovernment officials complete and verify the data. The keyindicators used to assess performance are the number ofprocedures the applicant is required to go through, the

30

ORGANISATION-LEVEL AND BOTTOM UP PRODUCTIVITY MEASUREMENT 31

number of days each procedure takes, and the cost of startup (The World Bank Group, 2006). The results for Irelandfor 2005 are given in Table 5.1 as an illustrative example.

Table 5.1: Starting a business in Ireland

This table summarises the procedures and cost associatedwith setting up a business in Ireland

Source: The World Bank Group (2006)

Another example of this bottom up approach toproductivity measurement is a study of the institutionalperformance of regional governments in Italy undertaken byPutnam (1993). In this study, one of the indicators used toassess institutional performance, and the most relevantfrom the point of view of productivity, is bureaucraticresponsiveness. Bureaucracies in each region wereapproached with mail requests for information about threespecific (but fictitious) problems:

The health department was asked about reimbursementprocedures for a medical bill incurred while the inquirerwas on vacation abroad.

The vocational education department was asked aboutjob training facilities for a brother just finishing juniorhigh school.

The agriculture department was asked, on behalf of a

Nature of Procedure (2005)

Procedure Duration (days) US$ Cost

The founder swears before a Commissioner for Oaths File application with register Make a company seal Register for taxes and employment payments Totals:

1 2 3 4 4

1

15 1 7

24

5.63

1,781.52

22.82

0.00

$1,809.97


farmer friend, for information about loans andsubsidies for experimental crops.

Replies were evaluated for promptness, clarity and com-prehensiveness. If no reply was received, follow uptelephone calls and subsequent personal visits were made.Information was brought together in a composite index ofthe responsiveness of the three agencies examined,comparable across twenty regions.

Similarly, the Department of Enterprise, Trade andEmployment, as part of its work on customer care research,has conducted mystery shopping surveys where queries onaspects of the department's work are put over the phone tothe relevant division. Examples of the kind of questionasked are what are the maximum hours that people undereighteen are permitted to work? and how are annualholidays calculated? Both the timeliness and quality ofreply to these and other scenarios are assessed. It would bepossible to repeat such surveys and track changes overtime.

5.4 ConclusionsOrganisation level productivity measurement is likely to bea feasible and useful tool for those organisations that haveclear, identifiable outputs that can be linked to inputs used.These measures do not necessarily need to cover the wholeorganisation, and may be indicators of productivity fordiscrete parts of the organisation. There are manyorganisations, and parts of organisations, where thedevelopment of such measures is not feasible, particularlywith regard to policy work. But this does not precludedevelopments for the large parts of the public sector wheremeasurement development is possible.

The bottom up/service user measurements examinedhere are not productivity measurements in the strict sense,as they are focused on the outputs and broad performanceof public sector organisations rather than linking this datato inputs in a direct manner. However, as outlined insection 2.1, such measures do help provide a picture ofwhat value is being delivered by public services in return for

32

ORGANISATION-LEVEL AND BOTTOM UP PRODUCTIVITY MEASUREMENT 33

the expenditure supports provided. As such, they have apotentially important role to play in productivitymeasurement in its broad sense. Bottom up measures canalso be a helpful source of information to providetriangulation data for more conventional productivitystudies.

6.1 IntroductionThe measurement of public sector productivity is clearly achallenging task. While there is a diversity of internationalexperience to learn from, no simple solution to measuringpublic sector productivity has been found. In particular,the idea of deriving a single measure of productivity for thenation, a sector or an organisation is unrealistic. Anyproductivity measures developed need to be interpretedcautiously and combined with other information onperformance to give a fuller picture.

For statistical and national accounts purposes, theinput/output ratio should inform the development ofproductivity measures. But more generally, a broaddefinition of productivity should be used in determiningproductivity in the public sector. The focus should be onthe value received from the services provided throughpublic funding, including the outcomes achieved.

6.2 Creating a framework for developing publicsector productivity measurement in Ireland

Information on public sector productivity in Ireland islimited. In order to develop a broad range of information onproductivity and not rely on single data sources, aframework for the development of productivitymeasurement is outlined in Table 6.1. This frameworkproposes that action be taken at a number of levels cross-national, national and sectoral, and organisation-based andbottom up - to develop information on public sectorproductivity in Ireland. In this way, a diversity ofapproaches to productivity measurement can be used toprovide a broad picture of productivity developments. Theframework draws from lessons learned from theinternational experience as outlined in this study.

Conclusions and recommendations aframework for the development of

public sector productivitymeasurement in Ireland

6

34

CONCLUSIONS AND RECOMMENDATIONS 35

Table 6.1 A framework for the development of public sectorproductivity measurement in Ireland

Productivity initiative

Action required

Cross-national comparative studies

Track Irelands comparative performance

in periodic studies of public sector performance and efficiency such as the World Bank and European Central Bank studies.

Actively participate in and encourage the OECD Management in Government: Comparative Country Data initiative.

National and sectoral initiatives

The Central Statistics Office should take

a lead role in the development of the measurement of government output and subsequent productivity studies.

Annual output statements being developed by government departments should inform productivity studies.

The health and education sectors should be priorities for productivity studies.

Relevant state bodies and academic institutions should be encouraged to undertake research into public sector productivity measurement.

Benchmarking of comparable organisations should take place.

Organisation-based and bottom up initiatives

Organisation-based measures of

productivity should be developed, using annual output statements as a basis for this work.

Central agencies should sponsor a number of service user based studies of the efficiency of public service provision across a range of sectors, repeated periodically.

Benchmarking with comparable organisations should be encouraged.


6.2.1 Cross-national comparative studiesThere are a small number of cross-national studies ofpublic sector efficiency and performance. These studieshave methodological limitations, but nevertheless providesome basis for discussion on productivity. Ireland isincluded in these studies, which provide an opportunity forcontrasting Irish experience with that of other countries. Itis suggested that:

Ireland's comparative performance is tracked in studiessuch as the World Bank, European Central Bank andNetherlands Social and Cultural Planning Officestudies. Examination of common trends anddifferences across the studies may highlight issues forfurther attention.

The Irish government should actively participate in andencourage the OECD Management in Government:Comparative Country Data project. This projectprovides an opportunity to develop performance andproductivity measures which can be tracked over timeand across all OECD countries.

6.2.2 National and sectoral initiativesCross-national comparative studies, while of interest, arelikely to be restricted in the amount of information theyprovide on productivity. High levels of aggregation, anddifferences in national practices and definitions mean thatthey are of limited value. National and sectoral trends overtime provide a more robust foundation for productivitymeasurement. It is suggested that:

The Central Statistics Office should take a lead role inthe measurement of government output andsubsequent productivity studies. The Eurostat directive(see section 1.1) suggests a key role for nationalstatistics offices in public sector output measurement.Productivity studies are a natural follow on once outputmeasures are in place.

Annual output statements should be developed by

36


government departments and should informproductivity studies. In Budget 2006, the Minister forFinance announced that from 2007 individual ministersmust publish an annual statement on the outputs andobjectives of their departments, and from 2008 theactual outturns, for presentation to the relevantOireachtas committee. In this context, the Taoiseachhas indicated that he wishes to see aggregate indicatorsdeveloped that show the impact of total public spending(Ahern, 2006).

The health and education sectors should be prioritiesfor productivity studies. Health and education aremajor components of public expenditure. There are alsoseveral studies of productivity in the health andeducation sectors in other countries to draw on.

Relevant state bodies and academic institutions shouldbe encouraged to undertake research into public sectorproductivity. In particular, adjusting output figures toreflect quality changes is an important topic for detailedconsideration. Bodies such as the Economic and SocialResearch Institute and Forfs with a track record inproductivity and performance issues are well placed toundertake or coordinate such work.

Where institutions provide similar services (localgovernment, hospitals etc), benchmarking ofperformance should be encouraged. This is in line witha call to improve productivity in the public sector by deBuitlir (2006).

6.2.3 Organisation-based and bottom up initiativesSectoral, national and cross-national studies of productivityare important in providing a macro-level overview. But it isalso important that public sector productivity is assessed atan organisational level. Moreover, getting a service userperspective of public sector efficiency at the micro level canfurther our understanding of productivity in its broadersense. It is suggested that:


Organisation-based measures of productivity should bedeveloped. For government departments, suchorganisation level measures should link in with andmake use of the output statements to be produced from2007 as part of the reforms of the budgetary processoutlined in Budget 2006 (Department of Finance, 2005).

Central agencies should sponsor a number of serviceuser based studies of the efficiency of public serviceprovision across a range of sectors. These studiesshould be repeated periodically to assess change overtime.

As at the national and sectoral level, benchmarking ofperformance with comparable organisations has a roleto play. Organisations should be encouraged to identifyappropriate benchmark organisations when assessingtheir efficiency. The Taoiseach (Ahern, 2006) hasindicated that he wishes to examine how Irish publicservices perform relative to their international peers,identifying how we compare with those who arerecognised as representing good practice.

6.3 ConclusionThe productivity of the public sector is as important to theeconomic performance of a country as the productivity ofthe private sector. Thornhill (2006) identifies three mainreasons why public sector productivity is important. First,the public sector is a major employer. Second, the publicsector is a major provider of services in the economy,particularly business services (affecting costs of inputs) andsocial services (affecting labour quality). Third, the publicsector is a consumer of tax resources. Changes in publicsector productivity can have significant implications for theeconomy.

But measuring public sector productivity presentsmajor challenges. Until recently, the convention in nationalaccounts because of the measurement difficulties was toassume that outputs equalled inputs, and that thereforeyear on year there was no productivity change taking placein the public sector. Clearly this is not the case, and

38


attempts are now being made in several countries todevelop productivity measures based on government outputdata. This paper has examined some of the main initiativesin public sector productivity measurement that are takingplace internationally. On the basis of these developments,proposals are made to improve productivity measurementin the Irish public sector. There is a strong case fordevoting more attention and resources to improving themeasurement of public sector productivity.

Listing of sources and conceptsincluded in the World Bank

Government Effectiveness indicator

Annex 1

40

Representative

Sources

Columbia University

State Capacity

Survey

Global Insight

Concept Measured

Rate the administrative and technical skills of the countrys

civil service (occupying middle and higher management

roles).

Rate the efficiency of the countrys national bureaucracies

overall.

Rate the efficiency of the countrys local-level government

bureaucracies overall.

Rate the effectiveness of coordination between the central

government and local-level government organisations.

Rate the states ability to formulate and implement national

policy initiatives.

Rate the states effectiveness at collecting taxes or other

forms of government revenue.

Does the central government produce a national budget in

a timely manner?

Do local governments produce budgets in a timely manner?

Rate the states ability to monitor socioeconomic trends,

activities, and conditions within its borders.

Rate the states ability to create, deliver, and maintain vital

national infrastructure.

Rate the states ability to respond effectively to natural

disasters.

Government instability: An increase in government

personnel turnover rate at senior levels that reduces the

GDP growth rate by 2% during any 12-month period.

Government ineffectiveness: A decline in government

personnel quality at any level that reduces the GDP growth

rate by 1% during any 12-month period.

ANNEX 1 41

Economist

Intelligence Unit

World Economic

Forum

Merchant

International

Group

Political Risk

Services

Institutional failure: A deterioration of government capacity to

cope with national problems as a result of institutional

rigidity that reduces the GDP growth rate by 1% during any

12-month period.

Quality of bureaucracy

Excessive bureaucracy / red tape

Public spending composition

Quality of general infrastructure

Quality of public schools

Time spent by senior management dealing with government

officials

Bureaucracy. The critical feature of bureaucracy is that it

raises issues more complicated than red tape alone.

Bureaucracy can be actively and deliberately obstructive to

foreign investors in response to political pressures, vested

interests and special interest lobbies. Some features that

determine the extent that bureaucracy could affect business

operations are the accountabilty of public officials;

politicisation of bureaucratic departments; regulatory

credibility and enforceability; size of the public sector and

transparency of decision making.

Government Stability. Measures the governments ability to

carry out its declared programmes, and its ability to stay in

office. This will depend on issues such as: the type of

governance, the cohesion of the government and governing

party or parties, the closeness of the next election, the

government's command of the legislature, and popular

approval of government policies.

Bureaucratic Quality. Measures institutional strength and

quality of the civil service, assesses how much strength and

expertise bureaucrats have and how able they are to manage

political alternations without drastic interruptions in

government services, or policy changes. Good performers

have somewhat autonomous bureaucracies, free from

political pressures, and an established mechanism for

recruitment and training.

ANNEX 142

World Markets

Online

Non-representative

Sources

African

Development Bank

Afrobarometer

United Nations

Economic

Commission for

Africa

Asian Development

Bank

Business

Environment and

Enterprise

Policy consistency and forward planning: How confident

businesses can be of the continuity of economic policy stance

whether a change of government will entail major policy

disruption, and whether the current government has pursued

a coherent strategy. This factor also looks at the extent to

which policy making is far-sighted, or conversely aimed at

short-term economic (and electoral) advantage.

Bureaucracy: An assessment of the quality of the countrys

bureaucracy. The better the bureaucracy the quicker

decisions are made and the more easily foreign investors can

go about their business.

Concept Measured

Management of public debt

Policies to improve efficiency of public sector

Revenue mobilisation

Budget management

What proportion of the countrys problems do you think the

government can solve?

Based on your experiences, how easy or difficult is it to obtain

household services (like piped water, electricity or telephone)?

Based on your experiences, how easy or difficult is it to obtain

an identity document (such as birth certificate, drivers

licence or passport)?

Executives effectiveness

Effectiveness of state structureGovernment services efficiency

Decentralisation of structures

Economic management

Civil service

Revenue mobilisation and budget management

Management and efficiency of public expenditures

How problematic are telecommunications for the growth of

your business?

How problematic is electricity for the growth of your

ANNEX 1 43

Performance

Survey

Business

Environment Risk

Intelligence

Bertelsmann

Foundation

Country Policy

and Institutional

Assessment

Global E-

Government

Freedom House

Latinobarometro

Institute for

Management

Development

business?

How problematic is transportation for the growth of your

business?

Bureaucratic delays

Consensus building

Governance capability

Effective use of resources

Reliable pursuit of goals

Welfare regime

Management of external debt

Management of development programs

Quality public administration

Revenue mobilisation

Budget management

Global E-government

Government and Administration: Government decentralisation,

independence and responsibilities of local and regional

governments, and legislative and executive transparency are

discussed

Trust in government

Government economic policies do not adapt quickly to

changes in the economy

The public service is not independent from political

interference

Government decisions are not effectively implemented

Bureaucracy hinders business activity

The distribution infrastructure of goods and services is

generally inefficient

Policy direction is not consistent

Source: Kaufmann. Kraay and Mastruzzi, 2005

Consumer Ombudsmans Office number of petitions to market court marketing instructions (number of) contractual terms negotiated statements on legislative initiatives cases solved individually replies to written inquiries

Courts such as The Supreme Court, Courts of Appeal, DistrictCourts, Provincial Courts and Supreme Administration Court number of cases settled

Helsinki City Police Department the output indicators of public order and security, such as

activities directed toward the protection of property and theindividual

crime prevention measured by the number of crimes solved the final products of traffic safety number of permit documents issued (number of passports,

identity cards, driving licences and firearms licences)

Housing Fund of Finland decisions about loans and interest subsidies measured as the

weighted number of decisions

National Board of Patents and Registration number of patents number of utility models number of trademarks number of pattern rights company register cases association register cases enterprise mortgage cases

Examples of output indicators in thecentral government in Finland

Annex 2

44

ANNEX 2 45

Source: Niemi, 1998

National Food Administration number of letters guiding supervision number of administrative decisions and memos number of publications number of statements number of training events new instruction materials

Prison system prisoner-days

Prosecutors Offices and Distraint Offices number of cases dealt with

State Audit Office number of annual audits supplementary audits international audits expertise activities, statements

Tax Administration numbers of private persons, agricultural entrepreneurs and

entrepreneurs and corporations subject to income andproperty tax

number of supervised registered employers numbers of primary producers and entrepreneurs subject to

value added tax the output indicator of real estate tax

Universities number of degrees completed (generally separated into

graduate and postgraduate degrees) adult education and continuing education measured, for

example, in days or number of courses (depending on theuniversity)

number of publications (research)

Performance as measured by the ODPM basket against a base of 100 in

2000/2001

Source: Martin and Bovaird, 2005

The UK Office of the Deputy PrimeMinister basket of cost effectiveness

indicators

Annex 3

46

2001/02 2002/03 2003/04

Performance Deflated by

expenditure


expenditure


expenditure

Primary education

Secondary education

Childrens social services

Adults social services

Housing

Benefits

Waste management

Transport

Planning

Culture

Community safety

All services

100.95

103.21

106.34

105.87

104.01

104.03

105.74

111.09

100.58

104.76

96.3

103.9

96.6

98.84

101.08

99.37

104.5

100.2

104.99

102.9

92.37

96.91

100.23

96.6

102.49

107.23

113.05

110.95

108.25

112.57

122.99

112.16

101.2

122.43

95.83

108.14

94.08

98.76

101.96

97.12

108.01

105.39

120.91

96.91

84.31

106.59

99.58

94.08

103.05

109.53

118.33

117.38

113.04

117.99

153.86

117.06

105.45

125.26

104.64

112.54

90.29

98.11

103.37

94.67

111.83

108.36

131.05

95.91

74.48

108.07

97.44

90.29

1 The Eurostat Price and Volume Handbook (2001) identifiesfour potential characteristics of output indicators:

1. Full coverage of all services provided to external users

2. Cost-weighted

3. Defined in as much detail as possible

4. Adjusted for qualityIt also identifies A, B, and C methods for individual services:

A methods: output indicators satisfying all four criteria B methods: output indicators not satisfying all four

criteria

C methods: methods based primarily on measuring inputs

2 Commission decision 2002/990 outlaws the use of C methodsreferred to in note 1 from 2006.

3 It should be noted that the main interest in this chapter is inthe approach taken to measuring performance andproductivity rather than the findings from the studies (thoughreference is made to the findings in some instances). Readersinterested in the findings are referred to the source documentsreferenced in the text.

4 The Atkinson review also suggests that this adjustment beused and interpreted cautiously pending further debate.

5 Frontier analysis is a statistical performance emasurementtechnique used for evaluating the relative efficiency of unitssurveye

measuring public sector productivity prelims - ipa · measuring public sector productivity: lessons...

Documents