Monitoring Performance or Performing Monitoring? Lessons on

the politics of monitoring

Monitoring for sustainable WASH services Symposium

Addis Ababa, 10th April 2013Katharina Welle, PhD candidate, University of Sussex

WHY FOCUS ON PERFORMANCE MONITORING?

What is Performance Monitoring (PM)?

Feedback

Objectives Activities Outputs Outcomes ImpactsInputs

Development Results

Source: OECD-DAC (2010)

OECD-DAC Results Chain

According to the OECD-DAC Glossary of Key Terms in Development Evaluation and Results-based Management (2010), performance monitoring refers to a continuous process of collecting and analysing data to compare how well a project, programme or policy is being implemented against expected results”.

Common assumptions on the linear relation between monitoring and decision making

The purpose of monitoring is “to track progress against given objectives” and “to inform decisions, focus and orient political and policy reforms, and to channel financial resources in the most effective way” (UN Water, 2006: 9)

Common criticisms of Performance Monitoring

• Misrepresents complex reality by reducing it to few, measurable results, expressed in quantitative indicators

• Quantification suggests a mechanical objectivity: sanctioned methods to produce ‘presumably neutral facts’

• Can create ‘perverse incentives’ and lead to ‘gaming behaviour’

Frederick Taylor (1856 – 1915)

Monitoring performance or performing monitoring? The importance of actors’ framings

”narratives … employing unifying metaphors …that tell what needs fixing and how it might be fixed ….” Rein and Schön (1996: 89)

INSIGHTS ON THE POLITICS OF PERFORMANCE MONITORING

Rural water access trends in the Southern Nations, Nationalities and Peoples (SNNP) Region

Official figures of water access in Ethiopia’s Southern Region from 2000 -7

0

10

20

30

40

50

60

70The Southern Region in Ethiopia

Source: Bureau of Water Resources 2009

ACTORS’ FRAMINGS OF ACCESS AND WHICH ONES GET ELEVATED IN PM

Feedback

Objectives Activities Outputs Outcomes ImpactsInputs

Development Results

Parameters for defining access to rural water in Ethiopia

Definitions of access

JointMonitoring Programme of WHO and UNICEF

EthiopianMinistry of Water and Energy

Volume 20 litres / person /day

15 litres / person / day

Distance 1 km 1.5 km

Access calculation (MoWR2009/10)

Handdug well

Protect-ed spring

Shallow well

Deep well

Averagebeneficiaries

270 338 457 3,313

‘Access’ results for my case study kebeleScheme Average

beneficiaries

3 Hand dug wells

810 (3 x 270)

1 Bore hole

3,313

Total 4,123

Scheme

Kebelepopulation

% served

CSA 5,885 70%

HEW data

~4,700 94%

Kebele residents’ issues with access

BH

Kebeleboundary

Water point

Public Tap

Public Tap

HDW1Church

Priest

HDW2

School

TeachersHP

HDW3

MONITORING DYNAMICS

Feedback

Objectives Activities Outputs Outcomes ImpactsInputs

Development Results

Using different calculation methods to stage ‘performance’ in review meetings

Regional Bureau of Water Resources

Calculation method used: average user estimates by scheme type

Rationale: reporting positive access trends at federal level

Result for case study woreda: Access: 58% in June 2009

Woreda (district) Water, Mining and Energy offices

Calculation method used: users living within 1.5 km of source

Rationale: obtaining budgets for water supply at the local level

Result for case study woreda: Access: 38.1% in December

2009

FACTORS AFFECTING RURAL WATER ACCESS DECISIONS

Feedback

Objectives Activities Outputs Outcomes ImpactsInputs

Development Results

Decisions on capital budget allocations for rural water supply in SNNP Region, Ethiopia, 2009/10

Who Factors affecting allocations

Channel 1a

Government block grants

Multi-sectoral ‘new budget grant formula’

Channel 1b

Sector donors (WB, UNICEF, etc)

Specific intervention woredas (80 out of 134 in 2009)

Channel 2

Multi-sector programmes (e.g.PSNP)

In case of PSNP(Productive Safety-Net Programme ), to food-insecure woredas, capital allocations to sectors depend on various factors

Channel 3

NGOs No direct control by government, many factors affect project sites of NGOs

Realities of “strategic planning” at woreda level

Financing modality Woreda blockgrant

WASH Programme PSNP NGO

Financing channel 1a 1b 2 3

Funding period 2006-2010 2005-2012 2007-2011 2007-2011

Schemes constructed until 2009/10

0 2 hand-dug wells2 shallow wells10 protected springs

8 protected springs

9 deep wells5 shallow wells

Construction costestimates based on MoWR (2005)

Not applicable ~600,000 ETB ~300,000 ETB ~7,000,000 ETB

Sources: World Vision project document, MoFED Water Supply and Sanitation fund utilisation sheet, MoFED PSNP Fund Utilisation Sheets & report prepared by officer for me

Borehole in case study kebele

Neighbouring kebele’s Tap

Multi-causality of factors affecting the repair

Inhibiting the repair

Facilitating the repair

Inadequate sector support to and

oversight of WASH committee

Cost implications of major maintenance

not coherently addressed

Government logistical and

capacity constraints

Climate of distrust and indications of

abuse of roles among water user

committee members

Local opinion leader pushed

repair

Drought in 2008 amplified water access problems of

local residents

Approach of general elections creates ‘window of opportunity’

Regional inventory results in decision to

address non-functionality

Woreda water office mobilised budget sources for new

reservoirs

Woreda water office reported

scheme breakdown

Implications for performance monitoring

PM is not objective but subject to actors’ framings – in performance monitoring, some factors are elevated while others remain neglected there is a need to bring perspectives from local users higher up on the

agenda

In addition to official monitoring objectives, people involved may have their own, private agendas, that influence the process and results It is important to pay attention to the practice of every day monitoring in addition

to redesigning global targets and indicators

Monitoring results are just one among many factors affecting service delivery decisionsCan be used as a tool to legitimise actors rather than really increasing

accountability

Thank youBelow are additional slides that I might use