caroline van den berg and sasha danilenko water...

21
1 Caroline van den Berg and Sasha Danilenko Water week, February 28, 2007 Baseline and Target values: What have we learned so far?

Upload: others

Post on 17-Apr-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

1

Caroline van den Berg and Sasha Danilenko

Water week,February 28, 2007

Baseline and Target values:

What have we learned so far?

2

Introduction

Guidelines on Results Measurement in Bank projects

To include baseline values for project outcomes by first ISR To determine target values for outcomes/results

What have we learned so far?

3

Baseline values for (intermediate) outcomes

Households have better access to sustainable improved water supply and/or sanitation services (household perspective)

Households have better access to improved WSS services (by income quintiles)Increase in average water consumption by residential hh% hh that have adopted good hygiene behavior (e.g. handwashing) at key times

Service providers are providing WSS services more efficiently (service provider/utility perspective)Reduction in pollution levels (e.g. BOD levels) in water at all monitoring stations downstream (water quality perspective)

4

Key Informant Interviews

Community Interviews Review of

administrative records

One-Time Survey

Impact Evaluation (Experimental Design)

CensusRigor/Difficulty

(Cost/Time/skills)

Focus GroupInterviews

Formal MethodsRapid Appraisal Methods

Direct Observation

Validity Reliability Credibility

Choice depends

1. Issue to be examined

2. Quality of information needed

3. Timeframe in which information is needed

4. Costs

Panel Surveys

5

Some thoughts on baseline data

Use of baseline data is directly linked to project design

What to do?Where to intervene?

Retrofitting of baseline values is a complex matter at best“Think outside of the box”

There is often more data available than you think:It is most likely not available in the MOW

6

Target values

Baseline indicator +

desired level of improvement

= target performance

0

10

20

30

40

50

60

70

80

90

100

Yr0 Yr1 Yr2 Yr3 Yr4 Yr5 Yr6

7

When setting target valuesBalancing between easy to achieve and impossible to realize: reality check on indicatorsKey factors influencing setting target values for basic indicators

Expected funding and resource levels – existing organizational capacity, personnel, funding, facilitiesPast performance to determine the realism of targetExternal environment – the economic and political environmentThe relationships between the different indicatorsThe type of indicators selected

8

Illustration of target values

Looking at indicators, baseline values and how to set target values Focus on

improving the efficiency which utilities provide WSS servicesImproving quality of service

Data from International Benchmarking Network (IBNET) at www.ib-net.org

9

IBNET website

www.ib-net.org

10

Illustration 1: past performance

Source: IBNET

Moldova: staff ratio by 1,000 people servedcompared over time and with region

0

2

4

6

8

10

12

Quartile 1(best) Quartile 2 Quartile 3 Quartile 4 (w orst)

1999 2004 ECA low -income 2004

11

Illustration 2: political context

Rezina, Moldova: hours of supply per day

0

4

8

12

16

20

24

1997 1998 1999 2000 2001 2002 2003 2004 2005

Source: IBNET

12

Illustration 3A: the impact of reform

Impact of reform in Moldova: consumption and metering

0

50

100

150

200

250

300

1996 2001 2004 2005

lcd

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

met

erin

g ra

tio

3_1_TotalWaterProdLCD 4_1_TotalWaterConLCD 7_1_Metering

Source: IBNET

13

Illustration 3B: the impact of reform

Moldova: consumption and price

0

50

100

150

200

250

300

1996 2001 2004 2005

lcd

0.00

0.20

0.40

0.60

0.80

1.00

1.20

US$/

m3

Total water production Total water consumption Billed tariff (US$/m3)

Source: IBNET

14

Illustration 3C: the impact of reform

Moldova: Staff ratio per 1,000 population

2.40

2.50

2.60

2.70

2.80

2.90

3.00

1997 1998 1999 2000 2001 2002 2003 2004 2005

Staf

f rat

io

Source: IBNET

15

Illustration 3D: the impact of reform

Moldova - Operating Cost Coverage Ratio

-0.20

0.400.600.801.00

1.201.401.601.80

2.002.20

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Source: IBNET

16

Illustration 3E: the impact of reform

Moldova: subsidies to consumers through underpricing of services (in US$ million) and the

operating cost coverage ratio

0

5

10

15

20

25

30

2000 2001 2002 2003 2004 2005

US$

mill

ion

0.000.200.400.600.801.001.201.401.60

Ope

ratin

g co

st

cove

rage

ratio

Subsidies Operating cost coverage ratio

Source: IBNET

17

Illustration 4A: indicator selection

Brazil: operating cost coverage ratio

-0.20

0.400.600.80

1.001.20

1.401.601.80

2.002.20

1996 1997 1998 1999 2000 2001 2002 2003 2004

Source: IBNET

18

Illustration 4B: indicator selection

Moldova: non-revenue water

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

NR

W in

%

0

10

20

30

40

50

60

70

NR

W in

m3/

km/d

ay

6,1 Non Revenue Water, % 6,2 Non Revenue Water, m3/km/ day

Source: IBNET

19

Some thoughts on target values

Setting targets without baseline data is difficult at bestThe indicators that are selected will be affected by the different interactions that take placeThe choice of indicators will affect the likelihood of achieving targets

It is more difficult to predict how composite indices are likely to behaveVolatility in indices can be smoothened out with e.g. moving averages

20

Some thoughts on target values

Target values are also be affected by events that are not directly linked by the project

External factors can have a big impact in the short run,And in the long run

21

Conclusions

Setting baseline and target values is an integral part of project design

How to design a project without baseline values? Retrofitting is not the way to go

There are many different options to measureDifferent indicators can measure similar-like project outcomesUnderstanding of past performance is a pre-requisite for understanding how to set targets