evaluating community-demand driven water & sanitation

[email protected]

[email protected]

[email protected]

Evaluating Community-Demand Driven Water & Sanitation Approaches

in Rural Maharashtra

…. a story of taps, toilets, and behaviors

Priti Kumar 1 and J Murty 2

S. K. Pattanayak 3, Sumeet Patil 4, K. Lvovsky 5, N. Raghava 6, J-C Yang 7, C. Poulos 7

1, 2, 5, 6 The World Bank

3 Duke University, USA (formerly RTI International)

4 Neerman, Mumbai (formerly RTI International)

7 RTI International, USA

2

Overview of Presentation

Context

Study Objectives

Study Methodology and Implementation Details

Results

Lessons Learnt

3

Context for Study

Child Diarrhea - key underlying link for India’s MDG

targets

11th 5yr Plan targets on WSS are ambitious, exceed the MDG goals,

and require multisectoral approaches

Significant success and scaling up RWSS reforms– (e.g.,

Jalswarajya, TSC)

Variety of settings, delivery mechanisms, and interventions

Focus on creating demand at the community level for increasing use of IHL; Incentives to the poor.

Need to understand how to improve health and non-health benefits of WSS programs

Limited knowledge in this area. Households behaviors or adaptation

modify the impacts, but very little is known

4

Context for Study

Importance of Water and Sanitation for health is not disputed. BUT…

Major knowledge gaps. Scientific literature is thin and inconclusive Limited knowledge on effectiveness of EH interventions (e.g.,

private/public taps, water quality treatment, long term behavior change etc.)

Until now more emphasis on clinical interventions (ORS, vaccines) for diarrhea reduction

Past evaluations do not meet quality standards Cross-sectional assessments (lacking baseline, control, statistical

power)

Insufficient designs for measuring broad range of program impacts (e.g.: equity)

5

Study Objectives

First scientific study of this kind in India to examine the

benefits of community demand driven approaches in RWSS

programs that are supported by GoI / GoM policies

Address question in a scientifically rigorous manner:

Do participatory water and sanitation projects deliver health, and socio-economic outcomes? (Maharashtra)

What factors affect / modify these impacts?

Contribute to global knowledge on health and WSS.

6

Specific Objectives

Determine whether WSH interventions due to Jalswarajaya Project cause differences in water, sanitation and health outcomes in villages that participate in the Project

Measure broader impacts of WSH interventions on rural livelihoods (e.g.: time savings, investments in coping activities, improvements in convenience/privacy and indirect benefits to caregivers)

Develop proxy indicators for monitoring that are correlated to health impact (water quality at point of use/source, retention of hygiene knowledge, use of latrines, condition of public stand posts, etc.)

Learn lessons for future implementation and evaluation

7

Why undertake the study in Maharashtra?

The Jalswarajaya Project was an ongoing Bank operation under implementation, with the schedule most suitable for the study.

A large RWSS intervention with a comprehensive range of WSS service packages, good geographical coverage.

Strong and committed government counterpart in GoM.

8

Study Team Partners

Rajiv Gandhi Drinking Water Mission, Govt of India –

Department of Water Supply and Sanitation, Govt of Maharashtra –

Multidisciplinary technical advisory group – WHO, USAID, UNICEF, Indian Council of Medical Research, National Institute of Epidemiology, CMC Vellore in GoI (reviewed/advised during initial phases of study design and development of study protocols)

RTI International (USA) – Principal investigating agency– study design, management, analysis and reporting.

TNS Mode (India) –field and data collection support

9

Jalswarajya – water self-rule (Maharashtra)

10

Interventions under Jalswarajya

Philosophy

community demand driven (Panchaayati Raj institutions)

decentralized delivery

cost sharing (10% of K, and 100% of O&M)

resources for capacity building and local institutional strengthening

2800 villages from 26 districts (400 households per village)

Pilot:30 villages from 3 districts

Phase I: Batch 1 - 225 villages from 9 districts

Phases I (Batch 2), and Phase II (17 additional districts)

Timeline

pre-planning – 1 month (IEC, community mobilization - VWSC)

planning – 6 months (VAP, appraisal and sanction)

implementation – 9 months

operation & maintenance – on-going

Process: inform, apply, select, plan, finance, construct, monitor

11

Study Design

Not a controlled experiment but evaluation of a real life program

Cannot randomly select treatments and control

Cannot dictate selection and timeline of the project. Evaluation had to be non-intrusive

Jalswarajya in Maharashtra: Quasi-experimental (matched) sample of 242 villages

Treatment: 95 from Phase 1, Batch 1 from 4 dry districts

Control: 147 matched controls from the above districts.

Treatment and controlled had similar probability of being selected in Jalswarajya Project

District selection Focus on rural, dry, and drought-prone (exclude coastal and urban

districts) and 4 geographic regions of Maharasthra – Buldana, Nashik, Osmanabad, Sangli.

Total: ~10,000 households

12

Study Design

Baseline (May & Aug 2005) and Endline (May & Aug 2007) data collection for intervention and control villages

Select controls using a matching technique to estimate counterfactual outcome and removes bias (Propensity Score Matching)

Difference in difference (before/after and with/without) estimation of impacts : Estimates population average treatment effect. Advantageous and robust compared to:

Only Before-After: Cannot demonstrate specific success of the program without controls

Only With-Without: Cannot demonstrate “progress” from a baseline. Cannot deal with different starting points.

Multiple observations from the same cluster (community) – ~50 HHs per village (randomly select from list and map all U5 hhs)

13

Data: Measurements

Outcomes:

Household pit latrines (IHL): constructed, operational and in-use

Diarrhea frequency & severity (> 3 episodes in 24-hr, 2-week recall)

Arm circumference

Additional parameters:

Individual - sex, age, education, anthropometric, health

Household - family size and composition, housing conditions, asset holdings, occupation and expenditures, SEC, services etc

Community – roads, electricity, environmental sanitation, employment, clinics, schools, credits, markets

Institutional - main governmental and NGO programs, local government size and composition

Water quality (E. coli & total coliform) – lab tested, spread plate method

14

Household Survey

Respondent - Primary Care Giver for U5 child

Water samples collected from approx 50% of surveyed households

Modular questionnaire

Knowledge, Attitudes

Household demographics

Sanitation Behaviors – outputs and outcomes

Hygiene Behaviors

Water Sources and their use

Water Treatment/safety behaviors

Food safety behaviors

Environmental conditions – HH and community

SEC & Budget constraints

Community Participation

15

Community Survey

Approx 150 minutes. Respondent – sarpanch, GP member, Informal leader, Doctor, etc

Water samples collected from up to 10 in-use drinking water sources

Modular questionnaire design

Background: population, households, area, arable land, major crop grown

Public infrastructure: roads, water supply, sanitation, hygiene, electricity, clinics, schools, STD booths, telegraph offices, post offices, credits and markets

Environmental sanitation: general cleanliness, drainage, animal and household waste, use of water sources, open defecation practices

WSS scheme: Jalswarajya, VWSC, NGO/SO, scheme details etc

Development Programs: Health, education, women support etc

Economy: employment opportunities, major governmental and NGO programs, prices

Local government: structure, composition, activities

16

Results-1: Outputs

Access to water supply and sanitation

increased in two years across programmes

Clear indication that investments are

reaching villages

17

% Households Using Taps

(by intervention, season, and year)

I indicates the 95% confidence interval.

35%

45%41%

44% 42%

51%

40%

47%

0%

10%

20%

30%

40%

50%

60%

2005 2007 2005 2007 2005 2007 2005 2007

Jalswarajya Villages Control Villages Jalswarajya Villages Control Villages

Dry Season Rainy Season

DID = 13%*** DID = ~

18

% Households Using Toilets



16%

34%

12%

22%19%

34%

14%

21%

10%

15%

20%

25%

30%

35%

40%

2005 2007 2005 2007 2005 2007 2005 2007



DID= 8*** DID= 6***

19

Results-2: Outcomes-Water

30% reduction in time spent in walking to and waiting at water source

(Access) across the board

Dry Season: 34 minutes in 2005 dropped to 22 minutes in 2007

Rainy Season: 23 minutes in 2005 dropped to 16 minutes in 2007

Across the board increase in LPCD consumption by households (Approx

30% jump)

10 to 11 LPCD increase from baseline levels depending upon season and

type of villages. (baseline levels: 27-30 LPCD increased to 38-40 LPCD)

Coping costs (water collection, treatment and storage, walking to

defecation site) (40% reduction)

Dry Season: Reduced from Rs 976 in 2005 to Rs. 590 in 2007

Rainy Season: Reduced from Rs 857 to Rs. 490 in 2007

20

Monthly coping costs of inadequate WSS (in Rs.)



DID= -202* DID= ~

926

605

836

458

869

513562

1064

0

200

400

600

800

1000

1200

2005 2007 2005 2007 2005 2007 2005 2007



21

Household water consumption in LPCD



30

41

31

39

27

38

27

38

25

30

35

40

45

2005 2007 2005 2007 2005 2007 2005 2007



DID = 4.5** DID = ~

22

Results-3: Outcomes on Behavior

Change

Not much impact seen on behavior change

Measured indicators are:

Water treatment

Safe water handling at hh level

Hand Washing

23

# of Critical Times a Child Washes Hands



1.1

1.3

1.2

1.3

1.7

1.2

1.7

1.2

0.0

0.3

0.6

0.9

1.2

1.5

1.8

2005 2007 2005 2007 2005 2007 2005 2007



DID = ~ DID = ~

24

% Households Treated Water before Drinking



63%68%

63%67%

75%69%

76%68%

0%

20%

40%

60%

80%

100%

2005 2007 2005 2007 2005 2007 2005 2007



DID = ~ DID = ~

25

# of Safe Water Handling Techniques HH Practiced



1.41.3

1.41.3 1.2 1.2 1.2

1.1

0.0

0.4

0.8

1.2

1.6

2005 2007 2005 2007 2005 2007 2005 2007



DID = ~ DID = 0.1*

26

E. Coli in Household Drinking Water

(by intervention, season, and year in log10)


0.6

0.8

0.5

0.8

1.1

0.7

1.2

0.7

0.0

0.3

0.6

0.9

1.2

1.5

2005 2007 2005 2007 2005 2007 2005 2007



DID = ~ DID = 0.23*

27

Diarrhea Prevalence among Children under 5



11%

9%10%

9%

12%

8%

13%

7%

0%

3%

6%

9%

12%

15%

2005 2007 2005 2007 2005 2007 2005 2007



DID = ~ DID = 2%*

28

Finer Analysis

Two types of finer analysis were carried out

Impacts due to higher coverage

Impact due to Jalswarajaya approach

29

Subgroup Analysis

1. Group 1: 80 control villages with >50% Taps, Toilets or Both

2. Group 2: 21 JS villages <50% taps or toilets.

3. Group 3: 10 JS Villages with >50% of HH with toilets but <50% using taps

4. Group 4: 31 JS villages >50% of HH using taps, but <50% using toilets

5. Group 5: 17 JS villages >50% of HHs using taps and toilets.

Control: 83 villages with <50% using taps and toilets

30

Subgroup Analysis Continued…

In dry season:

Group 5 have less e coli in their HH water

Lower COI and diarrhea in group 5

In rainy season:

Coping costs decrease in group 5 (more accessibility)

in home WQ worse in 1, 3 and 4. In group 1 and 3

WQ may worsen may be because WS is not

adequate enough to support suitable sanitation

Group 3 has higher cost of illness, diarrhea rates and

smaller arm circumference .. maybe because of

decline in water quality

Group 1 (80 non JS):

>50% Taps and/or

Toilets

Group 2 (21 JS):

<50% taps or toilets.

Group 3 (10 JS):

>50% toilets but <50%

taps

Group 4 (31 JS):

>50% taps but <50%

Group 5 (17 JS):

>50% taps and toilets

Control (83 non JS):

<50% taps and toilets

31

Jalswarajya Impact

Supply Side Support (definitely more structured than non-JS)

Jalswarajya villages have 21% higher likelihood of a proper water

supply scheme being implemented

Contribution and collaboration

JS villages were 50% more likely to contribute to the WSS

scheme (implication for sustainability and O&M)

JS villages were 25% more likely to have an active VWSC

(implication for capacity building, ownership, accountability and

sustainability)

Positive outlook for sustainable approaches…

32

Jalswarajya Impact

Compare BPL and SC/ST in Jalswarajya villages with those in

non-JS villages

less coping costs (water collection, walking to defecation site)

(Rs 200 in rainy and ~Rs 2000 in dry)

WQ better in dry season but worse in rainy season

Higher reported use of taps in dry season (weak but significant

evidence)

Differences not significant in APL and open caste HHs

targeting of “poor” OR interplay between program and “poor”

is happening

33

Lessons Learnt: Policy Implication

Promote both taps and toilets in higher number to see significant health impacts (integration of programs, coverage etc)

LG and community capacity building (like in JS) for sustainability, across programs

Improved IEC/ awareness for behavior change (very crucial)

Continuous and timely water quality management approaches

M&E systems to include indicators linked with impacts/outcomes (actual use, water quality, etc)

Lessons For Future IE…1

34

Early and sustained dialogue with counterpart and stakeholders is very important

Need a strong counterpart throughout the IE exercise..if not, spend time in building their interest .. and knowledge..

Do not outsource training of enumerators to external agencies. Should be the primary responsibility of PI and Co-PI, intensify efforts by reviewing early results of training

Simplify the study design, survey instruments and keep the sample size manageable

Study design should allow for mid-course corrections

Lessons For Future IE…2

35

Baseline assessments should be done very early on for a true picture’

Logistics related to sampling and data collection should be manageable - in this case WQ sample collection, transportation has become highly resource and time intensive

There needs to be a balance between the research cycle and time required to provide results to counterparts to influence sector programs/ polices.

evaluating community-demand driven water & sanitation

Documents