Cereal Systems Initiative for South Asia (CSISA)

Monitoring & Evaluation

Alwin Keil

Africa RISING–CSISA Joint Monitoring and Evaluation Meeting, Addis Ababa, Ethiopia, 11-13 November 2013

New varieties, improved management practices

Targeting farmers through trials, demonstrations, training

Building partnerships for research and technology delivery

Creating enabling environment (identifying change agents, linking

markets, service providers, addressing policies)

Increase food and income

security

Conserve natural & human resources

Reach 2 million farmers

Wide geographical coverage

Various objectives, institutions,technologies,partners and stakeholders

Work with male and female farmers

COND.MEANSGOAL

Key M&E Challenges

• Multiple data types (agronomic, socioeconomic, activity tracking, quantitative & qualitative…)

• Multiple data sources (by objective, country, institution)

• Multiple outputs and audiences (narrative reports, indicator reports, evaluations…)

• Large scale (and rapidly increasing!), but small M&E team

Monitoring

1. Number of individuals who have received USG supported short-term agricultural sector productivity or food security training

How to compile: Hubs report training data

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

2. Number of food security private enterprises (for profit), producers organizations, water users associations, women's groups, trade and business associations, and community-based organizations (CBOs) receiving USG assistance

How to compile: Hubs report their collaborators

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

3. Number of public-private partnerships formed as a result of FTF assistance

How to compile: Hubs report the government –private sector collaborations formed as a result of CSISA intervention

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

4. Number of policies/regulations/administrative procedures in each of the following stages of development as a result of USG assistance:

Stage 1 - Analyzed Stage 2 - Drafted and presented for consultation Phase 3 - Presented for legislation Phase 4 - Passed Phase 5 - Being implemented

Information compiled and submitted by IFPRI

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

5. Number of new technologies or management practices in one of the following phases of development:

Phase 1 - Under research Phase 2 - Under field testing Phase 3 - Available for transfer

How to compile: Hubs identify “CSISA technologies,” categorized by phase(s) of development.

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

6. Number of hectares under improved technologies or management practices as a result of USG assistance

How to compile: For a hub’s list of “CSISA technologies”, total number of hectares is calculated.

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

7. Number of farmers and others who have applied new technologies or management practices as a result of USG assistance

How to compile: For a hub’s list of “CSISA technologies”, number of adopting farmers is calculated.

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

8. Number of stakeholders with increased adaptive capacity to cope with impacts of climate change and variability as a result of USG assistance

How to compile: Indicator guidance still being produced by USAID.

USAID IndicatorsM

ea

sure

me

nt c

hal

len

ge

Field Technician

Service Providers

Farmer

Agriculture Specialist Scientist

Hub Based M&E

Central M&E

Donor (USAID/Gates Foundation)

Monitoring data flow

Challenge: how to keep track of beneficiaries once numbers go into the

ten-thousands?

Challenge: how to keep track of beneficiaries once numbers go into the

ten-thousands?

Need for credible estimates that may be based on…• clients’ lists of mechanized services providers; cross-

check validity in randomly selected villages.• information from ag input dealers on quantity of CSISA

supported seed/herbicides sold; but: credible extrapolation techniques needed and attribution must be clear.

• satellite imagery to track, e.g., advancement of planting dates of wheat.

Evaluation

CSISA

TechnologiesBusiness models

Change agentsPartners

CSISA

CSISACSISA

CSISA

CSISA

CSISA

CSISA

CSISA

CSISA

CSISA

CSISA

CSISA

Impact of CSISA varies by intervention, location, and time…

Impact of CSISA varies by intervention, location, and time…

Baseline – endline survey approach not adequate. Rather use…• Targeted surveys to evaluate specific, major

technologies (e.g. zero-tillage wheat, direct-seeded rice); use of propensity score matching or IV techniques (ideally, repeat such surveys to generate panel data).

• RCTs can be an option for specific interventions (e.g. new seed variety).

• Qualitative case studies as a complement to quantitative assessments.

M&E Technology

• Mobile data collection– Using Open Data Kit (ODK)– Brief surveys

• Surveybe– Comprehensive surveys

• Access, excel, cloud-based data bases

Thank You