going digital: innovative measurement technologies€¦ · going digital: innovative measurement...

Going Digital: Innovative Measurement Technologies

Nikhil Wilmink & Sree Sen CLEAR South Asia

July 27, 2013

CLEAR South Asia Mandate

CLEAR South Asia strategy includes a focus on the use of cutting-edge technology for effective collection and utilization of M&E

In collaboration with Fieldata.Org, CLEAR South Asia aims to promote the use of M&E technology, such as mobile phones and PDAs, for more real-time aggregation and effective utilization of M&E.

Context – E-society

Emphasis on the use of technology for more effective collection and utilization of M&E

The fundamental objective of all e-Society initiatives is to make Information, Communication & Technology (ICT) more inclusive – i.e. ALL members of society, especially those that are socially and economically marginalized, should gain access to and benefit from the knowledge, power and opportunities brought about by new ICTs

Sri Lankan Information and Communication Technology Agency has similar goals (http://www.icta.lk/en/programmes/e-society.html )

Mobile Technology in Evaluations

Improving transparency & accountability in development organizations and

government agencies, though technology-enabled M&E for better

monitoring, sharing, and application of data.

Enabling organizations, donors and citizens to use M&E data for real-time

decision-making, better implementation and delivery of projects and

services

Problems in Data Collection and Management

What do the following mean? Data reliability (will we get the same data, when collected again?)

Data validity (Are we measuring what we say we are measuring?)

Data integrity (Is the data free of manipulation?)

Data accuracy/precision (Is the data measuring the “indicator” accurately?)

Data timeliness (Are you getting the data in time?)

Data security/confidentiality (Loss of data / loss of privacy) Can technology help us address the above issues? How?

Mobile Technology Options

Mobile Technology Options

Data Quality & Real-time Data for Monitoring Purposes

Improving:

• Data quality

• Speed

• Transparency

• Accessibility

• Flexibility

Quick & Easy Set up:

Mobile-based Monitoring and Evaluation in Action:

MFI agents entering weekly loan repayments

for instant tabulation

Community health workers feeding back information on beneficiaries for automatic identification of high risk cases

Auditors collecting survey, observational, photographic and GIS data on infrastructure in slums.

Use of Mobile-based Technology in M&E

REPORTING/MIS Routine (Real-time) reporting

• Weekly loan repayment information of MFI clients reporting by field staff

MONITORING Ongoing program monitoring

• Beneficiary information collected and sent by health workers (for disease surveillance, delivery of benefits etc.)

Occasional (Surprise) checks • Spot checks by supervisors to monitor attendance and performance of

staff

EVALUATION Survey data

• Household survey data to assess impact of <<xx>> program

Use of Mobile Technology as key intervention in Programs

Information/Messages

• SMS reminders to beneficiaries about important health activities (treatment compliance)

Implementation Tools

• Videos and online teaching aids used in schools for regular or remedial education programs

Biometric, GIS information

• Record beneficiary information for easier tracking and follow up. Complementarities with other programs by same organization

Case Study: Delhi Voters Project

Project description and objectives

• An RCT evaluating whether providing information to government officials and slum dwellers can lead to higher accountability and thereby improved service delivery.

Interventions evaluated:

• The effect of providing information on spending and quality of public services delivered by elected officials during election sensitive periods

Role of mobile-based data collection

The intervention

• Field-based audits of public services in slums using mobile-based technology

• Surveys and observations

• Photographs

• GIS indicators

• Send report cards to elected representatives highlighting the quality and access to public services in their area

The implementation

GARBAGE FACILITIES

1) Dumpsters 2) Bins

3) Informal Points

TOILETS

Open Public Toilet Complex

Public Urinals

Started by auditing …

Audit Report Cards

Going a step further with Mobile Based GIS software

Accessibility to public services • How far is the nearest

toilet? • How far is the nearest

formal point of disposing garbage?

Spread Analysis

• Does cleanliness of a toilet affect household health?

MOBILE GIS !

Delhi Voters Project Technology Demo

Setting up Wireless (already done on the demo phones):

1. Click on the blue icon in the bottom right hand corner of the phone

2. Scroll right and click on the settings icon

3. Select Wireless and networks Wi-Fi Settings Connect to wireless network

4. Return to the main menu via the arrow in the bottom corner and select the Fieldata app on the main dashboard

Downloading Fieldata App (already downloaded on phones)

1. m.fieldata.org

2. Download first option (not the Oriya script)

3. Go back to the handset’s downloads and select Fieldata app

Designing the Survey Form

Filling out the Form

Select

Garbage Audit V3

TAKE 5-10 minutes to fill out the form. Take note of the “logic and control features” built into the survey form.

Fill Blank Form

Filling out the Form

Key Features:

1. Compulsory Questions

2. Grouping – (Repeats & Loops)

3. Location – Mapping

4. Skips & Branching

5. Photo

6. Multiple question types (GPS, photos, audio/video) to cover information not possible on paper

Editing or Sending Form

1. Edit and Check your data by selecting

2. See all your answer and change them and save these changes if you want

3. Finally

4. Mark the tick box green of your Saved Survey

5. Press “Send Selected”

6. Username: ClearSA (press Next).

7. Password: IFMR

8. Upload Results message saying: “Garbage Audit Survey v3 – Success”

9. Log on to immediately view your data on a map & as graphs!

www.fieldata.org

Edit Saved Form

Send Finalized Form

How much Time, Money and Effort Does it Take?

Less than what it is costing you now!

Selecting the Appropriate Technology Options

SMS USSD Call App IVRS

Low Hardware Costs

Low Running Costs

Will work with any Handset

Ease of Use / Interactive Interface

Not dependent on network availability

Not dependent on power reliability

Local Language

Low technology literacy / learning curve

Can handle large number of data sources

Can handle high frequency of collection

Can handle large amounts of data points

Provides “Audit” data (photos, GPS, etc.)

Data quality (validations, etc.)

Get data directly from respondents

About Fieldata.org

Fieldata.Org is a mobile-&-web portal for NGOs to raise funds, by offering real-time monitoring, and objective data for evaluation.

Mission is to improve transparency & decision-making in development

organizations and government agencies, by empowering them with technology-tools for better monitoring, sharing, and application of data.

Want organizations and donors to objectively answer:

Which projects should resources be allocated?

How well do projects utilize these resources?

About Fieldata.org