1

Monitoring & Evaluation (M&E)

- - an integral component of NRHM PIPs

Parameters of Health Care

AwarenessAvailableAccessible - equityAffordable

Quality

2

3

Monitoring & Evaluation

Requirements under the Logical Framework

Approach of the National PIP

4

Agreed Indicators … (1)

13 Process Indicators – 6 monthly review

1. % of ANM positions filled

2. % of states and districts having full time program manager for

RCH with financial and administrative powers delegated

3. % of sampled state and district program managers aware of

their responsibilities

1. % of sampled state and district program

managers whose performance was reviewed

during the past six months5. % of districts not having at least one month

stocks of (a)Measles vaccine, (b)Oral Contraceptive Pills and (c)Gloves

6

Agreed Indicators … (2) 13 Process Indicators – 6 monthly review (cont’d)

6. % of districts reporting quarterly financial performance in time

7. % of district plans with specific activities to reach vulnerable communities

8. % of sampled districts that were able to implement M&E triangulation involving communities

9. % of sampled outreach sessions where guidelines for AD syringe use and safe disposal are followed

10. % of sampled FRUs following agreed infection control and health care waste disposal procedures

11. % of 24 hrs PHCs conducting minimum of 10 deliveries/month

12. % of upgraded FRUs offering 24 hr. emergency obstetric care services

13. % of sampled health facilities offering RTI/STI facilities as per agreed protocols

7

Agreed Indicators … (3)

Output Indicators from Mid & End-line SurveysOutput Indicators from Mid & End-line Surveys

The states are to The states are to set levels of achievement based based on their on their own assessmentsown assessments..

1. 1. Contraceptive prevalence rateContraceptive prevalence rate2. % eligible couples using any spacing method for 2. % eligible couples using any spacing method for

> 6 months> 6 months3. % of women delivered during past one year who 3. % of women delivered during past one year who

received 100 IFA tabletsreceived 100 IFA tablets4. % deliveries conducted by skilled providers 4. % deliveries conducted by skilled providers

(doctors, nurses or ANMs) (doctors, nurses or ANMs) 5. % of 24 hrs PHCs conducting minimum of 10 5. % of 24 hrs PHCs conducting minimum of 10

deliveries/monthdeliveries/month

Agreed Indicators … (4)

66. % of upgraded FRUs offering 24 hr. emergency obstetric care

services

7. % of 12-23 months children fully immunized

8. % of mothers and newborn children visited within 2 weeks of delivery

by a trained community level health provider/AWW or health staff

(ANM/Nurse/Doctor)

9. % of children suffering from diarrhea during past 2 weeks received

Oral Rehydration Solution

10. Polio free status achieved since

Details of numerator and denominator and sources of information given in Appendix IV of National PIP

9

Program Planning & Service Delivery

MIES System Model in NRHM

MIES

Validation

EvaluationQuality Assessment

Monitoring

MIES

10

M&E Activities:–(1)

A - Surveys

3.1 Improve DLHS-3 (Mid-line Survey) to reflect Equity & Access

3.4 Equity Indicators for large national surveys like NFHS

Actions & Current Status

NFHS-III: – Key results out, Detailed results in August, 2007

DLHS-III: – Pilot Work to begin in July, 2007

Annual Health Survey (AHS) – being conceived with the RGI– District Health Profile

11

M&E Activities:–(3)

B. MIS3.23.2 Guidelines for local data to prepare & monitor

plans

3.5 CNAA Manual to be revised with field test3.6 Simplified MIES proforma

    Actions & Current Status

• MIES Format developed and sent to State Govts in August, 2006.

• Key Indicators for Health Programmes integrated and sent to States in November, 2006

• Data on revised formats have been received from 13 States o Andaman & Nicobar Isles, Chhattisgarh, Delhi, Goa, Gujarat, Himachal

Pradesh, Jharkhand, Madhya Pradesh, Maharashtra, Manipur, Meghalaya, Puducherry, Rajasthan

• User Guidelines for MIES Format finalised.

o Contains information on use of MIES data for local needs

12

3.7 Analytical reporting for outputs

Actions & Current Status.

• A web based information and communication system to be implemented in two phases.

– The first phase would essentially be a Bulletin Board for the NRHM/RCH.

– The second phase is to take care of the data ware housing requirement of MIES.

• In-house software being developed for routine MIS reports

M&E Activities:–(5) B. MIS

13

3.9 Concurrent evaluation studies through common format

Actions & Current Status

Evaluation presently being done through RETs – Max 168 (7x2x12) districts per annum– Needs reinforcement and realignment

PRCs undertake adhoc evaluation studies

Several Institutions approached for Concurrent Evaluation of NRHM• Workshop held in Feb, 2007• A Committee formed to evaluate the proposals • Framework for Evaluation being worked out

M&E Activities:–(6) B. MIS

14

3.10 Quality Assurance Surveys

Actions & Current Status

Monitoring tools, check lists, operational manual for QA finalized

Six States identified for piloting of QA and launched in December, 2006

- UP, Uttaranchal, West Bengal, Assam , Maharashtra, Karnataka

(1 Dist each, 2 in UP)

- Duration of QA piloting would be of 24 months

M&E Activities:–(7) B. MIS

15

3.11 Programme Management Reviews (Assessment of Management Capacity)

Actions & Current Status

Study by IIM-A completed and Tools prepared and circulated to StatesBased on feedback, the Tools were refined and up-scaled country-wide in January, 2007.Each State to canvas the Tool and report findings annually

M&E Activities:–(8) B. MIS

16

3.12 Community Monitoring & Triangulation

Actions & Current Status

Plan of action

1. Experts on Community Monitoring approached for a Concept Note on Triangulation

2. Based on the feedback from the experts, a Triangulation framework would be prepared and piloted

M&E Activities:–(9) B. MIS

19

Program Planning & Service Delivery

Status of MIES System Model

MIES

ValidationCommunity Monitoring & Triangulation

Evaluation•Surveys, RETs

•Concurrent evaluation Quality AssessmentPilot on

MonitoringFormat & guidelines

Programme Mgmt Review

Launched

Working Group on

M&E

20

The ten Steps to building a performance – The ten Steps to building a performance – based M&E Systembased M&E System

1.1. Conducting a Readiness AssessmentConducting a Readiness Assessment

2.2. Identifying & agreeing on Performance Outcomes to Identifying & agreeing on Performance Outcomes to Monitor and EvaluateMonitor and Evaluate

3.3. Selecting Key Indicators to Monitor OutcomesSelecting Key Indicators to Monitor Outcomes

4.4. Baseline Data on Indicators to Monitor OutcomesBaseline Data on Indicators to Monitor Outcomes

5.5. Planning for Improvement-Setting Realistic TargetsPlanning for Improvement-Setting Realistic Targets

6.6. A Monitoring and Evaluation System (HMIS, Web-based A Monitoring and Evaluation System (HMIS, Web-based ???)???)

7.7. The Role of EvaluationThe Role of Evaluation

8.8. Reporting the FindingsReporting the Findings

9.9. Using Own/Other’s FindingsUsing Own/Other’s Findings

10.10. Sustaining the M&E SystemSustaining the M&E System

21

M&E Plan: MDG goals achievement plan

1. Map information sources and flows– Performance data – Monthly, Qtrly, Annual– Surveys - NFHS, DLHS, NSSO, Other agencies– Evaluation Studies – RET, PRC, NGOs, Communities

2. Map M&E requirements• Household surveys• IT infrastructure mapping

Hardware, Software, Network, HMIS application, People – existing skills vs training needsState, District, Blocks

• Printing of Registers at SC, PHC, CHC etc• Map institutions (public & private) – by service, strength etc• Address vulnerable groups in IDHAP

3. Report on MIES format only• Don’t change the reporting format• Form 9 No longer required

4. Plan for sustainability of M&E

22

What we expect from the State Governments …(1)…(1)

Prepare a State-Logical Framework Approach (LFA) on the lines of the

National PIP to monitor the M&E Activities

Reporting Monthly Performance data on Revised MIES format.

Assess management capacity - Programme Management Tool

Undertake Household Surveys – to be updated annually

Revising and fresh printing of the primary registers from the Sub-centre

level onwards – capture data for SC/ST

Training of manpower on data collection, flow and relevance

Upgrading/modifying the hardware and software for States having an

HMIS.

For States not having an HMIS efforts are to be made for establishing one

The State Govts may revisit and revise their State PIPs to take the above aspects into consideration.

23

What we expect from the State Governments …(2)

Challenges

Getting the primary registers from the Sub-centre level onwards in position to capture data for vulnerable groups

Institutionalising the data flow mechanism for timley and quality data

Leveraging the advances in IT for expediting data flow Capturing information from private sector Have an inclusive Distt Action Plan to cover M&E requirements at

the Distt level and aggregated at State level

24

Triangulation – an emerging concept

Triangulation

Monitoring/ Performance data

Survey Data/ Evaluation studies

Community Monitoring

Concurrent Evaluation