Monitoring & Evaluation for Equity & Effectiveness

ME3The F1 M & E System

Rationale

• Need to measure progress, not merely on reform strategies, but more on how it has helped us attain the goals and results we want to achieve

• Need to provide data to stakeholders for outcomes valuable to the poor and the Filipino people, for which they are accountable for

•The F1 reforms were instituted to help us achieve goals faster and better for the poor

II. GOAL

To establish how reforms have improved outcomes for the poor, and how reforms have equitably and effectively achieved goals in the health systems

.

Measuring: EquityEquity

how reforms have improved outcomes for the poor

EffectivenessEffectiveness how reforms have achieved goals in the health systems

Purpose

1. Make available to all stakeholders of F1 relevant data on the health sector’s progress towards attaining improvements in F1 goals

2. Establish, improve, develop and maintain systems for generating, collecting, reporting, analyzing and using data to enhance attainment of F1 goals

Objectives

1. Defining Performance Indicators

2. Establish contribution of stakeholders to outcomes and outputs

3. Report to clients and stakeholders

4. To assess F1 policy levers for revision or augmentation

Performance Indicator Framework

HEALTH STATUS

Financial Risk Protection

ResponsivenessFinal

Performance Outcomes

SERVICEDELIVERY

Governance

Financing Regulation

Major Final Outputs

Access Quality Efficiency Financial Burden

Intermediate Performance Outcomes

HEALTH STATUS FINANCIAL RISK PROTECTION (FRP)

RESPONSIVENESS

Improved Health and Reduced Deaths(eg . CMR)

Equitable Financing or FRP for all

Client Satisfaction

Respect for Clients

ACCESS QUALITY

EFFICIENCY

FINANCIAL IMPACT

Protect Children from Immunizable Diseases(eg FIC)

Full govt. subsidy

SERVICE DELIVERY REGULATION

FINANCING

GOVERNANCE

Immunization Services to cover all 0-18 mos.(eg.# hours immunization services per RHU)

Delivery Services (eg.%LGUs received vaccines on time)

Vaccine Procurement Services (eg. #vials procured)

FINAL OUT-COMES

INTERMEDIATEOUT-COMES

MFO-LGUMFO-CHDMFO-CO

PIF

IV. COMPONENTS OF THE ME3

B. Scorecards and Performance Assessment• A tool for measuring and reporting on performance of

stakeholders for outcomes and outputs that can be attributed to them and for which they are accountable for

• Measure performance on outcomes valuable to the stakeholders clients and convey performance in a manner that clients and stakeholders can easily comprehend

• Serves as Tool for Performance Benchmarks, and for Accountability to clients

• Subset of indicators taken from overall PIF

Scorecards

• Categories of performance levels– Color Coded: Green, Yellow, Red– Easy Label of Status and Link to Decisions

• Challenge:– Selecting 10 Indicators from PIF

• That reflect performance

• Valuable & Understandable to clients

– 10 Themes with 2-3 summated PIs under it

IV. COMPONENTS OF THE ME3

C. Methods for collecting, storing and reporting on results

• Population or household surveys, facility surveys, routine records and reports in the health system, special studies, etc. shall be utilized

• Additional data sources shall be established• Data banking and reporting shall also be

developed

IV. COMPONENTS OF THE ME3

D. Baseline studies and parallel scorecards

• A baseline study shall be done utilizing indicators and assigned data source developed (2007 Q1)

• The baseline study shall provide information on

1) status of performance of the health system, particularly for incoming Local Chief Executives

2) areas of the PIF that need to be improved or revised

3) targets that need to be revised or set

IV. COMPONENTS OF THE ME3

E. System of governance of the F1 ME3

• Governance system shall be established which shall cover concerns such as level of independence of the ME3, internal rules, decision making process

VI. Functional Structure

DOH CLUSTERS TWGs(development of F1 ME3)

DOH EXECOM(Policy decisions)

TCG(Technical Approval)

DOH Task Force on F1 M&E(Coordination and Guiding Work)

Cluster TWGs

TWG PIF SCORECARD

PSDO Yes Central Office

Regulation Yes Central

SMCO Yes Central Office

Donors (BIHC)

CHD(BLHD)

IMCO Yes Central Office

FICO Luzon Yes DOH Hospitals

FICO Vis-Min Yes LGU

Technical Exchange FellowsUSAID

TWG PIF SCORECARD

PSDO-SD(PH) Yes Central Office

FICO Luzon Yes DOH Hospitals

FICO Vis-Min Yes LGU

Level/Stakeholder Assesment OversightClient HPDPB &

MIS

(Through a contracted external agent)

LGUProviders

LGU-Municipality ProvinceLGU-Province CHDCHD FICO LVMDOH Hospitals

BFAD, BBHFS, BHDT DOH PSDO Regulation

NCDPC DOH PSDO NCDPC

HPDPB,BIHC,BLHD,HRDB DOH SMCO

FS, PS, MIS DOH IMCO

Donors BIHC

ME3 IMPLEMENTATION

Implementation Structure

DOH EXECOM(Policy decisions)

TCG(Technical Approval)

HPDPB MIS

FICOs, PSDOs, SMCO, SMCT

LEVELS OF STAKEHOLDERSLEVELS OF STAKEHOLDERS

LEVELS OF STAKEHOLDERSLEVELS OF STAKEHOLDERS

LEVELS OF STAKEHOLDERS

IV. COMPONENTS OF THE ME3

F. Capability Building

• Competent units and personnel

• IT and MIS systems

• Funds for surveys and special studies

V. Phases of Activities and Timelines

Phase

1Drafting of PIF, Scorecards, and Data Sources for the 11stst set set of Final Outcomes

FOCUS: MDG

Develop Templates,Orientation Module

October-December 2006

Phase

2

•Baseline Studies

•Initial Publication of Scorecards

•Refinement of PIF for first set and ME3

January-April 2007

Phase

3

•Develop lower level output /process/ input indicators for 1st Set1st Set of FO

•Develop PIF for subsequent set of Final Outcomes consistent with F1 ME3

•Design of MIS, Capacity Building

2007

TASKS

Draft PIF & Scorecards

Consultations

Pretest

Baseline Survey

1. Final Outcomes2. Intermediate Outcomes3. Major Final Outputs4. Select key Indicators for

Scorecard1. Small Groups2. E-Mail3. National

Consultation(Dec.4-5)

For Feasibility

Q1 2007

Health outcome

Family/individual

Provider

SupplierLocal governm

ent

Central governm

ent

regional offices

FINAL OUTCOME INDICATOR NATIONAL EQUITY

Life expectancy Increase to…Baseline: 69.8 years in 2003 (Projections based on 1995 CPH)

LE of lowest 2 quintiles to approximate LE of highest 2 quintiles

Infant Mortality RatePneumoniaBacterial sepsis of newbornDiarrhea and gastroenteritis of presumed infectious origin

Reduce IMR to 17/1000 livebirths (MTPDP) Baseline: 29/1000 livebirths in 2003 (NDHS 2003)

Reduce IMR in lowest 2 quintiles to 18.5/1000 livebirths (TF) Baseline: 37/1000 lb in lowest 2 quintiles (NSO, ORC macro, NDHS 2003/Herrin 2006)

Under Five Mortality RatePneumoniaDiarrheas and gastroenteritis or presumed infectious originMeasles

Reduce U5MR to 32.24 (MTPDP) Baseline: 40 in 2000 (NSCB 2000)

UFMR in lowest 2 quintiles to approximate national target of 32.24 (TF)Baseline: 56.5 UFMR in lowest 2 quintiles (NSO, ORC Macro, NDHS 2003/Herrin 2006)

Maternal Mortality RateHypertension complicating pregnancy childbirth and puerperiumPostpartum hemorrhage

Reduce MMR to 90/100,000 livebirths (MTPDP)Baseline: 172/100,000 (NDHS 1998)

Reduction in baseline rates in lowest 2 quintiles by 70%

Malnutrition rate:Prevalence of underweight preschool children aged 0-5 years old

Reduce to 21.6% (MTPDP)Baseline: 27.6%(NNS 2003)

FINAL OUTCOME INDICATOR NATIONAL EQUITY

Total Fertility rate Reduce to 3.1 (MTPDP)Baseline: 3.5 in 2003 (NOH 2005-2010)

TFR in lowest 2 quintiles to approximate national target of 3.1 (TF)Baseline: 5.25 TFR among lowest 2 quintiles (NSO, ORC Macro, NDHS 2003/Herrin 2006)

HIV prevalence Contain ≤ 1% prevalence (MTPDP)Baseline(NOH): ≤ 1 % (DOH)

Hospital Net Death Rate

EQUITABLE FINANCING OR FINANCIAL RISK PROTECTION FOR ALL

Ratio of health expense to total non-food spending

Reduce ratio of health expense to total non-food spending to --%Baseline:---

Ratio of health expense to total non-food spending among lowest 2 income quintiles to be approximate to highest 2 income quintilesBaseline:---

Out of pocket spending for health of total health expenditure

Reduce out-of-pocket spending to 20% of total health expenditure (NOH/PNHA 2003)Baseline: 45% out-of-pocket spending out of total health expenditure (PNHA 2003)

Reduce out of pocket spending of lowest 2 income quintiles to ---% of total health expenditureBaseline: ---% out-of-pocket spending of lowest 2 income quintiles out of total health expenditure

FINAL OUTCOME INDICATOR

NATIONAL EQUITY

RESPONSIVENESS OF HEALTH SYSTEM

Client satisfaction on:Health facilityHealth agencyHealth programClient participation

Health facilityBaseline (WB/SWS 2000):Net satisfaction rate forFor- profit hospitals +96Traditional healers +94Nonprofit hospitals +91RHU +82Government hospital +79BHS +74

Net satisfaction rate to be equal/similar between lowest 2 income deciles and highest 2 income decilesBaseline: ---

Provider’s respect for personsDignityConfidentialityAutonomy

Scores to be equal/similar between lowest 2 income deciles

and highest 2 income deciles

Client orientationPrompt attentionAmenitiesAccess to social supportChoice of provider

Scores to be equal/similar between lowest 2 income deciles

and highest 2 income deciles

MDG-F1 Linkage Infant & Child Mortality Rate

ACCESS QUALITY FINANCIAL IMPACT EFFICIENCY

Pneumonia

Mortality Rate

Diarrhea Mortality

Rate

B. Sepsis Mortality

Rate

Malnutriton Rate

Water & Sanitatio

n

DOH HOSPITALS

CHD PSD-SD SMCO IMCOLGU

PROVIDER

(Public & Private

CLIENT

NCDPC

NCHFD

NCHP

NEC

HPDPB

BIHC

BLHD

HHRDB

Finance

Procurement

MIS

LGU Scorecard FrameworkHealth MDGs

Intermediate Outcomes

FINAL OUTCOME

Intermediate Outcome

Providers/StaffActions/Decisions

FacilityCharacteristics

ClientsActions/Decisions

ClientCharacteristics

LGU Actions/Decisions

InputsOutputs

Implementation Schedule

FO FO

FO FO FO

IO IO IO

MFOInput/Process

/Structure

MFOInput/Process

/Structure

MFOInput/Process

/Structure

MFOInput/Process

/Structure

MFOInput/Process

/Structure

Year 1 2 3 4 5

2007 2008 2009 2010 2011