scaling up performance based financing in rwanda 2004-2008
TRANSCRIPT
1
Scaling Up Performance Based Financing in
Rwanda 2004-2008
Rwanda PBF team: Paulin Basinga, Ghyuri Fritsche,
Bruno Meessen, Laurent Musango, Louis Rusa, Claude Sekabaraga, Agnes Soucat et al 1
2
2
Outline
1. Reconstruction and Innovations 2. Scaling Up Performance Based
Financing3. Results4. Reforms5. Impact Evaluation
3
3
Outline
Reconstruction and Innovations Scaling Up Performance Based
Financing Results Reforms Impact Evaluation
The post-colonial times .. Modern health introduced in Rwanda free of charge to users and
funded through direct public subsidy : infrastructure, equipments, personnel etc
1980s : shortages and rationing, dilapidation of health services 1992: community participation for financing and management of
health care (Bamako Initiative). 1994 : Genocide 1995-97: Reconstruction after the genocide : emergency
situations, NGOs, services free of charge. 1998-Willingness to come back to development and government
leadership: drug revolving funds re-established and cost sharing reintroduced
5
Innovation II: Scaling Up of Community Health Insurance
Source: Cellule d'appui aux mutuelles de sante. Ministry of Health / Rwanda 5
Innovation III: Scaling Up of Performance Based Financing
Phase -0 (white shaded): the three PBF pilot projectsPhase-1 (pink shaded): districts in which PBF was started in Jan 2006 Phase-2 (red shaded): the seven ‘control districts’ in which PBF was implemented in April 2008.
7
7
Outline
Reconstruction and Innovations Scaling Up Performance Based
Financing Results Reforms Impact Evaluation
Chronology of Performance-Based Financing in Rwanda
9
The PBF pilot experiments (2002-2005)
Three pilot schemes: Butare (since 2002) Cyangugu (since 2003) BTC (since 2005)
Led at provincial level by International NGOs. Priority health interventions: child immunisation, ANC,
assisted deliveries, family planning, curative care. A fee-for service at health center level
Scaling up: 2005-2008 2004: Evaluation of Butare and Cyangugu pilots
2005: Institutionalization:
2006: Scaling Up to 23 districts with 7 controls
2008: All districts
Evaluation: Cyagungu and Butare PBF2004
Use of Assisted Deliveries over time
0
5
10
15
20
25
30
Butare Cyangugu Gikongoro Kibungo
% o
f b
irth
s
200120022004
Performance Based Contracts Control
Institutionalization: integration in country budget
Since 2005, government pays outputs through recurrent budget (PBF budget line): 2005 US$ 800,000 for 4 districts 2006 US$ 5,000,000 for the country
Funds flow quarterly from Treasury directly to health facilities’ Bank Account on the basis of results of previous quarter
Since 2007, budget line item for PBF scheme for the District Steering Committee activities based at District level
Institutionalization: the HIVAIDS money
One national approach, one institutional set-up, same unit costs and same admin system facilitates alignment: Global Fund pays for HIV indicators into their supported sites .
Payment through same Bank Account: e.g MSH and ICAP – USG contractor-, FHI and BTC
Careful assessment of incentives through HIV monies in PBF: protecting PHC services by linking payments of HIV and PHC monies to levels of quality of general services.
Unit Fee * Quantity * % Quality = Payment;
National PBF Model : 2005-2008
15
National PBF model : 2005-2008
The national model for health centers is based on contracts between different levels: Steering Committee (comité de pilotage) with
representation of health authorities Three layers of contracts:
Contract between CAAC and comité de pilotage Contract between comité de pilotage and health facility Contract between Health facility and individual health workers
In-depth verification activities : Done by one focal point per administrative district for quantitative
evaluation Done by hospital for qualitative evaluation
Separation of functions
16
PBF Payment at Facility Level
Payments for performance are based on the quantity of outputs achieved (through case-based remuneration) conditional on the quality of services rendered.
The outputs (quantity) are measured monthly The quality is measured quarterly through the use of an elaborate
supervisory checklist. (13 services) The formula : PBF Payment HC = Quantity * % Quality
‘PBF Payment HC’ is the consolidated quarterly health center invoice (for either general or HIV),
‘Quantity’ stands for the quarterly provisory health center invoice (the sum of all indicators multiplied with their unit fees),
‘% Quality’ stands for the consolidated score—expressed as a percentage—obtained from the quarterly quality supervisory checklist
17
General health indicators and PBF prices Num INDICATEUR
Amount paid per case (Rwf)
Amount paid per case (US$)
1Number of New cases 100 $ 0.18
2Number of New cases received at the prenatal care (first visits) 50 $ 0.093Number of Women who received 4 prenatal consultations 200 $ 0.37
4
Number of women completed the 2 or 3 or 4 or 5 Tetanus vaccines 250 $ 0.46
5
Number of Women who received the 2nd dose of Intermittent Preventive Treatment of malaria 250 $ 0.46
6
Number of at risk pregnancies Referred before 9 months of pregnancy 1000 $ 1.83
7
Number of child aged 12-59 months seen at the curative care service for growth monitoring 100 $ 0.18
8
Family planning P new users (DIU, Pills, injections, implants)100 $ 0.18
9
Family planning : number of users: DIU, Pills, injections, implants 1000 $ 1.83
10Fully Vaccinated Child 500 $ 0.92 11Institutional Deliveries at the health center 2500 $ 4.5912Emergency referrals to the Hospital for obstetric care 2500 $ 4.5913Malnourished children referred 1000 $ 1.8314Others Emergency referrals 1000 $ 1.83
18
HIV/AIDS indicators and PBF prices (1$ = 545 RWF)
Num INDICATORS
Amount paid per case (Rwf)
Amount paid per case (US$)
1Number of clients tested for HIV at the VCT center 500 $ 0.92
2Number of couples/partners tested during the reporting month 2500 $ 4.59
3Number of HIV+ pregnant women on ART treatment during labor 2500 $ 4.59
4Number of infants born to HIV+ mothers tested 5000 $ 9.175
Number of HIV positives patients who received CD4 test 2500 $ 4.59
6Number of HIV + patients traited with cotrimoxazole each month 250 $ 0.46
7Number of new adults HIV+ on ART treatment 2500 $ 4.598Number of new infants HIV+ on ART treatment 3750 $ 6.889Number of HIV+ women on contraception 1500 $ 2.75
10Total number of HIV+ patients tested for tuberculosis 1500 $ 2.75
19
Process evaluation
Meeting of the Steering committee : quantity and quality assessment
Counter verification of the patients in the community : looking for the phantom patients
Counter verification of the quality score by hospital team: randomly selected site
Comparison of PBF data with HMIS data Peers evaluation (Hospital PBF)
Administrative & management coordination PBF admin system with internet based data entry and
retrieval facilitate decentralized management and future decentralized payments (by districts);
Semi-automated payment module, linked to central database, witch allow for ease of payments by MOF (Ministry of finances) and others (MSH; BTC; FHI and GF);
Central database allows for following trends and forecast accurately financial risk;
•INSERT GRAPHIC TO ADD MAP
•MAP IS 6.17” TALL
•ICT management tools: www.pbfrwanda.org.rw
How many persons to do that?
MOH central PBF Unit (CAAC): 1 coordinator and two full-time staffs;
A key role for partners (members of the CAAC and on the field)
An Extended team approach has been put in place to cover 23 districts, and includes PBF focal points from the MOH, eight NGOs and a bilateral agency as a coordination structure
23
23
Outline
Reconstruction and Innovations Scaling Up Performance Based
Financing Results Reforms Impact Evaluation
Results 1. Increases in the Volume of Services2. Increase of the Quality of Services3. Increase of staff productivity4. Provider Enthusiasm and Motivation
25
25
Outline
Reconstruction and Innovations Scaling Up Performance Based
Financing Results Reforms Impact Evaluation
Increase in Volume of Services (after 27 months)
PBF Indicator January 2006 average/month/
health center( 258 health centers on
average)
March 2008average/month/
health center(286 health centers on
average)
Percentage increase (linear/log R2)
Institutional Deliveries
21 37.5 78% (log 0.75)
New Curative Consultations
985 1,489 51% (log 0.19)
ANC: second dose of Tetanus Toxid
21 52.5 150% (log 0.63)
Family Planning new users
15.5 47.9 209% (linear 0.88)
Family Planning users at the end of the month
175.2 711.6 306% (linear 0.98)
Results for Family Planning Users at the end of the Month
Family Planning, Modern Methods, Users at the End of the MonthAverage Per Health Center per Month
R2 = 0.9784
0
100
200
300
400
500
600
700
1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12
2006 2007
Ave
rage
num
ber
per
mon
th
175
640
FP Injections and oral methods at Health Centers % Increase in Prevalence over 24 months; •January 2006 through December 2007
Other improvements Over 16 months of PBF, the Quality increased on
average by 7% across these 13 services. A sharp increase in staff productivity. Whilst all providers appreciate the additional bonuses
that they earn through PBF, most also see clear advantages in the better services they provide, and take clear pride and ownership of these activities which originate ‘from within’ as opposed to being dictated from above.
Improvements confirmed by survey based data
31
Assisted delivery – Modern contraceptive use
Meilleures pratiques en SM au Rwanda31Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et intérimaire 2007
Decrease of fertility
Fertility Rate 2005-2008
5.25.35.45.55.65.75.85.9
66.16.2
2005 2007
Source : Rwanda DHS 2005-2007
Dramatic Increase of Coverage of Insecticide Treated Nets
ITNs coverage 2005-2007Proportion of Children less than 5 sleeping under a
bed net
0%10%20%30%40%50%60%70%80%
2005 2007
Source : Rwanda DHS 2005-2007
Malaria out patient Non Malaria out patient
2001 2002 2003 2004 2005 2006 2007
0K
5K
10K
15K
20K
25K
Sum
of Con
f# o
utpa
tient
mal
aria
Sheet 1
Age group
5 years and above
Under 5 years
The trend of sum of Conf# outpatient malaria for Year G#C. Color shows details about Age group . The data is filtered on Country , which keeps Rwanda. The view is filtered on Age group , which keeps 5 years and above and Under 5 years.
2001 2002 2003 2004 2005 2006 2007
0K
20K
40K
60K
80K
100K
120K
140K
160K
180K
200K
220K
Sum
of Non
mal
aria
OPD
Sheet 1
Age group
5 years and above
Under 5 years
The trend of sum of Non malaria OPD for Year G#C. Color shows details about Age group . The data is filtered on Country , which keeps Rwanda. The view is filtered on Age group , which keeps 5 years and above and Under 5 years.
Decrease of malaria incidence
Malaria deaths decreased
Malaria death Non-Malaria Death
2001 2002 2003 2004 2005 2006 2007
0
50
100
150
200
250
300
Sum
of Mal
aria
dea
th (clin
ical
+ c
onf)
Sheet 1
Age group
5 years and above
Under 5 years
The trend of sum of Malaria death (clinical + conf) for Year G#C. Color shows details about Age group . The data is filtered on Country , which keeps Rwanda. The view is filtered on Age group , which keeps 5 years and above and Under 5 years.
2001 2002 2003 2004 2005 2006 2007
0
100
200
300
400
500
600
700
800
900
1000
Sum
of Non
Malaria
Dea
th
Sheet 1
Age group
5 years and above
Under 5 years
The trend of sum of Non Malaria Death for Year G#C. Color shows details about Age group . The data is filtered on Country , which keeps Rwanda. The view is filtered on Age group , which keeps 5 years and above and Under 5 years.
36
Infant and Under 5 mortality rate
37
37
Outline
Reconstruction and Innovations Scaling Up Performance Based
Financing Results Reforms Impact Evaluation
Reforms 2000-2008
Autonomization
Performance Based Budgeting
Decentralization,
Reform I: Autonomization Based on Bamako Initiative Health centers and hospitals fully autonomous : 60%
healh centers public autonomous, 40% faith- based private not for profit
Facilities are financially autonomous: Commercial bank account, revenue from user payments and payments from community insurance
Subsidized by the government: Needs based block grant for wages, Performance Based Grant for recurrent costs, and specific financing from public health programs (vaccines, contraceptives, TB drugs and ARV etc)
Reform II: Performance Based Transfers “IMIHIGO”: contract between the President of
the Republic and the district mayors Key health indicators integrated in the contract (in
2007: ITNs, Mutuelles, FP, safe deliveries, hygiene..)
Strong political commitment to results Quartely review with Prime Minister, President
attending twice a year
Reform III: Decentralization Administrative and fiscal
decentralization gives flexibility to local governments by providing them with needs and performance based block grants
Decentralization of wages sent as a block grant to facilities
Facilities have the authority to hire and fire
Facilities receive blockgrant from government
“People follow the money”
Fiscal and Financial Decentralization
0
20,000,000,000
40,000,000,000
60,000,000,000
80,000,000,000
Dis
burs
ed 2
002
Dis
burs
ed 2
003
Dis
burs
ed 2
004
Dis
burs
ed 2
005
Budg
et 2
006
Proj
ecte
d 20
07
Year
Am
ount
in R
WF
Transfers to Districts
CDF
Transfers to Provinces
42
42
Outline
Reconstruction and Innovations Scaling Up Performance Based
Financing Results Reforms Impact Evaluation
43
43
Study Rationale
No examples of rigorously evaluated bonus payment schemes to public sector health care providers in developing countries
No distinction between the incentive effect and the effect of an increase in resources for the health facilities
No unbundling of extrinsic and intrinsic-altruistic- motivation
Link between worker motivation programs and quality of care
44
44
Hypotheses
For both general health services and HIV/AIDS services, we test whether PBC:
Increases the quantity of contracted health services delivered
Improves the quality of contracted health services provided
Does not decrease the quantity or quality of non-contracted services provided,
Decreases average household out-of-pocket expenditures per service delivered
Improves the health status of the population
45
45
Evaluation Design
Make use of expansion of PBC schemes over time The rollout took place at the District level; random assignment at the
district level Treatment and control facilities were allocated as follows:
Identified districts without PBC in health centers in 2005 Group the districts in “similar sets” based on characteristics:
rainfall population density livelihoods
Flip a coin to assign districts within each “similar” to treatment and control groups.
46
More money vs. More incentives
Incentive based payments increase the total amount of money available for health center, which can also affect services
Phase II area receive equivalent amounts of transfers average of what Phase I receives Not linked to production of services Money to be allocated by the health center Preliminary finding: most of it goes to salaries
47
Baseline Survey: Sampling Strategy47
Lessons Learned: PBF
Start with easy things and then go progressively to complexity. Health centers before hospitals Simple quality indicators
Need for strong leadership and political will from authorities
Need for strong implementation oriented coordination structures and large pool of trainers
Lessons Learned: PBF
Institutionalization is the key phase: Importance of institutional contracts
Critical role of validation institution
PBF was used as a lever for reform: Allowed to raise reuneration n assoication with
performance Progressively shifted management of humn resources Assoicated decentralization with rapid results
Lessons learned Fiscal decentralization can help increase resources for health
facilities if well designed.. To serve purpose of service delivery.. autonomy of provision is essential…
Results Based Financing is a powerful mechanism to achieve the twin objective of increased performance and increased retention of qualified service providers
Combining public subsidy and private funding leads to increased remuneration and better adequacy with needs
Delinking healh workers from the central wage bill and civil service is possible..and health workers like it ..
Rwanda is back on track to reach the MDGs including MDG5
51
IMPACT EVALUATION OF PERFORMANCE BASED
FINANCING for
A collaboration between the Rwanda Ministry of Health, CNLS, and SPH, the INSP in Mexico, UC Berkeley and the World Bank
GENERAL HEALTH AND HIV/AIDS SERVICES in
RWANDA
51
52
52
Roll-out plan Phase 0 districts (white) are those districts in which PBF
was piloted Cyangugu = Nyamasheke + Rusizi districts Butare = Huye + Gisagara districts BTC = Rulindo + Muhanga + Ruhango + Bugesera + Kigali ville
Phase 1 districts (yellow) are districts in which PBF is being implemented in 2006, following the ‘roll-out plan’
Phase 2 districts (green) are districts in which PBF is not yet phased in; these are the so-called ‘Phase 2’ or ‘control districts’ following the roll-out plan. According to plan, PBF will be introduced in these districts by 2008.
53
Sets Phase I Phase II1 Kibungo all Kirehe all2 Nyanza remaining Kamonyi all3 Gakenke remaining
Rulindo remainingByumba remaining
4 Rwamagana remaining Kayonza west, allGatsibo west, all
5 Gatsibo east, all Nyagatare east,allKayonza east, all
6 Rutsiro all, south west Kibuye west, allNyamasheke remaining
7 Ngororero all Kibuye east, allGasiza
8 Rutsiro all, except south west Kabaya all9 Nyaruguru remaining Gikongoro all10 Burera all Ruhengeri all
Nyagatare west, all
Rollout plan for PBC in General Health
54
Immediately Phase I Phase IIThese places already have PBC in health centers for non-HIV / A IDS services
HIV / A IDS PBC to be introduced A FTER or SIMULA TEOUSLY with PBC for general services (ie. N OT before)
HIV / A IDS PBC to be introduced A FTER or SIMULA TEOUSLY with PBC for general services (ie. N OT before)
Rulindo Kibungo KireheKamonyi Nyanza KamonyiGisozi Gakenke Gisagara Rulindo Butare ByumbaBugesera Rwamagana KayonzaGasabo Gatsibo NyagatareNyarugenge Nyamasheke KibuyeCyangugu Ngororero Gasiza
Rutsiro KabayaNyaruguru GikongoroBurera Ruhengeri
Rollout plan for PBC in HIV/AIDS services
55
55
Program and Evaluation Roll-Out Plan
Jan-06 Mar-06 Jun-06 2007 Feb-08 Apr-08 May-08 Jul-08Treatment Start of interventionControl
GH Household HIV HouseholdSURVEYS
Timeline
Program Implementation
Impact EvaluationBaseline
FACILITYFOLLOW-UP
Start of intervention
56
Sampling Issues
Law of large numbers does not apply here… Proposed solution:
Propensity scores matching of communities in treatment and comparison based on observable characteristics
Over-sample “similar” communities in Phase I & Phase II It turned out
Couldn’t find enough characteristics to predict assignment to Phase I
Took a leap of faith and did simple stratified sampling
57
57
Analysis Plan
All analyses will be clustered at the district level Compare the average outcomes of facilities and
individuals in the treatment group to those in the control group 24 months after the intervention began.
Use of multivariate regression (or non-parametric matching) : control confounding factors
Test for differential individual impacts by: Gender, poverty level Parental background (If infant : maternal education,
HH wealth)
58
58
Difference in differences models To test the robustness of the analysis Control sample (both observed and unobserved) heterogeneity A two-way fixed effect linear regression:
α γ δ ε= + + + +∑Where:
is an outcome variable for facilitiy i in period t or for individual i who lives in the catchment area of facility i,
is an indicator of whether facility i
it it k itk i t itk
it
it
y PBC B X
y
PBC
δε
is being paid by PBC in period t, are time varying control variables,
i is a facility fixed effect, is a year fixed effect, is an error term.
itk
t
it
X
59
SECTION 4:Impact evaluation Implementation
SECTION 4.1 :Baseline surveys
60
Baseline Survey: Sampling Strategy60
61
The baseline has 4 surveys
December 2005-March 2006: General Health facility survey (166 centers)
Phase 1 : 80 facilities Phase 2 : 86 facilities
General Health household survey (2,016 HH) August – November 2006:
HIV/AIDS facility survey (64 centers) HIV/AIDS household survey (1994 HH)
HIV/AIDS study for another presentation
62
62
General Health Centers Survey:Content
General characteristics Human resources module: Skills, experience and
motivations of the staff Services and pricing Equipment and resources Vignettes: Pre-natal care, child care, adult care
VCT, PMTCT, AIDS detection services Exit interviews: Pre-natal care, child care, adult
care, VCT, PMTCT
63
Baseline Health Facility: Utilization
Num ber of Obs. cstd
Num ber of Obs. cstd
Curative child care 79 307.165 314.925 81 222.247 162.384 -2.151Curative adult care 79 581.608 799.890 81 461.543 632.451 -1.055Child grow th monitoring 74 129.770 227.201 70 152.100 357.436 0.450Prenatal care 77 77.065 45.741 77 76.013 62.327 -0.119Institutional delivery 74 17.041 17.940 74 13.230 12.221 -1.510Home delivery 49 1.408 5.330 49 1.776 7.811 0.272TB treatment 61 2.393 11.960 62 0.403 0.877 -1.307Malaria treatment 79 305.557 238.645 80 234.838 211.342 -1.979VCT 62 127.048 153.189 57 96.860 185.501 -0.971
NUMBER OF CONSULTATIONS IN AUGUST 2005
Services Provided
Phase I Phase II
T-Stat
64
64
Identified sectors and cells served by each of the 164 health facilities in the sample,
Randomly selected four cells from the catchment area,
For each cell, obtained number of zones (10-15 hh) Randomly selected three zones in each cell Obtained household lists for each of the zones Randomly selected one household for each zone Produced random sample of 12 households per health
facility, with a final sample size of 2,016 households.
General Health Household Level: Sampling Method
65
Baseline Field Sampling: GH HH
HEALTH FACILITY
CELL 1 CELL 2 CELL 3
HH 1
CELL 4
HH 4HH 3HH 2 HH 6HH 5 HH 7 HH 9HH 8 HH 10 HH 12HH 11
66
Baseline General Household Sample 2159 HH, 10,880 individuals Average HH size is 5.71 individuals, 75% of the sample is under the age of 30 years old.
(Sampling strategy) Not a nationally representative sample:
Sample of rural households with children < 6 years old
Baseline General Household Content
Socio-economic information Anemia finger prick test: children 12-71 months old Malaria dip stick test: children under age 6 Anthropometrics: <6 years old Mental health: mothers, pregnant women, adults over
age 20 Sexual history and preventative behavior knowledge Pre-natal care utilization and results Parents or caretakers were asked for information
regarding child (<5 years) health status
68
Baseline Household Data: Education68
Sam ple: ages 6 and up Variable Nr. Obs Mean St d. Error Nr. Obs M ean St d. Error T -st atEver at t ended 3104 79.68% 0.011 4042 80.88% 0.010 -0.825No schooling 2458 10.26% 0.007 3252 10.88% 0.007 -0.606At least som e P rim ary 2458 82.93% 0.010 3252 79.97% 0.010 2.195 **At least som e Secondary 2458 6.46% 0.008 3252 8.42% 0.007 -1.841 *At t ended school in last 12 m ont hs 2391 41.13% 0.015 3181 41.34% 0.012 -0.112Able t o read Kinyarwanda 2532 63.59% 0.015 3359 66.37% 0.014 -1.375
P HASE I (Int ervent ion) P HASE II (Cont ro l)
69
Baseline Household Data: Assets
Variable Nr. Obs Mean Std. Error Nr. Obs Mean Std. Error T -st atComplet e sofa set 921 2.18% 0.005 1147 3.08% 0.007 -1.073Radio 921 49.63% 0.023 1147 50.27% 0.017 -0.221Radio-casset t e or music system 921 4.29% 0.009 1147 5.89% 0.008 -1.274Telephone Mobile 921 1.07% 0.005 1147 3.50% 0.007 -2.778 ***Mosquito net s 921 22.53% 0.023 1147 28.00% 0.027 -1.542Sewing m achine 921 0.76% 0.003 1147 1.18% 0.004 -0.883A bed 921 62.82% 0.023 1147 58.71% 0.022 1.290W ardrobe 921 4.53% 0.009 1147 6.07% 0.010 -1.155Metalic library 921 0.57% 0.002 1147 1.28% 0.004 -1.645T able 921 63.47% 0.022 1147 63.40% 0.021 0.021Chair 921 85.48% 0.018 1147 84.82% 0.019 0.255A bicycle 921 16.37% 0.020 1147 17.26% 0.027 -0.262
P HASE I P HASE II
70
Baseline Household Data: Activities of Daily Living (21+ years)
Adults, mothers and pregnant womenVariable Obs Mean std Obs Mean std tstatTotal SampleAccepts to perform ADL 1705 79.71% 0.011 2081 81.28% 0.012 -0.988Nr of seconds for 1 sit-to-stand 1376 8.815 0.796 1680 8.807 0.680 0.008Nr of seconds for 5 sit-to-stand 1373 16.991 0.406 1676 16.161 0.437 1.392Nr of sit-to-stand in previous 1329 4.932 0.019 1619 4.904 0.015 1.114Squat for 30 seconds, seconds 1373 29.194 0.228 1676 28.879 0.231 0.970Balance on right foot, seconds 1374 29.074 0.223 1676 28.278 0.240 2.429 **Balance on left foot, seconds 1373 28.893 0.238 1674 28.190 0.248 2.046 **
Phase I Phase II
71
Baseline Household Data: Prenatal Care71
Variable Nr.Obs. Mean Std.Error Nr.Obs Mean Std.Error T -st atOf all b irths since Jan. 2005In facility birth 1238 32.05% 0.026 1462 34.87% 0.025 -0.777Of most recent pregnancyTimes received PNC 723 2.781 0.046 900 2.687 0.052 1.368Injection to prevent tetanus 728 74.52% 0.023 900 72.56% 0.019 0.654
P hase I P hase II
72
Baseline Household Data: Child Immunization
72
Variable Nr.Obs. Mean Std.Error Nr.Obs Mean Std.Error T -st at
12-23 months oldfully_immunized 262 75.36% 0.03 295 77.04% 0.04 -0.3312-71 months oldfully_immunized 1154 63.77% 0.03 1387 65.28% 0.02 -0.43
P hase I P hase II
73
Baseline Household Data: Child Health Care Utilization (<6 years)
Variable Nr.Obs. Mean Std.Error Nr.Obs Mean St d.Error T -st at
days_sick 1438 1.96 0.12 1717 2.59 0.13 -3.54 ***receive_care 597 25.21% 0.02 803 25.58% 0.02 -0.13times_receive_care 143 1.58 0.10 218 1.44 0.08 1.12cost_fees 135 246.20 43.80 213 285.71 42.82 -0.65cost_supplies 132 93.05 52.04 208 101.43 28.25 -0.14cost_medicine 133 287.77 64.66 207 475.54 76.57 -1.87 *cost_medicine_nopres162 73.48 19.46 283 109.43 30.67 -0.99cost_lab 134 77.13 21.38 208 78.92 21.43 -0.06cost_other 134 21.35 8.28 204 45.29 19.51 -1.13
P hase I P hase II
74
Baseline Household Data:Child Biomarkers (<6 years)
18.67%
7.51%
22.16%
27.82%
19.53%
7.33%
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
Anemic Malaria Fever and Malaria
Phase IPhase II
75
Validity of Sample
Require two different validations: Validate the sampling for the evaluation design
Diff in means tests between Phase I and Phase II to determine if intervention and comparison groups balanced at baseline
Validate the quality of data Compare descriptive stats to other sources of national data
(i.e.: 2005 & 2007 DHS, MOH data)
76
Validity of Sample and Data76
Evaluation design Of 110 key characteristics and output variables of HF, the
sample is balanced on 104 of the indicators.
Of 80 key HH output variables, the sample is balanced on 73
of the variables. Majority of the indicators which differ between Phase I and Phase II are
results from patient exit interview, which is not a random sample. Quality of data
HH Results comparable to the 2005 DHS, MOH data
77
SECTION 4.2 :Program implementation and
monitoring
78
Timeline of Activities
200512 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9
General Health Facility General Health HouseholdHIV/ AIDS FacilityHIV/ AIDS Household
PBF TRAINING PHASE I DISTRICTSPHASE I DISTRICTS: OUTPUTPHASE II DISTRICTS: INPUT
MONITORING DATA COLLECTION
Health FacilityGeneral Health HouseholdHIV/ AIDS HouseholdPHASE I DISTRICTSPHASE II DISTRICTS
2006 20082007
BASELINE DATA COLLECTION
PBF PAYMENTS
FOLLOW UP DATA COLLECTION
PBF TRAINING
79
Monitoring Program Roll-out
Regular participation in the PBF technical committee meetings by the impact evaluation team members
Monitor threat to internal validity of sample from: Political pressure to expand PBF into Phase II districts
before 2008 Many facility directors and providers in Phase II districts
heard of PBF through colleagues or media so attempt to imitate treatment
Ensure exposure to PBC for enough time Avoid contamination: Training of phase 2 started after
data collection for HF in May 2008
80
Monitoring Program Roll-out Additional data collection effort focused on
monitoring PBF roll-out at facility level Date received training Relationship with Comite de Pilotage: Number of
audits conducted Amounts received at facility due to PBF Allocation of PBF to salaries and other
Monitoring helped to ensure the evaluation team understood the actual roll-out
81
Monitoring Program Roll-out Key points for GH analysis
Baseline: Collected prior to training or first payments. Found little evidence of imitation of treatment in Phase
II Follow-up: The MOH initiated a revised training
course for ALL districts in 2008. Phase I districts received March-April 2008, and Phase II
districts received in May 2008 May look at indicators up to March 2008 for all health
facilities as health facility data collection didn’t end until July 2008
82
SECTION 4.3 :Follow up surveys
83
Follow-up surveys February-September 2008 3 surveys:
Combined health facilities survey for General Health - HIV/AIDS
Household survey for General Health (panel data) Household survey for HIV/AIDS (panel data)
84
Follow-up Field Sampling: GH HF Return to 166 facilities
Some GH facilities began offering HIV/AIDS services (VCT, PMTCT and/or ARV) between 2006-2008
Identified in the field and used the HIV/AIDS HF questionnaire; all GH HF questions still asked but in different format
94 (56.63%) GH 2006 & 2008 60 (36.14%) GH 2006; HIV/AIDS 2008 12 (7.23%) incomplete information
85
Follow-up Field Sampling: GH HH
Objective: Return to the same households to create panel data set
(2006-2008) Print baseline roster (names, codes) and keep consistent
across waves Account for household members who left and new
arrivals from 2006-2008
86
Follow-up Field Sampling: GH HH
Return to same households: In total 2159 were suppose to be surveyed in the
catchment area of 167 facilities. 1888 (87%) were interviewed in 2006 and 2008 267 (12%) were replaced 4 (0.2%) were not found and not replaced
% of replacement by region: South (24%), North (14%), East (4%) and West (13 %)
87
Follow-up Field Sampling: GH HH
External reasons: Migration result of 1) avoiding Gacaca, 2) employment in Kigali,
3) famine in South during the last 2 years Decentralization in 2006 renamed some areas in study sample;
impossible to locate based on baseline location Internal reasons:
For some health facilities, the baseline HH team didn’t follow sampling procedure
Given a cell to survey by the SPH team but difficult to reach Used the same cell information but surveyed households in another
area Health facilities with 10-13 hh replaced
88
Follow-up Field Sampling: GH HH
WHAT DOES THIS MEAN FOR ANALYSIS? Restrict to only matched households
1,888 households
89
Assisted delivery – Modern contraceptive use
Meilleures pratiques en SM au Rwanda89Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et intérimaire 2007
90
P ropo rtio nnal Malaria morb id ity in Health C entres vs Health Utiliz ation R ate
73,570,3
67,4
50,4
37,9
28,4
15
2527,4 29,9
37,8
44,4
71,175
0
10
20
30
40
50
60
70
80
2001 2002 2003 2004 2005 2006 2007
Malaria morbidity Health utilis ation rate
91
Infant and Under 5 mortality rate
92
Trend of Maternal Mortality ratio
Source : Demographic and Heath survey 1992, 2000 et 2005.
93
Section 5:Current and next steps
Current and Next Steps
Reformat, clean data bases to create panel data Initial GH HF results: January 2009 Initial GH HH results: March 2009 Plan dissemination workshop in Kigali to discuss
initial results with key stakeholders and TWG
95
QUESTIONS? SUGGESTIONS?
Performance Based Financing
Rwanda : Increase in utilization of services 2006-2008
(average of 206 health centers)
0
50
1 00
1 50
200
250
300
350In
stitu
tiona
lD
eliv
erie
s
New
Cur
ativ
eCo
nsul
tatio
ns
Teta
nus
Toxi
d
Fam
ilyPl
anni
ng N
ew
Fam
ilyPl
anni
ngTo
tal
Health Interventions
%