world relief’s cs health information system ccih m&e workshop may 28, 2005 melanie morrow...
TRANSCRIPT
World Relief’s CSHealth Information System
CCIH M&E Workshop May 28, 2005
Melanie MorrowWorld Relief
Presentation
• Overview of World Relief’s Vurhonga CSP in Mozambique
• Key components of the Vurhonga HIS– Regular surveys to monitor progress– Community HIS
• Tracking mortality
WR Vurhonga “Dawn” CSPs• Vurhonga 1 1995-1999
– Guija & Mabalane Districts– 107,000 population
• Vurhonga 2 1999-2003– Chokwe District– 2350 Volunteers trained in
173 Care Groups– 130,000 (140,000 EOP) pop– C-IMCI + HIV + BS
• Expanded Impact 2004-09– 5 New Districts in Gaza
Province– C-IMCI + HIV
Major Program Components
• Educating and mobilizing the community to prevent illness and seek appropriate treatment
• Creating and training VHCs to address health issues at village level
• Increasing access to care at village level
Vurhonga 2 Care Group Structure
5 Supervisors26 Animators
Care Groups of 10-15
Volunteers 10 HH per
volunteer
Reach: Every HH
One trainer can train & supervise 8 care groups, each with 8-10 volunteers.
Each volunteer is responsible for the 10-15 households on her block.
In Vurhonga 2: 26 trainers reached 24,500 HH via 2350
volunteers trained in 173 Care Groups
Care Group Meetings
• Volunteers verbally report and discuss statistics from the C-HIS
• Problem solve as a group
• Between meetings, Volunteers conduct home visits for the 10-15 HH in their “block”
Animator trains volunteers in interventions using pictures, stories, songs and drama
Home visit
• Volunteers greet family and inquire about their wellbeing
• Address current health concerns in HH
• Teach health lesson learned during most recent care group mtg.
• Make mental note of births, deaths or pregnancies
Other Care Groups
• Churches: Care Groups for pastors to teach BCC they share with their congregations.
• Grannies: Care Groups for grannies ensure support of elders.
Village Health Committees
Membership includes:– Chef de Saude – Village leader– Health Post Socorrista– Care Group leader– Neighborhood reps (max 5)– Church leader– Member of OMM (women’s organization)
Vurhonga HMIS Components
• Full count of beneficiaries at baseline and repeated as needed (can include retrospective birth and mortality questions)
• Baseline and Final KPC Survey• Monitoring surveys to track progress towards
project objectives (every 3-6 months)• Community-HIS (monthly Care Group statistics)
for monitoring vital events
Monitoring Surveys
• Abbreviated version of KPC instrument
• Similar to LQAS in that the sample is based on supervision areas; Random selection of 1 Care Group per animator
• Staff trade supervision areas and interview all HH with children <2 pertaining to that CG (sub sample OK)
• Track progress towards objectives
Monitoring Survey Form
Children 1-2 yrs old that have been fully vaccinated
88% 88% 86%89% 89% 91% 90% 91%
74%
71%59%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sept'9
9
Dec'9
9
Mar
'00
Jun'0
0
Sept'0
0
Dec'0
0
Mar
'01
Jun'0
1
Sept'0
1
Jan'0
2
Mar
,02
Jun'0
2
Sept'0
2
Dec'0
2
Jun'0
3
Jul'0
3
Sept'0
3
Target
Actual
Objective: Increase from 0.3% to 70% use of mosquito nets year-round by all children under 5
0%
79%
68%
81% 81% 85%90%92%95%91% 93%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Target
Actual
Objective: Increase from 28% to 75% children treated within 24h for fever (suspected malaria) at health facility
28%
82%73%
80%86% 89% 90% 92% 96% 97%
90%
0%
20%
40%
60%
80%
100%
120%
Target
Actual
Objective: Increase to 50% children treated within 24h for rapid, difficult breathing at appropriate health facility
76%
63%
91%85%
60%
2%
100%100%100%99%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Target
Actual
Objective: Increase from 1.6% to 40% women who recognize at least 3 signs of STIs
1.60%6% 5%
44% 42% 41%
57% 59%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sept'99
Dec'99
Mar
'00
Jun'0
0
Sept'00
Dec'00
Mar
'01
Jun'0
1
Sept'01
Jan'0
2
Mar
,02
Jun'0
2
Sept'02
Dec'02
Jun'0
3
Jul'0
3
Sept'03
%
Series1
Series2
Data use
• Revise strategies as appropriate (e.g. message re: mosquito nets; new pictures for reproductive health)
• Track performance by staff supervision areas, by individual villages & by district
• Share with care groups, VHCs and MOH to motivate and engage in problem solving
• Identify and respond to problems early
C-HIS (Monthly Care Group Stats)
Care groups form the basis for a sustainable community-HIS. Volunteers verbally report on vital events (births, deaths, pregnancies) that they discuss in their meetings. The information is shared with the community and MOH without dependence on project staff. Village Health Committees (VHCs) and the MOH make decisions using these data. Volunteers are motivated by the measurable impact they are having.
Care Group Statistics
• Volunteers verbally report vital events (births, deaths, pregnancies) for their block
• Mortalities are reported with the following information: Name, sex, age at death, date of illness onset, signs of illness, date of death.
• Literate volunteer records data
C-HIS (cont.)
• Discussion of illness signs during meeting used to determine most likely cause of death.
• “Questions of the month” can be added• Bi-directional learning with Animator• Summary data given to Vurhonga Animator and
to village Socorrista• Project staff together discuss monthly results and
implications during regular meetings, and take action.
Information Flow
.
CSP
Care Group
MOH
VHC
Animator
Socorrista
Sustainability of Info Flow
.
Care Group
MOH
VHCSocorrista
Vurhonga I volunteer retention 20 months post project
• Vols active at end of project: 1457• Vols who left post/moved: (92)• Vols who died: (44)• Replacement volunteers: 40
TOTAL No.VOLS STILL ACTIVE: 1361or 93%– Attrition 6.59%
50% of HH were visited by their volunteer in two weeks before survey
Time Cost of C-HIS Tabulation
• 30 minutes during CG meeting once/month
• 15 minutes for socorrista to compile village-wide data
• 30 minutes for district-wide tabulation at District Hospital
VHCs and CGs take action
• VHCs of 25 de Septembro and other villages noted increase in malnourished children in early 2002.
• Initiated Hearth community nutrition rehabilitation sessions using Care Group volunteers.
• Underweight children decreased from 13% in March 2002 to 7.2% in July despite food shortages.
Community goal setting
• Mapapa VHC noted that 19 HH lacked latrines; Set goal for all HH to have latrines within 3 months.
• 25 de Septembro VHC helped pass a local law requiring any HH that didn’t build a latrine to pay for the labor of others sent by VHC to do it for them.
Community-based accountability
• Muzumia village VHC noted pregnant women not using hygienic delivery huts assisted by TBA
• Data prompted community investigation
• Found TBA was demanding unauthorized payment
• Involved MOH to resolve issue
MOH preparedness
• Increases in diarrhea cases helped the MOH in Chokwe district to anticipate and stave off a cholera epidemic that other districts were unprepared for.
• Community has louder voice when backed by data
Benefits of community-specific data
• Local data used at local level prompted communities to:– Tackle malnutrition– Identify underlying problems– Involve district MOH in relevant matter– Set goals– Pass local laws– And appreciate impact of volunteers
Project benefits of regular HMIS feedback
• Keep project in touch with community
• Motivate staff—can see impact– Results graphed on office wall
• Identify and respond to problems – Personnel– Intervention/communication strategy
• Justify continued funding
Ingredients for Success
• Analysis and application of data by those involved in collecting it
• Only collect what actually use
• Link to lasting community structures (CGs and VHCs)
• Sustained volunteer participation (<2% drop out per year)
Volunteer Incentives
• Examples of tangible incentives – Year one: head scarf– Year two: kapulana traditional skirt– Year three: project T-Shirt
• Intangible incentives– Communication of respect and appreciation– Social support– Community recognition
End Result
As a result of CGs and VHCs using the C-HIS, the community has an effective system for monitoring and governing its own health—as well as interfacing with district MOH authorities.
200
3
FutureGenerations
Reasons to measure mortality rates and
changes in a CS project
Measurement of Mortality Rates and Causes of Mortality
• An Essential Tool for Maximizing Program Effectiveness?
• A CORE Function in Child Survival Programs?
The Initial Three-Tiered Approach to Monitoring and Evaluation
• Tier One: Counting the number of services provided
• Tier Two: Measuring coverage in the project population
• Tier Three: Measuring mortality impact
Arguments FOR Monitoring Mortality
• Is THE key indicator
• Can guide programming/increase program effectiveness
• Motivates staff
• Guides program policy formulation
Arguments AGAINST Monitoring Mortality
• Is too complicated, too time consuming, and takes high-level expertise, and must be carried out by outsiders
• Requires a “control” population• Is too expensive• Takes too many years to achieve impact• Takes a very large population in order to
document significant impact
“Gold Standard” for Demonstrating Mortality Impact
• Have an intervention and comparison area• Show that mortality rates in these two areas were similar
before the intervention• Show that the mortality decreased significantly more in the
intervention area than in the control area• Demonstrate that the mortality reduction should be
attributed to the intervention
Disseminating results
• Village chief and health committee are regularly informed of deaths and involved in discussion to learn from event.
• Trends are shared less often, at most every 6 months.
• Data are aggregated by project staff and (in Mozambique) by MOH, to promote sustainability.
C-HIS Information Flow from Care Groups
CSP
Care Group
MOH
VHC
Animator
Socorrista
Sustainability of data flow from Care Groups to Village
Health Committees & District MOH
Care Group
MOH
VHCSocorrista
Moz: U5 cause specific mortality Chokwe District (Mozambique) under 5 cause specific mortality March 2000-
July 2001
0
10
20
30
40
50
60
Malaria
Diarrhea
Pneumonia
Malnutrition
Cambodia: % of deaths attributable to EPI
EPI preventable deaths
19.921.3
15.3
11.78.6 9.5 8.5
1.8 2.14.2 3
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11
Quarters 2000-2002
Calculating mortality ratesInfant Mortality Rate:# deaths children 0-11 mo/1000 live births
Child Mortality Rate:# deaths in children age 12-59m/1000 live births
Under five mortality rate:• # deaths in children U5/1000 live births
Counts also useful if don’t know # births
Problems with Prospective Tracking
• Under-reporting of births—need to use a pregnancy register to catch all births
• Sensitivity needed to discern when culturally appropriate to visit family without waiting too long so that people forget important details.
• Hard to independently verify if all deaths have been captured.
Retrospective tracking of Mortalities in Mozambique
• Midterm count of all beneficiaries included inquiry about all births and deaths during preceding two years.
• Possible underreporting because– respondents inclined to leave out events that
occurred on the “border” of time asked about (though bounded by flood)
– Less likely to include more distant events
Mortality Data from Census/HIS
119
65
45
70
37
23.9
0
20
40
60
80
100
120
140
3/2000-2/2001 3/2001-2/2002 3/2002-2/2003
U5 Death Rate
Infant Death Rate
Retrospective tracking using pregnancy history
• At final eval, sample of 250 women interviewed about all pregnancies they have had during their lifetime and their outcomes.
• Intervals spanning 3 or more years without a birth were probed for possible miscarriages or unreported mortalities
Mortality Data from Pregnancy Histories
128.2
189.7
33.5
59.869
11.2
0
20
40
60
80
100
120
140
160
180
200
1998-1999 2000 2001-8/2003
Child Death Rate
Infant Death Rate
Comparison of Census vs. Pregnancy History
0
20
40
60
80
100
120
140
160
180
200
Year 2000** Year 2001 Year 2002
Census/HIS
Pregnancy History
**Pregnancy History data for 2000 includes flood deaths
Pregnancy History (cont)
• Pro: get complete history, has been validated in literature for accurate mortality estimates going back ~10 yrs.
Problems with Retrospective Studies
• Recall bias leading to under-reporting• If don’t use own staff, population reluctant to
talk about deaths; if use own staff scientific community reluctant to believe results
• Making lists is sometimes considered a suspect (politically destabilizing) activity
• Cultural definitions of child deaths (e.g. baby not considered a person until reaches a certain milestone or named)
Problems with Retrospective Studies (cont.)
• Determining adequate sample size for retrospective pregnancy history can be difficult
• Sample size can be quite large
• High maternal mortality rates could skew results
• Many confounders
Staff and Volunteers in Mapapa Village
Project Action
• Quarterly survey results showed children's ITN use changed as follows– Sept 1999 0%– April 2001 93%– Aug 2001 68%
• Refocused BCC to promote ITN use even in “winter” months– July 2002 81%
Key Ingredients for Success
• Analysis and application of data by those involved in collecting it
• Only collect what actually use
• Link to lasting community structures (VHCs)
• Sustained Volunteer Participation
Vurhonga I volunteer retention 20 months post project
• Vols active at end of project: 1457• Vols who left post/moved: (92)• Vols who died: (44)• Replacement volunteers: 40• TOTAL VOLS STILL ACTIVE: 1361
– Attrition 6.59%
• 50% of HH visited by volunteer in preceding two weeks
Volunteer Attrition in other WR CSPs
• 13.2% in Cambodia at end of year three (excluding deaths and relocation)– Lack of community identity
• 10% in Malawi at end of year one– Both men and women as volunteers– Association with established health institution
led to expectation of employment
Volunteer Motivation for Vurhonga
• Examples of tangible incentives – Year one: head scarf– Year two: skirt– Year three: project T-Shirt
• Intangible incentives– Communication of respect and appreciation– Social support– Community recognition
Benefits
• Accountability
• Contact with community
• Consensus-building
• Strengthening of partnerships
• Empowering communities to take responsibility for their health