assessing child/household needs and well being_senefeld_5.2.12
TRANSCRIPT
OVC Wellbeing Tool (OWT)
The OWT is based on the following guiding principles:
OWT Guiding Principles
1. Wellbeing from the child’s perspective
2. A valid and reliable measure of wellbeing
3. Age-appropriate
4. Applicable to multiple settings
5. Ease of Use
6. Repeated measure
Background
• CRS technical staff recognized the need for a comprehensive measure of OVC wellbeing
• Goal: create an instrument which can be used internationally to represent holistic OVC programming that is valid, reliable and practical to administer.
OWT Development
The OWT was developed over a two- year time period between 2006 and 2008 through a multi-step process:
1. CRS technical advisors brainstormed on domains of wellbeing
2. Domains subsequently verified through a process of review and comparison against other tools and definitions of wellbeing
3. Self-reported statements generated for each domain using a free listing methodology as well as adaption from other validated tools
4. Statements shared with expert judges within CRS, drawn from fields such as education, social protection, health, and food security
5. Draft tool piloted through a larger evaluation of PEPFAR-funded OVC programs
6. Advanced statistical analyses, along with feedback from the pilot countries further served to refine the OWT
Tool Development
• Originally 48 Questions• Self-Report Measure• Likert Scale• Used for Children Aged 13-18
• 10 Domains of Wellbeing – Nutrition and food security – Shelter and environment – Protection – Family – Health
– Spirituality – Mental health – Education – Economic – Community cohesion
Tool Development: Piloting the OWT
• 5 country evaluation:– Rwanda– Kenya– Zambia– Haiti– Tanzania
Tool Development: Analysis
• Data from 890 OVC from the 5 countries• Compared the OWT data to larger evaluation data• Validated OWT against
Hope Scale
• Statistical Analysis:– Cronbach’s Alpha– Confirmatory Factor
Analysis
Tool Development: Pilot Results
• Cronbach’s Alpha• Confirmatory Factor Analysis
– Reduced number of items in the scale– 48 to 36
Tool Development: Correlating OWT with larger survey
• Based on domains of traditional OVC intervention
• Similar across all five countries, corresponding to OVC sectoral foci
• Generally strong correlation between larger survey and OWT scores
Tool Development: The Children’s Hope Scale
• Stable tool shown to have internal consistency, convergent, discriminant and incremental validity
• Children’s hope conceptualized as positive expectations; agency and pathways
• Consistent with various definitions of resilience• 6 point Likert scale• Premise
• Children are goal directed• Higher “hope” → Increasing levels of agency and pathways thinking
• Validated against Children’s Hope Scale, the OWT showed a Spearman’s Rho of p<.01 for both the original long version and the shortened version.
1Snyder et al (1997). The Development and Validation of the Children’s Hope Scale. Journal of Pediatric Psychiatry 22(3), 399-421.
Tool Development: Challenges
• Hope Scale Validation
• Translation• Health domain
revisited• Data management
Tool Development Conclusion
• OWT now finalized– Reduced number of items
in the tool from 48 to 36 items
– Required approximately 20 minutes to complete
• End Result: A valid, practical tool to monitor OVC wellbeing according to the interventions that are being implemented.
Tool Use: Where and Who
• CRS: Botswana, Kenya, Rwanda, Haiti, Tanzania, Lesotho, Zambia, Malawi, Vietnam, Ethiopia, DRC, Pakistan, India
• Other agencies: Western Africa, Kenya, Tanzania, Malawi, India, Haiti, Botswana
Tool Use: How the OWT information should be used
• To monitor OVC programs at an aggregate level to identify patterns of change in OVC wellbeing within projects. – Weaker domains should be explored – Strengths identified
• NOT an in-depth assessment tool at the individual level!– However, rapid scoring can highlight need for
follow-up and more in-depth assessment with children who report significant problems.
Strengths and Challenges of such a tool
• Strengths:– Child’s perspective– Holistic– Universal tool – Age appropriate– Rapid– Low cost
• Challenges:– Universal tool– Translation accuracy– Eliciting information from children on sensitive
subjects
Feedback from the Children
• Initial feedback was critical in the adaptation of the tool (e.g. questions eliminated based on their feedback)
• Currently=Enthusiastic• Have said they are pleased
to be able to give their opinion on matters relating to them
• No current reported problems understanding the questions on the OWT
OWT Application: Kenya
• In April 2008, OWT baseline administered for 633 participants in OVC program in Nyanza Province
• Program Areas– Education and Vocational Training– Health– Psychosocial Support
• The OWT re-administered to 345 of these same OVC in March 2009
OWT Application: Kenya
OWT domain scores before and after one year of OVC programming
OWT Application: Kenya
OWT domain scores before and after one year of OVC programming
OWT & CSI
Some Comparisons Suggest Criterion Related Validity
• Measures consistent
1. Food security and economic issues
2. Health and wellness
3. Mental health
• Measures seem to assess different constructs
1. Shelter – CSI ratings seem to refer to physical structure; child self report seems related to care from adults
2. Child Protection – OWT child feels safe and treated like others, whereas CSI observes possibility of abuse, legal needs.
Latest work: adaptation for younger age group
• Two OVC programs, in Kenya and Rwanda, administered an adapted version of the OWT to children aged 7-12 (N=593; n=288 Kenya and n=305 Rwanda).
• The original OWT used a three-point Likert scale, which was easy for adolescents to understand, but rendered sensitivity detection regarding change over time more difficult. Thus, the current pilot included a five-point Likert scale response set (i.e., never true, rarely true, sometimes true, often true, always true).
• To ensure children understood the Likert scale responses, pictorial representations of the responses were developed, along with standardized enumerator instructions for explaining items, item response options, and query responses.
• Challenges in developing images that represented measurements in a culturally-appropriate way. Pre-testing demonstrated visual representation of cultural measurements varies from one context to the next; different graphics ultimately were used in Kenya and Rwanda.
Latest work: adaptation for younger age group (2)
• 56 items; stats TBD fewer items for final tool• 87.5% of enumerators (Kenya) & 89.1% in Rwanda
reported overall experience with the tool for this age group as either “great” or “good”
• 80.1% of enumerators Rwanda and 88.2% in Kenya reported that their experience with the pictorial representations was “great” or “good”
• All items were reported as easily understood by at least 70% of surveyed children. The majority of items (~80%) required no additional explanations beyond the initial instructions.
Thank youContact:
Dr. Shannon Senefeld
Catholic Relief Services
http://www.crsprogramquality.org/ovcwt/Senefeld, S., Strasser, S., Campbell, J., & Perrin, P. (2011). Measuring adolescent wellbeing: The development of a standardized measure for adolescents participating in orphan and vulnerable children programming. Journal of Vulnerable Children and Youth Studies, 6:4, 346-359.
Tool developed by a team of staff, led by Shannon Senefeld, Susan Strasser, and James Campbell.
Please note that the photographs in this publication are used for illustrative purposes only; they do not imply any particular health status (such as HIV
or AIDS) on the part of the person who appears in the photograph.