Community-Based Rehabilitation (CBR) Evaluation Framework
Manjula Marella
Co-authors: Ecosse Lamoureux and Jill Keeffe
Centre for Eye Research AustraliaUniversity of Melbourne
Community-based rehabilitation (CBR)
The World Report on Disability identifies CBR as an important component of health systems
Rehabilitation within one’s own community using local resources
The World Health Organization (WHO) guidelines promote a comprehensive system addressing Health, Education, Livelihood, Social participation and Empowerment of people with disabilities
Effectiveness of CBR services
Evaluations published on CBR programs lack evidence-based research
Published evaluations are mainly descriptions of services
Different methodologies and indicators used for evaluations
Common indicators and a systematic framework for evaluation that are valid for different settings of CBR are essential
Evaluation framework
To address the gap in the literature related to effectiveness of CBR programs, the Centre for Eye Research Australia (CERA)-CBR Framework (CCF) has been developed
The CCF includes indicators to comprehensively evaluate CBR programs
CERA – CBR Framework (CCF)
Service Delivery System
Availability of Services
Utilisation of Services
Quality of Services
Program Management
Program Planning
Human Resources
Infrastructure
Coverage
Sustainability
Networking
Linkages
Referral System
Advocacy
Support from families
Support from community
Outcomes
Intended outcomes
Functional independence
Psycho-social
Education
Economic Independence
Empowerment
Changes in Community
Satisfaction with services
Components
Key Areas
Manjula Marella, CERA November 2010
Utilisation of services
Enrolments
Waiting list
Dropouts
Eligibility to enrol
Client demographics
Average number of clients enrolled per year
Waiting time
Number of clients on waiting list
Number of clients who discontinue annually
Gender ratio of dropouts
Reasons for dropouts
Socio-demographic profile of dropouts
Themes/QuestionsKey area Indicators
Developing questionnaires Questions for each indicator Data collection methods and sources Key informant questionnaires
– CBR staff– Clients and Family members– Government (Health and Social Welfare)– Eye care practitioners– Disabled People Organisations (DPOs)– Non-government organisations (NGOs)
Face validity of the CCF
Reviewed by an expert panel (n=7)– CBR– Evaluation methods
To assess the appropriateness of the content and structure of the CCF
The CCF was modified based on the feedback from the panel
Modified version of the CCF included 230 indicators
Field-validation of the CCF Fiji Society for the Blind (FSB) Cambodian Development Mission for Disability
(CDMD)
Field-validation of the CCFFiji Cambodia
Total Population
944,720 14,494,293
Location of services
4 areas of Viti Levu Island 4 provinces (29 districts) in the south
Number of CBR workers
4 20
Types of services
Functional rehabilitation Referrals to hospitals, social welfare department and special education.
Inclusive education program is only for high school students
Functional, vocational and social rehabilitation
Empowerment Referrals to hospitals and inclusive education
Sample size 38 148
Analysis
Collated responses from various stakeholders NVivo 8 Qualitative analysis Strengths and limitations of the CBR programs Usefulness of indicators
– Applicable (relevant to the context of services)– Measurable (feasible to obtain data)
Triangulation – information obtained from different sources and compared
1. Service delivery systemFiji Cambodia
Strengths • Medical model of rehabilitation:
- functional skills training and - referrals to school for the
blind
• Social model of rehabilitation addressing all components in the WHO CBR matrix: health, education, social, livelihood and empowerment
• Regular follow-up and monitoring systems
Limitations • Lack of systematic follow-ups and internal monitoring protocols
• Unable to manage children <5 years
2. Program planning
Fiji Cambodia
Strengths • Saving costs associated with office space and transport by working in collaboration with Social Welfare and Eye departments
• Involvement of external stakeholders to plan activities
Limitations • Activities do not match with the goals of the program
• Poor financial sustainability
• Poor financial sustainability
3. NetworkingFiji Cambodia
Strengths • Collaboration with Social Welfare and Eye departments for resources
• Involving families in rehabilitation of the clients
• Good compliance with referral services
• Involving families in rehabilitation of the clients
• Advocating for inclusion of PWDs
Limitations • Poor collaboration with DPO and government
• Lack of community involvement in CBR program
• Accessibility to referral services
• Poor collaboration with DPO and government
• Lack of community involvement in CBR program
• Negative attitudes of staff at referral centres
4. OutcomesFiji Cambodia
Strengths • Improvement in daily living skills
• Changes in community attitudes
• Inclusion of clients in community as a result of self-help groups and income generation
• High client satisfaction - 35 (81%) clients and 52 (88%)
family members interviewed felt their needs were met
Limitations • Client satisfaction - 4 (36%) clients and 7
(50%) families interviewed felt their needs were met
• Lack of systems to evaluate outcomes of services
• Lack of systems to evaluate outcomes of services
Usefulness of the CCF
Fiji Cambodia Both
Applicable 201 (87%) 219 (95%) 197 (85%)
Measurable 167 (83%) 178 (81%) 156 (79%)
Overall 230 indicators in the framework
Discussion
The CCF is comprehensive and useful to evaluate the effectiveness of CBR programs
First study to compare two models of CBR Comprehensive model of CBR is feasible and more
effectively meets the needs of the clients Recommendations provided based on the findings using
the CCF helped the organisations to plan new services and strategies– Training of staff– Plans for financial sustainability
Application of the CCF
144 (63%) indicators were derived from the literature where they were not proposed for any specific disability
The majority (197, 85%) indicators were applicable to two different settings
The framework is likely to be applicable to different programs irrespective of model and type of disability
Limitations and recommendations for future research
The component ‘outcomes’ needs further validation
Only two case-studies in this study Does not include economic analysis Application in other settings including
developed countries
Summary
The CCF has been developed to evaluate the effectiveness of vision-related CBR programs
Field-tested in two different CBR settings in Fiji and Cambodia
Practical and useful to investigate the strengths and limitations of the two CBR programs
Acknowledgements
Supervisors: A/Prof Ecosse Lamoureux and Prof Jill Keeffe
Fiji Society Blind and Cambodian Development Mission for Disability
Vision Cooperative Research Centre (CRC) University of Melbourne
Thank you