community-based rehabilitation (cbr) evaluation framework manjula marella co-authors: ecosse...

22
Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia University of Melbourne

Upload: dominick-black

Post on 24-Dec-2015

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Community-Based Rehabilitation (CBR) Evaluation Framework

Manjula Marella

Co-authors: Ecosse Lamoureux and Jill Keeffe

Centre for Eye Research AustraliaUniversity of Melbourne

Page 2: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Community-based rehabilitation (CBR)

The World Report on Disability identifies CBR as an important component of health systems

Rehabilitation within one’s own community using local resources

The World Health Organization (WHO) guidelines promote a comprehensive system addressing Health, Education, Livelihood, Social participation and Empowerment of people with disabilities

Page 3: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Effectiveness of CBR services

Evaluations published on CBR programs lack evidence-based research

Published evaluations are mainly descriptions of services

Different methodologies and indicators used for evaluations

Common indicators and a systematic framework for evaluation that are valid for different settings of CBR are essential

Page 4: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Evaluation framework

To address the gap in the literature related to effectiveness of CBR programs, the Centre for Eye Research Australia (CERA)-CBR Framework (CCF) has been developed

The CCF includes indicators to comprehensively evaluate CBR programs

Page 5: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

CERA – CBR Framework (CCF)

Service Delivery System

Availability of Services

Utilisation of Services

Quality of Services

Program Management

Program Planning

Human Resources

Infrastructure

Coverage

Sustainability

Networking

Linkages

Referral System

Advocacy

Support from families

Support from community

Outcomes

Intended outcomes

Functional independence

Psycho-social

Education

Economic Independence

Empowerment

Changes in Community

Satisfaction with services

Components

Key Areas

Manjula Marella, CERA November 2010

Page 6: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Utilisation of services

Enrolments

Waiting list

Dropouts

Eligibility to enrol

Client demographics

Average number of clients enrolled per year

Waiting time

Number of clients on waiting list

Number of clients who discontinue annually

Gender ratio of dropouts

Reasons for dropouts

Socio-demographic profile of dropouts

Themes/QuestionsKey area Indicators

Page 7: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Developing questionnaires Questions for each indicator Data collection methods and sources Key informant questionnaires

– CBR staff– Clients and Family members– Government (Health and Social Welfare)– Eye care practitioners– Disabled People Organisations (DPOs)– Non-government organisations (NGOs)

Page 8: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Face validity of the CCF

Reviewed by an expert panel (n=7)– CBR– Evaluation methods

To assess the appropriateness of the content and structure of the CCF

The CCF was modified based on the feedback from the panel

Modified version of the CCF included 230 indicators

Page 9: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Field-validation of the CCF Fiji Society for the Blind (FSB) Cambodian Development Mission for Disability

(CDMD)

Page 10: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Field-validation of the CCFFiji Cambodia

Total Population

944,720 14,494,293

Location of services

4 areas of Viti Levu Island 4 provinces (29 districts) in the south

Number of CBR workers

4 20

Types of services

Functional rehabilitation Referrals to hospitals, social welfare department and special education.

Inclusive education program is only for high school students

Functional, vocational and social rehabilitation

Empowerment Referrals to hospitals and inclusive education

Sample size 38 148

Page 11: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Analysis

Collated responses from various stakeholders NVivo 8 Qualitative analysis Strengths and limitations of the CBR programs Usefulness of indicators

– Applicable (relevant to the context of services)– Measurable (feasible to obtain data)

Triangulation – information obtained from different sources and compared

Page 12: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

1. Service delivery systemFiji Cambodia

Strengths • Medical model of rehabilitation:

- functional skills training and - referrals to school for the

blind

• Social model of rehabilitation addressing all components in the WHO CBR matrix: health, education, social, livelihood and empowerment

• Regular follow-up and monitoring systems

Limitations • Lack of systematic follow-ups and internal monitoring protocols

• Unable to manage children <5 years

Page 13: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

2. Program planning

Fiji Cambodia

Strengths • Saving costs associated with office space and transport by working in collaboration with Social Welfare and Eye departments

• Involvement of external stakeholders to plan activities

Limitations • Activities do not match with the goals of the program

• Poor financial sustainability

• Poor financial sustainability

Page 14: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

3. NetworkingFiji Cambodia

Strengths • Collaboration with Social Welfare and Eye departments for resources

• Involving families in rehabilitation of the clients

• Good compliance with referral services

• Involving families in rehabilitation of the clients

• Advocating for inclusion of PWDs

Limitations • Poor collaboration with DPO and government

• Lack of community involvement in CBR program

• Accessibility to referral services

• Poor collaboration with DPO and government

• Lack of community involvement in CBR program

• Negative attitudes of staff at referral centres

Page 15: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

4. OutcomesFiji Cambodia

Strengths • Improvement in daily living skills

• Changes in community attitudes

• Inclusion of clients in community as a result of self-help groups and income generation

• High client satisfaction - 35 (81%) clients and 52 (88%)

family members interviewed felt their needs were met

Limitations • Client satisfaction - 4 (36%) clients and 7

(50%) families interviewed felt their needs were met

• Lack of systems to evaluate outcomes of services

• Lack of systems to evaluate outcomes of services

Page 16: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Usefulness of the CCF

Fiji Cambodia Both

Applicable 201 (87%) 219 (95%) 197 (85%)

Measurable 167 (83%) 178 (81%) 156 (79%)

Overall 230 indicators in the framework

Page 17: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Discussion

The CCF is comprehensive and useful to evaluate the effectiveness of CBR programs

First study to compare two models of CBR Comprehensive model of CBR is feasible and more

effectively meets the needs of the clients Recommendations provided based on the findings using

the CCF helped the organisations to plan new services and strategies– Training of staff– Plans for financial sustainability

Page 18: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Application of the CCF

144 (63%) indicators were derived from the literature where they were not proposed for any specific disability

The majority (197, 85%) indicators were applicable to two different settings

The framework is likely to be applicable to different programs irrespective of model and type of disability

Page 19: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Limitations and recommendations for future research

The component ‘outcomes’ needs further validation

Only two case-studies in this study Does not include economic analysis Application in other settings including

developed countries

Page 20: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Summary

The CCF has been developed to evaluate the effectiveness of vision-related CBR programs

Field-tested in two different CBR settings in Fiji and Cambodia

Practical and useful to investigate the strengths and limitations of the two CBR programs

Page 21: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Acknowledgements

Supervisors: A/Prof Ecosse Lamoureux and Prof Jill Keeffe

Fiji Society Blind and Cambodian Development Mission for Disability

Vision Cooperative Research Centre (CRC) University of Melbourne

Page 22: Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia

Thank you