aaces mid-term review (2014)

Australia Africa Community

Engagement Scheme (AACES)

MID‐TERM REVIEW

OF THE OXFAM

AACES WATER, SANITATION AND HYGIENE

PROGRAMIN SOUTH AFRICA

AND ZAMBIA

Date: March 2014

Research and report written by:

Margaret Roper, South Africa

South Africa MEL Review by:

Maud Mukova‐Moses, Australia

Table of Contents

ACKNOWLEDGEMENTS ........................................................................................................................... 3

ACRONYMS & ABBREVIATIONS ............................................................................................................... 4

GLOSSARY OF TERMS .............................................................................................................................. 5

EXECUTIVE SUMMARY ............................................................................................................................ 7

1. INTRODUCTION AND PURPOSE OF THE REPORT .......................................................................... 17

2. BACKGROUND TO THE AACES PROGRAM IN SOUTH AFRICA AND ZAMBIA ................................. 18

2.1 Background ................................................................................................................................. 18

2.2 Objectives of the AACES program implemented by Oxfam ........................................................ 18

2.3 Program approach in South Africa and Zambia .......................................................................... 19

2.4 Overview of the AACES WASH partners in South Africa and Zambia ......................................... 20

3. MID‐TERM REVIEW APPROACH, METHODOLOGY, LIMITATIONS AND VALIDITY ......................... 23

3.1 Approach ..................................................................................................................................... 23

3.2 Study design ................................................................................................................................ 23

3.3 Sampling plan .............................................................................................................................. 25

3.4 Field work and data collection .................................................................................................... 25

3.5 Data analysis plan ....................................................................................................................... 26

3.6 Limitations and validity ............................................................................................................... 26

4. FINDINGS OF THE MID‐TERM REVIEW .......................................................................................... 27

4.1 Assessment of relevance, effectiveness, efficiency and sustainability of the AACES WASH program............................................................................................................................................. 27

4.1.1 Relevance ............................................................................................................................. 27

4.1.2 Effectiveness ........................................................................................................................ 34

4.1.3 Efficiency and Value for Money approach ........................................................................... 44

4.1.4 Sustainability ........................................................................................................................ 48

4.2 Monitoring, Evaluation and Learning ...................................................................................... 56

4.2.1 South Africa .......................................................................................................................... 56

4.2.2 Zambia .................................................................................................................................. 59

4.2.3 Learning and sharing between Oxfam WASH AACES partners ............................................ 60

5 Assessment towards achieving AACES program objectives ......................................................... 62

AACES Objective 1: Marginalised people have sustainable access to the services they require ..... 62

AACES Objective 2: DFAT policy and programs in Africa are strengthened particularly in their ability to target and serve the needs of poor and vulnerable people .............................................. 67

Oxfam AACES MTR MARCH 2014 Page | 2

AACES objective 3: Increased opportunity for the Australian public to be informed about development issues in Africa. ........................................................................................................... 72

6. Success factors, challenges and gaps ................................................................................................ 73

6.1 Success factors ............................................................................................................................ 73

6.2 Challenges and areas for improvement ...................................................................................... 75

7. Recommendations to Improve Future Delivery ........................................................................... 77

7.1 Oxfam AACES Program level recommendations ......................................................................... 77

7.2 Partner and country level recommendations ............................................................................. 82

8. Conclusion ..................................................................................................................................... 85

APPENDIX 1: Oxfam AACES WASH MTR SEMI‐STRUCTURED INTERVIEW QUESTIONNAIRE ................ 87

REFERENCES .......................................................................................................................................... 90

ACKNOWLEDGEMENTS

The researcher would like to thank all who participated in this mid‐term review for their time,

information, willingness and openness in sharing their experiences, views and insights into the

program. In addition, to thank the Oxfam country offices for the support provided during the field

visits.


ACRONYMS & ABBREVIATIONS

AACES Australia Africa Community Engagement Scheme AFAP African Fertilizer and Agribusiness Partnership AFMAC Africa Mining and Communities ACFID Australian Council for International Development CADECOM Catholic Development Commission CLTS Community Led Total Sanitation CREATE Community Based Rehabilitation Education and Training for Empowerment DFAT Department of Foreign Affairs and Trade (Australia) ECD Early childhood development FS Fancy Stitch KZF Keepers Zambia Foundation LIMA LIMA Rural Development Foundation MEL Monitoring and evaluation MDIC Maputaland Development and Information Centre MEL Monitoring, evaluation and learning MTR Mid‐Term Review NGO Non‐government Organisations ODA Official Development Assistance OVSA OneVoice South Africa PSS People’s Participation Services RAPCAN Resources Aimed at Preventing Child Abuse and Neglect SMEC An international engineering company STC KZN Save the Children, KwaZulu‐Natal TREE Training and Resources in Early Education TU Tholulwazi Uzivikele WASH Water, sanitation and hygiene WM Woza Moya YWCA Young Women’s Christian Association ZAFOD Zambia Federation of disability organisations ZAPD Zambia Agency for People with Disability

NOTE ON CODING IN THE REPORT: Coding of the evidence is included in the report to confirm validity of the findings and triangulation of data. For purpose of confidentiality the coding sources are not included. All references starting with R refer to South Africa respondents, and Z refer to Zambia responses.


GLOSSARY OF TERMS12

Beneficiaries Individuals, groups or organisation’s that benefit directly or indirectly from the development intervention*

Capacity The knowledge, skills and competence to actEffectiveness The extent to which a program/intervention has achieved its objectives under

normal conditions in a real‐life setting** Efficiency A measure of how economically inputs (resources such as funds, expertise, time)

are converted into results** Evaluation The process of determining the worth or significance of an activity, policy or

program. An assessment, as systematic and objective as possible, of a planned, ongoing or completed intervention*

Induna A Zulu word meaning leader or advisorInputs The financial, human, and material resources used in a program/intervention**Intervention An instrument for donor or non‐donor support aimed to promote development*

specific activity or set of activities intended to bring about change in some aspect(s) of the status of the target population (e.g., HIV risk reduction, improving the quality of service delivery)**

Marginalized people A group of people that have been excluded or moved through a social process to the edge of society due to gender, age, race, geographic location or other factor

MEL Framework A document prepared at entry to enable performance assessment of an intervention* and to learn from the intervention

Monitoring A continuing function that uses systematic collection and analysis of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention, with indicators of the extent of progress and achievement of objectives and progress in the use of allocated funds*

Objective Intended impact contributing to the physical, financial, institutional, social, environmental, or other benefits to a society, community, or group of people via one or more interventions* a statement of a desired program/intervention result that meets the criteria of being Specific, Measurable, Achievable, Realistic, and Time‐phased (SMART)**

Outcome The likely or achieved short‐term and medium‐term results of an interventions outputs*

Outputs The products, capital goods and services delivered by a development intervention to direct beneficiaries*

People with a disability "Persons with disabilities include those who have long‐term physical, mental, intellectual or sensory impairments which in interaction with various barriers may hinder their full and effective participation in society on an equal basis with others." (Article 1, UN Convention on the Rights of People with Disabilities)

Program An overarching national or sub‐national response to a community development issue, human right or disease. A program generally includes a set of interventions marshalled to attain specific global, regional, country, or sub‐national objectives; involves multiple activities that may cut across sectors, themes and/or geographic areas**

Project An intervention designed to achieve specific objectives within specified resources and implementation schedules, often within the framework of a broader

1 Source of definitions with a *: Australian Government MEL Framework Good Practice Guide (exposure draft: March 17, 2006) 2 Source of definitions with a **: UNAIDS Working Document: MEL Definitions (www.unaids.org/en/media/.../11_ME_Glossary_FinalWorkingDraft.pdf ) accessed 18 February 2014


program** Relevance The extent to which the objectives, outputs, or outcomes of an intervention are

consistent with beneficiaries’ requirements, organisations’ policies, country needs, and/or global priorities**

Reliability Consistency or dependability of data collected through the repeated use of a scientific instrument or a data collection procedure used under the same conditions**

Stakeholder A person, group, or entity who has a direct or indirect role and interest in the goals or objectives and implementation of a program/intervention and/or its evaluation**

Sustainability Sustainability (of a program)—the likelihood that political and financial supportwill last to maintain the program**

Theory of change The building blocks required to bring about a long‐term goal usually described or depicted as a pathway towards change

Triangulation The analysis of data from three or more sources obtained by different methods. Findings can be corroborated, and the weakness or bias of any of the methods or data sources can be compensated for by the strengths of another, thereby increasing the validity and reliability of the results**

Value for Money Benefits or value of the program; investigated through examining the impact achieved for the money invested in the program

Water, sanitation and hygiene (WASH) services

Services including but not limited to access to clean water, hygiene facilities and appropriate sanitation; improved facilities; increased WASH knowledge and awareness; participation of communities and groups in decision‐making around WASH; provision and delivery of services related to WASH by duty‐bearers3

A review of the Oxfam AACES WASH Program Design document indicates that the stakeholders

include:

• Oxfam management and staff in Australia, South Africa and Zambia, as well as DFAT and the AACES implementing partners in Africa

• Oxfam partners including the directors and programme staff (which includes community development officers or facilitators, home based and community caregivers, mentors and teams or groups within the organisation that may not have a direct role to play in the WASH intervention)

• Beneficiaries of the WASH interventions including individual women, men and children; households; villages or neighbourhoods; community groups such as a women’s community gardening group;

• Civil society community structures including WASH committees, school management teams, School Governing Bodies, ECD Forums, community led decision structures, Parish Councils

• Traditional leadership at both individual level (Chief or Induna) and Council level

• Local (municipal), district, regional, provincial and national government structures

• Broader society and citizens primarily in South Africa, Zambia and Australia

3 Oxfam AACES WASH MEL Framework, November 2012


EXECUTIVE SUMMARY

1. Introduction

The Australian Department of Foreign Affairs and Trade (DFAT), through the Australia Africa

Community Engagement Scheme (AACES), funds the Oxfam Community Led Access to Water,

Sanitation and Hygiene (WASH) program in South Africa and Zambia. Oxfam is a global confederation

of seventeen affiliates dedicated to fighting poverty and related injustice around the world, through

working together to achieve greater impact by their collective effort. The Oxfam global program in

South Africa and Zambia is being implemented in targeted communities, where common themes

present an opportunity for linked programming and learning towards positive, sustainable change in

the lives of people living in poverty.

The overall purpose of the Mid‐Term Review (MTR), conducted in year three, is to make an

assessment of how the program is tracking against its objectives, and to identify gaps and areas for

improvement. The review included field research in South Africa and Zambia, and built on completed

MEL products such as baselines, program reports, research reports and case studies to further

provide evidence for program development and learning.

2. Background

The program is being implemented over a five year period (2011‐2016) and aims to enable

Australian NGOs and their partners to contribute to the DFAT strategy for Africa, through a

partnership program, focused on community‐based interventions across the sectors of water and

sanitation, food security, and maternal and child health.

The AACES objectives are:

1. Marginalised people have sustainable access to the services they require;

2. DFAT policy and programs in Africa are strengthened particularly in their ability to target and

serve the needs of poor and vulnerable people; and

3. Increased opportunity for the Australian public to be informed about development issues in

Africa.

The Oxfam AACES program focuses on water, sanitation and hygiene (WASH) and public health in

Zambia and South Africa. Both countries have large rural populations where WASH services are

lacking despite progressive policies and the good intentions of both country governments. The

overall goal of the program is to improve the health and quality of life of the poor and vulnerable in

targeted areas of Zambia and South Africa.

Oxfam’s approach to implementing AACES is a rights‐based, participatory and capacity‐building

approach that aims to support community members and social change organisations to improve


their communities’ health and quality of life. Implicit in the formulation of the AACES objectives is a

rights‐based argument that the duty to deliver essential services such as WASH rests with the state

and that citizens should have a voice and active role to play in the design, implementation,

governance and monitoring of these initiativesi.

In Zambia the program builds on Oxfam’s experience of implementing WASH programs and is

aligned with Zambian Government standards and institutions’ efforts to provide services over the

full five year period. The approach is to use WASH as a lever to help communities increase their

capacity and engage in opportunities to achieve their development aspirations – and thereby

shifting from a service delivery approach to one of enabling citizens to participate in their own

development and engage with governance processes. In South Africa the program is delivered

through pre‐existing non‐governmental partners across the three existing programs (food security,

HIV and child protection). The program also focuses on advocacy and capacity building to leverage

opportunities to increase impact, and influence policy in the region, across the continent and

globally.

3. Mid‐term review methodology

The MTR was designed to be participatory to enable stakeholders and beneficiaries to participate in

the design, data collection, analysis and reporting phases of the review. Feedback on the results of

the MTR will be shared with stakeholders, and where possible an opportunity for dialogue and

discussion on the findings will be held.

The research tools were designed by the external reviewer, and were reviewed by the Oxfam team

to ensure that the rights of the beneficiaries were not being compromised. An administrative guide

was developed, based on the Oxfam Ethics Guidelines, to standardise the approach to implementing

the tools across all sites, and to confirm that the research was conducted ethically. Informed consent

was gained prior to interviews, focus groups, and at the beginning of site visits. Purposeful and

convenience sampling was used. The sample included: Oxfam country staff; implementing partner

directors, management teams and program staff; beneficiaries in the AACES program in selected

sites in South Africa and Zambia; and service providers and technical experts who have been

engaged in elements of the program. The research methodology included a desk top review, key

informant interviews and focus groups. Data was collected through field visits in South Africa and

Zambia. The data was primarily qualitative and was triangulated across the three tools, across

different stakeholder findings, and across the different geographic sites.

The scope of the study was large and field work was limited due to time constraints, difficulty in

engaging with duty bearers in South Africa, and long travel distances to sites. The findings are


deemed as valid due to: the range of beneficiaries included in the sample; the triangulation of data

collected and analysed; and ethical adherence during the research process. Evaluation judgments

are made based on the evidence, and having given due consideration to Oxfam reflection on the

findings, implementing partner views and the evaluator’s understanding of the purpose, goals and

outcomes of the AACES WASH program.

4. Findings of the mid‐term review 4.1 Assessment of relevance, effectiveness, efficiency and sustainability of the AACES program

Relevance: Beneficiaries were involved through the partners in identifying and prioritizing their

WASH needs and program activities. Baselines were conducted in both countries and beneficiaries

were involved in this process. However, partners in South Africa indicated that it was not easy to

translate the Participatory Rural Appraisal baseline findings into activities that aligned to the AACES

program and their work, and therefore additional activities were required to be undertaken. All

partners required capacity building to address WASH issues across the full spectrum of

infrastructure, community engagement, increasing access for marginalised groups, and addressing

WASH governance and accountability. Technical support and capacity building were provided or

facilitated by Oxfam. Beneficiaries were involved in the design, planning, development of activities,

implementation and facilitating engagement with stakeholders. However, a further shift is required

to capacitate beneficiaries to take on more of a leadership, ownership and implementation role. The

approach of the Oxfam AACES program reflects the national political context. At local level, across

both countries, the approach and activities have had to respond to the local political, cultural and

social contexts. The engagement of beneficiaries with duty‐bearers (including traditional leadership,

governance structures at ward, district, province or national level) has been limited as the focus has

been on building capacity. However efforts have been made to develop working relationships or

engage with duty‐bearers, and to influence policy ‐ for example through the collective action of the

ECD Crèche Forum’s in South Africa, the work of MDIC with ward structures, and engagement by

Oxfam in Zambia with District officials and structures. Evidence is not available at this stage of the

program to determine the influence of the work on policy. A key finding is that partners and

beneficiaries need further capacity building to develop skills in advocacy and influencing policy.

Effectiveness: The evidence from the MTR, as presented in section 4.1.2 of the report, indicates that

the program is contributing to significant change in the lives of women, men, boys, girls and persons

with a disability. These changes are noticeable at a number of levels, and across the spheres of

change as articulated by Rao and Kelleher’s ‘Integral Framework’ which suggests that for long term

sustainable change the social systems and institutions that determine the distribution of power and

goods must be transformed (refer to the Oxfam AACES design document). Changes in knowledge,


awareness, beliefs and capacity were found at both the partner and beneficiary level; access to

WASH services has increased; efforts are being made to reduce WASH inequalities and this is

facilitating and contributing to changes in access, awareness and community structures’ ability to

respond; and there is increased engagement in WASH governance structures which is bringing about

clarity on roles, tasks and increased knowledge – however actual change at this level still needs to be

demonstrated. The evidence indicates that partner activities are contributing to improved health

outcomes and quality of life for children at crèches and at school, and at households of beneficiaries.

There is participation in the program and decision‐making structures at community level by men and

women; however the strong cultural approach towards men making decisions remains. There is

debate amongst partners as to how gender should be mainstreamed into the program, and further

development of this underlying theme is required. In Zambia, work on reducing gender inequalities

is being implemented together with local activities of a national campaign against gender based

violence by an Oxfam partner. In South Africa clarity is required as to how this element can be

strengthened. Significant efforts are being made in both countries to address inclusivity of people

with disabilities. Children and youth are involved in the program and the training in child protection

in Zambia appears to have had an important influence on participant attitudes and treatment of

children.

Efficiency and value for money: A number of examples from practice are emerging to illustrate

value for money in the AACES WASH program. However further research and analysis is required to

determine the significance of these examples, and to develop a clearer understanding of how value

for money approaches can be integrated into the program. The focus groups with the beneficiaries

in South Africa and Zambia suggest that the project is improving their wellbeing. Wellbeing is

definedii broadly to encompass various personal and social domains such as happiness, health, sense

of purpose, and safety. Responses from the MTR participants tended to focus on health aspects of

wellbeing, such as improved nutrition, increased amount of food; social aspects such as better

relationships between parents and children; an increased awareness of rights related to well‐being

such as an increase in water, health, the inclusivity of people with disabilities, and a sense of safety

for self and others (particularly for children who had fallen down wells in the past in KZN).

The evidence suggests that for sustainability and the achievement of the expected outcomes for the

program, the focus cannot only be on capacity building or infrastructure: without capacity building

the infrastructure cannot be maintained and the community remains dependent on external

resources to respond to shocks and the achievement of health outcomes, and without basic

infrastructure (such as a working water pump or water storage tanks) to enable communities to

overcome barriers to achieving short term outputs, the long term objectives will not be met.


Similarly, the program cannot only focus on infrastructure service delivery without emphasizing and

engaging in building capacity to do advocacy and influence duty‐bearers.

Sustainability: The sustainability approach for the Oxfam AACES WASH program is articulated in the

program design document. Sustainability is contingent on three building blocks: increasing the

capacity of men and women to claim their rights, respond to external trends and shocks, influence

those with power and to hold duty bearers to account; building the capacity of strong and well‐

functioning structures and institutions; and through Oxfam’s partnership strategy. Training,

mentoring, support, monitoring and oversight by partners is contributing to the development of

men and women, as well as community committee structures, to claim their rights, respond to

trends and to engage with duty‐bearers. ‘Communities of Practice’ are emerging between partners

and community groups, for example the two Crèche Forums, to learn and share from each other and

to develop themselves as individuals and as a collective group. Community structures are being

capacitated to function and respond to WASH and other developmental issues, however further

development and strengthening of these structures is required to build greater sustainability. There

is a need to clarify the financial transfer strategy for the WASH program at both the Oxfam level, as

well as at individual partner level. Emerging lessons from the withdrawal of two partners in South

Africa provide valuable insights into the need to have a clear strategy for handover and to build on

existing structures and practice –and that sustainability requires a long‐term investment in people

and structures. There are multiple levels of partnerships in the Oxfam AACES WASH program and in

general these partnerships are working collaboratively and along the Oxfam principles of good

practice. The MTR identifies elements where the partnership practice could be strengthened, this

includes clarifying roles and responsibilities; strengthening communication and planning; sharing of

resources and working more collaboratively across partners in each country.

4.2 Monitoring, Evaluation and Learning (MEL)

A MEL framework was developed at the start of the program to guide design, implementation,

monitoring, reporting and dissemination plans and activities. In South Africa, the approach taken by

Oxfam was to bring in a technical consultant to build the capacity of the implementing partners to

undertake baseline studies, strengthen MEL systems, build capacity to document practice and collect

relevant data, to use the results of their MEL systems to inform decision making and program

implementation, and to encourage partners to share learning, lessons, innovation and practice at

annual reflections. In Zambia, Oxfam decided to have a dedicated MEL officer involved in the

program to build capacity of partners, coordinate and quality assure data collection, reporting and

dissemination.


Partner’s perceptions of training and MEL support were generally positive in both countries. The

changes in MEL practice in the organisations in South Africa were largely attributed to the provision

of tailor‐made support to each organisation as well as opportunities for cross‐learning. In Zambia,

further capacity building is needed at partner level to strengthen MEL practice. There is improved

data collection, but all partners need further support in analysing data and using this data to inform

program and organisational development. Further engagement and understanding of the AACES

MEL Framework is needed and how partners can collect and use qualitative data more effectively.

There is a need to review the tools being used across partners at country level, and to review the

current reporting template in order that relevant data is being reported on across partners and

countries to meet the overall program reporting requirements. There is now a need to build the

capacity of beneficiary structures in monitoring systems and skills to contribute to the sustainability

of the program.

The Oxfam AACES program includes an objective to document and share learning in order to inform

policy, public engagement and program development and growth. Space has been created for

exchanging ideas, sharing experiences, reflecting on progress and practice, bringing in technical

consultants and innovative practices, in order for partners to learn, share and adapt ideas in their

programs. These have taken the form of annual reflections, training (such as the Disaster Risk

Management training), exchange and site visits have been undertaken between partners. Partners

indicate that these spaces are ‘incredibly valuable’ and have added enormous benefit to the growth

and development of themselves, their organisations and the programs.

Engagement amongst the AACES partners in South Africa and Zambia was generally viewed

positively, with respondents indicating that they had opportunities to share experiences and

establish new ways of working. The establishment of an AACES partners’ network in South Africa, in

which Oxfam is not a participant, was applauded as a great initiative amongst the partners and a

good indication of the sustainability of linkages and working relationships developed across the

partner portfolio. While all partners observed that there were frequent opportunities to interact, the

smaller organisations4 expressed that the workshops and cross learning events were time

consuming and tended to take partners away from their core work. One of the respondents in South

Africa noted that they sometimes felt intimidated, as a small organisation, to be seen as not actively

4 Perception of organisation size is based on respondents’ own interpretations.


participating in learning initiatives. Suggestions were made to reduce the length of workshops, and

encourage exchange visits to partners within the same province5.

5. Assessment towards achieving AACES program objectives

AACES objective one: Marginalized people have sustainable access to the services they require

The program is on‐track in meeting the Oxfam AACES WASH objectives one, two and three. Efforts

during year three will continue to increase the focus on Oxfam objectives three and four of the

program now that the ground work has been laid. This assessment is based on the results as

reported in the Oxfam AACES Six Monthly Snapshot Reportsiii, and the evidence from the MTR as

presented in the report (refer to Section Five). Areas that need strengthening include partner

clarity on the focus of the program and the tension between infrastructure/information and

advocacy; a focus on increasing the voice, advocacy and influencing skills of beneficiaries and

partner staff in the program; bringing the rights‐based approach to the forefront of the program

activities; and conducting an in‐depth study on the value for money approach and reviewing the

budget allocations in terms of ensuring efficiency of resources.

AACES objective two: DFAT policy and programs in Africa are strengthened particularly in their ability to target and serve the needs of poor and vulnerable people.

Since the inception of the AACES program in South Africa and Zambia, a number of activities have

been undertaken regarding Oxfam’s policy engagement with DFAT. The Oxfam approach has focused

on three core areas of A) policy dialogue relating directly to issues arising from the programs in

South Africa and Zambia, B) policy dialogue on issues broader than the Oxfam AACES program, and

C) regional and or international policy engagement on issues arising from, or of relevance to, the

programs of Oxfam and one or more other Australian NGOs working in Africa. A summary of the key

activities undertaken over the past two and a half years is provided in Section Five. Respondents

indicated that the success of Objective Two requires the participation of DFAT and Oxfam, and

despite efforts by Oxfam the response and level of engagement is not always forthcoming.

AACES objective three: Increased opportunities for the Australian public to be informed about development issues in Africa

Activities focusing on Objective Three were planned to start in year two of the AACES programiv.

Consequently, the focus in years one and two was on designing activities for implementation during

the final three years of the program. In year two efforts were focused on the development of the

concept for an Information Communications Technology (ICT)v approach to engage supporters in

5 Maud Mukova‐Moses, Oxfam Australia.


Australia with partner organisations, and potentially beneficiaries, in South Africa. The purpose is to

pilot a tool to engage Oxfam, the Australian public and program beneficiaries in a dialogue about

development issues in South Africa, including those relating to disability and WASH and to

demonstrate how Australian aid (through Oxfam’s AACES funded work) is seeking to respond to

these challenges. In year three the WASH AACES program in Zambia will be profiled at the

international WASH Conference to be held in Brisbane, Australia, in March 2014 in a joint poster

presentation with other AACES partners (CADECOM, CARITAS, AFAP and WaterAID).

A key focus in year one and two was on supporting a process to develop an all‐of‐agency Theory of

Change for in‐Australia community engagement. One of the purposes of engaging in this internal

work was to ensure that AACES could contribute to the existing agency priorities for community

engagement work in Australia, and to ensure that the AACES program was working with the relevant

internal units to implement the work rather than establishing parallel systems within Oxfam that did

not align with the organisation’s overall objectives. This work proved useful (as it informed the

current ICT project and the fruitful collaboration with the Youth Engagement Team, particularly the

Schools Program), but was a challenging process mainly because WASH is not a priority for Oxfam’s

public engagement in Australia {except for fundraising purposes which cannot (and should not) be

supported with AACES money}. In some instances, possibilities for public engagement by Oxfam in

Australia relate to sectors that AACES funds do not support in South Africa and Zambia (HIV/AIDS)

and/or relate to campaign activities (Food Justice or Climate Change) and, therefore, cannot be

supported under AACES Objective three.

6. Success factors, challenges and gaps

Success factors include the continuous engagement between partners and beneficiaries particularly

through relevant community structures which has provided a solid foundation on which the program

has grown; the approach of integrating WASH into existing programs and partnerships; innovation

which is being demonstrated across the partners and different sites including tippy taps, the ECD

Crèche Forums, saving clubs, the mapping of water points and the development and roll‐out of

WASH materials for crèches and schools; and the emerging collaboration between partners at

country level. The capacity building of partners is contributing to the delivery of activities, placing

WASH as a foundation of existing work, and is likely to contribute to the long‐term commitment of

partners to addressing WASH needs.

Challenges include the need to understand how the activities contribute to sustainability and

developing a clearer strategy as to how sustainability will be achieved; a tension over the past two

years between the primary focus on infrastructure/hardware vs. information/software in the

approach and activities of the partners, and more recently a debate in how to incorporate advocacy


and influencing work in the partner activities; a need to strengthen engagement with duty‐bearers

and build advocacy and influencing skills and practice of partners and beneficiaries; building the

capacity of stakeholders to manage and implement WASH programs; the integration of gender as a

cross‐cutting theme has not successfully been addressed in the program activities; and to strengthen

the feedback to technical advisors and how the program is developing based on decisions made as a

result of their engagement.

The gaps include the difficulty of bringing in new partners to meet program needs, such as partners

who can add value to the program e.g. an advocacy partner or partners to replace the two that left

the program at the end of year three, due to the belief of DFAT that the new partners would take

too long to come up to speed with the ways of working and the compliance issues of the AACES

program. The taking on of an advocacy partner in year three to build the capacity of partners to

implement advocacy activities makes absolute sense.

7. Conclusion and recommendations to improve future delivery

It is the researchers view, based on the findings of the MTR, now that the overall progress of the

Oxfam AACES program is at the stage where partners working at community level have a) the

capacity (knowledge, skills, practice), b) emerging lessons from implementation over the past two

and half years, c) the increased awareness of enablers and blockages that influence activities

towards achieving the program goals, and d) an understanding of the need to engage with duty

bearers on policy or development issues, that the focus can shift to how individually and

collaboratively they can achieve the long‐term vision and goal of the Oxfam AACES program. The

partner’s activities are relevant to the beneficiaries needs and are effectively contributing to

significant and sustained changes in the lives of women, men, boys, girls and marginalised groups,

particularly in terms of increasing access to WASH for people with disabilities. The findings suggest

that the program is adding value and there is efficient use of resources, however further research is

required to determine the value for money of the program, and to maximise the use of resources.

For example, a study examining infrastructural responses that maximise the return on the

investment, and that demonstrate how capacity building approaches can bring about change, could

add value to understanding efficiency and inform the program theory of change. There are

numerous emerging examples of value for money and innovation that should be documented to

share practice, influence policy and duty‐bearers accountability, and to leave an evidence‐base for

communities and partners to continue pursuing long‐term solutions to WASH beyond the AACES

WASH program. Examples of these are outlined below.

The MTR assessment indicates that the program is working towards achieving the three AACES

objectives, and significant progress has been achieved. Elements of the program that need further


strategic development and a focus on strengthening implementation over the next two years

include:

• Continuing to build partner and community capacity in monitoring and evaluation;

• Supporting communities to engage deeper in all aspects of the program cycle and objectives;

• Articulating strategies to ensure program sustainability at partner and community level;

• Reviewing and deepening the understanding of the theory of change;

• Strengthening advocacy and influencing work of partners and communities to engage effectively with relevant duty bearers;

• Continuing to share information, lessons and practice from the AACES program in South Africa, Zambia, and Australia with civil society, donors, governments, and accountability or leadership structures to inform and influence policy and practice.

The recommendations below aim to guide program activities to address these elements at both the

program and partner levels.

Oxfam AACES program level recommendations:

a) Review the understanding of the Theory of Change to strengthen program delivery

b) Strengthen strategy and practice for advocacy, rights based approach, holding duty‐bearers to

account and including communities in all stages of the program cycle

c) Strengthen program monitoring, evaluation, evidence‐base, documentation and learning

d) Clarify management structures, roles and responsibilities with partners

e) The program requires flexibility to bring on board additional partners to meet both the changing

needs of the program and to provide the extra capacity or skills to implement the AACES

program in South Africa and Zambia as it adapts and responds to the needs of the communities

on the ground.

Partner and country level recommendations:

a) Strengthen coordination and integration of partner activities

b) Finalise the two country advocacy strategies highlighting the linkages and areas of possible

collaboration and provide resources to strengthen the implementation of the advocacy strategy

and components of the rights based approach (including defining clearly the advocacy roles of

beneficiaries, partners and Oxfam)

c) Strengthen the capacity and skills of communities and beneficiaries across the program cycle,

including developing and mainstreaming appropriate gender responses into the program

d) Continue to build the skills and capacity of partners in MEL, and extend this to community

structures

e) Develop clear sustainability strategies and plans

f) Workshop the program’s theory of change


INTRODUCTION AND PURPOSE OF THE REPORT

The Australian Department of Foreign Affairs and Trade (DFAT), through the Australia Africa

Community Engagement Scheme (AACES), funds the Oxfam Community Led Access to Water,

Sanitation and Hygiene (WASH) program in South Africa and Zambia. Oxfam is a global confederation

of seventeen affiliates dedicated to fighting poverty and related injustice around the world, through

working together to achieve greater impact by their collective effort. The Oxfam global program in

South Africa and Zambia is being implemented in targeted communities, where common themes

present an opportunity for linked programming and learning towards positive, sustainable change in

the lives of people living in poverty. The program is being implemented over a five year period

(2011‐2016) and aims to enable Australian NGOs and their partners to contribute to the DFAT

strategy for Africa, through a partnership program focused on community‐based interventions

across the sectors of water, sanitation and hygiene, food security and maternal and child healthvi.

The purpose of this report is to document the findings from the Mid‐Term Review (MTR) which was

conducted in Year Three of the AACES WASH program in South Africa and Zambia. The overall

purpose of this review was to make an assessment of how the program is tracking against its

objectives, and to identify gaps and areas for improvement. The review included field research in

South Africa and Zambia, and built on completed MEL products such as baselines, program reports

and case studies to further provide evidence for program development and learning.

This report provides the background and conceptual framework of the AACES WASH program

(Section 2), details the review methodology and limitations (Section 3), and presents the findings

from the review (Section 4). Thereafter an assessment of the program against the AACES objectives

is provided (Section 5) and a summary of the successes, challenges and gaps from the findings

(Section 6). The final section provides recommendations for strengthening the Oxfam program in

South Africa and Zambia (Section 7).

The primary users of the report will be Oxfam and partner organisations. In addition, the report will

contribute to the DFAT program level MTR. The MTR will be used in a number of ways:

• For learning to inform decision‐making and program development by Oxfam and partners, and

specifically to inform Year Four Annual Plans;

• Accountability requirements (to DFAT, partners and communities/beneficiaries); and

• To inform and influence stakeholders.


BACKGROUND TO THE AACES PROGRAM IN SOUTH AFRICA AND ZAMBIA

2.1 Background

The overarching goal of the DFAT AACES program is to enable Australian NGOs and their partners to

contribute to the DFAT strategy for Africa through a partnership program focused on community‐

based interventions across the sectors of water and sanitation, food security, and maternal and child

health.

The AACES objectives are:

1. Marginalised people have sustainable access to the services they require;

2. DFAT policy and programs in Africa are strengthened particularly in their ability to target and

serve the needs of poor and vulnerable people; and

3. Increased opportunity for the Australian public to be informed about development issues in

Africa.

2.2 Objectives of the AACES program implemented by Oxfam

The Oxfam AACES program focuses on water, sanitation and hygiene (WASH) and public health in

Zambia and South Africa. Both countries have large rural populations where WASH services are

lacking despite progressive policies and the good intentions of both country governments. The

overall goal of the program is as per the AACES WASH program: to improve the health and quality

of life of the poor and vulnerable in targeted areas of Zambia and South Africa.

The Oxfam objectives for the program are:

1. Increased access to, and effective use of, improved integrated and sustainable WASH services;

2. Reduced WASH‐related inequalities in gender and vulnerable groups;

3. Strengthened capacity of stakeholders to manage and implement WASH programs on a

sustainable basis;

4. Improved WASH governance and effectiveness; and

5. Documentation and shared learning informs policy and public engagement, program

development and growth.

Oxfam’s approach to implementing AACES is a rights‐based, participatory and capacity‐building

approach that aims to support community members and social change organisations to improve

their communities’ health and quality of life. Implicit in the formulation of the AACES objectives is

a rights‐based argument that the duty to deliver essential services such as WASH rests with the


state and that citizens should have a voice and active role to play in the design, implementation,

governance and monitoring of these initiativesvii.

AACES Objective Two and Three (noted above) are respectively covered within the Oxfam Objectives

4 and 5 with policy engagement with DFAT occurring alongside broader policy engagement to ensure

improved WASH governance and effectiveness. Australian community engagement will occur around

issues and successes experienced within the program.

2.3 Program approach in South Africa and Zambia

In Zambia the program builds on Oxfam’s experience of implementing WASH programs and is

aligned with Zambian Government standards and institutions’ efforts to provide services over the

full five year period. The approach is to use WASH as a lever to help communities increase their

capacity and engage in opportunities to achieve their development aspirations – and thereby

shifting from a service delivery approach to one of enabling citizens to participate in their own

development and engage with governance processes. In South Africa the program is delivered

through pre‐existing non‐governmental partners across the three existing programs (food security,

HIV and child protection). The program also focuses on advocacy and capacity building to leverage

opportunities to increase impact, and influence policy in the region, across the continent and

globally.

Diagram 1: Overview of project process year one to three

In year one the program

identified the specific needs

across current programs in

South Africa and in the

districts in Zambia, with the

key learning, pilot activities

and research informing the

year two and longer term

integrated WASH strategy.

Project implementation & monitoring

Implementation of the AACES WASH program

Engage duty

bearers

Project design, development & testing

Community structures

Demo activities Capacity building

In year two, the focus of the program was on building capacity of partner organisations,

implementing activities based on the needs assessment process, strengthening relationships and

building or strengthening structures, and engaging with communities and marginalised groups. In

year three, the focus was on deepening the interventions, strengthening community and

stakeholder engagement, consolidating the approach, and beginning to mobilize beneficiaries to call

for and engage with duty‐bearers on their WASH rights.


Although not explicitly stated in any documents reviewed, the program approach over the five‐years

is based on the following phases: identifying needs through situational analysis and collecting

baseline data aligned to the Oxfam AACES MEL Framework; strategic design of partner level

interventions; the development and testing of different interventions; implementation and

monitoring; and finally an evaluation to determine impact. Inherent in the program design is the

expectation that the WASH program and responses at country level will be sustained.

2.4 Overview of the AACES WASH partners in South Africa and Zambia

The table below provides a brief overview of the focus of the core partner’s work and the approach

they are taking towards implementing the AACES WASH program.

Table 1: Overview of Oxfam AACES WASH partners in South Africa and Zambia

Country Partner Core program approach

South Africa

Fancy Stitch (FS)

(Left program end of year 2)

For the last ten years Fancy Stitch has focused primarily on improving the livelihoods of over 400 women through income generating activities as well as providing health and subsistence support. The organisation markets exquisite hand – embroidered products nationally and internationally. The organisation’s integrated support and the women’s active participation in the process materialized in increasing access to water for extended vegetable gardening activities during the winter months; increase access to water, sanitation and hygiene services in the Ingwavuma community on an individual as well as collective basis.

Zambia Keepers Zambia Foundation (KZF)

Keepers Zambia Foundation was established in 1996 as a response to increasing levels of poverty and deprivation among peri‐urban and rural households in Zambia. They aim to improve people’s livelihoods through sustainable utilisation of natural resources.

South Africa

LIMA Rural Development Foundation (LIMA)

Lima Rural Development Foundation (Lima) is a non‐governmental and non‐profit organisation established in 1989 with the aim of developing and improving the livelihoods of rural communities. To this end, Lima has been involved in various rural development activities ranging from health, water and sanitation, agricultural and general development facilitation in various communities. Lima’s AACES WASH work includes extending water as well as food security related projects to two communities. The objectives of the initiative are to reduce the vulnerability of orphans and other community members providing food to eat, water to drink and a livelihood; the provision of clean water and food to people living with HIV and AIDS and the provision of infrastructure for people to work, in their food gardens which may have other indirect effects such as reducing crime as


people will have means to produce food for themselves.

South Africa

Maputaland Development and Information Centre (MDIC)

(Left program end of year 2)

Local business centre, coaching and mentoring; social enterprise development and investment; rural development, community facilitation, community mobilisation and organizing; food security and poverty evaluation; HIV and AIDS awareness; research development and knowledge management; information and communication technology training. In terms of WASH its aims were to increase access and management of water resources at schools (installation of rain water infrastructure, establish water committee for each school, provide health trainings to capacitate sensitize and empower pupils and communities with key health messages around water, sanitation and hygiene.

South Africa

One Voice (OV) OneVoice South Africa (OVSA) is a non‐governmental organisation which uses creative ways of actively involving young people in schools in HIV and AIDS mainly schools from low socio‐economic status in KwaZulu Natal. The program provides young people with a platform to discuss and address: HIV, STIs and TB prevention; enhance their life skills so they can be better equipped and assertive in decision making pertaining to sexual reproductive health rights; gender, human rights so to promote healthy living. OVSA aim was to integrate elements of WASH into the life skills including HIV, STI’s and TB prevention educational program to grade 8 students, reaching fifteen. The project aims to positively support learners with a greater understanding of hygiene and sanitation and its link to the disease burden such as HIV and AIDS, STI’s and TB and other sexual and reproductive health and rights issues.

Zambia People’s Participation Services (PPS)

Peoples’ Participation Service (PPS) has been working with rural communities of the Western Province of Zambia to help improve the social and economic situation of the disadvantaged population. PPS aim to empower rural communities to address issues related to WASH, HIV and AIDS, natural resource management and food security.

South Africa

Save the Children KZN (STC KZN)

Save the Children KwaZulu‐Natal (SCKZN) established 51 years ago is dedicated to addressing the rights and needs of children in KwaZulu‐Natal. SCKZN developed a Forum model which uses planned outcomes rather than activity driven methods to develop the required capacity of its members, share ‘good’ practices and take collective action when necessary. SCKZN is implementing a WASH program specifically designed for the young child, ECD Service, family of the child and community at large where the ECD Service is situated

South Africa

Tholulwazi Uzivikele (TU)

Originally started with only a Home Based Care Program, but today offers a broad spectrum of programmes targeting all age groups in the area. These programmes encompass community based orphan care, early childhood development,


paralegal assistance, HIV prevention, youth development and community volunteerism. The WASH intervention aim is to increase knowledge on water and hygiene issues in eight communities in Northern uMhlabuyalingana.

South Africa

Woza Moya (WM)

Woza Moya (WM) is a HIV and AIDS community care and support organisation. Woza Moya’s mission is to provide quality care and support for people infected and affected by HIV and AIDS with a particular focus on children and their care givers. By providing home based care, child and youth care, food security and paralegal services, Woza Moya aims to alleviate poverty and suffering through an integrated approach to development. In relation to the WASH program, the partner has focused on innovative WASH technologies and developing the community’s knowledge, skills, awareness and capacity around related WASH issues which includes issues of maintenance to ensure sustainability of the new WASH infrastructures/ community investments.

In both countries additional support and capacity is provided through the services of specialized

organisations including Training and Resources in Early Education (TREE), the Zambia Association for

People with Disabilities, Community Based Rehabilitation Education and Training for Empowerment

CREATE, Resources Aimed at Preventing Child Abuse and Neglect (RAPCAN) and Young Women’s

Christian Association (YWCA), and technical expertise is brought in at specific points in the program

through sub‐consultants such as SMEC (South Africa) and Village Water (in Zambia). The role of

these organisations and individuals is to undertake specific research tasks, develop specific

materials, facilitate dialogue or activities, or to provide additional implementation support such as

increasing awareness of disability, child protection or gender, or to build technical skills. The

strategic approach is to build the capacity of the implementing partners to mainstream issues or

material use in their programs and WASH responses at community level. The involvement of these

service providers is detailed in this report.


MID‐TERM REVIEW APPROACH, METHODOLOGY, LIMITATIONS AND VALIDITY

The overall purpose of the Oxfam AACES mid‐term review (MTR) review is to make an assessment of

how the program is tracking against its objectives, and to identify gaps and areas for

improvement.

The objectives of the MTR are to:

• Assess progress against program objectives (at both Oxfam and partner levels) and ensure that these continue to be relevant for both South Africa and Zambia.

• Provide an assessment of the relevance of the Oxfam AACES program, and assess progress against effectiveness, efficiency and sustainability.

• To identify success factors and challenges in ensuring more equal access to services (of both the program and wider services) by women, men, and people with disability.

• Identify gaps and areas for improvement in both individual partner programs and Oxfam’s approach to the AACES program.

• Provide recommendations to address any issues identified for the remainder of the AACES program to improve future delivery.

3.1 Approach

The evaluation approach was participatory as the MTR aimed to enable stakeholders and

beneficiaries to participate in the design, data collection, analysis, and reporting phases of the

review. Feedback on the results of the MTR will be shared with stakeholders, and where possible an

opportunity for dialogue and discussion on the findings will be held (for example at the Oxfam

partner annual reflection event).

The research tools were designed by the external reviewer, and were reviewed by the Oxfam team

to ensure that the rights of the beneficiaries were not being compromised. An administrative guide

was developed, based on the OAU Ethics Guidelines, to standardise the approach to implementing

the tools across all sites, and to confirm that the research was conducted ethically. Informed consent

was gained prior to interviews, focus groups and at the beginning of site visits.

It was not deemed necessary to apply for formal ethical approval from an institution or government

in South Africa and Zambia as programmatic data was collected. No sensitive or personal data was

collected during the mid‐term evaluation.

3.2 Study design

The following study design process was undertaken:

1. Desk top review of literature


The first phase involved the review of literature of the program and country approaches, objectives,

challenges and successes. Literature included the program design document, country level program

reports, annual work plans, annual reports, research reports, case studies, baseline reports and the

snapshot reports.

2. Identification of stakeholders and sample

The primary program and evaluation stakeholders for South Africa and Zambia were identified

through the desk top review and discussions with Oxfam staff. The stakeholders were invited to

participate in the MTR through email, telephone or in person. The purpose, process and involvement

of beneficiaries were explained. Feedback on the mid‐term evaluation process and findings will be

presented and discussed with partners at the Oxfam AACES annual reflection. Feedback to

community participants will be given through the news brief and information sessions given by the

implementing partners in each country.

3. Development of research tools

A mixed method approach to data collection, using primarily qualitative data, was used. The

methods included:

a) Key informant interviews using a semi‐structured interview guide based on the research questions (refer to Appendix 1). Interviews were conducted with:

i. Oxfam country staff in South Africa, Zambia and Australia

ii. The director or head of the country implementation organisations, and with the program management team

iii. Country or district level stakeholders including duty bearers, traditional leaders, leaders of civil society organisations or community groups/structures

iv. Other key informants such as water experts, WASH community development researchers who have been involved in the AACES program, crèche forum chairperson, and school principals.

b) Focus group discussions with beneficiaries in each of the key geographic sites (e.g. UMkhanyakude and Uffafa in South Africa, Mongu and Kaoma in Zambia), particularly to gather data on gender, perceptions of success and challenges, participation, sustainability and value. In addition, a one‐on‐one interview was conducted with one person in Zambia and one in South Africa to gain empirical evidence from the perspective of people with disabilities.

c) Participatory tools were available for use with representatives from partner and implementation organisations to gather evidence on the extent to which the program is on track to achieve its objectives. The tool aimed to provide a platform for discussion on the effectiveness, relevance, efficiency and sustainability of the program to‐date. However due to time constraints in the distances to travel to partners, translation requirements and that data was gained through the semi‐structured interview guide, these tools were only used with two partners.


4. Data collection, field work and analysis

The data was collected during field visits in December 2013 to Zambia, and January/February 2014 in

South Africa. Data was analysed February 2014. Further detail is provided below on the field work

and analysis.

5. Draft report

A draft report was presented in February 2014 to Oxfam staff from South Africa and Zambia, and

circulated to staff in Australia. The purpose of the report and presentation was to present the

findings from the MTR; validate the data analysis; discuss the implications of the findings on the

program; and to discuss options on action to take to inform the recommendations.

6. Final report

The report was finalized and edited in March 2014.

3.3 Sampling plan

Two sampling methods were used:

i. Purposeful sampling was used for the key informant interviews and partner participatory

tool as these were dependent on the specific role and evidence that identified resources

could provide;

ii. Convenience sampling was used to select beneficiaries as they were selected on the basis of

availability to attend the focus group discussion on the day selected.

These samples included women, men, children and people with disabilities to ensure participation in

the mid‐term evaluation.

3.4 Field work and data collection

Data was collected during field work in Zambia and South Africa. Ethical considerations and practice

was upheld during the field work.

Date Country District/site Purpose 16‐20 Dec 2013 Zambia Western District,

Mongu and Kaoma

Stakeholder and partner semi‐structured interviews Field work with beneficiaries Interviews with Oxfam staff

13‐17 January 2014 South Africa Office based, South Africa

Key informant interviews with stakeholders and consultants

20‐25 January 2014 South Africa KwaZulu‐Natal, Uffafa and eThekwini

Stakeholder and partner semi‐structured interviews Field work with beneficiaries Interviews with Oxfam staff

2‐5 February 2014 South Africa KwaZulu‐Natal, UMkhanyakude

Stakeholder and partner semi‐structured interviews Field work with beneficiaries


In addition, a separate external review of the MEL in South Africa was undertaken as the researcher

provided MEL technical assistance to the South African AACES WASH partners during year two and

three.

3.5 Data analysis plan

The data collected was primarily qualitative. Extensive field notes were taken during the Key

Informant Interviews (KII), focus groups and participatory community dialogues. Data relevant to the

core MTR questions was entered into excel spreadsheets in order for triangulation across the three

tools, across different stakeholder findings, and across the different geographic sites. During the

report write‐up, validation of data was undertaken by cross‐referencing to the original transcripts.

The transcripts and data analysis spreadsheets are kept in a secure location in the office of the

researcher.

3.6 Limitations and validity

The following limitations and constraints of the MTR are noted:

a) The MTR was undertaken under tight timeframes due to conducting field work in two countries,

across the implementing partners, technical advisors and stakeholders. More time was required

in each site to implement the participatory tools, engage further with program staff, and

interview additional stakeholders.

b) The scope of the MTR and the research questions was extensive, and therefore priority

questions relevant to the key informant were selected, and during the interview further

questions were asked to probe for deeper insights and qualitative data. The original scope of the

MTR was too broad to conduct a value for money study, and this was kept at a high level. In

addition, the questions were adapted to meet the respondent’s language and literacy

competence. Questions were translated from English into local language to ensure participation

of beneficiaries, and responses were then translated back into English.

c) Distances to travel to visit communities were far, and due to the quality of roads and weather,

these had to be limited to sites near to the implementing partner offices. In addition, due to the

rains having started in Zambia, one site visit was cancelled and interviews were conducted

telephonically. One site visit was cancelled in UMkhanyakude due to the interview at a school

taking longer than expected.

d) KII with government duty bearers in South Africa were not undertaken. Partners indicated that

they find it difficult to arrange meetings for themselves – and therefore it was not possible to


arrange for interviews. The researcher was unable to contact relevant officials in the

departments during the research phase.

e) The study is primarily a qualitative study and quantitative data was not validated. Quantitative

data was sourced from the existing AACES WASH reporting mechanisms and reports.

The findings are deemed as valid due to the range of beneficiaries included in the sample, the

triangulation of data collected and analysed, and ethical adherence during the research process.

Evaluation judgments are made based on the evidence, and having given due consideration to

Oxfam reflection on the findings, implementing partner views and the evaluator’s understanding of

the purpose, goals and outcomes of the Oxfam AACES program.

FINDINGS OF THE MID‐TERM REVIEW

4.1 Assessment of relevance, effectiveness, efficiency and sustainability of the

AACES WASH program

This section provides the evidence and an assessment of the relevance of the Oxfam AACES

program, and assesses progress against effectiveness, efficiency and sustainability.

4.1.1 Relevance

Relevance refers to the extent to which participant priorities and needs are reflected in the program;

an assessment of how effectively communities, partners and stakeholders have been involved in all

stages of program cycle; and the ways Oxfam strategies coincide with or reflect the political context

at the local or national level.

Beneficiaries’ priorities and WASH needs

The AACES WASH program in South Africa and Zambia has reached a number of different

beneficiaries through direct engagement, and the program has benefited a secondary level of

beneficiaries. As illustrated in the diagram below, there are multiple inter‐connected levels of

beneficiaries.


Diagram 2: Multiple levels of beneficiaries

Community, neighbors, leadership,

governance

Children, youth, women, household members,

committees

Community Caregivers, school facilitators, ECD

Internal staff

Common across five of the six partners in South Africa was the need to build internal capacity on

WASH (knowledge) prior to developing relevant strategies to meet individual partner responses

aligned to their core business, as well as to their participant needs and priorities. For the sixth

partner, LIMA, had existing WASH technological and process knowledge and experience. LIMA

therefore supported and took the lead in conducting the Participatory Rural Appraisal (PRA) with

each of the initial five implementing partners. These partners found the PRA beneficial as it exposed

them to the WASH context in their communities, provided an opportunity to learn from LIMA about

different toilets, water sites, access to water and infrastructure in the community, and provided a

means to engage with community members on WASH. As one respondent in the MTR field work

said: “it was extremely helpful as we found out things” (RD).

However, partners struggled to translate the PRA findings into feasible, relevant and appropriate

responses that were driven by their existing programs and focus areas. Consequently, partners

undertook additional activities to identify community, household or their existing beneficiaries

needs and priorities. For example Woza Moya undertook community dialogues, MDIC consulted

local leadership and government, and Tholulwazi engaged communities through their participatory

drama process. A further activity was conducting the AACES WASH Baseline study which included

gathering district data available to the public, baseline surveys (such as the WASH Crèche Survey

conducted by Save the Children KZN, the WM school survey, and the WASH Household Survey

conducted by WM, MDIC and Fancy Stitch), consolidating information from community engagement

processes.


These processes provided the partners with a good understanding of what the needs and priorities

were at a community and household (or school or crèche) level. In general, as indicated in the AACES

WASH PRA Reportviii, the priorities and needs focused on the lack of resources and accessible or

working infrastructure, lack of knowledge about WASH practices, and in some communities a lack of

water.

Involvement of beneficiaries in the program cycle

The Oxfam partners in South Africa were involved in identifying the WASH problems and needs in

the communities in which they work; in planning and designing individual responses that met their

core business as well as the WASH needs: in developing and implementing materials or activities to

respond to the WASH issues; in managing the activities and facilitating engagement with the

beneficiaries, stakeholders and duty‐bearers; in the project management, finance, monitoring and

reporting of their WASH projects; and in the review of their progress. Oxfam provided technical

support in the areas of research pieces (that highlighted the intersection of WASH with HIV/AIDS,

food security, child protection and gender, and that mapped current relevant WASH materials

available), disability, MEL, child protection, exchange visits, site visits, access to conferences,

exposure to innovative technology, developing and disseminating crèche material (through TREE6)

and school based material (through One Voice), and linking partners to further technical or training

support.

Communities and beneficiaries participated at different stages of the program cycle depending on

the partner approach to community engagement, and the stage of the program over the past two

and a half years. All partners included beneficiaries in identifying community WASH needs, the

context of WASH services and in the development of the baseline study. How the partner responded

to these needs varied as the response had to be aligned with the programmatic approach of the

partner, for example by integrating WASH into the training and practice of community health

workers, ECD training, or through engagement with school management teams. Beneficiaries

participated in the activities, and there is a gradual shift towards community ownership and

commitment of the activities as capacity is being built, community members are involved in the

responses (for example by digging trenches for water pipes at Fancy Stitch), and structures are

taking on maintenance and monitoring roles (for example at schools).

6 The Training and Resources in Early Education (TREE) was contracted to develop WASH education materials for teachers and children in early childhood development centres.


In Zambia, the communities were also involved in identifying the WASH needs and priorities. The

method differed in that communities identified a person who was trained as a community facilitator,

and with the support of PSS and KZF, the community facilitator undertook a participatory process in

the community to develop a ward plan. The ward plan however not only focused on WASH, rather

on a broader community development perspective. As a result, the plans include needs such as

increased access to clinics, improved classrooms, and better roads. In addition, Village WASH

Committees7 (V‐WASH Committees) were established or existing committees strengthened to

provide leadership and ownership to the ward plans. The findings of the MTR indicate that these

committees are beginning to work well (i.e. good communication, meet regularly, members are able

to share their opinions) but further support by PSS and KZF is needed for the committees to move

from the planning stage to implementing action plans. The two focus groups (ZM, ZN) held with

committee members indicated that there is a dependency on the partner organisation and Oxfam to

provide and maintain infrastructure, to take their priorities to duty bearers and to lead the changes.

This is not unexpected given that these community facilitators were trained in mid‐2012 and have

only been operating for eighteen months, and that there is ongoing capacity development being

provided to the community facilitators. However, a barrier to the effectiveness of the community

facilitators, and the committee’s engagement with duty bearers, is the long distances in rural areas

from households to government offices, lack of infrastructure (particularly sand and mud roads,

poor telecommunications) and transport for them to engage with duty bearers.

Political context of country based programs

The engagement and involvement of government structures differs across South Africa and Zambia,

depending both on the Oxfam country approach as well as the political, governance and legislative

context of WASH. In South Africa the focus of the program was on integrating WASH into partner

programs and activities at the community or household level through direct engagement with

partner beneficiaries, while in Zambia the focus has been on supporting government’s programs

through partner implementation.

In Zambia, Oxfam has a history of engaging and working with government structures, and the WASH

AACES program was taken forward based on this foundation. The existing relationships with

Ministries, Provincial, District and Area Commissioners meant that a collaborative agreement could

be discussed and agreed to at the start of the program. Regular meetings and contact is maintained

with these duty bearers, and even if they change, there is established communication and

7 These are pre‐existing committees in communities in Zambia


expectations of engagement (as has occurred recently with many changes of personnel being made

by the change in government leadership and the creation of new districts). Duty bearers were

therefore involved in the initial discussions to select sites and the WASH approach within these sites,

are kept informed of progress and challenges, and a collaborative approach to the provision of

WASH is implemented. The duty‐bearers interviewed indicated that Oxfam is viewed as being very

supportive, sensitive to community processes, dynamics and needs, and that they have a good

partnership where they complement and supplement each other’s work.

In South Africa, an innovative approach was undertaken to integrate WASH into existing child

protection, food security and HIV programs, instead of focusing on providing WASH as a separate

program. This was in part due to the Oxfam Australia ‘One‐program approach’ix, but more

significantly due to the Constitution of South Africa which places a direct responsibility on

government to provide WASH infrastructure and services to its citizens. The challenge however is

that government does not provide uniform services across urban and rural areas, nor are they held

accountable for service delivery.

The Oxfam approach therefore reflects the national political context. At local level, across both

countries, the approaches and activities have had to respond to the local political, cultural and social

contexts. In all settings except in the crèches in the peri‐urban area of Wentworth where STC KZN

supports the early childhood development sector, the activities have had to engage with traditional

leadership. In all settings, including Wentworth, partners have had to engage with Municipal, Ward

or District government. All partners have had to find relevant approaches to meet their local

context. The findings indicate that engagement with traditional leadership and government has

varied across partners and countries.

Influencing policy

Evidence from the program has been made available to local, municipal or district, provincial and

national government in Zambia. This has been largely initiated and facilitated by Oxfam, and was

built from existing relationships and program work. The engagement at district level (quarterly

meetings, telephone calls, sharing of reports) continues to build a collaborative approach between

government and Oxfam, however, the government officials interviewed remained in the mind‐frame

that the role of Oxfam is to support the provision of infrastructure (i.e. provide funding) rather than

to facilitate communities leading their own development. The implementing partners indicated that

they have a good working relationship with the district government, however government does not

always deliver on their mandate, and are not accessible to rural communities due to long distances,

poor roads, limited transport and poor telecommunications infrastructure.


There was no evidence from the MTR research that this engagement has directly influenced Zambian

policies pertaining to WASH or development. However, government officials cited increased

awareness of the ‘clean water issue’ (ZD) and that, in one district the Council officials did visit one of

the Oxfam sites and are now taking steps to install a borehole. There is acknowledgement that the

program is assisting the Council in meeting their objectives through the additional capacity and

resources.

The South African approach of integrating WASH into existing HIV prevention, food security and

child protection programs has contributed to increased awareness of community members and

structures around WASH issues, and efforts have been made by the partners to engage with

government as the Constitutional and legislative provider of water and sanitation. As discussed

previously in this report, there have been varying levels of success. Evidence from the MTR

interviews and focus groups suggests that the ECD Forums, through the work of STC KZN, are

beginning to influence the norms and standards for registration of an ECD facility. In addition, the

engagement of FS in year two of the project with the local municipality to raise awareness and

advocate for accessible public toilets in Ingwavuma town has borne fruit: the municipality has

indicated they will support these but want FS to cover the costs. Despite FS leaving the AACES WASH

program, they continue to engage with the Municipality on this issue and are striving for the

Municipality to take full responsibility and ownership of them. The demonstration site of waterless

toilets at the Fancy Stitch tea garden, and the public use of them during the local marathon, made

an important contribution to the understanding of the effectiveness of this innovative solution to

sanitation. WM made numerous visits, phone calls and attended meetings with local government in

an attempt to raise awareness of the need for communities in the Uffafa Valley to have accessible

and clean water. However, no successes were noted by the organisation. However, in the last six

months the Municipality has begun extending the municipal water pipes to parts of the community

(along the main road) – but, as WM themselves acknowledge, it is not possible to determine if this is

as a result of their attempts to engage them or that this has been in the Infrastructure Development

Plan (IDP) of the district for a while. MDIC have successfully engaged with traditional and elected

government officials over many years, and established a collaborative working relationship with

them on community development issues (including WASH). However, as MDIC has withdrawn from

the program it is not possible to determine the results or impact of this collaboration, and the

possible influence on policy.

The rights‐based approach (as discussed previously) is central to how the program engages

communities, increases awareness of WASH and health rights, and builds capacity of community

members and structures to engage government and to increase access to their rights. However, the


findings of this review indicate that although awareness of rights has significantly improved (in that

all beneficiaries who participated in the MTR raised this as a result of the training and awareness‐

raising), beneficiaries have not yet been able to raise their voices to duty‐bearers beyond the

immediate village, household or community. Reasons cited (in both countries) include the distances

to get to official offices, transport, communication and time constraints, but also because they are

unsure of how to go about doing it.

The exception is STC KZN, who through the two Crèche Forums, have found a method to not only

raise awareness, but when necessary for the group itself to take collective action. For example, in

the past individual crèches have attempted to engage government on a one‐to‐one basis, with very

little effect, however, as a group there is greater voice in collective dialogue and holding duty

bearers to account. It is not a case of STC owning the forum, rather their role is to empower, monitor

and support the forum to: manage group processes, creating the space for beneficiaries to share

knowledge and practice, developing and implementing action plans that collectively work towards a

shared goal, facilitating problem solving as a group and deciding on group or collective action. Group

action refers to periodic action that the group takes, such as a WASH day in the community, whereas

collective action is when the group mobilizes around an issue (particularly when a right is infringed),

such as failure of the Municipality to remove refuse despite citizens paying for this service through

Municipal rates.

Partners described a significant moment in finding a relevant advocacy strategy that they were

comfortable with during the partner reflection in February 2013, when the WASH technical advisor

used the analogy of being a “tree shaker” or “jam maker” (RA, RF): in other words a provocateur or

strategic change agent. Since then, the partners have been more conscious about how to engage

with government (raise their voices) and how to build the capacity of beneficiaries in raising their

voices. Efforts going forward need to focus more on building the capacity of partners and

community structures to develop strategies and practices to promote the voice of community

members in claiming their rights. However, there were differing opinions across partners and with

Oxfam as to where the responsibility lies in advocating at a provincial, national and regional/Africa

level on WASH issues: Oxfam suggest the partners need to do this and partners indicated that this is

a role Oxfam needs to play as it is beyond their role as locally based civil society organisations.

In addition, as highlighted by one program manager “the rights‐based approach is too silent in the

program; it needs to come to the fore. The objectives of the program are good but too narrow for

real impact on rights. All we can do is mobilize people but they cannot take action as the budget and

program focus limits this. Across program rights are very low because dependency on a promise of

action is seen as a right, not the right itself. Government takes our money for tax so now they need


to use it for our rights. We need to shift from promise to plans to implementation” (ZP: paraphrased

quote). Similarly, one partner director raised the issue of a community paying municipal rates for

services and therefore when, for example refuse is not removed, the community has a right to

demand this service. As a collective voice it is more likely that action will be taken and this is the

value of a collective community structure, however as an NGO one needs to facilitate the process of

raising awareness and building consensus on action to be taken (RB).

4.1.2 Effectiveness

Effectiveness is determined by the program contributions to significant and sustained change, in

particular changes in the lives of women, men, boys and girls and marginalized groups, as well as

changes in policy, attitudes and practice.

The program has focused in the past two and a half years on: building capacity of stakeholders

primarily through awareness raising and building WASH knowledge within the partner organisation

and with their beneficiaries; increasing awareness of rights across the stakeholder sector; increasing

awareness and acceptance of people with a disability, and children’s role and rights to WASH;

increasing capacity of beneficiaries to engage with duty bearers; increasing capacity of civil society

structures to implement and manage elements of WASH.

Changes in knowledge, attitudes and practice

The evidence from the MTR indicates that there have been a number of changes that have come

about due to the partner activities. These changes are noticeable at a number of levels, and across

the spheres of change as articulated by Rao and Kelleher’s ‘Integral Framework’ which suggests that

for long term sustainable change in the lives of men and women, the social systems and institutions

that determine the distribution of power and goods must be transformed (refer to the Oxfam AACES

WASH design document). The changes are discussed below within the integral framework spheres of

change:

Table 2: Reported changes as a result of the Oxfam AACES activities in South Africa and Zambia

Oxfam objective

areas

Program: South Africa and Zambia

Beliefs, awareness,

knowledge and

capacity

• Increase in knowledge about safe and clean water, hygiene practices,

importance of a clean environment, water harvesting, collective

vegetable gardening, protecting water sources, health related

behaviour (STC, WM, OV, TU, MDIC, KZF, PPS)

• Increase in hand washing by children, household members and at


schools/crèches (RC, RA, RF, RB, ZJ, ZM,ZL)

• Reduced diarrhoea and diseases8 (ZI) (FS, WM, MDIC, STC)

• Anecdotal evidence that household members are using pit latrines

rather than open defecation in Zambia which was challenging until

there had been a mind shift once the health benefits were known (PPS,

KZF)

• Increased awareness of needs of people with disabilities (STC, WM,

OV, TU, MDIC, KZF, PPS)

• Increase in understanding the rights of children by parents, community

members and school principals and teachers (KZF, PPS), and how

organisational policies influence child protection practice (TU, WM,

MDIC)

• Awareness that WASH is at the centre of community development,

primary health care and management of a crèche and school (WM,

MDIC, STC, FS, PPS, ZB)

Access to WASH

services

• Increased access to improved sanitation facilities e.g. Pit latrines built

in Zambia, and people using them (PSS, KZF)

• Improved access to water in specific sites including crèches, rain water

harvesting at schools, community pumps protected, pipe to provide

water nearer to community, boreholes fixed and installed wells (STC,

WM, OV, TU, MDIC, FS, KZF, PPS)

• Emerging collective responsibility to access, protect, use and maintain

water and sanitation infrastructure by community members to sustain

WASH services, e.g. school principals and educators indicating they

have responsibility to provide access to drinking water (STC), and a

community building a wooden fence around the water pump to

protect it (KZF)

Reduced inequalities

in WASH

• Partners able to identify WASH needs for people with disabilities, and

beginning to respond through providing infrastructural support e.g.

design and building of accessible toilets for people in wheelchairs

(WM, TU, MDIC, KZF, PPS, FS), design of tippy taps for increasing hand

8 Self reporting of reduction, was not validated by district health statistics or clinic


washing (WM, FS), and increasing access for children with disabilities

to crèches and schools (STC, OV, MDIC, TU, FS, WM). Partners

increasing participation of people with disabilities in WASH activities

and community structures (TU, WM) and

• Awareness on the gender roles of men, women and children in terms

of access, use, and activities associated with WASH e.g. women collect

water and men dig pit latrines (STC, WM, OV, TU, MDIC, FS, KZF, PPS)

• Partner’s awareness on the differing needs and access to WASH

services between rural communities and urban communities, and the

identification of strategies to address needs of rural communities (STC,

WM, TU, KZF, PPS).

• Efforts are underway to develop committees or community group

structures to develop their own development and action plans to

address inequalities, access their rights to WASH services and engage

with duty‐bearers in different contexts e.g. crèches, schools,

community structures, wards/districts and provincial government (STC,

WM, TU, KZF, PPS, OV)

WASH governance • Increase in awareness of WASH and how communities can respond

through effective community structures, such as supporting V‐WASH

Committees and building capacity of Community Facilitators (PSS, KZF),

facilitating the establishment of Water User Associations (TU), and the

WASH Crèche Forum through (STC)

• Engagement with school management teams and School Governing

Bodies (SGBs) to increase knowledge and identify roles, responsibilities

and tasks (WM, OV, MDIC, FS), and with Parish Councils (LIMA)

• Oxfam and partners (and community structures beginning to) engage

with Area and District Commissioners, District Councils and Ward

Development Committees in Zambia to increase awareness of needs of

rural communities and highlight community identified priorities

(including but not limited to WASH services, for example the need for a

clinic to increase access to health services in the ward plans) and to

hold duty‐bearers to account e.g. confirming role of Oxfam, PPS and

KFZ in this program not as a provider of infrastructure, rather to

develop capacity of community structures to respond to


developmental needs.

• Engagement with traditional leadership by MDIC, PSS, KZF and initial

engagement with Municipal service providers by FS, TU, WM to

identify possible areas of support or collaborative responses (e.g.

alignment to IPDs and ward plans), increase awareness of roles and

responsibilities, and to begin to engage with duty‐bearers to hold them

accountable for WASH services.

• Engagement with South African Department of Basic Education by

OneVoice through a Memorandum of Understanding to define roles

and responsibilities; and with the Department of Social Development

and Department of Health by STC KZN in particular to ensure that

social workers and nurses respectively undertake their mandated visits

to the crèches to provide services.

Changes in the lives of women, men, boys and girls and marginalized groups

The marginalised groups involved through the partners in South Africa include children in early

childhood education and care, children and youth in schools, people with disabilities, women, and

households in rural, peri‐urban and low income urban areas. In Zambia, partners involve children

and youth in schools, women, and rural households and villages.

The training of women in construction in Zambia has resulted in increased skills and a sense of

empowerment by the women (ZJ). The long‐term investment in building capacity of the community

caregivers (all of whom are women) at Woza Moya to respond to social, health and community

challenges has meant that the inclusion of WASH as part of their response has resulted in individual

change in not only their own WASH practice, but in a strong belief that they are able to carry WASH

forward in their own households, as well as in the households they support. The success of the tippy

taps that they have installed and the rapid scale‐up in neighbour households and in the community

has made a significant contribution, above the health outcomes, to increased empowerment of

individual households taking action to respond to poor or inadequate access to clean water. For

example, one of the communities visited during the MTR to Woza Moya indicated that because of

the awareness raised through the tippy taps, they know they need to improve the cleanliness of the

river and spring, as this is contributing to continuous illness in the household.

The ECD practitioners working with STC KZN interviewed (all women) indicated that they used to be

anxious about the health and safety of children in their care, but now they had a sense of calm, they

felt settled and they had a system in place to monitor the hygiene of children. Similarly, the TU


crèche practitioner involved in the focus group discussion indicated that she had learnt how to

change washes in taps, and she was able to take initiative and find solutions if for example the water

supply was down. Previously she would have waited for it to be turned back on.

School‐based facilitators from OneVoice SA indicated that they had changed enormously through

the piloting of the school‐based youth materials. They had grown as facilitators, developed

confidence, were learning to listen to youth and respond appropriately (particularly if a youth

disclosed that they were HIV positive), and had developed their creativity in mobilising the school

community around WASH, in sharing knowledge, and in engaging with learners around behaviour

and practices (not only on WASH, but also on health, sexuality and relationships).

Across partner staff members interviewed, all

indicated that not only were they able to facilitate

changes in the lives of those they worked with, but

they had made changes in their own WASH practice, in

their households and that friends/neighbours were

also being influenced. Similarly, crèche practitioners

interviewed indicated that the household members of the children they cared for were also asking

for information about hygiene as the children were saying, for example, that they had to wash their

hands before eating and after using the toilets and where could they do it in the home.

“We eat, sell butternuts, we are strong and no longer starving, we

can feed our families” Vukazibuthe Participant, Women’s

Vegetable Gardening (FS)

The women involved in the Fancy Stitch food security project which benefited from water

harvesting, indicated that their nutrition and health had improved. LIMA indicated that through the

food gardens at the Parishes and increasing access to different crops, food storage and preparation,

there is an improvement in nutrition, however they indicated that this has not been confirmed

through research.

Men are involved in the WASH community activities and structures, and included in the capacity

building, infrastructure implementation, and as part of the community committee. As illustrated in

the graph below, there has been an increase in the number of men and women accessing WASH

services over the six‐month periods reviewed. There has been a total increase of 8,324 peoplex.


Graph 1: Increase in number of men and women accessing WASH services in South Africa and

Zambia from January 2012 to June 2013

661408

3661

5

1419

1765

0

1000

2000

3000

4000

5000

6000

Jan 2012 ‐ June 2012 July 2012 ‐ Dec 2012 Jan 2013 ‐ June 2013

Male

Female

Source: Oxfam AACES Six Monthly Snapshot Reports

In both the South African and Zambia rural community contexts there are strong traditional and

cultural practices. Men tend, in both countries, to be the decision‐makers in community structures,

and in fulfilling traditional leadership roles (except in the case of Woza Moya where the local chief is

currently a woman). The reported figure for the percentage of women participating in decision‐

making structures in Zambia remained at 44% for the period July 2012 to December 2013 (year 2)xi.

The focus on the program is on increasing the participation of women in decision‐making regarding

WASH, and this requires working with both men and women. In Zambia, the training for the

community facilitators aimed to have an equal percentage of women and men, however of the 30

trained, 10 were women (33%), and a number of women who had been selected from the

communities did not attend (reasons were not cited and it is assumed that this is indicative of the

gendered nature of participation in community processes).

Participation of children and youth

The participation of children and youth has primarily been as recipients of the program results, for

example increased access to water at schools for drinking and hand washing (WM, STC KZN, MDIC,

FS), reduced distances to travel to access water at schools, increased access to hand washing

facilities and toilets at school, a cleaner school environment and knowledge about WASH practices

and services. However, with the scale up of the OVSA program for schools and the campaign/school

mobilisation activities learners are encouraged to lead, the involvement of children and youth aims


to be strengthened. As indicated previously, children in crèches are instrumental in bringing about

changes in WASH practices at household level.

One of the DFAT policy requirements is that Oxfam and all its partners adhere to child protection

standards through adherence to child protection policies and code of conducts. In South Africa, the

organisation Resources Aimed at Preventing Child Abuse and Neglect were contracted to support

partners in adhering to child protection standards. As a result, they engaged with each partner to

review their existing policies, provide guidance and support to develop or strengthen both a child

protection policy as well as adherence to child protection standards across organisational policies,

and to provide awareness or training as necessary. Partners indicated that this was a valuable

process as they had not realised the importance of integrating child protection standards across

their organisational policies, structures and programs. However, as RAPCAN noted, this was a ‘top‐

down’ approach and focused on compliance rather than programmatic integration, child rights and

actual practice (engagement with children and families) which is the traditional approach RAPCAN

use to engage organisations, caregivers who engage with children, and interventions that work with

children. The approach was therefore not participatory, and although successful in terms of

compliance, has not maximised the opportunity of embedding a child rights perspective into

programs or necessarily translated policy into action. The ‘top‐down’ approach was not easy to

implement according RAPCAN, as they had very little room to manoeuvre from the contracted

approach, and they had to build the trust of the organisations to share policies for organisational

development and learning opportunities rather than be seen to be 'policing’ compliance.

One of the achievements was the development of individualized reports to each partner on the

standards and recommendations to strengthen practice which were submitted to each partner, and

the engagement of three of the five South African partners (TU, FS, WM) in demonstrating a

commitment to take this forward. The informal opportunities to engage with partner directors and

staff during the annual reflections and capacity building sessions were noted by RAPCAN as being

invaluable in building relationships and engaging in dialogue about policies, practice and including

standards in organisational development and systems.

In Zambia, training was provided to partners and communities on child protection. The training was

provided by a Zambian Police official and an official from the Zambian Community Development:

Mother and Child Health unit. The aim was to develop ‘child protection mentors’ who were

capacitated with knowledge and materials to support children. The value of this training was

highlighted by the school principals and teachers, as well as partner program staff (ZJ, ZK). Two key

child protection issues emerged from the community focus groups: what constitutes child labour


versus chores, and early marriages of girl children (ZJ, ZK). These issues continued to be debated by

community members during the focus groups.

Changes in the lives of children, as reported by the respondents, include:

• Parents now accept responsibility for clothing and feeding their children, before children had

to fend for themselves (ZJ)

• Parents used to abuse and neglect their children and would treat the child as if it wasn’t

theirs, now they know what abuse is and this has greatly reduced (ZJ)

• One of the schools visited indicated that the guidance counsellor is now far more proactive

and receives good support when engaging with children and their families on issues, and that

this making a difference to the child’s life (ZK)

• If a child drops out of school the principal will write a letter to the parent or guardian and

meet to find out why the child is not at school, explain the long term consequences of not

attending school, and encourage the child to return to school (ZK)

• In the two WASH schools where OVSA field tested the materials, the school facilitators

indicated that they had seen a change in the hygiene practice of girls, particularly in terms of

menstruation hygiene

The findings indicate that the first steps have been taken in developing appropriate engagement and

participation of children in WASH, but this has not yet been taken forward in the program. The

question is how, and of what relevance would it be, would further levels of engagement of children

and youth in the program contribute to the achievement of the objectives and goals, as well as what

value or return would this bring to the program. Once again, this speaks to the need for a clearly

articulated theory of change which would provide a common vision and identify strategies of how

children and youth can be further involved and the relevant changes brought about.

Disability

Extensive efforts have been made in South Africa to include people with disabilities (people with

disabilities) into the WASH activities, as well as into existing programmatic responses. CREATE was

contracted to undertake a baseline study and support partners in actively including people with

disabilities into the WASH program. Activities have included site visits, discussions and dialogues

with program staff, providing training manuals, conducting training and problem solving on specific

cases or situations. In Zambia two organisations, who work closely together, were brought on board

to lead and raise awareness of people with disabilities. ZAPD and ZAFOD are working with Oxfam on

an appropriate design for an accessible toilet for people with disabilities in rural areas, raising

awareness in communities, facilitating workshops and training sessions, and advocating for the


country’s national development plans to include

people with disabilities. Awareness raising focuses on

the rights of people with disabilities in the community,

schools and with people with disabilities themselves as

they often do not know their rights.

Partner activities have increased access to accessible services for 425 people with disabilities during

the period January 2012 to December 2013xii. Partners have embraced people with disabilities in

different ways, and therefore the different results are noted. A summary of these is provided in the

table below:

Table 3: Partner responses to inclusivity of people with disabilities

Partner Result

Fancy Stitch Design and construction of accessible toilet for household headed by wheel‐chair bound woman

Consideration of how people with disabilities access sanitation in public areas and engagement with Municipality to construct accessible toilets and hand washing facilities

MDIC Changes way viewed programs as realised they were excluding people with disabilities unintentionally

Began to look at school WASH infrastructure to facilitate access for people with disabilities

PPS Beneficiaries do not use derogatory language when talking about people with disabilities

Save the Children KZN

Consider design and infrastructure of crèche centres for access for children with disabilities

Tholulwazi Uzivikele

Change in language of staff toward more inclusive language

Employed person with disability

Consider how to include people with disabilities into all programs and increase their access to WASH facilities and services

Woza Moya Trained community health and care workers in identifying people with disabilities, how to treat them and who to refer them to: there is a specialist hospital for people with disabilities where cases are referred to, thereby increasing access to health services for a large number of children, women and men

ZAPD/ZAFOD These organisations have provided technical support and services in the wards through raising awareness on rights of people with disability, facilitating the inclusion of people with disabilities to participate in the baselines and V‐WASH Committees, and engaging with district officials on inclusion of people with disabilities. During the baseline study approximately 283 people with disabilities were registered with the government to receive benefits who previously were not registered.

“We realised that [our organisation] was unintentionally excluding

people with disability” Programme Manager, South Africa

partner


CREATE (South Africa) indicate that the disability baseline influenced the program design as it gave

an understanding of where each partner was at and what was lacking, and what was already in place

(even if the organisation was not aware of this beforehand). An important step in partners

identifying action based on increase knowledge came about as a result of the disability awareness

workshops. Although action has been taken in the WASH program, two and a half years has not been

sufficient time for partners to mainstream quality actions and practice across programs and

organisational practice. This will require further engagement, commitment by management, and

funding to support achievements at the output and outcome level. In addition, it will take time for

these activities to influence household and community practice.

There has been limited engagement with stakeholders and duty bearers regarding increasing

sustainable access to the WASH services they require. Fancy Stitch raised this issue with the

Municipality and is calling for an accessible toilet for people with disabilities to be made available in

the town. Awareness‐raising has been undertaken in the primary and secondary schools: however

this has not yet translated into action or into changing school policies to increase access. In most

districts in South Africa there is at least one school that has been classified as an ‘inclusive school’

where children with disabilities can access schooling, and in many regions there is a school primarily

equipped for children with disabilities (including learning disabilities).

One of the successes of the approach in South Africa has been the establishment of a partnership

between people with disabilities and the partner. As a result, people with disabilities are now

speaking for themselves on WASH, rather than the partner speaking on behalf of people with

disabilities. The aim is to empower people with disabilities to raise their voices and engage with duty

bearers ‐ and further capacity building is planned for people with disabilities. One of the areas that

need to be strengthened is the reporting on disability in the Oxfam report structure.

Gender

In Zambia, the inclusion of a partner to focus on this aspect will go some way to raise awareness of

the gendered nature of WASH. The partner, YWCA, indicated that there is a strong traditional

gendered way of working, which is slowly changing but one cannot work with women alone: it is

vital to work with men and the leadership. An effective strategy is to work with men and women to

develop appropriate role models and this attracts others and brings about change. The main activity

in Zambia has been to train twenty women in construction skills. The four women interviewed

during the MTR field work who have been trained in these skills were highly appreciative of the

training and spoke about how this had change their sense of purpose, well‐being and feelings of

being respected by their community.


However, concern was raised by four respondents that the success of these women in actually

undertaking contracts is limited because: a) lack of understanding and experience in contract

processes; b) it is difficult to get cash to buy building supplies and get them delivered to deep rural

sites; c) ongoing mentoring is required and who can provide this is not clear; and d) for real

empowerment they need to be supported by their family, by the community, and to form a business

group so that it becomes a collective effort to bring about change in their community (ZE, ZB, ZP, ZJ).

In general, gender has been addressed to a lesser

extent in both countries as the focus of marginalised

groups has tended to be on people with disabilities,

and empowering rural communities to respond to

WASH. There is some concern by partners about a

forced approach or an inappropriate response, and a

concern about the stated indicators relating to

gender. There remains a traditional and cultural gendered approach to community structures, the

role of women and men in decision‐making, and how and by whom issues or agreed actions are

taken forward. The community processes involve women to a greater or lesser extent dependent on

the culture, historical context, traditional leadership and the evolving democratic rights of people

being put into practice, and the empowerment of women. Partners in South Africa highlighted that

the strategies to raise gender at a program, partner, beneficiary, and community level are very

important. For example, explaining and engaging in the dialogue around the gendered nature of

WASH cannot only focus on women, it must include men and highlight their role, such as digging and

building pit latrines, or maintenance of water harvesting infrastructure. The engagement with a sub‐

consultant brought in to identify gender practices and norms did not meet the partner’s needs

(particularly as the consultant did not keep to their terms of reference and changed the focus of the

work session to more of a gender advocacy workshop), and was deemed inappropriate in approach,

content and messages (WM, TU).

4.1.3 Efficiency and Value for Money approach

Value for Money refers to the benefits or value of the program, and in essence examines the impact

achieved for the money invested in the program. Value therefore is explored across the results

chain: from inputs to processes, outputs, outcomes and impact. Value for Money (V4M) is based on

the following four components: economy, efficiency, effectiveness and equity.

Efficiency refers to the effective and efficient use of resources to contribute to program design,

implementation and outcomes.

“Gender must be a theme in a program, not a subject. It must be raised at program level, not at

individual level” Programme manager, South African

partner


The MTR field work did not undertake an in‐depth study on value for money and further

development of both the Oxfam approach as well as a rigorous study on value for money in the

AACES WASH program in South Africa and Zambia is required. An independent study is being

conducted to understand value for money in greater detail in the AACES WASH program. The focus

of the above mentioned research is to understand value for money in the Oxfam AACES program,

identify examples to demonstrate value for money, the extent to which beneficiaries believe the

project has improved their wellbeing, extent to which the project has leveraged resources from

government and others, the extent to which the project has influenced the broader work of Oxfam

and partners, and to identify how the value for money work could be improved and how the results

are sustainable.

A few questions were posed to participants to gather empirical evidence of perceived value of the

program focusing on the above scope during the MTR field visits. The findings are presented below.

Value was interpreted widely by respondents and included efforts to determine monetary returns,

health outcomes and individual changes in behaviour or practice, as illustrated below.

The focus groups with the beneficiaries in South Africa and Zambia suggest that the project is

improving their wellbeing (RA, RB, RE, RF, RG, ZI, ZJ, ZK, ZL, ZM, ZN). Wellbeing is definedxiii broadly

to encompass various personal and social domains such as happiness, health, sense of purpose, and

safety. Responses from the MTR participants tended to focus on health aspects of wellbeing, such as

improved nutrition and increased amount of food; social aspects such as better relationships

between parents and children; an increased awareness of rights related to well‐being such as an

increase in water, health, the inclusivity of people with disabilities, and a sense of safety for self and

others (particularly for children who had fallen down wells in the past in KZN).

The table below illustrates the emerging value for money according to the ‘4E model’ as perceived

by the MTR respondents.

Table 4: Perceived value for money of the Oxfam AACES WASH program

Economy:

Costs and

inputs

• Partners indicated that the budget is very limited and the program is being

implemented in a resource constrained environment.

• There is concern (RF, RE) that the input costs are not as low as they could be,

for example by using expensive mobile technology rather than open source,

high costs for travel, the use of limited resources for capturing data for

reporting rather than the funds being maximized on their return by investing in


infrastructure or a trainer

• Training was perceived to be of good quality (ZI), however the respondent did

not indicate when probed what constitutes “good quality”

Efficiency:

Inputs to

outputs

• Increase in awareness and knowledge which has led to action of some sort: as

one respondent said ‘we now have power to do things’ (RF). A wide range of

increased awareness and knowledge related to WASH, rights, disability and

processes (in particular through capacity building of community facilitators and

committees) was highlighted by respondents and how this has equipped

individuals and groups to develop action plans, share their knowledge, and to

make changes for example to install their own tippy tap, clean their yard or fix

their own washer in a dripping tap

• Increase in hand washing practice that has led to increased hygiene and

reduced diarrhoea and common infections such as eye infections (RE, RA, RG,

RF, RB)

• Through installing rain water harvesting systems Increase in access to water at

crèche, school and household level, and how to improve the quality of water

through easy methods such as using Jik9, sunlight or putting a cover over water

storage containers (RA, RB, RE, RG)

• Water harvesting or infrastructure (such as solar pumps, storage tanks, fencing

to keep animals out of food gardens) and the provision of essential equipment

(such as hoes, spades and buckets) increases food productivity e.g. production

of vegetable gardens increased by three fold10 (RE, RD), as well as school

attendance and learning outcomes as learners do not have to leave the school

property to get water to drink

• Consumers from the community were reported to value local vegetable

produce and were therefore prepared to pay a higher price than in the local

supermarket because, for example, traditional beans were seen to be of good

quality (these are traditional beans rather than commercial ‘green beans’, and

the community know the level of effort required to produce products for local

market (Fancy Stitch vegetable gardening project)

• There is a chain of community members who benefit from the building of

9 Jik is a brand name for bleach in South Africa 10 This figure has not been validated


knowledge, practice and/or infrastructure: for example children at crèche learn

about washing hands and how this contributes to health, and they go home

and share this with the caregiver who (as reported) then brings about change

in the household (STC KZN); or a community caregiver who installs a tippy tap

and discussed this with the neighbour who in turn installs one (WM); or the

community facilitator discussing waste management at a community

committee meeting which results in improved waste management, improving

the environment (PPS)

• The value of working together (as the community, community committee and

partner) has resulted in broad developmental ward plans that meet the needs

of the community (PPS, KFZ). WASH was central to bringing people together

and identifying community needs (ZA)

• The value of education is sustainable (ZD) and the investment in knowledge

changes mind sets and how individuals, committees or groups, and

communities respond to issues. For example the CLTS training was reported by

one respondent to have changed peoples mindsets and consequently resulted

in local government coming on board (ZI)

Effectiveness:

Outputs to

Outcomes

• There is an increase in the nutrition of vegetable garden families as they now

produce a regular supply of seasonal vegetables and this has increased their

health and productivity, and the excess sold to generate income for food,

education and clothing for family which contributes to an improved quality of

life (RE, RD)

• Through building the skills of ECD practitioners this has resulted in a change in

daily practices at crèche which has contributed to health outcomes (T1), such

as less eye infections, diarrhoea and ‘runny noses’ in crèches and schools (RB,

RA, RF, RG, ZA)

• Processes are now in place to deal with community development issues

through increased knowledge and awareness of rights, focus is not just on

infrastructure (ZB)

The implementing partners in both countries articulated during the MTR interviews the need to

balance the activities between infrastructure and capacity building: at different stages of the

program implementation cycle and in supporting community structures to respond to priority WASH

needs and concerns, there has been a tension and demand to deliver infrastructure. The evidence


suggests that for sustainability and the achievement of the expected outcomes for the program, the

focus cannot be on only capacity building or infrastructure: without capacity building the

infrastructure cannot be maintained and the community remains dependent on external resources

to respond to shocks and the achievement of health outcomes, and without basic infrastructure

(such as a working water pump or water storage tanks) to enable structures and communities to

overcome barriers to achieve short term goals or outputs, the long term objectives will not be met.

For example, the need for improved hand washing facilities in the Woza Moya community was

identified, and advocating for running water and hand basins was not feasible, however finding a

cost effective solution in the form of tippy taps meant that an accessible and easily replicated

approach could be taken. Similarly, increasing access to clean water at schools would have remained

at the talking and planning phase unless infrastructure was installed to demonstrate a solution ‐ now

there is a willingness to own the system and an emerging commitment to maintain it.

The evidence presented above highlights a number of emerging case studies that illustrate value for

money in the AACES WASH program. Samples of these are explored further in the study on

understanding value for money approaches in the Oxfam AACES WASH program.

4.1.4 Sustainability

How has or is the program design supporting long‐term sustainability? To what extent has a financial

and/or program transfer strategy been developed to ensure continuation or consolidation of the

program?

The sustainability strategy for the Oxfam AACES program is articulated in the program design

document (May 2011). The document indicates that sustainability is contingent on the following

three building blocks:

a) The capacity of men and women to claim their rights, respond to external trends and shocks,

influence those with power and to hold duty‐bearers to account.

b) In addition, the program aims to support sustainability of the outcomes and impact by

building capacity of strong and well‐functioning structures and institutions at various levels.

Therefore there is a focus on establishing or strengthening and coordinating community

structures, as well as providing technical and financial support to partners to enable them to

conduct WASH functions more effectively.

c) Through the partnership strategy, the program aims to rest responsibility and leadership for

WASH interventions with partner institutions and community structures.


The program design therefore incorporates the above elements for the long‐term sustainability of

the program. A review of the findings from the MTR indicates that progress is being made towards

establishing these blocks in programs, communities and practice. This is discussed below.

Increasing capacity of men and women

The capacity of men and women to claim their rights, respond to external trends and shocks, and

influence those with power and to hold duty‐bearers to account is being built through:

• Direct training to individuals and groups including ongoing training on WASH by all partners to

increase awareness and promote access to rights; training on child protection to increase

children’s rights and address trends in communities such as school dropouts in Zambia in year

two; and training on disaster risk management to partners in South Africa and Zambia, and to

community structures to increase capacity of individuals and groups to respond to external

shocks in year two

• Mentoring, support, monitoring and oversight through continuous engagement with

communities on project activities, for example the ongoing partnering with a school, crèche,

household or committee whereby the facilitator or caregiver provides opportunities to

incrementally demonstrate how problems are solved, community development processes unfold

(for example identification of needs followed by prioritisation followed by ward plans followed

by engaging with traditional leadership and ward councils) and how changes and progress over

time are possible

• Facilitating ‘communities of practice’ between community facilitators in Zambia, caregivers in

South Africa, the two Crèche Forum structures and program staff. Communities of practice in

this context are emerging groups of people who have a common interest or are responding to

common challenges where they learn from each other from sharing knowledge, practices and

experiences, and thereby develop themselves as individuals but also as a collective.

Key activities underway that have the potential to strengthen these activities and contribute to long‐

term sustainability include the development of the advocacy strategy and associated training that is

being planned, and strengthening access and the voice of communities to hold duty‐bearers

accountable through structures such as the Crèche Forums (STC), participation in the WAR rooms

(ward council planning forums) in South Africa by partners (TU) and engagement with government

departments at district and provincial level (TU, OV, STC). The key informant interviews with these

partners indicate that partners have thought about how to get community structures and members

to engage more with duty‐bearers. This will need to be a focus of the program over the next two

years to strengthen the community level engagement: however a few partners indicated that after


this five year program they are unlikely to exit the community and they will continue to build the

capacity of men and women as community development, and their work, is broader than the WASH

focus.

Building capacity of structures and institutions

In addition to building individual capacity, the program aims to support sustainability by building

capacity of strong and well‐functioning structures and institutions at various levels. These include at

the early childhood development and school level, as well as at community and household or village

level. The focus of these structures differs depending on if they existed prior to the program, if they

have a legislative mandate or if they emerged from the community level. The focus of the capacity

building of these structures in terms of the AACES WASH program is to strengthen community

responses to development needs and coordination between community members and duty‐bearers.

A brief description of the types of structures partners are working with is presented below:

COMMUNITY STRUCTURES PARTNER ARE ENGAGING WITH ON THE AACES WASH PROGRAM:

EARLY CHILDHOOD DEVELOPMENT MANAGEMENT & FORUMS

• ECD centre has responsibility to manage health and safety of children, and meet government requirements for registration, which includes WASH services

• Training of crèche managers to meet requirements however difficult to meet within broader community WASH context e.g. if no running water cannot provide flush toilets

• Community established ECD Forum which STC supports through ongoing capacity building, mentoring, and support

SCHOOL MANAGEMENT TEAMS & GOVERNING BODIES

• Existing structures in schools mandated to manage and govern school • Involved in developing and maintaining WASH services at school • Limited training in WASH provision to‐date but partners provide ongoing advice, mentoring and

support. In Zambia capacity building on child protection has been conducted. • One Voice engages with school management prior to and during curricula activities and through

school facilitators beginning to influence WASH management at school

PARISH COUNCILS

• Existing structure in churches • LIMA engages with them as part of the vegetable garden projects which contribute to

livelihoods and improved health and quality of life outcomes • Continuous engagement with Council but no direct WASH capacity building undertaken

V‐WASH COMMITTEES & WATER USERS ASSOCIATIONS

• V‐WASH Committees exist in Zambia , extensive and ongoing capacity building being undertaken direct with committee as well as identified community facilitators

• Water Users Associations being established by Tholulwazi Uzivikele in South Arica and capacity building has begun


Through the AACES WASH program, technical and financial support provided to partners has aimed

to enable partners to conduct WASH functions more effectively. Technical support in the form of

child protection policy review and development; ongoing MEL capacity building through ongoing

mentoring and development of skills and systems within the organisations; disaster risk reduction

training; capacity building in CLTS approach and practice; engagement with technologies that can

support WASH interventions such as the mapping of water sources with SMEC; development of ECD

materials and training by TREE; developing an understanding of inclusivity of people with disabilities

in relation to WASH and community development; and ongoing engagement with technical expertise

on engaging with duty‐bearers and developing awareness of right‐based approaches, WASH

governance and the provision of services by various departments and levels of government. The

findings of the MTR indicate that further technical support is required in relation to advocacy,

gender and in the provision of infrastructure advice and training.

There is a need to clarify the financial transfer strategy for the WASH program at both the Oxfam

level, as well as with individual partner level (AG1, AG2). Given that this is a five‐year program, the

financial resources will cease in 2016. Plans need to be made as to what financial resources are

required to continue to mentor and build capacity to sustain the outcomes going forward. LIMA and

TU highlighted that a three to five year program of this nature is unlikely to achieve sustainability as

there is insufficient time to mentor structures and monitor systems post direct activity and

intervention phase. There are many changes occurring within Oxfam in terms of both the Oxfam

Australia strategy as well as the establishment of Oxfam South Africa and the decisions taken (for

example what exactly will the WASH strategy be going forward) will influence the sustainability and

continuation of WASH in South Africa and Zambia. In addition, there has not been an articulated

strategy or policy approach from DFAT (AG1).

Lessons from AACES WASH practice:

The MDIC and Fancy Stitch withdrawal from the program provides valuable insights and lessons in

terms of sustaining the program activities and outcomes related to the above two building blocks.

For example:

• MDIC engaged with the Department of Basic Education district officials prior to their withdrawal

to develop a monitoring and maintenance plan to sustain the water harvesting infrastructure at

the schools. Indications during the MTR field visit suggest that a hand‐over strategy on its own is

insufficient for sustainability: a period of mentoring supported by training (for example in

maintaining equipment and accessing budget for repairs) is required.


• The provision of infrastructure at schools, in the case of MDIC water harvesting and Fancy Stitch

tippy‐taps, is an important step in the process of improving hygiene but must go hand‐in‐hand

with educator/teacher and learner curricula programs. For example, learners need to

understand why it is important to wash hands, educators need to be able to respond to waste

management in the classroom and school grounds, and awareness‐raising of menstrual

management with girl learners is required. On its own, the infrastructural response will not

enable the achievement of the intended health outcomes.

• The long‐term investment in building the capacity of women in Fancy Stitch is, according to the

researcher, a significant contributor to the successful empowerment of the vegetable gardening

group. The investment through Oxfam and the AACES program in providing resources for fencing

the land, erecting a shed and installing water harvesting infrastructure has altered the

livelihoods of these women and increased their health and quality of life. The withdrawal of

Fancy Stitch from the AACES program has not stopped the group: they continue to produce

food, improve their quality of life and support each other. They request advice from Fancy Stitch

when necessary, and engage on their own with local stakeholders on economic issues. This

points to the need for sustained mentorship beyond the AACES program, and partners need to

plan for this in mainstreaming support in their programs and organisations.

Based on monitoring program implementation, Oxfam staff in South Africa and Zambia highlighted

the following drivers or enablers to ensure that the program and results are sustainable:

The challenge of WASH is bigger than any one partner or community, as the duty rests with government systems that need strengthening. Oxfam Focus Group participant

• Community ownership of WASH practices, structures

and infrastructure, and a sense of belonging to a

community working towards the same goal, are

important foundations (ZF)

• Embed participatory processes in structures, plans

and activities so every participant realises the value

of participation (ZF) as an outcome in itself

• The context of WASH is not static and therefore programs, processes and structures must be

able to change and respond: what the program leaves behind is not fixed, rather it is the

learning, procedures, knowledge, curricula based materials, and functioning structures (RH)

• Relationships and how to access decision making bodies in a respectful, trusting and beneficial

way has been seen in other programs in South Africa to be an important element for

sustainability (RH)


Partnerships

The AACES program is implemented through local partner organisations in South Africa and Zambia.

As stated in the proposal for the AACES WASH program, “Oxfam’s partnership principles include

capacity building and a shared commitment to flexibility to ensure ownership and sustainable

outcomes. Oxfam partners are autonomous, independent, accountable organisations that share

Oxfam's core values and beliefs and work towards common goals on a long term basis. Oxfam

identified partners have been selected for their strong commitment to community engagement and

ownership principles and their grounding focus at community level. In line with the capacity building

objectives of the program, implementation will be shared between the various stakeholders, and as

local capacity is increased, management and oversight of the program will be transitioned to local

communities to ensure ongoing sustainability”xiv.

The MTR focused on exploring if partnerships were progressing in line with expectations and good

partnership practice. Oxfam understands partnerships as “mutually empowering relationships,

which are aware of power imbalances and focused on mutual growth, organisational development,

institutional strengthening and above all, on achieving impact”.xv

Partnership practice:

The Oxfam country offices are all part of Oxfam International and therefore there are overarching

coordination and management mechanisms. The design of the AACES WASH program included clear

roles, responsibilities, tasks and lines of accountability between the offices, however these have not

been translated into practice due to changes in staff, staff capacity and tensions in how the program

should run (AG2, AG1). As a result, there is confusion amongst partners on how the program is

managed, where the leadership rests, who is responsible for what, and where the accountability and

decision making rests (AG1, AG2, T1, T2, T3, RA, RB, RD, ZF). The consultants providing technical

expertise to the program also raised this as a concern.

In Zambia, the roles between Oxfam and the two partners were confused and the lines blurred in

the first two years of the program. Both partners felt that initially they were not given enough

authority to implement the program autonomously, and that Oxfam would “swing in and check‐up”,

or by‐pass them in decision making, or step in and take on the role of implementers. Conversely,

Oxfam indicate that the partners were not taking the lead and implementing activities, and would

turn to them for implementation activities. The review undertaken by a consultant in year two

provided insights into the situation, and the subsequent clarification of roles and implementation

approach has made a significant change (as reported by respondents in the MTR) in the

implementation of the program (T4).


A reflection on the partnership between Oxfam by the partners, particularly in South Africa but not

exclusively, indicated that there is a need to provide more timely communication and to follow up to

confirm arrangement or to notify if dates change, (RA, RF, ZA). They also highlighted that an annual

AACES WASH work plan is required to map out dates, deliverables, training and visits in advance (RA,

RF); that linkages on technical areas and sharing of resources could be strengthened between

partners and Oxfam should facilitate this to a greater degree (RA); and that the delay in funding

payments impacts negatively on the organisation’s finances, sustainability and accountability (RF).

Partner organisations indicated they valued the annual reflections, cross‐partner exchange visits (RA,

RF, RG), learning from each other, sharing resources and

connections. The AACES visit to Malawi was extremely beneficial

and resulted in TU adapting the WASH Ambassador approach. Oxfam identify six principles of good partnerships: 1. Shared vision and values 2. Complementarity of purpose and

value added 3. Autonomy and independence 4. Transparency and mutual

accountability 5. Clarity on roles and

responsibilities 6. Commitment to joint learning

Source: Working Together – Oxfam’s Partnership Principles, 2012

Partnerships between the communities or beneficiaries and the

partners are focused on the actual activities, and all participants

in the MTR indicated that partners are approachable, willing to

work with them, involve them in the decision making and

together they are contributing to the development of the

community and improving the lives of people. All partners and

beneficiaries were hugely appreciative of the support from

Oxfam and valued the openness with which they could engage

with staff.

Partnership implementation against good practice:

The findings of the MTR indicate that the partnerships are not necessarily meeting the expectations

of all stakeholders or along the lines of good practice. Responses to the MTR semi‐structured

interviews and focus group discussions indicate that:

• There is a shared vision towards achieving the WASH goals and on making an impact on poverty

and injustice. However the values and approach of how to achieve this vision differed with one

South African partner, Fancy Stitch, who withdrew from the program at the end of year two

because they could not reconcile the rights‐based or humanitarian approach with their faith‐

based approach. Despite the withdrawal of funds, some WASH activities are continuing such as

water harvesting, hand washing and basic hygiene, improving sanitation through installing

waterless toilets, and working with one primary school.

• There is a complementarity of purpose across the program interventions; however this has not

been clearly articulated to stakeholders, including the partners. The partners are focussing on


individual interventions, and there is a missed opportunity of more effectively maximising

efforts, resources and activities. For example, partners in South Africa and Zambia are working in

schools, however efforts are individual whereas a combined effort of sharing, engaging with

School Governing Body (SGB) forums and school management teams, and collective engagement

with district and provincial government could increase the effect and achievement of the

outcomes. Similarly, efforts in both countries are developing more accessible toilets for people

in wheelchairs, but it is not clear at a program level how these two efforts are being integrated.

The complementarity of purpose further highlights the need to clarify the theory of change for

the program, and to engage stakeholders in developing and understanding the complementarity

of the work they do to the overall changes the AACES WASH program aims to bring about.

• There are tensions between partners and Oxfam due to power imbalances based on funding,

transparency and accountability. During the annual reflection meetings for the program, space is

created for dialogue and debate around the values, goals, activities, results and priorities of the

program, and an openness and acceptance that people and organisations may have different

views and values. However, concern was expressed by two partners that there is a lack of

transparency about the funding of the program (RD, RF) and that the budget is not necessarily

being used to maximise effectiveness and add value (ZP). LIMA expressed a concern of their role

in the program and dissatisfaction at being side‐lined once the program began. Initially they

were included in the proposal in a technical lead capacity, however after the PRA process

(discussed previously), their role shifted to that of an implementing partner and the technical

capacity has not been provided to other partners or from a country level response in South

Africa. This has been raised and discussed with Oxfam; however there is no clarity on how this

issue can be addressed.

• As highlighted above, there is a need to clarify roles and responsibilities of Oxfam in the

leadership and management of the program at an inter‐country and country level. Agreements

are in place at all levels, however the practical application of roles has changed from program

inception and has not been clarified by Oxfam, and therefore no clear communication or

guidance has been provided to partners. Although the partners and Oxfam trust each other (this

was articulated during the KII) there are some concerns regarding the effectiveness and

efficiency of communication, receipt of funding tranches, and competence of Oxfam leadership.


Sustainability Overview

The sustainability strategy is embedded throughout the life of the program: from design,

implementation activities, management strategy, and in placing ownership at individual and

structural level. The program design considered how to embed and support sustainability elements.

Program practice, as presented in this report, suggest that the three building blocks towards

sustainability are being implemented and capacity

building efforts are focused on individuals and

structure. However, the findings indicate that the

partnership practice must be improved in order to

embed sustainability in program practice; the hand‐

over strategies and longer‐term financial support where necessary needs to be discussed with

partner now to allow them time to integrate the WASH approach further into programmatic

responses; the rights‐based approach needs to come to the fore of the program more and there is a

need to deepen the program to achieve the outcomes: in other words to shift from focusing on

outputs to strengthening the achievement of the outcomes.

“WASH is a lever to achieve sustained health outcomes”

Partner Program Manager, Zambia

Monitoring, Evaluation and Learning

A Monitoring, evaluation and learning framework was developed at the start of the program to

guide design, implementation, monitoring, reporting and dissemination plans and activities. In South

Africa, the approach taken by Oxfam was to bring in a technical consultant to build the capacity of

the implementing partners to undertake baseline studies, strengthen MEL systems, build capacity to

document practice and collect relevant data, use the results of their MEL systems to inform decision

making and program implementation, and to encourage partners to share learning, lessons,

innovation and practice at annual reflections. In Zambia, Oxfam decided to have a dedicated MEL

officer involved in the program to build capacity of partners, coordinate and assure the quality of

data collection, reporting and dissemination.

This section presents the findings of review of the MEL capacity building approaches in the two

countries.

4.2.1 South Africa11

Partners’ perceptions of training and MEL support received under the AACES program were

generally positive, with respondents highlighting that they have now developed MEL systems that

11 This review was conducted by Maud Mukova‐Moses, Oxfam Australia, Jan‐Feb 2014.


are accessible to all staff and collect more relevant data. These changes in MEL practices were

largely attributed to the provision of tailor‐made MEL support to each of the partner organisations

as well as opportunities for cross‐learning amongst the partners.

The collection of disaggregated data was described as invaluable, with some respondents noting that

this has enabled them to realise the differential impacts of gender in WASH. However, some

partners noted that while they were able to collect more data, they were not effectively utilising this

information, indicating a need for additional up skilling of staff to enhance data utilisation. Some of

the partners are also struggling to collate data to feed into analysis of longer term sustainability and

outcomes. OneVoice, Lima and CREATE12, noted that they were yet to receive MEL support to

ensure they are collecting and utilising data that is relevant to the AACES program.

Most partners stated that the AACES MEL framework was complex and initially difficult to

understand. However, through participation in AACES workshops and partner specific MEL support,

respondents noted that exposure to the framework had assisted them in focusing their activities and

clarified their organisation’s contribution to the overall AACES program. In addition, the framework

had become easier to use through on‐going application in the field. However, some partners

stipulated that the framework was still cumbersome and the data collection and analysis required

was time consuming.

One of the partners noted that the data reporting template focuses heavily on WASH infrastructure,

with limited emphasis on advocacy. This was viewed as a major gap as it reinforced the role of NGOs

as service providers rather than propellers of community led advocacy to hold duty bearers to

account. These sentiments were echoed by other respondents who highlighted that this was a broad

issue in South Africa, calling for a shift of NGOs’ focus from service provision to building the capacity

of citizens (rights holders) to demand services from relevant government authorities.

Oxfam staff noted that while partners had improved their MEL practices, they were still experiencing

difficulties in collating data provided by partners for AACES program reporting. This was attributed

to the current reporting framework which required partners to report against the No Longer

Vulnerable (singe country program) framework. While this approach is beneficial in facilitating

program integration, the current reporting templates do not capture data required for the AACES

report. Oxfam has recently started sharing the overall AACES program report and feedback from

DFAT with partners, which has increased their appreciation of data required and how it is used.

12 These partners joined the program in Year Three as implementing partners when MDIC and Fancy Stitch left; all three were involved in the program from inception. Their inclusion meant that core program activities could be scaled up.


While there has been limited data available for program reporting, staff noted that they have been

able to share available program data and learning with other Oxfam affiliates and through DFAT

newsletters and learning platforms. In addition, program information has been shared with

government stakeholders. For example, a recent Community Led Total Sanitation (CLTS) reportxvi has

been shared at a round table discussion with government authorities, with a follow up meeting

expected at the end of February 2014.

Oxfam shared similar sentiments with partners, noting limitations in service provision by

government authorities. Staff noted that this has had negative impacts on the sustainability of

program outcomes, with NGOs focusing on social support and welfare provision at the detriment of

active citizenship engagement to hold duty bearers to account for service delivery.

Qualitative data collection, data utilization and beneficiary participation

Respondents described different activities in data collection which included project reports, staff

meetings, community dialogues, focus group discussions and one‐to‐one discussions with

beneficiaries. Beneficiaries were also engaged in video recording and story writing. These processes

are used to facilitate feedback between partners and communities, as well as engage government

stakeholders. One of the respondents expressed that by providing feedback to communities, NGOs

may be creating expectations within communities that they will provide services where gaps are

identified. Instead, feedback should be provided by community members (e.g. Water User

Association members) so that the community focuses on engaging relevant government authorities,

rather than looking to the NGO.

All of the partners viewed qualitative data collection as instrumental in creating a space for

communities to share their perspectives based on their own understanding and interpretation. The

partners are primarily using narrative story telling techniques including dialogues, success stories,

photos and videos.

Partners have received training in mobile phone technology, storytelling and video making, with

most respondents noting that these methods were easier to use and understand than quantitative

data. In addition, these were described as more interactive, making it easier to engage communities

in MEL. However, one of the partners noted that video editing was time consuming and the

introduction of new software was affecting their computer systems. Suggestions were made to

provide training in the use of programs already available to partner organisations, such as Microsoft

Movie Maker.

One of the respondents highlighted that the AACES MEL framework had a strong focus on the

collection of quantitative data, and particularly on disability inclusion, and recommended a


framework review to include qualitative indicators that allow for the collection of disability inclusion

related changes in program implementation.

Qualitative data collected was largely used for reporting, project review and as evidence of change

and shared with stakeholders including donors, government departments and beneficiary

communities. Respondents noted that it was easier to demonstrate change using qualitative data,

suggesting that because communities were more engaged in the collection of qualitative data, they

found it easier to relate to.

MEL principles highlighted as important by partners included open and on‐going communication,

engaging different stakeholder levels, and high levels of stakeholder participation. Partners

expressed that a good MEL system ought to be easily understood and accessible to different

participant levels, from program directors to beneficiaries at the grassroots level. Respondents also

noted the needs for baseline data and reliable benchmarks to measure progress against.

4.2.2 Zambia

The MEL capacity of partners in Zambia is considered by partners and Oxfam staff to be “below what

is required” despite various training and workshop sessions partners have attended previously.

There are gaps in knowledge and practice at both a partner organisational and individual level. These

include understanding the MEL framework and indicators in sufficient detail to report on (ZA),

providing evidence for qualitative indicators (ZA), and understanding the outcomes in more detail

(ZA). Monitoring skills appear to be sufficient in terms of process indicators, but understanding what

to observe or report on for outcome (particularly qualitative) level indicators needs strengthening

(PPS). In addition, further support is required for partners themselves to develop MEL frameworks,

indicators and tools (ZE), and to document case studies (ZA, ZE). From the Oxfam perspective there

is increased support, commitment and dedication to developing an effective MEL system, however

individuals in the organisation’s lack experience and therefore capacity building is required (ZF).

There was an extended period when the program did

not have an Oxfam MEL officer, and therefore minimal

capacity building and support could be provided to

partners. However, the appointment of a new officer

who started at the beginning of year three has meant

that one training workshop has been held with all partners, individual discussions have been held to

improve partner data collection and reporting, joint monitoring visits have been undertaken, and

there is a plan going forward for both data collection and reporting, as well as strengthening

capacity through technical support.

“We need to shift from quantitative [activities] to influencing qualitative

[outcomes]. Partner Director, Zambia


The partners are only now significantly engaging with the MEL framework, and the understanding of

this has shifted the thinking and practice from understanding the program as providing aid to

empowering communities to influence and seek solutions to gaps in their development. Oxfam, at

the beginning of year three, was translating the MEL Framework into a MEL plan with partners to

strengthen implementation.

Qualitative data collection, data utilization and beneficiary participation

Data is collected through partner MEL systems and tools, including writing field visit reports,

observations and collecting data through the community structures. Respondents indicated that

they are now starting to document the process and case studies, but they are ‘not very strong’ at

this.

Oxfam indicated that across partners and the program, there are “too many tools used at different

frequencies” (ZE), and that one comprehensive set of tools is needed for the program to gather data

from the household level (ZE, ZA). In addition stories need to be collected to provide qualitative

indicator evidence however there is a lack of capacity and experience to do this to ensure value and

relevance of the story. PPS have written case studies on each of the geographic areas where they

operate. YWCA indicated that they have tried developing case studies but these have not been

successful. One of the priorities going forward is to share the information and data gathered with

the partners and stakeholders, to link partner efforts, and to create more space for sharing of

practice, challenges and solutions, and to engage in dialogue on how the program is meeting the

outcomes.

There has been limited involvement of beneficiaries in the MEL process, except as providers of data,

however they were extensively involved in the gathering of baseline data. The MTR process provided

an opportunity to engage beneficiaries in a reflective process of achievements and challenges to‐

date, and for beneficiaries to engage in the development of the program going forward (ZE).

The quarterly partner and Oxfam meeting provide an opportunity for reflection and to “sharpen

skills” in MEL practice (ZE).

4.2.3 Learning and sharing between Oxfam WASH AACES partners

Objective five of the Oxfam AACES WASH program is to document and share learning, and thereby

inform policy, public engagement and program development and growth. Space has been created

for exchanging ideas, sharing experiences, reflecting on progress and practice, and bringing in

technical consultants and innovative practices, in order for partners to learn, share and adapt ideas

in their programs. These have taken the form of annual reflections, training (such as the Disaster


Risk Management training), and exchange and site visits between partners. Partners indicate that

these spaces are incredibly valuable and have added enormous benefit to the growth and

development of themselves, their organisations and the programs.

Engagement amongst AACES partners in South Africa and Zambia was generally viewed positively,

with respondents indicating that they had opportunities to share experiences and establish new

ways of working. The establishment of an AACES partner network in South Africa, in which Oxfam is

not a participant, was applauded as a great initiative amongst the partners and a good indication of

the sustainability of linkages and working relationships developing across the partner portfolio.

While all partners observed that there were frequent opportunities to interact, the smaller

organisations13 expressed that the workshops and cross learning events were time consuming and

tended to take partners away from their core work. One of the respondents in South Africa noted

that they sometimes felt intimidated, as a small organisation, to be seen as not actively participating

in learning initiatives. Suggestions were made to reduce the length of workshops, and encourage

exchange visits to partners within the same province14.

The partners and Oxfam in both countries are documenting activities and changes (as discussed

previously in this report) at local level. This includes DVDs, photographs, stories, reports and the

development of case studies. These have been shared at annual partner meetings, however, further

work is required before they can be used externally or internationally as they are implementation,

operational and output level focused. For example, changes in the lives of people with a disability

and the challenges they face in accessing appropriate WASH infrastructure and services is being

documented, and as innovative or relevant solutions are being developed by partners these are

being documented by the partners themselves. However at this point it is still a work in progress (FS,

WM, ZAPD).

Oxfam recently started sharing the overall AACES program report and feedback from DFAT with

partners, which has increased their appreciation of data required and how it is used. While there has

been limited data available for disseminating, Oxfam have been able to share available program

approaches and learning with other Oxfam affiliates through newsletters, DFAT newsletters and

learning platforms (AG1).

13 Perception of organisation size is based on respondents’ own interpretations. 14 Maud Mukova‐Moses, Oxfam Australia.



Assessment towards achieving AACES program objectives

This section assesses progress against program objectives (at both Oxfam and partner levels) and

reflects, based on the evidence, if these continue to be relevant for both South Africa and Zambia.

AACES Objective One: Marginalised people have sustainable access to the services

they require

To a large extent the program is on track to achieve this objective. This assessment is based on the

results as reported in the Oxfam AACES Six Monthly Snapshot Reports,xvii and the evidence from the

MTR, which was presented in detail in the previous sections and is summarised against the AACES

WASH indicatorsxviii in the table on the following pages.

Table 5: Country program achievements towards AACES Objective One against the Oxfam MEL Framework indicators

South Africa Zambia

GOAL: The health and quality of life of the poor and vulnerable in targeted areas of Zambia and South Africa is improved.

1. Increased access to, and effective use of, improved integrated and sustainable water supplies, sanitation and hygiene (WASH) services.

Reported results for period June 2011 to December 2013xix: Number of additional people with access to sustainable and safe water = 12,941 Number of additional people with access to appropriate sanitation = 2,473 Number of additional locations with hand‐washing facilities and soap = 1,774 Number of reached with increased awareness of hygiene practices = 7,686

• Increased access to safe and clean water through water harvesting at schools (WM, MDIC), household (WM) and community vegetable projects (LIMA, FS), and working with communities to identify solutions to protect springs and wells, or extending water pipes to increase access, or through identifying non‐functioning wells and reporting this to duty‐bearers (WM, TU, FS)

• Increase to appropriate sanitation through promoting and demonstrating use of waterless toilets, and building them places where community members are able to access them (WM, FS)

• Increase in number of locations with hand washing facilities and soap (particularly through use of tippy taps) (WM, STCKZN, TU)

• Increase in knowledge and practice of hand washing and other hygiene practices at crèche, school, beneficiary and household level (STCKZN, FS, TU, WM)

• Increased access to safe and clean water through protection wells, springs and community water pumps (PPS, KFZ, VWZ)

• Increased access to appropriate sanitation particularly through building of pit latrines at schools (PPS, KZF)

• Increase in number of locations with hand washing facilities and soap (although not always available in poor and rural households) (PPS, KFZ)

• Knowledge and practice of hand washing and other hygiene practices with children and households, and through the work of the trained community facilitators (PPS, KZF)

2. Reduced WASH‐related inequalities in gender and vulnerable groups.

• Significant achievements have been made to integrate and mainstream the rights of people with disabilities into WASH programs, and increase capacity of partners, beneficiaries and people with disabilities to voice rights. Individual cases to reduce inequalities of people with disabilities have been addressed by partners (FS, TU, WM). WASH standards for people with disabilities underpinning approach and activities (CREATE)

• Increase in children and youth accessing improved sanitation, hygiene and quality water at crèches and schools (STCKZN, FS, TU, WM).

• Achievement in raising awareness of rights and WASH related issues of people with disabilities to partners, community structures and people with disabilities (PPS, KZN, ZAPD, ZAFOD).

• Increase in school going children to accessing improved sanitation, hygiene and quality water (PPS, KZF, VWZ).

• Increase in rural households accessing clean water, sanitation and improved hygiene (PPS, KZF)

• Increase in awareness, understanding, identification and possible action of children who are abused or neglected by school staff, and secondary awareness and


• Rural and peri‐urban households, schools or crèches increased access to clean water, sanitation and improved hygiene (STCKZN, FS, TU, WM).

• Increase in child protection awareness and inclusion in policies of partners (STCKZN, WM, TU, RAPCAN).

• Gender awareness with partners at annual reflection and workshops held (WM, TU).

• Women engaged in WASH training, community implementation, school or crèche action, and increased access to WASH services at household, crèche or school level (WM, STCKZN, FS, TU).

changes on parents/guardians (PPS, KZF, school focus groups).

• Awareness raising and engagement with partners and stakeholders on gendered responses to WASH (YWCA). Training of 20 women in construction aimed at economic empowerment (KZF) and increasing capacity within communities for building and sustaining WASH services This also increased the opportunity for men and women to benefit more equally form financial resources within the programme (narrowed gap in technical skills between men and women in project area).

• Women attend community meetings and elected to committees but limited participation in formal discussions (ZL, ZM, ZN), however valuable insight expressed during focus groups.

• Increased awareness of rights of people with disabilities, women and children in communities and amongst partner / government officials

3. Strengthened capacity of stakeholders to manage and implement WASH programs on a sustainable basis.

• ECD Forum at district level (STCKZN) are functional (meeting regularly, minutes of meeting, elect officials), raise awareness and group decides on collective action

• ECD practitioners have increased capacity to manage and implement WASH infrastructure and trying to overcome infrastructure or resource barriers (STCKZN, WM, TU), able to implement WASH programs (TREE, STCKZN, WM).

• School Principal, educators, Management Teams and School Governing Bodies engaging in management and implementation of WASH programs, but dependent on partner for leadership, technical expertise, awareness raising and education (FS, TU, MDIC, WM, OVSA).

• Increased awareness of structures roles and responsibilities in providing and managing WASH services, but not yet at stage of independent action, ownership, leadership or accountability (MDIC, WM, TU, FS, OVSA)

• Increased ability of 30 community facilitators and 15 district government officials in Mongu district in the use of participatory methods and tools to assist communities to identify, and analyse their own development challenges.

• Increased ability of communities to conduct participatory broad based baseline surveys and develop community plans with the help of community facilitators. Development of ward plans to create a basis for communities to engage with government and the private sector

• Enhanced abilities of over 77 Village WASH committees to conduct hygiene promotion, raise local resources and manage and implement WASH (and other) community projects for the long term (PPS, KFZ)

• Increased abilities of about 44 women and men Area Pump Menders to operate and maintain water points



• Engaging with Parish Councils to lead, guide and monitor vegetable and income producing projects, but long‐term investment required to build sustainability (LIMA).

• Establishment of two Water Users Association by TU and ongoing capacity building

• Integrating WASH activities into two existing Community Women’s Group food security programs (FS)

• Increased community awareness through WASH days (STCKZN), demonstration sites and community dialogue (WM), work of caregivers and facilitators / beneficiaries in partner programs (TU, MDIC, OVSA, LIMA, WM, STCKZN, FS)

• Engagement and inclusion of traditional and civic leadership structures and individuals in all stages of the WASH program by partners (PPS, KFZ)

• Training and capacity building of 20 women in construction skills to equip these women to manage and implement WASH and other community development programs (KPS)

4. Improved WASH governance and effectiveness.

• Increase in capacity of partners and communities to understand rights related to WASH, roles and responsibilities of duty‐bearers and beneficiaries , and more so in year 3 roles to support WASH accountability (WM, TU, STCKZN, OVSA, FS)

• Engagement by FS with Municipality on providing accessible sanitation services in the Ingwavuma town

• TU beneficiaries engaging with Manguzi local municipalities for WASH hardware and with local water schemes regarding the high water tariffs and calling on government to intervene when necessary.

• Improved dialogue amongst AACES partners as well as engagement on WASH issues by partners with all relevant stakeholders is ensuring a more coordinated and appropriate response to WASH issues.

• Working with School Governing Bodies and Education and Health department in the schools (OVSA, WM, TU, MDIC)

• Increase in capacity of partners and communities in their understanding of rights (WASH, gender, children and people with disabilities), as well as understanding of practices, roles and responsibilities of stakeholders in providing WASH services (KZF, PPS)

• Increased participation of communities in needs identification and assessment as well as development planning – increased decision making by communities (for instance communities decided which women were to be trained in construction)

• Regular discussions with duty bearers such as the Town Clerk and District Councils to share progress, challenges and engage with them on roles, responsibilities and accountability (Oxfam)

• Advocacy within District and Provincial Development Coordinating Committees on the strengthening of Ward Development Committees (Oxfam)

The table above illustrates the achievements against the program objectives, and indicates that the

program is on‐track in meeting the Oxfam objectives one, two and three. Efforts during year three

have an aim to continue to increase the focus on objectives three and four of the program now that

the ground work has been laid.

However the following barriers are noted that may affect the achievement, outcomes and

sustainability of these results:

1. There is a tension in the program during the first two years regarding the primary focus of

partners on infrastructure vs. capacity building, awareness raising and strengthening structures.

In many instances, partners and beneficiaries indicated that infrastructure, and often an

inexpensive or once‐off investment, would reap huge returns and without this they were

hindered in achieving the expected outputs and outcomes. However, the program focuses more

on capacity building, and this caused a tension in how they responded to beneficiary needs

when the demand for infrastructure is great. More recently, the debate and tension is around

how to incorporate advocacy and influencing work into the service delivery approaches.

2. The partner activities have built a sound basis in terms of increased awareness and knowledge,

influencing WASH practice at individual and household level, and strengthening structures

through knowledge, mentoring management and leadership, and increasing awareness of rights,

roles and responsibilities. This needs to be taken to the next level where there is increased voice

and action towards holding duty‐bearers to account – whether this is internal to the structure

(for example to SMT and SGBs) or external to traditional and government structures. This must

be scaled‐up in order to achieve the outcomes of the program and to sustain structures, systems

and behaviours in the long‐term.

3. The rights based approach and the program design highlight the need for an advocacy strategy

and activities in order to achieve the AACES objectives (at DFAT and Oxfam levels). A draft

strategyxx is being developed in the first half of year three for implementation.

4. There are budget constraints to implementing the program at partner and country level.

Although this MTR did not conduct an in‐depth value for money or costing study, concerns were

raised by respondents that the lack of transparency of financial allocations in relation to

outcomes is disempowering, and that there is no flexibility or access to funds for either

innovation or urgent responses that could make an enormous return on the investment if it is

out of the approved budget. For example, the women trained in construction are unable to use

their skills and generate income effectively as they cannot start a small‐scale business in the

community because they cannot afford the transport to get the necessary resources.

Oxfam AACES WASH MTR MARCH 2014 Page | 66

AACES Objective Two: DFAT policy and programs in Africa are strengthened

particularly in their ability to target and serve the needs of poor and vulnerable

people

Since the inception of the AACES WASH program in South Africa and Zambia, a number of activities

have been undertaken regarding Oxfam’s policy engagement with DFAT. The Oxfam approach has

focused on three core areas (as described in the design document). A summary of the key activities

undertaken over the past two and a half years is presented below to indicate how and to whom

evidence from the program has been made available to inform policy and practice.

A. Policy dialogue relating directly to issues arising from the programs in South Africa and Zambia

a) Oxfam, in consultation with Water Aid and other AACES partners commissioned research on

Community Led Total Sanitation (CLTS) within southern Africa to: understand how it is being

implemented; identify challenges and gaps with the approach; what participatory

approaches are being used; identify the perceived facilitators and inhibitors of successful

CLTS implementation in Southern Africa; and highlight government’s positions on CLTS in

selected countries.

b) The research report was finalised and published in year two, and was shared with AACES

partners and the South African Department of Human Settlements and Water Affairs, and

the eThekwini Municipal department of water and sanitation. In addition, the report has

been shared with DFAT in Australia. The aim is that the research will “influence donors and

government alike, feeding into their sanitation policy and practices to ensure appropriate

and sustainable practices are supported and funded”xxi.

c) A multi‐stakeholder CLTS roundtable was held on 28 August 2013 in Johannesburg, South

Africa. Beneficiaries included DFAT, national and local government departments,

international, national and local civil society organisations, and academia. Regional dialogue

around sanitation approaches was strengthened. Further areas for research and policy

development were identified. Further discussion on the CLTS findings are planned to take

place during the International WASH Conference in Brisbane, Australia in March 2014. In the

third quarter of year three (February 2014) the findings will also be presented at the joint

Water Research Council and Human Settlements CLTS meeting.

d) Oxfam has engaged DFAT staff with responsibility for WASH policy and strategy

development in Southern Africa regarding the research through the Australian High

Commission in Pretoria.


e) Exchange visits have been held between Oxfam and other AACES partners and countries.

These included the following visits:

• Oxfam staff attended an exchange visit to Zimbabwe in December 2013 as part of the

Steering Committee meeting. Insights included understanding how NGOs and government

operate in Zimbabwe whereby government determines and monitors where NGOs work and

receive regular reports as development is a joint responsibility of both parties however

provision mainly rests with the NGO sector; the provision of washing facilities at local

markets to increase access for community members; the protection of water sources for

human and animal use; exposure to innovative farming methods; and clearly defined roles

between the NGO and community in the design and follow up care of the water source.

• Oxfam program managers visited the Water Aid projects in Tanzania in June 2013, organised

by Water Aid. Staff indicated that it exposed them to the community and integrated school

WASH projects being implemented. This included observing inclusive designs of sanitation

facilities at schools. The designs included menstruation hygiene rooms as well as wash

basins. This was also supported by a team of mothers from the communities who gave

menstruation hygiene training to the school girls. The toilets were of disability inclusive

designs – with wheel chair access and ramps. However in some schools Oxfam staff noted a

lack of integration of programming, for example in most schools the school WASH project

had no running water systems or rain water harvesting technologies. School roofs were not

guttered to feed into the school toilet systems. The schools also did not have food gardens

on the premises. Most school toilets looked new compared to the general school

infrastructure. One headmaster even commented: “I might as well use the Water Aid

supported toilets as my new offices ‐ given that my own office is falling apart”.

• The Malawi visit included the “Mutumba” Community Led Total Sanitation (CLTS) program.

The most interesting aspect of this intervention was the social marketing and the trained

community sanitation technicians who engaged at a fee in various toilet construction

projects. Community/household commitment and contributions to their toilet construction

(after the no open defecation triggering process) was said to have been key to the success of

the project.

• Lessons from the visit included the importance of integrating entrepreneurship and

sanitation marketing, and the need to develop partnerships with strong autonomous and

sustainable local NGOs. Oxfam advised Water Aid and its partners to manage the risk of

strong local NGOs taking over the responsibility of local authorities and government using


donor funds. They were also advised to link latrine improvements with advocacy on broad

school improvements.

• The Steering Committee meeting was held in Kenya in December 2013. The meeting

explored amongst other topics related to the overarching MTR, change within the AACES

WASH program; to understand how the implicit theory of change for AACES objective one

works in the context and what changes; and to understand value for money and progress or

challenges in MEL practice.

The visits provided an opportunity to share practice, identify key learning points, and assist AACES

NGOs in identifying and sharing best practice ways of working, and possible joint areas of synergy

and divergence. A result of the above activities is increased cross‐learning and the engagement

between Australian NGOs working in Africa. Oxfam is the only AACES implementing partner in South

Africa and Zambia, and therefore cross‐country and sector relationships have been strengthened

between Oxfam offices as well as Australian NGOs. This has been achieved through sharing

documents (such as baseline studies, the CLTS research report); exchange visits; steering committee

meetings being hosted in different countries; and strengthening working relationships between

country based staff responsible for WASH and other aspects of the program (for example advocacy,

child protection, disability, youth empowerment). It is recommended that a case study of the

collaborative process is documented in order to highlight lessons for other programs and

organisations on how to work together and how to engage and influence policy.

B. Policy dialogue on issues broader that the Oxfam AACES program

Over the past two and half years activities have included engagement with DFAT and the office of

the Minister for Foreign Affairs bilaterally, with other agencies, or through the Australian Council for

International Development. Activities have included:

• A Dangerous Delay, a publication jointly produced with Save the Children regarding the

humanitarian response to the 2011 food crisis in East Africa which has been used to frame

discussions regarding early warning and response – including at the Australian Council for

International Development (ACFID) Humanitarian Forum in Canberra, Australia.

• Engagement with the Australian government on the response to the food crises in the Sahel

region of West and Central Africa which has included disseminating two reports and

discussions with DFAT.

• The Oxfam Australia mining team has been holding annual Mining Symposiums for the last

four years. This meeting brings together executives from the extractives sector, their

financiers and government to discuss key issues of policy and practice that could, if


improved, lead to enhanced outcomes for people in developing countries affected by the

extractives sector. Engagement with the development, implementation and evaluation of

the Australian Government’s “Mining for Development” initiative has included:

• Attending multi‐stakeholder consultations

• Providing formal written feedback to proposals such as the Africa Mining and

Communities (AFMAC) concept note

• Participating (through Oxfam’s Executive Director) as an NGO representative on

the Mining for Development Advisory Committee

• Ongoing dialogue between Oxfam Mining Advocacy staff and relevant staff

within DFAT in Canberra and the Australian High Commission in South Africa;

• Participation in the evaluation of the “Community Support Fund”.

• Engagement on the conflict in Mali and its humanitarian consequences. This has included

two briefing notes and related discussions with DFAT staff on the mandate of the proposed

UN peacekeeping mission and “Mali: A new Development contract” provided to DFAT prior

to the Brussels donor conference in May 2013.

• A confidential briefing to the Parliamentary Secretary for Foreign Affairs prior to the African

Union summit in January 2013.

• Ongoing engagement with DFAT on food security policy issues relevant to Africa.

• A briefing paper was submitted to the office of the Minister for Foreign Affairs, in year three

of the program regarding the future of Australia’s Humanitarian and Official Development

Assistance (ODA) Program. The brief emphasised the importance of Australian humanitarian

and development assistance to countries in Africa and made recommendations regarding

principles to inform the nature of this assistance and benchmarks against which its

effectiveness should be measured.

C. Regional and or international policy engagement on issues arising from, or of relevance to, the

programs of Oxfam and one or more other Australian NGO working in Africa

Oxfam hosted a joint learning event between four Australian NGOs funded through DFAT’s South

Sudan Early Recovery and Humanitarian Funding Round in April 201315. The learning and policy

report pertaining to Australia’s ongoing support for South Sudan will inform further engagement

with stakeholders.

15 This is not directly part of the AACES program.


As illustrated from the above activities, efforts have been made by Oxfam to provide information

and increase dialogue on development issues in Africa to inform policy and practice. Oxfam, as part

of the WASH reference group, shares knowledge and experience across the WASH sector in Australia

and has been influential in engaging with the Australian government in policy design and reform in

relation to the WASH sector. However, efforts to engage, even on agreed areas for engagement (for

example by the steering committee), were not responded to by DFAT. One example cited of this was

Oxfam’s extensive and formal feedback to the AFMAC proposal. Receipt was acknowledged, but

nothing further was heard despite requesting further information. Later Oxfam heard that DFAT was

disappointed that NGOs did not take up the opportunity to engage in dialogue on the matter. In

short, successful dialogue regarding DFAT policy requires the active participation of at least two

parties – one of which has to be DFAT. It was suggested that Australian NGOs are tasked and

resourced to coordinate cross‐agency policy coordination on particular sectoral priorities.

The success of the partnership’s efforts in relation to Objective Two should not be measured by

verifiable changes in DFAT policy and practice, as these policies are contingent on a number of

factors – including political imperatives that are outside the control of departmental staff or external

stakeholders such as NGOs. For example, the planned engagement by Oxfam with DFAT’s Africa

Strategy and “Country Delivery Plan” process is ‘on hold’ while the government makes strategic and

funding decisions for the Australian Aid Program in Africa, which is beyond the control of the

department and of Oxfam. The success of Objective Two should therefore be measured by the

efforts of all parties to the partnership (including DFAT) to engage in a meaningful, ongoing and

coordinated dialogue.

As discussed under the AACES Objective One findings in this report, cross program learning, research

and experience from country based partners has informed the development of the country and

individual partner responses to WASH. The findings from the MTR highlight opportunities for further

policy and practice engagement with DFAT, other Australian NGOs and stakeholders in Africa

(including for example the African Union and the Southern African Development Community) on

issues of access to WASH services by people with disabilities, CLTS in South Africa (which could also

be explored further in the Zambian context), and community engagement strategies with duty‐

bearers. Further education and policy dialogue on these issues could contribute in the long‐term to

strengthening programs that target and serve the need of poor and marginalised people. In addition,

further research on the Value for Money approach and outcomes will provide evidence to inform

and influence Australian policy and program practice regarding the approach on developing capacity

of communities to respond to WASH, external shocks and trends, influence and hold duty‐bearers to

account, and to claim their rights, rather than taking a WASH infrastructural development approach.


AACES objective three: Increased opportunity for the Australian public to be

informed about development issues in Africa.

Activities focussing on Objective Three were planned to start in year two of the AACES WASH

programxxii. Consequently, the focus in year one and two was on designing activities for

implementation during the final three years of the program. As indicated in the Oxfam Year One

Annual Report, these included “engagement with the ‘AACES Communicators Forum’ to support

objective three activities across AACES NGOs [which continued in year two]; supporting a major

evaluation of Oxfam Australia’s previous Diaspora engagement activities, and supporting a process

to develop an all‐of‐agency Theory of Change for in‐Australia community engagement”.

In year two efforts were focussed on the development of the concept for an Information

Communications Technology (ICT)xxiii approach to engage supporters in Australia with partner

organisations, and potentially beneficiaries, in South Africa. The purpose is to pilot a tool to engage

Oxfam, the Australian public and program beneficiaries in a dialogue about development issues in

South Africa, including those relating to disability and WASH and to demonstrate how Australian aid

(through Oxfam’s AACES funded work) is seeking to respond to these challenges. The ICT project

includes monitoring and an evaluation to determine reach, impact and influence on young people.

The AACES WASH program is subject to the ANCP Development Awareness Raising Guidelines

(March 2013) and therefore the funds cannot be used for fundraising purposes, or to support

specific campaign activities that target the Australian Government. One of the purposes of engaging

in this internal work was to ensure that AACES funds could contribute to the existing agency

priorities for community engagement work in Australia, and to ensure Oxfam were working with the

relevant units to implement the work rather than establishing parallel systems within Oxfam that did

not align with the organisation’s objectives. This proved useful (as it informed the current ICT project

and the fruitful collaboration with the Youth Engagement Team, particularly the Schools Program),

but was a challenging process mainly because WASH is not a priority for Oxfam’s public engagement

in Australia (except for fundraising purposes which cannot (and should not) be supported with

AACES money). In some instances, possibilities for public engagement by Oxfam in Australia relate to

sectors that AACES funds do not support in South Africa and Zambia (HIV/AIDS) and/or relate to

campaign activities (Food Justice or Climate Change) and, therefore, cannot be supported with

AACES Objective Three funds.

In year three the WASH AACES program in Zambia will be profiled at the international WASH

Conference to be held in Brisbane, Australia, in March 2014. The title of the draft paper is

“Experiences and Challenges of Engaging Governments”xxiv and explores the approaches of four


AACES partners (Oxfam, WaterAid, CADECOM, AFAP) to build capacity of national, provincial,

district, and local government to enable service delivery; communities to be more effective in

lobbying for WASH services; and all local partners to influence government policy. The conference

will provide an opportunity to engage with Australian and international WASH experts and profile

the developments, successes and challenges of WASH as a development issue in Africa. Oxfam staff

from South Africa and Zambia will be attending, and two partners from both South Africa and

Zambia will be attending the conference. It is expected that there will be increased awareness and

profiling of the work of DFAT and Oxfam in WASH in Africa.

Evidence from respondents in the MTR (RH, AG1, AG2 and Oxfam MTR validation discussion)

indicate that it has been difficult to design and initiate activities under this objective, however the

concept and strategy are now clear and activities are progressing according to the plan. Of

importance going forward is the need to monitor and evaluate the extent to which the ICT pilot,

conference presentations and use of other educational methods reach their intended beneficiaries

and achieve the intended outcomes.

6. Success factors, challenges and gaps

This section identifies success factors and challenges in ensuring more equal access to services (of

both the program and wider services) by women, men, youth and children and people with

disability.

6.1 Success factors

The evidence from the MTR presented in this report indicates that success factors (in other words

what has driven or contributed to successes in the program) include:

• A good foundation has been built through the continuous engagement by the partners with

beneficiaries and relevant community structures and where activities have focussed on the

outputs of the program (such as training, implementing crèche materials and the provision of

basic WASH infrastructure). This foundation has built good working relationships between

partners and the community, and has demonstrated that through collaboration changes in

accessing water, cleanliness of water, hygiene practices and sanitation can be achieved, and that

these changes contribute towards improved health and quality of life.

• The Oxfam approach to the AACES WASH program of integrating WASH into existing programs in

South Africa has added value. Partners built on existing relationships with beneficiaries and

communities, and are responding to the WASH needs identified as part of their existing

programs. For example, the lack of access to water in the food security program meant that the


women’s or Church groups could not increase food production; crèches could not meet the

government minimum standards for registration and thereby protect children; and water was

not available for hand washing as a primary health care response for people living with HIV

whose health was therefore being compromised.

• There are numerous incidents of innovation and successful practice at partner level including,

but not limited to:

• ECD Crèche Forums which address real issues facing ECD practitioners and children

in districts/municipalities and when necessary challenge the state from the

grassroots level by holding duty bearers responsible for services.

• Demonstration sites that have or are mobilising beneficiaries and communities to

find easy, inexpensive solutions including waterless toilets, tippy taps, and water

harvesting. In particular where there are integrated responses, such as the work

Woza Moya is doing across ECD, schools, households, community health work,

disability, youth empowerment, food security – all of which include WASH.

• Training women in construction in order for women living in rural areas in Zambia to

generate income, build toilets, and maintain infrastructure.

• Saving clubs and structures that are being put in place in Zambia for payment of

water, and funds are then available for infrastructure development, maintenance as

well as other community priorities, such as building a classroom.

• Mapping of water points was undertaken by TU and this was instrumental in

identifying needs, but also in creating the space to engage with traditional leaders

and government on the status of them.

• Development of materials for use by the implementing partners across different

sites, including the ECD Material (TREE), school based curricula and activities aligned

to the South African life orientation curricula (OneVoice SA), and the materials

related to identifying, supporting and facilitating inclusion of people with disability

(CREATE).

• These examples of innovation and their successful implementation have given

courage to partners and beneficiaries to continue to engage in responding to WASH

challenges. In addition they illustrate that the approaches of integrating WASH into

existing programs and building capacity results are possible.

• Emerging collaboration between partners at country level in both South Africa and Zambia in

order to maximise on resource development and training, provide an integrated WASH response

at geographic sites or districts. For example Woza Moya’s collaboration with OneVoice to


support training in school‐based WASH, CREATE in strengthening their capacity to respond and

include people with disabilities, and potentially LIMA for infrastructure support. In addition, the

benefit of cross‐country exchanges to share practice, create the space for dialogue and potential

establishment of communities of practice in Africa, and adapt what is working in other areas into

local context: for example the WASH Ambassadors TU is now implementing after the Malawi

visit.

• Linking and learning, research, the independent South Africa partner meetings, participation at

conferences, training, and engagement with technical experts have all contributed to creating

spaces to inform thinking and decision‐making, influence program design and implementation,

strengthen practice, and increase understanding, ownership, leadership and management of

WASH at partner level and for individuals involved in the program.

• The capacity building of partners and individual beneficiaries has contributed to an

understanding that WASH is at the centre of community development and core to responding to

health outcomes, livelihoods, children’s participation in education, and growth and development

of every person from childhood throughout life. Indications are that as a result of this shift

coupled with integrating WASH into existing program approaches will contribute to a long‐term

commitment towards addressing WASH needs.

6.2 Challenges and areas for improvement

The evidence from the MTR presented in this report indicates the following challenges in the design,

implementation, management and achievement of the program to meet the objectives and

outcomes:

• In order to strengthen and embed structures, processes and knowledge, required for, or that

contribute to sustainability, these needs to be raised and addressed in the program during year

three. Further discussion and dialogue is needed on what is required and how this can be

achieved. Agreement needs to be reached to agree

on what exactly is being handed over or left at the

end of the program, how these can be sustained,

and how aspects can be resourced (financially,

intellectually and physically), monitored and

mentored in the long‐term.

If we just focus on hardware we are at a dead‐end, and if we just focus

on software (processes and capacity) then also at a dead‐end. We need to be clear of what and

where we can specifically manoeuvre or manipulate to achieve the desired outcomes. Technical Expert to Oxfam

• The hardware versus software approach must be

explicitly discussed as this is having a negative

effect on the success of the program. In particular,


LIMA in South Africa could be adding greater capacity and value to the partners, and this is an

under‐utilized resource. Learning across LIMA and Village Water could be valuable in how the

hardware issues are addressed and what value they add, as well as how they contribute to the

outcomes. Without addressing this issue, the program is less likely to achieve meaningful impact

at the community and individual level, as well as dampen efforts for group empowerment.

• Engagement with duty bearers and addressing social and political influences are still at the

beginning stages: the needs have been identified and structures are being capacitated to

develop plans, but this is not yet being taken forward in meaningful action. Exceptions include

the ECD Forums. There is a need to understand in practice where and how the decision‐making

occurs and how it is played out, and identify key points where it is possible to influence

decisions, as otherwise there will be no or little change in the sector and the focus in the long‐

term will continue to be on building capacity and ‘doing governments work for them’ (T1). This

will continue to contribute to dependency on donors rather than citizens capable and

responsible to lead their own development (ZF).

• Building the capacity of stakeholders to manage and implement WASH programs on a

sustainable basis is challenging, and efforts need to be strengthened to build this aspect.

• The integration of gender as a cross‐cutting theme has not successfully been addressed into the

program activities.

• Technical support has been provided across gender, disability, ECD material development,

infrastructure responses, MEL, the programme approach in Zambia and South Africa,

government and stakeholder responses and in developing youth based material. All respondents

indicated that they have given input, but there has been a lack of feedback on how

recommendations have been taken forward and on the decisions taken, they have not had

opportunities to follow up or continue support for implementation, and that budget constraints

have limited their involvement – and the expectations at the start of the program have not been

lived up to. All are willing, eager and keen to continue involvement and make a far more

meaningful and significant contribution to the program.

6.3 Gaps

The MTR findings point to the following gaps or areas that need strengthening going forward:

• Revisit DFAT’s decision not to allow new partners into the program in year three in order to

meet the needs of beneficiaries and implementing partners when the response required is

beyond their own expertise, particularly in terms of ensuring more equal access to services, as

this could negatively impact on the program.


• Partner and beneficiaries have not had the capacity to lobby and undertake advocacy work up

till now in the program. However the strategy is being developed and plans are in place to

increase this capacity in year three.

• The program has not focussed on building the capacity of stakeholders to fulfil their

responsibilities as duty‐bearers, at school, ward and district level. The current approach assumes

that by building the capacity of community structures their voices will increase duty‐bearers

understanding of roles, responsibilities and they will take action. The findings at schools

however indicate that a more direct capacity building approach is needed to equip SGBs, SMT,

Principals and school caretakers with the knowledge of for example, how to access budgets to

maintain equipment, how to fix and maintain the infrastructure, how to integrate WASH as a

basic primary health care element that enables and improves learning outcomes.

Recommendations to address the challenges, strengthen program implementation and to guide the

achievement of the outcomes and goal of the program are detailed below.

Recommendations to Improve Future Delivery

The purpose of this section of the report is to provide recommendations based on the gaps and

areas for improvement that the evidence points to. These are presented below at the Oxfam AACES

program and partner level.

7.1 Oxfam AACES Program level recommendations

The evidence from the MTR presented in this report point to a number of recommendations to

improve the future development, delivery and sustainability of the program.

7.1.1 Review the understanding of the Theory of Change to strengthen program delivery

There is a need to map partner work against the program outcomes and to re‐visit the Theory of

Change now that the program approach has been translated into partner activities, lessons from

practice are emerging and there is a deeper insight into the realities of WASH and the expected

changes the program seeks to bring about. The deeper understanding of the theory of change could

help partners and communities to shift their thinking from the infrastructure versus

information/capacity building approach in that it would demonstrate the dual pathways and

interconnections of the approach: both are critical elements however what is important is how

infrastructural obstacles are overcome through the successful building of capacity.

The development and engagement with the theory of change needs to happen at the Oxfam

program level, as well as with partners and the beneficiaries. Currently, the theory of change

approach and understanding is highly theoretical, and the value of actual practice would enhance


the refinement of the approach, as well as significantly deepen partner and participant (particularly

committees, forums and facilitator/community worker) understanding of how their work

contributes to the overall outcomes and goal of the project, as well as what shifts and changes need

to occur to achieve this. Without the collective engagement with the theory of change approach and

a deepening of understanding practice it is the researchers concern that the program will continue

to be implemented but the outcomes related to stakeholder capacity, duty bearers accountability,

and real improvements in WASH governance and effectiveness will not be achieved.

The work in year three on the advocacy strategy and building capacity at partner and participant

level is a critical element to this, but without understanding how the different elements of the entire

program fit together and contribute to the overall success of the program, the final achievement of

the objectives will not be met.

7.1.2 Strengthen strategy and practice for advocacy, rights based approach, holding duty‐bearers

to account and including communities in all stages of the program cycle

There are four areas of the program that would benefit from a clearer strategic design, all of which

are essential elements of the program approach and will contribute to the achievement of the

objectives. These are:

a) Advocacy: the evidence from the MTR suggests that partners are at the stage where they would

benefit from capacity building in the area of advocacy and skills in how to influence policy and

practice. Partners in turn need to be able to build the capacity and skills of community structures

to influence policy, practice and duty bearers. The role of Oxfam is to develop, in collaboration

with partners, a strategy to integrate advocacy and influencing work across the WASH program

(for example identify where position papers could add value, host breakfast meetings etc.);

monitor the partner level of effectiveness of the strategy and the ability of communities to act

on their own to take forward; and to mentor, encourage and support partners in ‘knowing

whose door to knock on’, in other words to share structures and processes where partners and

communities can engage with duty bearers. Budget needs to be allocated for the achievement of

this.

Successes need to be built on incrementally to encourage partner and community engagement

with duty‐bearers and to take collective action. For example, in Zambia there is a platform for

engaging with government leaders (started by UNICEF), however the WASH partners need to

actively participate and claim this space. In South Africa, the districts now have ‘WAR Rooms’

which are monthly development coordination meetings of local government, communities,

business and civil society. Partners can use these as a platform of engagement. Lessons from the


engagement by TU for example could be shared with partners to support each other in this

engagement.

b) Rights‐based approach: This underpins all of Oxfam and partner’s work, including the WASH

approach and activities. As highlighted in the findings of the MTR, there is a missed opportunity

of bringing this rights based work to the centre of the activities. This is an important element of

the advocacy work, as without understanding and articulating how rights inform and drive the

influencing work the advocacy element will continue to be seen by partners as not part of their

direct community development approach and role. How the rights based approach is

understood within the context of each country will differ because the political, social, economic

and governance influences differ. Similarly, the understanding will differ at partner level

depending on the core focus of their work, the context within which they work, and their

approach to rights‐based work.

There is a need for approaches, practice, challenges and successes in working in a rights‐based

approach within the WASH sector to be articulated across the program and partner level, to be

documented (for learning, sharing and to inform practices) and to inform the sustainability

approach and practice at the community level.

c) Engagement with duty bearers: Building the capacity of partner and communities (through

advocacy and rights‐based practice) to influence and hold duty bearers to account is what the

program needs to focus on going forward, while at the same time continuing and sustaining the

work to‐date. Building trusting relationships and facilitating engagement between the

community and duty‐bearers is required for long‐term impact. The linkages in both countries are

not very strong, so there is a need to build the capacity and practice of communities so that their

voices are heard and there is meaningful engagement. The implications of providing capacity

building to duty bearers where relevant need to be considered at the program at partner level.

For example, school management teams could benefit from basic WASH training and this would

have a huge return on the sustainability and return on the investment of mainstreaming WASH

in school management, governance and contribute to teaching and learning outcomes.

d) Community participation across program cycle: Community participation is essential to the

sustainability of the program, and the level of participation needs to be enhanced through

awareness, capacity and practice. The program and partners are building this incrementally in

the sectors, however particularly at school level this needs to be enhanced and strengthened. At

the community level there is a need to move the community to think in a developmental way

and reach a point of activeness. The next eighteen months of the program will be critical in

building the capacity, mentoring and encouraging community structures to actively engage with


all stages of the program and with duty bearers. The concern is that without communities’ active

engagement the results will not be sustained and the long‐term benefits will not be achieved.

7.1.3 Strengthen program monitoring, evaluation, evidence‐base, documentation and learning

The qualitative aspects of the MEL framework require strengthening, particularly in how to use the

results. Partners are collecting stories, but capacity building and support is required to package and

present it for their use as well as for understanding program changes, barriers and successes. It is

also necessary to develop a collective understanding of what influence or results could be looked at

now that would be relevant at the end of the program: a strategy is required to understand what to

document and for data collection to begin for contributions to an evaluation report, documentary or

publication. A clearer understanding of the theory of change will inform this strategy and assist

program and partner staff to develop pathways or story‐boards of themes and stories that could be

collected. Further research and development of understanding value for money approaches need to

be undertaken and integrated into the future activities and plans.

There is a need to review the MEL Framework and include additional indicators pertaining to the

following measures:

a) Food security in South Africa and livelihoods in Zambia

b) Integration of WASH within existing programmatic areas

c) The influence of the program beyond the AACES partners, in particular the influence on other

Oxfam partners particularly around WASH, child protection, and disability

d) The leverage of additional funds and resources

e) The scope and understanding of inclusivity of people with disability suggests that further

indicators may be necessary as the current framework, developed at the start of the program,

may not adequately address the approach and practice.

The table below provides further detail on the recommendations going forward pertaining to MEL in

South Africa and Zambia:

MEL Capacity

Building

• Include support in data utilization for partners in the upcoming program annual plan towards more effective use of monitoring and evaluation data in program planning and improvement, as well as advocacy for policy formulation and implementation.

• Engage partners in reviewing their MEL systems to ensure there is a common understanding of terminology (e.g. impact, outcomes and outputs) and to initiate outcome mapping activities to assist partners in collecting and analysing data for longer term outcomes.

• Support partners to implement participatory planning, implementation, and monitoring and evaluation practices to enhance beneficiary and stakeholder participation.


AACES MEL

Framework

• Conduct a mapping of partners’ WASH project objectives and indicators against the AACES MEL framework to identify and address gaps in data collection and establish clear links with partners’ work.

• The AACES MEL framework should be reviewed to ensure that indicators clearly link to the program’s advocacy objectives. This will assist partners and beneficiary communities in articulating and demonstrating the roles of, and links between rights holders and duty bearers. This should include learning from the program in Zambia where Oxfam and partners have a closer working relationship with government authorities.

Program

reporting

• Consider reviewing the Oxfam ‘No Longer Vulnerable’ program templates to ensure that they are more flexible and accommodate data collection and reporting on individual projects. This should include determining points where projects come together and need to remain independent to ensure that all necessary data is captured.

Data collection,

data utilisation

• Consider utilising video editing software that partner organisations already have to ensure alignment with their computer operating systems.

Learning and

sharing

• More consideration should be given to availability of partners’ resources (time and personnel) when planning learning activities. This may include conducting learning activities for a shorter period, at longer intervals throughout the year, or in close proximity to partners’ project sites to assist partners in reducing time away from their core work.

• Consider platforms for engagement between the Oxfam teams in South Africa and Zambia to facilitate on‐going learning on the application of the AACES MEL framework. This may include sharing experience in qualitative data collection, analysis and utilisation, and collaborating in the interpretation of evaluation findings.

7.1.4 Clarify Oxfam management structures, roles and responsibilities

There is a need for Oxfam to clarify the leadership and management roles and functions of the

different country offices and staff in the program. It is further recommended that there is a

mapping of teams and functions across objectives to illustrate how partners contribute and link to

people/teams. The Oxfam AACES WASH program is now well into the implementation stage in South

Africa, and therefore a shift is required in how they work with partners ‐ levels of communication

and responsibility need to change. In Zambia program implementation has increased during the past

six months, however the partners will continue to require support in shifting from focusing on

output activities (such as numbers trained and provision of infrastructure) to understanding and

implementing activities that contribute to issues of governance and the long term goal of the

program.

Across both countries there is a need to provide an annual plan detailing when annual reflections

will be held, exchange visits and other key timeframes or deliverable. Ensure more regular

communication between Oxfam and partners to facilitate increased feedback on program progress

and support requirements. This may include telephone conversations, e‐mail updates or field visits.


Underlying this request by partners is a need to clarify and harmonize expectations as the messages

are often mixed. It is acknowledged that this may not be easy to undertake given that Oxfam works

towards an integrated program design that partners are familiar with, while AACES demands

separate meetings and forums.

7.1.5 The program requires a more flexible approach to bringing on board additional partners to meet changes in the design and implementation of the AACES WASH program in South Africa and Zambia

As the program has been implemented over the past two and a half years, there has been a huge

learning curve in how WASH can be implemented in both South Africa and Zambia, and what is

required for the successful achievement of the outcomes. As a result, in both countries there has

been a need to bring in additional partners to support implementation and address gaps, or the

withdrawal of partners from the program. However, the DFAT program design has not been flexible

about taking on new partners within the time frame due to a concern from their perspective on the

compliance of partners and how to bring them up to speed in the program to ensure effective and

efficient results. However, Oxfam implements programs through partnerships and ensures

compliance across all partners, and therefore bringing on new partners would not require a huge

effort or resource allocation to bring them into the program. In addition, a number of required

implementation services have been part of the program since its inception, and are therefore

already part of the program but are limited in how they contribute meaningfully to the objectives

and goal. For example, TREE could extend the crèche material implementation across many more

crèches, Zambia requires a gender partner (not as a service provider but to actually implement

necessary elements of the program), and South Africa now requires an advocacy partner, such as

Section 27. Oxfam works through partners, who are autonomous and have different strengths and

limitations, and therefore the work only gets implemented by bringing in partners and ensuring that

there is a mix of partners to address required implementation areas. The program is ready to

undertake advocacy work at a higher level; however the current approach will not achieve the

desired outcome without a partner who can do the work, and take the evidence and voices from the

communities to engage with the policy of the government, particularly in South Africa.

7.2 Partner and country level recommendations

The following recommendations are made based on the evidence presented in this report to

strengthen the implementation of the AACES WASH program in South Africa and Zambia.

a) In both countries, there is a need to strengthen the coordination and integration of

implementing partner’s activities. Partners have developed skills and expertise in their core

service area, and they need to continue to deliver these activities with their intended


beneficiaries. However, partners are working within the same structures or facing similar

challenges, and to prevent wasted resources and efforts a more collaborative approach needs

to be undertaken. For example partners are working in schools and are starting to share

resources, provide cross‐training opportunities and share practice. To scale‐up activities,

deepen WASH responses at sites and community level, and to achieve the expected outcomes

(such as improved WASH governance and accountability of duty‐bearers), greater collaboration

between partners is required. For example South African partners working in schools could

collaboratively engage with district managers and the provincial department to advocate for

adequate water and sanitation services or to influence norms and standards regarding school

infrastructure. The Zambia gender provider and disability organisations need to work closer

with the two implementing partners to deepen the gendered response within these

organisations for sustaining and mainstreaming gender within their sustainable livelihood

programs. Partners requiring technical services could benefit from input from the WASH

technical partners (such as LIMA and Village Water), but this appears to be ad hoc and value

would be added by having open discussions or agreed ways of working together that address

budgets, communication, reporting and accountability. A focus in the short‐term (by the end of

year three) will provide a basis for ongoing collaboration over the next two years (and beyond)

and allow partners to increase efforts on achieving outcomes.

b) The advocacy and rights‐based approach underpins the program approach and work of all

partners, however within the AACES WASH program has not been brought to the centre or

forefront of the activities or mindsets of the implementing partners or the beneficiaries. The

evidence from the MTR indicates that there are excellent examples of the rights‐based

approach and advocacy, for example through the disability work in both countries, the ECD

Forum collective action in two municipalities in South Africa, and increased awareness of WASH,

health and children’s rights. However, in general when engaging with beneficiaries in Zambia

during the MTR field work there remains a dependency on Oxfam to provide services, and the

acknowledgement of who duty‐bearers are and their role was not easily articulated. The

development of the advocacy strategy, training for partners and clear action plans will go

some way to strengthen the advocacy skills of partners and ultimately the voice of

beneficiaries in the program. It is recommended that at the annual reflections there is

discussion, dialogue and sharing of practices regarding how the rights‐based approach has

influenced program design, implementation and results.

c) The findings indicate that there are numerous successes on increasing access to, and the

effective use of improved water supplies, sanitation and hygiene services through the activities


of the partners. Efforts by partners need to continue to deepen the achievement and

participation of beneficiaries in obtaining these results. The participation of marginalized

groups including people with disability, men and women in rural contexts, children and youth

has been demonstrated in the identification of needs, the design of program activities at local

level, and now in beginning to manage WASH activities. There could be greater involvement of

these groups going forward in monitoring, management and increased ownership of the

program. Over the next eighteen months it will be important for partners to increase

engagement of beneficiaries in these aspects of the program cycle in order to build capacity for

community structures (committees, forums etc.) to carry the results of the program forward.

It is further recommended that the approach towards gender within the AACES WASH program

is addressed, as well as developing how individual partners (or collectively) can respond,

mainstream or model appropriate responses. It is suggested that a process is developed that

will allow partner engagement through dialogue and discussion. In addition, although at times

the focus is on increasing the participation of women in decision‐making, during the MTR

partners articulated that there also needs to be a focus on men. It is recommended that case

studies and sharing of lessons and findings explore the engagement, participation and changes

of both men and women.

d) It is necessary to continue to build capacity of partners in monitoring and evaluation skills,

and at the latest by the fifth year of the program for partners to have engaged beneficiaries in

monitoring, reflection and program learning events to build their knowledge, skills, practice and

ownership for WASH.

In South Africa16, there have been significant positive shifts in MEL capacity, with partners

demonstrating strengthened data management systems and fostering a culture of measuring

project results. Partners also showed positive changes in the use of evidence to inform decision

making and for accountability to stakeholders. The strengthening of MEL systems is also

evidenced by partners stipulating that they are now collecting more relevant data.

In Zambia, there is a need to increase partner capacity to collect reliable data, document

practice and provide technical support to partners to understand the MEL Framework and

translate this into MEL practice within their WASH activities.

16 Maud Mukova‐Moses. Oxfam Australia. Jan‐Feb 2014.


Across both countries there is a need to develop partner skills to analyse data and use the

results to inform program decision‐making, organisational development (for example strategic

plans, resource allocation, and efficiency) and to influence policy and practice.

e) It is recommended that there are discussions and engagement with partners individually and

collectively in relation to developing clear sustainability strategies and plans. These plans need

to include transfer strategies of knowledge, activities, results, infrastructure and financing to

encourage and enable results to be sustained. The emerging practice from the withdrawal of

MDIC and Fancy Stitch in South Africa provide valuable experience and insights into what needs

to happen at local level.

Conclusion

It is the researchers view, based on the findings of the MTR, now that the overall progress of the

Oxfam AACES WASH program is at the stage where partners working at community level have a) the

capacity (knowledge, skills, practice), b) emerging lessons from implementation over the past two

and half years, c) the increased awareness of enablers and blockages that influence activities

towards achieving the program goals, and d) an understanding of the need to engage with duty

bearers on policy or development issues, that the focus can shift to how individually and

collaboratively they can achieve the long‐term vision and goal of the AACES WASH program. The

partner’s activities are relevant to the beneficiaries needs and are effectively contributing to

significant and sustained changes in the lives of women, men, boys, girls and marginalised groups,

particularly in terms of increasing access to WASH for people with disabilities. The findings suggest

that the program is adding value and there is efficient use of resources, however further research is

required to determine the value for money of the program, and to determine greater efficiency of

resources. For example, a study examining infrastructural responses that maximise the return on the

investment, and that demonstrate how capacity building approaches can bring about change, could

add value to understanding efficiency and inform the program theory of change. There are

numerous emerging examples of value for money and innovation that should be documented to

share practice, influence policy and duty‐bearers accountability, and to leave an evidence‐base for

communities and partners to continue pursuing long‐term solutions to WASH beyond the AACES

WASH program.

The MTR assessment indicates that the program is working towards achieving the three AACES

WASH objectives, and significant progress has been achieved. Elements of the program that need

further strategic development and a focus on strengthening implementation over the next two years

include continuing to build partner and community capacity in monitoring and evaluation;


supporting communities to engage deeper in all aspects of the program cycle and AACES objectives;

articulating strategies to ensure sustainability of WASH programs and results at partner and

community level; reviewing and deepening the understanding of the theory of change (and ideally

evaluating); strengthening advocacy and influencing work of partners and communities to engage

effectively with relevant duty bearers; and continuing to share information, lessons and practice

from the AACES WASH program in South Africa, Zambia, Australia and with international NGOs,

donors and governments, and accountability or leadership structures to inform and influence policy

and practice. The recommendations aim to guide program activities to address these elements at

both the program and partner levels.


APPENDIX 1: Oxfam AACES WASH MTR SEMI‐STRUCTURED INTERVIEW QUESTIONNAIRE

DATE COUNTRY & VENUE NAME TITLE ORGANISATION Government/ Duty‐bearer / Stakeholder

Partner Program Manager/Coordinator Partner Director Oxfam staff Consultant

Introduction and purpose of interview, and how results will be used. Informed consent and confidentiality: Yes/No

QUESTIONS:

Please provide a short overview of the AACES WASH program, and what your organisation/department’s role is:

Please describe your role and responsibility in the program:

1. What is the purpose/ what are the objectives of the AACES WASH program? (Probe why they became involved in the program)

2. What do you expect the program to achieve over the 5 years? 3. What has been achieved (results) over the past two and half years?

What for you are the main outcomes (not outputs) of the activities to‐date? Probe for understanding and linking to program outcomes.

4. How do these results achieved (mid‐term) compare to the expected results at the half‐way point in the program? And compared to what is expected after 5 years? To what extent do you think the program is on track to achieve its objectives? Please explain

5. Who has benefited from the program – and how? 6. How were the WASH needs determined at the beginning of the program? How did this

influence the design and implementation of the program? 7. In what way has or is the program contributing to building and strengthened capacity of

stakeholders to manage and implement WASH programs on a sustainable basis (Objective 3)? To what extent and how has the program contributed to enhancing government/civil society response/s to WASH needs? Please give evidence if possible to support your answer

8. Zambia: Have there been any changes in the social/political context that have influenced the program? How did the program adapt or change (if at all)? Please explain

9. South Africa: How has the political/government context influenced the program? Please explain how the program has responded or not responded.

10. How is the program contributing towards reducing WASH inequalities in gender and marginalised groups (objective two)? Please explain your answer

11. How have women and men, boys and girls and marginalised groups been involved in the different stages of the program cycle? Refer to tool. Stages = SCOPING – INITIATING – DESIGN – IMPLEMENTING – CONSOLIDATING ‐ SUSTAINING


12. Do beneficiaries including women and men, girls and boys and marginalised groups including people with a disability believe the project has improved their wellbeing? What evidence is there to support your answer? (try and get case studies, examples & confirm with interviews)

13. To what extent has the program supported more equal access by women, men and people with disability to the benefits of the activity, and more broadly to resources, services and skills (e.g. by increased access directly, or by removing barriers to access)?

14. To what extent has the project developed the capacity of partner government and civil society to understand and promote gender equality?

15. Who are your partners in implementing the WASH program? What role do they play? How critical are they to the success/achievement of the objectives of the program?

16. What are your expectations of the following partnerships? (Use participation tool if necessary) o Government o Civil society o Community structures and leadership o Oxfam o Other

17. What is working in the partnership? What is not working? How can they be strengthened? Please explain your answers.

18. Has your organisation given thought to how the results achieved now and in the longer term (question three) will be sustained? If so what is your organisations sustainability strategy for the program? If not how sustainable do you think the results are?

19. How sustainable do you think the achieved results are? Please explain.

Efficiency and Value for Money questions:

20. What are the inputs / resources required for the program activities (money, time, people, equipment, knowledge etc.)?

21. Are inputs of good quality and timely? 22. How efficiently are they used? 23. What are the results because of these inputs? 24. What is the 'value' /return of these resources? 25. How do you make decisions on what inputs are used? 26. How do you ensure these achieve the intended results? 27. How do you ensure the opinions of those you are working with inform your decision‐making? 28. Has the program leveraged resources (financial and non‐financial) from government, other

duty bearers, or other donors? 29. How effectively and efficiently do you think the Oxfam/ community/ partner and stakeholder

resources have been used to contribute to program design, implementation and results?

Additional questions for Oxfam program staff:

30. How has Oxfam supported partners to maintain their relevance in the face of changing contexts?

31. How has Oxfam drawn lessons from partners in enhancing its own relevance? 32. What is the sustainability strategy for the program, and what has been the progress against

this? 33. What are the strategies for achieving and measuring value for money and how effective have

they been? 34. How does Oxfam define V4M given our particular approach and contexts? What are the best

examples to demonstrate Oxfam’s V4M, including equity? 35. Has the program influenced the broader work of Oxfam or partner NGOs? 36. How can the V4M of Oxfam’s work be improved?


MEL in Zambia:

Partners:

37. What training has your organisation had to MEL? 38. Have you been exposed to the MEL framework for the AACES WASH program? Please

describe when/how etc. and share any thoughts on it (probe for understanding of indicators) 39. What type of monitoring activities do your do as part of the WASH program? (probe for tools,

reports and activities) 40. How do you collect qualitative data and submit it to Oxfam in your reports? (process, data

quality check, accuracy) 41. A number of indicators focus more on the qualitative (story) aspect of the program, how are

you able to collect this information? What is working and what is not? If not collecting explore why and what could work.

42. How do you as an organisation use the information gathered? 43. How are the beneficiaries included in the MEL activities? How are findings or results shared

with them? 44. How is information or findings shared between the partners in Zambia? What are the learning

activities? Do you feel this is sufficient or are other mechanisms needed to strengthen the learning aspect of this program‐ is so what do you suggest?

45. What for you are the important principles/foundations for the MEL for this program?

Oxfam staff:

46. Please describe what has happened over the last 2.5 years in terms of MEL. 47. What are your thoughts on the MEL framework in terms of its relevancy, understandability

and application within the program? 48. What MEL capacity has been built with the partners over the last 2.5 years? What have been

the results of this capacity building? What else needs to be done? 49. Please give an overview of where in your experience the different partners are at in relation

to MEL within the WASH program implementation. 50. Do the reports and submissions from the partners provide sufficient evidence to report on the

indicators and meet the program reporting requirements? 51. Please describe how the partners, beneficiaries and stakeholders participate in the program

learning. What else, if anything, could be done and why? 52. How are or can the RSA and Zambia teams collaborate further over the MEL aspects? Should

they?

Please share any other thoughts on the program, or ask any questions you may have on this interview and MTR.

Close interview.



REFERENCES

i AACES WASH Terms of Reference, M Roper. Dec 2013. ii NEF. July 2012. Measuring Well‐being: A guide for practitioners. London iii Oxfam AACES Six Monthly Snapshot Reports: for 2011, 2012 and up until December 2013 iv Oxfam AACES Annual Reports Year 1 and Year 2. v Oxfam Australia. 4 February 2014. AACES Objective 3 Concept Note: ICT Pilot Project vi AACES WASH Terms of Reference, M Roper. Dec 2013. vii AACES WASH Terms of Reference, M Roper. Dec 2013. viii Oxfam Australia AACES WASH PRA Report. November – December 2011. LIMA ix Oxfam AACES WASH Program Design 2011‐ 2016 (revised 16 May 2011 – post peer review) x Oxfam AACES Six Monthly Snapshot reports for 2011, 2012 and up until Dec 2013 xi Oxfam AACES Six Monthly Snapshot report: 01/07/2013‐31/12/2013 xii Oxfam AACES Six Monthly Snapshot reports: 01/07/2013‐31/12/2013 and 01/01/2012‐30/06/2012 xiii NEF. July 2012. Measuring Well‐being: A guide for practitioners. London xiv Oxfam AACES WASH Program Design 2011‐ 2016 (revised 16 May 2011 – post peer review). Page 16. xv Oxfam. February 2012. Working Together: Oxfam’s partnership principles. xvi Galvin, M. (2013) Addressing Southern Africa’s Sanitation Challenges Through Community‐Led Sanitation (CLTS) 2013, Oxfam. xvii Oxfam AACES Six Monthly Snapshot Reports: for 2011, 2012 and up until December 2013 xviii Oxfam AACES WASH MEL Framework. 2012 xix Oxfam AACES Six Monthly Snapshot Reports: for 2011, 2012 and up until December 2013 xx Oxfam. 25 February 2014. Water, Sanitation and Hygiene in Underserved Communities, South Africa. Consultation with Partners and Allies: WASH Governance Advocacy Strategy. Outcome and Process Document. xxi Oxfam AACES Annual Narrative Report 2012‐2013. 10 September 2013. xxii Oxfam AACES Annual Reports Year 1 and Year 2. xxiii Oxfam Australia. 4 February 2014. AACES Objective 3 Concept Note: ICT Pilot Project xxiv Experiences and Challenges of Engaging Government. Draft: Feb 2014. AFAP; CADECOM, WaterAID, Oxfam.

aaces mid-term review (2014)

Documents