1

Concern Universal Impact Report An assessment of the long term impact of Concern Universal’s support to internally displaced people in Ibagué, Colombia.

December 2011

2

Contents

Acronyms 3

Executive Summary 4

Purpose of Study 5

Methodology 6

Findings 8

Graphics on results 11

Summary of Changes between 2005-11 12

Conclusions and recommendations 15

Annexes 17

3

Acronyms

CU-C Concern Universal-Colombia

FGD Focus Group Discussion

PSI Pastoral Social Ibagué, Archdiocese of Ibague (Concern Universal local partners)

UNHCR United Nations High Commission for Refugees

ICBF Instituto Colombiano de Bienestar Familiar Colombian Government of Family Welfare

JAC Junta de Acción Comunal Area Steering Committee

Acción Social Social Action: Colombian Government Department for Displaced and Poverty Reduction

WFP World Food Programme

4

Executive Summary

In August 2011 Concern Universal Colombia carried out an impact assessment to measure the impact, and identify the most significant changes for people who benefited from two projects working with Internally Displaced People (IDP’s) between 2005 and 2011. The projects carried out work in the communities of Los Álamos and Dos de Junio, in the city of Ibagué. Both communities are populated by IDP's who have been displaced by violence from rural areas of the Department of Tolima. The impact study interviewed people who have benefited from the project over the past six years, people who were resident in the area but who had not participated in the project, and other influential persons from the key stakeholders, including Pastoral Social of Ibagué (Concern Universal’s local partner), local government and the Concern Universal- Colombia team. There are three ‘most significant changes’ that the project brought about. We have defined these as changes that were defined as ‘most significant’ by more than 20% of beneficiaries. Firstly, the project was identified as improving social development and community organisation. 23% of respondents felt that the project had encouraged people, families, and local organizations to build a stronger community. Secondly, respondents stated there have been improvements in peace and security (20%). Respondents said there is now less violence in their communities.

Thirdly, infrastructure has improved. Over 19% of respondents believed that the physical transformation of their communities was the most significant impact of the project. Examples of the changes highlighted by respondents included; improvements in housing composition (tents to brick houses), better access to services (electricity/ water/ sewerage/ rubbish collection), and improvements in communal areas (pavements/ parks/ playgrounds/ sports field; building of a community hall). There were two further areas that were identified as significant changes (10-20% of respondents saw them as the most important changes). Owning a house was identified by 16% of respondents as a significant change. Many people targeted by the project, arrived in Ibagué with few or no possessions. Through support from Concern Universal they were able to lobby the municipal authorities for better services. They managed to get small and basic houses built and this constituted an enormous improvement from their previous conditions. 10% of respondents felt better able to Demand Rights due to the training and capacity building offered through the project. Beneficiaries now feel capable of actively negotiating with, and organising alongside, Government staff and other key actors. This ensures that their real needs are addressed as well as allowing beneficiaries the opportunity to actively engage in debates about what constitutes a just and sustainable society. The Impact Assessment shows that the work of Concern Universal Colombia and Social Pastoral

Two other areas were identified where the project has brought about relevant changes (up to 10% of respondents recorded these as a major change). Beneficiaries of the project felt that more attention should be given to these areas in future projects. Quality of life was mentioned by 7% of respondents as a major change brought about by the project, including an increase in their dignity and economic stability (3%).

has helped address the needs of IDPs and equipped them with the skills required to assert their rights and ensure that their dignity will not be lost.

5

Illustration 1: Dos de Junio 2005. Beginnings of street, houses and sewerage

Purpose of Study This study was conducted by CU-C and PSI to assess the impact of projects implemented between 2005 and 2011. There are a number of reasons why this study was conducted at the end of 2011, including:

(a) There are still a large number of displaced people in Colombia and it is important we learn more about why they are living in poverty and why they are discriminated against.

(b) CU and PSI have been working in these two communities for over 10 years and it is important to go back and assess the changes since the founding of the settlements.

(c) To demonstrate the effectiveness of a participatory development process: the project was planned and agreed with the beneficiaries, focusing on demanding their rights from local government and achieving results without causing violence.

(d) To understand why it is important for CU-C and PSI to continue supporting displaced people in Colombia.

This impact assessment is also timely because supporting IDPs has become an important policy issue and the Colombian government is currently trying to compensate victims of the continual violence of the last sixty years. This makes it a good moment for CU-C and PSI to measure and reflect on the impact of their projects that have sought to build peaceful co-existence,

development, rights awareness and support for internally displaced people. The newly implemented strategy of CU-C (2011-2016) makes this an appropriate time to take stock, as does the recent conclusion of two big projects funded by Cordaid, and the commencement a new project funded by AGEH in Germany, which will target the two assessed communities. Between 2001 and 2010 153,036 people were internally displaced in Tolima Department and the majority of these people came to Ibagué. Those who stayed in the city settled in 2 de Junio and Los Álamos. These two communities were chosen for the project because they have the largest populations of displaced persons in Ibagué, housing only displaced people in purpose built housing rather than existing housing. Both areas are poor and have only recently begun to have any formal community organising structures.

Illustration 2: 2 Dos Junio 2011. Street paved, improved

houses, with sewerage.

6

Methodology Participatory methodologies were selected to promote bottom-up accountability and to facilitate learning, of both past projects and CU’s own monitoring systems. The impact study reviewed progress since the implementation of two specific projects: ‘Project for people in state of displacement’ from 2005 to 2008 and ‘Reconciliation and local development’ from 2008 to 2011.

Illustration 3: Displaced youth formed an association in Dos de Junio.

The sample: 309 interviews were carried out for the impact assessment; interviewees included 113 women, 70 young people, 114 elderly people and 12 non participants. Two group workshops were carried out with a total of 70 people from the two communities, divided between the two workshops. Conversations were held with other organizations and local government. (See breakdown in Annex 2) The impact assessment focused on a number of ‘areas of enquiry’. Areas of enquiry allowed the team to look at actual change within the community as well as people’s perceptions of change and answer some specific questions: what has changed; what was the effect of the change; and was the change significant?

Illustration 4: Beneficiaries group workshop. Los Álamos.

Using the areas of enquiry as the basis of our assessment the study used a number of quantitative and qualitative tools to collect data and triangulate results. The research methods used were:

a) Direct observation: walking through the communities, going into houses for the interviews to observe their social, cultural and economic conditions.

b) Semi-structured interviews (direct and indirect beneficiaries): Using a number of open-ended questions around the areas of enquiry interviews were conducted with a representative sample of:

Beneficiaries

Local Partners

Local and National Government

International Organisation WFP

c) Stories of change: through FGD’s and semi-structured interviews the team identified individuals able to tell more detailed stories of change. Participants had the opportunity to use a camera / video camera to record the changes they described, or simply to write them down. The stories of change collected evidence of change and its effect(s) on the community. They also showed an individual’s personal journey of empowerment and how they took part in the community transformation.

d) Questionnaires (household survey): A short questionnaire was designed

7

with closed questions on the main areas of enquiry. The household survey design was based on a design previously used for the project.

e) Desk based research (baseline/ reports/ end of project findings/ evaluation): The team reviewed the project documents to draw out information on impact. An assessment of project documents was especially useful in developing impact indicators and to track what has changed since project inception. The initial conditions as described in the project proposal were used as a baseline.

The team made every effort to avoid bias by emphasising that it was necessary to hear about all changes, positive or negative, even when these changes were not associated with a particular organisation. We explained clearly, before each data collection session, that the research was going to help Concern Universal develop better projects so honesty was the best policy. Whether answers are positive or negative would not influence the chances of more project funding or of future work with the displaced communities.

The Impact Assessment framework also took into account other projects/ programmes that have been carried out in our project area that had the same objectives. The study attempted to assess whether CU’s interventions (or other interventions) have resulted in stronger community capacity to address the project issues.

Illustration 5: Displaced beneficiaries in Most Significant Changes workshop. Los Álamos.

8

Findings The findings of this assessment are divided into three categories: most significant, significant and relevant changes.

Most significant changes They were three areas of enquiry which were highlighted by over 20% of beneficiaries as having a significant, direct impact on their lives. We have called these the ‘most significant changes’ of the project.

1. Organizational capacity 23% of respondents stated that CU-C’s work has encouraged people, families, and local organizations to build and improve community cohesion. This change has given people a sense of ownership over decision-making in their community and has helped them to demand government accountability. It is now the norm for communities to feel confident to negotiate with local government over issues affecting their basic rights1. At the same time the number of leaders in the two communities has increased. These leaders have been able to increase self-esteem and awareness within their communities

and encourage others to organise.

As a result of the project women have learnt that income generating opportunities are available for them outside the home. They now believe that they have a choice how to use their time and skills. They have become authors and actors in their own development and that of their community

1 New element in CU projects, from article in Colombian

Constitution: Colombia is a social state of rights .

Illustration 6: Lady making shoes. Dos de Junio

2. Peace and security Over 20% of

respondents felt that communities have become less violent and that security has improved. Many people believed that each community had made progress towards reducing conflict.

3. Infrastructure 20% of people felt

that improvements in the physical infrastructure of their areas had the most significant impact on their lives. When the project began people were living in tents with few possessions. Six years later their communities are developed to a similar standard to other areas of the city, including new street paving, parks, a sports field, playgrounds, and community halls for meetings and events. Schools have been improved and enlarged. Basic services have also been improved with most respondents saving they now have greater access to electricity, water and sewerage systems and rubbish collection.

Illustration 7: Park in Los Álamos

9

Significant changes Two further changes were highlighted by up to 20% of beneficiaries as the most significant for their lives. We have called these the ‘significant changes’.

1. Owning a house 16% of the respondents identified owning a house as a significant change. After being displaced and losing everything in the violence, IDPs from Tolima Department moved to Ibagué, where they lived under plastic and cardboard huts in the middle of the road. The government moved them into communal tents with no electricity, water or sewerage. They stayed in these tents for over two years. With support from CU-C they began to organize themselves into community groups. The groups were able to demand the government to provide adequate housing. Over the last few years the municipal authorities have built small and very basic houses for IDPs. This is an enormous improvement from their living conditions in the tents.

2. Demanding Rights was mentioned

by 10% of the sample as a significant change. A major part of Social Pastoral and CU-C’s projects had been the provision of training on the negotiation of rights, and this shows how important the respondents felt this training was. The training has ensured the continued strength of the associations formed during the two projects (Women’s group, Youth group, Elderly people’s group and the official steering committee for each area). Evidence of the impact of this training can be seen in the increased knowledge and use of legal mechanisms to protect rights, such as “Tutela”, right of petition, action of compliance, and popular action. These processes have been used by the community in invoking their rights to housing, health care, formal education, basic sanitation and water and to have their status officially

recognized as displaced people with particular rights. Further evidence of the community’s ability to demand their rights can be seen in their participation in public policy formulation for displaced people, women, young people, children and adolescents and for elderly people. The beneficiaries have been actively involved and are now capable of negotiating and organizing with government staff and other key actors to come to agreements about real needs. During and after the projects they have also taken part in debates for a just and sustainable society.

Relevant Changes Two changes were highlighted by less than 10% of respondents. These are significant changes for only a small number of respondents. At the same time a number of beneficiaries felt that these areas were often overlooked during the project and greater attention should be given to them in future projects:

1. Quality of life: 7% of respondents stated that their dignity had increased as a result of the projects implemented by Social Pastoral and CU-N. However, a number of people felt that too great a focus was given to economic empowerment and there could have been more attention paid to the social issues. A majority of beneficiaries felt more attention should be given to psychological support, such as increasing the confidence and dignity of women (“we learnt to talk first and then how to talk to government staff without feeling inferior”) 2 . However, respondents also acknowledged the importance of addressing their basic material needs (house and food), social services (education), and economic welfare (work or income), and ecological and sanitation needs (sewerage, cleaning up communal ground and parks, and rubbish collection) stating that these have

2 Words of a beneficiary from Dos de Junio, first workshop.

10

been satisfied to a degree through the work of CU-C and PSI.

2. Economic stability: A minority of

respondents felt that the projects had given them greater economic independence. A number of respondents mentioned that the rotating fund supported small economic initiatives and that this was a factor in improving the families’ economic wellbeing. Participants also mentioned that they had found the training in economic administration and management of small businesses a great help. The majority of respondents stated that they had had some success in income generating activities. This is a significant achievement taking into account the high levels of unemployment that have existed in Ibagué in recent years. These are well above the national average for Colombia. (2005: Ibagué 19.3%, national 11.7%; 2008: Ibagué 19.1%, national 11.9%; 2011: Ibagué 15.5% and national 9.2%).

3.

Illustration 8: Rotating Fund project

The Impact Assessment shows that the work of Concern Universal and Social Pastoral has helped address the needs of the target communities and equipt them with the skills required to assert their rights and ensure that their dignity will not be lost.

Illustration 9: Income generation

11

Graphic representation of Most Significant Change results

The following charts show the results from the group workshops and interview responses given by both beneficiaries and non-participants in both areas.

Graph 1: Most Significant Changes by % of response. When asked if these seven key changes made a ‘significant difference’ to their lives 81% of beneficiaries agreed that they did. Each participant had to name one key change amongst the seven changes. Organisational capacity was the most widely chosen area with 23% of respondents, whilst economic change was chosen by the fewest number of participants (3%)

Graph 1 the categorisation of changes according to number

Graph 2 Important changes mentioned by respondents in Dos de Junio

Graph 3 Important changes mentioned by respondents in Los Alamos

Graph 4: When asked to name who brought about the impact of each change respondents very clearly recognised that in most cases, the change was due to the work of Concern Universal, followed by the fulfilment of the duties of the municipal authorities, then to the local partner, Social Pastoral. Put together CU-C and PSI were

23 20 19

16

10 7

3

Org

anis

atio

n

Pe

ace

an

d S

ecuri

ty

infr

astr

uctu

re

Ow

nin

g

hou

se

Dem

an

din

g R

igh

ts

Qu

alit

y o

f liv

e

Econ

om

ic

Most significative Significative Relevant

What positive changes have occurred in your life

(last 5 years)

32

21 18

7 5

Pavedstreets

Emergencyaid

/improvedhouses

Building ofcommunal

hall

Reductionof crime

andviolence

Rotatingfund /smallbusinesses

Which are the most important changes in your community (Dos

de Junio)?

10 10 9 9

7

Which are the most importants changes in your community (Los

Alamos)?

12

mentioned most frequently for being responsible for positive impact, a good indicator of the success of the jointly implemented project. This response was echoed by the other people asked, including the local and departmental authorities.

Graphic 4: Catalyst for Change

Summary of Changes between 2005 and 2011: Update on situation in 2005 In 2005 a significant number of people living in rural areas in Tolima were forced to leave their land, houses, belongings and communities because of armed conflict. They arrived in a town which they did not know, and lived in very difficult conditions without access to basic services. They did not know their rights as displaced people and as they all came separately from different places and circumstances, did not realize the importance of organizing themselves as groups to demand their individual and collective rights. Their physical, emotional and social health was damaged because of the trauma of the displacement. In 2005 the families were not yet established in the two areas, they were all living in one camp, in difficult conditions, in shelters made

of cardboard, black plastic bags and bamboo, without water, sewerage or electricity. There were no hygiene facilities causing many health problems but they had no access to health services, education or any other basic service. With no income or subsidies, people were forced to beg for food whilst living in deplorable conditions and this had a very negative impact on their mental, physical and social health and their dignity as people.

How the situation had changed by 2008 During the first Concern Universal Colombia and Pastoral Social project beneficiaries received psychological support. This helped them to mourn the loss of family, friends and neighbours. It also helped them come to terms with the loss of their livelihoods and housing. It helped them to accept, forgive and begin new lives, thus adapting to their new home and community. The project supported the communities through an understanding of their rights, food packages, the building of small houses (with electricity, water and sewerage), access to preventative and curative health services and formal education for children and young people. They also achieved official recognition of their displaced status from relevant department of municipal authorities, a vital step for all other legal and judicial processes. In the wider community they formed functioning associations with the different population groups from the two areas. Six associations exist and have been legally registered. Their aim is to continue the development of the communities and to try to raise funds for future activities. In 2008 the two communities were living with a degree of peace, violence had decreased considerably within the communities and the cases of ill treatment of children had been reduced. Those breaking the law were generally reported to the authorities. Alternative solutions were being used for conflict resolution instead of verbal or physical violence. The greatest interest from beneficiaries was in forming associations of

0

5

10

15

20

25

30

35

40

Catalyst for changes

CU-C

Alcaldía

PSI

JAC

ACNUR-UNHCR

ICBF

13

women, young people and older people enabling these groups to work together more effectively to improve their houses, roads, parks, playgrounds, schools, water supply, sewerage system and other facilities. They were learning and using their knowledge to use legal mechanisms to reclaim rights. As one respondent stated ‘since 2005, with the help received from the project with CU, we have managed to change, to free ourselves from the displaced mentality. Now we feel that we are people with dignity, with hopes and worthy of rights3”

Illustration 10: Workshop and community lunch Dos de Junio

The overall impact in 2011 A majority of the internally displaced people now living in Dos de Junio and Los Alamos have received emergency aid, and also recognise their rights and understand the importance of uniting and organizing themselves. The associations established through the project have internalized values and customs that demonstrate the ideals of social and gender equality, peace and democracy. These associations are now registered and have established relationships with neighboring communities. The organizations have the capacity to a) work for viable solutions for the development needs of the communities b) assist in the organization of associations in other communities c) establish alliances with other communities, government personnel and other bodies who can help to increase their ability to lobby decision making bodies d)

3 Words of a woman, part of the steering committee of

the area of Los Álamos. Systematization 2008.

raise funds and administer themselves transparently. These results show that it is possible for such organizational practices to have a significant impact in reclaiming economic, social, cultural and environmental rights. In 2011 all the displaced families have houses, with water, electricity, sewerage and even gas. The areas now have a regular rubbish collection. All children and young people are in school. All families have a health service. The rotating fund money has been lent to some families to begin or improve a small business. The associations have been proactive in formulating public policies for the town and the geographical department. The displaced communities are integrated into the town and participate in many ways. The area of Los Álamos is now being considered by the United Nations High Commission for refugees (UNHCR) as a positive example of development and community building, with full rights for displaced people.

A 2011 external evaluation of the project identified CU-C and PSI’s impact on the beneficiaries lives; In Los Álamos in 2008 there were 7 community leaders with some skills, now there are 13, of whom 8 are adult women, 2 young women, 1 is a young man and one an adult man. In Dos de Junio in 2008 there were 6 leaders with some skills, now there are 10, of whom 5 are adult women, 1 is a young women, 2 are young men and 1 is an adult man. CU-C and PSI have been great allies and guarantors of the beneficiaries´ associations in their relations with other organisations and to government bodies. They help them to make contact with other groups, teach them how to use advocacy to claim their rights and services, promote agreements and alliances with them so that the other organizations or government bodies can work in a coordinated way in the two communities, with active participation of the leaders and the associations. At the beginning of the project in 2008 only two small businesses existed in Dos de

14

Junio. Now there are 34 in both areas, 22 in Los Álamos and 11 in Dos de Junio, financed with the funds lent from the rotating fund. Of these 34, 10 have been functioning for two years, 7 in Los Álamos and 3 in Dos de Junio. 23 small businesses have been working for a year, 15 in Los Álamos and 8 in Dos de Junio. Illustration 11: Association of elderly people making brooms and mops Los Alamos

Of the 34, 29 have survived, made a profit and gained in value, such as incorporating other small businesses, selling other products, or using new designs. This has worked particularly well in Dos de Junio, where all the small businesses are achieving this. In Los Álamos, which is poorer and more remote, 18 of 22 are achieving this. Of the 34, 21 have grown, i.e. their sales have improved and they have generated further employment : 9 of the 11 in Dos de Junio and 12 of the 22 in Los Álamos. All the families have built basic houses all of which have water, electricity and sewerage. 93% have gas piped to the house, and 43% of houses have improved and expanded. 80% of women, children, young people and older people in Los Álamos and 90% in Dos de Junio have access to the basic health services offered by the municipal authorities and public health services, including vaccinations, the healthy school programme, and visual, dental, mental and environmental health programmes. 90% of women, children and young people and 70% of older people in Los Álamos, as well as 95% of women, children and young people, 90% of older people in Dos de Junio have access to food packages from the World Food Programme

as part of the project, and this has greatly improved nutrition. The beneficiaries indicated clearly that CU-C had been the most important organization in achieving this positive impact and PSI is another of the most important.

15

Conclusions and recommendations CU-C have always had the clear intention to transform the problem which made the projects necessary into a more positive and sustainable situation. The replies and comments in the impact assessment made it clear that the two projects carried out by CU-C and PSI did fulfil the expectations of the beneficiaries and the agreement with the donor, and also made lasting changes in the lives of the beneficiaries, on personal, family and community levels. The actions of CU-C and PSI strengthened community associations in the two communities, gave them the knowledge and the confidence to talk to local government staff, to understand and use the political constitution, and to use the appropriate legal mechanisms and to reclaim their rights. It is, however, important to continue designing and delivering multi-year community-led food security projects in partnership with existing local structures. The custom of negotiating times, dates and durations of activities with the beneficiaries to ensure there are no conflicts with their other commitments has worked well and it would be advisable to continue using this method in future work. The participation in local government bodies is an excellent example of organized civil society engaging with and ensuring government fulfils public policy: this should be valued and continued. It would be useful for future projects to give more attention to short and medium term aims. These two projects had only final situations planned, in order to perceive changes during the project duration it would be more effective to encourage the beneficiaries to mark milestones by acknowledging and outlining short to medium term aims. The reports of activities from all six years working with these two communities show a high level of compliance and fulfilment of the project plan. It is evident that an excellent

participative planning process has been carried out and carefully monitored using the annual operative plans. Working together with other organisations and government bodies has proved very constructive for the project beneficiaries. The local government staff themselves praised this method and expressed admiration for the experience, strategies and methods of CU-C and PSI in working with displaced beneficiaries. CU-C and PSI’s experience in working with communities, their accountability and transparency, and their ways of working with beneficiaries all contributed to achieving the impact of the two projects. The direct intervention, the field work and the daily contact with beneficiaries were all important in getting to know beneficiaries and their needs, making possible in-depth understanding, flexible attention and a harmonious process. The planning, monitoring, evaluation and systematisation process could be improved to be even more precise and so more effective from the very beginning of project activities.

Illustration12: Community activity Los Alamos

The training and capacity building of how to claim rights through legal and judicial mechanisms according to the Colombian constitution should be continued with the two displaced communities. This would give them even more possibility of understanding local government staff, of demanding more

16

of their rights in the future and so improving even more their situation and that of other displaced people. For future work more insistence on creating networks and alliances could ensure greater impact and sustainability.

The principles used such as equality, horizontal relations, working for a social State of Rights and helping beneficiaries to become really involved in democratic participation did bring about real impact. In the work carried out in the two communities, inclusion of all was ensured, with people from the country, some indigenous, people with special needs, women, children, young people and the elderly all participating. CU-C´s gender policy and ideas definitely helped the community be aware of, and work correctly with gender issues. The educational approach used in the two projects should continue to be used in the future. This approach was explained in the 2011 External Evaluation: “Concern Universal-Colombia and Pastoral Social´s way of working, based on creating alliances and agreements with other organizations and government bodies, is an excellent way to achieve greater efficiency and effectiveness but also to have greater impact in development projects, plus being a good way to replicate with the other organizations involved the approaches, good practice and methods of working and the successful processes with beneficiaries. “ A recommendation also made in the External Evaluation 2011 about the

economic aspects of the project should be adopted in the future: in this aspect it is very important that the beneficiaries in the two areas continue to develop the internal market for the products produced in the small businesses in the two areas which were supported by the rotating fund of the project. As other buyers do not normally come into these areas, it is vital that the inhabitants do use and consume the products. It is also important to promote a sort of “social stamp” for the products to identify them and make them more attractive, thereby adding social and political incentives for buying and involve any other organization coming into the areas in purchasing and supporting these income

generating initiatives. The Project approach developed by CU-C and PSI is appropriate for achieving positive impacts because it combines aspects of personal and community development, social and economic need, political and judicial demands for rights, and community development, all of which are done through training and capacity building, promoting and supporting the formation of alliances and guiding beneficiaries and their community associations. It would be wise to continue to be involved with local governance and local administrative processes to help beneficiaries assert their needs more effectively and sustainably. The beneficiaries´ needs should be considered in the Territorial Ordinance Plans and in the Collectives. Inter Commune Plans, in order to guarantee long term Government support.

17

Annex 1: Area covered by assessment

Annex 2: Number of participants in the study

Number of participants per community

2 de Junio Los Álamos

Women Men Women Men

Women 59 -

54

-

Young people 21

13

19

17

Elderly people 20

8

32

54

Non participants 4

2

4

2

TOTAL 104 23

109

73

ITEM %

Total Population 2005 1.561

Population two communities 2011

1.404

Total direct beneficiaries project

927 66,03

Non participants 477 33,97

Total Interviews 309 19,80

Women 113 7,24

Young people 70 4,48

Elderly people 114 7,30

Non participants 12 0,77

18

Annex 3: Impact according to the number of mentions by beneficiaries for community

Impact for the beneficiaries in the two communities

NUMBER OF MENTIONS

CHANGE

195 Own house

174 Infrastructure (parks, playgrounds, schools, community buildings, water, electricity, gas, sewerage and rubbish collection )

156 Social organization within communities (associations of women, young people, older people, steering committee of each area, community participation in events, campaigns, marches, demonstrations. )

127 Peaceful co existence as a community, more united, more peaceful in general

90 Reduction in violence, increased security, less robbery, less crimes, less violent acts)

89 Improvement in quality of life, of self confidence in the women, in health, useful use of free time, leadership skills, improved parenting, training and capacity building.

Training in rights and correct demanding of rights from the relevant Government staff by the beneficiaries.

60 Education and cultural events in the community

54 Organization and paving of roads

37 Productive projects carried out by families

35 Young people and children taking part in community events and activities.

26 Promoting development by advocacy with local authorities

22 Commitment and democracy in the communities.

15 Social development in the communities

10 Good use of free time by children and Young people

10 Government fulfilling its duty with people displaced by violence

19

Annex 4: Key Informant Interviews Women´s associations : President, Secretaries and members. Elderly People´s Associations : President and members Young people´s Associations : President and members Area steering committees : President and secretary. Social Pastoral Social : Director and staff. Colombia Government Welfare department ICBF : staff members Government department Acción Social, Tolima : director

Annex 5: Research Team Members CU-C team

Siobhan McGee Country director

Jaime Bernal Project Coordinator

Alejandra Moreno Financial assistant

Liliana Rivera Administrative assistant

Dina Bernal González Educator

Nicole Landmann International worker –AGEH Germany

Hannah Richardy International Volunteer Lay People´s Council Aachen, Germany.

Aoife Itziar Bernal McGee Volunteer, interviews and writing up

Rubén Dario Gómez Educator

Luis Alfredo Páez Educator

Gabriel Bernal McGee Volunteer, writing up, photos, videos.

20

Annex 6: Assessment Expenditure In order to enable judgements to be made about the cost-benefit this assessment exercise, a summary of the costs involved is provided below.

Annex 7 Limitations/ Lessons

a) One limitation was the need to carry out an impact assessment without having had the time to prepare the work team for it, this made it more difficult for them to understand and dedicate themselves to it. Another difficulty was that the assessment coincided exactly with the ending of two projects, which made it less possible to go out frequently to these areas and dedicate significant amounts of time to this assessment.

b) We learnt that is would be better to do this kind of assessment when there was time beforehand to give a workshop to the team on impact assessment and most significant change and then to have a team dedicated to the impact assessment, rather than a team all working full time on other projects and just giving a few hours once a week when they could get away.

Working days Person Cost per day Days Total

Country Director £65.20 £20.00 £1,304.00

Project Coordinator £35.69 £30.00 £1,070.64

Financial Assistant £14.25 £5.00 £71.26

Administrative Assistant £10.69 £3.00 £32.07

Educators £15.32 £15.00 £229.80

AGEH worker £35.63 £3.00 £106.89

German Volunteer £6.77 £18.00 £121.85

SUBTOTAL £2,936.50

Quantity Valor Total

Transport Project Coordinator £6.00 £2.14 £12.84

Educators and others £20.00 £2.14 £42.80

Country director £4.00 £2.14 £8.56

Group workshops £4.00 £10.69 £42.76

SUBTOTAL £106.96

Refreshments Group workshops £70.00 £0.53 £37.10

team £110.00 £0.53 £58.30

SUBTOTAL £95.40

Materials Photocopies group workshop £100.00 £0.05 £5.00

Photocopies interviews £800.00 £0.05 £40.00

Materials report (ink,paper) £3.00 £30.28 £90.84

SUBTOTAL £135.84

TOTAL £3,274.70

21

Annex 8: Timetable of study

DATE ITEM

19 July Preliminary information to Advocacy and

Learning Manager

Jaime & Siobhan

12 August Send TOR to Advocacy and Learning

Manager

Jaime & Siobhan

26 September-10

October

Draw up template of Impact Assessment report Design of primary research instruments: significant change discussion guidelines; household survey; key informant interview schedule.

Siobhan & Jaime

Monday 10 October

2011 to Friday 13th

January 2012.

Field work : questionnaires with samples in

both areas

Diana Bernal , Nicole

Landmann, Hannah

Richardy, Alejandra Moreno,

Rubén Gomez, Luis Paez ,

Jaime Bernal & Siobhan

McGee.

1-31 October Semi structured interviews with local NGOs,

municipal and departmental authorities, local

partner etc

Jaime & Siobhan

31 Oct – 4 Nov Document review Jaime & Siobhan

18 January 2012 Finish report Siobhan & Jaime

20th January 2012

Send final document & summary in English to

Advocacy and Learning Manager.

Share with stakeholders and use with donors.

Siobhan

Jaime & Siobhan.

Translation of report to English in UK by

volunteer

Volunteer with good Spanish

in UK office

22

Annex 9: Terms of Reference

1. Study title: Assessing the long term impact of Concern Universal’s support to displaced people in Ibagué, Colombia, through an ex-post evaluation of the displacement, reconciliation and local development projects.

2. Rationale

During 2011-12 Concern Universal will carry out four separate impact assessments in Ghana, Colombia, Kenya, and The Gambia. The Colombia impact assessment is being carried out to assist CU Colombia learn from its work with displaced people. The assessment will demonstrate the effectiveness of our approach and further improve current programming. The Impact Assessment will be conducted to answer the question ‘what long-term difference did we make in the project areas?’ The final external evaluation demonstrated that the activities were implemented on time and to budget, but so what? In the long term, what difference have these activities made to people’s lives?

The impact study will attempt to answer the following key questions:

What are the sustained changes in people’s lives brought about by CU / Social Pastoral projects? There will be an emphasis on finding out the changes related to local development, human rights and peace building.

What are the ‘significant’ changes (positive/ negative/ intended/ unintended) brought about by the CU/ Social Pastoral projects?

3. Focus

Local development, human rights and peace building with displaced people has been a priority for the CU Colombia programme over the last six years and a significant part of funding has been spent in this area of work. Since Concern Universal Colombia is in the process of renewing its country strategy and has had democracy and human rights, poverty reduction and health as key programme focal areas, this is a good time to reflect on, and capture learning from, past experience. The new strategy is likely to include governance, human rights and peace; rural development and environment. Concern Universal Colombia accepts that these are interrelated. The displaced population, themes and the town of Ibagué have been chosen because Concern Universal and partner Social Pastoral have been working there for the past six years and the intervention ended in March 2011, making it perfect for assessing long term impact of past interventions. The intervention with funding from Cordaid consisted of two projects: Project for people in state of displacement from 2005 to 2008 and Reconciliation and Local development from 2008 to 2011. Reports from other projects of Concern Universal –Colombia will be used in the desk based research, although they are in other villages outside Ibagué.

4. Selection of sample villages and households Over the last six years Concern Universal has worked with 341 households and 1561people in two displaced communities in Ibague. The beneficiaries for the exercise will be divided according to the percentage of the 309 in each displaced area and then into population groups within the projects, just as the associations are formed : young people, women and elderly people. The questionnaires will be done with a sample of non participants as well as project participants. It will use 5% of

23

the 309 project participant as the non project group, which is 15 people, divided between the two areas. These same associations of young people, women and older people will participate in the focus group discussions.

5. Sampling frame The impact assessment has a sampling size that will give us a confidence level of 95% and a margin of error of 5%. We have decided on the final sampling size by using the Raosoft sample size calculator4.

Item Displaced projects Number of communities 2

Number of households 341

Number of individuals 1561

Proposed sample people 309

Proposed sample households 309

6. Proposed methodology

The methodology proposed is simple and user friendly. It will build on existing monitoring and evaluation systems. The methodologies have been chosen to be participatory and help in the promotion of further downward accountability, as well as support programmatic learning. The research team will make every effort to avoid bias by emphasising that we want to hear about all changes, positive or negative, and whether or not associated with a particular organisation. We will clearly explain, before each data collection session that this research is going to help Concern Universal develop better projects so honesty is the best policy. Whether answers are positive or negative will not influence the chances of more project funding or of future work with the displaced communities. The study will follow up specifically on the two projects of the intervention: Project for people in state of displacement from 2005 to 2008 and Reconciliation and Local development from 2008 to 2011. The final situations of the two projects were the following: The associations in the first phase of the intervention in the areas Álamos and 2 de Junio in Ibagué will have strengthened their organizations and will have social and economic stability , which will favour their human, social, political and citizen development . The women and girls , men and boys , protagonists of the project, trained and with capacity, will have skills for income generation, can enjoy their rights as citizens and recognize themselves as subjects of rights and development in the commune and the town. There are two baselines studies, one for each project, not in great detail but they will be used for this exercise. The impact assessment will focus on a number of ‘areas of enquiry’. Areas of enquiry will allow us to look at change within the community and people’s perceptions of change. When looking in to the areas of enquiry we will attempt to answer some specific questions: what has changed; what was the effect of the change; and was the change significant? All these referring to the themes of the project. Using the areas of enquiry as the basis of our assessment the study will use a number of quantitative and qualitative tools to collect data and triangulate results. The most likely research methods to be used are:

4 http://raosoft.com/samplesize.html

24

f) Direct observation: walking through the communities, going into houses for the

interviews to observe their social, cultural and economic conditions. a) Semi-structured interviews (direct and indirect beneficiaries): Using a number of

open-ended questions around the areas of enquiry interviews will be conducted with a representative sample of:

g) Beneficiaries (people in representative sample of communities) h) Partner (Social Pastoral ) i) Local government (Acción Social, ICBF, municipal authorities and departmental

authorities) j) National government (Acción Social, World Food Programme )

b) Stories of change: through the FGD and semi-structured interviews the research

team will identify individuals that are able to tell more detailed stories of change about the project. Participants will be offered the opportunity to use a camera / video camera to record the changes they describe, or simply to write them down (with accompanying photo). A story of change will…

a community has transformed itself.

possible.

c) Questionnaires (household survey): A short questionnaire will be designed with closed (Yes/No) questions on the main areas of enquiry. The household survey design will be based on a design previously used for the project. These questionnaires are being designed using the project indicators.

d) Desk based research (baseline/ reports/ end of project findings/ evaluation): The programme team will review the project documents to draw out information on impact. An assessment of project documents will be especially useful in developing impact indicators and to track what has changed since project inception. The initial situation as defined in the project proposal will be used.

The IA framework will also take in to account other projects/ programmes that have been carried out in our project area that had the same objectives (may also be private sector/ community initiatives). The study will attempt to assess whether CU’s interventions (or other interventions) have resulted in stronger community capacity to address the project issues.

7. Expected Outputs

Colombia Impact Assessment (max 15 pages for publication, plus appendices), including summary on a page and one page introduction

Framework for Impact Assessments that could be used by Colombia team

Training on impact assessments

Videos/ photos – to use for report and website. These would be best if they were in the form of case studies (with photos) and video clips (no more than 1 minute each that we could use on the website

25

8. Timeframe (ToR)

DATE ITEM

19 July Preliminary information to Advocacy and Learning Manager

12 August Send TOR to Advocacy and Learning Manager

26September-

31 October

Document review

Draw up template of Impact Assessment report

Design of primary research instruments: significant change discussion guidelines; household survey; key informant interview schedule.

Fieldwork

7-11

November

Final write up

17 November Translation of summary to English

21 November Send final document & English Summary to Advocacy and Learning

Manager.

Share with stakeholders and use with donors.