improving impact - do accountability mechanisms deliver results - short report, 2013

7/28/2019 Improving Impact - Do Accountability Mechanisms Deliver Results - Short Report, 2013

1/16

IMPROVING IMPACT:DO ACCOUNTABILITY MECHANISMSDELIVER RESULTS?

A joint Christian Aid, Save the Children,Humanitarian Accountability Partnership report,June 2013

Andy Featherstone,research consultant


2/16

2 Improving impact Chapter heading

Foreword 3

Introduction 4

The research case studies 6

Accountability mechanisms and project quality 8

The findings 9

Key conclusions and recommendations 14

Endnotes 15

Front cover photoWomen are now actively determiningdevelopment plans and projects in MakueniDistrict, Kenya

Christian Aid/Amanda Farrant

CONTENTS


3/16

3impvng mpact Foreword

ForeWord

Why does accountability matter? Why doso many humanitarian and developmentorganisations put such eort into ensuringnot only that the wishes o the people they

aim to help are taken into account, butalso that benefciaries are able to hold aidorganisations to account?

Most o us would respond thataccountability matters because we aremorally obliged to use the resources heldin trust or other people according to thewishes and best interests o those people.We are also morally obliged to show that

we have done so.

We could stop there, but manyother compelling moral and practicalarguments have been made or havingaccountability to aected communities. Itis suggested that accountability improvesthe eectiveness o humanitarian anddevelopment programmes by ensuringthat goods and services are relevant topeoples needs and that this helps ensuresustainability; that where accountabilitysystems support community participationin programmes, they can contributeto political and social empowerment;that accountability can even makeprogrammes more efcient, by allowingpeople to identiy and correct waste andmismanagement.

All o these arguments make good sense.They sound as i they ought to be true.Perhaps this is why, over the years, wehave done so little to investigate whether

they are true on the ground.

This report, then, is rather special in thatit goes beyond assumptions. It records amethodical investigation o the eects oaccountability mechanisms on the qualityand results o aid. The author and designteam made strenuous attempts to ensurethat the methods used were robust andthat, as a result, the conclusions are as

evidential as possible. Given the nature othe topic and the difculty o measuringindividual perceptions, social dynamicsand cultural values, this is a signifcantachievement.

The report is the product o a collaborativeand collegial design process, led byChristian Aid, Save the Children UK and theHumanitarian Accountability Partnership,and including colleagues rom morethan 15 other organisations. Its fndingssuggest that, in developing accountabilitymechanisms, a modest investmentbrings a signifcant return. By deliveringconvincing evidence through a single jointstudy, it shows that the same can be trueo research.

Paul Knox-ClarkeHead o Research and Communications, ALNAP


4/16

4 Improving impact Introduction

In recent years, the aid sectorhas made significant efforts tostrengthen accountability betweenimplementing agencies and projectparticipants. But accountabilitymechanisms that include

the provision of information,participation of stakeholders, andthe means to elicit and respondto complaints are still far fromcommonplace and evidence oftheir contribution to project qualityand impact is scant.

A more systematic examinationof the role that these mechanismsplay in strengthening programmequality and impact is required toensure the best possible servicesare provided to the communities

that need them. This report gathersevidence that will start to fill thisinformation gap.

We propose that accountabilitymechanisms improve the quality ofprojects, including their relevance,effectiveness, efficiency andsustainability.

To test and advance this theory, wereviewed evidence from existingdocuments and supplemented itwith case study research in Kenyaand Myanmar.

The findings are expectedto contribute to the case forintroducing accountabilitymechanisms into projects anddemonstrate the importance ofpromoting them.1

The state of accountabilityin the sector

The humanitarian and development

sectors accountability to projectparticipants has, for many years, laggedbehind other aspects of internationalassistance. Until recently, agenciesrarely put into place or formalisedmeasures to provide information tothose receiving aid, to ensure their

participation in projects or to provide ameans for their feedback or complaint.

A story of progressThis accountability deficit is now beingaddressed. A revolution in qualityand accountability, which gainedprominence in the mid-1990s, has led tosignificant progress in the sector.

Driven in part by a move towardsrights-based programming, andinformed by humanitarian principlesand the Red Cross/NGO Code ofConduct, there has been a fundamentalshift in practice. In humanitariansettings, in particular, accountabilitymechanisms are becoming the normrather than the exception. A range of

methodologies and approaches havedeveloped that seek to pass power tothose receiving assistance.

Organisations and inter-agencyinitiatives such as the HumanitarianAccountability Partnership (HAP)and the Active Learning Networkon Accountability and Performancein Humanitarian Action (ALNAP),have helped make the humanitariansector more accountable to affectedcommunities.

Many of the country-level UNcoordination clusters have alsodeveloped tools to make thehumanitarian sector more accountableto affected communities, as have theterms of reference for humanitariancoordinators and country teams. Therecent inclusion of accountabilityto affected populations as a keycomponent of the TransformativeAgenda agreed by the Inter-AgencyStanding Committee in December 2011 has now provided the foundation foreffective collective action.

Work still to doThese are all encouraging steps, butmuch more still needs to be done. Thisis particularly true of collective effortsacross the sector, where reports

and evaluations continue to highlightdeficiencies. While the 2012 edition ofthe ALNAP State of the HumanitarianSystem report praises the progressmade by humanitarian agencies inestablishing information, participation

and complaints systems, thestakeholder survey (see figure I) revealsthat this progress has yet to meet theexpectations of project participants.For each of the questions onaccountability mechanisms, the level ofdissatisfaction with the performance ofaid agencies was considerably greaterthan the level of satisfaction.

The challenge of(mis)perceptionsThere are compelling operational

reasons for strengtheningaccountability. In a growing numberof countries, relief and developmentagencies operate in highly politicisedand insecure contexts, where the aidcommunity rubs shoulders with less-principled actors and where foreigngovernments often seek to diluteimpartial assistance to meet broaderpolitical goals.

In such places, it is not only importantfor agencies to do what they say theywill do, they also need to distinguish

themselves from those that may createthe perception of a lack of impartialityand independence.

This gap is further widened by the poorperformance of some aid providers.Several recent studies2 have shownthat in some of the most complexcontexts there is a significant gapbetween the type, quality and quantityof assistance required and that which isprovided. This reveals an accountabilitydeficit for agencies that has potentialoperational and staff-security

consequences.

Conversely, by building trust,understanding and a sense ofownership of projects, effectiveaccountability mechanisms may havepositive implications for staff security.

INTRODUCTION


5/16


6/16

6 Improving impact The research case studies

THE RESEARCHCASE STUDIES

Methodology

The research focused on threebenchmarks of accountability andquality management from theHumanitarian Accountability

Partnership (HAP) Standard:information sharing, participation,and complaints handling. Togetherthese were considered to form anaccountability mechanism.

The research drew on a literaturereview (including an analysis ofdocuments submitted by membersof the HAP peer-learning group5)and on research involving two casestudies piloting the methodology, inKenya and Myanmar.

The first step in the case studyresearch was to assess how wellcomponents of the accountabilitymechanism functioned against thethree HAP benchmarks, using anadaptation of the Listen FirstFramework.6 The second step wasto assess the contribution of theaccountability mechanism toprogramme quality.

Details of the methodology andfindings are provided in a longer, online,PDF version of this report.

It can be found on the websites ofthe commissioning agencies.7

The findings of the research providesignificant evidence of the link betweenaccountability mechanisms and therelevance, effectiveness, efficiency andsustainability of projects.

Christian Aid partnerUkamba ChristianCommunity Services in

Makueni County, KenyaWith support from Christian Aid,and funded by DFIDs ProgrammePartnership Arrangement, UkambaChristian Community Services (UCCS)

has been implementing a resilienceproject in Makueni County in lowerEastern Kenya. Adopting aparticipatory vulnerability and capacityassessment methodology, UCCSworked with the community to identifythe main issues preventing families

from establishing and maintaining agood living. The community identifieddrought, conflict, environmentaldamage, poor health, low incomes,youth unemployment, substanceabuse and lack of school funds as thebiggest obstacles preventing themfrom getting out of poverty.

Community-based organisations(CBOs), led by programmemanagement committees, wereestablished in each area to organiseand oversee project work. Theyhave been targeted by Christian Aidfor initiatives to strengthen theirorganisation and activities.

On the basis of the participatoryexercises and focus-group discussions,

the benchmark components of theaccountability mechanism in eachvillage were assessed and given arating, using the adapted Listen FirstFramework.

The participation of the CBOs in

decision-making (supported by thesystem of mass meetings), meant thatin all villages information sharing wasjudged to be at an intermediate stage.The most significant limitation was alack of shared financial information.

Because the CBO had responsibilityfor articulating the communitys needs(proposing projects and implementingnon-technical aspects of projects), in allvillages participation was considered tobe at a mature stage.

In the villages that had longer

relationships with UCCS, theknowledge and use of the formalcomplaints and response mechanism(CRM) with opportunity to give directfeedback to the project officer meantthat complaints handling was judged

Christian Aid-funded sand dam, Makueni County, Kenya

ChristianAid/A

mandaFarrant


7/16

7Improving impact The research case studies

Save the Children creates and worksthrough village-level CBOs, developingtheir ability to identify, implement andadminister projects over a numberof years. The work of the CBOs isenhanced by the involvement ofchildrens groups, which develop,implement and oversee aspects ofthe work.

The knowledge that communitieshad of Save the Children, theprogrammes, and of progress madeagainst objectives, demonstratedthe effectiveness of the informationsharing and engagement of thecommunity.

The level of participation in projectselection, design and delivery, meantthat both information sharing andparticipation were considered mature.(The sharing of information about thedirect costs associated with village-level projects, and participation inthe management of the budget, areworthy of particular mention.)

In the villages where the formal CRMhad been rolled out, the availabilityof formal and informal methods forfeeding back and raising complaintsmerited a mature score. Butpreference for use of the informalmechanisms suggests there maybe further scope for working withvillage members to explore issues ofrelevance.

Where the CRM had yet to be rolledout, the villages received a basicscore. That said, the communityhad substantial confidence in theinformal mechanisms that existed forcomplaints and feedback.

When the scores for each of thethree accountability componentswere added together, the villages

that benefited from a formal CRMwere judged as mature, whilethose without were consideredintermediate.

to be at a mature stage. In the villagesthat had a far shorter relationship withUCCS there was a gap in understandingof the CRM and complaints handlingwas judged to be at an intermediatestage.

When the scores for each of thethree components were addedtogether, the accountabilitymechanisms in the three villages thathad the longer relationships with UCCSwere given an overall assessmentof mature, while the accountabilitymechanisms in the villages where therelationship was shorter were assessedas intermediate.

Save the Children

in Meiktila andKyaukpadaungtownships, Myanmar

Save the Childrens programme inMyanmars Kyaukpadaung and Meiktilatownships focuses on child protectionand non-formal education. The LearningEducation and Active Participationproject aims to ensure children,communities and civil-society networksin Myanmar actively contribute to thecountrys development and positivesocial change. To that end, it increasesaccess for out-of-school children, aged9-16, to basic competencies and lifeskills, and supports education-focusedCBOs.

Funded by the United States Agencyfor International Development, Savethe Childrens Civil Society, Communityand Township programme seeks topromote good governance by buildingon its distinctive competence andexperience in child protection andchild-rights governance programmes.Through the programme, Save theChildren supports CBOs, children andyoung people. It also helps community-based structures and mechanisms tomonitor, respond to and prevent childrights and protection violations.

Children in Meiktila play during break-time at the summer school run by the local CBO

SavetheChildre

nInternationalMyanmar


8/16

8 Improving impact Accountability mechanisms and project quality

Having assessed the functioning ofthe accountability mechanisms in thetwo case studies, the next step wasto explore the contribution made bythe mechanisms to project quality andimpact. This was investigated through

participatory exercises, and the resultswere used to validate the assumptionsdeveloped by the HAP peer-learning

group. The findings for the casestudies are summarised by village infigure II, below.

As well as the two case studies,relevant examples from the documents

submitted by the HAP peer-learninggroup and the literature review havebeen included in the analysis of the

findings. In the analysis, instanceswhere the assumed contribution ofthe accountability mechanisms toproject quality was validated during theresearch have been highlighted and anidentifier has been inserted to clarify

the link with figure II.

Criteria and Assumed contribution of 11 villagesidentifier accountability mechanism (five intermediate and six mature)

(information, participation,complaints) to project quality

RelevanceR1

R2

EffectivenessE1

E2

E3

E4

E5

EfficiencyV1

SustainabilityS1

S2

ImpactI1

Assessment against the adaptedListen First Framework

Village reference

Assisting in the identification and targeting ofthe most vulnerable or relevant participants

Ensuring the assistance is most suited to theneeds and priorities of the participant group

Increasing participant understanding anduptake of the project

Strengthening the relationship between theparticipants and the agency

Respecting the dignity of participants andempowering communities

Identifying and addressing problems swiftly

(including fraud and mismanagement)

Strengthening operational security

Optimising the use of programme resources

Strengthening the contextual basis for theproject (improving the relevance of theproject will increase its sustainability)

Increasing participant ownership of theprocess

Exposure to and/or use of the accountabilitymechanism has had intended/unintended

impact

Myanmar Kenya

Intermediate Mature Intermediate Mature

M1 M2 M3 M4 M5 K1 K2 K3 K4 K5 K6

Figure II: Summary table of the contribution of the accountabil ity mechanism to project quality, Myanmar and Kenya

ACCOUNTABILITYMECHANISMS AND

PROJECT QUALITY


9/16

had been raised either formally orinformally that, because Save theChildren had responded, had increasedthe benefit of the programme to thecommunity.

The NFE students suggested thatthe library should be moved tothe middle of the village, from itslocation near to the monastery, asit was problematic for mixed-agechildren to use it. Save the Childrenagreed and it was moved, whichallowed more children to benefitfrom it. If there hadnt been an opendiscussion, then this wouldnt havebeen possible and [the project]wouldnt have been so relevant.

Childrens group member, Yone village,

Myanmar

In all of the villages where the researchwas conducted, participationwas considered an importantcontribution to ensuring thatthe project met the needs andpriorities of communities (R2).Lack of participation was considereda significant hindrance to successfulinterventions. And while the UCCSprogramme in Kenya was felt tobenefit from very strong involvementby the community, examples were

given of other projects where a lackof participation had resulted in poorproject outcomes.

We agreed that a dam would addressboth the water and food securityproblems. The community used its

knowledge of the context to ensurethe project was targeted correctly andthis made it more successful.

Programme Management Committee

member, Kalawani location, Kenya

EffectivenessAccountability mechanisms,particularly the provision ofinformation, were considered tohave facilitated project progress andparticipation in 9 of the 11 villages. Theavailability of information about theagency and the project in both casestudies played an important part in theprojects success.

In the Kenya case study, the use

of mass meetings to inform thecommunity about the project ensuredthat knowledge was widespread. Forthis reason, people trusted UCCS andhad a good understanding of what theorganisation had achieved.

In the context of a participatory projectrelying, to a large extent, on thecommunity sharing the responsibilityfor undertaking the work (by providinglabour and sand for dams, overseeingthe work of contractors and keepingrecords for stock levels), provision

of information was consideredessential for the work to progress(E1) and it meant UCCS only neededto provide minimal supervision.

Information has helped peoplecome together to work in the project

9Improving impact The findings

RelevanceThe participation of affectedcommunities in the project wasconsidered to have made an importantcontribution to the success of theproject in 4 of the 11 villages. In each,an explicit link was made betweenthe participation of affectedcommunities and the successfultargeting of the most vulnerableparticipants (R1).

For the selection of students for thenon-formal education programme,the villages have the informationand so we are better able to selectthe right people. We can decidefor ourselves what is best for thecommunity and are better able toget the targeting right.

Male focus-group member, Nat Gyi Kone

village, Myanmar

We prioritised the dry areas andin doing so made choices aboutwho was most in need. This ledto a better outcome as we knowthe needs of the community others dont.



More specifically, the documents

submitted by the HAP peer-learninggroup identified several instanceswhere the use of complaintsmechanisms by community membershighlighted errors of inclusion orexclusion, and in so doing improvedthe targeting of programmes.

Based on complaints, thecommittee held an open villagemeeting and a decision was made,based on the selection criteria, thatone name should be removed fromthe list and another one added.

World Vision International, accountabilityto communities feedback and complaints

mechanisms in Bangladesh

In discussion with childrens-groupmembers in Myanmar, exampleswere given of programme issues that

Save the Children complaints response mechanism in Myanmar

THE FINDINGS


10/16

stalled and there would have beenanimosity in the community.


member, Mutulu location, Kenya

The provision of information andgood communication betweenUCCS and the community was feltto have built on the contribution ofthe accountability mechanisms toensure project progress and to havelimited the possibility of fraud. In sodoing, it ensured that the intendedbeneficiaries received the maximumbenefit.

The contractor overseeing the workon the dam said the communityshould rent him a house. We wereuncertain of this and checked withthe UCCS office, who respondedimmediately to say that they hadalready given him the money todo this.



The literature review highlightedseveral examples demonstratingthe link between the participation ofcommunities in the implementationand management of project workand the best use of resources andvalue for money (V1). It also providedexamples of how accountability

mechanisms can help ensure theimplementing organisation delivers onits commitments.

Monitoring committees cancheck that the work is done ontime and to a good standard. InBurkina Faso, a new school wasbeing funded by local governmentwho contracted a local buildingcompany to do the work. Previously,government contractors would beleft unsupervised by governmentofficials, and this could cause

problems as sometimes buildingwork could be sub-standard and notfinished on time. To address thisproblem, the monitoring committee,made up of community members,was responsible for checkingthe quality of the work and they

10 Improving impact The findings

as they know what is happening andhow to assist.



Save the Childrens transparency

was felt to have helped gain thecommunities trust and respect (E2),contributing to the effectiveness of theproject. The importance of two-waycommunication, through the provisionof information and communityparticipation, was considered to havebenefited the project in 7 of the 11villages in which the research tookplace. During focus-group discussionsin two of the villages, an explicitlink between empowerment andaccountability (E3) was made.

Save the Children listen to usand encourage us to decide byourselves. This has helped build trustbetween us.

Female focus group members, Lat Pan

Khar Kough village, Myanmar

In the building of the village library[which children helped plan andimplement], Save the Childrenlistened to us and, as a result, theproject is of better quality. It makeschildren feel valued.

Childrens group members, Lat Pan Khar

Kough village, Myanmar

In 2 of the 11 villages, discussionshighlighted how accountabilitymechanisms can raise andresolve issues of potential fraudor mismanagement (E4). In Kenya,the formal complaints mechanismhad addressed a problem in theway in which the community wasparticipating in the project.

Each of the different groups took

it in turns to assist with the damconstruction, but not all groupsbothered to turn up, which manythought was unfair. If we didnthave a complaints mechanism wewouldnt have been able to addressthis, and the programme would have

Signboards: at a Christian-Aid-funded sand dam, Makueni County, Kenya

Andy

Featherstone


11/16


could also directly feedback to thegovernment to challenge the qualityof the work if they deemed it to bepoor. The committee also oversawthe workers as they built the school,to ensure that they worked the hours

that they were contracted to do.Christian Aid, Accountability to Affected

Populations and Value for Money8

With the setting up of complaintsand response mechanisms (CRM),and the communities and childrenbeing made aware of CRM andinformation boards, an increasedtrust has been created between thecommunity and the partner NGO.The staff also realise their increasedaccountability towards the childrenand community, as they know that

the community and children arenow empowered It has also led toproactiveness among staff in dealingwith issues of childrens feedbackfrom the India programme.

Save the Children, Breakthrough quarterly

reports 2011-2012

Neither of the research locationssuffered from insecurity at the timethe research was conducted. Butan example in a paper written byCDA Collaborative Learning Projects(2012)9 did provide insight into howaccountability mechanisms canstrengthen the security of thoseproviding aid (E5).

For several agencies, instancesof violence against staff workingin difficult environments werereduced after they improvedtheir communication andfeedback processes with affectedcommunities. Through increaseddialogue and better communication,organisations that had beenexperiencing violence or threatseventually found themselveson positive terms with localcommunities, and in some caseswere even protected by the localcommunity from armed groups.

EfficiencyThe two case studies didnt offer anyevidence about whether accountabilityimproved efficiency. This wasdisappointing, but understandablegiven the nature of the interventionsand how information was shared. InKenya, the community had very limitedaccess to financial information andtherefore little input into budgetarydiscussions. Conversely, in Myanmarthe community had control over howvillage-level budgets were spent.

Better evidence was collectedwhere both the community and theorganisation had discussed howfunds were spent, for example in acommunity resilience programme in

Chin State, Myanmar:It had initially been planned to buypaddy in Paletwa, but at the timethe procurement was happeningthere was insufficient stock availablein the township. The project teamdiscussed this with the concernedcommunity development committees(CDC) and agreed to procure thepaddy from Kyauk Taw, which islocated in Rakhine state and is alsothe entrance of the plane landsto Paletwa. The CDC from each

of the villages went to Kyauk Tawto purchase the paddy, includingvisiting the paddy field during whichthey learned new crop practicesand networked with local farmers.In addition to securing cheaperprices, they also made importantcommercial links.

Similar to the paddy example, theoriginal plan was to procure timberfor rice-bank construction in Paletwa.When the time came to place theorder, it was found to be very difficultto procure sufficient quantities andquality of timber in Paletwa, andthe transportation cost was foundto be high. In discussion betweenthe project team and CDCs from thetarget villages, it was agreed to buy

wood in their respective villages. Thechange in plan reduced procurementcosts, in addition to negating theneed to pay for transport. It alsocreated income for some of thevillagers.

SustainabilityAccountability mechanisms wereconsidered to have contributed tothe sustainability of projects in 10 ofthe 11 villages. Reference was madeto the link between the contextualrelevance of a project and itssustainability (S1) in one village,emphasising both the importanceof community participation in theselection of the intervention and howthis can increase the longevity of a

projects benefits.The community, CBO and childrensgroup all discussed and preparedthe proposal. We had the bestinformation and were able to takethe decision ourselves. This isan important building block forsustainability. We cant sustain theproject ourselves at this time, but ourparticipation has helped us towardsthis [goal].

Childrens group member, Nat Gyi Kone

village, Myanmar

Frequent reference was made tocommunities owning projectsthat they had been involved in andhow this increased sustainability(S2). Communities having access toinformation and being able to raiseconcerns was also felt to make projectsmore sustainable.

Because we participated in selectingthe project it is ours, and so werewilling to give our time to it and wevalue the intervention.

Programme Management Committeemember, Kyawango location, Kenya

[Participation] in the decision-makingprocess is important as it brings


12/16

12 Improving impact The findings

peoples ownership, which willcontinue beyond the project lifespan.

Mens focus group member, Lat Pan Khar


Conversely, there was considerableconcern about the sustainability ofprojects in which communities wereunable to participate.

If we hadnt participated in the

decision-making about the project,we would not be willing to providelabour and would not take care of itin the same way. There would be agreater likelihood that it would fail tomeet our need and there would be agreater risk of bringing conflict to thecommunity.



ImpactThe Myanmar case study offered anumber of examples of how exposureto, and use of, accountabilitymechanisms had influenced theactions of communities in waysthat went beyond the expected

outcomes of the project (I1). Theseranged from community decision-making processes through to raisingcomplaints with banks. While it isdifficult to quantify the influences, ineach circumstance the communitymade explicit reference to theaccountability mechanism as aprimary sway.

Having participated in decision-

making processes [with Save theChildren] we have now adoptedthe process to help organise ourvillage rice donation. Previously wedidnt know how to make decisionstogether, so this is a lesson for uswhich we now use for broadercommunity decision-making.

Mens focus group member, Lat Pan Khar


Previously the village elders had nopractice of mass meetings; now they

choose to hold more formal meetingsand have a proper [transparent]system for budget management, andwe vote on important issues.Male focus group member,

Yae Cho village, Myanmar

Opinion-ranking exercise, Meiktila Township, Myanmar

AndyFeatherstone

In the Kenya case study, familiaritywith UCCS participatory planningand decision-making processes wasfelt to have had a broader impacton the community (I1). One of thevillages proposed a more participatory

approach to decision-making in religiousgatherings. This was felt to reduceconflict among members and make theproject more successful.

Were using the knowledge wehave of participation to bring peopletogether in the church, in order toassist them in identifying issues...which has reduced conflict amonggroups.

Programme Management Committee,

Itoleka location, Kenya

Analysis of the findings

The findings from the two casestudies, supplemented by the evidencesubmitted by the HAP peer-learninggroup and literature review, addconsiderable weight to the hypothesisthat accountability mechanisms makean important contribution to programmequality. The findings also provide someimportant insights into how they dothis and of the contribution made by

specific accountability components tothe relevance, effectiveness, efficiencyand sustainability of a project. Thesecontributions are summarised below.

Relevance: strengtheningthe targeting and qualityof assistanceAccountability mechanisms improvedthe targeting of assistance, the natureof supported interventions and thelocation of services. Communityparticipation provided agencies

with a better understanding of localvulnerabilities and increased theusefulness of projects to communities.

AndyFeatherstone


13/16


Effectiveness: promotingtrust, empowerment andacceptance, and addressingmismanagementAccountability mechanisms have

strengthened trust between agenciesand project participants, andhighlighted the link betweencommunity participation and ownership.

The research revealed evidence ofincreased empowerment and self-esteem among project participants. Italso noted greater willingness ofgroups to demand accountability fromother duty bearers schools, localauthorities and even private companies.

The literature showed thataccountability mechanisms made a

contribution to trust dividends withcommunities in insecure environments,and that there was an associatedreduction in violence against staff andincreased attention paid to fraud andmismanagement.

Efficiency: optimising theuse of resources andpromoting value for moneyThe literature review highlightedseveral instances where communityinvolvement in procurement had

increased a programmes efficiency.Where communities had beenempowered to monitor contractors,there was greater efficiency and valuefor money.

Sustainability: enhancingcommunity ownership ofprojectsAn important link was identifiedbetween the participation of acommunity in a project and perceptionsof its sustainability. The case studies

demonstrate that participation canincrease the relevance of projectsto their context and strengthen acommunitys ownership of processesand results.

Women and men participating in the research survey, Makueni County, Kenya

AndyFeatherstone


14/16

14 Improving impact Key conclusions and recommendations

The results will perhaps come asno surprise: a modest investmentin information sharing (in terms offinancial resources, staff time andagency commitment), involvement byproject participants in the design and

delivery of programmes, and ensuringthere is a means of listening to andacting on feedback, brings a significantreturn not only in participantsatisfaction and engagement inprojects, but also in the tangiblesuccess of projects.

The research, supported by theliterature review, suggests that theuse of accountability mechanismscan help ensure that more ofthe most vulnerable people getthe sort of assistance they most

need. Our examples suggest thatmismanagement and corruptioncan be raised and tackled at thesame time as efficiency and valuefor money is strengthened. Ininsecure environments, accountabilitymechanisms can increase theacceptance and operational securityof aid programmes. More excitingstill is the contribution accountabilitymechanisms can make to empoweringcommunities and in so doing havean impact that goes beyond theimmediate objectives of aid projects.

Examples from the research suggestthat a robust system of accountabilitybetween those offering and receivingassistance can provide a compellingmodel for others to claim their rightsfrom duty bearers.

While the literature review thatpreceded the research highlightedthe progress made in strengtheningaccountability across the sector, itis a concern that development andhumanitarian projects seem opaque

to so many people. It is hoped thatreplicating the methodology tested inKenya and Myanmar, to strengthenthe evidence base and inform agencypractice, will address this perceivedaccountability deficit.

A second key lesson from the researchis the enduring need to contextualiseaccountability, ensuring its relevanceto those communities receivingassistance. As the nature of need, andthe best way to address it, changes,

so should the means for agencies toaccount for their actions.

But while it is important foraccountability mechanisms to becontextualised, the responsibilitiesof those providing assistance remainthe same. If common perceptions ofaccountability imbalances are to bechallenged and changed, there is aneed to ensure that whether themechanisms are considered formalor informal issues are routinelydocumented, and the need for change

assessed, responded to and, whereappropriate, redressed.

Implications for the sectorIn drawing conclusions from theresearch, it is clear that progress isnecessary at both individual-agencylevel and collectively across the sector.

Being better at doing better

The results, even given the smallnumber of case studies, arecompelling. Project-level accountability

mechanisms cannot be consideredan add-on to tick an organisationalor donor box. They are an essentialcontribution to the developmentprocess, irrespective of the natureof the programme (be it short- orlong-term) and the context of theintervention.

Recommendation: Organisationsacross the sector need to be farbetter at routinely ensuring theexistence of strong accountabilitymechanisms, monitoring their

effectiveness and periodicallyevaluating how they contribute toproject outcomes and impact.

The need for a step-changeacross the sector

Linking the research to the HAP peer-learning group provided an excitingopportunity to gather evidence andgood-practice case studies from arange of organisations. Yet, evenwith the support of agencies thathave an organisational commitmentto accountability to those with whomthey work, there were only a handfulof case studies available documentingthe contribution of accountabilitymechanisms to project quality andimpact.

Recommendation: Given the findingsof the research, there is both an urgentneed to strengthen practice and acompelling case to more rigorouslydocument contribution. At best, thelack of evidence represents a missedopportunity At worst it highlights afailure to understand and communicatethe impact that assistance is havingon communities.

Building the evidence base

The study has provided someimportant signposts about thecontribution that accountabilitymechanisms make to project quality,but there are limits to what can be

achieved by analysing just two casestudies. There is significant scope forbuilding on the findings. In particular,research in different contexts wouldstrengthen the applicability and value ofthe results. Cross-agency research ina rapid-onset emergency may provideopportunities to contrast the benefitsof different agency approaches andoffer counterfactual examples thatwould strengthen the data.

Recommendation: Given the priorityplaced on accountability by the

Transformative Agenda, sanctioningand support of the research by aHumanitarian Country Team wouldprovide an exciting opportunity forreal-time feedback, in addition tocontributing important evidence.

KEY CONCLUSIONS ANDRECOMMENDATIONS


15/16

15Improving impact Endnotes

ENDNOTES

1 The terms of reference for the

research can be found in the full

report (see endnote 7).

2 Featherstone A and Abouzeid

A, Its the thought that counts -

humanitarian principles and practice

in Pakistan, ActionAid International,

2010. Available at www.actionaid.

org/publications/its-thought-counts-

humanitarian-principles-and-practice-

pakistan

Also see Medicins sans

Frontires, In the Eyes of Others:

how people in crises perceive

humanitarian aid, 2012. Available at

http://www.msf.org/article/eyes-

others-how-people-crises-perceive-

humanitarian-aid

Also see Anderson M, Brown D

and Jean I, Time to Listen: hearing

people on the receiving end of

international aid, CDA Collaborative

Learning Projects, Cambridge MA,2013. Available at www.cdainc.com/

cdawww/publication.php

3 Featherstone A and Abouzeid A,

Its the thought that counts -

humanitarian principles and practice

in Pakistan, ActionAid International,

2010. Available at www.actionaid.

org/publications/its-thought-counts-

humanitarian-principles-and-practice-

pakistan

4 Glyn Taylor et al, The State of the

Humanitarian System, 2012 Edition,

ALNAP. Available at www.alnap.org/

ourwork/current/sohs.aspx

Survey of recipients of humanitarianaid, p48. Field-based surveys

gathered responses of 1,104 people

who received humanitarian aid during

2009 -2010 in Haiti (179), DRC (325),

Pakistan (100) and Uganda (500). The

questions covered experience with

humanitarian assistance and how the

system could improve.

5 Agencies in the HAP peer-learning

group include Save the Children,

Christian Aid, HAP, Oxfam, World

Vision International, CAFOD, Plan

International, Church of Sweden,

ALNAP, MERLIN, Care International

and HelpAge International.

6 Listen First is a draft set of

tools and approaches that NGOs

can use to make themselves more

accountable to the people they

serve. It was developed jointly by

Concern Worldwide and MANGO

(financial management and

accountability of non-governmentalorganisations). Details of the

approach are available at www.

listenfirst.org/introduction

7 The full report can be found on

the websites of the commissioning

agencies:

HAP International:

www.hapinternational.org/projects/

research/hap-research.aspx

Save the Children UK:

www.savethechildren.org.

uk/resources/online-library/

improving-impact-do-accountability-

mechanisms-deliver-results

Christian Aid:www.christianaid.org.uk/resources/

policy/programme-practice.aspx

8 Christian Aid, Accountability to

Affected Populations and Value for

Money in VOICE Out Loud, issue 16,

October 2012, p6. Available at www.

ngovoice.org/documents/voice%20

out%20loud%2016.pdf

9 CDA Collaborative Learning

Projects, Feedback Mechanisms

in International Assistance

Organizations, Cambridge, MA,

August 2011. Available at www.

cdainc.com/cdawww/pdf/other/

lp_feedback_research_report_Pdf_1.pdf


16/16

d.ie

The Christian Aid name and logo are trademarks of Christian Aid

UK registered charity number 1105851 Company number 5171525

Scotland charity number SC039150

Northern Ireland charity number XR94639 Company number NI059154

Andy Featherstone developed the methodology, led the study and prepared the research report.

Mechack Mutevu facilitated the community discussions in Kenya, and Salai Khin My Aye facilitated

discussions for the Myanmar case study.

In Kenya, thanks are due to Christian Aid for agreeing to support the research and to Nicholas

Abuya, in particular. Ukamba Christian Community Service did an exceptional job of hosting the

research, and Lydia Muithya and John Malonzo both provided excellent support throughout. For

the Myanmar case study, thanks are due to Denis de Poerck of Save the Children for agreeing tohost the study and to Ei Thant Khing (Gracy) and Htoo Lwin for providing such diligent assistance

throughout the trip.

David Loquercio of the Humanitarian Accountability Partnership, Juliet Parker of Christian Aid and

Burcu Munyas of Save the Children UK deserve special mention for their roles in conceiving the

research and providing such steadfast support and encouragement throughout the process.

Members of the HAP peer-learning group have played an important role in supporting the study

in the design of the terms of reference, the development of the methodology and offering case

study material.

It is hoped that the findings of the research presented in this report do justice to the efforts of

all those who have supported the study and offer a practical tool that can be used to build the

evidence-base for the impact of accountability mechanisms to affected communities in the future.

Edited, designed and produced by Christian Aids Publishing team

The Humanitarian Accountability

Partnership is registered as a

non-profit organisation in Geneva

Save the Children registered charity numbers

England & Wales 213890 Scotland SC039570

improving impact - do accountability mechanisms deliver results - short report, 2013

Documents