the global evaluation of emergency response funds (erfs) · needs of those beset by natural...

The Global Evaluation of Emergency Response Funds (ERFs) Volume II: Country Case Study Reports

March 2013

Volume II: Country Case Study Reports

Table of Contents CC OO UU NN TT RR YY CC AA SS EE SS TT UU DD YY RR EE PP OO RR TT -- AA FF GG HH AA NN II SS TT AA NN

CC OO UU NN TT RR YY CC AA SS EE SS TT UU DD YY RR EE PP OO RR TT -- CC OO LL OO MM BB II AA

CC OO UU NN TT RR YY CC AA SS EE SS TT UU DD YY RR EE PP OO RR TT -- EE TT HH II OO PP II AA

CC OO UU NN TT RR YY CC AA SS EE SS TT UU DD YY RR EE PP OO RR TT -- II NN DD OO NN EE SS II AA

CC OO UU NN TT RR YY CC AA SS EE SS TT UU DD YY RR EE PP OO RR TT -- OO CC CC UU PP II EE DD PP AA LL EE SS TT II NN II AA NN TT EE RR RR II TT OO RR YY

EE VV AA LL UU AA TT II OO NN MM AA TT RR II XX AA NN DD TT EE RR MM SS OO FF RR EE FF EE RR EE NN CC EE


The Global Evaluation of Emergency Response Funds (ERFs) Country Case Study Report - Afghanistan

March 2013


i

Disclaimer The contents and conclusions of this evaluation report reflect strictly the opinion of the author, and in no way those of UN, OCHA or donors.

Evaluation Management Evaluation Manager: Tijana Bojanic

Chief, Evaluation and Guidance: Scott Green


ii

Table of Contents Acknowledgements ..................................................................................................................................................... i

Disclaimer .................................................................................................................................................................... i

Evaluation Management .............................................................................................................................................. i

Table of Contents ....................................................................................................................................................... ii

Abbreviations ............................................................................................................................................................. iii

Introduction ................................................................................................................................................................. 1

Purpose of the Evaluation .......................................................................................................................................... 1

Methodology ............................................................................................................................................................... 2

Overview .................................................................................................................................................................... 2

Limitations/Challenges ............................................................................................................................................... 2

The Nature of This Report .................................................................................................................................. 2

Limitations .......................................................................................................................................................... 2

The ERF in Afghanistan - An Overview ...................................................................................................................... 3

Findings ...................................................................................................................................................................... 6

Overall Observations .................................................................................................................................................. 6

Operational Impact ..................................................................................................................................................... 7

Operational Effects ..................................................................................................................................................... 7

Relevance ........................................................................................................................................................... 7

Timeliness .......................................................................................................................................................... 8

Outcomes ................................................................................................................................................................... 8

Leadership and Coordination .............................................................................................................................. 8

Critical Gaps Filled.............................................................................................................................................. 8

Partnership ................................................................................................................................................................. 8

Processes and Inputs ................................................................................................................................................. 9

Review of Proposals ........................................................................................................................................... 9

Funding and Project Issues ........................................................................................................................................ 9

Mainstreaming Gender ....................................................................................................................................... 9

Operation of Advisory and Review Boards ......................................................................................................... 9

Monitoring and Evaluation ........................................................................................................................................ 10

Country Level Recommendations............................................................................................................................. 11

Appendix I: Programme Agenda............................................................................................................................... 13

Appendix II: Stakeholders ......................................................................................................................................... 15


iii

Abbreviations AO Administrative Office

CAP Consolidated Appeal Process

CAR Central African Republic

CERF Central Emergency Response Fund

CHF Common Humanitarian Fund

DAC Development Assistance Committee

DfID Department for International Development, UK

DRR Disaster Risk and Reduction

EGS Evaluation and Guidance Section

ERF Emergency Response Fund

EU European Union

FCS Funding and Coordination Section

FTS Financial Tracking System

GHD Good Humanitarian Donorship

HC Humanitarian Coordinator

HCT Humanitarian Country Team

HRF Humanitarian Response Fund

IASC Inter-Agency Standing Committee

IDP Internally Displaced Person

INGO International Non Governmental Organization

NGO Non Governmental Organization

NNGO National Non Governmental Organization

OCHA Office for the Coordination of Humanitarian Affairs

OECD Organization for Economic Cooperation and Development

OIOS Office of Internal Oversight Services (United Nations)

oPT Occupied Palestinian Territory

RC Regional Coordinator

TOR Terms of Reference

UMG Universalia Management Group

UN United Nations

UNICEF United Nations Children’s Fund


1

Introduction This is one of five Country Case Study Reports that are mandated as a contributing part of the Global Evaluation of ERFs, which is taking place in the latter half of 2012. This case study report reflects the findings arising from a 10 person day mission to Afghanistan by a staff of two, one from Universalia and one from OCHA’s Evaluation and Guidance Section (EGS). Details of the mission can be found in Appendices I and II.

This Country Case Study Report is firmly focused on conditions in Afghanistan. While it is obvious that global conditions and the overall management system of the ERFs impacts on the delivery of services in Indonesia, this report will not directly address these global issues. Rather, how these global considerations affect the performance of the ERFs as a whole will figure largely in the overall report of this evaluation

Universalia wishes to thank the management and staff of OCHA, that of its UN systems partners, donor representatives, government stakeholder and the representative of the NNGOs and INGOs who have generously given their time and opinions. Without them, this evaluation as a whole would not be possible.

PURPOSE OF THE EVALUATION In a protracted process such as this, there is a clear benefit to beginning a Country Case Study Report by reprising the heart and soul of the exercise as a whole, namely the Objectives and Key Areas of the evaluation. Therefore, as per the TOR, the Objectives of the evaluation are to:

• Provide an independent assessment of the contribution of ERFs to the humanitarian community’s ability to address critical unforeseen humanitarian needs in a timely and effective manner;

• Examine the contribution of ERFs to strengthening the leadership of Humanitarian Coordinators, coordination role of the cluster system, and building partnerships (in particular with national and local NGOs);

• Examine the role of the elements of OCHA that plan, administer and report on ERFs, map the progress made since 2009 in improving the effectiveness and efficiency of ERFs, and identify the areas of internal strengths and weaknesses.

The TOR laid out a set of Key Areas to be examined. They are:

• The success of ERFs in meeting their goal to support early action and response to critical unforeseen humanitarian needs in a timely and effective manner;

• The performance, ‘value added’, and cost effectiveness of ERFs in the context of the overall humanitarian architecture and humanitarian reform process (e.g. RC/HC strengthening, cluster approach, development of effective partnership between UN and non-UN humanitarian actors, including local and national NGOs);

• The level and nature of accountability between ERF recipients to RC/HC, donors, beneficiaries, and others; • The strategic, managerial and operational aspects of ERFs, including structure and focus, the efficiency and

effectiveness of administration, fundraising, needs assessment, criteria for allocation of funds, monitoring and reporting, and connectedness with other funding sources;

• Internal (e.g. staffing capacity, management arrangements) and external (e.g. contextual and political) factors which affect ability of ERFs to deliver on their objectives; and

• The accountability and risk management systems in place.

This evaluation will generate specific, actionable recommendations whose purpose is twofold:

• At the global level, to inform the review of ERF Guidelines in 2013 and the development of policy in relevant areas; and

• At the country level, to lead to improvements in ERF management, processes and operations.

.


2

Methodology OVERVIEW In our proposal, the methodology proposed for the assignment consisted of a mixture of qualitative and quantitative data collection, with attention both to forward-looking, improvement oriented questions and to assessment of impact. In summary, we will follow the OCHA Evaluation Policy objective of determining:

• Is the right thing being done? • Is it being done well? • Could it be done any differently?

The general steps envisaged for this evaluation remain largely unchanged, and include the following steps:

• Data collection (including survey administration, field missions and desk-top country cases); • Data analysis; • Presentation of initial findings at the conclusion of data collection; • Preparation and submission of a draft final report for comments by OCHA; • Preparation of the Final Report, including executive summary and all appendices; and • Presentation of the final findings to the Evaluation Reference Group.

LIMITATIONS/CHALLENGES

The Nature of This Report

This is one of five field mission reports for the Global Evaluation of ERF. It is not a specific and detailed evaluation of the ERF in Afghanistan, but an overview report. In light of the approved methodology, this report drills down using Afghanistan as an example to explore at a practical level the key evaluation issues. Given the resources available for it (see below) it is unreasonable to consider that such an overview and summary of the ERF in Afghanistan could be considered to be an exhaustive. Specifically, unlike other four country-based reports, this overview does not attempt to provide a comprehensive assessment of the ERF’s overall performance. Rather it highlights the nature of ERF behaviours and practices within the context of the key evaluation issues.

Limitations

The evaluation team wishes to thank the staff and management of the ERF in Afghanistan for developing a comprehensive program. Timing constraints, given only a one week window of opportunity, reduced the team’s ability to meet face-to-face with some of the stakeholders identified by the country staff, largely due to schedule conflicts.

Another limitation with respect to Afghanistan was the extent of the impact of the national security situation on the ability to engage stakeholders. Even in the capital of Kabul, security considerations somewhat restricted the mobility of the team and also the ability of some stakeholders to meet with the team even in the now relocated OCHA office space. The number of projects in Afghanistan as a whole is relatively few, with many involving NNGOs in somewhat remote and insecure locations. As well, during the time of this evaluation, the third and fourth quarters of 2012, few projects remained in operation given the fact that the ERF had not approved any new projects since early in 2012. This naturally reduced the pool of NGO stakeholders.

Nevertheless, the breadth and scope of those engaged is clearly sufficient to ensure triangulation of data as well as a variety of sources of opinion.

The documentary data provided by the ERF staff more than met the requirements of this evaluation. However, the small number of projects over the last several years makes it difficult to disaggregate any trends within this data other than the most basic with respect to accuracy and timeliness.

There were several more strategic limitations that impacted on the evaluation. The agreed upon methodology of allocating about the same amount of time for data collection in each of the five locales proved to be somewhat problematic in that HRFs/ERFs are vastly different in size, ranging from ones like Afghanistan whose annual project expenditures are in the range of US$ 2-4 million to Ethiopia where its Review Board on a frequent weekly basis regularly assesses about US$1 million per week.

As well, the Terms of Reference for the assignment and the evaluation questions laid out in the TOR appear to have been based on some form of standardized ERF/HRF and were not sufficiently discrete to address the implications of such a degree of difference in magnitude. The evaluation team worked around this latter challenge by adapting interview protocols to more adequately reflect conditions in Afghanistan..


3

The ERF in Afghanistan - An Overview The ERF in Afghanistan was established in July 2009 and became operational in February 2010 to provide national and international NGOs with access to rapid and flexible funding to support life-saving needs of the vulnerable population. The ERF is an impartial source of humanitarian funding and supports activities to meet emergency needs of those beset by natural disasters and conflicts. OCHA ensures the management and financial administration of the ERF on behalf of the Humanitarian Coordinator (HC).

In November 2011, the first meeting of the Afghanistan ERF Advisory Board was convened under the chairmanship of the Humanitarian Coordinator, Mr. Michael Keating with membership drawn from UN Agencies (UNHCR, UNICEF, WFP, WHO and IOM), four NGOs nominated from the NGO forum, ACBAR and participating donors. The Board endorsed the global guidelines and requested a greater focus on conflict-induced emergencies with allocations more broadly distributed across the geographical regions, particularly in the south and southeast. In order to promote greater donor awareness and participation in the ERF beyond the five current contributing donors to achieve the fundraising target set, it was agreed to invite non-contributing donors to participate as observers. NGO members indicated that the reconfirmation of nominees would be subject to an internal consultation process within ACBAR. Following the Advisory Board meeting, the ERF Review Board was reconstituted under OCHA leadership to review and recommend proposals to the Humanitarian Coordinator within the policy parameters now prescribed.

In 2012, Afghanistan remains a country with significant development deficits and shortfalls which are exacerbated by protracted conflict and recurrent natural hazards. Notwithstanding the various improvements recorded over the past decade, Afghanistan remained towards the bottom of all human development indicator league tables. Although 5.7 million refugees have returned to Afghanistan over the past decade with 67,000 returning during the past year, in excess of five million Afghans remain in neighboring countries, more than three million as registered refugees. The combined total represents almost 40% of the population; conditions for reintegration into Afghan community life remain challenging. Within Afghanistan, rising insecurity – armed conflict, intimidation and harassment, tribal conflict and cross-border shelling - has resulted in an increase of the number of conflict-induced displacements by 185,000 to 447,000 by 31 December 2011, an increase of 45% over 2010 levels. Rising insecurity has also adversely impacted on the ability of humanitarian workers to access people in need and for these same people to access basic services. Afghanistan’s recurrent natural hazards include extreme winter conditions, flooding and landslides, drought and earthquakes. The drought declared in July 2011, the eighth in eleven years, affected almost three million people across fourteen provinces in the north, northeast and west of the country when rain-fed wheat crops failed.

The ERF in Afghanistan is one of the smaller ERFs. The following tables illustrate its size, and more importantly, how it was unable to grow to meet needs, leaving it effectively without resources at the time of the evaluation. It should be noted that the balance which existed at the end of 2011 was largely allocated during the first months of 2012, resulting in a situation where the Fund was unable to approve projects post March 2012.

Exhibit 1 Funding trend 2009-2011 (in US$)

2009 2010 2011 2012

Target 5,000,000.00 8,000,000.00 8,000,000.00

Received 3,450,250.00 2,832,064.00 4,586,370.00 0

Allocated 4,072,560.00 3,598,760.82 2,695,069.21

Expenditures 0 3,797,474.25 3,524,945

The following tables show the distribution of the Fund since commencement in terms of type of NGO. It is interesting to note that the ERF in Afghanistan has not provided any support to UN bodies. The table below also illustrates the sector disbursement, showing a very large concentration in shelter and non-food items, a somewhat different pattern than other ERFs.


4

Exhibit 2 Disbursement by Sector

Finally, this table shows the geographic concentration of ERF activity and the heavy concentration in one or two provinces.

NNGO, 4,225,402.82, 41%

INGO, 6,141,014.21, 59%

WASH11%

Emergency Shelter

44%

Health2%

NFI distribution

10%

Food Security27%

Nutrition3%

Protection3%


5


6

Findings This country case study is structured differently from its four companion documents. This is in large part due to the very different circumstance of the ERF in Afghanistan and its present condition. As a preface to either general conclusions or specific findings, it is important to underscore that virtually all OCHA and UN system stakeholders recognized that the ERF in Afghanistan had evolved toward a strategic crossroads where the choice between closing it outright or restarting it with a completely new approach had become imperative.

It was also evident that OCHA stakeholders in particular recognized the degree to which the ERF mechanism required a complete restart and simply could not be amended, or “fixed”. Therefore, this country case study is set within the context of an ERF that is presently not operational and which will be restarted.

This leads to a different sort of set of findings and conclusions, mainly ones directed to assist OCHA staff and managers in the restart by identifying areas where the recurrence of past performance might detract from the future success of the ERF.

OVERALL OBSERVATIONS While initial progress was promising, the ERF in Afghanistan has not met its potential.

Unlike many other locales, the combination of natural disasters and conflict situations has led to a highly problematic environment in which to operate a demand-driven pooled fund. One of the key limitations to the operation of the ERF in Afghanistan in the past appears to have been a highly conservative approach to risk management. Difficult circumstances, combined with conservative leadership have led to missed opportunities to demonstrate its worth.

For example, the majority of ERF projects in Afghanistan have addressed the provision of shelter, in most instances, months after a flood or winter storm. Land restoration was also a major project focus. In discussion with both recipients and OCHA staff past and present, it became apparent that such projects carried fairly low levels of risk to their attainment. By contrast, projects such as one very recently approved to provide emergency health services for civilians living in a high risk combat prone area, were characterized as “too risky” and lacking in key components such as statistical baselines. Decisions such as these have been recently rescinded. In short, the idea of an ERF being a relatively rapid gap filler or response mechanism to an unforeseen need was not realized in Afghanistan to any significant degree.

A sufficient critical mass does not presently exist.

The ERF in Afghanistan was not able to attract a sufficient degree of donor support to give it the critical mass required to be able to support a range of interventions and to be able to show what difference it was making. As noted above, highly conservative leadership combined with a very low tolerance for risk resulted in the Fund losing visibility and to some extent, losing credibility.

The potential to expand appears not to have been explored actively.

Several gaps in the ERF architecture since 2009 point to a conclusion that very little was done to expand it or to raise its credibility. The establishment only in the latter months of 2011 of an Advisory Board is evidence of this tendency. However, there are also operational reasons for the apparent lack of efforts to increase the profile of the ERF or to sustain its donor base, which had fallen off significantly. The history of having a series of short-term Heads of Office of OCHA in Afghanistan combined with a similar pattern with respect to the position of HC clearly would lead to a degree of instability and uncertainty and thus detract from the need for a pooled fund like this one to have established a network of willing partners, among both UN agencies and donors. Donor stakeholders and UN stakeholders alike support this contention relative to leadership.

A Complete Restart is needed

The above general conclusions confirm the recent decision by the OCHA Head of Office to in effect, restart the ERF in Afghanistan and to take all appropriate measures to address what are generally recognized to be a series of strategic missteps. The evidence presented below confirms the need for a restart and hopefully will give such a process some lessons learned so as to improve the chances for a successful revitalization of the ERF.

The following set of specific findings is derived from the questions contained in the approved Evaluation matrix, which can be found as Error! Reference source not found..


7

OPERATIONAL IMPACT Finding 1. The ERF has made a minor but valued contribution to the overall humanitarian response,

however has not been able to expand beyond a narrow base of largely settlement and WASH related activity

This first Finding underscores a positive observation that, notwithstanding some of the issues discussed above with respect to general conclusions and issues that will be addressed subsequently, the Afghanistan ERF has made a difference, albeit a narrowly defined one in the area of shelter and WASH in the northern regions of the country and in response to natural disasters. Its contribution in relation to what are generally perceived to be the greater challenges facing Afghanistan, namely, complex emergencies involving a combination of natural disaster and conflict, has been nominal until very recently, with a number of projects in 2012 that have begun to expand the boundary of the ERF and in some way have begun to show donors that the mechanism has intrinsic worth.

Finding 2. Before the middle of 2012, the ERF does not appear to have had an effect on the overall level of humanitarian financing

The ERF in Afghanistan does not appear to have made any contribution to the overall level of humanitarian funding. Indeed, the Fund itself, until October 2012, was in effect bankrupt and had lost donor support. In the concluding months of 2012, new leadership at the OCHA Office in Afghanistan and a new commitment to the ERF and to a more versatile and diversified ERF, has resulted in two donor actions, the first a Nordic nation making a first ever contribution and the second, another Nordic nation renewing its pledge. The cumulative effect has been to provide the ERF, through the balance of 2012 and into 2013, with the funds it requires to demonstrate the new strategic approach now being implemented.

Other donors who were engaged by this evaluation indicated that should the ERF be restarted with the general directions being contemplated, there would be a strong likelihood for donor support. This possibility of increased support is not only due to the strategic commitment of current OCHA management to renewal, but also the geopolitical evolution in Afghanistan with the impending withdrawal of foreign military forces in 2014.

Finding 3. The Fund in Afghanistan is not well understood.

The above Finding points to a significant challenge that faces the ERF in Afghanistan. NGO, Government and donor stakeholders alike are not clear about the direction of the Fund. This has led the ERF to be poorly situated among the range of humanitarian instruments. This problem is in large part due to the lack of resources for the Fund itself and the generally weak cluster environment in Afghanistan, a condition which even UN stakeholders reluctantly admit.

OPERATIONAL EFFECTS This particular set of questions as laid out in the approved Evaluation Matrix had a tendency to duplicate other issues. For the sake of brevity and clarity, these duplications which related to the nature of the projects and whether they generated benefits and synergies, have been addressed earlier.

Relevance

Finding 4. The ERF projects supported since 2009 have been narrowly relevant in the area of response to natural disaster

This Finding amplifies that above related to the overall operational impact of the ERF in Afghanistan. Although the mechanism as a whole has made only a minor impact, it is important to stress that for the beneficiaries in question, those affected by weather conditions and flooding largely in mountainous regions, the projects that the ERF has supported are indeed relevant to their immediate needs.

The size of the Fund and the growing pattern of it being severely challenged for resources probably resulted in at least implicit decision-making to concentrate in areas where relatively small amounts of funding could have some operational effect - relevance. While this strategy was probably well intentioned, it inadvertently led to perceptions of the Fund not being more broadly relevant to what were perceived to be emergency situations.

Finding 5. The limited cluster engagement weakens the potential effect of projects (lack of synergies)

It is important to note that, the crucial advisory board mechanism did not exist in the Afghanistan context prior to November 2011. Cluster partners engaged in this process very much indicated that prior cluster coordination with respect to the ERF was uneven and that the concentration of ERF proposals in a narrow geographic scope and largely with respect to one theme reduced the viability of cluster coordination in the first instance.

This lack of cluster coordination until very recently probably had the consequence of reducing the effect of ERF-supported projects and reducing the level of their overall engagement.


8

Timeliness

Finding 6. In comparison to some other local sources, the ERF is seen as responsive to NGO needs (esp. NNGOs)

The issue of the timeliness of the ERF mechanism has been a worldwide concern. In the context of prior Afghanistan projects and given the extremely conservative and risk averse nature of these projects, the amount of time needed to undertake prior assessment was not a significant factor, given that the projects in question were being proposed well after the crisis had occurred. The actual processing time for the ERF in Afghanistan approximates those of others, about four to five weeks, with some projects being approved more rapidly than others. NGO stakeholders, both NNGO and INGO, saw the ERF as a relatively inviting mechanism and one which was able to respond to their needs more rapidly than other sources. In this context, it is important to underscore that some NNGOs look to INGOs as an active source of funding, as they have neither the capacity nor track record to warrant direct access to many bilateral bodies.

However, should the ERF adopt a more aggressive and risk tolerant approach and especially expand its scope of activities to include a broader geographic focus, the question of ERF timeliness may become more germane.

OUTCOMES

Leadership and Coordination

Finding 7. Information sharing is limited in terms of follow-up and lessons learned processes led by OCHA.

Until October 2012, the ERF in Afghanistan had not undertaken any real coordination or outreach activities. While fact sheets had been prepared and of course the Annual Reports submitted, level of coordination on the ground in Afghanistan, was nominal. It bears repeating that for the first two years of operations, an Advisory Board was not in place, thus in many ways critically limiting the ERF process. As well, the assumed consent model for the Review Board, while applicable within the context of Guidelines, probably was less than fully inclusive. Furthermore, the opportunities to engage in collective dialogue appear to have not been recognized, let alone realized.

The current situation however, is rapidly evolving toward a highly coordinated and outreach-based ERF process. The addition of a Deputy Head of Office for pooled funds and coordination demonstrates the degree to which new management recognizes the importance outreach and communications. This more senior person can undertake the time-consuming but vital work of building new partnerships, broadening liaison and more directly engaging donors, all activities that were only scantly addressed in the past. In this context, the ERF Manager, the sole employee dedicated to the ERF process per se, can assume responsibilities more related to the strengthening of the granting processes of the ERF.

Critical Gaps Filled

Finding 8. It is not certain whether the most pressing needs have been addressed by the HRF

The distribution of projects by sectors since 2009 shows a high degree of concentration on shelter and WASH. Stakeholders also pointed out that many of these initiatives were launched many months after the natural occurrence. More importantly, few HRF supported projects seem to focus on addressing the humanitarian challenges presented by the on-going insurgency, or were geographically sited to locales where the insurgency has been more active.

One case in particular is striking. Prior HRF management started clearly that they had resisted supporting a health clinic and outreach activity designed to address the medical needs of persons in high risk zones. Eventually the project was approved. Stakeholders identified it as one of the most innovative that the HRF had ever supported and identified it as a “way forward”. More generally cluster stakeholders were nearly unanimous in their view that the HRF, until very recently had taken a very conservative and cautious approach to needs assessment.

PARTNERSHIP Finding 9. OCHA in Afghanistan is not seen as a strong partner in terms of the HRF function

Government stakeholders appeared to be supportive of the HRF as a partner in the context of it being a relatively easy to access fund, although a very small one.

The level of concern about the quality of the HRF in terms of partnership tends to be most pronounced with UN partners and some donors. Lack of clarity, narrow focus and an absence of communications with respect to the HRF were identified as factors limiting the quality of relationships.


9

PROCESSES AND INPUTS

Review of Proposals

Finding 10. The existing quality control mechanisms are weak

This Finding speaks to the systemic issue of the weakness of the ERF mechanism in the past, combined with the difficulties in UN system coordination in Afghanistan. The ERF process, aside from procedural quality control and audit, relies on consultation and information sharing to assess qualitative importance of projects. The ERF process in Afghanistan, until very recently, did not have the means to do so.

As well, given the inherent complexity of the humanitarian environment in Afghanistan, procedural compliance was also made more problematic. In the first instance, monitoring has been uneven. More recently however, OCHA senior management in Afghanistan are committed to using OCHA field office staff as ERF monitors, thereby potentially increasing the effectiveness of this process. Second, the prior approach to contracting audits individually led to exceptionally long delays in and thus the release of the final 20% payment. This issue is not so much related to the payment as it is related to the use of the audit as an after-project quality control mechanism. More recently, the appointment of a regional audit firm on a standing offer basis has made considerable improvement in audit times, probably making these audits more relevant as tools to assess the capacity of NGOs with respect to subsequent proposals.

Finding 11. OCHA staff levels have been minimal (one full time person).

From its commencement in 2009 until October 2012, the ERF in Afghanistan had only one full time employee dedicated to it. Virtually all stakeholders agreed that, given the complexity of the humanitarian environment in Afghanistan, such levels of staffing were inadequate from the outset and probably constituted a contributing factor to the difficulties which the ERF faced from its very beginning.

The degree of managerial uncertainty over this period of time, as is evidenced by the rapid rotation of Heads of Office, probably contributed to this situation. However, in the last year, the situation is beginning to turn around. The arrival in October 2012 of Deputy Head of Office responsible for coordination in general and pooled funds in particular has led to the ERF mechanism now having a relatively senior manager responsible for its strategic direction.

FUNDING AND PROJECT ISSUES

Mainstreaming Gender

Finding 12. Gender and other disadvantaged groups have not received a high degree of consideration.

In Afghanistan, as is the case everywhere else, the Gender Marker is now a required instrument. However, in Afghanistan, the overall gender ratings are lower than most other ERFs. The nature of the projects selected, largely shelter and land restoration, do not easily lend themselves to the direct consideration of their impact on disadvantaged groups as a whole.

In discussion with cluster representatives, the difficulties of implementing gender sensitive or pro-disadvantaged persons programming in Afghanistan were highlighted. However, these same stakeholders noted that there were lessons being learned with respect to effective mechanisms to raise gender sensitivity and improve the effectiveness of humanitarian programming for disadvantaged persons in general. The ERF mechanism does not appear to have taken advantage of these networks. In fact, it could be argued that the prior approach was largely passive.

Operation of Advisory and Review Boards

Finding 13. Advisory and review bodies are not engaged thus limiting responsiveness and outreach.

As noted earlier, the Advisory Board for the ERF in Afghanistan was only established in November 2011. The existing Review Board mechanism utilizes the assumed consent model. Among stakeholders there are serious concerns as to whether the Review Board in fact exists or whether it is really only one or two committed individuals. The lack of resources available to administer the ERF in Afghanistan probably has led to lost opportunities to create synergies and partnerships that are necessary.

These gaps are more than simply procedural irritants. The establishment of the ERF in Afghanistan in 2009 does not seem to have been accompanied by a sufficiently sensitive needs assessment which might have indicated that the conditions in Afghanistan required the expenditure of resources from the outset to better engage elements like these Boards, clusters and NGOs.


10

MONITORING AND EVALUATION Finding 14. Project duration and level of resources (within project and with OCHA in Afghanistan) impede

effective monitoring and evaluation.

Many donors worldwide have been vocal in seeking from ERFs additional performance information related to the impact of programming on beneficiaries. Globally, this presents a systemic challenge to OCHA. In Afghanistan, given the overall size of the Fund and the very small size of grants, the challenge to go beyond simple output reporting on an individual project basis may be insurmountable from a resource consideration and either irrelevant or cost prohibitive from methodological considerations.

Finding 15. No efforts at sectorial review (review of shelter as a whole, WASH, etc.) as other ERFs have done.

The resource levels of the ERF in Afghanistan appear to have made it impossible for it to have considered attempting to conduct cross-project evaluations. For example, the number of similar projects that were funded in relation to shelter would have lent themselves to a sectorial review and thus would have provided an opportunity for a more qualitative assessment of performance along with opportunities for improving capacity and the sharing of relevant information. In short, due to the apparent absence of funding, an important opportunity for learning was not undertaken.


11

Country Level Recommendations These recommendations are directed at the country level management of the ERF. Global recommendations, partly drawing on the conclusions of this case study, will be presented through the Final Evaluation Report.

With the increasing standardization of the ERF process the opportunity for innovation at the country level is becoming constrained. Therefore this case study presents only a limited number of interim recommendations that ultimately should be read in conjunction with the final evaluation report.

As noted earlier, the recommendations that follow are in large part designed to emphasize some prior lessons learned so as to complement the now underway “Restart”.

Recommendation 1. Announce a complete restart of the ERF in Afghanistan

This first Recommendation is in effect a reprise of decisions taken during the final months of 2012 that have been alluded to earlier. The ERF, as it was constituted from its inception until the early months of 2012, was beset with structural issues as well as resource limitations that resulted in it having a very limited impact.

However, notwithstanding the only limited success of the ERF in the first two years of its operation, the concept of pooled funding at the country level remains acceptable to major donors as is witnessed by the commitments made in the final months of 2012, after some “restart” measures had been introduced.

It is also important to stress that from the perspective of building support in the period leading up to 2014, it is very important for the ERF and for its managers and staff to clearly differentiate the future from the past, so as to neutralize to the greatest extent possible, prior criticisms.

Recommendation 2. Clarify the focus of the ERF in Afghanistan

The issue of the tug and pull between a classic response to an emergency and the evolving notion of resilience building along with the promotion of preparedness did not emerge in Afghanistan to the degree it did in other ERFs. Nevertheless, it is likely to do so as the ERF in Afghanistan secures more support. However, this issue can become very problematic and require sensitive management.

Given that the Advisory Board was established less than a year ago and that the Review Board has never been used in any context other than the assumed consent model, the new management of the ERF and the OCHA Office in general have an opportunity to directly engage key stakeholders about the focus of the ERF. The ambiguity that has beset other ERFs, and the process as a whole worldwide, needs to be clarified in the Afghanistan context so as to build solidarity and so as to actively promote the ERF as a mechanism that can foster synergies and in the 2014 and beyond period, catalyze resources. In addition, from a more tactical standpoint, engaging major stakeholders in a discussion about the ERF in Afghanistan will visibly demonstrate a commitment to a more collaborative and consensus-based approach.

This recommendation also speaks to issues about geographic focus and subject matter focus. The prior highly conservative approach to risk management and project selection has led to perceptions that the ERF is not fully engaged across Afghanistan and engaged in responding to the most needed humanitarian challenges. This is not to say that the ERF should stray into the uncertain waters surrounding early recovery activities. It is to emphasize that, for the ERF in Afghanistan to be successful, it will likely have to considerably broaden its outreach from the very outset by approving projects that visibly are “different” from those in the past.

Recommendation 3. Recruit more donors

This Recommendation follows on information secured during the course of the Afghanistan country case study. One new donor was identified, whose immediate pledge was sufficient to warrant OCHA management beginning the process of restart. Subsequently, a second donor has been identified, in this instance, a prior donor making a recommitment. During the course of the mission, references were made to other donors, some of whom had not been prior donors in Afghanistan and seldom made contributions to the pooled funds at the country level.

The ERF mechanism, now somewhat augmented as to its resource levels, would gain significantly if it were to undertake very active outreach to donors, emphasizing the benefits of using the ERF mechanism in the run up to 2014 as a trial vehicle for the benefits of pooled funding in Afghanistan in general, up to and possibly including the establishment of a CHF, if sufficient critical mass is secured.

However, it is important to underscore that, while the longer range goal with respect to humanitarian coordination may involve consideration of a CHF, the shorter term objective for the ERF is to re-establish its credibility and to rebuild a donor pool, increasing it in a step by step way where positive results are used in an incremental fashion to build new support, one donor at a time. This clearly implies a resource commitment on the part of OCHA in


12

Afghanistan and a commitment by the newly arrived HC to champion the ERF mechanism. More directly engage the Advisory and Review Boards

This Recommendation again mirrors ongoing developments that commenced in the second half of 2012. The Advisory Board now exists but there is in fact another gap that could be filled and which might strengthen the overall architecture of cooperation and collaboration in Afghanistan.

While the Review Board in the past has been used as a passive mechanism due to the usage of the assumed consent model, engaging it more directly as part of the revitalization of the ERF in Afghanistan probably would have considerable benefits. The Advisory Board can “work from top to bottom” better linking the ERF with other processes of coordination. By contrast, a more actively engaged Review Board can work from bottom to top assisting the now augmented staff at OCHA with more technical level advice about projects and priorities. This does not mean that the Review Board would be called upon to meet weekly or even monthly. Rather, it is to suggest that the Review Board be convened three or four times a year to advise OCHA with respect to technical issues and project level priorities.

Recommendation 4. Engage in more active information sharing with clusters and among NNGO/INGO recipients.

The prior Recommendation spoke to unique characteristics of the ERF process. This recommendation addresses how the ERF mechanism needs to be better positioned among cluster partners and with NGOs more generally in Afghanistan.

With respect to clusters, it is recommended that OCHA staff take an active role in engaging cluster leads and clusters themselves in promoting the ERF as being more accessible to a broader range of project areas.

In relation to NGOs in general, two areas of strengthening need to be undertaken. First, the ERF needs to proactively engage NGOs, both national and international, to the renewal of the ERF and the fact that the ERF intends to give priority to new types of programming in new geographic locales. Put simply, NGOs who may have been prior partners need to be aware that the ERF is evolving toward a new paradigm. Second, and in line with Findings about information sharing, the ERF should undertake several feedback/lessons workshops which would bring ERF recipient organizations together so as to broaden information sharing and hopefully begin to build an “ERF in Afghanistan” network.

Recommendation 5. Strengthen OCHA internal capacity for planning, quality control and M&E.

The recent augmentation of the overall level of staff at OCHA in Afghanistan points the way to the recognition that in circumstances such as these, the minimal approach to staffing and resources which is practiced by a number of ERFs was simply insufficient for the challenges inherent in Afghanistan. While it is not the purpose of this country case study report to make specific administrative recommendations, it is necessary to underscore that, the for restart process to be effective, several actions need to be seen as priorities.

First, sufficient resources for liaison and coordination must be available. Second, OCHA resources in field offices need to be called upon to provide monitoring or advice to project recipient organizations and to give a face to the ERF in the field. Third, sufficient resources need to be secured either from OCHA or by a process of coordination with cluster partners to mount additional performance reporting exercises, for example, sectorial review in areas like shelter where there has been a large number of projects.


13

Appendix I: Programme Agenda 14 – 18 OCTOBER 2012

Day Date Time Activity Location Participants

SUNDAY 14 OCT

10:00 hrs Arrival in Kabul – Team Member and transfer to accommodation facility – Ms. Tijana Bojanic

Kabul International Airport Driver –

Security – Ahmad Rashid Yaqubi

15:15 hrs Arrival in Kabul – Team Member and transfer to accommodation facility – Mr. Dale Thompson

Kabul International Airport Driver –

Security – Ahmad Rashid Yaqubi

17:00 hrs Security Briefing UNDSS Joint Operations Centre, UNOCA

Security - Ahmad Rashid Yaqubi

19:30 hrs Briefing with HoO, DHoO and ERF Manager

UNOCA Aidan O’Leary, Arnhild Spence and Sayed Farid

20.00 hrs Dinner Serena

MONDAY 15 OCT

8:00 hrs Transfer from Serena to Qalai Fathullah Office

OCHA Dispatch

Sayed Farid

9:00 hrs Group Meeting/interview with donors

OCHA Qalai Fatullah Aidan, Dominic and Arnhild

10:15 hrs Meeting with Government emergency response entity/ANDMA

OCHA Qalai Fatullah Shams Khalili and Sayed Farid

11:15 hrs Meeting with Afghan Red Crescent Society (ARCS)

OCHA Qalai Fatullah Shams and Sayed Farid

12:30 – 13:30 hrs

Lunch Break

13:30 hrs Group Meeting/Interview with Review Board members – RB meeting

OCHA Qalai Fatullah Arnhild Spence, Dominic Parker, Megbaru, Sayed Farid and Gift Chatora

16:30 hrs Briefing with Humanitarian Coordinator

OCHA Qalai Fatullah Michael Keating, Aidan O’Leary

16:30 hrs Transfer to Serena OCHA Dispatch

19:00 hrs Dinner Serena

TUESDAY 16 OCT

8:00 hrs Transfer from Serena to OCHA Qalai Fathullah

OCHA Dispatch

Sayed Farid

9:00 hrs Groups/interviews with INGO IPs

OCHA Qalai Fatullah Sayed Farid, Megbaru and Malikzai

10:00 hrs Groups/interviews with NNGO IPs

OCHA Qalai Fatullah Farid Sayed Farid and Dr. Naseer

12:00 – 13:00 hrs

Lunch Break

13:00 hrs Follow-up individual interviews with INGOs and NNGOs

OCHA Qalai Fatullah Farid and Malikzai


19.00 hrs Dinner Serena

WEDNESDAY 17 OCT


OCHA Dispatch

Sayed Farid

9:00 – 10:00 hrs

Meeting with OCHA Cluster Coordination Team

OCHA Qalai Fatullah Dominic, Jessica, Sarah and George

12:00 – 13:00 hrs

Lunch Break


14

Day Date Time Activity Location Participants

14:00 – 15:30 hrs

Inter Cluster Coordination Monthly Meeting – Session of ERF

OCHA Qalai Fatullah Jessica/ Sarah/George and Sayed Farid


19:00 hrs Dinner Serena

THURSDAY 18 OCT


OCHA Dispatch

Sayed Farid

8:30 hrs Debriefing OCHA Office, Qala-e-Fatullah Aidan, Arnhild, Dominic and Farid

10:00 hr Transfer to the Airport and Departure

Airport Driver, OCHA Dispatch


15

Appendix II: Stakeholders • Achaya, S • WHO

• Aman, A • AREP

• Aman, J • NPO

• Ameriahulah • KSRO

• Askew, C • UNHCR

• Barrow, M • OCHA

• Bowers, J • OCHA

• Bubic, T, • IOM

• Chidambaram, C • ACTED

• Conlan, E, • WFP

• Ellis, R • UNHCR

• Esmahillah, S • NARDOA

• Farid, S F • OCHA

• Frigen F • Emergency

• Habibullah, M • NRC

• Hamra, F • Award

• Harrison-Prentice, T • PIN

• Jafri, G J • CIDA Canada

• Katar, M D • Ministry of Finance, ANDMA

• Keating, M • H/C

• Lais, M • Ministry of Finance, ANDMA

• Marjoire • Tear Fund

• Martin, F • Intersos

• O’Leary, A • OCHA

• Omarz, B • ACRO

• Parker, D • OCHA

• Petropolis, G • OCHA

• Radaeu, L • Emergency

• Rafiq, S, • NGO, Protection Cluster

• Salam, A • SFRD

• Salam, A • SHARQ

• Salim, M • UNDP

• Shahrukhi • SRP

• Spence, A • OCHA

• Tariq, A • Afghan Aid

• Tawab, S A • ADEO

• Tawab, S • ADEC

• Vijselaar, L • DACAAR

• Virdee, G • OCHA


The Global Evaluation of Emergency Response Funds (ERFs) Country Case Study Report - Colombia

March 2013


i





ii


Disclaimer .................................................................................................................................................................... i




Introduction ................................................................................................................................................................. 1


Methodology ............................................................................................................................................................... 2

Overview .................................................................................................................................................................... 2

Case Study Methodology ........................................................................................................................................... 2

Local Limitations/Challenges ...................................................................................................................................... 2

The ERF in Colombia ................................................................................................................................................. 3

Findings ...................................................................................................................................................................... 4



Outcomes ................................................................................................................................................................... 6

Partnership ................................................................................................................................................................. 7



Monitoring, Reporting and Audit ............................................................................................................................... 10

Overall Conclusions .................................................................................................................................................. 11





iii

Abbreviations AAH Action Against Hunger

AECID Agencia de Cooperación Internacional (Spain)

APC Agencia Colombiana para la Cooperación internacional

CAP Consolidated Appeal

CDPMM Corporación Desarrollo y Paz del Magdalena Medio

CHF Common Humanitarian Framework

COCOMACIA Consejo Humanitario Mayor de la Asociación Campesina Integral del Atrato

CRC Cruz Roja Colombiana

EHL Equipo Humanitaria Local

ELN Ejercito de Liberación Nacional


FARC Fuerzas Armadas Revolucionarias de Colombia

FSC Funding Coordination Section

GoC Government of Colombia

HAO Humanitarian Affairs Officer




ICRC International Committee of the Red Cross

IFRC International Federation of the Red Cross

IHL International Humanitarian Law

IM Information Management


IOM International Organization for Migrations

LHT Local Humanitarian Teams



NSDPR National System for Disaster Prevention and Response

OCHA Office of Coordination of Humanitarian Affairs

PAHO Pan American Health Organization

RC Resident Coordinator

RET Refugee Education Trust

ROLAC Regional Office for Latin America and the Caribbean

RR Rapid Response

SCF Save the Children Fund

SIDA Swedish International Development Agency


UF Underfunded


iv

UN United Nations

UNCT United Nations Country Team

UNDP United Nations Development Program

UNFPA United Nations Fund Population Fund

UNHCR United Nations High Commissioner for Refugees


UNETT United Nations Emergency Technical Team

WASH Water, Sanitation and Hygiene

WHO World Health Organization

WFP World Food Program


1

Introduction This is one of five Country Case Study Reports that are mandated as contributing part of the Global Evaluation of ERFs, which is taking place in the latter half of 2012. This case study report reflects the findings arising from a 5-7 day mission to Colombia. Details of the mission can be found in Appendices I and II.

This Country Case Study Report is firmly focused on conditions in Colombia. While it is obvious that global conditions and the overall management system of the ERFs impacts on the delivery of services at country level, this report will not directly address these global issues. Rather, how these global considerations affect the performance of the ERFs as a whole will figure largely in the overall report of this evaluation.

Universalia wishes to thank the management and staff of OCHA Colombia, that of its UN systems partners, donor representatives, government stakeholders and the representatives of the NNGOs and INGOs who have generously given their time and opinions. Without them, this evaluation as a whole would not be possible.










• Internal (e.g. staffing capacity, management arrangements) and external (e.g. contextual and political) factors which affect ability of ERFs to deliver on their objectives;





.


2

Methodology OVERVIEW The methodology for the global evaluation consists of a mixture of qualitative and quantitative data collection, with attention both to forward-looking, improvement oriented questions and to assessment of impact. This follows the OCHA Evaluation Policy objective of determining:




CASE STUDY METHODOLOGY The findings of this country report are based on a visit to Colombia and a complementary documentary review of documents related to the Colombian humanitarian, security and political context. As per the TOR, the primary focus of the discussions was ERF funding initiatives in Colombia during its two and a half years of existence.

The Evaluation Mission that took place in Colombia from the 1st to the 8th of October was conducted by two staff members. . The Universalia evaluation mission consisted of a series of interviews with those involved in ERF processes in 2012, including OCHA staff, officials of the Government of Colombia, and international and national NGOs. Interviews with stakeholders were semi-structured and based on interview protocols designed prior to the field missions. In order to see firsthand the implementation of one of the most important ERF funded initiatives, the team carried out a two-day mission to Quibdó, the capital of the Department of Chocó, an area affected by a mix of humanitarian natural disasters taking place in a particularly insecure conflict environment. The full program is presented in Appendix I; a full list of people interviewed is presented in Appendix II.

LOCAL LIMITATIONS/CHALLENGES The evaluation team faced limited challenges in collecting the information required for its analysis. The evaluation team had the possibility to meet directly with two of the three main donors of the ERF, Spain and Sweden. Norway closed its embassy in Colombia some years ago and the officer in charge of following the ERF could not be reached by telephone as anticipated. The evaluation team could also meet directly with several international NGOs receiving funds from the ERF, with the exception of the French Red Cross (which has no representation in Colombia). The situation was a bit more difficult with National NGOs as many of them have local representation on the field. The missions could meet with half of the NNGOs who had already implemented projects with ERF funding.

Unfortunately, meetings with the new Director and sub-director of OCHA office in Colombia were limited because of their internal and international agenda. The absence of the Humanitarian Coordinator (HC) during the presence of the evaluation mission prevented collecting the views of the senior humanitarian representative in country. The presentation of the main findings and recommendations of the evaluation team took place a few days after the conclusion of the field mission through a telephone conference with OCHA Colombia’s Deputy Director and the ERF manager.

.


3

The ERF in Colombia Despite recent peace negotiations between the Government of Colombia (GoC) and the Revolutionary Armed Forces of Colombia (FARC) guerrillas, Colombia remains adversely affected by a 50 year legacy of internal armed conflict. Moreover, recent natural disasters have further exacerbated an already tenuous situation and while ranked a mid-income country, profound inequalities and structural voids persist in several regions. The tragic outcome of these combined factors is that in a country populated by 44 million people, ten percent are internally displaced (IDPs).

Put succinctly, natural disasters combined with the precarious internal dynamics of the country present unique challenges in terms of mobilizing humanitarian relief and rapid response to disaster areas. Throughout these recent events, the GoC has made significant efforts to mobilize their own resources and address disaster response and reconstruction needs. However, while the GoC made sizable national budget allocations (US$2.7 billion for emergency assistance in 2010) only US$0.6 billion (less than 10 per cent of total planned assistance) was used for short term humanitarian assistance. Also, while the GoC requested international support from individual countries and UN agencies, it categorically refused using a Flash Appeal or Consolidated Appeal (CAP) as instruments for resource deployment.

It is within this atmosphere that the Colombian ERF was established in 2009 with the mandate of providing up to 6 months of humanitarian support to crisis situations requiring rapid emergency response. The Fund was launched with contributions from Norway, Sweden and Spain totalling US$2.4 million. The first ERF-supported humanitarian response occurred in February of 2010 and was implemented by the French Red Cross following severe flooding in the area of Guapi, in the department of Cauca.

Since its inception in the summer of 2009, the ERF in Colombia has supported 18 humanitarian rapid response initiatives and disbursed more than US$3.3 million in 2010 and 2011 (with US$750,000 pending disbursement). At the time of the evaluation, an additional US$1.4 million was to be disbursed in 2012, with possible new proposals in the pipeline. Some 14 organizations, 8 INGOs and 6 NNGOs have received funding to implement rapid humanitarian response in a wide range of municipios and departments of Colombia (Antioquia, Atlántico, Bolivar, Caquetá, Cauca, Cesar, Chocó, Cordoba, Magdalena, Santander, Sucre, and Valle del Cauca).

The current humanitarian architecture of the UN system in Colombia is modelled on the Humanitarian Reform coordination structure. It includes seven clusters, which coordinate the work of some 12 UN agencies and 35 national and international NGOs. While OCHA has been present in Colombia since 2001, it has never been in a position to develop a Consolidated Appeal (CAP). Moreover, in the context of Colombia, whereby the GoC is the main provider of humanitarian assistance, government-led emergency relief efforts to some areas have been hindered by the ongoing internal conflict. In some areas, Government assistance/presence is limited or non-existent. Thus, the flexibility of the ERF funding mandate extends humanitarian coverage to areas that would otherwise not receive any governmental relief. In doing so, Colombian ERF and its stakeholders are filling a very critical gap.

Coverage of ERF Projects in 2011 (OCHA)


4

Findings The TOR for the assignment laid out a number of questions to be answered by the evaluation. These relate to the ERF Results Framework. The evaluation team prepared an evaluation matrix to amplify each key question by articulating sub questions, illustrative indicators, and data sources (see Appendix III).

OPERATIONAL IMPACT Finding 16. The Colombian ERF provides relative advantage in small-scale disaster relief and response

efforts undertaken by recipient organizations.

The operational impact of the ERF mechanism in Colombia relies on three main factors that highlight the value-added attributes of the mechanism:

1. Complementarity with existing mechanisms: This can occur at the level of local humanitarian response efforts; other UN agencies in the management of complex humanitarian crises; bilateral programming in specific areas where humanitarian initiatives are complemented by bilateral funding commitments; CERF supported projects; or, at the sectoral level with other Clusters/actors when operational humanitarian initiatives lead to other mainstreamed concerns in regard to protection of children or other vulnerable groups;

2. Flexibility of the humanitarian response: Where possible, flexibility is applied in the conception, revision or “adaptation” of operational concerns at the field level; in the adoption of holistic and integrated early recovery approaches and initiatives as a necessary follow up to humanitarian actions; in scheduling, management and operations at the field level when the local environment becomes an adverse factor to implementation;

3. Geographic or thematic “gap filling” in the humanitarian landscape of Colombia: The third key principle animating the ERF since its inception is the idea of “filling gaps” where state institutions do not or cannot have access to populations in need. ERF in some instances enabled interventions that restored access to indigenous communities permitting subsequent humanitarian relief efforts. By working with important national actors or organizations such as the Pastoral Social and potentially investing in their emergency rapid response mechanisms, ERF ensures greater access to zones and areas where there is no presence at all of state or non-governmental actors.

Finding 17. OCHA/ERF initiatives have contributed to capacity building and collaboration between recipient organizations.

International organizations supported by ERF do not need particular support in regard to expertise or financial/administrative functions. At the opposite end of the spectrum, local organizations are divided in two groups. The first group has a large, national and relevant experience (e.g. Pastoral Social, Cruz Roja Colombiana, Opción Legal) and barely needs additional support for the implementation of rapid response interventions. The second group is composed of local organizations supported by ERF funding. These local organizations have benefited from support, expertise, and training provided by larger national and international organizations. This collaboration has increased their capacity to solve problems, identify gaps and priorities, to train local implementers and better articulate their ERF proposals. The CDPMM has developed its intervention in Magdalena through UNFPA promotional and technical support to vulnerable women. COCOMACIA has stated it has benefited greatly from PAHO expertise in training and supporting local community health workers and midwives in the nine Afro-descendant areas of concentration of the ERF health funded project. Last but not least, some OCHA sub-offices and antennas also play a similar key capacity building and coordination role in support of monitoring, programmatic design and implementation.

OPERATIONAL EFFECTS Finding 18. ERF mechanisms and funding have strengthened two cooperative tools: the ERF-CERF working

complementarity and the ERF-UN Clusters relation.

ERF in Colombia is one of the tools that supports and strengthens the institutional and operational cooperation between OCHA and the current clusters in Colombia. To some extent, the ERF could be viewed as a valuable funding option for UN agencies, which are officially eligible for ERF funding1, although this opportunity remains unexplored. In Colombia, the ERF has, on many occasions, played a complementary role to the CERF.2 In 2010,

1 ERF Colombia – Lineamientos generales, p. 2 2 CERF Resources: “In 2010, Colombia received a total of $6,607,366 from the CERF. The UFE provided $2,966,946 at the very start of the year and $3,640,647 from the RR widow”, See Footnote 14.


5

the three last projects funded by ERF were articulated as CERF Rapid Response (RR) projects, as was a 2011 Water and Sanitation project implemented by AAH that complemented a CERF Underfunded Emergency (UF).3 This move towards a more substantial articulation between the two funding mechanisms enabled the Humanitarian Country Team (HCT) to cover close to 11% of emergency response financial requirements in 2011.4 Indeed, the CERF and the ERF share common principles and criteria for interventions; in the Colombian experience this was largely focused in double affected areas ravaged by conflict and flooding.5

After working together for two years, ERF and CERF are progressively building upon this relationship. It has been suggested that further institutionalization of joint needs assessments and resource allocations could also be explored in this context.6 For example, more recent needs assessments and rapid responses in the Department of Putumayo (border with Ecuador), whereby INGOs and NNGOs are coordinating their efforts in response to the latest floods, partly with ERF support, will possibly lead to the financing of a CERF RR that will complement and strengthen current inter-institutional humanitarian efforts.

Finding 19. Health and WASH sectors remain the two main pillars of ERF thematic humanitarian interventions in Colombia.

It may be too early to evaluate the thematic and programmatic evolution of ERF Colombia. The consolidation of 2010-2011 data shows an increased investment in value of health projects from one year to the next. WASH interventions constitute the second most important humanitarian response investment in Colombia. The increased concern about early recovery on the part of donors and recipient organizations can be seen in 2011, with 17% of total ERF investment, while funding for Shelter was clearly a response to the 2010 “Ola Invernal” (Rainy Season) and is not foreseen to become a trend. Exhibit 3 Representation of Thematic Sector Interventions from ERF Colombia (2010/2011)

No of projects in 2010

Funding (US$) % No of projects in 2011

Funding (US$) %

HEALTH 2 452,720 25 3 651,773 42

WASH 2 496,111 27 2 411,988 26

SHELTER 6 309,528 17 0 -- 0

EDUCATION 2 333,935 18 1 238,842 15

RECOVERY 0 -- 0 2 259,563 17

FOOD/SECURITY 1 230,108 13 0 -- 0

TOTAL 13 1,822,402 8 1,562,166

Finding 20. Relevance and efficacy of ERF processes is weakened by the continued absence of a clear strategic vision.

Many stakeholders who were interviewed observed that what is missing from the ERF processes in Colombia is a more strategic vision, namely a way for the ERF to rank its priorities based upon thematic, sectoral, geographic, or political considerations. For instance, informants offered suggestions regarding how the Fund should respond to highly political and sensitive humanitarian needs when the Colombian State may not be in a position to respond. A new strategic direction would involve making decisions on specific intervention focuses and avoid responding to all types of requests that are not interconnected. A more strategic intervention would not necessarily reject the fundamental criteria of “double affectation” but could open new avenues and new types of responses to humanitarian man-made crises in a more sensitive and political context.

Finding 21. The ERF has demonstrated operational flexibility in regions where the GoC has little to no access.

The ERF provides relevant and necessary funding to INGOs and NNGOs for rapid response initiatives to areas that would not otherwise receive it.

However, representatives of the GoC suggested certain changes to ERF implementation in order to improve its reach. For instance, informants stated that the ERF should consider revising the selection criteria of “double

3 Emergency Response Fund, Annual Report 2010, p.3. 4 Emergency Response Fund, Annual Report 2011, p.12. 5 Independent Review of the Value Added of the Central Emergency Response Fund (CERF) in Colombia, Final Report, Glyn Taylor, Humanitarian Outcomes, July 2011, p.7 6 Ibdem, p. 25


6

affectation” in order to expand the coverage and flexibility of its current programming. It was also suggested that the US$50,000 floor for ERF projects be changed to become more accessible to smaller NGOs.

Finding 22. Lengthy response time detracts from the effectiveness of the ERF.

Most informants who met with the evaluation team found that the application process took between 2 to 3 months, and even longer in certain cases. The consensus observed that the duration between initiation and disbursement was too long given the nature of the emergencies for which ERF was sought. However, they also concluded that the process was faster than other financing mechanisms.

The question of timeliness will be revisited again in a later section (Finding 35) as it seems to be a persistent obstacle especially in regards to what appears to be a fairly inflexible and cumbersome administrative process.

OUTCOMES Finding 23. Increasingly, ERF is being called upon to balance its primary objectives with those of

emergency preparedness and early recovery initiatives.

Emergency preparedness is not a systematic component of local humanitarian disaster response supported by ERF Colombia. However recent innovative approaches to disaster preparedness and early response demonstrate how these two critical gaps can be filled through a flexible small-scale funding mechanism like the ERF. To this end, the Advisory Board of the ERF invited the HC and OCHA to determine a certain percentage of the fund that should be dedicated to emergency preparedness.7

Humanitarian response in Colombia tries whenever possible to integrate local communities in early recovery activities, although many of them are voluntarily giving their time and energy to realize the initiatives undertaken on their behalf. COCOMACIA claims that local communities provided 50% of the work that was required in supporting local logistics and material in a very difficult geographic environment. Solidaridad Internacional distributed seeds and tools in order to restart agricultural activities. At its biannual meetings, the Advisory Board has reiterated the importance of better articulating rapid response with early recovery initiatives that would support mid-term recovery of affected communities.

In response and for Colombia in particular, the Colombia Advisory Board has agreed to the principle of supporting up to 10% of each proposal for Disaster Preparedness and Reduction.

Finding 24. ERF Colombia implementation capacity is stretched to fully manage ERF projects throughout the entire project cycle.

The implementation and management of ERF projects rely on the ERF manager and OCHA’s sub-offices and antennas.

The ERF manager receives support from a series of internal and external stakeholders throughout the design and the revision processes, as well as the implementation, and monitoring, of each ERF funded project. The ERF manager also plays a secondary capacity building role in support of national and local agencies which present proposals to OCHA. Small NNGOs need more technical assistance than large INGOs. Project formulation with smaller NGOs requires more time and resource investment in order to help articulate conceptual and technical tools, formats and presentations that support their response plan.

The OCHA sub-offices and antennas play a key role in identifying possible humanitarian interventions that may require the support that neither the GoC nor other national or international agencies have responded to. The role of OCHA’s sub-offices often goes beyond the needs assessment and identification process and often continues with particular support in terms of proposal design and conception. The RET recognized the key role played by the antenna of OCHA in Cordoba in supporting the coordination of national and local actors in the humanitarian response implemented by the “Education Board” of the Department of Atlántico.

The OCHA sub-offices and antennas also aid in managing the coordination, information sharing and needs assessments of the Local Humanitarian Teams (LHT), which regroup representatives from UN agencies, local NGOs and other national representatives of Colombia’s humanitarian response. In some instances, OCHA officers are sharing their offices with other UN agencies, which facilitate, to some extent, the coordination of “pooled” activities. It also prevents potential overlapping of humanitarian initiatives and ensures consistency and relevance in the overall humanitarian response on behalf of the LHT and the HCT.

Finding 25. Changes in management have hindered ERF leadership and coordination.

In 2012, several management changes occurred at the Colombian ERF. The ERF manager in place since the inception of the fund departed and was replaced by an interim manager until a new manager was hired in June

7 1ra reunión de la Junta Asesora del ERF Colombia, 18/11/2009, PNUD, Bogotá.


7

2012. This series of changes led to slower disbursements and to date, only one project has been approved in 2012 and only one disbursement has been made. The thematic trend for the upcoming eight projects in the pipeline for 2012 confirms the central importance of WASH investments in the humanitarian response in Colombia (the current approved amount for all sectors is US$1,572,314.00).

Finding 26. Risk management in an environment as dynamic and fragile as Colombia cannot be expected to adhere to classic risk indicators and management.

ERF funded projects in Colombia are subject to risk: logistical, financial, administrative, safety, political and reputational. The risk management strategy of ERF in Colombia is based on two separate tracks: the first relies on the seriousness of the project vision, feasibility and viability at its inception phase; the second relies on the perceived continuous information sharing among all interested stakeholders throughout institutional, geographic or thematic networks. Complex local operations (e.g. summer 2012 floods in the Department of Putumayo) can be the source of mutually agreed local humanitarian strategies whereby risk is shared, accepted and monitored by all relevant stakeholders. Proximity of OCHA’s antennas and sub-offices, as well as LHT, allows for closer monitoring of risk at local levels with all respective humanitarian, community, political, religious and/or safety institutions.

PARTNERSHIP Finding 27. Partnerships play a major role in consolidating the important role of ERF within the wider

Humanitarian Community.

The ERF has a wide range of partners and stakeholders, either at the local, NGO, governmental, or UN level. Partnerships provide a key component to fulfilling the mandate and objectives of the ERF. However, it appears that this aspect may be overlooked to a certain degree. Many informants have suggested that the ERF can broaden its outreach and organizational impact through greater collaboration, outreach, information sharing, consultation or flexibility in regards to its relationships with its partners. But given the already overstretched conditions of ERF personnel, this seeming oversight is understandable.

PROCESSES AND INPUTS Finding 28. Needs assessment and the application process is lengthy and delays the start of programming.

ERF needs assessments and project allocations are reviewed within the context of the following criteria8:

• “Double affectation” (i.e. natural disaster with conflict affected area): most beneficiaries are indigenous, afro-Colombians and/or IDP communities (and many times a mix of one category with the other);

• Vulnerable populations: although protection is not a main intervention area through ERF funds per se, projects highlighted the particular needs of vulnerable populations, especially women and children under 18 years of age (as Exhibit No. 9 demonstrates, these categories comprise 75% of the target population).

• Generally inaccessible locations where there is little or no State presence or institutional response capacities, and very often with presence of non-State armed groups;

• No other funding sources available - Although ERF funds may have a leverage effect for some organizations that benefit from additional funding sources, this is not always the case for recipient organizations involved in ERF implementation. Indeed, ERF was the sole funding source of all five NNGOs engaged in ERF implementation9;

• Cluster-based decision - Following analysis and, in most cases, previous approval of Local Humanitarian Teams, ERF funds can be used to augment “regular funds” of the humanitarian UN agencies present in Colombia, Central Emergency Response Fund (CERF) Underfunded (UF ) and Rapid Response Funds, and after analysis and previous (in most cases) approval of Local Humanitarian Teams.

Administrative protocols and processes through to the external audit review often have a negative impact on timeliness of emergency funding and can strain relationships with partners and stakeholders. The first step, application to ERF funds, is triggered by a humanitarian emergency. This first step is relatively easy to manage for large organizations that have the capacity to work with international standards. However, when smaller local NGOs want to intervene, they have to be supported either by a national branch (if applicable), or by the ERF manager if there is no national chapter. Thus, the ERF manager and OCHA often provide the necessary support to local NGOs with programmatic and technical assistance throughout the ERF cycle.

When an emergency is identified, a needs assessment is conducted by the local humanitarian team, local government representatives and the NGO. Clusters are not yet operational at that level and do not actively participate in this process. When the assessment identifies the need for a response, NNGOs and INGOs can

8 Ibid, p.11 9 Ibidem, Cruz Roja Colombiana, Opcion Legal, Pastoral Social, RET.


8

proceed to develop a formal ERF proposal. Although clusters are identified as playing a crucial role in the development of proposals, again it is the ERF manager who informally reviews the proposal prior to sending it to the technical committee to avoid unnecessary delays. Since the number of organizations able to intervene in a humanitarian capacity is fairly limited in Colombia, OCHA takes this extra step to avoid rejecting valid proposals lacking in technical content and to avoid unnecessary delays wherein the technical committee would be required to analyze a proposal twice.

Between the identification of an emergency, the needs assessment and the revision of the proposal by the ERF manager and the applicant, a month can easily pass. Once a proposal is judged as adequate by the ERF manager, it is vetted by the technical committee to ensure efficacy. In principle, this step should be completed in 1 to 3 days, but it can take 5 to 7 days. When a reviewer is out of the country, this step can take up to several weeks.

Finding 29. The ERF Technical Committee contributes to the relevance and feasibility of humanitarian proposals submitted to ERF Colombia.

All proposals submitted to OCHA ERF in Colombia are reviewed and revised by a 14 member technical committee who rotate on a semester basis. Each project is reviewed by a small panel of these experts. Their comments and inputs are directly integrated in the ERF online management tool that supports the development, monitoring and follow up of each proposal funded by the ERF. In theory, the technical review board has up to three days to review a proposal, although in practice, it often takes more time.

Finding 30. Lack of understanding in application preparation compromises efficacy, timeliness and coverage of ERF funding in emergency response.

The flow of recent proposals presented to the ERF suggests a concerning trend: one out of every two proposals did not conform to basic required application principles. From July to September 2012, the ERF Colombia has received 12 proposals, six of which were meant for core financing activities of NGOs. This observation illustrates a basic lack of understanding of the key functions, principles and objectives of the ERF. As cited earlier, this puts considerable strain on ERF/OCHA personnel; in 2012, four out of the eight new proposals to be funded were administered by the same OCHA Field officer.

Finding 31. Allocation and disbursement processes are lengthy and cumbersome.

Once a proposal is endorsed by all reviewers and judged as technically viable, ERF moves to the final approval and disbursement process. The humanitarian coordinator (HC) signs off an endorsement that is then sent to the recipient agency and to OCHA Geneva. This is a crucial step in initiating the approval process, yet it can be delayed for several weeks if, for example, the HC is out of the country for an extended period; there is no other responsible signing authority when the HC is unavailable.

When the endorsement is signed off by the HC, ASB in Geneva is activated. ASB requires that a bank certification be sent by recipient agencies. In certain cases, it has taken up to 15 days to have the appropriate banking documentation ready for the ASB. Local NGOs in Colombia do not have bank accounts in the US and cannot provide bank certification in English. This step therefore usually takes more than 1-3 days since the recipient agency may have several exchanges with the ASB to provide all the required banking information.

ASB also reviews the proposal and the budget and develops a Memorandum of Understanding (MoU) to be signed by the HC, ASB and the recipient organization. The contract is standard for all ERF projects, and it is only available in English. It should be noted that several local NGOs whose personnel are unable to read English have signed contracts without a full understanding of their responsibilities. Once all these steps have been successfully taken, 80% of the proposed budget is disbursed.

FUNDING AND PROJECT ISSUES Finding 32. The ERF has integrated cross-cutting issues in its processes and has been particularly

successful at ensuring gender equality in funded initiatives.

ERF plays a critical role in putting a particular focus on protection of vulnerable populations. As ERF is intended to target men and women equally, statistics demonstrate that a total of 55% of the target recipients were women who benefited directly from emergency response in the health, education, water and sanitation and early recovery sectors. Nevertheless, finalization of the complete integration of the IASC Gender Marker10 in the information system as well as in the ERF application form needs to be addressed. While gender analysis prevails throughout

10 The Gender Marker is a tool that codes whether or not a humanitarian project is designed to ensure that women, men, boys and girls of all ages will benefit equally from it, and that it will contribute to increasing gender equality.


9

the cycle of each project, as well as in the final presentation of outputs at the global level11, gender mainstreaming remains a constant challenge in the present day.

The table below shows a breakdown of ERF beneficiaries by gender. There seems to be an equitable distribution of beneficiaries among men and women and among girls and boys. Exhibit 4 Relative representation of Women, Men, Girls and Boys through ERF projects/sectors (2010/2011)

Sector Year Women Men Girls* Boys* Total

HEALTH 2010 1,170 1,309 950 950 4,379

2011 19,008 12,513 6,823 7,498 45,842**

WASH 2010 2,765 2,708 3,732 3,732 12,937

2011 1,829 1,667 1,904 1,945 7,345

EDUCATION 2010 3,759 3,906 7,665

2011 10 10 1,310 1,110 2,440

SHELTER 2010 130 123 218 218 689

EARLY RECOVERY 2011 691 768 480 428 2,367

FOOD SECURITY 2010 800 700 800 700 3,000

TOTAL 2010

4,865 (17%)

48,40 (17%)

9,459 (33%)

9,506 (33%) 28,670

2011 21,538 (37%)

14,958 (26%)

10,517 (18%)

10,981 (19%) 57,994

TOTAL 2010-2011

26,403 (30.5%)

19,798 (22.8%)

19,976 (23%)

20,487 (23.6%)

86,664

(*) For 2010, disaggregation between Girls and Boys did not exist. All were "Children"

(**) This explains itself largely by the nature of the project implemented in el Chocó by the local NGOCOCOMACIA

Finding 33. Donor support for ERF Colombia remains strong although future expansion of the fund should rely on a strategic diversification of donors.

ERF Colombia was initially supported by Norway, Sweden and Spain. These three donors continue to be its greatest supporters. While only one new donor (San Marino) has been identified in the short term, OCHA has expressed its desire to increase donorship and double the amount of the Fund.12 Exhibit Error! No text of specified style in document.5 Donors contributions to ERF Colombia – 2009-2012

Year Donor(*) Funding (US$) Decision Dates Total/Year

2009 Sweden 431,790 No data 1,425,274

2009 Norway 993,484 No data

2010 Sweden 418,410 No data

2,130,836 2010 Norway 1,012,146 No data

2010 Spain 700,280 No data

2011 Norway 1,023,541 9 Dec. 2011

2,410,470 2011 San Marino 14,245 14 June 2011

2011 Spain 686,813 15 March 2011

2011 Spain 685,871 24 may 2011

TOTAL 5,966,580 5,966,580

National disbursement rules and regulations in some countries (Sweden and Spain) have resulted in late disbursements from some donors, i.e. towards the end of UN fiscal year. The impact of this imbalance is that ERF Colombia may be perceived as having important levels of carry-over from one year to another. This imbalance may give the false perception that the ERF is either: a) over-funded in comparison to the number and funding level of the initiatives it supports; or b) that the capacity of disbursement of the ERF is uncertain. The following table illustrates the recent fiscal position of the ERF.

11 See Emergency Response Fund 2010 and 2011 – Results of ERF projects per Cluster. 12Emergency Response Fund, Annual Report 2011, p.8


10

Exhibit 6 Contributions and Disbursements – ERF Colombia – September 2012

Year Contributions Available Allocations (Us$) (*)

Disbursed (Us$) Pending To Disburse (Us$)

Balances Of The Year (Us$) (**)

2009 1.425.274

1.425.274

2010 2.130.836 1.819.994 1.485.187 334.807 1.736.116

2011 2.410.470 1.560.166 1.248.133 312.033 2.586.420

2012

1.182.598 400.000 782.598 1.403.822

TOTAL 5.966.580 4.562.758 3.133.320 1.429.438

(*) Referring to 2012 figures: projects endorsed by Humanitarian Coordinator during 2012.

(**) Near USD 547,000 are in the pipeline with new ERF projects that were not endorsed by the end of 2012.

Finding 34. The Advisory Board takes an active role in Colombia ERF strategic and operational planning.

The findings throughout this report demonstrate the presence and impact that the Advisory Board (AB) has within the Colombia ERF structure. The AB appears to be a progressive influence in this regard. For example, they have promoted the adoption of disaster preparedness, early recovery initiatives and capacity strengthening within the ERF scope. Moreover, the AB plays an active role in monitoring and consulting ERF operations at regular bi-annual meetings.

MONITORING, REPORTING AND AUDIT Finding 35. ERF protracted external audit processes are detrimental not only to these recipient

organizations, but to the reputation and efficacy of the Colombia ERF.

In accordance with ERF guidelines, ERF funded projects are required to undergo an external audit following implementation. To ensure accountability, 20% of the total funding allocation is withheld from the recipient organizations until such time as a positive audit is produced. While fiscal accountability is important to maintaining the integrity of the ERF, it is equally crucial to ensure that recipient organizations, especially small-budget national or local NGOs are able to move through the audit process as swiftly as possible.

To date, recipient organizations who received ERF funding in 2010 still have not received the 20% of funding to be granted after project completion. The procurement process to hire an audit firm for the 2010 projects started during the fall of 2011. The audit firm completed its work in March of 2012 and audit reports were sent to OCHA Geneva. Since then, the still-unapproved audit reports have been shuttled back and forth between Geneva and Colombia with no resolution in sight. Put succinctly, this process is still ongoing, almost two years after the completion of the first ERF projects. For small and local NGOs this period, for which there is no timeframe identified in ERF existing guidelines, is unacceptable and poses serious risks to their survival.

This present course is unsustainable for small-scale recipient organizations. For example, RET, an INGO that committed itself with two ERF projects in 2010 and 2011 has a shortfall of more than US$67,000 with its two projects. COCOMACIA a local organization is struggling with a US$50,000 shortfall.

However, this present course will also prove unsustainable for the Colombia ERF. Most NNGOs have received clear instructions from their boards to stop working with the ERF. This will remain so until reliable financial rules and procedures are put in place guaranteeing rapid audit and disbursement of the withheld funds. It has also been suggested that decreasing participation of NNGOs within integrated and concerted humanitarian rapid response will weaken the image and credibility of the ERF; it will de-legitimize the ERF in the eyes of the GoC as it will mainly support INGOs; it will obviously limit the scope of the ERF humanitarian response in areas or thematic sectors where local and/or national NGOs prevail; and, based on this, it will possibly affect the interest (and investments in the Fund) of some donors who are/would be interested in strengthening local humanitarian response capacities.


11

Overall Conclusions The overall implementation of the Colombia ERF has been generally positive.

For the three years it has been operational, the ERF has established itself as a relevant organization; it has provided innovative responses to the challenges posed by the ongoing crises affecting Columbia. It has been able to do so because of the unique attributes framing its objective:

1. Complementarity with existing mechanisms;

2. Flexibility of the humanitarian response;

3. Geographic or thematic “gap filling” in the humanitarian landscape of Colombia.

The degree to which each of these directives has been fully realized can be debated. However, given that the Fund is still in relative infancy it would be unfair to judge success or failure as though it were a well -established humanitarian organization. The fact that the ERF has collaborated with and funded Local, National and International NGOs in a number of rapid relief initiatives; that fact that it has worked with other mechanisms, e.g. CERF in capacity building; the fact that it has demonstrated flexibility in its interpretation of relief models and that it has filled critical gaps, especially in those crises-affected areas where the GoC cannot; all of these accomplishments indicate that the Colombian ERF has not lost sight of its core objective and will no doubt continue to play a modest yet pivotal role within the Humanitarian Community.

Still, there are many areas that need improvement in order to reach the full potential of its mandate, and to benefit the most those who often have least. While recommendations will be made in the following section, it is important to note that, as with any relatively new organization or enterprise, streamlining, especially in regards to process, vision, and relationships, is key.


12

Country Level Recommendations The following recommendations are intended for country-level operations. Global recommendations will be drawn from all field reports and presented in the final evaluation.

Recommendation 6. The ERF needs to improve its strategic direction and should take a leadership a role in solving existing concerns regarding its interventions.

As described in the evaluation findings of this report, there are some ERF global guidelines that have been questioned by stakeholders interviewed for this case study as well as pending issues on the strategic vision of the ERF that may ultimately affect its relevance if no action is taken to address these concerns and issues.

The most important and pressing concerns were related to the auditing rules and the fact that NGOs had not received the remaining 20% of their funding, almost two years after the end of their grant.

Concerning the ERF Colombia procedures, several stakeholders felt that the ERF should feed and lead the debate on the revision and expansion of the “double affectation” rule in order to respond more effectively to humanitarian emergencies generated in conflict-affected areas.

Another local guideline that should be revisited is the US$50,000 minimum floor and the US$250,000 ceiling for ERF funded projects. It has been suggested that in complex and protracted humanitarian responses, there is often a need for smaller interventions (for instance with very small NNGOs that do not have the capacity to implement a US$50,000 disbursement, or to address small localized situations) or larger and longer term emergency response that would require more than US$250,000.

Strategic concerns about the ERF vision and operations were also expressed. ERF therefore needs to work in a participatory manner with all interested stakeholders in developing a more strategic vision for the Fund regarding future sectoral or thematic interventions. Presently, ERF projects represent a sum of parts rather than a coherent and prioritized whole. A more strategic vision articulating where or what type of interventions to allocate emergency funds would be a good step toward improving the ERF’s strategic vision.

In terms of overall improvements to be made to its strategic direction, the ERF in Colombia should also consider paying greater attention to disaster preparedness. Disaster preparedness was perceived as a key sustainable element of humanitarian response in the country and could be pursued with, for example, prepositioning, training, or community awareness.

Strategically, ERF in Colombia can definitely position itself as a leader in the country in defining best practices for early recovery in a humanitarian response context. The definition of these parameters should be done through a collaborative process convened by ERF in which national and international humanitarian actors participate.

Recommendation 7. If the ERF in Colombia is to flourish and assume a leadership role in the humanitarian response community, its resource allocation will need to be revisited and increased.

If the ERF is to become a more prominent actor in the humanitarian community, resource acquisition and allocation will need to be planed accordingly. For instance, if strategic changes are made to the double affectation criteria or to the amounts given for project implementation, the workload for the ERF manager will become overwhelming. In this sense, augmenting human resources should be considered a priority, especially in terms of ERF personnel.

ERF and OCHA could also consider allocating a certain percentage of its budget, or a surcharge applied to implementing organizations to pay for technical needs assessments, monitoring and evaluation of ERF funded projects.

Recommendation 8. The ERF in Colombia, in collaboration with OCHA should consider developing short and mid-term sustainability strategies that will support ERF funded initiatives beyond immediate humanitarian response.

Stakeholders interviewed mentioned that some ERF iprojects (e.g. shelters or health posts) did not include plans for upkeep. Therefore, longer term governmental relief required for infrastructure maintenance should be detailed in the proposals and the appropriate government authorities should be consulted. Short and mid-term monitoring should take sustainability issues into consideration. Early recovery initiatives should also aim at strengthening local sustainability strategies.

Recommendation 9. The ERF electronic platform should integrate financial information (budget, interim and final reports) that could be overseen by the administration and finance section.

Making these small changes to the rather excellent ERF electronic platform would provide better support the ERF manager. It would allow other OCHA staff (e.g. finance staff) to jump in and provide support on the ERF monitoring and auditing tasks.


13

Recommendation 10. OCHA Colombia should develop more oriented results-based management operations, evaluation tools and capacities in order to promote ERF outcome and impact results.

OCHA Colombia needs to go beyond the measure of ERF outputs and be in a position to better promote final outcome and impact results of its intervention in support of the populations who benefited from ERF interventions. A stronger results-based management alignment of its evaluation tools and capacities should support incoming operations and evaluations.

Recommendation 11. OCHA Colombia needs a wider and stronger communication strategy directed towards national humanitarian institutions and organizations in order to bridge the “communication gap” obscuring ERF objectives, policies and procedures.

Several stakeholders mentioned that they were not sufficiently aware of ERF activities throughout project cycles. In addition, norms and guidelines that apply to ERF funding are not well known among potential recipient organisations and OCHA field staff.

To bridge that gap, wider promotion of ERF achievements through a major overhaul of the OCHA website would be welcome. OCHA sub-offices and antennas should also receive adequate policy and operational guidelines on ERF project cycles in order to support local humanitarian organizations when implementing ERF projects.

Recommendation 12. Monitoring of ERF projects should be a task shared among OCHA field staff, the ERF technical committee, and clusters.

Monitoring of ERF projects is a complicated task in Colombia. Several stakeholders need to pitch in if the ERF monitoring function is to be improved. ERF Technical Committee experts should be involved in the follow-up, evaluation and technical guidance regarding project development for proposals they reviewed. OCHA field staff should also play an important role within the monitoring team since they are directly on the field.


14

Appendix I: Programme Agenda INTRODUCCIÓN DEL EJERCICIO DE EVALUACIÓN

LUNES 01 DE OCTUBRE DE 2012

Desde Hasta Descripción de la actividad Metodología/Recursos Responsables

08h00 09h:00 Debriefing con el Sr. Gerard Gomez (Head of OCHA Office in Colombia) y el Sr. Max Bonnel (Deputy HoO)

N/A Oficinas de OCHA Bogotá

Organiza y gestiona: Luis Fernando Amaya - ERF Manager Magda Cifuentes: Asistente HoO OCHA. CONFIRMADA

9h30 10h30 Debrief de la misión y revisión de plan de trabajo/ arreglos logísticos necesarios con el ERF Manager

Oficinas de OCHA Bogotá Organiza y gestiona: Luis Fernando Amaya - ERF Manager

10h30 11h30 Revisión del proceso de trabajo del ERF en Colombia (Luis Fernando Amaya – ERF Manager)

Oficinas de OCHA Bogotá Organiza y gestiona: Luis Fernando Amaya - ERF Manager

11h30 12h30 ACCIÓN CONTRA EL HAMBRE (INGO) Sr. José Luis Barreiro

Oficinas de OCHA Bogotá

Organiza y gestiona: Luis Fernando Amaya - ERF Manager// Equipo de evaluación externa ERF. CONFIRMADA

13h00 14h00 Almuerzo Almuerzo Almuerzo

14h00 15h00 Reunión con equipo de Coordinación de la Oficina de OCHA en Colombia

Revisar estado de la estructura de coordinación humanitaria en Colombia. Principales logros y retos. (MCH, agenda transformadora del IASC, Coordinación en Desastres Naturales) Revisar los conocimientos del equipo de Coordinación de OCHA Colombia con el ERF.

Organiza y gestiona: Luis Fernando Amaya - ERF Manager / Chiara Capozio - Responsable Unidad de Coordinación OCHA Colombia. (Nicolas Mayr, Maria Claudia Marin)

15h30 17h00 OXFAM GB Sr. David Huey/Charles Rowley

Oficinas OXFAM GB

Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF. CONFIRMADA

17h30 Cierre del día Cierre del día Cierre del día

MARTES 02 DE OCTUBRE DE 2012

Desde Hasta Descripción de la actividad Metodología/Recursos Responsables y ubicación

08h30 10h00 Entrevista con Unidad Administración OCHA OCHA Colombia

Oficinas OCHA Bogotá

Luis Fernando Amaya – Valerie Valdivieso CONFIRMADA

10h30 11h30

RET ROLANDO MORALES (INGO) Coordinador Nacional de Programas (e) RET - Protecting Trough Education

Cra. 11 Nº 71 - 40 oficina 1002, Bogotá, Colombia Tel 312 5535377


12h00 13h00 Espacio de trabajo con el ERF Manager OCHA Colombia

Oficinas OCHA Bogotá Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF.


15

Desde Hasta Descripción de la actividad Metodología/Recursos Responsables


14h30 15h30 OPCION LEGAL (NGO Nacional) Sr. Raúl Hernández Coordinador Nacional

Calle 43 No. 27-20 Bogotá, Colombia PBX (571) 3407848

Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF.

16h00 17h30

SAVE THE CHILDREN Sr. Diego Vila – Oficial de programa Sra. Claudia Vega – Responsable de relaciones externas

Carrera 7 No. 32-85 Oficina 302, Bogotá, Colombia Tel: +57 1 2854850 Fax: +57 1 2854536 (Ext 105)

Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF.

Cierre del día Cierre del día Cierre del día

MIERCOLES 03 DE OCTUBRE 2012

Desde Hasta Descripción de la actividad Metodología Responsables

8h30 10h30

Reuniones bilaterales / Grupo focal con coordinadores de Clúster (Seguridad Alimentaria y Nutrición, Agua y Saneamiento, Salud, Educación en Emergencias, Protección) Sra. Manuela Angel – Coordinadora del Cluster SAN Celular:310 3213872

Oficinas de OCHA Bogotá

Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF. CONFIRMADOS SAN, ALBERGUES Y SALUD

JUEVES 04 DE OCTUBRE 2012


09h00 10h30

Reunión Unidad de Gestión del Riesgo ante Desastres Naturales - COLOMBIA Sr. Natalia Segura Cooperación Internacional UNGRD

Oficinas de la UNGRD Calle 13 - Carrera 37 Bogotá, Colombia

Luis Fernando Amaya - ERF Manager María Claudia Marín Equipo de evaluación externa ERF.

11h00 12h00 Misión de Terreno - Departamento del Chocó

Se visitará a la Organización Comunitaria COCOMACIA, ejecutora del ERF-DMA-0467-018. Reunión el 5 de Oct. a las 2 pm, en COCOMACIA y con la CRUZ ROJA HOLANDESA – 4 de Octubre Entrevista equipo de campo OCHA.

VISITA DE CAMPO HASTA EL VIERNES 5 DE OCTUBRE (REGRESO A BOGOTA PM) RESPONSABLE: Katherine Catamuskay Humanitarian Affairs National Officer - OCHA Chocó

11h00 13h00 Revisión de temas administrativos y operativos ERF Colombia – ERF Manager y equipo de evaluación

Oficinas de OCHA Bogotá.

Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF. Atiende uno de los evaluadores


14h00 15h30 SOLIDARIDAD INTERNACIONAL (INGO) Sr. Daniel Martin – Jefe de País

Carrera 6 Bis No. 26 – 51 Oficina 301 Bogotá Teléfono (1) 2431438 – 316 8763072


16h00 17h30 Revisión de material y solicitud de retroalimentación – reuniones adicionales

Oficinas de OCHA Bogotá.


Cierre del día Cierre del día


16


VIERNES 05 DE OCTUBRE 2012

09h00 10h00

AGENCIA COLOMBIANA PARA LA COOPERACIÓN INTERNACIONAL – APC Sra. Luz María Salazar – Sra. Lucia Toro

Oficinas APC Colombia Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF.

10h00 11h00 Espacio de trabajo para el equipo de Evaluación

Oficinas de OCHA Bogotá Equipo de evaluación externa ERF.


14h00 17h00 Reunión de clarificación en ERF Manager

Oficinas de OCHA Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF


LUNES 08 DE OCTUBRE DE 2012


8h30 10h00

SIDA – Agencia Sueca para el Desarrollo Internacional Internacional - Sra. Emma Nilenfors – Responsable Cooperación SIDA / Sr. Sergio Arboleda – Oficial de Programas

Oficinas ASDI Bogotá Tel: +57 1 3256196


10h30 12h00 Entrega de resultados al Deputy HoO Max Bonnel y al equipo de OCHA Colombia.

Oficinas de OCHA Luis Fernando Amaya - ERF Manager / Equipo de evaluación externa ERF.



17

Appendix II: Stakeholders Met Name Position Organization Type of interview

1. AMAYA ORTIZ, Luis Fernando ERF Manager Colombia OCHA Face-to-face interview

2. ANGEL, Luis Former legal representative COCOMACIA Face-to-face interview

3. ANGEL, Manuela Coordinator Food Security and Nutrition Cluster

Focus Group

4. ARBOLEDA, Sergio Program Officer Embassy of Sweden Face-to-face interview

5. BARREIRO, José Luis National Director Action Against Hunger Face-to-face interview

6. BONNEL, Max Deputy Director OCHA Telephone interview

7. BUDDEN, Provash Country Director Mercy Corps Face-to-face interview

8. CAICEDO, Juan Pablo National office of information and reporting

OCHA Focus Group

9. CAMACHO, Claudia Responsible Emergency Education Cluster

Focus Group

10. CAPOZIO, Chiara Humanitarian Affairs Officer OCHA Focus Group

11. CASTANEDA, Sandra Responsible for emergencies - IOM Shelter cluster Focus Group

12. CORZO, Orlando Accountant Mercy Corps Face-to-face interview

13. DAVILA Roger Program Officer Save the Children Face-to-face interview

14. FERRERO, Sergio Advisor, International Emergency Program WHO/PAHO

Health Cluster Focus Group

15. GOMEZ, Gérard Head of Office OCHA Face-to-face interview

16. HERNANDEZ, Raul National Coordinator Opcion Legal Face-to-face interview

17. HERRERA, Angie Fundraising and Donor Relations Oxfam GM Face-to-face interview

18. KRUIS, Rode Local Representative Netherlands Red Cross Face-to-face interview

19. MARIN, Maria Claudia Information Officer OCHA Focus Group

20. MARTINEZ RUEDA, Javier Eduardo Project Coordinator Corporación Desarrollo y Paz del Magdalena Medio

Telephone interview

21. MAYR, Nicolas National officer OCHA Focus Group

22. MORALES, Rolando Emergency Education Coordinator RET – Refugee Education Trust

Face-to-face interview

23. MORENO, Maria Cristina Project Coordinator Mercy Corps Telephone interview

24. MORENO, Olga Lucia Administrative Coordinator Opcion Legal Face-to-face interview

25. PEREA, Imer Project Coordinator COCOMACIA Face-to-face interview

26. PERSICO, Daniele Project Coordinator COOPI Telephone interview

27. REY YÉBENES, Fernando General Coordinator Embassy of Spain Face-to-face interview

28. REYES, Ana Milena Administrative Assistant Opcion Legal Face-to-face interview

29. ROJAR, Octavio Legal Representative COCOMACIA Face-to-face interview

30. ROWLEY, Charles Program Manager Oxfam GB Face-to-face interview

31. SAAVEDRA DUARTE, Jacqueline National Emergency Service Pastoral Social Face-to-face interview

32. SANTOS, Daniel Martin Country Representative Solidaridad Internacional Face-to-face interview

33. SEGURA, Natalia International Cooperation Unidad de Gestión del Riesgo ante Desastres Naturales


34. TORO, Lucia Directorate of Demand Colombian Presidential Agency for International Cooperation


35. TORO ZULUAGA, Guillermo Alberto Disaster and Risk Reduction Manager

Oxfam GB Face-to-face interview

36. VALDIVIESO, Valérie Administrative and Finance Officer OCHA Face-to-face interview


18

Name Position Organization Type of interview

37. VEGA, Claudia Responsible for external relations Save the Children Face-to-face interview

38. VILA PALA, Francesc Responsible for humanitarian projects

Embassy of Spain Face-to-face interview

Advisory board members COCOMACIA Focus Group


The Global Evaluation of Emergency Response Funds (ERFs) Country Case Study Report - Ethiopia

March 2013


i





ii


Disclaimer .................................................................................................................................................................... i




Introduction ................................................................................................................................................................. 1


Methodology ............................................................................................................................................................... 2

Overview .................................................................................................................................................................... 2

Country Case Study Selection ............................................................................................................................ 2


Limitations .......................................................................................................................................................... 2

The ERF in Ethiopia - Overview ................................................................................................................................. 4

Findings ...................................................................................................................................................................... 7

Operational impact ..................................................................................................................................................... 7


Outcomes ................................................................................................................................................................... 8

Partnership ................................................................................................................................................................. 9


Funding and Project Issues ...................................................................................................................................... 10







iii

Abbreviations CERF Central Emergency Response Fund

DfID Department for International Development


EHCT Ethiopian Humanitarian Country Team










OFDA Office of Foreign Disaster Assistance


RBM Results Based Management




UN United Nations



1

Introduction This is one of five Country Case Study Reports that are mandated as a contributing part of the Global Evaluation of ERFs, which is taking place in the latter half of 2012. This case study report reflects the findings arising from a 12 person day mission to Ethiopia by a staff of two, one from Universalia and one from OCHA’s Evaluation and Guidance Section (EGS). Details of the mission can be found in Appendices I and II.

This Country Case Study Report is firmly focused on conditions in Ethiopia. While it is obvious that global conditions and the overall management system of the ERFs impacts on the delivery of services in Ethiopia, this report will not directly address these global issues. Rather, how these global considerations affect the performance of the ERFs as a whole will figure largely in the overall report of this evaluation

















2

Methodology OVERVIEW In our proposal, the methodology proposed for the assignment consisted of a mixture of qualitative and quantitative data collection, with attention both to forward-looking, improvement oriented questions and to assessment of impact. In summary, we will follow the OCHA Evaluation Policy13 objective of determining:




Country Case Study Selection

Based on initial document review and suggestions received from our key respondents during the Inception Phase, five field missions were conducted in countries affected by humanitarian natural disaster or in complex emergencies. Assessments of two of the ERFs, in oPT and Colombia, have been requested by in-country stakeholders, and three, Ethiopia, Indonesia and Afghanistan have been selected by the evaluation team taking into consideration size, type of emergency, country context, donorship profile, and evaluations and reviews previously conducted.


This is one of five field mission reports for the Global Evaluation of ERF. It is not a specific and detailed evaluation of the HRF in Ethiopia. In the past, OCHA had conducted specific and intensive country level evaluations of individual ERFs. As well, audits have been conducted of various HRFs and ERFs.

This report on the HRF in Ethiopia is an overview report. In light of the approved methodology, this report drills down using Ethiopia as an example to explore at a practical level the key evaluation issues. Given the resources available for it (see below) it is unreasonable to consider that such an overview and summary of the HRF in Ethiopia could be considered to be an exhaustive report. Specifically, unlike prior country-based reports, this overview does not attempt to provide a comprehensive assessment of the HRF’s overall performance. Rather it highlights the nature of HRF behaviours and practices within the context of the key evaluation issues.

Limitations

The field mission to Ethiopia was exceptionally well planned with a very high participation rate of all local stakeholders. The evaluation team therefore wishes to congratulate the management and staff of OCHA in Ethiopia and through them, to thank the many participants who took of their time and talents to advise our mission. Aside from a scattered number of timing conflicts due to travel or otherwise scheduled meetings, the evaluation team, as is evident in the Appendix, met a broad cross section of relevant personnel. The extent of the data provided by the HRF staff and management was more than adequate; indeed it exceeded in some respects, our requirements.

That said, however, there were some conceptual and structural limitations to the field mission. The first limitation is a conceptual one that is rooted in the design of the evaluation itself. The design provided for approximately equal time for each of the five country case studies. While these field missions were in no way designed to be an in-depth country-based evaluation as was conducted previously, the size of the Ethiopian HRF in comparison to the other 11 HRF/ERFs would have probably warranted the allocation of several more days. In particular, several of the other field missions were able to undertake out of the capital meetings, (Indonesia, Colombia and oPT). In at least the case of Indonesia, the contrast between the size and scope of the HRF in Ethiopia versus the very small HRF in Indonesia illustrates this problem vividly. The limitation therefore was that while the participation rate among stakeholders in Ethiopia was very high, the mission did not have the opportunity to seek a field context so as to better orient delivery partner views within the overall national context.

13 OCHA Policy Instructions: Evaluations, 2010.


3

A second limitation of the mission again related to timing. If a differential approach to mission scheduling had been adopted at the outset, it is likely that the HRF in Ethiopia would have been able to drill down deeper into the pool of stakeholders, especially those in the clusters most directly linked to HRF project activity. As well, probably more time would have been secured to meet with government representatives and possibly even government representatives outside the capital.

While the HRF staff and managers made every effort to brief in advance NGO stakeholders in particular, there remained a degree of apprehension among some about the nature of the evaluation. Even with the best of efforts, it is difficult to convey a message that an evaluation such as this is not primarily about judging performance on an individual project basis. As well, some of the NGO stakeholders appeared to be reticent in expressing their views about challenges, and especially about the magnitude of the grants. This again was to be expected; however it did to some extent limit the openness of the dialogue.


4

The ERF in Ethiopia - Overview The HRF is a pooled fund managed by the Humanitarian Coordinator with the operational support of OCHA. Established in 2006, the HRF is used to respond to disasters triggered by natural hazards, such as floods, droughts and outbreaks of diseases, and complex conflict-related crises. The distribution of funds is based on the assessment of needs within the context of an annual plan developed by the Government as well as other unanticipated humanitarian situations that may emerge. The four main objectives of the HRF are to:

1) Ensure more adequate, timely, flexible and effective humanitarian financing through the use of the pooled funding mechanism;

2) Empower the humanitarian coordinator system;

3) Support development of the cluster approach to coordination;

4) Improve partnerships between UN and non-UN actors.

It is most important to underscore that the HRF in Ethiopia currently is nearly as large, in terms of contributions, as all other HRFs/ERFs combined, thus placing it in a separate category from its sister funds. Indeed, one weekly Review Board meeting, of which there are between 30 and 40 formal meetings a year, can transact more project support than several HRF/ERFs have access to on an annual basis.

Although exceptions have been allowed where justified (for instance, in the bulk purchase of therapeutic supplementary food by UNICEF on behalf of the cluster to support nutrition response), HRF grants usually have a grant ceiling of US$700,000 per project, with a duration usually not exceeding 6 months.

Unlike a number of other HRF/ERFs, the HRF in Ethiopia has benefited from a wide and sustained donor base, with DFID (UK) and the Netherlands being the largest donors. The table below shows overall revenue since inception. Exhibit 1 Overall Contributions since 2006

Donor Contributions US$ (2006-2011)

2006 2007 2008 2009 2010 2011 Total per Donor

DFID 8,861,284 7,632,793 28,489,852 5,573,248 5,600,000 8,025,682 64,182,859

Netherlands 5,012,000 3,395,940 25,675,614 15,441,176 9,790,210 59,314,940

Norway 1,553,277 2,022,776 2,644,228 2,232,764 747,031 1,238,281 10,438,357

Ireland 4,352,602 2,033,133 6,385,735

Sweden 577,564 4,090,487 6,476,850 3,696,600 14,841,501

Spain 1,293,661 2,664,002 3,957,663

Switzerland 247,934 689,853 699,301 1,637,088

Italy 857,233 1,010,101 1,867,334

Denmark 4,893,092 4,893,092

Misc 295,517 295,517

Total 15,674,495 13,629,073 68,389,047 32,456,472 28,401,036 9,263,963 167,814,086

Given the chronic nature of emergencies in Ethiopia, with many that impact food and water supplies, the distribution among types of support noted below is to be expected, with nutrition and WASH-related activities predominating. Exhibit 2 Funded Projects by Sector

HRF Funded Projects by Sector

Sector US$

2006 2007 2008 2009 2010 2011 Total

Food 355,397 398,358 2,000,000 4,778,400 1,731,080 9,263,235

Health 2,341,991 783,233 3,731,898 4,354,129 75,304 11,86,554

Nutrition 490,545 241,246 20,560,367 11,843,571 3,734,186 15,718,345 52,588,260

WASH 6,194,050 2,502,556 7,486,368 6,617,857 8,699,462 1,169,844 32,670,137

Water Tankering - 2,171,099 461,815 0 4,591,456 7,224,370

Livestock 3,713,749 1,297,793 3,486,696 1,080,965 4,937,562 3,491,372 18,008,137


5

HRF Funded Projects by Sector

Sector US$

2006 2007 2008 2009 2010 2011 Total

NFI 1,615,846 1,233,524 681,592 1,248,927 2,710,524 299,715 7,790,128

Agriculture - 4,588,860 827,432 2,710,085 188,009 8,314,386

Assessment - 184,404 478,305 303,094 965,803

Flood Mitigation 2,402,886 2,402,886

Refugee 1,589,457 1,589,457

Service Support 2,692,071 500,809 2,398,178 5,591,058

Education 466,126 12,480 478,606

Urban Shelter 615,117 615,117

Capacity Building 197,415 197,415

Immediate Response Contingency 185,800 266,155 80,000 531,955

TOTAL 14,711,577 6,456,709 44,891,284 39,642,858 11,375,793 27,936,921 159,517,505

By design, the HRF is intended primarily to support NGOs, and in the case of Ethiopia, due to existing government regulation, INGOs. The following table shows the 2011 balance.

In 2011, drought was the major cause of the natural disasters that the HRF responded to.

The following map shows the diversity of the HRF response, although as the table below shows, the vast majority was directed towards severely drought affected areas in northern Ethiopia.


6

It should also be noted that the original policy documents establishing the Fund envisaged an Advisory Board tasked to give policy guidance related to the Humanitarian Coordinator and to HRF to OCHA and the Review Board. In the past, this Board has proved very difficult to convene and now, with the recent inclusion of donors on the Ethiopian Humanitarian Country Team (EHCT), there is some suggestion that the EHCT should provide the HRF with this level of support.


7

Findings OPERATIONAL IMPACT Finding 36. The HRF in Ethiopia is an important and invaluable tool in the overall humanitarian response

The above Finding is grounded on a combination of documentary review, the analysis of the most recent HRF client feedback survey and extensive interviews with recipient NGOs, UN partners and government representatives. Very simply, in Ethiopia at this time, the HRF is a unique element within the humanitarian architecture in that it provides a direct means to engage NGOs, at this moment in time only INGOs due to national government regulation, in filling unanticipated gaps to respond to slow or rapid onset humanitarian crises.

The evaluation team, by participating in a regularly scheduled HRF Review Board meeting, was able to witness the way the HRF team and its Review Board addressed both slow onset emergencies (a drought situation) and a more rapid onset emergency where conflict between two tribal groups resulted in food shortages.

Virtually all stakeholders indicated that the size of the HRF in Ethiopia was such that while it could not respond to every gap that might exist, it possessed a sufficient critical mass so as to make a welcome contribution in relation to the relatively large range of circumstances , especially in the last several years.

In short, there is nearly unanimous support about the overall relevance of the HRF in Ethiopia and the unique role that it plays.

This Finding is tempered by several factors that are generally outside the control of the HRF in Ethiopia. First, while the size of the grants, sub US$700,000 with the majority in the US$300,000-500,000 range, is larger than in any other ERF, it may not be sufficient to fully meet the emergency needs in Ethiopia. Second, the timeframe of the HRF, six months excluding any no-cost extensions, is somewhat limited given the nature and the prolonged duration of emergencies that Ethiopia faces.

Finding 37. The overall impact of the HRF however is limited to some degree by the degree of time that it takes to process requests, in comparison to a few other entities which have equally if not faster response mechanisms.

The major reservation with respect to the relevance of the HRF is in perceptions about its timeliness, some of many of which appear to be based on erroneous assumptions about the very nature of the HRF itself.

In the first context, the HRF in Ethiopia over the last two years has reduced the amount of time from the receipt of an application to the release of a cheque to approximately one calendar month. The in-Ethiopia processes of a formal review by cluster (not a virtual agreement by assumed consent) and a similar kind of formal review by the Review Board takes about 60% of this period, with the balance being required by Geneva to conduct due diligence and processing requirements .

Donors and NGOs alike are under the impression that this process takes too long. However, they also, based on the\responses to the client survey and interview data, praise the HRF in Ethiopia for the transparency and rigorous nature of both the cluster and Review Board processes. It is logically inconsistent to seek a faster process while at the same time seeking the continuation of the degree of clarity in the current one. Granted, it may be possible at the global level to identify administrative streamlining in this one month duration.

The second dimension of timeliness related to the amount of time before the formal submission of a proposal. There is very little data in Ethiopia or elsewhere in the world to formally track what is in essence an informal process of consultation and advice-giving. Clearly, the overall effectiveness of the cluster system in a country can impact on this upfront time. In Ethiopia, it is apparent that the current cluster system works adequately in the needs identification processes.

The HRF is faced with a near Hobson’s Choice, reduce the amount of upfront time and possibly reduce the quality and connectivity of proposed projects, and thus delay formal approval processes by having to “fix up” deficiencies, or maintain a process of due diligence and support to NGOs so as to improve the incoming “quality” of proposed projects and thus speed their overall acceptance. One thing is clear, unlike other ERFs; the HRF in Ethiopia appears to have sufficient staff, to actually engage in this upfront process of improvement.

There is one organization in Ethiopia that plays a significant role in humanitarian response that is clearly faster than the HRF. It is USAID’s Office of Foreign Disaster Assistance (OFDA). OFDA in Ethiopia and elsewhere has the option of pre-approving selected NGO partners and providing them with a pool of funding (in the Ethiopian context up to US$5 million) that the NGO in question can disburse on 48 hours notice of receipt of a verbal approval by OFDA. In Ethiopia, only two NGOs, GOAL and IRC, have this authority granted to them. Others do not and face a grant approval timeframe with OFDA that exceeds the timeframe of the HRF.


8

Finding 38. The HC function and cluster coordination has been strengthened by the presence of the HRF, in large part due to its size and, longevity.

One of the strategic objectives world-wide of the HRF/ERF mechanism is to strengthen the HC function within the context of the cluster system. In Ethiopia, based on triangulated information secured from a variety of types of partners, the HC role appears to have been strengthened by the presence of the HRF. This is in large part, due to the fact that the HRF has sufficient critical mass to be able to influence other UN and NGO partners within the clusters. In short, in the case of Ethiopia, “money talks”. But that is not all.

There is very positive support for the way the HRF team worked in country, the way the Head of Office participated as a key leader and coordinator, and the degree to which the HC himself has been willing to promote the HRF in Ethiopia. These characteristics relate in part to the fact that the HRF has sufficient critical mass to be a useful tool; but they are probably more influenced by the personalities of the current set of players. The extent to which the organizational strengthening of clusters and the HC is based on idiosyncrasies in personalities involved is discussed in greater detail in the Partnership section of this Report.

Finding 39. The HRF has generated a limited degree of catalytic support.

This Finding is based on scattered evidence and relates to some WASH projects funded by HRF which were subsequently built upon by other partners in order to remove the possibility of the negative impact of further drought. It also reflects testimony related to seed provision programming in Southern Ethiopia, where one-time distributions supported by HRF were followed by other projects designed to improve overall agricultural livelihood.

The key question here, however, is the degree to which the HRF and its approval processes in particular have been designed to catalyse follow-up action, because in many instances, the HRF-supported project cannot be assumed to respond to the totality of the emergency or its duration. In that context, the HRF presently does not have the tools to fully coordinate post-project activity and without the tools, including resources, might run the risk of delaying project approval while seeking to map out a longer term future.

OPERATIONAL EFFECTS Finding 40. The formal pre-screening by clusters, combined with a rigorous review by the Review Board,

improves relevance and allows for a degree of prioritization.

As mentioned previously, the HRF in Ethiopia utilizes a process of formal and regularized consultative meetings with clusters and the Review Board. This process, while possibly more time consuming than the virtual assumed consent model contained in the new Guidelines, has enabled the HRF to maintain an on-going relationship with the members of the Review Board. The Review Board, in meeting on a regular basis and in having significant participation of both NGOs and UN partners, enables the HRF to establish a process where an individual project can be assessed within the context of other activities known to the Review Board members and not strictly on a “one off” basis. Thus, the overall relevance of the HRF is increased as is its ability to make positive prioritization among projects. It also should be noted that the Review Board turns down about a quarter of all submissions made to it, even after the rigorous process of pre-application consultation and cluster review.

OUTCOMES14 Finding 41. The national government’s definition of an “emergency” and the focus of the biannual

Humanitarian Response Plan of the government tend to result in the HRF concentrating on larger projects/emergencies and having fewer resources to address small scale situations.

This Finding speaks to the relationship between the government of Ethiopia and various members of the humanitarian community, especially UN partners. First, it should be noted that the HRF is the only pooled fund or humanitarian fund in general provided by UN bodies that operates at partial arm’s length from mandated government-wide planning and resource allocation. This differentiation has positive benefits for the HRF and has enabled it to establish with civil society in Ethiopia, via INGOs, a direct rapport with respect to emergencies and gap filling.

The government has established a biannual, rolling and revised, national disaster planning process. While this may be beneficial given the chronic, slow onset nature of disasters in Ethiopia, the global mandate of the HRF/ERF process is in filling unforeseen humanitarian gaps that are small to medium sized. The larger scale approach of the biannual rolling plan tends to overlook short, more rapid onset situations, for example, rapidly emerging crises that are caused by community conflicts, as opposed to the consequences of long term drought over an entire province.

14 This particular set of questions as laid out in the approved Evaluation Matrix had a tendency to duplicate other issues. For the sake of brevity and clarity, these duplications which related to the nature of the projects and whether they generated benefits and synergies, have been addressed earlier.


9

Given the fundamental requirement for all HRFs/ERFs to fill gaps that have been identified on the basis of larger project planning, this national process in Ethiopia may limit to some degree the ability of the HRF to respond to smaller situations, as it must in the first instance, secure government approval for participation.

This national condition, which is “part of the rules of the game” in Ethiopia, can serve to influence NGOs and donors to the erroneous assumption that the HRF is not timely, which is supported to some degree by the demand-driven nature of the HRF; namely, that it responds to requests as opposed to initiating action in its own right.

PARTNERSHIP Finding 42. In Ethiopia, the HRF and OCHA as its manager are seen as a good partner by virtually all

stakeholders.

Given the size of the HRF in Ethiopia and the number of personnel in comparison to virtually any other ERF, it is not surprising stakeholders indicated that they had on-going contact with the HRF itself. Virtually all categories of stakeholders expressed strong and positive beliefs about the quality of the partnership with HRF. The transparency of HRF and especially of its selection process was identified as key element in this exercise of good partnership. The degree to which the staff of the HRF assisted grant applicants and worked with cluster partners was also identified as a strong benefit.

The one area where stakeholders, especially NGOs, expressed an interest in improvement was follow-up and opportunities for lessons learned. While the regular HRF communications tools including its newsletter were seen as highly beneficial in terms of both communications and the practice of good partnership, there was a desire for the HRF to undertake more follow-on/follow-up activities that would bring partners together and thus promote a greater degree of sharing.

Finding 43. The quality of this partnership has enabled the HRF to acquire the status as the preferred partner to respond to life and livelihood threatening gaps in the humanitarian response.

Virtually all groups of stakeholders, in supporting the quality of the partnership with the HRF, indicated that the HRF was in many respects the preferred partner on a national basis to address unfilled humanitarian gaps. With the exception of a handful of partners who had access to OFDA pre-committed funds (only two INGOs), stakeholders indicated that they felt that the HRF was the most appropriate entry point into the humanitarian system.

The only reservation was around the issue of timeliness, as there is a general perception that the HRF could be faster as well as broadly-based concern about the impact of audit delays in the recent past. However, in both these instances, the stakeholders recognized that their concerns lay not so much with the HRF in Ethiopia as they did with global ERF procedures.

Finding 44. Although good partnership is practiced, the concept of “good partnership practices” is not well understood.

At present, OCHA itself lacks a consistent policy or framework to enable and empower its staff to exercise good partnership. In Ethiopia, it is therefore not unreasonable to assume that staff and managers did not have an appreciation that partnership is in fact a managerial practice where there has been considerable study even within the UN system as to the effects of certain partnership practices on the attainment of goals and objectives. In short, practicing good partnership is more than “just common sense”.

Finding 45. Good partnership in Ethiopia depends on interpersonal relationships and personal practices in the absence of any procedures to encourage it.

This Finding demonstrates how good partnership and the benefits accrued from it can be placed at risk in the absence of at least some degree of direction about the nature of what constitutes good partnership practice. The evaluation team noted with interest the degree to which all categories of stakeholders outside of OCHA identified with individual members of the HRF staff and identified these individuals and their behaviours as the key components of what they saw to be a successful relationship.

PROCESSES AND INPUTS Finding 46. The existing quality control mechanisms combined with rigorous selection processes ensure

strong and effective programming.

This Finding speaks to quality control mechanisms that exist specifically within Ethiopia and does not address quality control mechanisms that are centrally administered in Geneva.

The HRF in Ethiopia, as noted above, has developed a series of well-accepted and transparent project review mechanisms. These include informal consultations with applicants before the formal submission of a grant application, active Review Board processes, and a degree of project monitoring.


10

This continuum of mechanisms is more transparent and more proactive than those suggested in the new Guidelines, especially where the assessment of applications by clusters and the Review board is concerned. Presently, applications approved at the country level appear to meet or exceed the level of specificity that is now required by the new Guidelines.

Finding 47. Although the HRF in Ethiopia has developed a set of formal and informal information and feedback mechanisms, post project feedback is insufficient to promote capacity building and information sharing

While most of the Findings are very positive with respect to the work done by the HRF, this one points to a shortcoming in its work processes, albeit one that may be largely caused by factors outside of its direct control.

Notwithstanding the degree of coordination and information sharing which characterizes the HRF in Ethiopia, its fundamental design remains based in an individual project assessment that is generated on a demand-driven basis. Weaknesses in monitoring noted earlier limit the possibilities for lessons learned opportunities and capacity building of NGOs.

Likewise, the fact that the HRF has not conducted systematic follow-up and lessons learned exercises (largely as a result of resource shortfalls) has reduced a vital opportunity for it to strengthen the capacity of the cluster as a whole and build recipient capacity.

FUNDING AND PROJECT ISSUES Finding 48. While HRF in Ethiopia is the largest ERF, its size is not sufficient to meet the chronic

life/livelihood saving crises that are facing the country

While the funding for the HRF in Ethiopia has varied over the years and has considerably increased when the country faced near national levels of crises, it remains, the largest, most sustained among its sister funds.

Nevertheless, virtually all stakeholders indicated that the gap filling demand vastly exceeds the level of resources available. While this Finding is probably not surprising given the chronic nature of disasters in Ethiopia, it is important to underscore that the HRF really does not at this time have sufficient reserves to invest in areas that may be outside of the traditional mandate of filling emergency gaps.

Finding 49. Notwithstanding the above, some long-term activities that border on DRR/resilience have been supported, and while increasing the overall relevance and responsiveness of the HRF to uniquely Ethiopian needs, any drain the capacity to some degree to respond to emergencies.

This Finding speaks to a strategic issue of whether ERFs should be supporting activities more directly related to building resilience at a community level and to some degree supporting DRR considerations. While activities such as providing seeds and shoots to farmers and supporting animal husbandry programming could be seen as being very close to resilience-related activities, they are not so much related to community planning, building networks to promote resilience, and capacity building for DRR. In the Ethiopian context these activities are in fact directly related to livelihood protection and providing a sort of immediate response to mitigate against a chronic pattern of repeating crises. .

A small number of key donors have argued that the HRF in Ethiopia should be more actively engaged in resilience building and DRR-related activities, in line with some global trends, and that preventative measures need to be given priority so as to attempt to avert humanitarian crises. At the same time virtually all stakeholders in Ethiopia, including donors who are more supportive of DRR and resilience building, recognize that the HRF does not have the resources to meet all the emergency humanitarian small to medium size gaps that may arise. Therefore, diluting these resources to any significant degree for projects that would not deliver an immediate humanitarian benefit would seem to be counterproductive.

MONITORING AND EVALUATION Finding 50. There are gaps in current project monitoring that result in lost opportunities for learning at the

project and thematic level.

The HRF in Ethiopia has established wide coverage in project monitoring, with a current monitoring rate in the 75-80% range. The lack of full coverage remains problematic, and is due in part to access constraints and resource levels. More importantly, efforts to promote a more participatory approach to monitoring through peer to peer monitoring generally have been limited.

As noted earlier, while the HRF has played a limited but valuable role in improving cluster coordination, that has not fully translated into wide-spread acceptance of the benefits of peer to peer monitoring by cluster partners. Moreover, the reductions in administrative overheads provided by the HRF seem to have limited the ability and willingness of NGO recipient partners to participate in peer to peer monitoring.


11

This gap is to some extent exacerbated by the fact that the HRF in Ethiopia does not appear to have sufficient resources to conduct comprehensive and thematic follow-up and lessons learned exercises, continuing the global practice of generally seeing HRFs as a series of isolated projects, (with several exceptions as below). Therefore, a variety of opportunities to promote organizational learning and capacity building among NGOs in particular seems not to have occurred.

Finding 51. The HRF Annual Report, while providing a solid basis for basic information sharing at a high level, lacks sufficient monitoring data so as to promote lessons learned.

The HRF in Ethiopia, given the size of the pooled fund itself, has access to levels of administrative resources that vastly exceed those available to the remaining dozen ERFs. This level of resources allows the HRF to have staff and financial capacity that exceeds others and that provides it with an administrative critical mass that is unique.

This has enabled the HRF in Ethiopia to increase the size and depth of the Annual HRF Report to a degree which others cannot do. This clearly promotes more learning and more information sharing and thus contributes to strengthening the transparency and buy-in for the HRF as a whole. Likewise, this administrative critical mass enables the HRF to be able to special reporting about the outputs of the HRF and provide on-going briefings to its partners and to some degree, an overview of the humanitarian challenges in Ethiopia that it faces. However, monitoring data and the general absence of evaluations somewhat limits the utility of these otherwise highly competent reporting tools.

Finding 52. Project duration and level of resources (within project and with OCHA in Ethiopia in general) impede the assessment of beneficiary impact.

Many donor representatives in both Ethiopia and Headquarters are very comfortable with the quality and frequency of HRF reporting. They however are also looking for outcome reporting on consequences of the HRF intervention, so as to demonstrate to their headquarters what actual difference in the human condition has ensued as a result of their support. Whether the donor in question favours a more conventional emergency response paradigm or one more oriented to the promotion of resilience building capacities, there is no difference in the degree to which they are seeking reporting beyond that currently available.

The principle causes for the gap in outcome reporting are twofold. First, size and duration of individual HRF projects in Ethiopia make it very difficult to demonstrate the contribution that an HRF project has made to alleviating an emergency situation and thus improving the human condition from a methodological standpoint. Far too many external factors can influence the course of events and thus the humanitarian outcomes. Second, the HRF as a tool is insufficiently financed at either the project level or the level of administrative set-asides so as to begin to contemplate a rigorous approach to evaluation at the project level. Indeed, given global experience with similar attempts, it is probably unrealistic to conduct individual project evaluations.

Finding 53. Early efforts at sector reporting (water trucking review) show that with modest levels of resources at the country level, evaluation is possible and can go beyond output reporting and process compliance.

The Finding above speaks to the difficulty in undertaking individual project evaluations of the outcomes or impacts on beneficiaries. However, the HRF in Ethiopia has recently undertaken a horizontal review of a number of water trucking projects in drought stricken areas. This was done after cluster discussion surrounding the overall attributes and benefits of water trucking to meet a crisis as opposed to the possibility of supporting longer term measures such as deep bore hole drilling and restoration of water pans.

This horizontal review of water trucking explored the consequences of a number of water trucking projects. It was financed by adding into the budget of an upcoming project, a substantial amount of funds (12% or more of the entire project) to pay for this horizontal study. It should be noted that the magnitude of this line expenditure caused some degree of questioning during the project’s review by Geneva-based staff and an explanation of the strategic nature of this line expenditure had to be provided as it exceed virtually all other levels of expenditure for evaluation in the HRF/ERF context at a project level.

The product of this horizontal review was what in essence would constitute a portfolio evaluation of water trucking activities that has enabled the HRF Review Board and the clusters to seriously examine more broadly alternatives to water trucking as a mechanism to alleviate water shortage in either the short or longer term timeframes.


12

Overall Conclusions The HRF in Ethiopia is a necessary and strongly relevant element of the humanitarian architecture

The Findings above illustrate the degree to which the HRF in Ethiopia, the largest ERF world-wide, plays a crucial role in the humanitarian response in a country prone to continuous and chronic slow onset and rapid onset disasters. Notwithstanding reservations about the balance between emergency gap filling and programming more directed toward resilience and support for DRR, the HRF in Ethiopia has made a significant contribution and remains highly relevant now in its sixth year of existence.

The relevance and effectiveness of this HRF is in large part due to the fact that it possesses a sufficient critical mass of discretionary grant resources to make a major difference that is highly discernible to all actors. The HRF management also has recognized its overall importance and has allocated to it adequate human resources to be able to institutionalize the HRF as an on-going element in the humanitarian system. The processes used in Ethiopia are seldom replicated in other HRFs/ERFs in large part due to the availability of human resources.

The HRF in Ethiopia is both effective and more rapid than most other mechanisms of the UN or bilateral donors

An analysis of recent data shows that Ethiopia HRF is clearly among the most rapid mechanisms of humanitarian response with respect to individual granting activity. As noted above, it is a misperception to believe that the HRF/ERF model can be a near immediate response mechanism. Rather, within the confines of its general placement in the humanitarian architecture globally and in Ethiopia in particular, it plays a highly effective role.

It is very important to underscore that the HRF in Ethiopia, even using what could be potentially time-consuming formalized engagements with the clusters and the Review Board, remains among the fastest world-wide with a current average one month timeframe from formal receipt of application to first payment of 80percent. This duration may be largely influenced by the growing quality and comprehensiveness of HRF project proposals which stand the test of scrutiny both at the country and the Headquarters levels.

The HRF in Ethiopia is well managed and has established a unique pattern of highly effective coordination, consultation/approval, and quality control mechanisms

As should be evident from the above two general a conclusion, the HRF in Ethiopia is well managed and appears to have sufficient resources with the exception of those related to monitoring. The level of resources allocated to informal consultations with stakeholders, and administering the formal approval processes at the cluster and the Review Board levels have strengthened the effectiveness and improved transparency of HRF. It is a visible and valuable element of the overall humanitarian architecture and given its size, has provided the HC with a viable means of promoting greater cluster coordination at the country level, another strategic objective of the HRF/ERF model.

Resource constraints have limited the ability of HRF to demonstrate its results through monitoring and evaluation techniques

As previously discussed, the level of resources available to HRF in Ethiopia are inadequate to ensure either complete monitoring or the exercise of program evaluation techniques which could begin to better tell the full story of the contributions that the HRF programming is making. The recent stipulation in the new Guidelines that individual projects will no longer fund evaluation-related activities is likely to place a severe restraint on this largest HRF which has already experimented with an innovative approach to horizontal portfolio evaluation at the country level.

This new stipulation in essence eliminates the ability of ERFs to earmark the resources to conduct more broadly-based evaluations and individual project evaluations from projects.


13

Country Level Recommendations These recommendations are directed at the country level management of the HRF. Global recommendations, partly drawing on the conclusions of this case study, will be presented through the final valuation report.


Recommendation 1. OCHA in Ethiopia should begin a process of engaging NNGOs in a more proactive fashion so as to ensure their entry into the ERF systems, as has been suggested by many stakeholders.

This initial recommendation speaks to a structural difference between the fund in Ethiopia and several other countries. As noted in the Findings above, national policy and law in Ethiopia have made it problematic for both the HRF and NNGOs to enter into formal arrangements. Put simply, the legal implications for the NNGOs outweigh, in their mind, any benefits for the humanitarian response. INGOs indicated that they frequently are contracting NNGOs at the local level as immediate service providers, thus to some extent being able to counteract current government policy.

As part of the field mission, senior government representatives indicated a willingness to explore with OCHA, the Humanitarian Coordinator and HRF staff, the possibility of engaging NNGOs in a more direct fashion. This level of support was echoed by UN partners and by a number of large size INGOs, all of these recognizing that more actively engaging NNGOs would strengthen the overall humanitarian response due to their field-based presence.

This recommendation, however, needs to be cautiously advanced so as to avoid unduly raising expectations or creating an erroneous understanding of what appears to be an evolving government position. The fact that senior government representatives are willing to discuss arrangements which would better harness the totality of the NGO presence in Ethiopia shows the extent to which the HRF has been able to establish credibility and to begin to lobby or advocate for more comprehensive solutions to chronic humanitarian crises.

Recommendation 2. OCHA in Ethiopia should engage more proactively with cluster partners in order to ensure more consistent follow-on for some approved projects in chronic situations.

One of the predominate discussion items with stakeholders in Ethiopia was the fact that Ethiopia itself is impacted annually by near constant life and livelihood threatening emergency situations. In essence, Ethiopia represents a chronic emergency situation. While this has focussed the ERF primarily to emergency humanitarian responses, stakeholders noted that in some circumstances the gap filling was sequential, being repeated in a number of instances.

While the HRF response mechanisms in Ethiopia are clearly high quality and as timely as is currently possible to contemplate, given the chronic situation that afflicts Ethiopia and the recurring nature of some of the specific crises, the HRF and the HC may wish to explore building into the humanitarian response follow-on procedures that would be implemented by other partners. In essence, the HRF grant in this instance would be the first step of a more substantial response and eventual recovery process. In this way, the HRF would be the first response and OCHA therefore would be better able to coordinate follow-on.

For example, drought in certain regions of Ethiopia is endemic. While HRF emergency responses may have focussed on water trucking or very rapid restoration of facilities, the linking of a first response kind of HRF grant to a longer strategy would better link the HRF to broader issues of the relationship between initial humanitarian responses and overall basic development objectives.

Recommendation 3. In conjunction with other ERFs globally, OCHA in Ethiopia should set aside a small percentage of donor contributions to finance new approaches to evaluation and identification of results, using the recent water trucking sector review as an example.

The HRF in Ethiopia, while possessing one of the highest rates of monitoring among sister funds, has been less successful in undertaking evaluations. For the reasons noted above, the evaluation of individual HRF grants at a country level is problematic, especially if the purpose of the evaluation is to promote organizational learning and to demonstrate outcomes or impact. However, the recent horizontal review of water trucking as a drought response mechanism demonstrates that ERFs can conduct useful exercises in evaluation at a country level. The problem in replicating these studies lies in funding inadequacies.

The new Guidelines, in eliminating the prior line for individual evaluation of projects, (probably a wise choice in its own right) tend to downplay evaluation as a whole, relying on OCHA EGS to undertake the primary activities. This may not be an appropriate course of action especially in the case of Ethiopia. One possible solution would be for


14

the HRF in Ethiopia to develop a modest multi-year evaluation plan which would use the sectoral approach of the water trucking review on which to base in the study of other issues, for example, the benefits of seed distribution, or livestock support. With a modest plan for possibly no more than two such studies per year, OCHA might consider seeking the agreement of its donor stakeholders to specifically set aside the resources necessary to finance this evaluation model from the totality of their contributions to the HRF. Doing so would, to a great degree, respond to donor concerns about how to demonstrate the “worth” of the HRF at a country level and how to strengthen the overall quality of the HRF response through evaluation. It is unlikely that the HRF in Ethiopia currently possesses the resources to undertake even this modest approach to the promotion of learning. Thus if it wishes to do so, it should either seek new or re-profiled resources from donors; or OCHA centrally should provides the resources to do so.

OCHA in Ethiopia may wish to engage in post project lessons learned exercises with both project recipients and clusters (likely by sectors or themes).

While the HRF in Ethiopia uses strong feedback mechanisms including its user survey and its existing range of communications instruments, there is a gap in terms of the extent to which NGO partners are brought together by the HRF to engage in lessons learned or Ethiopian best practices workshops.

The size of the HRF in Ethiopia in comparison to that of virtually all other ERFs lends itself to economies of scale and the opportunity for significant degrees of cross-fertilization. While peer to peer monitoring in some situations can result in a minor degree of information sharing, if one of the implicit objectives of the ERF is to promote NGO capacity building through “learning by doing” as opposed to training per se, then lessons learned exercises would be valuable means of pursuing this goal. However, it is obvious that to do so, adequate communications and liaison resources to mount a lessons learned series would have to be provided. In the present fiscal climate, it is unlikely that these resources would be immediately available. While the HRF in Ethiopia and the HC might wish to identify a non-traditional donor to partner with, for example philanthropic or corporate interests, it is more likely that resources for lessons learned exercises would have to be provided centrally by OCHA as part of an overall response to a number of the communications and organizational learning challenges that face HRFs and ERFs world-wide.

Recommendation 4. OCHA in Ethiopia should explore the possibility of notionally setting aside a small percentage of available funds for more DRR/resilience-related activities, recognizing the primacy in Ethiopia of the need to ensure adequacy of life/livelihood saving programming.

Globally, this evaluation encountered numerous situations where the differentiation between emergency responses and the promotion of DRR or resilience building activities was uncertain. In Ethiopia there was a perception, notwithstanding the extent of gap filling for emergencies, there would be benefit in having at OCHA’s disposal funds to be able to promote resilience building at the local level and delivered by NGO partners. This would, in essence, mean anticipating as opposed to strictly responding.

While this more proactive model would seem to be highly desirable, an excessive focus on promoting proactive resilience measures clearly goes beyond the strategic intentions of the HRF/ERF model. In the case of Ethiopia where there are numerous unfilled humanitarian gaps on an annual basis, care would have to be taken to ensure a sufficient reserve to meet the primary goals of an emergency humanitarian response. Accordingly, it would probably be desirable for the HRF in Ethiopia to actively engage its partners and the government in a consultation process so as to identify the strengths and weaknesses of placing a small percentage of overall HRF resources aside for resilience building activities.

It would be unwise at this time to recommend a specific level for such earmarking. However, from a purely practical sense, an earmark of less than 10% would probably not result in a sufficient critical mass of new projects so as to be able to eventually evaluate their worth. Conversely, earmarking more than 20% would appear on first insight to possibly threaten the viability of the demand-driven funds to respond to emergencies.

Recommendation 5. OCHA in Ethiopia should actively engage its cluster partners in galvanizing cluster support for more peer to peer-based monitoring and generally increased monitoring wherever viable.

This recommendation is very operational and in line with the philosophy that underpins the new Guidelines as well as one that promotes a greater degree of organizational learning. While the level of monitoring of projects in Ethiopia is among the highest on a world-wide basis, OCHA stakeholders as well as those from partner organizations indicated that peer to peer monitoring was not wide-spread. There was a general consensus that this approach to monitoring was probably more effective in a learning and sharing context than in the context of monitoring for compliance alone. However, many stakeholders candidly indicated that for peer to peer monitoring to work, clusters and major NGO partners would have to be convinced to lay aside resources to do so. Again, the benefits of this recommendation are linked to resource availability and to the degree that the cluster system as a whole recognizes the benefits of the HRF mechanism.


15

Recommendation 6. OCHA in Ethiopia, led by the HC, should highlight to OCHA senior management the specific elements of the newly approved Guidelines that would weaken the effectiveness and timeliness of its activities, if so as to better inform the 2013 revision of the existing Guidelines.

During the interviews and meetings in Ethiopia, there was a general apprehension voiced about the impact of the new Guidelines which came into effect in October, 2012, but which have been in discussion for nearly two years. As noted earlier, this country case study is not the place to make recommendations about the Guidelines per se or their impact. However, it is important to stress the degree of concern that was voiced by all categories of stakeholders about their impact.

At present, these Guidelines are beginning to be implemented and as such, it is not possible to assess on an evidentiary basis their real impact. However, it is incumbent on OCHA in Ethiopia to raise, based on evidence, the impact of these guidelines given the commitment of senior management in OCHA to review them in 2013. For example, while NGO stakeholders had apprehensions about the reduction from 10% to 7% for administrative overhead and expressed similar concerns about strictures related to transferring funds from one line to another, there is no hard evidence as yet to show the impact of such changes.

Equally, it is incumbent on OCHA in Ethiopia, given the size of the HRF in comparison to all other ERFs combined, to address issues of a more strategic nature such as how the new Guidelines may result in the limitation of evaluations at the country level and in the fashion the prototype sectoral work that Ethiopia has already undertaken. As well, OCHA in Ethiopia currently possesses probably the most transparent and proactive review process for grants, which has resulted in active review as opposed to passive and which has also generated overall response times that are superior to other ERFs that use the more passive approval processes that the new Guidelines mandate. In short, as a policy matter, OCHA in Ethiopia may wish to strongly advocate, at least for itself, the need to maintain the current system of approvals which, in many respects, is the foundation on which the apparent success of the HRF in Ethiopia is built.


16

Appendix I: Programme Agenda Friday, 5 October 2012

Time Activity Venue / Details Comments

09:00 – 11:00 hrs Meeting with HRF manager OCHA

12:00 – 13:00 hrs Meeting with UNICEF OCHA Narine

Lunch

14:00 – 15:30 hrs HINGO meeting ChildFund Charlie Danzoll Tel. 0911-250738

Confirmed

Monday, 8 October 2012

Time Activity Venue / Focal person Comments

11:00 – 12:30 hrs Meeting with NGO implementing partner

CONCERN office Linda Horgan

Tel: 0911215697

Confirmed

12:30 – 13:30 hrs Lunch

14:00 – 15:30 hrs Meeting with NGO partner IRC office David Murphy Tel. 0911241779

Confirmed

14:00 – 15:30 hrs Meeting with UN women OCHA Georgia McPeak Tel. 931304481

Confirmed

16:30 – 17:30hrs Meeting with HC

HC office Confirmed

Tuesday, 9 October 2012


09:00 – 11:00 hrs Meeting with IMC IMC office Sisay Seyoum Tel. 0911516136

Confirmed

11:00 -12:30 hrs Bilateral meeting with Netherlands Embassy

Embassy of Netherlands Anitra Tel. 011371 1100

Confirmed

11:00 -12:30 hrs Meeting with World Vision World Vision Desta Arega Tel. 0911 840604

Confirmed

12:30– 13:30 hrs Lunch with Jeanine Cooper

14:00 – 15:30 hrs Meeting with GOAL GOAL John Rynne Tel. 0911 214432

Confirmed

14:30 – 15:30 hrs Meeting with UN implementing partner

FAO Adrian Cullis Tel. 0920341384

Confirmed

16:00 – 17:00 hrs Meeting with Government DRMFSS (debrezeit rd) Tadesse Bekele Tel. 0115529388

Confirmed


17

Wednesday, 10 October 2012


09:00 – 10:30 hrs Meeting with Cluster leads (Dale)

OCHA (ENCU, UNICEF, FAO, UNHCR, WFP, UNDP, WHO, UN Women)

UNDP – apologized

UN women – confirmed

WHO – confirmed

WFP – confirmed

ENCU – confirmed

FAO – Confirmed

UNHCR - confirmed

09:30 – 11:00 hrs Meeting with OFDA (Scott) OFDA Kate Farnsworth Tel. 0911511422

Confirmed

11:00 – 12:00 hrs Meeting with UN IPs (Emergency Nutrition Coordination Unit)

OCHA Isaack Manyama Tel. 0913202151

Confirmed

12:30– 13:30 hrs Lunch

14:00 – 15:00hrs Bilateral meeting with DFID OCHA Shaun Hughes Tel. 0910 103 949

15:00 – 17:00 hrs Meeting with HRF donors OCHA Swiss – confirmed (Serge)

Netherlands – Confirmed (Geert Geut and Anitra)

Norway – confirmed (Johan)

Ireland – confirmed (Anne)

Spain – confirmed (Alessandra)

DFID – (Shaun)

Sweden – (Aklog)

Italy – (Fabio Melloni)

17:00 – 18:00 hrs (after donors meeting)

Meeting with SIDA OCHA Aklog Laike Tel. 0115180000

Confirmed

17:00 -18:00 hrs (after donors meeting)

Meeting with Swiss OCHA Serge

Confirmed

Thursday, 11 October 2012


09:00 – 10:30 hrs Meeting with ECHO ECHO Laurent Saillard Tel. 0911 255 600

Confirmed

09:15 – 10:45 hrs Meeting with ERCS ERCS Afework Teshome Tel. 0115-159858

Confirmed

11:00 - 12:30 hrs Meeting with WFP WFP Andreas Hansen Tel. 0922446642

Confirmed

11:00 - 12:30 hrs Bilateral meeting with NGO CRS Carlos Sanchez Tel. 0112788819

Confirmed

14:30 – 16:30 hrs HRF Review Board meeting OCHA

17:00 hrs - Meeting with UNICEF UNICEF Shadrack Omol Tel. 0911 254011

Confirmed


18

Friday, 12 October 2012

Time Activity Venue Comments

09:00 – 10:00 hrs Meeting with IOM representative

IOM Josiah Ogina Tel. 0911202777

Confirmed

11:00 -12:30 hrs Debriefing /ppt presentation OCHA

12:30 -13:30 hrs Lunch

14:00-15:00 hrs Meeting with Plan International Plan International David Throp Tel. 0911 206759

Confirmed


19

Appendix II: Stakeholders • Alemayehu, B • Plan International Ethiopia

• Ali, M • Mercy Corps

• Arega, D • World Vision

• Asianyan, N • UNICEF

• Bacha, Z • IMC

• Bekele, T • DRMFSS ( Government of Ethiopia)

• Bezabeh, S • CRS

• Cooper, J • OCHA, Rep to AU

• Cullis, A • FAO

• Danzoil, C • ChildFund

• Davis, M • CRS

• DelConte, D • OCHA

• Farnsworth, K • OFDA ( USAID)

• Firke, D • SPO

• Getahua, S • CRS

• Geut, G • Embassy of Netherlands

• Gezahegn, N • World Vision

• Girmai, H • Irish Aid

• Haines, A • Irish Aid

• Hansen, A, • WFP

• Hartevelt, I • IFC

• Horgan, L • CONCERN

• Hughes, S • DfID

• Kumar, P • CONCERN

• Laike, A, • SIDA

• Lasada, A • Embassy of Spain

• Manyama, I • UNICEF

• McPeak, G • UN Women

• Mekonnen, T • IMC

• Melloni, F • Embassy of Italy

• Mondiwa, A • Help Age

• Murphy, D • IRC

• Ogina, J • IOM

• Omol, S, • UNICEF

• Owusu, E, • UN RC/HC

• Patenaude, S • SDC

• Rynne, J • GOAL


20

• Saillard, L • ECHO

• Sanchez, C • CRS

• Seyoum, S • IMC

• Teshome, A • ERCS

• Throp, D • Plan International

• Yemarra , H • IMC


The Global Evaluation of Emergency Response Funds (ERFs) Country Case Study Report - Indonesia

March 2013


i





ii


Disclaimer .................................................................................................................................................................... i




Introduction ................................................................................................................................................................. 1


Methodology ............................................................................................................................................................... 2

Overview .................................................................................................................................................................... 2

Country Case Study Selection ............................................................................................................................ 2

Limitations/Challenges ............................................................................................................................................... 2


Limitations .......................................................................................................................................................... 2

The ERF in Indonesia – An Overview ......................................................................................................................... 4

Findings ...................................................................................................................................................................... 6

Overarching Conclusion ............................................................................................................................................. 6



Relevance ........................................................................................................................................................... 7

Timeliness .......................................................................................................................................................... 7

Outcomes ................................................................................................................................................................... 8

Leadership and Coordination .............................................................................................................................. 8

Critical Gaps Filled.............................................................................................................................................. 8

Partnership ................................................................................................................................................................. 8


Review of Proposals ........................................................................................................................................... 9


Mainstreaming Gender ....................................................................................................................................... 9

Operation of Advisory and Review Boards ......................................................................................................... 9




Strategic Recommendations .................................................................................................................................... 12

Operational Recommendations ................................................................................................................................ 13




iii

Abbreviations CERF Central Emergency Response Fund

DRR Disaster Risk and Reduction



FM Fund Manager



HO Humanitarian Officer


IMDFF DR Indonesia Multi Donor Fund Facility for Disaster Recovery


M&E Monitoring and Evaluation





OFDA Office of Foreign Disaster Assistance


RBM Results Based Management


SIDA Swedish International Development Agency


UN United Nations


WASH Water, Sanitation & Hygiene


1

Introduction This is one of five Country Case Study Reports that are mandated as a contributing part of the Global Evaluation of ERFs, which is taking place in the latter half of 2012. This case study report reflects the findings arising from a 10 person day mission to Indonesia by a staff of two, one from Universalia and one from OCHA’s Evaluation and Guidance Section (EGS). Details of the mission can be found in Appendices I and II.

This Country Case Study Report is firmly focused on conditions in Indonesia. While it is obvious that global conditions and the overall management system of the ERFs impacts on the delivery of services in Indonesia, this report will not directly address these global issues. Rather, how these global considerations affect the performance of the ERFs as a whole will figure largely in the overall report of this evaluation
















.


2

Methodology OVERVIEW In our proposal, the methodology proposed for the assignment consisted of a mixture of qualitative and quantitative data collection, with attention both to forward-looking, improvement oriented questions and to assessment of impact. In summary, we will follow the OCHA Evaluation Policy objective of determining:




Country Case Study Selection

Based on initial document review and suggestions received from our key respondents during the Inception Phase, five field missions were conducted in countries affected by humanitarian natural disaster or in complex emergencies. Assessments of two of the ERFs, in oPT and Columbia, have been requested by in-country stakeholders, and three, Ethiopia, Indonesia and Afghanistan have been selected by the evaluation team taking into consideration size, type of emergency, country context, donorship profile, and evaluations and reviews previously conducted.

LIMITATIONS/CHALLENGES


This is one of five field mission reports for the Global Evaluation of ERF. It is not a specific and detailed evaluation of the HRF in Indonesia, but an overview report. In light of the approved methodology, this report drills down using Indonesia as an example to explore at a practical level the key evaluation issues. Given the resources available for it (see below) it is unreasonable to consider that such an overview and summary of the HRF in Indonesia could be considered to be an exhaustive. Specifically, this overview does not attempt to provide a comprehensive assessment of the HRF’s overall performance. Rather it highlights the nature of HRF behaviors and practices within the context of the key evaluation issues.

Limitations

The evaluation team wishes to thank the staff and management of the HRF in Indonesia for developing a comprehensive program, which included an extremely valuable field mission to Jakarta which resulted in the team having the opportunity to meet with nearly a dozen INGOs, several of whom were represented by multiple participants.

Timing constraints, given only a one week window of opportunity, reduced the team’s ability to meet face-to-face with some of the stakeholders identified by the country staff, largely due to schedule conflicts. Nevertheless, the breadth and scope of those engaged is clearly sufficient to ensure triangulation of data as well as a variety of sources of opinion.

The documentary data provided by the HRF staff more than met the requirements of this evaluation. However, the small number of projects over the last several years makes it difficult to disaggregate any trends within this data other than the most basic with respect to accuracy and timeliness.

There were several more strategic limitations that impacted on the evaluation. The agreed upon methodology of allocating about the same amount of time for data collection in each of the five locales proved to be somewhat problematic in that HRFs/ERFs are vastly different in size, ranging from ones like Indonesia whose annual project expenditures are in the range of US$1.25 million to Ethiopia where its Review Board on a frequent weekly basis regularly assesses that value of projects.


3

As well, the Terms of Reference for the assignment and the evaluation questions laid out in the TOR appear to have been based on some form of standardized ERF/HRF and were not sufficiently discrete to address the implications of such a degree of difference in magnitude. The evaluation team worked around this latter challenge by adapting interview protocols to more adequately reflect the small size of the HRF in Indonesia.

.


4

The ERF in Indonesia – An Overview The Humanitarian Response Fund (HRF), previously the Emergency Response Fund (ERF), was first introduced in Indonesia in 2001 to address gaps in humanitarian response. The HRF facilitates quick disbursement of funds to enable NGOs to respond to unmet emergency needs. The Fund’s name was changed in 2010 to reflect a wider scope of assistance towards disaster management. The HRF also supports activities that promote durable solutions for displaced persons and targeted communities by meeting humanitarian and early recovery needs, particularly in areas where funding gaps exist. By providing NGOs with a rapid and flexible funding mechanism, the Fund meets the short-term emergency needs of vulnerable communities.

In 2011, the HRF disbursed a little over US$1 million to address the lingering needs of vulnerable populations following the Mt. Merapi eruptions, the Mentawai Island earthquake and tsunami, and the Wasior flash floods.

The HRF in Indonesia is one of the smallest currently in operation. Established in 2001, it is presently the longest continually operating ERF. Given its duration, and the chronic disaster prone nature of Indonesia where Level 3 emergencies occur every several years and over 100 Level 1 and Level 2 emergencies occur on average each year, it is paradoxical that the HRF has undergone a steady reduction in size of available resources and number of donors, with it having only one donor, Sweden, since 2008. The table below shows overall donorship patterns over the last several years. This table reflects a local custom in Indonesia to attribute specific projects to specific donors and also the fact that Sweden has been the sole donor since 2008. Exhibit 7 Donorship Patterns

Donors Funds (US$) Project Period Number of Project

UK Department for International Development 2,863,900 2002-2006 42

Australian Government’s Overseas Aid Program 999,000 2003-2006 14

The Kingdom Foundation, Saudi Arabia 266,576 2007 5

OPEC Fund for International Development 499,975 2007-2008 8

Swedish International Development Agency 1,115,375 2007-2008 15

Swedish International Development Agency 1,876,460 2009 - 2010 21

Swedish International Development Agency 2,070,150 2010 22

Swedish International Development Agency 3,068,650 2011-2012 22 *

TOTAL 12,791,436

149

The table below shows the magnitude of Sweden’s contribution up to and including the current year. However, it is essential to note that for 2012, Sweden did not make a separate annual contribution. Rather, given a low rate of disbursement, the second half of Sweden’s 2011 contribution was re-profiled into 2012. Although there indications that Sweden may provide on-going funding, no decisions have been taken as yet.

Thus the HRF fund faces the real possibility of having no new funding for upcoming years and thus basically falling dormant. At present (March 2013), the fund has not received any new support, even from Sweden. However, the Government of Sweden has agreed to again re-profile the remaining balance of its contributions to allow for some degree of activity in 2013 in the absence of any new funding from any source. Exhibit 8 Sweden’ Contribution

SIDA funding to Indonesia HRF

2007- 2012

Funding (SEK) Funding (US$) Project Period No. of ERF projects

7,500,000 1,115,375 2007-2008 15

13,000,000 1,876,460 2008-2009 21

15,000,000 2,070,150 2010 21

20,000,000 3,068,650 2010-2012 11 in 2011 5 in 2012 (up to June)

Source: OCHA contribution tracking system


5

Although the HRF has only supported a small number of projects in 2011 and again in 2012, the map below shows the geographic distribution of HRF-supported activities which range across many islands of Indonesia. This map also shows the basic distribution of HRF grants across sectors and illustrates the degree to which over two-thirds of the funding has been directed in recent years to resilience building activities, or activities undertaken in a recovery context, both being somewhat at the outside limits of the current global Guidelines regarding the choice of programming.

The HRF in Indonesia probably has directed more of its efforts to attempt to provide access to NNGOs. The information for the past two years shows a majority of the small number of projects being ones initiated by NNGOs. The data for 2010 in particular reflects the impact of the Mt. Merapi eruption and the role played by INGOs in a more immediate response environment.

Number of projects per year:

• 2012: 11 (5 by Int’l, 6 by National) • 2011: 11 (5 by Int’l, 6 by National) • 2010: 22 (18 by Int’l and 4 by National) • 2009: 21 (19 by Int’l and 2 by national)


6

Findings The following set of specific findings is derived from the questions contained in the approved Evaluation Matrix, which can be found as Appendix III.

The findings that follow are fully triangulated, many times combining documentary evidence with testimony from several different groups of stakeholders. It also should be noted that the team that mounted the Indonesia case study was comprised of Mr. Dale Thompson, the overall Team Leader and Mr. Scott Green, Head of OCHA’s EGS. Messrs. Green and Thompson participated in all the interviews and group meetings. Moreover, in late February 2013, an inspection mission from FCS was conducted, the conclusions of this mission being discussed with the author of this case study on March 4-6. There was a great deal of similarity in terms of findings.

OVERARCHING CONCLUSION Like many ERH/HRF mechanisms the level of staff available is inadequate to meet the extent of the expectations that are involved. The level of liaison and coordination activities at the country level probably are more important that grant processing. Yet, OCHA itself does to appear to have recognized this at the Strategic Level. Likewise, the growing global demand for greater accountability and transparency has not been translated into adequate resources for monitoring, let along evaluation at the country level.

Many of the specific Findings below are the direct consequence of a general inadequacy of resources and although specifically cast in terms of Indonesia echo findings made in other country case mounted for this Global Evaluation.

OPERATIONAL IMPACT Finding 1. The HRF has made a minor but valued contribution to the overall humanitarian response,

however has not been able to expand beyond a narrow base.

Although very small in size, the HRF in Indonesia has been able to make a valuable albeit limited contribution at the national level, namely filling unforeseen gaps in the instance of humanitarian emergencies. The Indonesia HRF has been able to do so in two contexts.

First, in the larger Mount Merapi eruption, a disaster that attracted world-wide attention, the HRF was able to fund nearly a dozen NNGOs to support land and livelihood restoration efforts. Second, more recently and with an even more limited budget, the HRF has been able to support flooding and storm damage ranging across the nation from the Thousand Islands north of Jakarta to Ambon and Sulewasi.

Both types of intervention however are limited by the historic small size of the HRF, now in fact shrinking to approximately US$1.25 Million for the current year, with no clear indication of continuity even from the current single donor, Sweden. Indonesia experiences over 100 Level 1 and Level 2 disasters per annum and while the national response capacity has steadily grown, there is clearly a role for the HRF especially in relation to the support of NNGOs.

Finding 2. The HRF does not appear to have had an effect on the overall level of humanitarian financing.

There is very little evidence that this small HRF has had any catalytic effect in relation to any increase in the overall level of humanitarian support for Indonesia, one of the central objectives of all HRF/ERF mechanisms. One former donor to the HRF, Australia some five years ago transferred its support away from the HRF to provide core funding for the OCHA office in Indonesia as a whole so as to ensure that OCHA would have the means to lead the humanitarian community in the Indonesian context. The rationale for this action was that the chronic disaster prone nature of Indonesia required an on-going OCHA presence even if a Level 3 emergency was not immediately at hand.

They also pointed out that the fundamental nature of the HRF and the way it has to respond to emergencies within the context of the national planning framework led to what they considered to be excessive time delays between the identification of an emergency situation and OCHA having the ability to respond to it.

Likewise, New Zealand, in partnering with the Government to establish a new pooled fund to support resilience building and DRR planning, indicated that in their view, these activities were more suited to the needs of Indonesia as a middle income country than episodic and somewhat after the fact project support at a very small level provided by the HRF. It should be noted that while this new pooled fund has only one donor, New Zealand, other major donors are considering joining. It also should be noted that this fund has begun to disperse DRR-related support. This may in some contexts may result in an overlap with the smaller HRF.


7

Indeed, by March 2013, the fund itself had not even been able to secure any new resources for itself, even from its long-standing donor, the Government of Sweden. Rather, Sweden agreed again to re-profile the remaining balance of its prior to allow a degree of activity in 2013.

Finding 3. The direction of the fund in Indonesia is not well understood.

The above Finding points to a challenge that faces the HRF in Indonesia. Virtually all groups of stakeholders, NGO, Government and donors alike are not clear about the direction of the fund.

This has led the HRF be seen by outside stakeholders and partners as to be ambiguously situated among the range of humanitarian instruments. This problem is in large part due to the lack of resources for the fund itself. Notwithstanding past and present communications efforts to publicize the fund, this finding is triangulated by virtual unanimous testimony from UN and donor stakeholders.

OPERATIONAL EFFECTS

Relevance

Finding 4. In comparison to some other local sources, the HRF is seen as responsive to NGO needs (esp. NNGOs).

Notwithstanding the above considerations, the HRF is viewed by both UN and NGO stakeholders and especially NNGOs, as considerably more flexible and responsive than other granting mechanisms. While US AID’s OFDA has the ability to, in essence, advance considerable sums of money to a few NGOs so as to provide for immediacy of response, other donors both UN and bilateral generally lack the capacity to respond even within the timeframes of the HRF (see below).

More importantly, the fact that the HRF offers the ability to offer a more holistic mix of activities, some relating to resilience building and some related to restoration and gap filling even within the same project, increases its relevance especially to NNGOs who may be more attuned to the breadth of challenges that exist at the local level. However, this obvious benefit is hampered to a large degree by the current size of the fund and the relatively small size of the projects (see below).

Finding 5. Some NGO stakeholders have had some concerns about targeting, communications and transparency.

All NGO stakeholders, both INGOs and NNGOs, were supportive of the HRF in general. While NNGOs indicating that it provided them one of the few opportunities to access internationally-based funding, there was an undercurrent of concern about how decisions were taken at the HRF. The current model of assumed consent by the Advisory Board was not well understood, with some NGO stakeholders knowing that the engagement of the Advisory Board was scant and seeking a more visible level of transparency.

This Finding again is likely the product of the level of internal resources available in Indonesia for the HRF, a thread which runs through many subsequent Findings.

Timeliness

Finding 6. The HRF response, while seen as slow by some, is more timely than other sources.

In Indonesia, like in many other HRF/ERF locales, considerable gaps in statistical data before 2010 make analysis and cross-analysis problematic. It should be noted that the administrative reforms undertaken by OCHA and the policy reforms undertaken by FCS in particular have considerably rationalized data.

For the period 2011 and 2012 to date, the HRF in Indonesia has averaged approximately two days between receipt of a grant application and its onward approval by the HC. This is faster than many other ERFs but it also points to a challenge which will be addressed in subsequent Findings – quality control. There is however, somewhat of a delay in the next administrative step, namely the processing of the grant application in Geneva. For this same time period, some 23 days on average have been required, although grants processed in 2012 are considerably below this average, in the 10-14 day range. Based on discussion with applicants and staff, this average is in large part caused by the frequency of the need to provide additional information. Finally, for this time period, it takes on average 11 days to process the first (80%) payment. Again, it should be noted that this duration is falling sharply in 2012 to the single digit range.

Finding 7. INGOs are comfortable with HRF processes, even the timelines, but NNGOs are less satisfied.

In spite of what some stakeholders perceive as delays in approval, and what others have seen as delays in final auditing, INGOs in particular recognize that the ERF is more timely than many of the other instruments at their disposal. Their ability to call upon global reserves has enabled them to start earlier; and their relative cash strength has limited the impact of prior audit delays. Such is not the case with NNGOs. The NNGO stakeholders engaged in


8

this evaluation were nearly unanimous in indicating that what they perceived to be unnecessary delays in approval impacted on their ability to deliver the services. However, on careful inspection, given that projects were being put forward sometimes months after the incident in question, such reservations may not truly be relevant.

OUTCOMES


Finding 8. Information sharing with NGOs is limited in terms of follow-up and lessons learned processes led by HRF personnel.

The level of the administrative budget of the HRF in Indonesia has resulted in an inability to share information. While basic information is available via quarterly bulletins, there is only scattered evidence of specific follow up and lessons learned exercise, key elements in any learning based approach.

Recently, fund staff had to draw back a plan to conduct orientation briefings in areas other than Java. This clearly limited the capacity of the HRF to reach out to NNGO stakeholders who are more likely not to have representatives in the capital. Likewise, the possibility of bringing together recipient groups so as to share experiences and in some way build capacity has not ensued due to funding limitations. It is interesting to note that the mission to engage about a dozen stakeholder groups in Jogakarta who had provided services post the eruption of Mt. Merapi, was the first time that these groups had come together either before starting programming or more importantly, after.

While there had been other OCHA missions to the areas post Mount Merapi, when directly asked all participants at the Jogakarta workshop confirmed that this was the first time they has been brought together. Needless to say, these stakeholders in particular indicated that they would welcome more opportunities to share experiences.

Critical Gaps Filled

Finding 9. Although critical gaps were filled to some degree, project size is somewhat smaller on average that of other ERFs - lack of critical project size (the HRF itself has been very small).

Given the size of the HRF in Indonesia, a decision was taken several years ago to limit project size to a maximum of US$100,000. Of course, the global stipulation with respect to six months duration is in force in Indonesia.

While a project of this size may respond to some of the needs of smaller scale emergencies, NNGO stakeholders who received support post Mt. Merapi eruption indicated that their need was greater than that which could be supported by US$100,000. On balance however, gaps, even small ones were filled, but probably not fully.

More importantly however, UN stakeholders tended to consider that while the HRF had value, projects of this size lacked the critical mass so as to be catalytic. Several stakeholders indicated that the Government decision-making processes combined with the size of the HRF made it very difficult to expect that such small projects, implemented somewhat after the fact, could be expected to catalyze or even be closely linked to broader objectives.

Finding 10. DRR considerations constitute the majority of recent programming choices and as such may exceed the bounds of the intent of the HRF/ERF model. This policy challenge is to some degree the result of it being supported by one donor; and to a larger degree, the result of government-wide policy direction.

The Government of Sweden, as a policy issue, is firmly committed to the support of resilience building and DRR-related activities as a primary objective; namely being proactive as opposed to reactive. Likewise, the Government of Indonesia is internationally recognized as being one of the leading middle income nations in the support for proactive approaches to disaster management.

These two factors, have led the HRF over the last several years, toward granting a majority of its funds (and a majority of the projects supported) to projects that are clearly proactive in nature or which seek to build DRR capacity after the fact. In this context, this small fund is now in some ways competing with other larger initiatives including the new pooled fund. This course of action also is the cause for the degree of ambiguity about the very purpose of the fund that was identified in earlier Findings. While there is no doubt that this element of the project portfolio is making a positive contribution in its own right, the question remains as to whether a fund of this size makes any real and sustained contribution in the Indonesian environment, and even whether it is cost effective.

The relatively lengthy Government decision-making process has resulted in the HRF supporting small projects, on an individual basis, and with very little possibility of linking them to larger or more sustained initiatives so as to promote synergies, catalytic funding and in the longer term, sustainability- all key goals of the HRF/ERF mechanism.

PARTNERSHIP Finding 11. OCHA is generally seen as a valued partner in terms of the HRF.


9

Even in the face of the considerable funding limitations that challenge the HRF, it is seen by many stakeholders as a good partner. This level of support appears to be concentrated among both INGO and NNGO partners. The NNGOs in particular see OCHA as a very valuable partner and one which can provide them access to sources of international funding, UN, that they otherwise would not have at their disposal. NNGOs also were particularly supportive of the efforts of the local Indonesian staff of the HRF and the degree to which they have been responsive and helpful.

UN system partners, however, while recognizing the contribution, albeit small, that the HRF has made, are not as certain about the quality of partnership with the HRF. There was some degree of apprehension expressed by some partners about the level of OCHA’s communications with respect to the role of HRF and or its general directions. Government stakeholders appeared to be supportive of the HRF as a partner in the context of it being a relatively easy to access fund although a very small one.

PROCESSES AND INPUTS

Review of Proposals

Finding 12. The existing quality control mechanisms are weak

The HRF/ERF process as set out in the new Guidelines contains a number of steps designed to ensure quality control as well as relevance. In Indonesia, due to the factors described above, many of these steps are either missing or not operational, notwithstanding staff assertions to the contrary.

The ambiguous and infrequent ( in the immediate present context) Advisory and Review processes limit the ability of the small HRF staff to have access to a critical and external assessment of the relative worth of the projects. Likewise, as will be described below, budget cuts have reduced monitoring to the range of approximately 70%, thus limiting, especially for NNGOs the opportunity to benefit from mid-course advice and correction. This level of monitoring is directly attributable to budget cuts over the last several years.

Finding 13. OCHA staff levels are minimal (two part time staff).

Aside from the part time participation of the Humanitarian Officer (HO), the current staff of the HRF in Indonesia consists of a part time fund manager (FM) and a part time administrative assistant. While the recent IOSS audit set down some suggestions for staff size for ERF funds, the practicability of these recommendations is problematic in many instances. However, as has been evidenced by a number of the above Findings, the current staff size, along with administrative budget, is inadequate to meet even the demands of this small fund. It should be noted however that this issue of lack of staff is endemic among many HRF/ERFs worldwide


Mainstreaming Gender

Finding 14. Gender and other disadvantaged groups have received a high degree of consideration.

Among sister HRFs world-wide, the HRF in Indonesia has one of the highest ratings over the last two years with respect to the Gender Marker. Project recipients as well as UN partners indicated that OCHA had taken special efforts to highlight the importance of these cross-cutting considerations. In that context and although quite small, the HRF in Indonesia has shown that gender considerations can be successfully integrated into a variety of project types in more than a nominal or symbolic way.

Engagement with NNGO stakeholders in particular illustrated that these positive words were being translated into concrete actions.

Operation of Advisory and Review Boards

Finding 15. Advisory and Review Boards are not engaged thus limiting responsiveness and outreach.

The Advisory Board has not met for some time with the most recent effort being several years ago and being unsuccessful. Likewise, the assumed consent model used by the Review Board disaggregates the Board and leaves the possibility open, because of the nature of the assumed consent model, that no one has actually reviewed a project proposal. Staff noted that some projects had been approved without any feedback within the set timeframe.

These gaps in the HRF architecture are more than irritants to day-to-day operations. They represent fundamental gaps in the ability of the HRF to be relevant. To some extent, it can be argued that these gaps are contributing factors to the lack of understanding about the HRF in general.

Finding 16. In relation to the HRF mechanism in particular, the current approach to the two Boards deprives feedback to OCHA on its performance and direction and diminishes accountability.


10

The present state of both the Advisory and Review Boards, beyond infrequent engagement on an assumed consent basis has resulted in a gap in the range of performance measurement techniques available to the HRF. It appears that Boards have not recently played a challenge role in terms of the assessment of performance on a country basis. In making this statement, it should be emphasized that the concept of performance being discussed herein is a qualitative value adding and learning based concept, and is not the compliance-based approach to financial and programmatic performance which tends to predominate thinking about the ERF/HRF mechanism all across OCHA.

MONITORING AND EVALUATION Finding 17. There are gaps in current project monitoring that result in lost opportunities for learning at the

project and thematic level.

With a current monitoring rate in the 70% range, the HRF in Indonesia has established coverage; however the lack of full coverage in itself remains problematic. More importantly, efforts to promote a more participative approach to monitoring through peer-to-peer monitoring generally has not occurred due to resource limitations. The recent reductions in administrative overheads provided to the HRF are limiting the ability and willingness of NGO recipient partners to participate in peer-to-peer monitoring.

This gap is to some extent exacerbated by the fact that the HRF in Indonesia clearly does not appear to have sufficient resources to conduct comprehensive and thematic follow-up and lessons learned exercises, continuing the global practice of generally seeing HRFs as a series of isolated projects. Therefore, a variety of opportunities to promote organizational learning and capacity building among NGOs in particular seems not to have occurred.

Finding 18. Project duration and level of resources (within projects and with OCHA in Indonesia in general) impede the assessment of beneficiary impact.

Many donors world-wide and Sweden in particular, have been vocal in seeking from HRFs/ERFs additional performance information related to the impact of programming on beneficiaries. Globally, this presents a systemic challenge to OCHA. In Indonesia, given the overall size of the fund and the very small size of grants (sub US$100,000) the challenge to go beyond simple output reporting on an individual project basis may be insurmountable from a resource consideration and either irrelevant or cost prohibitive from methodological considerations.

Finding 19. No efforts at sectorial evaluation (review of shelter as a whole, WASH, etc.) as other ERFs have done.

The resource levels of the HRF in Indonesia appear to make it impossible for it to have considered attempting to conduct cross-project evaluations, for example a selection of WASH initiatives over several years, as have occurred elsewhere. While these resource limitations are clearly apparent and clearly have stymied the ability of the HRF to go beyond basic output performance, and have even limited its ability to conduct complete monitoring, there are numerous opportunities in Indonesia, even in its resource-constrained environment to have reviewed similar projects on a sectorial basis.

For example, the number of similar projects that were funded in relation to the Mt. Merapi eruption would have lent themselves to a sectorial review and thus would have provided an opportunity for a more qualitative assessment of performance along with opportunities for improving capacity and the sharing of relevant information. In short, due to the apparent absence of funding, an important opportunity for learning was not undertaken.


11

Overall Conclusions A sufficient critical mass does not presently exist.

As the Introductory Remarks about the nature of the HRF in Indonesia showed, the fund is very small, has become reliant on only one donor and has not been able to secure any additional resources as yet for the 2013 and onward period of time. The effect of this, with the exception of the Mt. Merapi eruption, has been a fund that distributes in the US$1.25 Million range per annum and issues less than 15 contracts on average per year. Limited usage of the crucial mechanisms to build outreach and partnership like the Advisory and Review Boards has further weakened the HRF.

These basic conditions are further exacerbated by limited staff resources that have resulted in limited outreach and communications activities. New pooled funds that have engaged both prior donors and other cluster partners further complicate an already problematic situation.

All these factors lead to a difficult but necessary conclusion, namely: the HRF in Indonesia lacks a critical mass not only in relation to funding but also in relation to the identification of its role within the humanitarian community in Indonesia and even the articulation of its very purpose within that community.

The potential to expand or even sustain current resource levels appears not to have been explored actively.

Presently Sweden is the only donor to the HRF and has been such for the last five years. Australia, previously a donor to the HRF has chosen to provide OCHA with core funding as opposed to supporting the HRF directly.

The general climate in Indonesia in relation to emergency preparedness and emergency response is in large part influenced by the Government of Indonesia’s firm commitment to resilience and DRR-related activity. As well, Indonesia’s status as a middle income country results in its having sufficient resources to respond to levels of emergencies that in other lower income countries would require external support. The impact of this reality is that there are inherent difficulties in securing donor support for multilateral programming in such environments, in the face of overwhelming demands for support from lower income countries. This however, does not fully explain why the HRF in Indonesia appears not to have been able to attract even one additional donor at a nominal level over a period of five years.

A small niche is being served, but this value added role is not well understood.

In conclusion, the HRF in Indonesia appears to be an incomplete project and one, if it is to be prolonged, which will require a considerable degree of re-invigoration. The calculus for OCHA centrally and or in Indonesia is stark. Is maintaining an HRF in Indonesia worth it? Many factors tend to answer this question in the negative. There is however, one very key positive value added.

The HRF in Indonesia, albeit very small and under-resourced, has played a different role than other bodies in its ability to support NGOs, both INGOs and NNGOs. Although the rate of participation of NNGOs has tapered off slightly over the last year, the level of NNGO participation in HRF grants on a percentage basis exceeds that of virtually every other HRF/ERF. The NNGOs clearly indicated that their capacity to plan and manage projects was increased by participating in the HRF and by delivering programmatically and thus “learning by doing”. Even with serious limitations of size, the HRF staff have attempted to address the geographic diversity of Indonesia with a number of small projects in various outlying regions. Efforts are being made and a niche is being served, albeit a small niche which is considerably underfunded. Again, it should be remembered that on an annual basis, there are over 100 natural disasters a year in Indonesia. Notwithstanding the capacity of government agencies, it is reasonable to assume that a slightly better funded HRF would be capable of responding to a number of these situations.

As well, and as noted immediately above, the development of a new pooled fund led by the Government of Indonesia but with an advisory body which includes the HC has complicated the understanding of the HRF. While the HRF has been moving toward a resilience building / DRR focus, its role is not well understood and its primary role of gap filling has been blurred to a considerable extent thus reducing its effectiveness and its relevance.


12



Before laying out specific recommendations, given the nature of the Findings relative to Indonesia, it is probably necessary to set out some overarching conclusions about the value of the HRF in the context of Indonesia:

• First, while a small niche is being filled, especially in relation to NGOs and small-scale resilience building, the level of expenditures in recent years has been so small so as to not contribute to the broader aims of the HRF/ERF mechanism in relation to leadership and coordination in the humanitarian sphere.

• Second, the lack of donor support, as is evidenced by Sweden being the sole donor and only having agreed to re-profile lapsing funds for 2013, illustrates that notwithstanding the strong role of the government of Indonesia, there does not appear to be consistent support to maintain the HRF at any reasonable level of visibility or programmatic effectiveness.

• Third, the HRF, while admittedly serving a small number of NGOs, has been and will continue to be important only for a very small number of NGOs in comparison to the large number of INGOs and NNGOs that operate in Indonesia.

Therefore, and in light of the provisions of the new Guidelines with respect to the closure of an ERF, it would appear that the HRF in Indonesia is a prime example of a mechanism which has outlived its primary usefulness and which unless considerably reinforced in terms of finances and staffing, would have little chance to play a significant contributing role. In short, unless higher level considerations come into play, the HRF in Indonesia probably should be closed.

STRATEGIC RECOMMENDATIONS The following Recommendations are presented in the eventuality that OCHA decides to continue the HRF or at least to give it a period of grace so as to re-establish itself. These Recommendations are extremely serious and onerous. Simply providing the prior level of donor support ($1.5 Million and one donor) would be inadequate, given the magnitude of the challenges that the mechanism faces. Likewise, the HRF in Indonesia must be re-targeted and clarified in order for it to have any opportunity to make a discernible contribution to the humanitarian response in Indonesia as a whole.

Recommendation 7. Recruit more donors if a decision to continue the HRF is taken.

As was shown above, the HRF in Indonesia does not possess sufficient critical mass nor does it have a sustainable donor mass. Indeed, the resources that it currently possesses have already been re-profiled, and will lapse at the end of 2012. While there is some indication that Sweden, the sole donor since the departure of Australia five years ago, may provide some degree of limited support, the inescapable reality is that a one donor to HRF in the range of less than US$1.5 Million per year is neither sustainable, nor cost-effective.

Therefore, notwithstanding any specific recommendations related to the operation of the HRF, OCHA managers have an initial and in essence, mandatory responsibility, to seek to replenish the fund and to broaden the donor base. In the absence of such efforts, it would seem only rational to proceed to wrap up the HRF as soon as possible, in line with some of the specific guidance laid out in the new globally applicable Guidelines.

Recommendation 8. Clarify the focus of the HRF in Indonesia.

Assuming the HRF can be re-financed, or as part of the re-financing, OCHA management in Indonesia needs to engage its stakeholders in a process of prioritization. The current HRF is largely directed toward very small scale resilience building programming; while in the immediate past given the Mt. Merapi eruption, more immediate life and livelihood saving activities occurred.

It should be noted that in Indonesia, over 100 Level 1 and Level 2 natural disasters occur annually. While the Government of this middle income country has grown in capacity to respond to many of these crises, stakeholders, especially NGOs point out that the HRF could have a greater role to play if it was sufficiently financed.

Another challenge of the HRF that needs to be remedied is its overall size. Put simply, a handful of annual grants in the sub US$100,000 range regardless of their focus do not constitute a sufficient critical mass. Therefore, along with clarifying the focus of the HRF and returning it to the greatest degree possible to the gap filling model, serious consideration about project size needs to be taken.


13

OPERATIONAL RECOMMENDATIONS The following set of Recommendations assumes that OCHA has decided to continue the HRF and that the Strategic Recommendations presented above have been fulfilled.

Recommendation 9. Clarify future / role of HRF vis-à-vis that of other emerging pooled funds (IMDFF DR).

As noted in the Findings, a new pooled fund has emerged in Indonesia that is under the aegis of government disaster management authorities and which has an advisory committee composed of government and UN partners, including the HC. This fund, although presently supported by New Zealand, is likely to gain immediate support from other donors. Virtually all stakeholders who were aware of this new fund commented that the relationship between it and the HRF remains unclear and that additional efforts needed to be taken to clarify this relationship and to ensure that overlap and duplication did not exist. The fact that there was near unanimity among external stakeholders shows that prior efforts, however cast, have yet to result in a clear picture.

Recommendation 10. Restart and more directly engage the Advisory and Review Boards.

The Findings above demonstrated the degree to which the HRF in Indonesia has become disconnected from its consultative bodies. The absence of meetings in the recent past, combined with the passive assumed consent model for project ratification, have in effect disengaged OCHA’s HRF in Indonesia from the very stakeholders it needs to engage if it is to be renewed in a sustainable fashion. Specifically, it is recommended that the Head of Office convene a meeting of the Advisory Board as soon as possible in 2013 and make special personalized efforts to ensure a high level of attendance. The HOO may wish to consider as the key agenda item, the renewal and redirection of the HRF, as well as collective efforts in recruiting donor support for it. The Advisory Board also need to be physically convened to better address project selection technique (this latter suggestion being a common one for all ERF/HRF mechanisms at the country level).

Recommendation 11. Engage in more active information sharing with clusters and among NNGO/INGO recipients.

The current level of resources allocated to the staff of the HRF and for administrative purposes is inadequate. If the OCHA Head of Office considers that the HRF still remains a viable instrument, additional resources will be necessary to broaden the base of the HRF, especially in the ways of how it engages with its non-UN stakeholders. NNGOs and INGOs both reported the degree to which they felt there was a need for communications and networking, activities which have not been able to be supported in the past. Likewise, if the HRF is to be rejuvenated, senior staff resources will need to be allocated for a more proactive approach to higher level communication (regarding the HRF).

Recommendation 12. Strengthen OCHA internal capacity for planning, quality control and M&E.

All the above recommendations point to an inescapable reality: the current level of resources, both human and financial, allocated to the HRF is inadequate. Even if the HRF were to be refinanced at its current minimal level and the present level of administrative resources retained, this level itself is inadequate in every context. Liaison resources at present levels are nominal. Quality control and advance coordination with potential NGO partners is limited. Adequate monitoring cannot occur. Any efforts at evaluation so as to show the work and contribution of the HRF are non-existent.

This final Recommendation also addresses the interplay between the size of an HRF and the level of resources that are required to make an HRF of any size viable. While recent audits suggested three levels of notional human resources, these levels appear to have been arbitrarily determined. A much more country specific approach to resource allocation needs to be undertaken and while this country level report will not make recommendations at the global level, this does not preclude the HRF staff along with the Head of Office to undertake an objective needs assessment for Indonesia in terms of internal resources based on a minimal scenario which might be renewing the HRF at a level below US$2-2.5 Million per annum in contributions. It should be noted however that these conditions and gaps are not unique to Indonesia. Rather they affect nearly all HRF/ERF mechanisms worldwide.


14

Appendix I: Programme Agenda Monday, 5 November 2012

09:00 – 11:00 Meeting with OCHA Indonesia

Venue: OCHA office

Participant: Ms. Knarik Kamalyan, Ms. Arie Kustanti and Iwan

12:00 - 13:00 Lunch

14:00 – 15:00 Meeting with Jakarta BPBD (Provincial Disaster Management Agency)

Venue: Jl. Medan Merdeka Selatan

Participant: Mr. Edy Djunaedy and Iwan

15:30 – 16:30 Meeting with National NGOs

Venue: Aceh Room, 7th Fl. Menara Thamrin Building

Participants: MITA, HFI, NU, PKPU, PADMA

Tuesday, 6 November 2012

09:00 – 10:00 Meeting with International NGOs


Participants: Action Contre le Faim, Church World Service, Habitat for Humanity, Hope Worldwide, Islamic Relief, Muslim Aid, Plan International, Save the Children, World Vision

10:30 – 12:00 HRF Board Members / Cluster Leads


Participants: FAO, IFRC, UNDP, UNICEF, UNFPA, UNHCR, WFP, WHO

12:00 – 13:00 Lunch

13:30 – 14:30 Meeting with AusAID

Venue: Cyber 2 Building, Rasuna Said

Participant: Mr. Jeong Park, Mr. Ben O’Sullivan

15:00 – 16:00 Meeting with Sida

Venue: Menara Rajawali, 9th Fl. Kuningan

Participant: Ms. Annika Siwerz

Wednesday, 7 November 2012 9:00 – 10:00 Meeting with NZAid

Venue: Sentral Senayan 2, 10th Fl, Gelora Senayan

Participant: Mr. Kirk Yates

10:30 – 11:30 Meeting with USAID

Venue: Jl. Medan Merdeka Selatan No. 5

Participant: Mr. Harlan Hale

12:30 – 13:00 Teleconference with Board Member – Wayne Ulrich IFRC

13:00 – 14:00 Lunch

14.00 – 14.30 Meeting with Board Member – Budhi Ulaen UNDP

15.00 – 15.30 Teleconference with Board Member – Lina Sofiani UNICEF


15

Thursday, 8 November 2012 5.30 To airport, heading to Yogyakarta

Lion Air JT 552. ETD 07.35 ETA 08.45

10:00 – 13:00 Meeting with National NGOs in Yogyakarta

Venue: tba

Participants:Inprosula, PALUMA, YEU, Satunama, Dejarup, IDEA, and Yayasan Dian Desa

16:00 To Airport, heading to Jakarta

Lion Air JT 551 ETD 17.40 ETA 18.55

Friday, 9 November 2012 09.00 – 10.30 Exit and debrief with OCHA Indonesia

11.00 – 12.00 Meeting with BAPPENAS

Venue: BAPPENAS office


16

Appendix II: Stakeholders • Ali, A • PKPU

• Apsoro, H, • Habitat for Humanity

• Arimbi , ? • DEJARUP ( Yogyakarta)

• Ashio, H • PALUMA ( Yogyakarta)

• Azhima, R • YEU ( Yogyakarta)

• Bayu, ? • MITA

• Djunaedy, E • BPBD ( Gov of Indonesia)

• Edo, S • YOD ( Yogyakarta)

• Gengaje R • HoO OCHA

• Haid, M, • Muslim Aid

• Hamm, C • HOPE World Wide

• Huwlyani, S • PALUMA ( Yogyakarta)

• Kamalyan, K • OCHA

• Karim, B A • FAO

• Narjowae, S • IDEA ( Yogyakarta)

• Novirani, D • YEU ( Yogyakarta)

• O’Sullivan, B • AUSAID

• Park, J • AUSAID

• Purwari, A • SATUNMA ( Yogyakarta)

• Rahman, M • Muslim Aid

• Satria, D • Church World Services

• Shafikin, N • WHO

• Sidabe, R • SATUNMA ( Yogyakarta)

• Siwerz, A • SIDA

• Soedjowoo, R • YOD ( Yogyakarta)

• Sumuan, B • World Vision

• Surya, R • HFI

• Wawan, Y • Islamic Relief

• Widowah, ? • HFI

• Wiji, D, V • IDEA (Yogyakarta)

• Yates, K • NZ AID

• Youianus, T • PADMA

• Yuniarti, W • WHO


The Global Evaluation of Emergency Response Funds (ERFs) Country Case Study Report - Occupied Palestinian Territory

March 2013


i





ii


Disclaimer .................................................................................................................................................................... i



Abbreviations ............................................................................................................................................................. iv

Introduction ................................................................................................................................................................. 1


Methodology ............................................................................................................................................................... 2

Overview of global approach ...................................................................................................................................... 2

Case Study Methodology ........................................................................................................................................... 2

Emerging Issues ......................................................................................................................................................... 3

Where do ERFs Fit? ................................................................................................................................................... 3

The Critical Mass of ERFs .......................................................................................................................................... 3

The Transformative Agenda ....................................................................................................................................... 3

The Consequences of ERFs ...................................................................................................................................... 3

The Focus of ERF Activity .......................................................................................................................................... 3

The Life Cycle of ERFs ............................................................................................................................................... 4

Gender and Disadvantaged Persons Considerations ................................................................................................. 4

The ERF in the Occupied Palestinian Territory .......................................................................................................... 5

Findings ...................................................................................................................................................................... 8



Relevance ........................................................................................................................................................... 8

Flexibility ............................................................................................................................................................. 9

Timeliness ........................................................................................................................................................ 10

Coverage .......................................................................................................................................................... 11

Outcomes ................................................................................................................................................................. 11

Critical gaps filled ............................................................................................................................................. 11

Leadership and Coordination ............................................................................................................................ 12

Partnership ............................................................................................................................................................... 12

Processes and Inputs ............................................................................................................................................... 13

Needs assessment and application .................................................................................................................. 13

Review of proposals ......................................................................................................................................... 14

Timely allocation and disbursement ................................................................................................................. 15

Funding and Project Issues ...................................................................................................................................... 16

Mainstreaming gender ...................................................................................................................................... 16

Cost efficiency .................................................................................................................................................. 16

Operation of advisory board ............................................................................................................................. 16

Monitoring, Evaluation Reporting and Audit ............................................................................................................. 17




iii


Appendix II: Stakeholders Met .................................................................................................................................. 23


iv

Abbreviations ACF Action contre la faim (Action Against Hunger)

ACTED Agency for Technical Cooperation and Development

ARA Access restricted areas

CERF Central Emergency Response Fund

CISP Comitato Internazionale per lo Sviluppo dei Popoli



GVC

HC

Gruppo di volontariato civile

Humanitarian Coordinator





MAAN MAAN Development Center






PNGO Palestinian Non-Governmental Organizations Network




UN United Nations


WASH Water Sanitation and Hygiene

WHO World Health Organization


1

Introduction A Global Evaluation of OCHA managed Emergency Response Funds (ERFs) is being conducted during the latter half of 2012. As part of this exercise a case study of the Occupied Palestinian Territory (oPT) Humanitarian Response Fund (HRF) was conducted between the 1st and 9th October 2012, which is reported on in this document . An additional four country case studies are being conducted in Ethiopia, Afghanistan, Columbia and Indonesia.

This Country Case Study Report focuses on the operation of the HRF as a response to the humanitarian crisis in the oPT. The report findings, conclusions and recommendations are directed primarily to stakeholders at the national level. The implications management of the ERFs at the global level will be consolidated and presented as part of the global ERF evaluation report.

Universalia wishes to thank the management and staff of OCHA, that of its UN systems partners, donor representatives, national authority stakeholders and the representative of the NNGOs and INGOs who have generously given their time and openly shared their opinions. Without them, this evaluation as a whole would not be possible.

PURPOSE OF THE EVALUATION The objectives for the evaluation, as given in the overall TOR, are given as:

• To provide an independent assessment of the contribution of ERFs to the humanitarian community’s ability to address critical unforeseen humanitarian needs in a timely and effective manner;

• To examine the contribution of ERFs to strengthening the leadership of Humanitarian Coordinators, coordination role of the cluster system, and building partnerships (in particular with national and local NGOs);

• To examine the role of the elements of OCHA that plan, administer and report on ERFs, map the progress made since 2009 in improving the effectiveness and efficiency of ERFs, and identify the areas of internal strengths and weaknesses.

The TOR laid out a set of Key Areas to be examined. These are to scrutinize:





• Internal (e.g. staffing capacity, management arrangements) and external (e.g. contextual and political) factors which affect ability of ERFs to deliver on their objectives;






2

Methodology OVERVIEW OF GLOBAL APPROACH The methodology for the global evaluation consists of a mixture of qualitative and quantitative data collection, with attention both to forward-looking, improvement oriented questions and to assessment of impact. This follows the OCHA Evaluation Policy objective of determining:


The general steps for the overall evaluation include the following steps:


CASE STUDY METHODOLOGY A range of approaches were used to assist in answering these questions, including:

• Semi-structured interviews with key informants and stakeholders. Selected respondents included the HRF fund management team, the Humanitarian Coordinator (HC) other OCHA staff (HoA, FCU staff), project implementing partners (NNGOs, INGOs and UN agencies – including agencies who had applied unsuccessfully for funds), donors (both contributors and non contributors), cluster leads, and representatives of the Palestinian Authority. This group of stakeholders covered the membership HRF Advisory Board and Review Board.

• Consulting a large number of project reports and documents. These included oPT HRF guidelines, monthly and annual progress reports, Advisory Board minutes and minutes from various workshops and other associated meetings. At the project level the evaluation reviewed sampled proposals, documentation on project review processes, memorandum of understanding, monitoring reports, final narrative and financial reports, and audit reports.

• A field visit to a sample of HRF projects in the West Bank to examine the implementing context and beneficiary perceptions.

To ensure the accuracy and validity of study findings, the evaluation used a process of triangulation. The findings of the study do not rely on the opinions of individual interlocutors, but reflect interlocking evidence drawn from multiple sources.

At the end of the field mission, the preliminary findings were presented to OCHA and the Advisory Board members in Jerusalem. The comments received during this debrief have been incorporated into the case study report.


3

Emerging Issues In the evaluation inception process several key strategic issues emerged. While these issues are for the most part echoed in the TOR (see Appendix IV), it is useful to highlight and reiterate these topics as these provide additional core direction to the evaluation as a whole.

WHERE DO ERFS FIT? Probably the most widespread concern raised relates to where ERFs fit in the totality of the humanitarian response, and the difference between ERFs and other pooled funds. Institutional stakeholders very much seek an evaluation that will address the strategic positioning of country level pooled funds and especially those that have civil society, as opposed to UN organizations, as their recipients.

While the overall issue of the design of pooled funds in general is beyond the mandate of this evaluation, the TOR permit us to give strategic emphasis to this issue due to its elements related to clusters and overlap in particular. The impact of this strategic issue implies that we intend to give increased emphasis to exploring strategic considerations at one level and focusing our work in this area at the country level on the tactical and operational considerations related to day-to-day cluster activity and field level instances of potential friction.

THE CRITICAL MASS OF ERFS There was broad consensus among all groups of stakeholders that there were questions of the critical mass of ERFs and that these questions existed in two dimensions. First, whether ERFs as a whole were large enough in comparison to the whole of the humanitarian response to constitute a significant critical mass that could make a difference through engagement with civil society. Secondly, are individual project sizes sufficiently large to produce results that can be sustained? This issue as a whole, therefore, probably is best addressed at the country level; with some consideration of the impact of the overall architecture of the humanitarian response at the global level, an echoing of “where do ERFs fit”.

THE TRANSFORMATIVE AGENDA The IASC Transformative Agenda is the set of actions within which OCHA and the humanitarian system is collectively working to improve leadership, coordination and accountability of humanitarian response. These include: A mechanism to deploy strong, experienced senior humanitarian leadership to guide the humanitarian response; The strengthening of leadership capacities and rapid deployment of humanitarian leaders, to ensure the coordination architecture functions well; Improved strategic planning at the country level; Enhanced accountability of the Humanitarian Coordinator (HC) and members of the Humanitarian Country Team (HCT) for the achievement of collective results; and streamlined coordination mechanisms. OCHA’s ERFs need to be placed strongly within the context of the Transformative Agenda, in particular where OCHA demonstrates progress made through the Transformative Agenda which also impacts OCHA’s role as both a coordinator and source of funds.

We recognize that the Transformative Agenda is a relatively new initiative and that the more retrospective elements of our evaluation will not be relevant to it. However, given that this evaluation is firmly forward looking, the potential impacts of the Transformative Agenda on ERF will be explored at the Headquarters and field level both.

THE CONSEQUENCES OF ERFS Although the TOR clearly recognized the difficulties of attribution, we encountered a fairly consistent set of observations that called for this evaluation to at least begin to explore the consequences of ERFs. Donors in particular were interested in having some greater understanding of how ERF projects actually made a difference, as opposed to strictly delivering what was contracted for. However, given the scope of the evaluation, we will likely have to rely on the testimony of NNGO and INGO recipient bodies, as well as that of OCHA staff.

THE FOCUS OF ERF ACTIVITY It has become apparent that a number of donors and stakeholders other than those from OCHA had some concerns about perceived mission creep with respect to the focus of OCHA’s ERF activities. While nearly all were supportive of the existence of a timely and flexible fund directed toward civil society recipients, there was some concern that overlap and duplication of functions was occurring or that the ERFs, especially those of some longevity, were moving away from a paradigm based on emergency response to issues of human suffering and threats to life, toward a protracted response paradigm, blurring the line between the humanitarian response as a whole, and development. The most notable example given of this issue related to the role that ERFs may be playing in disaster responses and the promotion of resilience, areas most stakeholders took to be part of a protracted relief paradigm.


4

THE LIFE CYCLE OF ERFS It is apparent that the ERF process is rapidly evolving and that conditions that might have prevailed two years ago in response to the 2007 Formative Evaluation, probably have been superseded by more recent developments and even today, by the anticipated promulgation of a set of ERF guidelines and tools. Most stakeholders identified flexibility as a key characteristic of a responsive approach and were of the view that timeliness and local sensitivity were key positive characteristics of ERFs. However, few understood how ERFs came to be in the first place and how OCHA manages their life span. This again also echoes the strategic issue of “where do ERFs fit”. It also addresses the perception of some stakeholders that one of the benefits of an ERF is to provide medium level donors with a mechanism to funnel resources to civil society groups in instances where the donor does not have the infrastructure itself to respond to the emergency. This therefore, leads to the issue of what are the costs and benefits of accommodating donors in this manner; and also, can ERFs be used as a mechanism to encourage broader donor pools, or potentially even new donors, other than those from OECD nations.

GENDER AND DISADVANTAGED PERSONS CONSIDERATIONS We came to recognize that the management and administration of the ERFs had to take into account considerations of gender equality and the impact on women and girls, children in general, and the elderly.. To this end, the evaluation team will give special emphasis to how ERF related activities and procedures address the needs of females, children and other disadvantaged groups, who in many instances are impacted more seriously in humanitarian emergencies than other types of people. For greater clarity, the notion of other disadvantaged groups may, in certain circumstances under review, include ethnic or religious minorities.


5

The ERF in the Occupied Palestinian Territory The humanitarian crisis in the oPT is foremost a human rights crisis, grounded in the on-going conflict and occupation. Restrictions on the movement of goods and people have negatively impacted on peoples’ livelihoods and the ability to deliver basic services. In Gaza, there are severe restrictions on access to areas with the most viable agricultural and fishing prospects and there are chronic problems in service delivery of health, education and adequate water for drinking and personal hygiene. In the West Bank Palestinian movement is also severely impeded by a variety of mechanisms. Palestinians in Area C continue to be displaced due to demolitions of homes and schools by Israeli forces and are vulnerable to Israeli settler violence. Bedouin and herder communities are amongst the worst affected groups.

In response to this protracted crisis the Consolidated Appeal Process (CAP) 2012-2013 articulates a two-year strategy of the humanitarian community to tackle the most urgent humanitarian needs. This CAP focuses on two strategic objectives: enhancing the protective environment including access to services, and tackling food insecurity. It targets the most vulnerable communities with priority in the Gaza Strip, the Area C of the West Bank, the Seam Zone and East Jerusalem, where the Palestinian Authority has limited or no access.

It is widely accepted that a resolution to the crisis is dependent on progress in the peace process. In the interim, humanitarian assistance is programmed to prevent a further deterioration of the situation. A primary entry point for humanitarian aid is protection from violence, displacement and dispossession. Consequently education, health and nutrition and WASH services are creatively aligned with promoting the rights-based agenda.

There have been growing concerns in the oPT about the dependence of vulnerable populations on emergency assistance and the urgency of offering Palestinians more sustainable solutions. Consequently humanitarian organizations are striving to become more strategic in their provision of relief assistance and draw a clearer line between emergency programmes, recovery and development interventions. As the latter projects are being excluded the size of the CAP is shrinking. The current CAP request is more than 25% lower than the previous year, and the lowest since 2006.

Exhibit 1 CAP Funding Trend Analysis

The 2012 CAP requested US$416.7 million to implement 149 relief projects in 2012 (17 from local NGOs, 84 from international NGOs and 48 from UN agencies). The CAP is currently 66% funded, with contributions of US$273 million reported from donors. This currently makes the oPT the best funded – in proportional terms – of all the 23 active CAP appeals.

Within the larger humanitarian response, the oPT Humanitarian Response fund (HRF) is designed as a complement the CAP process. It was created in August 2007 with the support of Sweden and Spain. The HRF is managed by OCHA oPT, on behalf of the Humanitarian Coordinator (HC), with the technical support of the humanitarian cluster/sector leads and member agencies.


6

Funding is provided primarily to NGOs, but there is the flexibility to fund UN agencies if needed. There are two priorities – or objectives – for HRF funding:

• To provide an immediate response to unforeseen emergencies through life saving interventions and/or preventing the further erosion of livelihood assets and coping mechanisms of the affected communities;

• To strategically fill gaps in funding within a humanitarian response plan until main-stream humanitarian funding is available.

The primary focus of the HRF has been on the first objective - to respond to unforeseen emergencies – defined as events that fall between CAP planning cycles. These have traditionally included a range of agro-climatic shock and pest and disease outbreaks, such as:

• Insufficient rainfall: The average rainfall during the 2010-2011 season was below average with an uneven distribution. The effects of this were compounded by restrictions on water projects and infrastructure imposed by the Israeli authorities. The HRF contributed to mitigating the effects of water scarcity by providing subsidized clean water and fodder distribution.

• Pest and disease outbreaks: Responses included vaccination campaigns against Chlamydia and foot and mouth, management tuta absoluta virus in tomato crops and red palm weevil.

• Severe windstorms: Repair of greenhouses destroyed by a storm in December 2010 in Nablus. • Flooding: Flood damage affected the Gaza strip in 2010.

Over the last year the definition of unforeseen emergencies has been strategically expanded to include response to the demolition of residential and livelihoods-related structures in area C of the West Bank and other consequences of settler violence. More than 387 demolitions of Palestinian structures by Israeli authorities were recorded between January and September 2011. The HRF supports reconstruction efforts, with the principal aim of protecting livelihoods and preventing further population displacement.

In addition, the HRF also supports ‘gap filling’ of other critical needs. The HRF is very conscious of avoiding potential competition with the CAP and a usual selection criterion is that the ‘gap’ falls outside of the CAP. Examples of gap filling projects include; assisting children suffering from genetic disorders, enabling water and sanitation facilities to function and reducing health risks due to inadequate wastewater treatment in refugee camps. The HRF has been used to rehabilitate and construct temporary classrooms and facilities for schools in East Jerusalem and Area C.

In 2011 the HRF received 42 project proposals for a total amount of $8,394,062.Of these 42 proposals, 14 projects were approved at a total amount of US$2,667,112 allowing the HRF in 2011 to provide direct assistance to 624,731Palestinians. WASH was the biggest recipient of funds, followed by Agriculture. The majority of funds were disbursed to INGOs, although the share directed to NNGOs has increased significantly in 2012, with 16 of the 18 HRF projects approved so far being managed by NNGOs.

Exhibit 2 2011 HRF projects by sector and partner

Sector Projects US$

WASH 5 1,170,542

Agriculture 4 909,028

Health and Nutrition 1 222,036

Shelter & NFIs 2 214,481

Education 2 51,025

Total 14 2,667,112

Recent donors of the HRF also include Ireland, UK, Norway, Netherlands and Switzerland, who have collectively contributed US$ 23.3 million to the ERF over the last five years (see Exhibit 3). A comparison with the CAP funding (see Exhibit 4) shows that ERF funding was the equivalent of only 0.8% of the funding provided to the CAP over this period.


7

Exhibit 3 Funding to oPT ERF by Donor (2007-11)

Between 2007 and 2011 the ERF approved project funding for a total of US$11.4 million. Expenditure peaked in 2009 at US$5.4 million in response to operation Cast Lead. As contributions have consistently exceeded expenditure the fund has built up a large reserve of over US$11m by 2012. The Advisory Board has encouraged this as a large planned reserve against potential larger-scale unforeseen emerging needs.

Exhibit 4 ERF Contributions compared to expenditure


8

Findings The following set of specific findings is derived from the questions contained in the approved Evaluation matrix, which can be found as Appendix III.

OPERATIONAL IMPACT Finding 20. The HRF has a comparative advantage in addressing small-scale unforeseen emergencies and

protection issues, in partnership with NGOs.

The HRF in oPT plays an important and unique role in supporting the overall humanitarian response in the oPT. Particular strengths of the mechanism have proved to be in relation to:

• Effectively responding to small-scale unforeseen emergencies that have developed in-between funding CAP funding cycles, such as weather and disease events;

• Working through a collective instrument to address sensitive areas of protection, including interventions in Area C of the West Bank; and

• Maintaining a strong reserve to buffer immediate response to potential large-scale unforeseen shocks.

No evidence was found of significant unmet critical humanitarian needs. The evidence base does not allow a more precise judgement on either the severity of the assessed needs or the specific role of the HRF in addressing these needs. However, it is evident that the HRF has resulted in additionality of action. The HRF has facilitated interventions to take place that otherwise would not have happened. The impacts of this have been disproportionate to the relatively modest levels of HRF funding within the overall humanitarian response.

An important element of the uniqueness of the HRF instrument has been providing access to NGOs for emergency response funding. This has primarily benefited INGOs, but is increasingly being channelled to NNGOs.

Within the overall humanitarian response significant funding gaps remain in relation to (i) funding of NGO projects within the CAP, (ii) funding of preparedness activities, in particular those relating to protection, and (iii) basic service delivery. To some extent the ERF has served as a funder of last resort and assisted in addressing critical gaps in response.

Finding 21. The HRF has had a positive, but modest, effect in reinforcing the Humanitarian Reform process. The HRF is not a primary instrument for developing partnership.

The impact of the HRF in reinforcing the Humanitarian Reforms has been more modest. The existence of the HRF is widely perceived to positively reinforce both humanitarian leadership and coordination. However, given the scale of the funding, the HRF has limited influence on these processes. The main tangible benefit has been encouraging the active engagement of a selected group of NNGOs in cluster led processes.

The role of the HRF in promoting partnership has been ambiguous; not least because the goals of improved partnership itself are not clearly articulated within the humanitarian system in oPT. The goal of partnership appears to be interpreted in oPT as a comparatively narrow one of building NNGO capacity as an exit strategy for other humanitarian agencies. A broader notion of the HRF being used as a tool to support more collaborative efforts among UN bodies does not seem to have taken hold; not has the notion that the HRF indirectly strengthens the clusters.

The challenge may in large part relate to the degree to which the HRF is seen largely as a granting mechanism and not as a mechanism that can be used to strengthen the overall UN response.

OPERATIONAL EFFECTS

Relevance

Finding 22. The HRF has been in relevant in responding to quick onset disasters and maintaining a reserve to respond to major new emergencies.

The oPT is primarily experiencing a rights-based, rather than needs-based, crisis. Globally, humanitarian aid is more commonly aligned to meeting needs. In the oPT humanitarian needs – as measured by standard indicators and standards – typically fall well below accepted thresholds for emergency intervention. Given this context there is a continued questioning of the role of humanitarian action against other more political instruments. Consequently the HRF has to define a rather different role compared to other humanitarian crises. However, there is agreement that the HRF has managed to successfully fill a relevant niche.

The HRF has proved relevant in addressing a range of quick onset shocks, including water shortages, floods, storms, and pest and disease outbreaks. The heavy constraints that the political context places on access to


9

services and longer-term development make it necessary to continue to respond to these events through an emergency instrument. Whilst the consequences of non-intervention are not generally life threatening, the HRF does act with success to stabilize livelihoods.

The lack of comparable and transparent needs assessment makes it hard to objectively verify the extent to which the portfolio of projects reflects the most pressing needs. Although outside of the scope of this evaluation it is noted that there may be scope to improve the objectivity and comparability of needs assessments.

The HRF also maintains an important reserve function against possible future major emergencies. A worst case scenario could see a rapid escalation of needs following an earthquake or major escalation of violence. Whilst the HRF itself does not have the independent capacity to manage a major crisis, it would contribute to short-term bridge funding until alternative mechanisms such as a flash appeal or CAP revision occurred.

Finding 23. The HRF has acted flexibly to address protection, but these needs may ultimately be better addressed through preparedness funding.

The HRF is attempting to further strengthen its relevance by making a stronger link to protection goals. A major focus of the HRF in the last year has been to rebuild structures demolished by the Israeli administration Area C. The primary objective is to prevent displacement of communities, by re-establishing the means to continue their livelihoods. The multi-donor and multi-agency nature of the HRF has been critical in allowing it to address this need. The political ramifications of these actions would expose a bi-donor and implementing agency to a potential level of confrontation that most seem unwilling to risk. The pooled nature of the HRF spreads this risk.

However, there are questions on whether the HRF is the most appropriate way to respond to household level events. There is a general agreement that it would be preferable to establish contingency, or preparedness, fund which could be called down much more rapidly to respond to these idiosyncratic events. There is a strong argument for providing funds in advance as the continuing levels of demolitions can be reasonably forecast. A contingency fund would relive the responding agency of the need to formulate a specific proposal around individual event and significantly cut the response time. Preparedness funding could logically be included within the CAP.

Flexibility

Finding 24. The HRF has struck a fair balance between flexibility and maintaining a focus on its core mandate and areas of comparative advantage.

The challenge for the HRF lies in striking an appropriate balance between defining a core identity – which is partly defined with reference to global norms – and adapting to the demands of the local situation and diversity of stakeholders. Here it has appeared to achieve a reasonable balance.

The HRF has maintained a focus on the primary objective to “provide an immediate response to unforeseen emergencies through life saving interventions and/or preventing the further erosion of livelihood assets and coping mechanisms of the affected communities”. There was a strong awareness amongst all stakeholders of this core objective. Whilst the arguments could at times appear somewhat contrived – especially in relation to a ‘life saving’ goal, there was a general agreement on the need to align proposals to this objective including the livelihood protection element. Given the nature of the crisis in the oPT, it is unclear why would ‘live-saving’ be imposed as a criteria. It seems that an unnecessarily restrictive option was selected.

Considerable flexibility has been demonstrated in the definition of trigger events. In the oPT context this has been expanded from traditional agro-climatic events and disease outbreaks to include idiosyncratic events such as demolitions in Area C. As described above the HRF has some unique advantages in responding to these events.

The small scale of the HRF funding has been a further contributor to its flexibility, allowing it to support well defined and local needs that other funding sources often struggle to deliver.

There is a potential opportunity to invoke more flexibility under the second objective to “strategically fill gaps in funding within a humanitarian response plan until main-stream humanitarian funding is available”. This provides a safety valve both in terms of meeting any critical unmet needs, as well as allowing the HRF mechanism to tick over in the absence of unforeseen events. Indeed, given the build-up of unspent reserves, there was some demand from donors to loosen the funding criteria and take fuller advantage of this objective.

A further example of flexibility relates to the debate on opening a temporary, 3rd objective within the HRF to fund “preparedness” activities. The argument was that as this element of the CAP had remained unfunded, the HRF should finance this activity as a critical gap. This again raises the question of the relationship of the HRF to the CAP. Ultimately the need for this particular objective dissipated as DG ECHO provided ACTED with contingency funding for this purpose.


10

Timeliness

Finding 25. To be ‘fit for purpose’ a substantial reduction in the time taken to approve proposals and release funds is still required.

As an emergency response to unforeseen shocks it is imperative that the HRF is agile and releases funds in a timely manner. Consequently performance on this issue was closely examined during the case study.

The analysis looked at two main phases in the overall sequence from proposal development, to the release of funding. Phase one is from the identification of the shock or problem to the submission of a formal proposal to OCHA. Phase two is from the receipt of the proposal to the release of funds to the implementing agency. This initial breakdown provides a useful distinction – as OCHA can be held more directly accountable for delays in the second phase, but has much less influence on the first.

Estimates of the duration of the first phase relied on personal recall and estimation. These were drawn out by discussing specific projects. Agencies estimated this process as typically requiring a range of four to six weeks.

Fairly accurate data on the duration of the second phase is available from OCHA ERF records (see chart below). This indicates that the average time lapse between the submission of the proposal and signing of the MoU was two months. In total this suggests that from problem identification to the release of funds takes on average in excess of three months.

Exhibit 5 Timeline for approval of 2011 HRF projects (average in days)15

Despite this extended timeline, there was widespread agreement that this is better than other available funding sources. The main alternative for funding emergency response is bi-lateral funding from donors such as DG ECHO or a sub-contract with UN agencies. These sources were estimated to require 6 months or more to mobilize. Agencies may have other quick response funds available, for example from unrestricted internal funds, but these are rarely mobilized in oPT as more acute needs elsewhere tend to take precedence.

However, the core question is whether a 3-4 month delay is acceptable. As emergency response in oPT is rarely about saving lives, the bench mark for acceptable timeliness is whether this is sufficient to protect communities, maintain livelihoods and prevent displacement. The answer here was mixed.

For several ERF interventions the delay in disbursement was deemed acceptable. For example, as the ICRC provide tents as short-term shelter or relatives may host families for a period affected by demolitions, the structures may be rebuilt over time. Greenhouses only needing repair prior to the next growing season. For other responses timing was more critical – for example families without adequate shelter at the onset of winter or timely vaccination campaigns to stop the spread of disease.

Coverage 15 Source OCHA oPT


11

Finding 26. The HRF provides good coverage against its objective of reaching major population groups facing life-threatening risk.

The HRF is a relatively small fund in absolute terms, with expenditure of US$2.7 million in 2011. It is also relatively small in relation to the total oPT humanitarian budget – equivalent to just over 0.5% of the CAP appeal in 2011. However, judged against the objective – of meeting critical unforeseen needs and filling gaps outside of the CAP – this budget has proved adequate. In both geographical and sectoral terms there was general agreement that the most pressing needs are generally being met. The cluster leads did not argue for an increase for immediate emergency response.

The planned reserve of US$5 million, although untested, appears to offer a reasonable buffer against possibly larger future shocks.

Currently there are no other active pooled humanitarian funds in oPT for the HRF to link to. A Country Humanitarian Fund (CHF) has not been established, nor has the option been widely discussed yet. In such an instance however, the existing HRF would be subsumed by a CHF. A request was made to underfunded element of the CERF at the end of 2010, but all CERF projects have now been closed.

Finding 27. Linkages to other transitional or development funds or processes are weak, largely as a consequence of the operating environment.

An important challenge for the HRF is achieving better linkages between ‘emergency’ response, and rehabilitation or development. This is rarely achieved in practice. The problem is not a lack of awareness or interest in developmental outcomes, or a willingness of donors to fund developmental outcomes, but an environment that does little to permit longer-term development. As a consequence, ERF projects have rarely progressed beyond self-contained interventions, often destined to be repeated as an emergency response rather than transitioned to other more appropriate and sustained services. At best there may be a transition from HRF funding to other sources of larger-scale or longer-term humanitarian funding.

OUTCOMES

Critical gaps filled

Finding 28. The HRF serves as a ‘funder of last resort’ but is poorly adapted to filling gaps beyond unforeseen emergencies.

In addition to meeting unforeseen needs the HRF has a second objective “to strategically fill gaps in funding within a humanitarian response plan until main-stream humanitarian funding is available”. This leads to a rather grey area between the boundaries of the HRF and other funding streams including the CAP and other transitional funds.

The principled approach taken by OCHA is that the HRF should not compete with the CAP process. The argument lies in protecting the integrity of the CAP process, as an expression of the collective need. HRF funding, operating outside of the CAP structure, could potentially undermine this. OCHA emphasize that the HRF is not designed to fill gaps in CAP funding.

In practice this remains a difficult boundary to police. At times conflicting views were reported of what could, or could not, be funded at the boundary of the HRF. Consequently rather a lot of energy has been expended at times in debating questions of eligibility, often relating to the degree of specificity of existing CAP proposals. At other times a more relaxed interpretation has been used, especially after CAP projects have remained unfunded for several months.

This remains a problematic issue as, despite overall good funding levels of the CAP, significant gaps in funding are reported in particular by the NGOs. CERF is unlikely to be invoked here given the overall positive funding and there is no other adapted mechanism designed to specifically fund gaps in CAP funding.

A second boundary for HRF gap funding is the boundary between emergency and chronic needs. The crisis in oPT is protracted and many of the needs are effectively chronic in nature. In principle the HRF does not fund repeated interventions in areas such as water supply resulting from neglect in basic service provision. However, the boundary becomes blurred when the lack of services leads to symptoms that are as acute as an emergency. As noted above, the lack of development options placed humanitarian aid in a difficult situation.

In effect the HRF has been forced to act a provider of last resort, stretching its mandate to avert a deterioration of livelihoods and quality of life. However, it is not well adapted to this role. In addition to questions of mandate, it is too small to have real traction at this level.


12


Finding 29. The HRF reinforces humanitarian leadership and coordination arrangements, principally by encouraging the active engagement of NGOs in these structures.

In oPT the HRF benefits from the existence of a set of well-established coordination structures. The successful operation of the HRF is heavily reliant on the leadership and coordination arrangements established as part of the humanitarian reforms. The backbone of this is a set of clusters covering Protection, Agriculture, WASH, Education, Cash Transfers and Food Aid. These are supplemented by NGO focal points for Shelter and NFIs, and a sectoral coordination group for health.

The HRF is heavily reliant on technical assistance from clusters in identifying needs, working with partners on proposal development, vetting proposals and assistance with monitoring. The quality of the coordination bodies is acknowledged to be variable. However, the HRF itself has a limited ability to improve the performance of these coordination bodies directly. From the cluster perspective the main indirect benefit of HRF funding is the incentive for NGOs to participate in their activities.

The HRF – which some perceive as being effectively controlled by the cluster leads – maintains a limited incentive for the continued involvement of NGOs. As one cluster lead said, the HRF demonstrates to NGOs the tangible benefits of participating in cluster processes. This is of particular significance given the problems around the CAP process. All NGOs, both INGO and NNGO, reported that the CAP is largely ineffective for them as a fund raising tool. Therefore interest in devoting resources to participation in cluster led CAP processes is waning.

In reality the HRF provides more of an incentive for INGOs who tend to receive a larger share of funding. The pool of NNGOs partners is vast – estimates varied of between 1,000 and 3,500 registered NNGO in oPT. The HRF – and indeed the clusters – can only work with a mere handful of the strongest and most relevant NNGOs.

The HC has played an important role in supporting the HRF. Convening all stakeholders around a common humanitarian plan provides the framework for the HRF activities and has promoted coherent action. In the oPT the HC has played a prominent role in leading the Advisory Group. Given the intrinsically political ramifications of humanitarian action in the region, building a common position amongst donors is crucial in maintaining humanitarian space. In return the existence of a pooled fund, under the control of the HC, bolsters his authority and leadership role.

PARTNERSHIP Finding 30. The HRF has limited comparative advantages in promoting effective partnership amongst the

different stakeholder groups.

Under the ERF results framework there is a generic objective of strengthening, or building, partnership between various actors. For the HRF, this primarily relates to operational partnerships between the UN agencies, international and local NGOs. A starting point for the evaluation is a determination of what good ‘partnership’ is, so that progress towards this goal can be assessed.

NGOs and UN agencies need to partner as they cannot reach their development goals alone. Working cooperatively allows agencies to achieve greater innovation, impact, scale, reform and sustainability. The most effective examples of humanitarian partnership are based on a voluntary collaboration, sustained over the longer-term in which each party shares benefits, costs and risks to achieve a jointly defined objective.

However, in relation to the HRF in oPT and more generally there is little clarity on what is meant by the concept of partnership. There is no accepted framework for action, or reference to common documents such as the UN principles of partnership. This means that the HRF operates without a clear set of partnership goals. Consequently there is little surprise that examples of the HRF nurturing complementary and creative partnership between multiple stakeholders are sparse.

If anything there seems to be a dominant concept of ‘replacement’. In discussing the issue, NNGOs repeatedly expressed a desire to access humanitarian funds directly, avoiding the necessity of ‘partnering’ with INGOs.

The HRF has increased the proportion of grants made directly to NNGOs. Measures to promote this include working through local NGO associations to publicize the existence of the fund and translating materials into Arabic. There was a push from the NNGO representative in the HRF Advisory Board that a condition for all HRF grants should be an NNGO implementing partner. However, this was rejected by the HC on the basis that whilst desirable, it should not be a binding obligation with the most important consideration being the most effective implementing partner.

In terms of wider system wide efforts to capacitate local NGOs, there are already a plethora of on-going initiatives to build NNGO capacity. The HRF, with its’ emergency focus, does not have an obvious comparative advantage in


13

working with NNGOs on this theme. This may be better addressed through sustained engagement in a less time sensitive context, and a more tolerance learning during implementation.

PROCESSES AND INPUTS The process of application, review, approval, implementation, monitoring and reporting on HRF projects is summarized in Exhibit 6. Observations on the various steps in the process are given below. Exhibit 6 Steps in the HRF Process

Needs assessment and application

Finding 31. The role of the clusters in early warning, needs assessment and developing applications is critical and could be further systematized.

The starting point for the HRF response is typically the identification of a ‘shock’. Early warning and needs assessments are relatively informal processes in oPT. Information on some shocks is centrally compiled and monitored, such as demolitions. In other cases, such as agro-climatic shocks, information is more informally gathered through the NGO network. Given the small area and population of the oPT this informal network appears to provide adequate information on emerging hazards and impacts. Whilst there may be room to improve early warning this analysis falls outside of the scope of the HRF mechanism.

How this information on a shock is transformed into a proposal follows a number of possible pathways. At one extreme an NGO may identify the problem, conduct its own needs assessment and independently submit a proposal for ERF funding. At the other end of the spectrum the cluster plays a core role in the identification of the problem, conducting the needs assessment, encouraging a suitable agency to prepare and application and mentoring the process of proposal development. Often the process lies somewhere in between, with an on-going interaction between the NGO and cluster.

There are clear benefits of close involvement of the cluster in an early stage of the proposal development. The main advantage is avoiding the submission of unsuitable applications – projects that are either out of scope or replicate other interventions. Making the involvement of the cluster mandatory at an early stage may be worth considering as it saves devoting resources on developing unnecessary proposals.

A number of other issues emerged around the responsibilities of the clusters for nurturing applications. Firstly, there is a potential conflict of interest between the coordination role of the cluster lead agency and its own role in programming response. There is the risk that the cluster lead may use its privileges position to identify funding opportunities for their own agency, rather than working through the cluster membership to identify implementing actors. Concrete examples of this behaviour emerged in the consultation. The need to firewall the two functions in a cluster lead is important – possibly by using a peer review mechanism rather than relying on the cluster lead. This is a general issue for cluster operations and management and lies largely outside of the control of the HRF.


14

Secondly, there is the question of what to do where no cluster exists. This including situations where the emergency cluster has transformed into a development orientated sectoral coordination body – as for health, where no formal structure has been established or where the ‘cluster’ has minimal capacities. In the case of oPT the OCHA Field Coordination Units provide an important and effective back-up system, but these cannot be relied on in all contexts.

Thirdly, there is a tension between the centrality of the cluster and a desire for greater NNGO involvement. Many smaller NNGOs reported a limited capacity to engage in coordination meetings, so that the cluster becomes a barrier to NGO involvement. One option is the use NNGO coordination bodies, such as the Palestinian Non-Governmental Organizations Network (PNGO), as representing the stakeholder group in cluster meetings. However, this has had limited success. Ultimately it may be reasonable to expect one criterion for using an NNGO implementing partner is its ability to participate in coordination mechanisms.

Review of proposals

Finding 32. The HRF secretariat manages the review process relatively efficiently.

The HRF is managed by a two person team in the OCHA office. This team is responsible for all aspects of the in-country administration of the HRF fund. At the level of proposals this includes providing guidance to potential implementing partners on the application process and managing the project review, approval, monitoring and close-out.

The proposal submission process is detailed in HRF guidance notes (a general note and sector specific notes for agriculture and WASH) drafted by OCHA at the country level. These were found by implementing agencies to provide clear guidance on eligibility criteria, minimizing the submission of proposals that are subsequently rejected as falling outside of the scope of the HRF.

Within the review process the HRF management team is responsible for an initial screening of proposals to determine if these meet the eligibility criteria and have been correctly completed from an administrative perspective. This team then manages the remaining steps of the review process, specifically seeking technical inputs and clearances.

The evaluation received generally positive feedback on the HRF management, specifically the HRF secretariat but also the support offered by other members of the OCHA country team when needed. However, the analysis of the detailed project timelines (see exhibit 5.3) suggest that there could be room for improving the efficiency of the secretariats’ processing of project proposals.

Finding 33. Budget review guidelines are overly restrictive and could be relaxed.

The guidelines on the structure of the project budget are quite restrictive and adherence is closely scrutinized during the review process. This includes an expectation that 80% of the budget is spent on direct costs, that overheads are limited to 7%, and staffing costs limited to 10%. The underlying assumption is that HRF projects should leverage support from other sources, or from the established infrastructure of the implementing agency in the area of operation.

Budget structure and level of cost sharing are subjects of an on-going debate between donors, UN agencies and NGOs. However, there does seem to be a valid case that HRF projects do require a degree of cross subsidy from the applying agencies own resources. This may have negative implications for the participation of NNGOs or mounting a genuine rapid response to an emergency, where the necessary complementary resources are lacking.

Finding 34. The technical review process is cumbersome and could be streamlined.

Following an initial screening of the proposal by the OCHA ERF manager, proposals pass through two stages of technical review. Firstly, the formal approval of the cluster lead is sought. At this point the opinion of the relevant OCHA Field Coordination Unit (FCU) is also canvassed, but their input is non-binding. Secondly, proposals which are approved by the cluster are passed electronically to the HRF Review Board for their endorsement. The cluster review tends to be much rigorous and methodological than the Review Board function.

There appears to be an unnecessary degree of overlap between the two processes, with overlapping (and on occasions contradictory) feedback provided. This inevitably delays the review process. The benefits of the double review do not appear to add sufficient value to justify the delays caused. There is an opportunity to streamline the process by consolidating the two step review. This could be achieved either by integrating the cluster role in the review board, or reinforcing the cluster level review and dispensing with the current RB.

As a related comment, it was noted that the process of approving project amendments during implementation is also extremely heavy. Until recently with the promulgation of new Guidelines, even minor changes – such as no cost extensions or approval on utilizing the contingency fund - have to go through the whole approval process again, including the authorization of the HC.


15

Timely allocation and disbursement

Finding 35. Timeliness can be significantly improved by changes to administrative procedures.

As discussed earlier there is a need to improve the timeliness of the allocation and disbursement of funds. Various options were identified to improve the timeline, while working within the general parameters of the current process:

• Requiring applicants to systematically consult the clusters (or relevant coordination body) from the start of the process. This would avoid situations where clusters are only consulted after a formal proposal has already been submitted to OCHA.

• Simplifying the review process, and consolidating the two step process into one. • Changing the eligibility date for expenditure to the first date of submission of a proposal, from the current date of

signing the MoU. If the proposal is not approved then no expenditure would be reimbursed. This requires the implementing partner to have a limited pre-financing capability and a willingness to bear the risk of the proposal being rejected. However, this is an established model already in use by agencies and some donors (e.g. DG ECHO) that could be an important option for genuine sudden onset emergencies.

• Pre-registration of partners. Currently each application has to be supported by quite specific legal and financial documentation from the applicant, irrespective of whether they are a previous recipient of a HRF grant. Possibly these requirements could be relaxed for proven partners or the process of clearance started earlier in the overall process.

One suggestion mooted locally was that responsibility for financial disbursements could be shifted from OCHA Geneva to UNDP in Jerusalem. However, on closer examination it appears that UNDP would still have to act on the specific instructions of OCHA Geneva. In addition UNDP’s systems are aligned with developmental, rather than emergency, timeframes. Therefore this change is unlikely to improve timeliness and may even worsen it by introducing a new actor.

However, alternative ways of shaving time from this step of the process should be considered – as clearance by Geneva adds an average of 10 days to the process16 (see exhibit 7). In the context of OCHA’s core responsibility for pooled funding, it might be appropriate to initiate a wider discussion on delegating greater financial authority to OCHA to manage its own finances directly at the country level. Under this scenario significant time savings could conceivably occur.

The table below displays timing in two dimensions. “Days Between” addressed how long an individual step may require to be completed. “Total Time” is a representation of the total running time for the entire process and not for any one step. Exhibit 7 Timeline for Project Review and Approval (2011 average)17

Days between steps

(e.g. from receipt of proposal to submitting it to cluster)

Total time (days from receipt of proposal)

1. Proposal received

2. Submitted to cluster 7.6 7.6

3. Cleared by cluster 15.3 22.9

4. Final proposal submitted 0.4 23.3

5. Sent to Review Board 2.8 26.1

6. Review Board feedback 1.1 27.2

7. To HC or approval 8.4 35.6

8. Approved by HC 0.3 35.9

9. Draft MoU to GVA 3.7 39.6

10. MoU cleared by Geneva 10 49.6

11. MoU signed 5.1 54.7

Ultimately the problem is not so much inefficiency at any one step in the process, but rather the number of steps that have to be completed. There are limits in the ability to further streamline the HRF process without 16 Part of this time involves delays associated with revisions and approvals of the MoU and other administrative clearances. 17 These intervals may include the time necessary for the applicant to provide additional information, ie these do not necessarily reflect on just the time taken by the cluster, OCHA or ERF boards to process the documentation.


16

compromising administrative safeguards. There is space for some rebalancing between accountability concerns and rapidity, but absolute limits will apply. This highlights the need for alternative approaches – such as preparedness funding where agencies have stand-by contingency funds that can be used without prior approval – to complement the HRF.


Mainstreaming gender

Finding 36. Gender aspects are weakly articulated in HRF projects, but roll-out of gender marker is in progress.

The HRF application form asks applicants to articulate how gender was taken into account in the identification of needs and formulation of the response. However, in practice gender aspects appear to be weakly addressed in many of the projects. The HRF Secretariat undertook a baseline analysis of all HRF projects approved in 2011 against the gender marker classification. Of the 14 projects approved for 2011, gender was mainstreamed in 4 projects, 4 projects had weak gender analyses in their assessment, and 3 projects showed no attention to gender mainstreaming and gender equality in their project design.

The HRF Secretariat recognizes the need for additional steps to raise awareness and ensure that gender considerations are mainstreamed in all HRF projects. The introduction of the Gender Marker tool in 2012, which encourages agencies to include gender analysis in their needs assessments and integrate the advancement of gender equality into all activities and outcomes, will be key in supporting this endeavour. These new initiatives will formally come into effect once Global ERF Guidelines have been endorsed. A Gender Marker self-assessment form has already been included as part of the screening process for projects in 2012.

Cost efficiency

Finding 37. Total management costs are relatively high, highlighting the need for the HRF to be utilized selectively and not compete inappropriately with other cheaper funds.

Tracking and estimating the total overhead costs of the HRF fund is difficult to measure accurately. The most transparent element is the 3% overhead charged by OCHA on the contributions. In practice this covers only a relatively small part of the real overhead costs and is directed to support part of the OCHA HQ costs – specifically the Administrative Services Branch (ASB) in Geneva.

Many other contributions are made to the operation of the HRF. This includes:

• The HRF secretariat costs – with two locally hired professional staff. Although they support other tasks (including CERF if activated) over 80% of their time is devoted to HRF management.

• OCHA country office staff. Given the political sensitivities of HRF projects, a considerable amount of senior management time (by the HoO and others) has been used to support this instrument.

• OCHA field staff from the four FCUs support project identification, review and monitoring activities. • The clusters, principally the cluster leads, support project formulation, review and to some extent monitoring. • A large group of donors support the HRF administration and attend the Advisory Board meetings. To put this in

perspective the members of the Advisory Board meets around three times a year and the last meeting included eight donors.

• The implementing partners cross subsidize the operation of the HRF as the chargeable overheads reportedly fall short of real costs.

If these costs were accurately accounted for the effective costs of operating the HRF may be much higher than assumed. This is especially true for a relatively small fund such as the oPT HRF. Potentially this makes it relatively expensive source of financing.

Given the lack of data it is hard to draw accurate conclusions on real costs. However, it does tend to reinforce the need to apply the HRF strategically in areas of highest comparative advantage and avoid competing with other potentially more cost-efficient (e.g. bi-lateral) sources of funding.

Operation of advisory board

Finding 38. The Advisory Board has been an essential ingredient of the success of the HRF in the oPT context.

The Advisory Board (AB) consists principally of the HRF donors, along with a UN agencies, INGO and NNGO representatives. It is chaired by the Humanitarian Coordinator.

The AB plays a central role in setting the strategic direction for the HRF. In the politically charged environment of the oPT, it has proved particularly important to have strong representation of donors on the AB. For example, the


17

HRF interventions in Area C have raised tensions with the Israeli authorities. The collective ownership of projects through a pooled fund – and reinforced through the active engagement in the AB - has been critical in defusing this pressure. Indeed in the context of the oPT the collective nature of the HRF has been as important as the quick disbursement element.

The AB also serves as a useful counter-check on allocative decisions at the strategic level. This complements the project level review by ensuring a prioritization between sectors, geographic areas and recipients.

MONITORING, EVALUATION REPORTING AND AUDIT Finding 39. Projects are adequately monitored to ensure minimum standards for accountability in the use

of funds.

The monitoring of HRF projects is carried out by the implementing organizations as well as externally by OCHA. Organizations are requested to submit monthly progress reports during the implementation period, in addition to mid-term and final reports. The requirement for monthly reports seems somewhat over burdensome and in any case adherence to this is weak.

Given the relatively small number of projects, the HRF managers have directly monitored a large proportion. Of the 14 projects implemented during 2011, 10 projects (70%) were visited by the HRF manager. Additional monitoring visits were carried out by the HRF Advisory Board and OCHA field staff. However, there is no formal ‘target’ on the proportion of projects that should be visited. This might be usefully established.

The OCHA monitoring uses a standard monitoring proforma. This provides the necessary assurances that works have been implemented as agreed and canvasses levels of satisfaction amongst users. Monitoring is also being strengthened to ensure that projects are mainstreaming gender. OCHA, not being a technical agency, has actively sought the participation of cluster leads in monitoring visits. This in turn has been (partially) resisted by cluster staff due to time constraints.

It is suggested that it is not realistic or necessary to expect systematic technical / cluster involvement in monitoring all HRF projects. The scale of the projects often does not justify a heavy monitoring process and the existing OCHA led monitoring provides adequate assurances and quality control. Where technical aspects are critical, for example building works, there should be provision within the project to utilize specialist supervision such as the Ministry of Works. Therefore a continued ad hoc cluster involvement is probably sufficient.

Finding 40. Routine audits are overly precautionary and could be conducted on a sample basis.

The current HRF requirement is that 100% of the projects are audited. This process has been streamlined with the introduction of a central contract with an auditing company under OCHA management, rather than requiring each agency to conduct its own audit.

Judged against industry standards this requirement seems excessive. Even well established partners have to complete an audit for each project. In addition it causes significant problems to some implementing agencies as a 10% retention is held until a satisfactory audit report is received. For smaller partners, with limited cash flow, this can be problematic.

It may therefore be appropriate to consider developing criteria for conducting audits on a sample, rather than routine, basis.

Finding 41. Detailed impact evaluations are best conducted at the sectoral, or crisis, level rather than at the level of the project or HRF.

There has been an on-going debate on improving reporting at the level of results. This applies not only to HRF projects but across the humanitarian sector. HRF projects are expected to provide final narrative and financial reports within three months of conclusion. These reports are reviewed by the ERF manager, but rarely reach a wider audience in an unprocessed form. For short-duration projects the final reports were judged to be appropriate and informative, containing useful information on implementation and immediate outcomes. The implementing partners appreciated that the reporting requirements were not unnecessarily onerous.

This does not obviate the need for a deeper understanding of the impacts of these humanitarian interventions. However, it is questioned whether it makes sense to attempt more detailed impact evaluations at either the project - or at the HRF mechanism level. Impact evaluations are inevitably strategic in nature and draw from an evidence base of a range of interventions, often over an extended period. The HRF universe of projects is probably too small to provide a useful basis for enquiry. It is therefore suggested that it would be far more productive to investigate questions of impact through evaluations at either the sectoral- or crisis-level.


18

Finding 42. HRF level reporting could strengthen sharing of best practice

OCHA provides a number of summarized reports on HRF performance for a wider audience. As well as an Annual Report, OCHA introduced a HRF monthly update in 2011. These reports contain basic statistics on income and expenditure. An analysis is provided of projects by sector, partner and geographical area. These are shared with Members of the Advisory Board and humanitarian partners to ensure the transparency and accountability of the fund.

One area of improvement could lie in providing more substance on selected HRF project activities. At the moment this is lacking. Case studies of best practices could be better publicized to help foster replication of success stories. This information would also assist donors in advocating for continued support to the HRF.


19

Overall Conclusions The HRF in oPT plays a unique role in supporting humanitarian response, The HRF has proved to be flexible, adapting to meet evolving needs. Particular strengths of the mechanism have proved to be in relation to:

• Effectively responding to small-scale unforeseen emergencies that have developed in-between funding CAP funding cycles, such as weather and disease events;

• Increasing NGO access to emergency response funding, with increasing attention to the inclusion of NNGOs; • Working through a collective instrument to address sensitive areas of protection, including interventions in Area

C of the West Bank; • Maintaining a strong reserve to buffer immediate response to potential large-scale unforeseen shocks.

The management by the OCHA HRF managers has been relatively efficient and effective. The big challenge remains in reducing the time taken to appraise and approve projects. Some fairly dramatic changes are needed in procedures if the mechanism is to be truly fit for purpose.

The value of the HRF mechanism is witnessed by the additionality of action, rather than the additionality of funds. The HRF has facilitated interventions to take place that otherwise would not have happened. The impacts of these have been disproportionate to the relatively modest levels of funding.

The impact in relation to supporting the Humanitarian Reforms has been positive, but modest. The existence of the HRF is perceived to positively reinforce both humanitarian leadership and coordination. However, given the scale of the funding, the HRF has limited influence on, or through, these processes. The main tangible benefit has been encouraging the engagement of a selected group of NNGOs in cluster led processes.

The role of the HRF in promoting partnership has been ambiguous; not least because the goals of partnership are not clearly articulated within the humanitarian system in oPT. The implicit goal is a comparatively narrow one of building NNGO capacity as an exit strategy for other humanitarian agencies. Against this goal the HRF has a number of comparative disadvantages as a capacity building agent.

In the oPT context specific funding and implementation gaps remain in relation to (i) funding of NGOs within the CAP, (ii) funding of preparedness activities, in particular those relating to protection, and (iii) basic service delivery. The HRF has served as a funder of last resort, addressing the most acute gaps. However, it is not the ideal instrument to address these wider needs. Continued efforts are need to construct a better adapted solution to address these important gaps.


20



Recommendation 13. Given the high exposure to a range of shocks, and limited resilience, donors in oPT should maintain a pooled fund to respond to unforeseen emergencies.

The objectives set for the oPT HRF continue to be relevant, and the mechanism has a proven added value within the available funding instruments. Therefore the continuation of an HRF like pooled funding instrument is recommended for the immediate future. This should continue to focus on funding NGOs – both INGOS and NNGOs.

Funding for this function should be similar to current levels. This may be either a standalone fund – or an element of an adapted instrument (see below).

Consideration should be given to revising the definition of the funding objectives. Given the nature of the crisis in the oPT, it is unclear why would ‘live-saving’ be imposed as a criteria; this is not a funding criteria for all ERFs, and it is not in global guidelines. It seems as the most restrictive option was selected.

Recommendation 14. OCHA should explore with donors the possibility of adapting the current mechanism to support critical gaps in the response to the chronic crisis in oPT as a whole and preparedness funding in particular.

There is a gap in humanitarian funding to meet critical gaps in addressing chronic needs in oPT. A second gap is for funding of preparedness measures – many protection issues almost by definition are best addressed through enhanced preparedness. HRF funding has been stretched to cover some of the most pressing gaps.

In the context of this protracted crisis it is suggested that the relevance of other country based pooled funds should be examined. The most obvious option would be a CHF which is designed to fill gaps in CAP funding (including preparedness actions), reinforce NGO participation and partnership and absorb the scale of donor funds that appear to be available.

Within the operation of the current HRF OCHA should consider changes to improve transparency, timeliness and visibility. Specific recommendations on administrative and financial procedures are limited as many of these would be logically require centralized changes.

Recommendation 15. OCHA should re-consider whether the cluster leads should be responsible for formal approval of projects.

The current review process has weaknesses, is somewhat repetitive and slow. OCHA oPT should consider options for strengthening the process. In particular, in order to better firewall the role of lead agency as coordinator and implementor, OCHA should consider reducing the individual role of the cluster lead in project approval. This may be achieved either through shared responsibility at the cluster level, or centralizing formal responsibility for approval in a (strengthened) review Board.

Recommendation 16. OCHA improve the reporting on the oPT HRF results through a revised Annual Report and collaboration with the clusters on impact evaluations.

Donors and other stakeholders lack sufficient information on the substance of HRF achievements. OCHA could consider addressing two principle approaches to improving this information flow. Firstly the current HRF Annual Report, whilst informative, lacks sufficient information on the essence of HRF interventions. OCHA should consider adapting these reports to include mini case studies of the most relevant and successful interventions.

Secondly, OCHA through the HC should collaborate with the cluster leads to investigate how sectoral level evaluations could be conducted. The role of the HRF would be one element within such studies.


21

Appendix I: Programme Agenda

01 Oct

7:30 - 8:30 Briefing with OCHA Head of Office

9:00 - 11:00

12:00 - 3:00

02 Oct

9:00 - 10:30

11:00-12:00

12:30 - 2:00

(ESDC, RCSO, Palestinian Youth Union)

2:00 - 3:00

3:00- 4:00

03 Oct

8:30 - 9:00 Meeting with the Humanitarian Coordinator

9:00 - 11:00

11:30-1:00

1:30 - 3:00

04 Oct

8:30 - 9:00

9:00 - 11:00

12:00-1:00

1:30 - 2:30 Meeting with OHCHR (in Ramallah)

05 Oct

9:00 - 10:00

10:30 - 11:30

12:30 - 1:30

2:00 - 3:00

Meeting with OCHA Administrative Services

Meeting with OCHA Field Coordination UnitsMeeting with UNDP

Meeting with CESVI

Meeting with Save the Children

FR

I

Focus group with cluster coordinatorsTH

UR

S

Meeting with Palestinian Authority Ministry of Planning (in Ramallah)

Focus groups with INGO stakeholders. (GVC, SCC, ACF, CISP)

Meeting with HRF Advisory Board

Briefing with HRF Manager and Assistant

WE

D

Meeting with ESDC

Meeting with MAAN

Briefing by OCHA on Humanitarian Situation

Meeting with DG ECHO

Focus groups with NNGO stakeholders (PHG, MAAN, UAWC, SHAMS, PARC)

SCHEDULEM

ON

Focus group with UN stakeholders (UNICEF, WHO, FAO, WFP, UNRWA)

Briefing with HRF Manager and Assistant

TU

ES


22

08 Oct

9:00 - 16:00 Field visits to West Bank

PHG demolition response (Nablus)

09 Oct

9:00 - 10:30

11:00 -12:30

Tu

e

Clarification meeting with ERF Manager

Exit debrief with OCHA and Advisory Board

CISP water projects (Jericho)

Mo

n


23

Appendix II: Stakeholders Name Organization Position

Saad Abdel-Haq OCHA HRF Fund Manager

Amal Al-Rafiq SHAMS

Diana Anani OCHA Head West Bank - South - Field Coordination Unit

Muhorram Barghouti Palestinian Youth Union

Jamal Barnat ESDC

Michael Beeman ACTED Shelter Focal Point

Francesca Bianchini CISP Administrator

Nicola Crissell DFID Team Leader

Maher Daoudi Consulate General of Sweden Deputy Head Development Cooperation

Nadejda Dagasu FAO Agriculture Cluster Coordinator

Arif Daraghmeh Paletinian National Authority Village Council

Mutaz Ewidat RCSD

Marco Ferloni GVC Country Representative

Cyril Ferrand FAO Senior Emergency Coordinator

Thierry Foubert UNICEF WASH Cluster Coordinator

Juan F Garcia Consulado General de Espana Responsible for Agriculture and Natural Resource Projects

Diego Gutierrez ACF Representative

Douglas Higgins UNICEF Deputy Special Representative

Ayman Ibrahim OCHA Central West Bank Field Coordination Unit

Rema Imseis UNSCO Special Assistant

Kahramana Kelani-Jarrar Palestinian Authority Head of UN and INGOs Department

Sami Khader MA'AN Development Center Director General

Mohammed Khaled Swedish Cooperative Centre Country Director

Peter Lundberg Consulate General of Sweden Deputy Consul General

Letizia Montecalvo UNRWA External Relations

Yousef Muhaisen WHO Health Sector Coordination

Isra Muzaffar OCHA Head Central West Bank Field Coordination Unit

Stephanie O'Brien Irish Aid Deputy Head of Mission

Tatiana Olivero CEVSI Head of Office

Gaincarlo de Picciotto Swiss Agency for Development and

Cooperation Head of Office

Ramesh Rajasingham OCHA Head of Office

James Rawley UNSCO Humanitarian Coordinator

Taha Rifaie UAWC

Adeeb Salman OCHA Humanitarian Affairs Assistant

Yasmine Sherif UNDP Deputy Special Representative

Maryanna Schmuki UNICEF Education Cluster Coordinator

Shrine Smeir OCHA OiC Administration

Saul Takahashi UNHCHR Deputy Head of Office

Jesus Tome Consulate General of Spain Technical Cooperation Office

Laura Turner WFP External Relations

Stephen Williams DG ECHO Ag. Head of Office

Izzat Zeidan PARC Director of Extension


The Global Evaluation of Emergency Response Funds (ERFs) Evaluation Matrix and Terms of Reference March 2013

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Eva

luat

ion

Mat

rix

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

Op

erat

ion

al Im

pac

t

Cri

tica

l un

fore

seen

hu

man

itar

ian

n

eed

s ad

dre

ssed

How

, and

to w

hat e

xten

t has

ER

F

mec

hani

sm c

ontr

ibut

ed to

the

hum

anita

rian

com

mun

ity’s

abi

lity

to a

ddre

ss c

ritic

al u

nfor

esee

n hu

man

itaria

n ne

eds

in a

tim

ely

and

effe

ctiv

e m

anne

r?

Wha

t spe

cific

feat

ures

of t

he E

RF

lend

th

emse

lves

to im

prov

ing

the

hum

anita

rian

resp

onse

?

Wha

t are

the

view

s of

rec

ipie

nts

and

natio

nal g

over

nmen

ts r

egar

ding

the

over

all o

pera

tiona

l im

pact

of t

he E

RF

s?

Wha

t are

the

view

s of

oth

er

deve

lopm

ent p

artn

ers

dono

rs o

ther

UN

et

c.)?

Evi

denc

e of

new

feat

ures

/new

type

s of

pr

ojec

ts/r

ecip

ient

s/ga

ps fi

lled

Evi

denc

e of

leve

rage

of E

RF

with

oth

er h

uman

itaria

n do

nors

Res

pons

e ra

tes

for

ER

Fs

sinc

e 20

07 (

timel

ines

s)

Rat

e of

sat

isfa

ctio

n w

ith E

RF

by

natio

nal

gove

rnm

ents

Rat

e of

sat

isfa

ctio

n of

rec

ipie

nts

Rat

e of

sat

isfa

ctio

n w

ith o

ther

UN

par

tner

s

Rat

e of

sat

isfa

ctio

n w

ith d

onor

s

Iden

tific

atio

n of

ER

F u

niqu

e fe

atur

es a

nd d

egre

e of

sa

tisfa

ctio

n in

/ acc

epta

nce

of t

hem

Deg

ree

of s

atis

fact

ion

in th

e tim

elin

ess

of E

RF

s

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evie

ws

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Fin

anci

al /

tran

sact

ion

data

Wha

t has

the

perf

orm

ance

and

‘v

alue

add

ed’ o

f the

ER

F

mec

hani

sm b

een

in th

e co

ntex

t of

the

over

all h

uman

itaria

n ar

chite

ctur

e an

d hu

man

itaria

n re

form

pro

cess

(e.

g. R

C/H

C

stre

ngth

enin

g, c

lust

er a

ppro

ach,

de

velo

pmen

t of e

ffect

ive

part

ners

hip

betw

een

UN

and

non

-U

N h

uman

itaria

n ac

tors

, hu

man

itaria

n fin

anci

ng)?

How

has

the

RC

/HC

func

tion

been

af

fect

ed?

Is th

ere

evid

ence

of i

ncre

ased

clu

ster

co

oper

atio

n?

Wha

t are

the

spec

ific

“val

ue a

dded

” el

emen

ts?

Has

the

ER

F p

rodu

ced

syne

rgie

s or

ca

taly

zed

addi

tiona

l fun

ding

?

Rat

e of

sat

isfa

ctio

n in

ER

F m

odel

by

clus

ter

part

ners

Iden

tific

atio

n of

cha

nges

in R

C/H

C r

oles

Cha

nges

in o

vera

ll hu

man

itaria

n fin

anci

ng p

atte

rns

( va

lue,

type

, tar

gets

)

Iden

tific

atio

n of

new

ele

men

ts o

ver

time

(200

7 to

pr

esen

t)

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Fin

anci

al /

tran

sact

ion

data

Op

erat

ion

al E

ffec

ts

Rel

evan

ce/A

pp

rop

riat

enes

s -

Fle

xib

ility

– T

imel

ines

s -

Co

vera

ge

To

wha

t ext

ent h

as th

e us

e of

E

RF

fund

s im

prov

ed o

pera

tiona

l ef

fect

s of

the

hum

anita

rian

resp

onse

, in

part

icul

ar

rele

vanc

e/ap

prop

riate

ness

, fle

xibi

lity,

tim

elin

ess

and

cove

rage

?

Wha

t spe

cific

feat

ures

of E

RF

s ha

ve

cont

ribut

ed to

thes

e im

prov

emen

ts?

Deg

ree

to w

hich

sta

keho

lder

s pe

rcei

ve E

RF

to b

e a

posi

tive

addi

tion

to th

e o

vera

ll hu

man

itaria

n re

spon

se

Iden

tific

atio

n of

spe

cific

feat

ures

/attr

ibut

es in

eac

h E

RF

Leve

l of s

atis

fact

ion/

per

ceiv

ed p

ositi

ve b

enef

it of

key

el

emen

ts b

y in

tern

al a

nd e

xter

nal s

take

hold

ers

Inte

rnal

and

ext

erna

l su

rvey

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

18 S

elec

tion

of in

dica

tors

will

dep

end

on th

e av

aila

bilit

y of

dat

a

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

How

rel

evan

t/app

ropr

iate

are

the

obje

ctiv

es th

e E

RF

mec

hani

sm

and

the

port

folio

of p

roje

cts

finan

ced

to h

uman

itaria

n ne

eds

in

resp

ectiv

e co

untr

ies?

Has

ER

F fu

ndin

g be

en d

irect

ed to

the

high

est p

riorit

y ar

eas

(geo

grap

hica

l, th

emat

ic, a

nd s

ecto

ral)

with

in th

e re

spon

se?

Dis

trib

utio

n ra

tes

acro

ss th

emes

/sec

tors

on

a co

untr

y ba

sis

over

tim

e

Deg

ree

of s

uppo

rt b

y IN

GO

s, N

NG

Os

and

UN

pa

rtne

rs o

f the

dis

trib

utio

n pa

ttern

Inte

rnal

and

ext

erna

l su

rvey

s

Cou

ntry

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Fin

anci

al /

tran

sact

ion

data

How

ada

ptiv

e is

the

ER

F

mec

hani

sm to

cha

ngin

g ne

eds?

H

ow fl

exib

le is

ER

F fu

ndin

g?

Wha

t spe

cific

mec

hani

sms

diffe

rent

iate

E

RF

s fr

om o

ther

com

mon

fund

s re

: ad

apta

tion

and

flexi

bilit

y?

Wha

t ins

tanc

es o

f the

ir us

e ar

e th

ere

in

mee

ting

“cha

ngin

g ne

eds”

?

Wha

t are

the

perc

eptio

ns o

f int

erna

l and

ex

tern

al s

take

hold

ers

re: f

lexi

bilit

y an

d ad

apta

tion?

List

of m

easu

res

in E

RF

that

diff

er fr

om C

HA

, CE

RF

, et

c.

Deg

ree

of s

uppo

rt fo

r th

e pr

opos

ition

that

ER

F is

fle

xibl

e an

d ad

apta

ble

List

of i

nsta

nces

whe

re “

flexi

ble

mea

sure

s” w

ere

used

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

T

o w

hat e

xten

t has

the

avai

labi

lity/

use

of E

RF

fund

s im

prov

ed th

e tim

elin

ess

of th

e re

spon

se?

Wha

t spe

cific

mec

hani

sms

diffe

rent

iate

E

RF

s fr

om o

ther

com

mon

fund

s re

: tim

elin

ess?

Wha

t ins

tanc

es o

f the

ir us

e ar

e th

ere

in

impr

ovin

g tim

elin

ess?

Wha

t are

the

perc

eptio

ns o

f int

erna

l and

ex

tern

al s

take

hold

ers

re:

ER

F a

s a

mec

hani

sm to

impr

ove

the

timel

ines

s of

re

spon

se?

Tim

elin

ess

of E

RF

s si

nce

2007

, rat

es b

y co

untr

y ra

tes

vers

us e

xpec

tatio

ns a

nd s

tand

ards

List

of t

ools

/pro

cedu

res

deve

lope

d to

impr

ove

timel

ines

s si

nce

2007

Rat

es o

f sat

isfa

ctio

n of

inte

rnal

and

ext

erna

l st

akeh

olde

rs r

e tim

elin

ess

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Fin

anci

al /

tran

sact

ion

data

How

do

ER

Fs

link

with

oth

er

hum

anita

rian

pool

ed fu

nds

- na

mel

y C

ER

F, a

nd o

ther

MD

TF

s?

Wha

t is

the

valu

e ad

ded

of h

avin

g E

RF

s as

a c

ompl

emen

t to

othe

r fu

ndin

g st

ream

s?

Wha

t is

the

effe

ct o

f the

pre

senc

e of

an

ER

F o

n th

e ov

eral

l lev

els

of

hum

anita

rian

fund

ing

avai

labl

e fo

r a

cris

is?

Inst

ance

s of

leve

rage

/cat

alyt

ic e

ffect

to a

ttrac

t oth

er

dono

rs

Iden

tific

atio

n of

val

ue a

dded

ele

men

ts o

f ER

F w

ith

poss

ible

com

paris

on b

y lo

cale

Rat

e of

cha

nge

of s

hare

of h

uman

itaria

n fin

anci

ng

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Dat

a fr

om e

xter

nal

stak

ehol

ders

Fin

anci

al /

tran

sact

ion

data

How

doe

s th

e si

ze o

f ER

Fs

impa

ct

thei

r ad

ded

valu

e, in

par

ticul

ar

whe

re o

ther

poo

led

fund

s ar

e pr

esen

t?

How

doe

s th

e op

erat

ing

envi

ronm

ent

impa

ct th

e ef

fect

iven

ess

and

valu

e-ad

ded

of E

RF

s?

Wha

t is

the

perc

enta

ge o

f the

ER

F in

re

gard

to th

e ov

eral

l hum

anita

rian

resp

onse

and

fina

ncin

g?

Iden

tific

atio

n of

val

ue a

dded

ele

men

ts o

f ER

F w

ith

poss

ible

com

paris

on b

y si

ze

Inst

ance

s of

leve

rage

/cat

alyt

ic e

ffect

to a

ttrac

t oth

er

dono

rs b

y si

ze

Inst

ance

s of

leve

rage

and

per

cept

ion

of v

alue

add

ed

taki

ng le

vel o

f dev

elop

men

t int

o qu

estio

n

Rat

e of

cha

nge

of s

hare

of h

uman

itaria

n fin

anci

ng

Inte

rnal

and

ext

erna

l su

rvey

s

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Fin

anci

al /

tran

sact

ion

data

Wha

t pro

gres

s ha

s be

en m

ade

sinc

e th

e in

cept

ion

of E

RF

s (h

ow

wer

e sh

ortc

omin

gs id

entif

ied

by

prev

ious

eva

luat

ions

/aud

its

addr

esse

d; w

hat a

re th

e le

sson

s le

arne

d fr

om im

plem

entin

g th

ese

reco

mm

enda

tions

)?

Wha

t rec

omm

enda

tions

wer

e m

ade

by

eval

uatio

ns a

nd a

udits

?

Wha

t was

the

rate

of p

ositi

ve

man

ager

ial r

espo

nse?

Wha

t has

bee

n th

e le

vel/r

ate

of

exec

utio

n?

Wha

t are

the

perc

eptio

ns a

bout

the

usef

ulne

ss o

f the

se c

hang

es?

Rat

e of

acc

epta

nce

of r

ecom

men

datio

ns s

ince

200

7

Rat

e of

impl

emen

tatio

n of

agr

eed

upon

re

com

men

datio

ns

Per

cept

ion

of th

e ut

ility

of s

uch

reco

mm

enda

tions

(c

ount

ry le

vel v

ersu

s H

Q v

ersu

s do

nor)

List

of o

ther

mea

sure

s/m

eans

impl

emen

ted

outs

ide

of

thos

e fr

om fo

rmal

rec

omm

enda

tions

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Ou

tco

mes

Cri

tica

l gap

s fi

lled

- L

ead

ersh

ip

and

co

ord

inat

ion

To

wha

t ext

ent h

as th

e av

aila

bilit

y of

initi

al a

nd b

ridge

fund

ing

and

un-e

arm

arke

d na

ture

of t

he fu

nds

allo

wed

bet

ter

targ

etin

g an

d co

ntrib

uted

to fi

lling

crit

ical

gap

s in

th

e re

spon

se?

Wha

t gap

s ha

ve b

een

fille

d?

List

of t

ypes

of g

aps

fille

d, c

ompa

red

by E

RF

to

esta

blis

h pa

ttern

s of

inte

rven

tions

In

tern

al a

nd e

xter

nal

surv

eys

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

Par

tner

ship

H

ow, a

nd to

wha

t ext

ent h

as th

e E

RF

mec

hani

sm c

ontr

ibut

ed to

en

hanc

ing

part

ners

hips

, in

part

icul

ar w

ith th

e N

GO

co

mm

unity

(in

tern

atio

nal a

nd

natio

nal)?

Doe

s O

CH

A h

ave

a po

licy

abou

t re

latio

ns w

ith N

GO

s in

the

cont

ext o

f E

RF

/CE

RF

act

iviti

es?

Doe

s O

CH

A h

ave

any

spec

ific

perf

orm

ance

mea

sure

men

t tec

hniq

ues

to a

sses

s “e

nhan

cing

par

tner

ship

s”?

How

doe

s O

CH

A s

ecur

e fe

edba

ck fr

om

its N

GO

par

tner

s, fr

om a

ll m

anne

r of

pa

rtne

rs?

Wha

t are

the

perc

eptio

n of

OC

HA

NG

O

part

ners

and

oth

er p

artn

ers

such

as

dono

rs a

bout

“en

hanc

ed p

artn

ersh

ip”

and

if po

sitiv

e w

hat s

peci

fic e

xam

ples

ca

n be

sho

wn?

Exi

sten

ce o

f Par

tner

ship

Pol

icy

or s

imila

r

leve

l of a

war

enes

s of

it a

nd it

s co

nten

ts

Circ

ulat

ion

of it

(ye

s/no

) to

pot

entia

l rec

ipie

nts

Exi

sten

ce o

f NN

GO

and

ING

O fe

edba

ck m

echa

nism

s

Leve

l of a

war

enes

s am

ong

dono

rs o

f par

tner

ship

po

licy

(if a

ny)

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

To

wha

t ext

ent i

s O

CH

A s

een

as a

“g

ood

part

ner”

in th

e co

ntex

t of

ER

Fs?

Wha

t are

the

perc

eptio

ns o

f OC

HA

ER

F

part

ners

abo

ut

the

qual

ity o

f the

pa

rtne

rshi

p re

latio

nshi

p?

Are

ther

e di

fferin

g pe

rcep

tions

bas

ed o

n di

fferin

g ki

nds

of p

artn

ers?

Wha

t are

the

perc

eive

d st

reng

ths

and

chal

leng

es o

f the

OC

HA

ER

F n

atio

nal

Adv

isor

y G

roup

mec

hani

sm?

Leve

l of s

atis

fact

ion

with

OC

HA

as

a pa

rtne

r

Diff

erin

g pa

ttern

s of

sup

port

/crit

icis

m a

mon

g in

tern

al

or e

xter

nal s

take

hold

ers

and

ING

Os

vers

us N

NG

Os

Leve

l of s

atis

fact

ion

with

cou

ntry

leve

l adv

isor

y m

echa

nism

s

Leve

l of s

atis

fact

ion

amon

g do

nors

re

OC

HA

as

a “g

ood

part

ner”

Inte

rnal

and

ext

erna

l su

rvey

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Pro

cess

es a

nd

Inp

uts

Eff

ecti

ve a

nd

tra

nsp

aren

t p

rop

osa

l re

view

- T

imel

y al

loca

tio

n a

nd

d

istr

ibu

tio

n -

Co

llect

ive

pri

ori

tiza

tio

n o

f n

eed

- R

epo

rtin

g

How

effe

ctiv

e/ef

ficie

nt/ t

imel

y ar

e E

RF

s in

pro

vidi

ng in

itial

and

br

idge

fund

ing

to a

ddre

ss c

ritic

al

unfo

rese

en h

uman

itaria

n ne

eds?

Do

any

have

qua

lity/

serv

ices

sta

ndar

ds?

Do

any

have

tim

elin

ess

stan

dard

s?

Typ

es o

f qua

lity

cont

rol m

echa

nism

s in

use

List

s of

sta

ndar

ds a

nd th

eir

usag

e

Deg

ree

of s

atis

fact

ion

with

sta

ndar

ds a

nd th

eir

usag

e by

OC

HA

per

sonn

el

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Are

bot

h he

adqu

arte

rs a

nd

coun

try

leve

l sta

keho

lder

s re

ceiv

ing

all n

eces

sary

and

re

leva

nt in

form

atio

n?

How

and

on

wha

t fre

quen

cy a

re v

ario

us

grou

ps o

f sta

keho

lder

s en

gage

d?

Wha

t are

the

perc

eptio

ns o

f the

se

vario

us g

roup

s of

HQ

and

cou

ntry

leve

l st

akeh

olde

rs a

bout

the

scop

e, q

ualit

y an

d tim

elin

es o

f the

info

rmat

ion

they

re

ceiv

e?

List

of p

roce

sses

aut

horis

ed fo

r st

akeh

olde

r en

gage

men

t

Com

paris

on o

f the

est

ablis

hed

proc

esse

s ve

rsus

ac

tual

del

iver

y/di

ssem

inat

ion

Rat

es o

f sat

isfa

ctio

n w

ith fe

edba

ck m

echa

nism

s co

mpa

red

acro

ss s

take

hold

er ty

pe

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t re

view

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

H

ow a

dequ

ate

is O

CH

A’s

ca

paci

ty

to a

dmin

iste

r an

d m

anag

e E

RF

s,

and

to w

hat e

xten

t is

it de

pend

ent

on th

e op

erat

ing

cont

ext?

Wha

t lev

el o

f res

ourc

es is

ava

ilabl

e at

H

Q a

nd a

t the

cou

ntry

leve

l to

man

age

ER

Fs?

Are

ther

e tr

ends

in th

ese

leve

ls?

Are

ther

e an

y be

nchm

arks

/sta

ndar

ds

that

reg

ulat

e or

gui

de th

e al

loca

tion

of

reso

urce

s to

ER

Fs

at b

oth

HQ

and

at

the

coun

try

leve

l?

Art

e th

ere

wid

e di

spar

ities

at t

he c

ount

ry

leve

l and

if s

o w

hy?

Wha

t are

the

perc

eptio

ns o

f int

erna

l st

akeh

olde

r re

: ade

quac

y?

Wha

t are

the

perc

eptio

ns o

f ext

erna

l st

akeh

olde

rs r

e: a

dequ

acy?

Rat

es o

f sta

ff re

sour

ces

com

pare

d ov

er ti

me

Rat

es o

f sta

ff us

age

com

pare

d by

fund

siz

e

Exi

sten

ce, i

f any

, of s

tand

ards

Rat

es o

f HQ

(G

enev

a an

d N

YC

) st

aff u

sage

pos

t 20

09

Per

cept

ion

of a

dequ

acy

of s

taff

allo

catio

n by

st

akeh

olde

r ty

pe

Fin

anci

al /

tran

sact

ion/

HR

da

ta

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Are

the

indi

rect

pro

gram

me

supp

ort c

osts

cha

rged

by

OC

HA

ap

prop

riate

and

how

do

they

co

mpa

re to

oth

er fu

ndin

g m

echa

nism

s?

Wha

t is

the

aver

age

leve

l of s

uppo

rt

cost

s?

Wha

t con

stitu

tes

such

cos

ts?

If th

ere

are

larg

e va

rianc

e (+

-10%

),

why

?

Has

ther

e be

en a

ny c

hang

e in

the

patte

rns/

leve

l of t

hese

cos

ts o

ver

the

lifet

ime

of th

e E

RF

pro

gram

me?

How

doe

s th

e le

vel o

f cos

ts c

ompa

re

with

oth

er O

CH

A a

ctiv

ities

and

with

ot

her

gran

t pro

gram

mes

of h

uman

itaria

n pa

rtne

rs?

Rat

es o

f sup

port

cos

ts v

ersu

s C

ER

F o

ver

time

and

by

loca

le

Deg

ree

of v

aria

nce

Fin

anci

al /

tran

sact

ion

/ HR

da

ta

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

H

ow c

lear

are

rol

es a

nd

resp

onsi

bilit

ies

of F

CS

and

oth

er

adm

inis

trat

ive

units

with

in O

CH

A,

HC

s, in

-cou

ntry

ER

F A

dvis

ory

Boa

rds

and

clus

ters

?

Are

rol

es a

nd r

espo

nsib

ilitie

s fo

rmal

ly

set o

ut?

Are

ther

e an

y ov

erla

ps o

r am

bigu

ities

(p

erce

ived

or

actu

al)?

Do

exte

rnal

sta

keho

lder

s pe

rcei

ve

clar

ity?

Do

stak

ehol

ders

inte

rnal

to O

CH

A

perc

eive

cla

rity?

Exi

sten

ce o

f rol

es a

nd r

espo

nsib

ility

ag

reem

ents

/pol

icie

s, e

tc.,

2007

to p

rese

nt

Evi

denc

e of

ove

rlaps

Cha

nges

sin

ce 2

007,

if a

ny

Per

cept

ion

of c

larit

y by

var

ying

type

s of

sta

keho

lder

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Ext

erna

l and

inte

rnal

su

rvey

s

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

Fu

nd

ing

an

d p

roje

ct is

sues

Co

mp

lem

enta

rity

– A

dd

itio

nal

ity

– Q

ual

ity

- A

pp

rop

riat

enes

s

How

effe

ctiv

e ar

e ap

proa

ches

to

ensu

re th

at th

e re

cipi

ent a

genc

ies

have

a c

ompa

rativ

e ad

vant

age

(suc

h as

cap

acity

to im

plem

ent

with

in th

e tim

efra

me

of th

e gr

ant,

past

per

form

ance

, spe

ed o

f di

strib

utio

n an

d ab

sorp

tive

capa

city

)?

Wha

t mec

hani

sms

are

used

to a

sses

s th

e or

gani

zatio

nal c

apac

ity o

f re

cipi

ents

?

Are

rem

edia

l/int

erna

l cap

acity

bui

ldin

g ac

tiviti

es p

erm

itted

?

Exi

sten

ce a

nd r

ate

of u

sage

of c

apac

ity a

sses

smen

t to

ols

as p

art o

f sel

ectio

n

Rat

es o

f exp

endi

ture

on

trai

ning

and

oth

er c

apac

ity

build

ing

activ

ities

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Fin

anci

al /

tran

sact

ion

data

How

is s

ucce

ss/fa

ilure

of p

roje

cts

fund

ed b

y E

RF

s m

easu

red?

D

o pr

ojec

ts h

ave

theo

ries

of c

hang

e /

logi

cal f

ram

ewor

ks b

uilt

in a

nd if

so

how

ar

e th

ey a

sses

sed?

Has

OC

HA

dev

elop

ed a

sel

ectio

n cr

iteria

pro

toco

l/ pa

radi

gm; a

nd if

so

wha

t con

stitu

tes

it?

Wha

t doe

s O

CH

A c

onsi

der

to b

e “h

igh

qual

ity”?

Doe

s it

vary

as

to c

onte

xt?

Req

uire

men

t to

have

logi

c m

odel

s/lo

g fr

ames

Exi

sten

ce o

f log

fram

es, e

tc.,

by c

ount

ry, i

f app

licab

le

List

of t

ypes

of o

ther

tool

s to

est

ablis

h an

ticip

ated

ou

tcom

es

Exi

sten

ce o

f sel

ectio

n to

ols

Rat

es o

f the

ir us

age

Rat

e of

per

cept

ions

of w

hat “

qual

ity”

impl

ies

by lo

cale

Exi

sten

ce w

ithin

ER

F g

rant

s of

tim

etab

les

for

mon

itorin

g

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

A

re th

ere

firm

or

flexi

ble

patte

rn o

f pr

ojec

t mon

itorin

g; a

nd if

they

are

fle

xibl

e w

hat c

onst

itute

s th

e “f

lexi

bilit

y”?

How

do

coun

try

leve

l adv

isor

y le

vel

(incl

udin

g ci

vil s

ocie

ty)

and

stak

ehol

der

grou

ps c

ontr

ibut

e to

sel

ectio

n de

cisi

ons?

Rat

e of

act

ual m

onito

ring

by O

CH

A s

taff

vers

us p

lans

by

loca

le

Exi

sten

ce o

f loc

al a

dvis

ory

bodi

es in

pro

ject

sel

ectio

n

Rat

e of

eng

agem

ent i

n se

lect

ions

by

loca

l adv

isor

y bo

dies

How

effe

ctiv

e is

OC

HA

in

iden

tifyi

ng r

isks

(in

im

plem

enta

tion,

fina

ncia

l, ac

coun

tabi

lity,

rep

utat

iona

l) an

d m

itiga

ting

them

?

Doe

s O

CH

A h

ave

a ris

k m

anag

emen

t st

rate

gy r

e: E

RF

s?

If so

how

is it

use

d?

Evi

denc

e of

ris

k id

entif

icat

ion

Evi

denc

e of

ris

k m

itiga

tion

stra

tegi

es

Evi

denc

e of

less

ons

lear

ned

on r

isks

Evi

denc

e of

inno

vatio

ns d

evel

oped

to r

educ

e ris

k

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

To

wha

t ext

ent d

o E

RF

-fun

ded

proj

ects

take

into

con

side

ratio

n ne

eds

of v

ulne

rabl

e gr

oups

and

cr

oss-

cutti

ng is

sues

suc

h as

ge

nder

, age

, and

env

ironm

ent?

Are

thos

e is

sues

a fa

ctor

in th

e se

lect

ion

of in

divi

dual

pro

ject

s?

Doe

s O

CH

A h

ave

form

al s

elec

tion

crite

ria th

at a

ddre

ss s

uch

horiz

onta

l pr

iorit

ies?

If so

how

doe

s it

use

them

?

Are

civ

il so

ciet

y gr

oups

rep

rese

ntin

g th

ese

vuln

erab

le g

roup

s in

clud

ed in

O

CH

A d

ecis

ion-

mak

ing/

advi

sory

m

echa

nism

s?

Evi

denc

e of

sel

ectio

n cr

iteria

and

form

al d

ecis

ion-

mak

ing

patte

rns

Cha

nges

ove

r tim

e

Evi

denc

e of

spe

cific

eng

agem

ent o

f dis

adva

ntag

ed

grou

ps

Evi

denc

e th

at s

elec

tion

crite

ria s

peci

fical

ly a

sk a

bout

ro

le/p

artic

ipat

ion/

impa

ct o

n di

sadv

anta

ged

grou

ps

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

To

wha

t ext

ent h

as th

e E

RF

co

ntrib

uted

to lo

cal c

apac

ity

build

ing

and

mad

e lin

kage

s to

di

sast

er r

isk

redu

ctio

n, r

ecov

ery

and

long

er te

rm d

evel

opm

ent

prog

ram

mes

?

Wha

t fac

tors

hin

der

or s

tren

gthe

n ca

paci

ty e

nhan

cem

ent?

Wha

t spe

cific

evi

denc

e ex

ists

of

cont

ribut

ions

by

ER

F to

long

er te

rm

goal

s

How

are

ER

F m

easu

res

re: D

RM

in

tegr

ated

/ co

ordi

nate

d w

ith th

e D

RM

w

ork

of o

ther

s?

Is th

ere

any

evid

ence

of o

verla

p be

twee

n de

velo

pmen

t par

tner

s?

Evi

denc

e of

DR

M c

onte

nt in

ER

F g

rant

ing

Rat

e of

sam

e, if

app

licab

le

Deg

ree

to w

hich

nat

iona

l gov

ernm

ent s

take

hold

ers

supp

ort D

RM

con

tent

Evi

denc

e of

pos

t pro

ject

mon

itorin

g to

ass

ess

capa

city

dev

elop

men

t (an

d ra

te o

f sam

e if

appl

icab

le)

Rat

e of

sat

isfa

ctio

n w

ith D

RM

as

a pr

ogra

mm

e el

emen

t by

stak

ehol

der

type

Evi

denc

e/pe

rcep

tions

of

over

lap

or la

ck o

f int

egra

tion

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings

Fin

anci

al /

tran

sact

ion

data

Per

form

ance

Mea

sure

men

t D

o ex

istin

g m

onito

ring,

rep

ortin

g an

d ev

alua

tion

syst

ems

prov

ide

rele

vant

and

tim

ely

info

rmat

ion

to

the

right

aud

ienc

e? H

ow is

the

info

rmat

ion

prov

ided

use

d? W

hat

are

the

barr

iers

(an

d op

port

uniti

es)

to s

tren

gthe

ning

im

pact

/out

com

e re

sults

rep

ortin

g?

Wha

t sys

tem

s/pr

oced

ures

/pol

icie

s cu

rren

tly e

xist

?

Wha

t is

thei

r fr

eque

ncy?

Who

are

the

audi

ence

s?

Wha

t are

see

n as

cur

rent

str

engt

hs; a

nd

wha

t are

see

n as

cur

rent

lim

itatio

ns?

To

wha

t deg

ree

is im

pact

/out

com

e re

port

ing

re: b

enef

icia

ry im

pact

bei

ng

unde

rtak

en?

Wha

t bar

riers

exi

st?

How

cou

ld th

is b

e im

prov

ed?

Typ

e of

per

form

ance

mea

sure

men

t too

ls a

nd th

eir

freq

uenc

y an

d au

dien

ces

Rat

e of

app

licat

ion

of a

bove

(to

asc

erta

in c

ompl

ianc

e)

Leve

l of s

atis

fact

ion

with

OC

HA

inte

rnal

sta

keho

lder

s

Exi

sten

ce o

f out

com

e/im

pact

indi

cato

rs a

nd r

ate

of

thei

r us

age

acro

ss 1

2 ac

tive

ER

Fs

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Wha

t key

indi

cato

rs a

re u

sed

for

mea

surin

g th

e su

cces

s of

ER

Fs

and

how

can

dat

a co

llect

ion

proc

esse

s be

str

engt

hene

d?

Wha

t are

cur

rent

key

indi

cato

rs; a

nd is

th

ere

agre

emen

t on

com

mon

ality

?

To

wha

t deg

ree

has

data

col

lect

ion

harm

oniz

atio

n be

en u

nder

take

n?

Wha

t are

see

n as

cur

rent

str

engt

hs; a

nd

wha

t are

see

n as

cur

rent

lim

itatio

ns r

e:

key

indi

cato

rs?

List

of c

urre

nt in

dica

tors

, deg

ree

of c

omm

onal

ity

Rat

e of

usa

ge

Leve

l of s

atis

fact

ion

with

cur

rent

indi

cato

rs

Leve

l of s

atis

fact

ion

with

pro

pose

d ne

w in

dica

tors

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Vol

ume

II: C

ount

ry C

ase

Stu

dy R

epor

ts

Issu

e/T

hem

e M

ajo

r E

valu

atio

n

Qu

esti

on

s/S

ub

-Th

eme

Exa

mp

le o

f S

ub

-Qu

esti

on

s Ill

ust

rati

ve In

dic

ato

rs18

S

ou

rces

of

Dat

a

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

W

hat e

valu

atio

n m

etho

dolo

gy

mig

ht b

e be

st a

pplie

d to

ER

F

gran

ts, g

iven

thei

r av

erag

e si

ze

and

the

fact

that

they

ofte

n pr

ovid

e on

ly p

artia

l fun

ding

for

proj

ects

?

Wha

t met

hodo

logi

es h

ave

been

use

d in

th

e pa

st if

any

by

OC

HA

to a

sses

s sm

all

size

d ac

tiviti

es?

Wha

t met

hodo

logi

es a

re u

sed

by o

ther

hu

man

itaria

n pa

rtne

rs?

Wha

t are

the

cost

s ve

rsus

ben

efits

of

eval

uatio

n at

the

scal

e of

ER

F g

rant

s?

List

of m

etho

dolo

gies

use

d in

indi

vidu

al E

RF

ev

alua

tions

Deg

ree

of c

omm

onal

ity a

mon

g th

em

Too

ls u

sed

by o

ther

s to

ass

ess

sub

US

$500

K

proj

ects

List

/type

of c

apac

ity s

tren

gths

and

cha

lleng

es

com

parin

g co

untr

y le

vel v

ersu

s H

Q le

vel

List

/type

of c

apac

ity s

tren

gths

and

cha

lleng

es o

f re

cipi

ents

ass

esse

d ag

ains

t lev

el o

f cou

ntry

de

velo

pmen

t

Inte

rnal

and

ext

erna

l su

rvey

s

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

Rep

orts

/ au

dits

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sho

uld

com

mitm

ent t

o ev

alua

tion

be in

clud

ed a

s a

crite

rion

for

futu

re fu

ndin

g de

cisi

ons?

Wha

t is

the

expe

rienc

e of

oth

er

hum

anita

rian

part

ners

in th

e as

sess

men

t of m

ediu

m s

ized

gra

ntin

g pr

ogra

mm

es s

uch

as E

RF

s?

Wha

t are

the

cost

s ve

rsus

ben

efits

of

eval

uatio

n at

the

scal

e of

ER

Fs

and

ER

F in

divi

dual

gra

nts?

Deg

ree

to w

hich

don

ors

use

eval

uatio

ns o

f ind

ivid

ual

hum

anita

rian

proj

ects

sub

US

$500

K

Leve

l of s

uppo

rt fo

r pr

ojec

t lev

el e

valu

atio

n am

ong

OC

HA

sta

ff

Leve

l of s

uppo

rt a

mon

g IN

GO

s an

d N

NG

Os

Des

k R

evi

ews

Cou

ntry

doc

umen

t rev

iew

s

HQ

doc

umen

t rev

iew

s

Inte

rvie

ws

with

inte

rnal

and

ex

tern

al s

take

hold

ers

Sm

all g

roup

mee

tings


Terms of Reference

GLOBAL EVALUATION

OF THE

EMERGENCY RESPONSE FUNDS (ERFS)


RATIONALE AND USE OF THE EVALUATION

In 2011, in consultation with donors, OCHA instituted an evaluation policy to conduct triennial global evaluations of country-based pooled funds. As such, a global evaluation of the Emergency Response Funds (ERFs) will be conducted in 2012, replacing individual ERF evaluations periodically undertaken in the past. This evaluation meets the requirements of the Country-Based Pooled Funds evaluation policy by providing an independent assessment of the contribution of ERFs to the humanitarian community’s ability to address critical unforeseen humanitarian needs in a timely and effective manner.

The evaluation will identify strengths and weaknesses of the ERF mechanism and provide specific recommendations regarding areas that need to be strengthened, as well information on progress made since the creation of OCHA Funding and Coordination Section in 2009.The evaluation will also explore how the mechanism has contributed to humanitarian reform and response, and will take into account, to the extent possible, the implications of the ongoing IASC Transformative Agenda. The results of the evaluation will at the global level inform the review of ERF Guidelines in 2013 and development of policy in relevant areas; at the country level, the evaluation is expected to lead to improvements in ERF management, processes and operations. The recommendations of the evaluation will be addressed through the Management Response Plan as per OCHA Evaluation Policy.

Overview of the Emergency Response Funds (ERFs) The name Emergency Response Fund (ERF) is an umbrella term19 covering a broad range of country-based pooled funds in use since 1997. The main objective of ERFs is to provide Non-Governmental Organizations (NGOs) and United Nations (UN) agencies with rapid and flexible in-country funding to address unforeseen humanitarian needs. ERFs provide governments and the private sector with an opportunity to pool their contributions to a specific country to enable timely and coordinated humanitarian assistance in response to emergencies. The aim of ERFs is to provide initial funding, fill critical gaps and bridge funding to enable humanitarian partners to meet the short-term emergency needs of vulnerable communities without delay. They are usually established to meet unforeseen needs not included in the Consolidated Appeals Process (CAP) or similar coordination mechanism, but in line with Common Humanitarian Plan (CHAP) objectives and identified priorities. ERFs are not intended to provide core funding to projects or programmes in protracted crises, although some ERFs, especially when they are sizeable, may fund critical gaps in the CAP (Consolidated Appeals Process). OCHA created a Funding and Coordination Section (FCS) in 2009 to oversee funding mechanisms such as ERFs and to provide technical advice and support to OCHA at the country level in the development of policy and procedure relating to funding mechanisms.

As of March 2012 there are 12 ERFs in operation: Afghanistan, Colombia, Democratic Republic of Congo (DRC), Ethiopia, Haiti, Indonesia, Iraq, Kenya, Myanmar, Nepal, occupied Palestinian Territory, Pakistan, Uganda, Yemen and Zimbabwe. Three ERFs, namely Iraq, Nepal and Uganda, have closed at end of 201120.Generally, ERFs are relatively small in size (less than $10 million), provide small to medium sized grants (less then $500,000), and predominantly fund NGOs. The first ERF, established in Angola in 1997 to respond to increasing humanitarian needs caused by years of conflict, had received US$24.5 million from eight donors until its closing in 2004. The Ethiopia HRF is continuously the largest fund since the creation of ERFs and is unique as it operates in a country with no CAP and where it is the only humanitarian financing mechanism. The Haiti ERF, with the largest annual portfolio in the history of the mechanism, has received a total of approximately $82million in donations from at least 42 different donors since August 2008 (majority of the funding was triggered by the 2010 Earthquake).

An ERF is under the overall management and oversight of the Humanitarian Coordinator (HC) with day-to-day management and financial administration performed by OCHA. In response to emerging humanitarian needs, partners submit proposals for ERF funding to OCHA, and the HC, supported by a technical Review Board and by sector/cluster groups, makes decisions on grants. An Advisory Board, comprised of donor, UN and NGO representatives, advises the HC on policy issues and strategic direction of the fund. The specifics of the individual funds reflect the country contexts in which they have been established and therefore differ to varying degrees in purpose, approach and practice.

19 ERFs are also known as Humanitarian Response Funds (HRFs), Emergency Response and Reconciliation Fund (ERRF), Humanitarian Multi-Stakeholder Fund (HMSF) 20 The list of ERFs active during the period of 2009 to 2011, together with the dates of their creation and annual and total contribution, is presented in Annex to this ToR


Previous Evaluations and Reviews A number of evaluations and studies since the Humanitarian Reform in 2005 have looked at humanitarian pooled funding mechanisms managed by OCHA. Two evaluations of Common Humanitarian Funds (CHF) were commissioned21, and three of Central Emergency Response Fund (CERF)22. The 2007 Review of OCHA Emergency Response Funds23 emphasized their contribution to humanitarian response, particularly with respect to improved ability to quickly respond to sudden emergencies, fillgaps in humanitarian response, and scale up. Also noted, however, was the need to further refine and strengthen ERF processes, connectedness to other funds, and institutional relations. Two evaluations of individual ERFs were conducted in 2011, that of the Haiti Emergency Relief and Response Fund (ERRF) and of Pakistan ERF. In addition to these evaluations, there were some critical audits of the country-based pooled funds: Sudan CHF in 2010, Ethiopia and Somalia HRFs in 2010 and Haiti ERRF in 2011. This evaluation will consider relevant findings and recommendations made in previous evaluations and audits and follow-up actions taken.

A number of significant changes occurred in the humanitarian financing arena since the Review of OCHA Emergency Response Funds was conducted in 2007, making this evaluation particularly timely. These include the adoption of humanitarian financing by OCHA senior management as a core function, establishment of OCHA FCS in New York and a dedicated unit in OCHA Administrative Services Branch to support ERFs, and the creation of an IASC Sub-Working Group on Humanitarian Financing. In addition, seven new ERFs were established during this period.

Evaluation Objectives, Scope, and Purpose The objectives of the evaluation are to:

Provide an independent assessment of the contribution of ERFs to the humanitarian community’s ability to address critical unforeseen humanitarian needs in a timely and effective manner;

Examine the contribution of ERFs to strengthening the leadership of Humanitarian Coordinators, coordination role of the cluster system, and building partnerships (in particular with national and local NGOs);

Review the accountability and governance structure of ERFs;

Examine the role of the FCS, Coordination and Response Division (CRD), Donor Relations Section (DRS) and Administrative Service Branch (ASB) as well as the role of Head of Country Office and the Emergency Relief Coordinator (ERC), map the progress made since 2009 in improving the effectiveness and efficiency of ERFs, and identify the areas of strengths and weaknesses.

The evaluation will examine the processes, outcomes, effects and operational impact of ERFs as outlined in the Results Framework depicted in the section VI of this TOR. It will cover all ERFs in operation during the period from 2009 to 2011, with five currently active ERFs to be selected for field visit.

While the evaluation does not cover countries where ERFs operated and were subsequently merged into CHFs, it will consider both the closed funds and the ones incorporated in CHFs to provide insight into: (a) issues of ERF closure; (b) overall usefulness of ERFs for the HC; (c) integration of policy and management of ERFs with other humanitarian funds; and (d) policy and operational issues of ERF support to NGOs and UN agencies.

The evaluation will generate specific, actionable recommendations whose purpose is twofold: at the global level, to inform the review of ERF Guidelines in 2013 and the development of policy in relevant areas, and at the country level, to lead to improvements in ERF management, processes and operations.

ERF Result Framework and key areas of inquiry Key areas to be examined are:

The success of ERFs in meeting their goal to support early action and response to critical unforeseen humanitarian needs in a timely and effective manner;

The performance, ‘value added’, and cost effectiveness of ERFs in the context of the overall humanitarian architecture and humanitarian reform process (e.g. RC/HC strengthening, cluster approach, development of effective partnership between UN and non-UN humanitarian actors, including local and national NGOs);

The level and nature of accountability between ERF-recipients to RC/HC, donors, beneficiaries, and others;

21 Evaluation of Common/Pooled Humanitarian Funds in DRC and Sudan in 2007 and the Global CHF Evaluation in 2011 22 2007 Interim Review, 2008 Two-Year Evaluation, and 2011 Five-Year Evaluation 23Review of OCHA Emergency Response Funds, Development Initiatives, 2007


The strategic, managerial and operational aspects of ERFs, including structure and focus, the efficiency and effectiveness of administration, fundraising, needs assessment, criteria for allocation of funds, monitoring and reporting, and connectedness with other funding sources;

Internal (e.g. staffing capacity, management arrangements) and external (e.g. contextual and political) factors which affect ability of ERFs to deliver on their objectives;

The accountability and risk management systems in place.

The analysis should be structured around the ERF Results Framework depicted in the figure below. The evaluation team should test the validity of the Framework and may suggest modifications to the Framework itself or to the proposed list of key questions in the inception phase of the evaluation.

ERF Results Framework/Key Questions

Operational Impact

Critical unforeseen humanitarian needs addressed

How, and to what extent has ERF mechanism contributed to the humanitarian community’s ability to address critical unforeseen humanitarian needs in a timely and effective manner?

What has the performance and ‘value added’ of the ERF mechanism been in the context of the overall humanitarian architecture and humanitarian reform process (e.g. RC/HC strengthening, cluster approach,

development of effective partnership between UN and non-UN humanitarian actors, humanitarian financing)?

Operational Effects

Relevance/Appropriateness - Flexibility – Timeliness - Coverage

To what extent has the use of ERF funds improved operational effects of the humanitarian response, in particular relevance/appropriateness, flexibility, timeliness and coverage? What specific features of ERFs have contributed to

these improvements?

How relevant/appropriate are the objectives the ERF mechanism and the portfolio of projects financed to humanitarian needs in respective countries? Has ERF funding been directed to the highest priority areas


(geographical, thematic, and sectoral) within the response?

How adaptive is the ERF mechanism to changing needs? How flexible is ERF funding?

To what extent has the availability/use of ERF funds improved the timeliness of the response?

How do ERFs link with other humanitarian pooled funds - namely CERF, and other MDTFs? What is the value added of having ERFs as a complement to other funding streams? What is the effect of the presence of an ERF on

the overall levels of humanitarian funding available for a crisis?

How does the size of ERFs impact their added value, in particular where other pooled funds are present?

How does the operating environment impact the effectiveness and value-added of ERFs?

Outcomes

Critical gaps filled - Leadership and coordination -Partnerships

To what extent has the availability of initial and bridge funding and un-earmarked nature of the funds allowed better targeting and contributed to filling critical gaps in the response?

To what extent has the ERF mechanism contributed to further strengthening the role of the RC/HC and the clusters? What opportunities does the IASC Transformative Agenda provide in terms of contribution of ERFs in

strengthening the humanitarian response leadership?

To what extent has the ERF mechanism help improve prioritization discussions and decisions within and across clusters?

How, and to what extent has the ERF mechanism contributed to enhancing partnerships, in particular with the NGO community (international and national)?

Processes and Inputs

Effective and transparent proposal review - Timely allocation and distribution - Collective prioritization of need - Reporting, monitoring, evaluation and audit

How effective/efficient are ERFs in providing initial and bridge funding to address critical unforeseen humanitarian needs?

How timely are allocation, disbursement, and transfer of funds? How does timeliness of ERF funding compare with other funding sources?

To what extent do the vetting and quality control mechanisms impact allocation and timely disbursement of funds to partners (particularly national NGOs)?

How transparent is management, allocation, and distribution of the funds?

To what extent are the prioritization and vetting processes inclusive, thorough and based on good-quality criteria? Do these processes represent due diligence?

Do existing monitoring, reporting and evaluation systems provide relevant and timely information to the right audience? How is the information provided used? What are the barriers (and opportunities) to strengthening

impact/outcome results reporting?

Are both headquarters and country level stakeholders receiving all necessary and relevant information?

How effective, efficient and timely are quality control mechanisms in place for: 1) planning and design, 2) approval of projects, 3) fund disbursement; 4) monitoring, reporting and review; 5) auditing?

How adequate is OCHA’s capacity to administer and manage ERFs, and to what extent is it dependant on the operating context?

How well are the individual ERFs managed?

How effective are the structures established for governance and accountability, for individual funds and globally?

Are the indirect programme support costs charged by OCHA appropriate and how do they compare to other funding mechanisms?

How effective and efficient is FCS in overseeing ERFs and providing technical advice and support to OCHA at the country level in the development of relevant policy and procedures? How effective and efficient is the support from

the CRD, DRS and ASB?


How is the role of ERC as a decision maker for establishment/closure of ERFs and administrator for OCHA's trust funds (which are used as ERF channels) defined and executed? Have the appropriate measures to mitigate the

risks associated with those roles been taken?

Funding and project issues

Complementarity – Additionality – Quality - Appropriateness

How effective are approaches to ensure that the recipient agencies have a comparative advantage (such as capacity to implement within the timeframe of the grant, past performance, speed of distribution and absorptive

capacity)? How clear and transparent are the criteria for eligibility of funding and selection of projects?

How is success/failure of projects funded by ERFs measured? Are the projects funded of demonstrated high quality?

How effective is OCHA in identifying risks (in implementation, financial, accountability, reputational) and mitigating them?

To what extent do ERF-funded projects take into consideration needs of vulnerable groups and cross-cutting issues such as gender, age, and environment? Are those issues a factor in the selection of individual projects?

To what extent has the ERF contributed to disaster preparedness, in particular contingency planning and pre-positioning of supplies?

How have ERFs affected the capacities of recipient agencies? To what extent has the ERF contributed to local capacity building and made linkages to disaster risk reduction, recovery and longer term development

programmes?

Additional Areas of Investigation A few areas of investigation outside the Result Framework are suggested below. Additional issues, if identified during the inception phase of the evaluation (either specific issues regarding individual ERFs or any other question that may arise during consultations) may be included in the analysis.

How clear are roles and responsibilities of FCS and other administrative units within OCHA, HCs, in-country ERF Advisory Boards and clusters?

What progress has been made since the inception of ERFs (how were shortcomings identified by previous evaluations/audits addressed; what are the lessons learned from implementing these recommendations)? How can ERFs continue to be strengthened and processes improved?

What key indicators are used for measuring the success of ERFs and how can data collection processes be strengthened?

What evaluation methodology might be best applied to ERF grants, given their average size and the fact that they often provide only partial funding for projects? Should commitment to evaluation be included as a criterion for future funding decisions?

Evaluation Methodology The evaluation will use mixed method analysis, employing the most appropriate qualitative and quantitative approaches and data types, and triangulating various data sources for validation. Data will be derived from primary and secondary sources, including, i.a., key informant interviews in headquarters and field, focus groups, surveys of stakeholder groups, direct observation in the field, in depth financial analysis and analysis of monitoring reports, disbursement time frames and processes, meeting minutes, previous evaluations and audits of humanitarian pooled funds, and evaluations of the use of funds or of the projects funded that might have been conducted by recipient agencies. All data, quantitative and qualitative, will be disaggregated by gender and age where applicable and possible.

The Evaluation Team will conduct field visits to five recipient countries and produce a short standalone report for each country’s ERF in addition to the main evaluation report. The country reports should identify any improvements that would help strengthen the functioning of the funds, and areas working particularly well which might be systematized and applied in other ERF contexts, as well as documenting areas where a flexible approach has benefited the country.


The evaluation team will propose the sampling criteria to identify a cross-section of ERF recipient countries for field missions24. The final decision on the country selection will be made by the OCHA Evaluation and Guidance Section. The data collection tools and methods will be standardized across countries; fund-specific issues or questions may, however, be addressed in country reports should they arise during consultations or are requested by the ERF Advisory Boards. Selected projects funded by ERFs should be analyzed to provide insight into full project cycles and the impact that ERF funding had on particular projects.

Perspectives from all stakeholders should be solicited including: recipient organizations, the RC/HC, the HCT, advisory groups, clusters, OCHA, NGOs, government stakeholders in recipient and donor countries, civil society groups and members, and beneficiaries. Local and national NGO perceptions will be sought throughout the evaluation process.

The Evaluation Team will propose detailed methodology during the inception phase, which will include any revisions to the ERF Results Framework proposed below, a detailed description of indicators, tools, triangulation plan, gender/age analysis to be used, and validation strategy.

The evaluation will employ the criteria for humanitarian evaluations25 recommended by ALNAP, namely: Effectiveness, Efficiency, Coverage, Appropriateness and Relevance, and Coordination. The evidence will be collected at all levels of the results hierarchy, including process/inputs, outcomes, effects and operational impact. The analysis will focus on ‘operational impact’, that is, the ‘impact’ of the ERFs on the humanitarian system to respond to sudden onset, unforeseen new developments and their role in the larger humanitarian response. While the impact at the beneficiary level is not the primary focus of this evaluation - the underlying assumption is that effective and principled financing will contribute to better humanitarian outcomes - the evaluation will address the issue to the extent possible.

Management and Governance Arrangements

Evaluation Management

The evaluation will be managed by OCHA’s Evaluation and Guidance Section (EGS). OCHA EGS will appoint an Evaluation Manager. Evaluation and Guidance Section in its position as managing entity will:

Ensure the availability of funding for the evaluation;

Manage the evaluation in accordance with agreed budget and timeline;

Coordinate, convene and chair Reference Group;

Draft and finalize the ToR, with input from the Reference Group;

Advertise for, select and recruit the evaluation team;

Facilitate the evaluation team’s access to key stakeholders and specific information or expertise needed to complete the evaluation;

Help coordinate field research and workshops/ presentations;

Advise the evaluation team on strategic direction of the evaluation and provide guidance and input on methodology, content and recommendations;

Provide assistance with technical standards;

Serve as principal interlocutor between the Evaluation Team and Reference Group;

Ensure all stakeholders are kept informed;

Review, comment and approve all deliverables of the evaluation, including the final Evaluation Report; and

Review and facilitate the follow up and a management response to the evaluation, ensuring that the evaluation findings conclusions and recommendations are correctly interpreted.

24 Due to demand from the field and ERF donors, oPT and Columbia will be included in the list of countries selected for field missions; the two countries should be the first ones visited. As ERFs in Pakistan and Haiti have been evaluated within the past 2 years, those countries will not be selected for field missions. 25 Tony Beck, Evaluating humanitarian action using the OECD-DAC criteria for humanitarian agencies: An ALNAP guide for humanitarian agencies. Overseas Development Institute: London (March 2006).


Reference Group (Headquarters-Level)

A headquarters-level Reference Group (RG) will be established to guide the evaluation and help ensure its relevance and transparency, as well as accuracy, credibility of findings, correspondence to the practical concerns of stakeholders and their ownership of the process. RG will be chaired by EGS, and comprised of primary external stakeholders of the evaluation and representatives from OCHA FCS and Administrative Services Branch (ASB). The primary external stakeholder groups identified are recipient agencies (UN and NGO) and donor countries. As such, FAO, UNICEF, and WFP (as the largest UN recipients of ERF funding); Oxfam, SCF, ICVA and Interaction (representing the NGOs); and Sweden, UK, and Netherlands (as the constant largest donor countries) will be invited to nominate representatives to the RG.

Reference Group will serve in an advisory capacity without management responsibilities. Members will serve as focal points within their organizations, collecting and disseminating information (including any available monitoring and evaluation data on the use of ERF funds), organizing the follow-up to recommendations within their organizations, and may assist the Evaluation Manager with the coordination of field missions on behalf of the Evaluation Team. The RG members will be expected to review and provide appropriate and timely feedback on all evaluation products (i.e. ToR, Inception Report, Country Reports, and draft Report) and on the selection of the evaluation team.

An additional responsibility of a designated RG member will be to provide occasional updates to the two key coordination bodies related to humanitarian financing, e.g. the IASC Sub Working Group on Humanitarian Financing (IASC SWG HF) and the Pooled Fund Working Group (PFWG). Once the membership of the reference group is finalized one UN Agency will be asked to provide this function to the IASC SWG HF. DfID has agreed to act as a focal point for the evaluation with the PFWG, as well as to work on facilitating participation of emerging donors in the evaluation process.

The Reference Group will be the sole advisory body to the evaluation process. The IASC HF SWG and the PFWG will be kept informed during the process and should direct any type of communication through the respective RG member.

OCHA Country Offices and country-level ERF Advisory Boards

OCHA Country Offices (COs) will help facilitate evaluation team’s access to key informants and relevant documentation in the countries selected for field visits, and may propose additional issues specific to the ERF under consideration. The Humanitarian Coordinator/ERF Advisory Boards will be provided an opportunity to review and comment on the draft Country Reports and to suggest additional areas of enquiry; their input will be coordinated by OCHA COs.

Timeline and Phases of the Evaluation The evaluation has been divided into four phases for which dates are estimated. Final dates for some components on the evaluation will be dependent upon the date of contracting the evaluation team.

Inception July 2012- August 2012

Meetings with headquarters-based stakeholders

(New York and Geneva)

Key informants interviews

and methodology development

Draft and final Inception Report

Research August – October 2012

Field research

Desk Review

Country validation presentations

Country reports

Reporting December 2012 – January 2013

Production of draft and final reports

Presentation of findings

(New York or Geneva)

Follow-up January 2013 Management Response Plan prepared


Deliverables and Reporting Requirements The quality of the evaluation report will be assessed according to the UNEG Evaluation Standards and ALNAP Quality Proforma26.

All deliverables listed below will be written in good Standard English. If in the estimation of the Evaluation Manager the reports do not meet this required standard, the Evaluation Team will ensure at their own expense the editing needed to bring it to the required standard.

Inception Report

The Evaluation Team will produce an inception report not to exceed 4500 words, excluding annexes, setting out:

The team’s understanding of the functioning of emergency response fund mechanism and OCHA’s mandate in managing ERFs;

Stakeholder analysis and plan for their involvement in the evaluation process;

The team’s initial understanding of the context(s) in which the ERFs operate Methodology, including any changes to the proposed results framework and key questions, comparative analysis, detailed triangulation strategy, approach to qualitative data analysis and gender analysis, and a strategy for addressing impact on beneficiary level given the constrains faced in measurement and attribution;

Any suggested deviations from the ToR, including any additional issues raised during the initial consultations;

An evaluation matrix showing, for each question, the indicators proposed and sources of information;

Data collection plan;

Detailed fieldwork plan, including sampling criteria and proposal of the countries for field visits;

Remaining evaluability issues and how they will be addressed;

Interview guides, survey instruments, and/or other tools to be employed for the evaluation;

Draft dissemination strategy of the evaluation findings and recommendations; and

Draft outline for the evaluation report and country level reports.

Evaluation Report

The Evaluation Team will produce a single report which is comprised of:

Executive summary of no more than 2500 words;

List of acronyms;

Table of contents;

Summary table of conclusions and recommendations, including where responsibility for follow up should lie;

Analysis of context in which ERFs were implemented and operating(including individual country contexts as well as the humanitarian reform process and other financing tools);

A desk review of previous evaluations and studies which relate to ERFs, and a summary explanation of how this evaluation is positioned among them;

Overview of how the ERF is being used in each country: objectives, amounts to various categories of agency and types of activity, etc;

Methodology summary – a brief chapter, with a more detailed description provided in the annex;

Main body of the report, including findings in response to the evaluation questions, conclusions and recommendations;

Annexes will include: (1) ToR, (2) Detailed methodology, (3) Analysis of ERF funding flows from 2004 onwards, (4) List of persons met, (5) Details of all surveys undertaken, (6) Details of any quantitative analysis undertaken, (7) Team itinerary, (8) All evaluation tools employed, and (6) bibliography of documents (including web pages, etc.) relevant to the evaluation.

For accuracy and credibility, recommendations should be the logical implications of the findings and conclusions. Recommendations should:

26Availableviawww.alnap.org/pdfs/QualityProforma05.pdf.


Follow logically from the evaluation findings and conclusions;

Be relevant to the intervention and reflect the reality of the context within which ERF operates;

Be clearly stated and not broad or vague;

Be realistic and reflect an understanding of OCHA and potential constraints to follow-up;

Be prioritized with a timeframe for follow-up; and

Suggest where responsibility for follow-up should lie.

Country Reports

The Evaluation Team will produce a short report (aide-memoire) of no more than 10,000 words for each of the five Countries selected for the field visit. The country reports should present findings and conclusions relating to the particular ERF examined, identify any improvements that would help strengthen its functioning and identify areas working particularly well which might be systematized and applied in other ERF contexts. Country reports are expected to be drafted during the mission and serve as the basis for the in-country validation workshop. Minimal revisions should be concluded in the following week.

Reporting and dissemination strategy

The Evaluation Team will conduct the following presentations:

At the end of each field visit, the Team will conduct a validation presentation with an out-brief of main findings to primary stakeholders;

Upon completion of the draft evaluation report, validation workshops will be held in New York or Geneva;

Once the evaluation is completed, presentations of the main findings and recommendations will be made to various fora as decided by OCHA and the Reference Group. The Evaluation Team may be requested to assist with these presentations.

For recommendation relating to OCHA, Management Response Plan will be prepared and follow up actions tracked by OCHA Strategic Planning Unit, as per OCHA Evaluation Policy. Response to recommendations and follow-up plans will be made public, as will any follow up action taken.

In addition to the Evaluation Report and oral briefings, the evaluation findings and recommendations can be presented through alternative ways of dissemination, such as video. The Evaluation Team will consider possible ways to present the evaluation and include a dissemination strategy proposal in the Inception Report.


Annex 1 to the ToR

Emergency Response Funds active during the period from 2009 to 2011

ERF Dates Donations 2009 Donations 2010 Donations 2011 Total 2009 to 2011

Afghanistan 2009 $3,450,250.00 $2,829,261.00 $4,800,880.00 $11,080,391.00

Colombia 2009 $1,425,274.00 $2,130,836.00 $2,410,470.00 $5,966,580.00

DRC 2004 $0.00 $0.00 $1,000,000.00 $1,000,000.00

Ethiopia 2006 $45,631,278.00 $15,471,870.00 $42,952,693.00 $104,055,841.00

Haiti 2008 $0.00 $81,833,084.00 $471,647.00 $82,304,731.00

Indonesia 2001 $2,986,650.00 $728,200.00 $2,340,450.00 $6,055,300.00

Kenya 2009 $2,592,709.00 $0.00 $3,751,651.00 $6,344,360.00

Myanmar 2007 $1,988,271.00 $0.00 $2,199,674.00 $4,187,945.00

oPt 2007 $7,551,379.00 $5,313,378.00 $2,432,646.00 $15,297,403.00

Pakistan 2010 $0.00 $36,655,292.00 $1,951,726.00 $38,607,018.00

Yemen 2010 $0.00 $2,510,023.00 $6,499,960.00 $9,009,983.00

Zimbabwe 2008 $3,883,495.00 $717,538.00 $888,415.00 $5,489,448.00

Nepal 2008 - 2011 $0.00 $0.00 $0.00 $0.00

Uganda 2009 - 2011 313,972 $618,687.00 $0.00 $932,659.00

Iraq 2007 - 2010 $621,012.00 $0.00 $0.00 $621,012.00

Total 2009 - 2011 $290,952,671.00

the global evaluation of emergency response funds (erfs) · needs of those beset by natural...

Documents