decentralized evaluation - docs.wfp.org

73
Decentralized Evaluation Decentralized evaluation for evidence-based decision making Prepared by Khadijhatou Seck, Regional Consultant September 2017 UN Network for SUN (UNN)/REACH Secretariat Evaluation Manager: Tania Goossens Thematic Evaluation – End of Term Evaluation “Renewed Efforts Against Child Hunger and undernutrition (REACH)” Burkina Faso Case Study

Upload: others

Post on 27-Jan-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Decentralized Evaluation - docs.wfp.org

Decentralized Evaluation

Dec

entr

ali

zed

ev

alu

ati

on

fo

r ev

iden

ce-b

ase

d d

ecis

ion

ma

kin

g

Prepared by

Khadijhatou Seck, Regional Consultant

September 2017

UN Network for SUN (UNN)/REACH Secretariat

Evaluation Manager: Tania Goossens

Thematic Evaluation – End of Term Evaluation “Renewed Efforts Against Child Hunger and undernutrition (REACH)” Burkina Faso Case Study

Page 2: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study i | P a g e

Acknowledgements

The evaluator sincerely thanks the UNN/REACH Secretariat in Rome and REACH

facilitators in Burkina Faso for facilitating the evaluation process. The staff gave

generously of their time to support the team and provide relevant information.

Our gratitude also goes to all stakeholders (the government, the donor community, the

United Nations agencies and non-governmental organizations), who dedicated their time

to participate in informative discussions and contributed significantly to the evaluation

findings.

Disclaimer

The opinions expressed in this report are those of the Evaluation Team, and do not

necessarily reflect those of the World Food Programme (WFP). Responsibility for the

opinions expressed in this report rests solely with the authors. Publication of this

document does not imply endorsement by WFP of the opinions expressed.

The designation employed does not imply the expression of any opinion whatsoever on the

part of WFP concerning the legal or constitutional status of any country, territory or sea

area, or concerning the delimitation of frontiers.

Page 3: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study ii | P a g e

Table of Contents

Acknowledgements .................................................................................. i

Disclaimer ............................................................................................... i

Executive Summary ............................................................................... iii

Key Findings .............................................................................................................................. iv

Overall Conclusions ................................................................................................................... v Recommendations .................................................................................................................... vi

1. Introduction ....................................................................................... 1

1.1. Evaluation characteristics ................................................................................................. 1 1.2. Country-specific context .................................................................................................... 2

1.3. REACH in Burkina Faso .................................................................................................... 3

2. Evaluation Findings ............................................................................ 5

2.1. Evaluation Question 1 — What are REACH results? ..................................................... 5

Effectiveness .......................................................................................................................... 5

Efficiency............................................................................................................................. 10

Equity .................................................................................................................................. 11

2.2. Evaluation Question 2 — What are the explanatory/contributing factors explaining results? ............................................................................................................ 12

2.3. Evaluation Question 3 — To what extent are the results achieved and the REACH operational models sustainable? ..................................................................... 14

3. Conclusions and Recommendations ................................................... 15

3.1. Conclusions .............................................................................................................. 15 3.2. Lessons learned and good practices......................................................................... 16 3.3. Recommendations .................................................................................................... 16

Annex 1: Terms of Reference ..................................................................................................... 18

Annex 2: Evaluation Matrix ....................................................................................................... 48

Annex 3: Country Visit Schedule ............................................................................................... 56

Annex 4: List of People Interviewed .......................................................................................... 58

Annex 5: Data Collection Tools ................................................................................................. 59

Annex 6: Bibliography ............................................................................................................... 60

Annex 7: Table - CIP Planned Outcomes, Outputs and Activity Categories versus AWP ........ 61

Annex 8: UNN/REACH Secretariat Support ............................................................................. 63

Acronyms .................................................................................................................................... 64

Tables Table 1: Progress in outcome 1 ...................................................................................................... 7

Figures Figure 1: Planned budget ...................................................................................................... 4

Figure 2: Planned budget versus implemented resources 2014-2016 ................................ 11

Page 4: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study iii | P a g e

Executive Summary

Introduction

1. This evaluation is commissioned by the UN Network for Scaling Up Nutrition (SUN)/Renewed Efforts against Child Hunger and undernutrition (UNN/REACH) Secretariat. It is undertaken as per agreement of the UNN/REACH Secretariat with Global Affairs Canada (GAC), which provides funding to REACH in Burkina Faso, Haiti, Mali, Myanmar and Senegal to support government-led nutrition governance efforts by pursuing four outcomes: 1) increased awareness and consensus among stakeholders of the nutrition situation and the best strategies and priorities for improvement; 2) strengthened national policies and programmes that operationalize and address nutrition through a multi-sectoral approach; 3) increased human and institutional capacity on nutrition actions at all levels; and 4) increased stakeholder effectiveness and accountability in implementing and supporting nutrition actions.

2. The objectives of this evaluation are two-fold: 1) accountability—assess and report on the performance and results of REACH; and 2) learning—determine the reasons why certain results did or did not occur, draw lessons and derive good practices. The evaluation covers the period from June 2014 to August 2017 and was timed so as to allow the country visit to be undertaken while the international and national facilitators were still in country.

3. The main stakeholders and users of the Burkina Faso evaluation are: the UNN/REACH Secretariat; REACH facilitators; the REACH Country Committee, made up of country-based heads of partner agencies the World Food Programme (WFP), the United Nations Children’s Fund (UNICEF), the World Health Organization (WHO), the Food and Agriculture Organization of the United Nations (FAO); UN agencies nutrition focal points, SUN government focal point; the Ministry of Health; the Ministry of Water and Sanitation; the Ministry of Basic Education and Literacy; Secrétariat Exécutif du Conseil National de Sécurité Alimentaire (SE-CNSA, Executive Secretariat of the National Food Security Council); Secrétariat Permanent du Conseil National de Protection Sociale (SP-CNPS, Permanent Secretariat of the National Social Welfare Council); Canadian Cooperation; and SUN Networks (Civil Society, Business, Parliamentarians, Donors).

4. The UNN/REACH Secretariat’s exploratory mission to Burkina Faso was carried out in May 2014. Considering the country’s existing achievements, the mission focused on areas that warrant improvement: analyses like the Infant and Young Child Feeding intervention (IYCF) mapping had been carried out, but were lacking the in-depth analysis needed to provide a global picture of the multi-sectoral approach; nutrition policy was health-oriented and needed revision to account for contributing sectors; several coordination frameworks existed, but none were structured to coordinate the implementation of a multi-sectoral approach; existing monitoring systems did not allow for a multi-sectoral dimension in data collection and evaluation.

5. The evaluation is based on three criteria: effectiveness, efficiency and sustainability. The main evaluation questions, as indicated in the Terms of Reference (TOR), were: 1) what are REACH results in each country (effectiveness, efficiency and equity); 2) what are the explanatory/contributing factors explaining results; and 3) to what extent are the results achieved and the REACH operational models sustainable? To respond to these questions, the evaluation team collected primary qualitative data through semi-

Page 5: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study iv | P a g e

structured interviews with REACH stakeholders and triangulated this information with secondary data retrieved from documents and the REACH monitoring and evaluation (M&E) system. The limitations encountered are not country-specific; the two most significant were indicators lacking sufficient specificity, and difficulty attributing certain results to REACH activities.

Key Findings

6. The conclusions outlined below are presented according to key evaluation questions.

Evaluation Question 1 — Performance at the country level

7. Strong consistency has been observed between the Country Implementation Plan (CIP) and the Annual Work Plans (AWPs); nearly all CIP activities have been applied to AWPs. However, facilitators have readily made necessary modifications (the addition of new activities, rewording, etc.) to better respond to context-specific needs. REACH has focused on analytical tools and policy and planning documents to support reflection on the multi-sectoral approach and consolidate an appropriate framework. REACH has also dedicated much time to facilitation activities, networking, etc. which are not measurable, but which have played a decisive role in obtaining the outcomes presented in the following paragraph.

8. Concerning outcome 1, all planned exploratory analyses were carried out with the personal involvement of facilitators. The preliminary results were presented to the Direction de la Nutrition (DN, Directorate of Nutrition) and focal points for commentary and technical validation. Certain outputs like La Stratégie de Communication et de Plaidoyer Nationale (SCPN, National Advocacy and Communication Strategy) underwent validation at the national level under the aegis of the Conseil National de Concertation en Nutrition (CNCN, National Council for Nutrition Coordination). Furthermore, facilitators ensured the wide distribution of tools through the CNCN and Partenaires Techniques et Financiers/Nutrition (PTF/Nutrition, Technical and Financial Partners/Nutrition). Concerning outcome 2, all planned activities have been carried out or are being carried out: nutrition is included in the 2016 Plan National de Développement Economique et Social (PNDES, National Economic and Social Development Plan) and the United Nations Development Assistance Framework (UNDAF); the Plan National de Nutrition (PNN, National Nutrition Policy) and the Common Results Framework (CRF) have been revised; the Plan Stratégique Multisectoriel de Nutrition (PSMN, Multi-Sectoral Nutrition Plan) has been created. Policy and strategy documents have been validated by the CNCN, but not yet approved by the government. The decision to develop national policies lies with the government and the documents created are the result of efforts on behalf of all stakeholders, but REACH’s contribution (technical, financial, facilitation) was decisive and very much appreciated. Performance for outcome 3 is impacted by the PSMN coordination structure, which is taking a long time to implement, despite appeals from agencies and REACH. However, it is important to recognize the positive national dynamic in favour of nutrition, which is spearheaded by very active coordination mechanisms such as PTF and SUN networks, in particular civil society. Concerning outcome 4, a monitoring system capable of generating and analysing data from a multi-sectoral perspective is not yet in place. However, the UNN functions at the instigation of REACH, which handles the secretariat; monthly meetings with the technical group are consistently held and the common agenda has just been signed. A joint programme on nutrition based on the PSMN has not yet been established, but the

Page 6: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study v | P a g e

common agenda is nevertheless an important tool to strengthen the agencies’ “harmonized efforts”.

9. Efficiency. Budget implementation rates (2014–2016) for activities within the four outcomes vary: 59 percent (Outcome 1), 51 percent (Outcome 2), 17 percent (Outcome 3), 5 percent (Outcome 4). Several reasons can be cited: efficient management of the mapping exercise; merging activities initially planned separately; high-budget activities not yet carried out (capacity gap analysis); partner contributions, etc. The situation could change by the end of 2017; activities with elevated budgets are planned (resource mobilization workshop, capacity gap analysis, etc.).

10. Equity. Four actions were planned in the CIP (integration of gender into policies/sectoral plans; indicators broken down by sex; advocacy for women to be represented in the different coordination mechanisms at all levels; and advocacy for gender-sensitive messages to be disseminated by the different partners/channels). Other than the gender indicator included in the Multi-Sectoral Nutrition Overview and the Situation Analysis Dashboard, no other REACH action regarding gender has been noted.

Evaluation Question 2 — Contributing Factors

11. Factors affecting REACH’s performance and results are: the stability of the DN, the existence of dynamic consultative frameworks, REACH governance based on a “client approach”, and the support of the UNN/REACH Secretariat. Facilitators are integrated into existing structures (DN, PTF/Nutrition, etc.) and have endeavoured to respond to stakeholder expectations, enabling the establishment of a climate of trust favourable to REACH. Facilitators were able to rely on the support of the UNN/REACH Secretariat and felt more at ease in carrying out their functions.

Evaluation Question 3 — Sustainability

12. Tools have been appreciated, but national ownership was more readily apparent in the Policy and Plan Overview, which is a factor of sustainability. However, no tools have been the subject of skills transfer. The sustainability of REACH functions seems compromised; agency focal point TOR are tied to agency mandates and respond more to donor interests. No solution has been found to anchor the national facilitator within the government.

Overall Conclusions

13. REACH performance varies according to outcomes. It is higher on outputs which fall under REACH control (stocktaking exercises) than on those outputs which fall under government leadership (document approval, coordination structure creation, monitoring systems, etc.). Facilitators dedicated much effort to accompanying the government (advocacy, facilitation, etc.), but these actions are non-quantifiable; they are difficult to highlight when presenting results. Stocktaking exercises were carried out at an opportune time and served to develop policies and strategies, and according to reports, they contributed to increased awareness of the nutrition situation among stakeholders. Nutrition is considered a national priority (PNDES 2016-2020) and the PNN and the PSMN have been created and validated. However, the reference framework could be more solid if these documents were approved by the government, which is not yet the case. Despite concerted efforts by stakeholders and REACH, human and institutional capacity, and coordination and monitoring mechanisms remain challenges. The CNCN operates like a coordination mechanism; coordinating the implementation of the multi-sectoral approach is undertaken by the DN, which has a

Page 7: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study vi | P a g e

disadvantage due to its administrative position, and the technical secretariat attached to the Ministry of Health is taking time to establish. Sectoral focal points are in place, but they do not have the capacity, nor the administrative clout required to raise awareness about and coordinate the implementation of nutrition actions within their ministries. Existing monitoring systems are sectoral and do not allow for multi-sectoral data analysis. Progress has been made to implement a multi-stakeholder platform, and SUN Networks have been created or are in the process of being formalized. The UNN is operational and the UN Nutrition Agenda has been signed.

14. Concerning equity, actions formulated in the CIP were not included in the initial CIP action plan, nor in the AWPs, which compromised their implementation from the outset.

15. Tools are appreciated and ownership has been observed for tools like the Policy and Plan Overview, but their sustainability is limited by the absence of skills transfer. The sustainability of REACH functions is not guaranteed, for lack of a consensual solution to anchor the national facilitator.

Recommendations

16. R 1 — Strengthen governance elements in TOR of UNN nutrition focal points

Agency nutrition focal points are often heavily engaged in operational activities, hence the need to mention nutrition governance in their TOR, in line with their agency mandate, in order to avoid neglecting this issue.

Responsibility: REACH Country Committee Deadline: First quarter 2018

17. R 2 — Support countries to better formalize the designation of sectoral focal points

REACH could develop generic TOR to be adapted for each country so that designated sectoral focal points respond to the profile and receive the necessary training/orientation to carry out their responsibilities.

Responsibility: UNN/REACH Secretariat Deadline: First quarter 2018

18. R3 — Establish a website for countries that have benefited from REACH where they could request advice remotely and receive information or get help doing so

REACH is not present long enough in-country for the country to master analytical tools and have a high-performing nutrition management mechanism. The website would allow REACH to continue its support role and strengthen capacities remotely using new technologies.

Responsibility: UNN/REACH Secretariat Deadline: First quarter 2018

19. R4 — Strengthen gender awareness within REACH

To strengthen gender awareness, REACH should ensure that gender actions appear in the initial action plan; include a gender indicator/component in more tools; integrate a gender indicator within the REACH M&E system.

Responsibility: UNN/REACH Secretariat

Page 8: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study vii | P a g e

Deadline: First quarter 2018

20. R5 — Continue to encourage the request for a longer transition period in order to support important planned activities

REACH’s contribution will be decisive for important upcoming activities, in particular through supporting the technical secretariat’s functions; development of a multi-sectoral information platform; development of simplified guides/tools for community actors and assisting with testing them in several communities.

Responsibility: UNN/REACH Secretariat Deadline: before December 2017

Page 9: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 1 | P a g e

1. Introduction

1.1. Evaluation characteristics

Overview of the evaluation subject

1. Renewed Efforts against Child Hunger and undernutrition (REACH) is an inter-agency initiative established by four UN partner agencies—the Food and Agriculture Organization of the United Nations (FAO), the United Nations Children’s Fund (UNICEF), the World Food Programme (WFP) and the World Health Organization (WHO)—in 2008 to strengthen nutrition governance. The International Fund for Agricultural Development later joined as an adviser. Initiating partners signed a Memorandum of Understanding (MOU) in December 2011 and REACH was fully operational by 2012.

2. In supporting government-led nutrition governance efforts, REACH uses a set of analytical tools and resource materials and tailored support (facilitation, coaching, mobilization) to attain the following four outcomes: 1) increased awareness and consensus among stakeholders of the nutrition situation and the best strategies and priorities for improvement; 2) strengthened national policies and programmes that operationalize and address nutrition through a multi-sectoral approach; 3) increased human and institutional capacity on nutrition actions at all levels; 4) increased stakeholder effectiveness and accountability in implementing and supporting nutrition actions. Since 2016, “Joint UN Effectiveness”, one of the outputs under outcome 4, became a separate fifth outcome “Harmonized and coordinated UN efforts” in alignment with the UNN Strategy.

3. In 2011, Global Affairs Canada (GAC) provided funding to REACH in eight “generation 1” countries (Bangladesh, Ghana, Mali, Mozambique, Nepal, Rwanda, Uganda and the United Republic of Tanzania), and in 2014, to four additional “generation 2” countries (Burkina Faso, Haiti, Myanmar and Senegal) and further funding to Mali. The present evaluation concerns these five countries. It is undertaken as per agreement of UNN/REACH Secretariat with GAC.

4. Its main objectives are: 1) accountability—assess and report on the performance and results of REACH; and 2) learning—determine the reasons why certain results occurred or not, draw lessons and derive good practices. The evaluation covers the period from June 2014 to August 2017 and was timed so as to allow the country visit to be undertaken while the international and national facilitators were still in country.

5. Main stakeholders and users of the Burkina Faso evaluation are: the UNN/REACH Secretariat; REACH facilitators; the REACH Country Committee, made up of country-based heads of partner agencies (WFP, UNICEF, WHO, FAO); UN agencies nutrition focal points; the SUN government focal point; the Ministry of Health; the Ministry of Water and Sanitation; the Ministry of Basic Education and Literacy; Secrétariat Exécutif du Conseil National de Sécurité Alimentaire (SE-CNSA, Executive Secretariat of the National Food Security Council); Secrétariat Permanent du Conseil National de Protection Sociale (SP-CNPS, Permanent Secretariat of the National Social Welfare Council); Canadian Cooperation; Réseau de la Société Civile pour la Nutrition (RESONUT, Civil Society SUN Network); Réseau des Acteurs Privés pour la Nutrition (RAPNUT, Private Sector SUN Network); parliamentarians; and donors. The UNN/REACH Secretariat and its United Nations partner agencies will use these evaluation findings to inform REACH’s operational and strategic decision-making. The lessons learned will be used to improve current and future programmes.

Page 10: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 2 | P a g e

Evaluation methodology

6. The evaluation focuses on three criteria: effectiveness, efficiency and sustainability (Terms of Reference: TOR in Annex 1). It addresses three key questions 1) performance at the country level (effectiveness, efficiency and sustainability; 2) contributing/explanatory factors explaining the results; and 3) sustainability. An evaluation matrix (Annex 2) has been prepared; for each evaluation question, it provides sub-questions, measures and indicators, data sources and approaches to data collection.

7. The visit to Burkina Faso, carried out by the regional consultant, took place from June 28 to July 6 (Evaluation Schedule: Annex 3). Primary qualitative data were collected through semi-structured interviews with REACH stakeholders (List of People Interviewed: Annex 4 and Data Collection Tools: Annex 5). These were triangulated with secondary data retrieved from documents (e.g., mission reports) and the REACH monitoring and evaluation (M&E) system (Bibliography: Annex 6). A debriefing with people interviewed and the UNN/REACH Secretariat was organized by teleconference on July 6 to obtain feedback on preliminary findings.

8. No major limitations were observed in the country, and those encountered are not country-specific. They are: 1) some indicators are weak in terms of relevance and specificity—for example, a quantitative indicator is used to measure a qualitative result for outcome 1; 2) difficulty attributing certain findings to REACH, particularly when REACH support takes the form of facilitation rather than service provider, and the fact that benchmarks are not under REACH control; 3) reliability of information due to a lack of flexibility within the baseline/endline template (for instance entering “not applicable” is not proposed as an option): all indicators are filled out even though the initially planned activity or deliverable was not retained or was done without REACH contribution; 4) data availability for outcome 3, which is obtained through nationwide surveys or national information systems that are beyond REACH control; furthermore, REACH timelines are too short to produce changes in coverage or behaviour.

1.2. Country-specific context

9. This sub-chapter gives a brief overview of the socio-political context and the nutrition governance situation prior to REACH activities. Information on governance is based on information retrieved from the CIP, mission reports and interviews. They are presented in the following paragraphs according to the four REACH outcomes.

10. Over the last decade, Burkina Faso has recorded sustained average economic growth of 6 percent; the poverty rate has decreased, but remains high: 46 percent (2009), 40.1 percent in 2014. 1 The food and nutrition situation remains concerning. 2 Food insecurity is present in the Plateau (43 percent), Centre Sud (36 percent) and Centre-Ouest (33 percent) regions, and in the Sahel, where 65 percent of households are at risk of food insecurity. Malnutrition has decreased, but rates remain near critical thresholds as defined by WHO: 35.1 percent (2009) and 31.5 percent (2013) for chronic malnutrition; 11.3 percent (2009) and 8.2 percent (2013) for Global Acute

1 http://databank.worldbank.org 2 WFP, 2014, Burkina Faso — AGVSAN, July 2014

Key Indicators Human Development Index (2014) 181 of 187 Global Hunger Index (2014) 19.9 (alarming) Gender Inequality Index (2015) 146 of 188 Female literacy (2014) 58% Source: PNUD; IFPRI

Page 11: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 3 | P a g e

Malnutrition. Regarding gender, inequalities persist. For example, women represented 24.2 percent of the public and private sector workforce in 2015, and for elected positions, women represented only 18.9 percent of the legislature in 2012-2014.3

11. Outcome 1 — Increased awareness and consensus. Burkina Faso has made a number of achievements: data on the nutrition situation is regularly provided through Standardized Monitoring and Assessment of Relief and Transition (SMART) surveys which have been carried out annually since the Sahel pastoral (2009-2010) and food (2011-2012) crises; a list of 45 Core Nutrition Actions (CNAs) has been established; the study “The Cost of Hunger” is being finalized. However, stocktaking exercises to enrich analysis and better understand the multi-sectoral dimension were useful.

12. Outcome 2 — Strengthened national policies and programmes. Nutrition is clearly included in axis 2 of the United Nations Development Assistance Framework (UNDAF) (2011-2015) and less so in the Stratégie de Croissance Accélérée et de Développement Durable (Strategy for Accelerated Growth and Sustainable Development) (2011-2015). The Plan National de Nutrition (PNN, National Nutrition Policy), included in the Plan Stratégique Multisectoriel de Nutrition (PSMN, Multi-Sectoral Nutrition Plan) (2010-2015), is strongly health-oriented. Because of this, the Africa Nutrition Security Partnership (ANSP) and UNICEF began heavily advocating for the multi-sectoral approach and organized a national workshop in 2014, which laid the foundation for its implementation.

13. Outcome 3 — Increased human and institutional capacity. The country has several institutions that address nutrition, including the Conseil National de Sécurité Alimentaire (CNSA, National Food Security Council) and the Conseil National de Concertation en Nutrition (CNCN, National Council for Nutrition Coordination). The latter was created in 2008 and oversees coordination, but according to people interviewed and the Strengths, Weaknesses, Opportunities, Threats (SWOT) analysis carried out by the ANSP and UNICEF, its capacities are weak and integration is insufficient to manage a multi-sectoral approach. The SUN Civil Society network held its constituent assembly in 2014, and the Partenaires Techniques et Financiers (PTF, Technical and Financial Partners)group must be mentioned, as it is very active and unites all nutrition stakeholders.

14. Outcome 4 — Increased effectiveness and accountability. Several monitoring systems are in place: surveys (SMART, MICS, SAP, food security, etc.), validated by the National Institute of Statistics and Demography; and sectoral information systems (National Health Information System, Ministry of Agriculture and Food Security Harmonized Framework). However, this remains insufficient for decision making with a multi-sectoral approach.

1.3. REACH in Burkina Faso

15. In May 2014, the UNN/REACH Secretariat undertook an exploratory mission in Burkina Faso to examine nutrition governance using data gathered through document review, SWOT analysis and interviews with stakeholders. A second mission was carried out in July 2014 to follow up and make proposals for REACH’s anchorage in the country according to different terms and conditions: administrative hosting by WFP; operational hosting at the level of the Resident Coordinator Office; and government entry points provided by all coordination mechanisms. The international facilitator started in December 2014 and the national facilitator in April 2015. REACH’s

3 http://www.un-page.org/files/public/pndes_2016-2020-4.pdf

Page 12: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 4 | P a g e

engagement in the country terminates end of December 2017; a request for an extension was made.

16. The initial REACH budget was USD 845,833 for 2014-2016. In 2016, GAC approved an extension until December 31, 2017. A total of USD 388,000 was allocated for 2017 (balance as of December 31, 2016 USD 298,334 left over from the initial three-year budget, in addition to remaining funds intended for generation 1 countries).

Figure 1: Planned budget

2014-2016

2017

Source: REACH Budget CIP Burkina Faso Final Excel Sheet and Copy of PA REACH 2017VF

Résultat 110%

Résultat 27%

Résultat 318%

Résultat 47%

Soutien technique

pays (frais des facilitateurs)

42%

Soutien Secrétariat

13%

Soutien administratif PAM3%

Résultat 13%

Résultat 28%

Résultat 322%

Résultat 46%

Soutien technique pays (frais des facilitateurs)

45%

Soutien Secrétariat13%

Soutien administratif PAM3%

Page 13: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 5 | P a g e

2. Evaluation Findings

Overview of completed activities, output and results

17. There is strong alignment between the CIP and the Annual Work Plans (AWPs). Except for activity 1.3.1, “Facilitate the integration of recommendations from ‘The Cost of Hunger’ study, including in the communication plan”, all CIP activities have been reaffirmed (Table: CIP Planned Outcomes, Outputs and Activity Categories versus AWP in Annex 7). However, facilitators have readily made context-relevant modifications, for example: 1) the addition of new unplanned activities (resource mobilization workshop, advocacy workshop for the new government); 2) content changes (facilitation of targeting by intervention replaced by the dialogue workshop for the scale up of the Common Results Framework-CRF); 3) activity break-down (support for CNA integration/implementation in regional development plans broken down into three activities: sharing stocktaking exercises, consultation on the integration process and development of a guide for integrating CNAs into plans).

2.1. Evaluation Question 1 — What are REACH results?

Effectiveness

18. The results are presented according to the four REACH outcomes. For each outcome, findings on outputs are followed by an assessment of progress towards the outcome based on the REACH M&E system together with stakeholder views. Effectiveness is analysed based on progress made toward achieving the fifteen quantifiable expected REACH outputs. However, it must be stressed that facilitators dedicated much time and effort to facilitation, networking, and motivation to create favourable conditions for achieving results. These REACH actions are more or less apparent in the following paragraphs, but specific attention must be drawn to them because they are not easily measured.

Outcome 1

19. Output 1.1 Multi-sector and multi-stakeholder stocktaking. All planned deliverables were achieved. The first analysis for the Multi-Sectoral Nutrition Overview, (including trends and causal analysis), was carried out in 2014 by the UNN/REACH Secretariat using data from the 2010 Demographic and Health Survey. Since 2015, facilitators have made annual updates after each SMART survey publication. The preliminary findings of the analysis are presented to the agency nutrition focal points for feedback, then to the UNN/REACH Secretariat for counter-verification. A condensed version was prepared for the Direction de la Nutrition (DN, Directorate of Nutrition) and for meetings with the CNCN and PTF/Nutrition. Situation Analysis Dashboards have also been created for the 13 regions and illustrate 2009 to 2016 trends. The analysis process was not participatory, but people interviewed found the documents very useful. They provided contextual analysis for policy and programme document development.

20. For stakeholder and nutrition action mapping, facilitators called upon a consultant, a former REACH facilitator who had experience with mapping in Niger, to share experiences. This enabled them to clearly understand the challenges inherent in the exercise and inspired personal engagement. The facilitators met with the DN to present their objectives and the mapping development process, and to request that a committee be established to lead the exercise. The resulting committee included two facilitators and three government representatives (DN, Ministry of Agriculture and Food Security

Page 14: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 6 | P a g e

and the Statistical Division), all of whom were trained on the tool by the UNN/REACH Secretariat. The meticulous preliminary work carried out included: developing a questionnaire and analysis plan; reviewing the CNA list; meeting with agency focal points to gain shared understanding of target definitions for each CNA and consensus on the level of geographic disaggregation; identifying stakeholders based on a list provided by the Office for the Coordination of Humanitarian Affairs and NGOs that work with the agencies; classifying NGOs (local, international, consortium); and interviewing stakeholders who must fill out the questionnaire, to avoid persistent errors in interpretation. An additional precaution was taken by the national facilitator to encode the questionnaires in order to correct repetitions, especially for the consortium NGOs. Following this work, facilitators who did not yet master the analytical component requested help from the UNN/REACH Secretariat to improve their understanding, and those who had not mastered map creation sought the help of a national consultant to better understand the mapping exercise. The preliminary findings were first presented to the different focal points (SUN, sectoral and agencies), then a validation workshop was organized in July 2015. A hard copy version was created to facilitate exchange. Mapping is considered a tool for knowledge and awareness; the people interviewed say they have been enlightened on the diversity of stakeholders that contribute to their sector. According to information collected, the mapping results also contributed to developing the PSMN.

21. Policy and Plan Overview. The UNN/REACH Secretariat provided facilitators with an intern and ensured this person’s technical supervision. Facilitators were involved in this exercise: they gathered documents and supervised the intern through weekly monitoring, in addition to Skype discussions with the UNN/REACH Secretariat, in which they participated. The facilitators then analysed preliminary findings and made the following two changes: introduction of explicative data to a purely descriptive analysis; and enlargement of the analysis to include the planning framework for increased awareness of nutrition in programme planning. In December 2016, the document was presented to the CNCN, which recommended that this tool inform community planning and NGO programmes. To this end, consensus on the process of integrating nutrition into community/regional plans was undertaken and the development of a guide for integrating nutrition into community/regional plans is planned.

22. Output 1.2 Consensus on CNAs. In 2014, a list of 45 CNAs was established, with the support of ANSP/UNICEF. During the mapping exercise in 2015, REACH facilitated a review of this list, deemed ambitious, using a participatory process that brought together stakeholders from government, NGOs and the UN. The list was reduced to 27 CNAs, selected on the criteria of proven nutritional impact and feasibility. In addition, some wording was corrected.

23. Output 1.3 Cost-Benefit Analysis. The study “The Cost of Hunger in Burkina Faso” was undertaken by WFP in 2014 and launched in 2015. The document’s proposed recommendations (establish a multi-sectoral policy; strengthen stakeholder coordination, etc.), correspond to REACH objectives. Thus, the activities initially planned in the CIP to “Facilitate the integration of the ‘The Cost of Hunger’ study recommendations into the communication plan” became obsolete. However, facilitators supported the study’s dissemination. The document was used in advocacy activities, notably the meeting with parliamentarians.

Page 15: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 7 | P a g e

24. Output 1.4 National advocacy and communication. The Stratégie de Communication et de Plaidoyer Nationale (SCPN, National Advocacy and Communication Strategy) was developed in 2016, with the support of a consultant, who used the following process: meet with sectors to collect data; develop a first version submitted to the DN for review by an ad hoc committee; carry out a second consultation with sectors to identify areas of collaboration between sectors and establish annual communication plans; develop a second version for committee review (they decided to shorten the document, which was deemed too long, and strengthened the connection with the PSMN). Through this consultative process, the level of appropriation among people interviewed was high. The document was validated during the national workshop in April 2017.

25. Progress achieved on outcome 1 — Increased awareness and consensus. The monitoring data, confirmed through interviews, indicates that the target number of stakeholders involved in nutrition has been attained as planned, and that there is an upward trend among donors and UN agencies. Qualitatively speaking, facts attesting to an increased awareness were observed. For example, sectors recognize that they had previously been leading nutrition sensitive actions without realizing it; now, they have a better understanding of the connection between these sensitive interventions and nutrition. The new Canadian cooperation team says they have included nutrition in their reproductive health project following advocacy by a RESONUT stakeholder.

Table 1: Progress in outcome 1 Stakeholder

group Baseline Endline Trend Target Comment as per REACH endline data

analysis

NGOs 100% 100%

→ 80% Baseline: the largest NGOs (HKI, ACF, CRS, GRET

and Save the Children) support nutrition actions. Endline: CRS replaced by MDM.

Donors

60% 100%

80% Baseline: the largest donors World Bank, ECHO, Canada, USAID, Japan. The first three support nutrition. Endline: all five donors support nutrition.

Government Ministries

100% 100%

80% Baseline: Agriculture, food security, water and sanitation; Health; Education; Ministry of Social Action and National Solidarity; Commerce. Endline: The first ministry has been divided in two, Commerce is no longer on the list.

UN Agencies 80% 100%

↗ 80% Baseline: UNICEF, FAO, WHO and WFP active

and UNFPA not active on nutrition. Endline: all five agencies support nutrition.

Source: REACH ME and Endline Burkina Faso

Outcome 2

26. Output 2.1 Integration of nutrition in government strategies and UNDAF. The UNDAF (2018-2020) is currently being developed, and facilitators have seized every opportunity to advocate for nutrition. For example, they participated in training on planning and formulating the UNDAF, and attended the strategic retreat to analyse context and determine priorities. In 2016, the government developed its Plan National de Développement Economique et Social (PNDES, National Economic and Social Development Plan), which became the national reference document. The integration of nutrition into this document was a team effort undertaken by the DN, RESONUT, PTF and REACH. They drafted proposals and led many lobbying activities with the writing team. Nutrition is positioned in the PNDES and has been given an impact indicator (chronic malnutrition) and a coverage indicator (community management of acute malnutrition). Also of note is the budget line item regarding nutrition inputs that was included in the budget thanks to the support of a team from the Ministry of Economy and Finance and the power of dialogue of the European Union, which approved the

Page 16: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 8 | P a g e

addition of this item to its budgetary support. The integration of nutrition into sectoral policies is a long-term process and will happen when revisions are carried out. But an example was observed in 2017: REACH participated in revising the Politique Nationale de Sécurité Alimentaire et Nutritionnelle (PNSAN, National Food and Nutrition Security Policy) to ensure that a connection between the PNN and the PNSAN be established.

27. Outcome 2.2 Review/update of multi-sector national nutrition policy/strategy/action plan. The review of the PNN and the CRF, and the development of the PSMN were carried out under the DN’s leadership. The DN established a PNN working group to review the 2007 version of the policy and strengthen the multi-sectoral component. REACH participated in the working group and ensured that documentation was made available (policy/strategy of other countries, stocktaking, etc.). For the review of the initial CRF in 2014, several mini-workshops were organized to better account for key sectors. The CRF is a consensual document that was validated in 2016 by all stakeholders. The PSMN was developed by a consultant who had worked with the technical group, which was established by the DN. Several sessions were held with REACH facilitators and members of the technical group who supported preparations for work sessions; shared the stocktaking exercises to guide document development; shared best practices; and ensured consideration of international and national recommendations. The technical component of the PSMN was validated in 2017 by all sectors, under the leadership of the General Secretary of the Ministry of Health. Costing was undertaken by the World Bank and UNICEF, and is being finalized. The two documents, the PNN and PSMN, have not yet been approved by the government.

28. Output 2.3 CNA uptake in sectoral annual work plans. The 2017 annual work plan for the DN is based on the CRF, but the other sectors have not begun to proceed in the same way.

29. Output 2.4 Sub-national CNA Uptake. REACH presented an analysis of the 2016 Plans communaux de développement (PCD, Community Development Plans) during a meeting of the CNCN, indicating that the PCD mentioned nutrition objectives, but not nutrition activities. Subsequently, in 2017, the DN, supported by REACH, organized a national workshop to reflect on integrating nutrition into the PCD. The following recommendations were made: increase awareness among local elected officials; make simplified booklets available; identify and involve local partners in the planning process; organize trainings; mobilize community resources.

30. Progress achieved on outcome 2 — Strengthened national policies and programmes. According to the indicators, outcome 2 has been achieved in Burkina Faso: the PNN and PSMN have been developed and validated. The decision to develop these two documents rests with the government, and all stakeholders have contributed. However, the people interviewed recognize that REACH played a decisive role, providing technical and financial support, and facilitating the process by motivating stakeholders to respect timeframes.

Outcome 3

31. Output 3.1 Management of implementation capacity by the coordination mechanisms. REACH supported all activities related to negotiation, consensus-seeking, etc. that were undertaken to create a multi-sectoral coordination structure integrated at the highest level, but no consensus was reached. The alternative solution is to create a technical secretariat associated with the Ministry of Health; this solution

Page 17: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 9 | P a g e

has faced delays. REACH’s support for the establishment of the SUN Network was more successful. Several knowledge and awareness actions were undertaken, such as mini-workshops with academics supported by WHO and the advocacy workshop targeting parliamentarians. Certain networks are in place: Civil Society (2014), Parliamentarian (2016) and United Nations (2016), and they are operational. The Business Network (2016) is being formalized, and the Donor and Academic networks are being created. The following are examples of their activities: the Civil Society Network has carried out various studies (accountability, nutrition funding, etc.) and contributed to launching the Parliamentarian Network; the Parliamentarian Network supports advocacy for the integration of nutrition into the budget.

32. Output 3.2 Human capital allocated and institutions in place for nutrition scale-up. The sectoral ministry focal points have been designated and involved in various activities (mapping exercises, developing policy documents, etc.). However, they unanimously recognize difficulties encountered within their ministries in establishing dialogue and creating an environment favourable to nutrition.

33. Output 3.3 Governance, management and nutrition-related training strengthened at all levels. The planned activities to strengthen capacity have not begun either for the coordination framework or at the community level, and the training manuals on the multi-sectoral approach have not yet been produced.

34. Output 3.4 Knowledge-sharing network. REACH has been very active in activities related to “knowledge sharing/studies/research” and “documentation and sharing of best practices with stakeholders”. These are REACH’s ongoing actions. The latest example is a published article on the added-value of REACH in SUN implementation and the interview with the SUN Focal Point, which appeared on the new Media Hub site of the Emergency Nutrition Network. Other activities include: facilitators participating in annual REACH and SUN meetings; supporting the SUN joint self-assessment workshop; and facilitating an inter-country exchange between parliamentarian networks in Chad and Burkina Faso. In particular, the Burkina Faso parliamentarians learned about the role their colleagues in Chad played in scaling up nutrition interventions, and organizing the parliamentarian workshop and the National Nutrition Forum in 2015.

35. Progress achieved on outcome 3 — Increased human and institutional capacity for multi-sectoral nutrition governance at all levels. According to the evaluation matrix, outcome 3 has been achieved. However, it must be noted that coordination capacities are still very weak. Everyone acknowledges that the CNCN in its current form is not capable of coordinating the implementation of the PSMN. As previously mentioned regarding the technical secretariat, REACH and the agencies have invested much effort in advocating for its creation, but this decision rests with the Ministry of Health.

Outcome 4

36. Output 4.1 Multi-sectoral M&E system and processes in place. Planned activities (PSMN M&E framework, integration of indicators into sectoral M&E systems, strengthening the capacity of the coordination body) could not be carried out. REACH efforts focused on developing the PSMN, which was validated in 2017. Discussions about the PSMN monitoring system have begun, and changes will come following the EU’s recent proposal to implement a National Information Platform for Nutrition.

Page 18: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 10 | P a g e

37. Output 4.2 Results disseminated to relevant stakeholders. Dashboards have been created and are updated periodically at the national level and for the 13 regions (see output 1.1). However, the performance review of the implementation of the multi-sectoral approach has not been carried out; only the Ministry of Health had developed an AWP in 2017 related to the CRF.

38. Output 4.3 Nutrition as a key area for "UN delivering as One" established. The UN Network for SUN was formalized in 2016. To render it more operational, REACH undertook a series of activities: organizing the first meeting with agencies to review objectives, composition and the role of the Chair; supporting the development of a work plan and organizing monthly monitoring meetings; carrying out the inventory of UN nutrition actions for agencies in 2016; interviewing agency nutrition focal points on nutrition governance; holding a strategic workshop with the support of the UNN/REACH Secretariat to achieve consensus and propose a common agenda; meeting with the UN Country Team to verify the links between the agenda and nutrition objectives defined in government positioning documents. The final version has been validated.

39. The network does not have a joint nutrition programme inspired by the PSMN. However, two MOUs between several UN agencies have been implemented: Accelerating Nutrition Improvements 2013-2016 (WHO, UNICEF, WFP) in three regions to strengthen nutrition surveillance; and Child Friendly Quality School (2014 to 2017) (UNICEF, WFP and FAO), in the Sahel region. Signing the common agenda was an important step towards harmonizing the efforts of partner agencies.

40. Progress achieved on outcome 4 — Mechanisms to track impact, implementation and funding established. Different impact monitoring systems are in place (SMART, SAP, etc.) and sectoral ministries have their own information system (health information system/Ministry of Health Database, Food Security Information System, etc.). However, a system with a multi-sectoral approach and a system to track PSMN funding has yet to be developed.

Efficiency

41. Planned/mobilized resources compared to utilized resources. Overall, budgetary implementation rates (2014 to 2016) for the four outcomes are low (Figure 2: CIP planned budget versus implemented budget 2014-2016). Several reasons can be cited. First, efficient management of the mapping exercise reduced respective expenses by 57 percent. Facilitators were personally involved in the exercise, which reduced the number of paid consultant days. Secondly, some planning was overly optimistic: activity 3.1—capacity gap analysis—depends on the coordination structure, which is difficult to implement, and activity 4.1—M&E system—depends on finalizing the PSMN, which is a long process; these outputs have the highest budget allocations (USD 70,000 and USD 40,000) and, for lack of time, were not carried out. Thirdly, budgets were overestimated for activities like integrating nutrition into national reference documents and the UNDAF. Fourthly, contributions from other actors also contributed to economizing resources: WHO contributed to CRF mini-workshops and the establishment of business and academic networks; UNICEF provides constant financial and technical support, and specifically contributed to the PSMN budgeting exercise; FAO brought in a consultant for the mapping exercise; as host agency, WFP provides continuous support in numerous forms.

Page 19: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 11 | P a g e

42. The budget implementation situation has changed little as of June 2017. The expected rate of budget implementation in 2017 is 18 percent. The situation could change by the end of 2017, as activities with elevated budgets are planned (resource mobilization workshop, capacity gap analysis, etc.).

43. Regarding compliance of expenditures with approved budget plans and timeliness of funds requisition and release, no difficulties were observed.

Figure 2: Planned budget versus implemented resources 2014-2016

Source: REACH Budget CIP Burkina Faso Final Excel Sheet and REACH Burkina Faso - Expenditures tracking_270317

Equity

44. The first planned action related to gender in the CIP consists of “supporting the integration of gender equality and women’s empowerment in the different policy documents and strategies and in planning, implementation and monitoring and evaluation of the different sectors engaged in nutrition”. The data collected does not provide proof of any action led by REACH on this issue. The Policy and Plan Overview could have served to understand the degree of women’s empowerment in policies, and to identify opportunities for support, but this analysis was not carried out.

45. The second planned action regarding gender pertains to “gathering indicators broken down by sex and data analysed with a gender perspective”. The Situation Analysis Dashboard has a gender indicator, but in the M&E table, coverage indicator 3.A has not been broken down by sex, though data is available in the Department of Health and Sanitation and SMART surveys.

46. The last two planned actions in the CIP are as follows: “strengthen the capacities of women’s organizations and advocate for women to be represented in the different

59% 51%

17%

5%

87%

100%

92%

65%

0%

20%

40%

60%

80%

100%

120%

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

900,000

Résultat 1 Résultat 2 Résultat 3 Résultat 4 Soutientechnique

pays (frais desfacilitateurs)

SoutienSecrétariat

Soutienadministratif

PAM

Total

Total planifié Total dépensé Taux dépenses

Page 20: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 12 | P a g e

coordination mechanisms at all levels” and “ensure that messages disseminated by the different partners/channels at all levels are gender-sensitive”. For these issues, no tangible REACH action has been identified; furthermore, these activities seem premature.

Key Findings – Question 1 Performance

Effectiveness

Despite the relevance of activities initially included in the CIP, almost all of which are included in the AWPs, there is still a need to make changes to better adapt to the context.

All stocktaking analyses were carried out and facilitators were personally involved in certain analyses. This explains savings made on stakeholder and nutrition action mapping, and innovations made to the Policy and Plan Overview, which has become a more analytical tool.

Tools underwent technical validation (DN, focal points), followed, where necessary, by national

validation under the supervision of the CNCN (PSMN, SCPN).

There is a national dynamic in favour of nutrition, illustrated by the combined efforts of stakeholders to integrate nutrition into the PNDES and the creation of a budget line item for nutrition inputs in the Ministry of Health budget.

Through REACH, all SUN Networks have been established, but they are not at the same level. Some (Civil Society, UN, Parliamentarian) are functional, while others are being formalized (Business) and others are being created (Academic, Donor). Multi-stakeholder exchanges have already been observed: Civil Society contributed to advocacy for the establishment of the Parliamentarian network. The UN Network Agenda, an import step towards harmonizing the efforts of partner agencies, has been validated.

The establishment of a structure with the human and institutional capacity required to coordinate the implementation of the PSMN remains the biggest challenge and exceeds REACH’s expertise.

The DN undertook discussions with partners about developing the PSMN M&E system. The EU’s proposal offers possibilities for an even more efficient system.

Efficiency

Low implementation rate (2014-2016) for the four outcomes and for diverse reasons (budget savings, cost sharing with other partners, planning of activities difficult to complete under agreed-upon timelines, etc.). The situation remains unchanged in 2017; rates of budgetary implementation are still weak, but they could evolve by year’s end.

Equity and gender

The actions foreseen in the CIP were not planned in the AWP, which in part explains why they were not implemented.

2.2. Evaluation Question 2 — What are the explanatory/contributing factors explaining results?

Exogenous factors (political stability, policy environment, human resources in government entities, natural hazards)

47. Changes in government (two Health Ministers) delayed the signing of an order for the creation of a technical secretariat to oversee coordination of the PSMN. But overall, REACH benefited from a favourable national context: the clear stability of the DN and the open-mindedness of management created a favourable working environment; the engagement of focal points in the work of various committees for policy review, development of a strategic plan, the communication and advocacy strategy, etc. were also observed. In addition to these human resources, the country has a database

Page 21: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 13 | P a g e

(SMART and Harmonized framework of the Ministry of Agriculture) that facilitated stocktaking exercises.

REACH governance, facilitators’ hosting arrangements and funding

48. Concerning REACH governance, the following positive factors were observed:

i. planned outputs adequately aligned with national priorities. Activities planned by REACH are discussed with the DN to ensure a connection with the needs of the government. When a relevant government need is not planned in the CIP, facilitators negotiate with the UNN/REACH Secretariat for funding, or propose to present the request to other agencies. Thus, REACH was able to establish a much-appreciated trust-based relationship with the DN;

ii. substantial involvement of facilitators in implementing activities. This enabled quality tools, like mapping, to be produced and at a lower cost;

iii. facilitators’ interpersonal skills. Interviews revealed that facilitators succeeded in uniting stakeholders around nutrition and motivating them to adhere to agendas;

iv. support from the UNN/REACH Secretariat. Facilitators benefited from the support of the Secretariat (UNN/REACH Secretariat Support: Annex 8); they particularly appreciated the promptness with which the Secretariat responded to their questions and were reassured to know they had a respondent to guide them in case of difficulty.

REACH partners’ commitment

49. At first, REACH was misunderstood. It was viewed as a new structure for coordination or leadership, and some partners were not very cooperative. It took some time for them to better understand and agree to collaborate. Later, after trust had been established, REACH took advantage of existing dynamic platforms (PTF/Nutrition; Agriculture; Food Security). Monthly meetings with PTF/Nutrition enabled REACH to quickly gain visibility. Furthermore, partners had already begun to discuss the multi-sectoral approach; in 2014 and 2015, ANSP/UNICEF organized workshops on nutrition dialogue and multi-sectoral coordination.

Key findings — Question 2 Factors affecting performance

Factors with negative impact on activity implementation or effectiveness

Changes in government

Initial misunderstanding of REACH mission

Enabling Factors

Alignment of REACH activities with national priorities

Favourable working climate due to the DN’s stability and management’s open-mindedness, and the engagements of sectoral focal points

Existence of dynamic platforms for exchanges (PTF/Nutrition; Agriculture; Food Security)

Discussion about the multi-sectoral approach underway

Support to REACH by host agency WFP

Facilitators’ interpersonal skills

UNN/REACH Secretariat support

Page 22: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 14 | P a g e

2.3. Evaluation Question 3 — To what extent are the results achieved and the REACH operational models sustainable?

Regarding achieved outcomes and REACH operational models

50. In accordance with REACH guidelines, tools were validated. First, preliminary findings were systematically presented to the PTF/Nutrition and the DN, followed by different processes to ensure validation: sectoral mini-workshops for the Policy and Plan Overview; validation workshops expanded to other sectors and decentralized stakeholders for the PSMN, stakeholder and nutrition action mapping, and the SCPN.4

51. A transition plan (2017) was developed, but it was established according to REACH actions that remain to be finalized and not according to the question of sustainability. For example, the document makes no mention of skills transfer with regard to stocktaking exercises such as “support sectoral focal points to update analytical tools”.

52. Continuation of the national facilitator’s functions within the government is hypothetical. The facilitator’s anchorage was the subject of several discussions within agencies; three options were proposed: position the facilitator in the Ministry of Health (insufficiently unifying to some); at the Prime Minister level (a good strategic position but no technical counterpart); or in the CNSA, which would become the Conseil National de Sécurité Alimentaire et Nutritionnelle (CNSAN, National Food Security and Nutrition Council) (certain agencies would not relate to this structure). Due to a lack of consensus, agency advocacy for facilitator anchorage within the government was not undertaken.

53. Government uptake of REACH tools is apparent for the SCPN and the Policy and Plan Overview. The sectoral focal points were involved in developing the communication strategy and they say they relate to the document. For the Policy and Plan Overview, the CNCN gave instructions for the tool to inform community planning and NGO programmes. However, the skills transfer exercises were not carried out. People interviewed said they would not be able to repeat the exercises, which could limit sustainability.

54. Concerning agencies, agency focal point TOR are tied to agency mandates and respond more to donor interests, including FAO and WHO, who recruit focal points for specific projects.

REACH’s contribution to increased national ownership and its leadership role in multi-sectoral nutrition governance and coordination

55. National ownership of nutrition governance begins with integrating nutrition into the national reference document and the existence of a national nutrition policy. In Burkina Faso, nutrition has been integrated into the PNDES and the PNN has been developed; REACH provided technical and financial support. Nonetheless, the PNN has not yet been approved by the government. The desired level of ownership has not yet been reached.

56. The sustainability of nutrition governance also depends on the existence of a coordinating mechanism capable of implementing the PSMN. This has not yet been achieved and remains one of the largest challenges to overcome.

4 REACH-Facilitators-Manual_FR_MPL_Final_24102013.doc

Page 23: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 15 | P a g e

Key Findings – Question 3 Sustainability

Validation and dissemination of tools observed, but ownership more visible for the

SCPN and the Policy and Plan Overview.

Continuation of the national facilitator’s functions within the government hypothetical, due to lack of agreement; agency advocacy for facilitator integration within the government not carried out.

Establishment of a PSMN coordination mechanism remains one of the largest challenges to overcome.

3. Conclusions and Recommendations

3.1. Conclusions

57. Performance. REACH activities in Burkina Faso were dominated by the production of analytical tools intended to support reflection on the multi-sectoral approach already underway in the country, and the development of policy and programme documents to consolidate the reference framework for this multi-sectoral approach. Overall, REACH performance is higher on outputs for which it was the service provider and that fall under its control (stocktaking) than on those outputs which fall under government leadership (creation of coordination structures, monitoring system, etc.), where REACH actions were characterized more by facilitation and mobilization. It must also be noted that REACH performance was analysed using expected REACH outputs; it is difficult to highlight facilitation, mobilization and other activities that were decisive in REACH’s approach and take up a large part of facilitators’ time, but which are not easily measured.

58. Concerning outcome 1, stocktaking analyses were carried out at an opportune time to contribute to the development of policy and strategy documents (PNN, PSMN, PNDES, etc.). They constitute advocacy tools that contributed to raising awareness among stakeholders. An example of this is the dynamic in favour of nutrition observed among stakeholders who were involved in developing policy and strategy documents, and during the integration of the nutrition budget.

59. Concerning outcome 2, REACH worked to see nutrition integrated into national priorities (PNDES 2016-2020), and the PNN and PSMN produced. These documents have been validated, but have not yet been approved by the government.

60. Outcome 3 presents the weakest level of performance: the CNCN continues to oversee nutrition coordination, but it operates more like a consultative and technical validation framework than as a management body implementing the PSMN. The decision to create a technical secretariat to replace it is slow to be endorsed. The sectoral focal points are in place, but they do not have the necessary capacity or administrative clout to coordinate multi-sectoral interventions within their ministries. Integrating nutrition into sectoral policies and plans is one thing; leading the Ministry to be accountable for the results of the CRF to which it has committed is quite another. Sectoral focal points have significant responsibilities: they must advocate within their ministries to raise awareness among their colleagues; be able to interact with all services and departments involved in nutrition sensitive

Page 24: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 16 | P a g e

interventions; and ensure coordination within the ministry. They have not been prepared for this new task.

61. Concerning outcome 4, existing information systems are sectoral. The EU’s proposal is a great opportunity to implement a system that integrates the multi-sectoral dimension. REACH played its role as secretariat of the UN Network; the technical group holds regular monthly meetings; a joint programme specific to nutrition has not been carried out, but the UN agencies’ common agenda has been signed.

62. Equity. Activities relevant to gender do not appear in AWPs, which compromised their implementation from the outset.

63. Sustainability. The positive appreciation for the stocktaking exercises is the first factor in establishing sustainability. The people interviewed confirm they have a better understanding of the nutrition situation and the challenges posed, thanks to REACH tools. However, skills transfer is lacking. Regarding the sustainability of REACH functions, the problem of anchoring the national facilitator within the government has not been resolved. Similarly, the UN focal points TOR are defined according to agency mandates and donor interests, and do not specifically mention institutional nutrition governance.

3.2. Lessons learned and good practices

64. The personal involvement of facilitators in the mapping exercise was a good initiative. They committed to ensuring meticulous preliminary work, monitoring the data collection process, and ensuring quality data entry. Controlling these steps was crucial to avoiding problems interpreting data collected during the mapping exercise.

65. Attentive facilitators who seek to respond to new implementation needs can play an important role in enriching REACH tools. The Policy Overview that became the Policy and Plan Overview has become a guiding document for planning exercises.

66. A dynamic multi-stakeholder platform enables efforts to be shared among stakeholders. For example, during the integration of nutrition into the PNDES, RESONUT supported advocacy activities, the UN Network supported proposals for insertion; donors used their negotiating power for integration of a nutrition budget line item; and the UN Network and Donors will support the organization of the partners’ conference, which is currently being prepared.

3.3. Recommendations

67. Recommendation 1 — Strengthen governance elements in TOR of UNN nutrition focal points

Agency nutrition focal points are often heavily engaged in operational activities, hence the need to mention nutrition governance in their TOR, in line with their agency mandate, in order to avoid neglecting this issue.

Responsibility: REACH Country Committee Deadline: First quarter 2018

68. Recommendation 2 — Support countries to better formalize designation of sectoral focal points

Sectoral focal points are key stakeholders in the multi-sectoral approach. They are often designated without having a good understanding of their roles and responsibilities. REACH could develop generic TOR to be adapted for each country so that designated

Page 25: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 17 | P a g e

sectoral focal points respond to the profile and receive the necessary training/orientation to carry out their responsibilities.

Responsibility: UNN/REACH Secretariat Deadline: First quarter 2018

69. Recommendation 3 — Establish a website for countries that have benefited from REACH where they could request advice remotely and receive information or get help doing so

To establish nutrition governance, countries must: i) regularly conduct stocktaking exercises like those developed by REACH, in order to maintain stakeholder awareness on nutrition and gather relevant information for use in decision-making; ii) have up-to-date policy and strategy documents; iii) have human and technical capacities, and effective M&E mechanisms. REACH supports country efforts to establish the foundation for governance, but REACH is not present long enough in-country for the country to master analytical tools and have a high-performing nutrition management mechanism. The website would allow REACH to continue its support role and strengthen capacities remotely using new technologies.

Responsibility: UNN/REACH Secretariat Deadline: First quarter 2018

70. Recommendation 4 — Strengthen gender awareness within REACH

To strengthen gender awareness, REACH should: i) ensure that gender actions cited in the CIP text appear in the initial CIP action plan and AWPs; 2) include a gender indicator/component in more tools, which has already been undertaken for the Multi-Sectoral Nutrition Overview and the Situation Analysis Dashboard, but remains important to add to the Policy and Plan Overview; integrate a gender indicator into the REACH M&E system.

Responsibility: UNN/REACH Secretariat Deadline: First quarter 2018

71. Recommendation 5 — Continue to support the request for a longer transition period in order to support important planned activities

REACH’s contribution will be decisive for important upcoming activities in Burkina Faso: i) the technical secretariat will be created and the human resources needed to make it operational must be put into place relatively quickly; ii) a large project is on the horizon with the establishment of the multi-sectoral information platform, which will require the contribution of all stakeholders (sectoral ministries, agencies, PTF), and a diversity of actors (nutrition focal points, sectoral M&E staff, decentralized stakeholders, etc.). The added-value of REACH lies in the organization’s capacity for facilitation and uniting stakeholders; iii) the government requested the support of REACH to produce simplified guides/tools for community-level stakeholders and assist with testing them in several communities, and identify difficulties and lessons learned. These activities are important, considering the role that communities can play in scaling up nutrition interventions.

Responsibility: UNN/REACH Secretariat Deadline: before December 2017

Page 26: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 18 | P a g e

Annexes

Annex 1: Terms of Reference

THEMATIC EVALUATION:

End of Term Evaluation of

Renewed Efforts against Child Hunger and undernutrition (REACH) in

Burkina Faso, Haiti, Mali, Myanmar and Senegal from 2014-2017

UN Network for SUN (UNN)/REACH Secretariat

Table of Contents

1. Introduction ..................................................................................... 20

2. Reasons for the Evaluation ................................................................21

2.1. Rationale ............................................................................................................................ 21

2.2. Objectives ........................................................................................................................... 21

2.3. Stakeholders and Users .................................................................................................... 22

3. Context and subject of the Evaluation ............................................... 24

3.1. Context ............................................................................................................................... 24

3.2. Subject of the evaluation .................................................................................................. 25

4. Evaluation Approach ........................................................................ 27

4.1. Scope ................................................................................................................................... 27

4.2. Evaluation Criteria and Questions ................................................................................. 27

4.3. Data Availability ................................................................................................................ 28

4.4. Methodology ...................................................................................................................... 29

4.5. Quality Assurance and Quality Assessment .................................................................. 30

5. Phases and Deliverables .................................................................... 32

6. Organization of the Evaluation .......................................................... 33

6.1. Evaluation Conduct .......................................................................................................... 33

6.2. Team composition and competencies ............................................................................ 33

6.3. Security Considerations ................................................................................................... 34

7. Roles and Responsibilities of Stakeholders ....................................... 34

8. Communication and budget .............................................................. 36

8.1. Communication ................................................................................................................. 36

8.2. Budget ................................................................................................................................. 36

Annex 1 REACH Theory of Change ........................................................ 37

Annex 2 Conclusions and Recommendations of the Joint Evaluation of REACH 2011-2015 ................................................................................. 38

Annex 3 REACH deliverables and tools ................................................. 41

Annex 4 REACH Log frame ................................................................... 42

Page 27: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 19 | P a g e

Annex 5 Overview of REACH Resources and Country Budgets for Burkina Faso, Haiti, Mali, Myanmar and Senegal ............................................... 43

Annex 6 Membership of the Evaluation Committee and of the Evaluation Reference Group .................................................................................. 44

Annex 7 Evaluation Schedule ................................................................ 45

Annex 8 Acronyms ................................................................................ 46

Page 28: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 20 | P a g e

1. Introduction

1. These Terms of Reference (TOR) are for a thematic evaluation of REACH in Burkina

Faso, Haiti, Mali, Myanmar and Senegal. This is an end of term evaluation

commissioned by the UN Network for SUN (UNN)/REACH Secretariat and will

cover the period from 2014-2017.

2. These TOR were prepared by the Evaluation Manager (EM), Tania Goossens, in

consultation with the UNN/REACH Secretariat, following a standard template. The

purpose of the TOR is twofold. Firstly, it provides key information to the evaluation

team and helps guide them throughout the evaluation process; and secondly, it

provides key information to stakeholders about the proposed evaluation.

3. REACH - Renewed Efforts Against Child Hunger and Undernutrition – is an inter-

agency initiative that was established by the four initiating UN partner agencies:

Food and Agriculture Organization (FAO), United Nations Children's Fund

(UNICEF), World Food Programme (WFP) and World Health Organization (WHO)

in 2008 in an effort to strengthen the fight against poverty and undernutrition. It

was later joined by the International Fund for Agricultural Development (IFAD) as

an adviser. REACH takes place in the context of the Scaling Up Nutrition (SUN)

Movement which was established in 2010. SUN is currently active in 59 countries,

galvanizing the support of multiple stakeholder Networks, including the UN

Network for SUN (UNN), to reduce malnutrition. REACH is a country-centred,

multi-sectoral approach to help strengthen national capacity for nutrition

governance, which also includes support to all SUN Networks and other partner

organisations to ensure effective engagement in multi-stakeholder processes and

platforms. REACH is based on a theory of change5 which envisages that the

nutrition of children under 5 and women can be enhanced if country-level nutrition

governance is improved6. It also assumes that improved nutrition governance

requires progress towards increased awareness and stakeholder consensus,

strengthened national policies and programmes, increased human and institutional

capacity, and increased effectiveness and accountability. After three pilot countries

started in 2008, the REACH Memorandum of Understanding (MOU) was signed by

the initiating partners in December 2011 and REACH was fully operational by 2012.

In March 2015, the initiating partners agreed to extend REACH through a re-

validated MOU with WFP remaining as designated host agency. It was also

confirmed that REACH serve as the secretariat for the UN Network for SUN (UNN),

previously co-facilitated with the UN Standing Committee for Nutrition.

5 Please see annex 1 for the full theory of change. 6 Mokoro 2015. Strategic Evaluation: Joint Evaluation of Renewed Efforts against Child Hunger and under-nutrition

(REACH) 2011-2015: Volume I Evaluation Report. Oxford: Mokoro Ltd, October 2015.

Page 29: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 21 | P a g e

2. Reasons for the Evaluation

4. The reasons for the evaluation being commissioned are presented below.

2.1. Rationale

5. Monitoring and evaluation is a high priority for REACH in order to build

understanding of its effect on improving nutrition governance and ultimately

nutrition outcomes in participating countries; for knowledge sharing and learning

across REACH countries and with other stakeholders. Since nutrition governance

must be tailored to each unique situation and is led by government, lesson learning

and knowledge sharing are strongly linked to REACH’s goal achievement and has,

therefore, been a high priority. The evaluation aims to address aspects that cannot

be understood through routine monitoring, in particular the extent to which

REACH’s outcomes have been achieved, factors affecting REACH outcome

achievement and a comparison of country experiences in REACH implementation.

6. An independent external evaluation7 (IEC) of REACH, covering the period 2011 to

2015, was conducted in eight generation 1 countries that were funded by the

Canadian government8. Serving the dual purpose of accountability and learning, it

assessed REACH's relevance and appropriateness, performance, the factors

explaining results, and sustainability. A summary of the findings can be found in

Annex 2. In 2014, Global Affairs Canada (GAC) funded four additional REACH

generation 2 countries (Burkina Faso, Myanmar, Haiti and Senegal) and provided

additional funding to Mali. The generation 2 countries were not part of the IEC

given the short implementation time at the time of the evaluation. However, as per

the donor agreement, each country is expected to have an external evaluation linked

to their Country Implementation Plans (CIP). As funding for these countries will

terminate at the end of 2017, this end-term evaluation will focus on these four

countries and Mali. The evaluation is timed so as to allow country visits to be

undertaken while all facilitators are still in country.

7. The findings and recommendations of the evaluation will inform the UNN/REACH

Secretariat and participating countries of progress and effects and enable them to

understand how their own experiences compare to those of other countries. This is

important information to improve current and future programmes. The findings of

this evaluation will likewise provide evidence on which the Canadian government,

and other donors can make a decision about future funding.

2.2. Objectives

8. The evaluation will address the dual and mutually reinforcing objectives of

accountability and learning.

7 Mokoro 2015. Strategic Evaluation: Joint Evaluation of Renewed Efforts against Child Hunger and under-nutrition

(REACH) 2011-2015: Volume I Evaluation Report. Oxford: Mokoro Ltd, October 2015. 8 Bangladesh, Ghana, Mali, Mozambique, Nepal, Rwanda, Uganda and the United Republic of Tanzania

Page 30: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 22 | P a g e

Accountability – The evaluation will assess and report on the performance and

results of REACH in 5 GAC-funded countries. A management response to the

evaluation recommendations will be prepared by the UNN/REACH Secretariat to

document the level of agreement with the recommendations and the steps to be

taken to address the recommendations; and

Learning – The evaluation will determine the reasons why certain results occurred

or not to draw lessons, derive good practices and pointers for learning. It will enable

learning of particular countries, especially through the case studies, as well as

highlight lessons learned across countries. The evaluation will also provide

evidence-based findings to inform REACH’s future operational and strategic

decision-making. Findings will be actively disseminated and lessons will be

incorporated into relevant lesson sharing systems.

9. The evaluation will give equal weight to both accountability and learning.

2.3. Stakeholders and Users

10. A number of internal and external stakeholders have interests in the results of the

evaluation and some of these will be asked to play a role in the evaluation process.

Table 1 below provides a preliminary stakeholder analysis, which will be deepened

by the evaluation team as part of the Inception phase.

Table 1: Preliminary Stakeholders’ analysis9

Stakeholders Interest in the evaluation and likely uses of evaluation report to this stakeholder

INTERNAL STAKEHOLDERS

UNN/REACH

Steering Committee

(representatives from

FAO, IFAD, WHO,

WFP and UNICEF)

The SC is the main governing body for REACH and is closely involved in the decision making and direction setting of REACH. The SC has an interest in the performance and results of REACH as well as in recommendations to be applied for any future REACH countries. SC members will act as key informants and are also members of the Evaluation Reference Group (ERG).

UNN/REACH

Secretariat

The Secretariat carries out global level activities of REACH and manages and monitors progress at country level. It has an interest in the performance and results of REACH in the 5 countries and what should be used in the future. The evaluation will also be useful for fundraising. Secretariat staff play a role as key informants and selected staff are on the Evaluation Committee (EC).

Global Affairs

Canada (GAC)

GAC has funded REACH in 12 countries since 2011. GAC has an interest in an impartial account of the performance and results of REACH in the 5 countries funded for accountability purposes and future funding decisions. GAC is represented on the ERG.

9 This builds on the list of stakeholders identified during the 2015 evaluation of REACH.

Page 31: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 23 | P a g e

REACH facilitators The facilitators have an interest in the country case studies but also in the findings of the evaluation as a whole with regards to performance and results and how their experiences compare to those of the other REACH countries. REACH facilitators (both past and present) play a role as key informants. They will also assist with the provision of country level documentation, the programme for country visits and facilitate access to key stakeholders.

Members of REACH

Country

Committees

These are the stakeholders (country representatives of the REACH agencies) who are appointed in country to govern the REACH process. Their role in the evaluation is as key informants, and it will be important to have as many of them as possible in the final debriefing meeting in country.

Nutrition Focal

Points at country

level (FAO, WFP,

WHO, UNICEF,

IFAD)

The nutrition focal points work closely with the facilitators in the implementation of REACH. They have an interest in the country studies and in learning from other countries. Their role in the evaluation is that of key informants and liaison within their agencies. They should be able to comment on the effectiveness of REACH in facilitating UN coordination.

Regional Nutrition

Advisors (FAO,

WFP, WHO and

UNICEF) (IFAD does

not have)

The regional nutrition leads do not play a direct role in REACH but may offer a regional and, therefore, a more external perspective of the impact of REACH at country level as key informants. They may be interested in the final evaluation report, as well as country studies if within their region, depending on how much exposure they have had to REACH.

EXTERNAL STAKEHOLDERS

SUN (global and country level)

The role of REACH past, present and future is key to SUN, and therefore, the evaluation is of interest to SUN at country level (SUN government focal point) and the SUN Movement Secretariat (global). Both the SUN focal points (country level) and the Country Liaison Team at the SMS will act as key informants in the evaluation. SUN Focal Points and a representative of the Country Liaison Team are also members of the ERG.

Government Ministries (MoH, MoA and Food, Social Welfare, water etc. as relevant)

Government Ministries, in particular those involved in nutrition policy, practice and budgeting, are a key external partner to REACH (though the role will depend on the set up in country). They would be interested in lessons learned from REACH in their countries as well as others. They will act as key informants on experience to date of REACH as appropriate.

SUN Networks at

country level

CSOs, donors and the private sector at country level are working within the context of the SUN networks, where these have been established and/or supported. As a service of the UNN, REACH facilitates harmonised and coordinated UN nutrition efforts. REACH in some countries is also supporting the functioning of other SUN networks. Members of the SUN networks at country level will be key informants.

While the ultimate beneficiaries of REACH are women and children under five years of age,

REACH support, given its focus on strengthening the capacity of national governments and

supporting UN agencies, impacts these beneficiaries only indirectly. They will, therefore, not

be included in the evaluation.

11. The primary users of this evaluation will be:

Page 32: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 24 | P a g e

The UNN/REACH Secretariat and its UN agency partners in decision-making,

notably related to REACH establishment, implementation and management

across countries. Lessons learned will also be used to improve current

programmes and when expanding REACH to other countries in the future.

In-country stakeholders, including government (SUN Focal Points in

particular), UN, non-governmental partners, key donors, REACH facilitators to

know how effective REACH is, how to redirect if and when needed to improve

effectiveness, and how lessons can be shared across countries.

Global Affairs Canada (GAC), as the donor with the highest level of interest since

the evaluation focuses on countries funded by the Canadian government. Other

donors may be interested in the results because of their potential to fund the

REACH approach in other countries.

Other global actors, in particular the SUN Movement Secretariat (SMS) and

SUN Networks, with an interest in coherence and synergies between SUN and

REACH at country level; including also the role played by REACH in supporting

the establishment and functioning of SUN Networks including UNN.

3. Context and subject of the Evaluation

3.1. Context

12. In 2008 the Directors-General of FAO and WHO and the Executive Directors of

UNICEF and WFP wrote a letter to Country Representatives recognizing

undernutrition as a key component to malnutrition and health. The letter noted

that the causes of undernutrition are preventable and linked undernutrition to

overall economic and social development. The letter committed the agencies to

developing a partnership called the Renewed Efforts Against Child Hunger and

undernutrition (UN REACH) in an effort to strengthen the fight against

undernutrition. IFAD later joined REACH in an advisory role. REACH was initially

intended to help countries accelerate progress towards the Millennium

Development Goal MDG1, Target 3 (to halve the proportion of underweight children

under five globally by 2015) primarily through a public health oriented approach.

This approach evolved over time to reflect an evolving broadened multi-sectoral

approach which was articulated also in the 2013 Lancet Series10.

13. REACH takes place in the context of other UN and global initiatives on nutrition.

The SUN Movement was launched in 2010 and is currently active in 59 countries.

With the governments of countries in the lead, it unites stakeholders from civil

society, the UN, donors, businesses and academia in a collective effort (SUN

Networks) to end malnutrition in all its forms. REACH is a country-centred, multi-

sectoral approach to help strengthen national capacity for nutrition governance,

10 Mokoro 2015. Strategic Evaluation: Joint Evaluation of Renewed Efforts against Child Hunger and under-nutrition

(REACH) 2011-2015: Volume II Annexes. Oxford: Mokoro Ltd, October 2015.

Page 33: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 25 | P a g e

which also includes support to all SUN Networks and other partner organisations

to ensure effective engagement in multi-stakeholder processes and platforms.

14. In March 2015, the four principals of FAO, UNICEF, WFP and WHO agreed to

extend REACH through a re-validated MOU and WFP remain the designated host

agency. The principals also confirmed that REACH serve as the secretariat for the

UNN, a role previously co-facilitated with UNSCN. The UNN supports the

achievement of all Sustainable Development Goals (SDGs) and the Agenda 2030,

with a specific focus on Goal 2, as endorsed by the United Nations Decade of Action

on Nutrition (2016-2025). The UNN Strategy (2016-2020) further situates REACH

within the UNN with tools, human resources and experiences that can be drawn

upon, for support in response to assessed needs, where extra support is needed and

where funding is available. UNNs are present in all SUN countries while REACH

support is present in only a sub-set of SUN countries, depending on demand from

national government and the UNN.

3.2. Subject of the evaluation11

15. REACH aims to reduce maternal and child undernutrition in participating countries

as part of country efforts to achieve development goals. REACH’s contribution is to

strengthen nutrition governance and management in the countries in which it

works. Two overarching theories underlying REACH are that:

a. Through better coordination and less duplication, nutrition actions will

be more efficiently and effectively delivered.

b. By taking a multi-sectoral approach to nutrition, both nutrition direct

and sensitive interventions will have a bigger impact on nutritional status of

women and children.

16. To strengthen national governance and management, REACH implements

standardized approaches and tools in each country (see Annex 3). Capacity

strengthening of national actors is a critical dimension.

17. REACH’s modus operandi is to establish national facilitation mechanisms to

support countries to intensify coordinated action to address undernutrition and

stunting. An international facilitator is usually teamed up with a national facilitator

to support the establishment of effective systems for nutrition governance and

management, which are defined as sustainable, government-led, multi-sectoral and

solution-oriented and partnerships-based. Implementation arrangements have

varied from country to country depending on the national context.

18. REACH has a multi-tiered management structure with an international secretariat

based at WFP in Rome and governance in the form of a steering committee that

includes representatives of all partner agencies, in addition to its country level

governance.

11 Mokoro 2015. Strategic Evaluation: Joint Evaluation of Renewed Efforts against Child Hunger and under-nutrition

(REACH) 2011-2015: Volume II Annexes. Oxford: Mokoro Ltd, October 2015.

Page 34: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 26 | P a g e

19. Knowledge sharing systems are established and coordination mechanisms are set

up. The multi-sectoral approach aims to engage relevant government ministries

across relevant sectors on nutrition-specific and nutrition-sensitive actions to

ensure resources are used most effectively to reach those children in need.

20. The ultimate beneficiaries of REACH are women and children under five years of

age, the most affected vulnerable populations with nutritional deficiencies. REACH

supports the integration of gender equality and women’s empowerment in the

different policy documents and strategies and in planning, implementation and

monitoring and evaluation of the different sectors engaged in nutrition. Indicators

are broken down by sex and data is analysed with a gender perspective.

21. As shown in the REACH log frame12 (see Annex 4), REACH established a high level

impact aim of improving the nutritional status of children under five years of age

and women. This would be achieved by addressing the four REACH outcomes:

Outcome 1: Increased awareness and consensus of stakeholders of the nutrition situation and the best strategies and priorities for improvement

Outcome 2: Strengthened national policies and programmes that operationalize and address nutrition through a multi-sectoral approach

Outcome 3: Increased human and institutional capacity on nutrition actions at all levels

Outcome 4: Increased effectiveness and accountability of stakeholders in implementing and supporting nutrition actions

22. REACH began in three pilot countries13. Building on those experiences, the

Canadian International Development Agency (CIDA) funded REACH efforts in 2011

in eight additional countries14. In 2014, the Canadian Department of Foreign

Affairs, Trade and Development (DFATD) signed a grant to provide funding to four

generation 2 countries (Burkina Faso, Haiti, Myanmar and Senegal) and additional

funding to Mali, a generation 1 country. Implementation began in mid-late 2014

(Burkina Faso and Senegal) and early-mid 2015 (Haiti and Myanmar). An overview

of REACH resources to and country budgets can be found in Annex 5.

23. REACH has been successful in providing a unique, neutral facilitating and catalytic

function at country level, resulting in it being recognized as SUN “boots on the

ground” in the 2015 evaluation. It has been equally recognized for its quality tools

and strong competent staff. Challenges with REACH have been with regards to

building national ownership of the approach and its tools as well as UN agency

participation, both of which have impacted the sustainability of efforts post-

REACH. This appears less of a challenge for generation 2 countries following the

establishment of UNN for SUN at country level and clarity around the role of

12 The REACH log frame was first drafted in 2011 and a second version, with a reduction in the number of impact,

outcome and output indicators, was produced in 2013. The log frame has not undergone any further changes; except

that the language around Core Priority Interventions has been changed to Core Nutrition Actions. 13 Laos and Mauritania in 2008 followed by Sierra Leone in 2010 14 Bangladesh, Ghana, Mali, Mozambique, Nepal, Rwanda, Tanzania, Uganda.

Page 35: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 27 | P a g e

REACH as a service of the UNN. REACH tools have also been fine-tuned and

become much more embedded in the country nutrition governance process.

Cumulative processes and learnings of REACH have helped accelerate progress in

generation 2 countries. One remaining challenge for REACH is in mobilizing long-

term funding to be able to implement the approach over a five year period, as

recommended by the evaluation in 2015, and to be able to respond to country

requests for support. REACH has, however, managed to diversify its donor base.

4. Evaluation Approach

4.1. Scope

24. The evaluation will assess the effectiveness and efficiency of REACH, its

progress/achievements of results and the sustainability of those achievements in

five countries, including country case studies. The evaluation will also examine

issues that are cross-cutting in nature (such as gender and equity, participation,

national ownership, use of evidence, progress monitoring and reporting). The

evaluation will assess to what extent REACH outputs and outcomes addressed

gender and equity considerations. The evaluation will assess processes,

coordination arrangements, governance and partnerships at country level and

assess the support provided by the UNN/REACH Secretariat to the five countries.

25. Funding was received in March 2014 and activities are ongoing in all five countries

up to the present time. Therefore, the evaluation reference period will be from June

2014 up until August 2017, when the evaluation’s data collection will take place in

order to assess the fullest extent of results achievement.

4.2. Evaluation Criteria and Questions

26. Evaluation Criteria The evaluation will apply the international evaluation

criteria of Effectiveness, Efficiency, and Sustainability. The evaluation will assess

what has been achieved by REACH at country level and its overall performance and

effectiveness in achieving its objectives and outcomes, which are to improve

nutrition governance and management and, ultimately, improve nutrition in the

five countries covered by the evaluation. The evaluation will focus on assessing

changes at the outcome level using both quantitative and qualitative data. It will

also assess REACH’s efficiency and the extent to which REACH has been able to

build sustainable nutrition governance and management mechanisms in the five

countries including policies, systems and capacity. Impact will not be assessed as

the length of the REACH implementation period has not been long enough to see

changes at the impact level. The evaluation will not assess the relevance of REACH

since this was assessed during the 2015 evaluation. This evaluation will include an

assessment of gender and equity issues, which is particularly important considering

that REACH aims to positively impact women and children.

Page 36: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 28 | P a g e

27. Evaluation Questions Allied to the evaluation criteria, the evaluation will

address the following key questions, which, collectively, aim at highlighting the key

lessons and performance of REACH. The selected evaluation team will be expected

to develop the exact questions during the Inception phase:

Question 1: Performance at the country level15:

i) Effectiveness: Analysis of the nature, quantity and quality of results against

those intended; and unintended, including both positive and negative effects.

The focus is on to what extent REACH has been able to achieve its intended

outcomes and to what extent REACH’s efforts are being reflected and taken up

in policy and action planning at country level;

ii) Equity: Extent to which REACH outputs and outcomes address equity

consideration, including gender equity which is relevant to all four outcome

areas: awareness raising and consensus building; policies and action planning;

country priority interventions and coordinating mechanisms; and tracking and

accountability systems; as well as the extent to which outputs and outcomes are

moving towards achieving REACH’s intended impacts on women and children;

iii) Efficiency: Quantitative and qualitative assessment of the observed outputs

produced in relation to inputs; how efficient are the administrative structures

that REACH has put into place; are the current and/or proposed arrangements

for managing REACH the most cost and administratively effective; and, could

the results have been achieved more efficiently through other means.

Question 2: Contributing/explanatory factors: Analysis of the factors which affect REACH’s performance and results, including inter alia:

i) The operational and policy environments, capacity and resources, skills and

knowledge in participating countries;

ii) The governance and management of REACH at the country level;

iii) REACH partnerships at country level including: whether the necessary

commitment, agreement and actions were taken by partners to support REACH

to achieve its objectives.

Question 3: Sustainability

i) Sustainability of the results achieved and of the REACH operational model;

ii) The extent to which REACH is contributing to increased national ownership and

its leadership role in multi-sectoral nutrition governance and coordination.

4.3. Data Availability

28. The REACH log frame includes a range of qualitative and quantitative indicators.

The evaluation team will be given baseline and end line monitoring data for each of

the five countries. No data have been collected on the impact indicators as they are

long-term and it is too early to see impact.

15 Mokoro 2015. Strategic Evaluation: Joint Evaluation of Renewed Efforts against Child Hunger and under-nutrition

(REACH) 2011-2015: Volume II Annexes. Oxford: Mokoro Ltd, October 2015.

Page 37: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 29 | P a g e

29. Due to the nature of REACH, many of the REACH indicators are perception based.

While REACH has put in place tools for the collection of these data and a clearly

defined scoring system, the primary data source for many of the indicators is the

UN focal point team and the REACH facilitator’s observations.

30. The factors discussed above have implications for the reliability of data as well as

in terms of data comparability across countries. Not only are there differences in

the way that the indicators have been applied at country level but the subjectivity of

some of the scoring processes makes verifying the data challenging. As a result, the

evaluation conducted in 2015 did not include an analysis against all of the outcome

and output indicators. Instead, broader analysis and observations were noted.

31. The evaluation team will be given additional information including the Country

Implementation Plans, budgets and annual work plans. Monthly reports, minutes

of calls and meetings and donor reports will also be made available.

32. Concerning the quality of data and information, the evaluation team should:

a. assess data availability and reliability as part of the inception phase

expanding on the information provided in section 4.3. This assessment will

inform the data collection

b. systematically check accuracy, consistency and validity of collected data and

information and acknowledge any limitations/caveats in drawing

conclusions using the data.

4.4. Methodology

33. This section presents the overall preliminary methodology for the evaluation.

Building on this, a complete methodology guide will be designed by the evaluation

team during the inception phase. It should:

Employ the relevant evaluation criteria [effectiveness; efficiency; sustainability];

Demonstrate impartiality and lack of biases by enabling findings to be triangulated

from a variety of information sources and both qualitative and quantitative data

derived primarily from interviews with the full range of REACH stakeholders, data

analysis, and document and records reviews;

Apply an evaluation matrix geared towards addressing the key evaluation questions

taking into account the data availability challenges, the budget and timing

constraints;

Carry out case studies in all five countries to capture the diversity of country context

and operational modalities employed. An explanation of how country level findings

will be analysed and, where possible, synthesized should be included in the

Inception Report. Case studies are to explore the achievement of outputs and

outcomes, whether or not REACH is on track to achieve the planned impact,

indications of the sustainability of efforts, and the processes and methods used as

Page 38: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 30 | P a g e

well as the different modus operandi employed and their effectiveness. Case studies

will be based on document review and interviews with stakeholders and those

implementing REACH. The sampling technique to impartially select stakeholders

to be interviewed will be specified in the Inception Report;

Include an analysis of available baseline and end line data on REACH outcomes

which will be analysed at country level and across countries (where possible);

Enable an assessment of the effectiveness and efficiency of the governance and

management of REACH at country level including the REACH Country Committee

and technical group, as well as support provided by the REACH Secretariat;

Enable an assessment of the effectiveness of REACH partnerships at country level,

including whether the necessary commitment, agreement and actions were taken

by all partners to support REACH to achieve its objectives;

Where relevant, data will be disaggregated by sex, by age group and by country. The

evaluation findings and conclusions, including the country case studies, will

highlight differences in performance and results of the operation for different

beneficiary groups as appropriate.

34. The following mechanisms for independence and impartiality will be employed:

An Evaluation Committee (EC) will be established to support the Evaluation

Manager (EM) throughout the process, review evaluation deliverables and

submit them for approval to the Chair of the EC.

An Evaluation Reference Group (ERG) will be established to review and

comment on evaluation TOR and deliverables. ERG members act as experts in

an advisory capacity without any management responsibilities.

Further information on both mechanisms can be found in section 7 below. A

list of members of the EC and ERG can be found in Annex 6.

35. Potential risks to the methodology include timing of the evaluation, in particular

with regards to the availability of key stakeholders including facilitators (some

whose contracts are ending mid-year and there is the risk they may leave earlier for

other employment). This will be mitigated by confirming the country visit agenda

as early as possible and plan in line with people’s availability and contract end dates.

Additional risks are with regards to unforeseen political instability or security

issues. This will be mitigated again through mission planning, including identifying

beforehand any upcoming events such as elections and liaising with security staff.

4.5. Quality Assurance and Quality Assessment

36. WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the

quality standards expected from this evaluation and sets out processes with in-built

steps for Quality Assurance, Templates for evaluation products and Checklists for

their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance

Page 39: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 31 | P a g e

system (EQAS) and is based on the UNEG norms and standards and good practice

of the international evaluation community and aims to ensure that the evaluation

process and products conform to best practice.

37. DEQAS will be systematically applied to this evaluation. The WFP EM will be

responsible for ensuring that the evaluation progresses as per the DEQAS Process

Guide and for conducting a rigorous quality control of the evaluation products

ahead of their finalization.

38. WFP has developed a set of Quality Assurance Checklists for its decentralized

evaluations. This includes Checklists for feedback on quality for each of the

evaluation products. The relevant Checklist will be applied at each stage, to ensure

the quality of the evaluation process and outputs.

39. To enhance the quality and credibility of this evaluation, an outsourced quality

support (QS) service directly managed by WFP’s Office of Evaluation in

Headquarters provides review of the draft inception and evaluation report (in

addition to the same provided on draft TOR), and provide:

a. systematic feedback from an evaluation perspective, on the quality of the

draft inception and evaluation report;

b. recommendations on how to improve the quality of the final

inception/evaluation report

40. The EM will review the feedback and recommendations from QS and share with the

team leader, who is expected to use them to finalise the inception/ evaluation

report. To ensure transparency and credibility of the process in line with the UNEG

norms and standards[1], a rationale should be provided for any recommendations

that the team does not take into account when finalising the report.

41. This quality assurance process as outlined above does not interfere with the views

and independence of the evaluation team, but ensures the report provides the

necessary evidence in a clear and convincing way and draws its conclusions on that

basis.

42. The evaluation team will be required to ensure the quality of data (validity,

consistency and accuracy) throughout the analytical and reporting phases. The

evaluation team should be assured of the accessibility of all relevant documentation

within the provisions of the directive on disclosure of information. This is available

in WFP’s Directive (#CP2010/001) on Information Disclosure.

43. All final evaluation reports will be subjected to a post hoc quality assessment by an

independent entity through a process that is managed by OEV. The overall rating

category of the reports will be made public alongside the evaluation reports.

[1] UNEG 2016 Norms and Standards states Norm #7 states “that transparency is an essential element that establishes trust and

builds confidence, enhances stakeholder ownership and increases public accountability”

Page 40: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 32 | P a g e

5. Phases and Deliverables

44. The evaluation will proceed through the following phases. The deliverables and

deadlines for each phase are as follows:

Figure 1: Summary Process Map

45. During the preparation phase, the EM develops the evaluation TOR in line with

procedures. The EM will support the contracting of consultants and prepare a

document library and communication and learning plan. Deliverables: evaluation

TOR, TORs for EC and ERG, document library, communication and learning plan.

46. During the inception phase, the EM will organise an orientation meeting and

share relevant documents with the evaluation team for the desk review. The EM

will help organise inception meetings (remote) with key stakeholders. The

evaluation team will be responsible for drafting the inception report, including an

evaluation matrix and stakeholder analysis. This will be shared with the outsource

Quality Support Advisory service and updated accordingly by the EM before being

shared with the ERG for comments. Final inception report will be submitted to the

EC for approval. Deliverable: inception report.

47. To initiate the data collection phase, the EM will work with the evaluation team

on a country visit agenda, including meetings, identifying stakeholders and

providing administrative support as required. The evaluation team will undertake

data collection as per the agreed agenda. At the end of the field work, the evaluation

team will conduct a PPT debriefing based on data gathered and early analysis

conducted. Deliverable: debriefing PPTs (one per country).

48. The report phase includes the analysis of data gathered and the drafting, review,

finalisation and approval of the evaluation report. This phase is largely the

responsibility of the evaluation team, with inputs from the EM, EC and ERG. The

draft evaluation report will be shared with the outsource Quality Support Advisory

service and updated by the EM before being reviewed by the ERG. A final evaluation

report will be submitted to the EC for approval. Deliverable: final evaluation report.

49. During the dissemination and follow up phase, the EC will develop a

management response to the evaluation recommendations. Both the evaluation

report and the management response will be made publicly available by the EM. All

stakeholders involved in the evaluation will be requested to disseminate the

1. Prepare

•TOR; selection and contracting of

consultants; provisions for

impartiality and independence

2. Inception

•Inception Report

3.Collect data

•Country visits; data collection; debriefing PPT and case study

reports

4. Analyze data and Report

•Evaluation Report

5.Disseminate and follow-up

Page 41: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 33 | P a g e

evaluation report. UNN/REACH Secretariat will prepare a Management Response

and follow up on the status of implementation of the recommendations.

50. A more detailed evaluation schedule can be found in Annex 7.

6. Organization of the Evaluation

6.1. Evaluation Conduct

51. The evaluation team will conduct the evaluation under the direction of its team

leader and in close communication with Tania Goossens, the Evaluation Manager.

The team will be hired following agreement with WFP on its composition.

52. The evaluation team will not have been involved in the design or implementation of

the subject of evaluation or have any other conflicts of interest. They will respect

that people share information in confidence and inform participants of the score

and limitations of confidentiality. Neither EC members nor staff implementing

REACH will participate in meetings where their presence could bias the response of

the stakeholders. Further, the evaluation team will act impartially and in an

unbiased manner and respect the code of conduct of the evaluation profession.

6.2. Team composition and competencies

53. The evaluation team is expected to include 4 members, including the team leader.

The team leader will be international and will be joined by a regional consultant for

West Africa and a national or international consultant for Haiti (1) and Myanmar

(1), respectively. To the extent possible, the evaluation will be conducted by a

gender-balanced, geographically and culturally diverse team with appropriate skills

to assess gender dimensions as specified in the scope, approach and methodology

sections of the TOR. At least one team member should have WFP experience.

54. The team will include members with expertise and practical knowledge in the

following areas:

Food security and nutrition issues and governance, policy and advocacy.

Multi-sectoral nutrition programming at country level.

Coordination mechanisms, multi-sectoral partnerships or leadership.

Institutional change and capacity building.

Gender expertise / good knowledge of gender issues

All team members should have strong analytical and communication skills,

evaluation experience and familiarity with the countries they are evaluating

The team should have the appropriate language capacity (English, French).

55. The Team leader will have technical expertise in one of the areas listed above as well

as in designing methodology and data collection tools and demonstrated experience

in leading similar evaluations. She/he will also have leadership, analytical and

Page 42: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 34 | P a g e

communication skills, including excellent English writing and presentation skills.

The Team Leader should also have French language capacity.

56. Her/his primary responsibilities will be: i) defining the evaluation approach and

methodology; ii) guiding and managing the team; iii) leading the evaluation mission

and representing the evaluation team; iv) drafting and revising, as required, the

inception report, the end of field work (i.e. exit) debriefing presentation and

evaluation report in line with DEQAS.

57. The team members will bring together a complementary combination of technical

expertise required and have a track record of written work on similar assignments.

58. Team members will: i) undertake documentary review; ii) conduct field work; iii)

participate in relevant meetings including the debriefing; iv) draft and revise case

studies for their respective countries; v) contribute to the final evaluation report.

6.3. Security Considerations

59. Security clearance where required is to be obtained for all travel:

Consultants hired independently are covered by the UN Department of Safety & Security

(UNDSS) system for UN personnel which cover WFP staff and consultants contracted

directly by WFP. Independent consultants must obtain UNDSS security clearance for

travelling to be obtained from designated duty station and complete the UN system’s Basic

and Advance Security in the Field courses in advance, print out their certificates and take

them with them.16

60. However, to avoid any security incidents, the EM is requested to ensure that:

The WFP CO registers the team members with the Security Officer on arrival in

country and arranges a security briefing for them.

The team members observe applicable UN security rules and regulations.

7. Roles and Responsibilities of Stakeholders

61. The UNN/REACH Secretariat:

a- The Global Coordinator of the UNN/REACH will take responsibility to:

o Assign an EM for the evaluation: Tania Goossens, Programme Officer.

o Compose the internal EC and the ERG (see below).

o Approve the final TOR, inception and evaluation reports.

o Ensure the independence and impartiality of the evaluation at all stages, including

establishment of an EC and of an ERG.

o Participate in discussions with the evaluation team on the evaluation design and the

evaluation subject, its performance and results with the EM and the evaluation team.

16 Field Courses: Basic https://dss.un.org/bsitf/; Advanced http://dss.un.org/asitf

Page 43: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 35 | P a g e

o Organise and participate in two separate debriefings, one internal and one with

external stakeholders.

o Oversee dissemination and follow-up processes, including the preparation of a

Management Response to the evaluation recommendations.

b- Evaluation Manager:

o Manages the evaluation process through all phases including drafting this TOR

o Ensure quality assurance mechanisms are operational

o Consolidates and shares comments on draft TOR, inception and evaluation reports

with the evaluation team

o Ensures expected use of quality assurance mechanisms (checklists, quality support)

o Ensure that the team has access to all documentation and information necessary to

the evaluation; facilitate the team’s contacts with stakeholders; set up meetings and

field visits; provide logistic support; and arrange for interpretation, if required.

o Help ensure the organisation of security briefings for the team as appropriate.

62. An internal Evaluation Committee has been formed as part of ensuring

independence and impartiality. The EC is composed of key staff of the

UNN/REACH Secretariat17. The EC will oversee the evaluation process by making

decisions, giving advice to the EM and commenting on and clearing evaluation

products submitted to the chair for approval. EC members will also be responsible

for ensuring evaluation recommendations are implemented.

63. An evaluation reference group has been formed and is composed of REACH

internal and external stakeholders18. The ERG will review the evaluation products

as further safeguard against bias and influence.

64. WFP Country offices will provide logistical and administrative support to the

evaluation team as appropriate

65. Stakeholders in in participating countries and at the REACH Secretariat will be

asked to provide information necessary to the evaluation; be available to the

evaluation team to discuss REACH, its performance and results; facilitate the

contacts with stakeholders; and help set up meetings. A detailed agenda will be

presented by the evaluation team in the inception report.

66. The Office of Evaluation (OEV). OEV will advise the EM and provide support

to the evaluation process where appropriate. It is responsible to provide access to

independent quality support mechanisms reviewing draft inception and evaluation

reports from an evaluation perspective.

17 A list of members can be found in Annex 6. 18 idem.

Page 44: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 36 | P a g e

8. Communication and budget

8.1. Communication

67. The EM will ensure consultation with stakeholders on each of the evaluation phases

as shown in Figure 1 (above). In all cases the stakeholders’ role is advisory. The

evaluation team will conduct country debriefings at the end of country data

collection. Participants unable to attend a face-to-face meeting will be invited to

participate by telephone. A communication plan for the evaluation will be drawn up

by the EM during the inception phase. The evaluation report will be posted on

WFP’s external website and the UNN/REACH website once complete.

68. Key outputs during the evaluation phase will be produced in English. Country case

studies for Haiti, Senegal, Mali and Burkina Faso will be produced in French.

Should translators be required for field work, they will be provided.

69. As part of the international standards for evaluation, WFP requires that all

evaluations are made publicly available. Following the approval of the final

evaluation report, it will be translated into French and any French language country

case studies will be translated into English. During the inception phase, the EC will

agree on a plan for report dissemination in line with evaluation objectives.

8.2. Budget

70. Budget: For the purpose of this evaluation, the budget will include:

Hire of individual consultants through Human Resources (HR) action and thus be

determined by “HR regulations on consultancy rates;”

Coverage of travel expenses and subsistence fees for consultants as appropriate;

Provisions for stakeholder workshops as defined in the evaluation timeline and

country mission schedules;

Translation of final evaluation products.

GAC has provided funding for the evaluation, through the REACH Trust Fund. The

overall expected cost of the evaluation, including preparatory work, is estimated at

USD 120,000. This includes an estimated 83 days for the Team Leader, 47 days for

the Regional Consultant and 16 days each for the two national consultants.

Please send any queries to Tania Goossens, Evaluation Manager, at [email protected]

or (+39) 06 6513 2348.

Page 45: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 37 | P a g e

Annex 1 REACH Theory of Change

Page 46: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 38 | P a g e

Annex 2 Conclusions and Recommendations of the Joint Evaluation of REACH

2011-201519

CONCLUSIONS

1. Across the eight countries, most of REACH’s progress was made towards outcomes 1 and 2, with less or no progress on outcomes 3 and 4. This was related in part to limited timeframes and the sequential nature of REACH’s outcomes.

2. REACH’s progress was significantly influenced by the performance of the Secretariat in Rome. The process of launching REACH was slow and in some respects disjointed and confused. The Secretariat’s system has gradually introduced a reasonably standardized programme of effort across eight or more countries.

3. REACH fits well with the international nutrition agenda and convening UN agency priorities; and has been broadly relevant to country policies and priorities. There are limitations in applying a standard model insufficiently adjusted to local realities and under tight timeframes.

4. REACH has provided relevant, timely and well-prioritized facilitation and support, which has furthered the nutrition response in the countries where it has been present. REACH has successfully contributed to greater stakeholder engagement, with progress in REACH countries in the level of commitment to nutrition, more effective priority setting, and capacity building. REACH has also made, but with more variable levels of success, a contribution to monitoring and to accountability.

5. The achievements and weaknesses of REACH reflect its key design and implementation qualities. Positive features include: flexibility of procedures and arrangements; on the ground presence; quality tools and instruments; strong dialogue; neutrality; and a focus on processes as well as results. REACH has also effectively supported SUN in furthering the nutrition agenda. However, there has been an element of overshadowing by the SUN movement, which has contributed to REACH being relatively less known and understood.

6. The challenges that REACH has faced reflect: its weak TOC; the ambitious nature of its plans and timeframes; the sequential nature of REACH’s outcomes (requiring more time to be implemented); varying levels of ownership by governments; and lack of partnership strategy that caused low levels of buy-in and support from its partner agencies. The REACH TOC did not sufficiently take account of outcome to impact level factors such as the importance of high level political commitment by Governments, the political economy of the UN, and the lack of clear accountability and incentives for support to REACH within the UN. The latter was undermined by the absence of: i) sustained commitment from the highest level of the UN organizations; ii) a clear mandate by the UN to coordinate and work together; and iii) strong and enforced accountability mechanisms.

7. In practice, government and UN commitments were not always strong and clear enough for things to move forward. In terms of internal governance, the variable and in some cases low level of commitment and buy-in of the Technical Group and the REACH Coordinating Committee (RCC) at country level were key factors affecting performance. In a crowded global landscape, the establishment of REACH and its existence continues to be questioned by some nutrition actors.

8. Overall, the results and achievements of REACH are unlikely to be sustainable unless additional investments and efforts are made. There has been insufficient attention to the effects on SUN when REACH ends. The strategies for exiting from countries were premature compared to the level of progress in country, and were developed late in the process.

19 Mokoro 2015. Strategic Evaluation: Joint Evaluation of Renewed Efforts against Child Hunger and under-nutrition (REACH) 2011-2015: Volume II Annexes. Oxford: Mokoro Ltd, October 2015.

Page 47: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 39 | P a g e

Recommendations

41.The evaluation team formulated these recommendations at a time when various far-reaching decisions had recently been made, including on: i) REACH becoming the secretariat of the UN Network for SUN; and ii) in parallel, the roll-out of arrangements for funding REACH in additional countries. These decisions assume that there is a continued need for REACH and influence its future role, functioning, structure and scope.

42. Recommendation 1: The core function of REACH should continue to be facilitation and coordination of country-level nutrition responses, with a strong focus on maintaining and developing its reputation for neutrality. This function should be based on two modes of intervention: one should involve multi-year facilitation services, building on the approach adopted to date; and the other should involve specialized short-term facilitation and related services for countries meeting specific criteria.

43. Continued support at the country level to strengthen facilitation in the SUN countries20 should recognize that it may be possible to continue multi-annual “REACH-like” engagements in selected countries – subject to full appraisals – but that in other countries the REACH contribution will have to be on a smaller scale, with specific criteria developed to ensure feasibility. REACH’s perceived neutrality has allowed it to be effective as a broker among different organizations and entities. To maintain this neutrality, clear limits should be placed on the time, type of engagement and resources that REACH dedicates to supporting the UN Network for SUN.

44. Recommendation 2: REACH should develop a medium-term vision, strategies and an operating plan for its second phase, which has a five-year timeframe to align effectively with SUN’s five-year timeframe and strategy.

45. This will require:

extending the timeframe in existing REACH countries by two more years to consolidate gains and

move towards sustainability (Bangladesh, Ghana, Mali, Mozambique, Nepal, Rwanda, Uganda and

the United Republic of Tanzania); and

adopting a five-year timeframe in new countries from the outset.

46. Recommendation 3: As part of its key strategies for engagement, REACH should encourage the UN Network for SUN – which REACH now coordinates – to align its focus with REACH’s core function of facilitation and coordination. The network – and REACH’S support to it – would thus have a central mission in mobilizing the technical strength of the United Nations for facilitating scaled-up and effective country-level nutrition responses.

47. REACH’s new and additional responsibility as Secretariat of the UN Network for SUN provides the possibility of greater alignment between SUN and REACH. There is opportunity and potential risk in the new arrangement. The opportunity lies in the fact that the valuable resources and leveraging power of the UN can be used effectively in the nutrition response. The risk is that of side-tracking what REACH has done well and of REACH losing its valuable neutrality. To address this risk, there is a need for clarity on what the UN Network for SUN can achieve and for this to align with the focus and mandate of REACH.

48. Recommendation 4: The next phase of REACH – and further decisions on funding multi-year, country-level interventions – should be based on a thorough reappraisal of the REACH theory of change, which should recognize that the role of REACH is facilitation and related services, rather than technical assistance or support. The new theory of change should form both the role of REACH as the implementer of SUN in the field and its support to the UN Network for SUN. It should be broadly disseminated to contribute to better understanding of REACH’s role in the overall nutrition environment.

20 SUN covers 55 countries (http://scalingupnutrition.org/sun-countries).

Page 48: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 40 | P a g e

49. The design of any future REACH multi-year intervention should explicitly state and test the assumptions on which it is based and identify the conditions for receiving REACH support. The evaluation identified five conditions for implementation of REACH multi-year programming: i) a senior REACH facilitator should be in-country for a minimum of five years; ii) thorough consultative preparation by and commitment from all parties; iii) plans for supporting immediate start up; iv) financial commitments from UN partners to supporting the REACH approach; and v) early work on approaches to sustainability.

50. Recommendation 5: To inform the new theory of change, REACH should commission a study of the architecture of technical assistance for scaling up nutrition. The study should include facilitation and identify priority areas for REACH, taking into account the work of other technical-support partners. The study should be used to inform REACH’s medium-term plan of action and its strategies for engagement in the coming five years (see recommendations 1–4).

51. Recommendation 6: Participating UN agencies should sign a new MoU with stronger provisions that include strategic decision-making and accountability mechanisms at the most senior level of UN agencies; commitment to contributing funding to country-level REACH activities; and commitment to better coordinating their planning, resourcing, implementation and advocacy efforts in the nutrition sector at the country level.

52. Future work to support country-level coordination of nutrition interventions through REACH should be contingent on serious and public commitment at all levels of UN agencies to better coordinate their planning, resourcing, implementation and advocacy efforts in this sector. To this end, high-level commitments from agencies need to be matched with commitments to collaboration at technical level, underscoring that this will entail a less agency-centred approach. In the absence of these commitments, there is the risk that REACH will lose focus, waste effort and ultimately fail.

53. Recommendation 7: The REACH partnership should proactively explore and develop funding options and sources for its second phase. Recognizing its recently augmented role regarding the UN Network for SUN, it should particularly encourage appropriate financial allocations from member agencies (see recommendation 6), donors and host countries. Funding from host governments should be encouraged as a means of ensuring sustainability in countries where multi-year engagement is foreseen.

54. Recommendation 8: Country-level implementation of REACH should continue to be guided by CIPs and annual plans. However, CIP processes should be revised to ensure maximum leadership and buy-in from all stakeholders. CIPs should also adopt an approach to ensuring that equity and gender issues are part of the country-level work and global advocacy on nutrition. Ensuring that REACH has expertise in gender and equity, establishing incentives for national actions on gender and equity in nutrition, and monitoring progress against indicators are all essential.

Page 49: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 41 | P a g e

Annex 3 REACH deliverables and tools

Page 50: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 42 | P a g e

Annex 4 REACH Log frame

Page 51: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 43 | P a g e

Annex 5 Overview of REACH Resources and Country Budgets for Burkina

Faso, Haiti, Mali, Myanmar and Senegal

REACH active donor grants

Donors Contribution USD Grant Validity Countries

EU EUR 550,000 586,980 Feb 2017-April 2018 Chad

Irish Aid EUR 1,000,000 1,086,957 Dec 2016-Dec 2017 Lesotho, Sierra Leone, Zimbabwe & Tanzania

Canada - GAC - Generation 2*

CAD 5,000,000 4,488,330 2014-2017 Burkina Faso, Haiti, Mali, Myanmar & Senegal

Canada - GAC - Generation 1

CAD 15,000,000 15,290,520 2011-2016 Bangladesh, Ghana, Mali, Mozambique, Nepal, Rwanda, Tanzania & Uganda

Canada - 2. grant agreement

Country* USD

(2014-2017)

Burkina Faso 845,833

Haiti 764,500

Mali** 285,000

Myanmar 760,000

Senegal 925,833

Total 3,581,166 *NB: A no-cost extension has been granted for the five countries to 31.12.2017 **Mali had received funding from a previous grant which expired in 2016

Page 52: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 44 | P a g e

Annex 6 Membership of the Evaluation Committee and of the Evaluation

Reference Group

Evaluation Committee

Nancy Walters, UNN/REACH Secretariat (Chair of EC)

Nicolas Bidault, UNN/REACH Secretariat

Tania Goossens, UNN/REACH Secretariat (Evaluation Manager)

Christine Wenzel, UNN/REACH Secretariat

Evaluation Reference Group

Martin Bloem, WFP (replaced by Lauren Landis, WFP)

Anna Lartey, FAO

Victor Aguayo, UNICEF

Francesco Branca, WHO

Juliane Friedrich, IFAD

Isabelle Laroche, Global Affairs Canada (replaced by Joyce Seto, GAC)

Maimouna Doudou, REACH Burkina Faso

Ousmane Ouedraogo, REACH Burkina Faso

Bertine Ouaro, SUN Focal Point Burkina Faso

Souleymane Diallo, REACH Mali

Amadou Fofana, REACH Mali

Dr Djibril Bagayoko, SUN Focal Point Mali

Sophie Cowppli-Bony, REACH Senegal

Aida Gadiaga, REACH Senegal

Abdoulaye Ka, SUN Focal Point Senegal

Agnes Solano, REACH Haiti

Marie-Mona Alexis, REACH Haiti

Dr. Joseline Marhone, SUN Focal Point Haiti

SanSanMyint, REACH Myanmar

Dr. May Khin Than, Director of the National Nutrition Center (NNC) (SUN Secretariat Myanmar)

Delphine Babin-Pelliard, SUN Movement Secretariat (replaced by Fanny Granchamp and Thahira Mustafa, SMS)

Page 53: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 45 | P a g e

Annex 7 Evaluation Schedule

Phases, Deliverables and Timeline Key Dates

Phase 1 - Preparation 2017 Desk review, first draft of TOR and quality assurance March 8 Circulation of TOR and review by ERG and EC March 21 Identification and recruitment of evaluation team March 31 Final TOR March 31 Phase 2 - Inception Data library to evaluation team for desk review April 7 Orientation call with evaluation team April 12 Inception mission to Rome April 25 Review documents and draft inception report including

methodology. April 25-May 5

Submit draft inception report to Evaluation Manager May 5

Quality assurance and feedback (EM and quality support system)

May 12

Revise inception report May 17

Submit revised inception report to Evaluation Reference Group

May 17

Revise inception report May 24-26

Submit revised inception report to Evaluation Committee

May 26

Sharing of inception report with stakeholders for information May 29

Phase 3 – Data collection and analysis

Field work (Senegal, Mali, Burkina Faso, Haiti, Myanmar) (on average 10 calendar days per country)

May 28-August 15

In-country Debriefing (at end of each country visit) June 5-August 15 Phase 4 - Reporting

Draft evaluation report August 15-September 22

Submit Draft evaluation report to Evaluation Manager September 22

Quality assurance and feedback (EM and quality support system)

September 29

Revise evaluation report October 6 Submit revised evaluation report to Evaluation

Reference Group October 24

Consolidate comments November 2 Revise evaluation report November 20

Submit final evaluation report to Evaluation Committee

November 25

Phase 5 Dissemination and follow-up

Final report disseminated to all stakeholders December 1 Follow up on recommendations December

onwards

Page 54: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 46 | P a g e

Annex 8 Acronyms

CIDA Canadian International Development Agency

CNA Core Nutrition Action

CO Country Office

CSO Civil Society Organization

DEQAS Decentralized Evaluation Quality Assurance System

DFATD Canadian Foreign Affairs, Trade and Development

EC Evaluation Committee

EM Evaluation Manager

ERG Evaluation Reference Group

FAO Food and Agriculture Organization

GAC Global Affairs Canada

IFAD International Fund for Agricultural Development

M&E Monitoring and Evaluation

MDGs Millenium Development Goals

MoA Ministry of Agriculture

MoH Ministry of Health

MoU Memorandum of Understanding

OEV Office of Evaluation

REACH Renewed Efforts against Child Hunger & undernutrition

SC Steering Committee

SDGs Sustainable Development Goals

SMS SUN Movement Secretariat

SUN Scaling Up Nutrition

TOR Terms of Reference

UN United Nations

UNDAF United Nations Development Assistance Framework

Page 55: Decentralized Evaluation - docs.wfp.org

TOR REACH Evaluation March 2017 47 | P a g e

UNDAP United Nations Development Assistance Plan

UNDSS United Nations Department of Safety & Security

UNICEF United Nations Children’s Fund

UNN UN Network for SUN

UNSCN United Nations Standing Committee on Nutrition

WFP World Food Programme

WHO World Health Organisation

Page 56: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 48 | P a g e

Annex 2: Evaluation Matrix

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q1. Performance at the country level

Q1.1 Effectiveness: how effective has REACH been in achieving intended outcomes?

1.1.1 What progress has been made in delivering outputs and achieving REACH’s four outcomes:

a) Increased awareness and consensus

b) Strengthened national policies and programmes

c) Increased human and institutional capacity on nutrition

d) Increased effectiveness and accountability

1.1.2 Was there any intended positive or negative outcome?

1.1.3 How did the realization of intended outcomes vary between countries?

1.1.4 Where was REACH most successful, where least and why?

Actual versus planned REACH outputs:

a) State of completion of: stocktaking exercise; consensus on CNAs; cross-benefit analysis; joint advocacy

b) State of completion of: nutrition in government & UN strategy; multi-sector national nutrition action plan; sector/CNA update; sub-national CNA update

c) State of completion of: capacity gap analysis & planning; capacity development; guidance materials & training

d) State of completion of: multi-sector M&E; accountability; joint UN effectiveness

Intended outcomes versus actual outcomes (end-line compared to baseline data)

Stakeholders interviews

REACH SC REACH Secretariat REACH CC REACH facilitators UN agencies nutrition focal points Sector ministries (members of national multi-sector platforms) CSO alliance (Chair and co-chair) Donor network (Chair, co-chair) and GAC

Document review

REACH documents and data (CIP, annual work plans, baseline and end-line data; meetings and workshop reports)

National policy and strategy documents

Semi-structured individual interviews

Document review: systematic analysis of different types documents (REACH, Government)

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Page 57: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 49 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q1.2 Equity: to what

extent have REACH

outputs and outcomes

addressed equity

considerations,

including gender

equity?

1.2.1 To what extent were gender commitments in respective CIPs implemented?

1.2.2 To what extent are REACH outputs and outcomes moving towards achieving intended impacts on women and children?

1.2.3. How did equity considerations vary between countries?

1.2.4 Where was REACH most successful, where least and why?

Evidence of REACH work plans addressing: integration of gender equality/women’s empowerment in relevant sector policies and strategies; 2) analysis of relevant indicators with a gender perspective; advocacy for women to be represented in the different coordination mechanisms at all levels; and advocacy for gender sensitive messages disseminated by the different partners/channels

Evidence of prioritization in country of women and children under 5

Stakeholders interviews

REACH Secretariat REACH CC REACH facilitators UN agencies nutrition focal points Sector ministries (members of national multi-sector platforms)

Document review

REACH documents and data (CIP, annual work plans, mission reports) National policy and strategy documents

Semi-structured individual interviews

Document review: systematic analysis of different types documents (REACH, Government)

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Page 58: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 50 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q1.3 Efficiency: to what extent were resources/inputs (such as funds, expertise, time, etc.) used optimally to achieve intended outputs?

1.3.1 Were resources optimally planned and used in relation to intended outputs?

1.3.2 Were REACH administrative/management arrangements conducive to timely delivery of set outputs?

1.3.3 Where was REACH most efficient, where least and why?

Rate of budgetary implementation

Compliance of expenditures with approved budget plans

Timeliness of funds requisition and release

Timeliness of delivered outputs

Adequacy of planned outputs vis-à-vis national priorities and identified gaps

Stakeholders interviews

REACH Secretariat REACH CC REACH facilitators UN agencies nutrition focal points Sector ministries (members of national multi-sector platforms)

Document review

Annual Progress Reports

Expenditure tracking sheets

Semi-structured individual interviews

Collecting and analysing secondary information from existing databases

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Q2 Contributing/Explanatory Factors

Page 59: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 51 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q2.1 How have REACH performance and results been affected by the operational and policy environments, capacity and resources, skills and knowledge?

2.1.1 Were REACH implementation plans negatively or positively affected by exogenous factors? And if so which?

2.1.2 How did positive and negative contributory/explanatory factors vary between countries? Are there communalities between countries?

2.1.3 Where was REACH most

successful, where least and

why?

Positive and negative exogenous factors that affected implementation of planned outputs, such as: political stability; policy environment; climatic hazards or man-made disasters; technical and human resources capacity of relevant government entities

Awareness/knowledge/perceptions of internal and external stakeholders of REACH mandate, facilitators role and work plan

Positive and negative factors that affected adherence to annual work plans

Stakeholders interviews

REACH Secretariat REACH CC REACH facilitators UN agencies nutrition focal points Sector ministries (members of national multi-sector platforms)

Document review

Country sector analysis reports/nutrition profiles from different sources

Minutes of multi-stakeholders meetings

Semi-structured individual interviews

Document review: systematic analysis of different types documents

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Page 60: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 52 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q2.2 How have REACH performance and results been affected by its own governance and management at country level?

2.2.1 Were REACH implementation plans negatively or positively affected by institutional arrangements? And if so which?

2.2.2 How did positive and negative factors vary between countries? Are there communalities between countries?

2.2.3 Where was REACH most successful, where least and why?

Areas where governance and management have been a positive influence and where negative (intentional or not): placement arrangements, funding mechanisms, procedures, etc.

Stakeholders interviews

REACH CC REACH facilitators UN agencies nutrition focal points

Document review

Semi-structured individual interviews

Document review

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Page 61: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 53 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q2.3 To what extent have REACH’s partners demonstrated the necessary commitment, agreement and actions to support REACH to achieve its objectives?

2.3.1 Are processes put in place

to ensure dialogue and joint

actions?

2.3.2 How did partners’ commitment and engagements vary between countries? Are there communalities between countries?

2.3.3 Where was partners’ involvement most successful,

where least and why?

Existence of processes for dialogue and joint actions

Levels of commitment amongst partners (attendance at meetings, interactions, evidence of joint working/ joint initiatives)

Knowledge and perceptions of REACH amongst external partners

Type and regularity of interactions between REACH facilitators, SUN Focal point and SUN networks

Stakeholders interviews

REACH SC REACH Secretariat REACH CC Regional nutrition advisors REACH facilitators UN agencies nutrition focal points SUN focal point Sector ministries (members of national multi-sector platforms) CSO alliance (Chair and co-chair) Donor network (Chair, co-chair) and Canada Document review CIPs, minutes of meetings

Semi-structured individual interviews

Document review

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Q3. Sustainability

Page 62: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 54 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q3.1 To what extent are the results achieved and the REACH operational models sustainable?

3.1.1 Were REACH outputs officially endorsed by relevant national entities and national resources (human and financial) made available to sustain them?

3.1.2 Where is sustainability most likely, where least and why?

Official endorsement of REACH outputs by relevant national entities

REACH Transition plan planned or in progress

Evidence (steps taken) for uptake of REACH functions and tools into country nutrition governance processes

Evidence (steps taken) for phasing-over UN coordination-related REACH functions to the UN Network in-country (clearly defined priorities, budgets and responsibilities

Stakeholders interviews REACH SC REACH Secretariat REACH CC Regional nutrition advisors REACH facilitators UN agencies nutrition focal points SUN focal point Sector ministries (members of national multi-sector platforms) CSO alliance (Chair and co-chair) Donor network (Chair, co-chair) and Canada Document review Transition plan, minutes of meetings

Semi-structured individual interviews

Document review

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Page 63: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 55 | P a g e

Key Question Sub-question Measure /indicator Source of information

Data collection Methods

Data analysis methods

Q3.2 To what extent is REACH contributing to increased national ownership and its leadership role in multi-sectoral governance and coordination?

3.2.1 Did REACH contribute to increased national ownership and leadership role in multi-sector governance and coordination? And if so how?

3.2.2 Where was national ownership and leadership most enhanced, where least and why?

Stakeholders perceptions about REACH facilitators capacities to mobilize/facilitate/coach and about usefulness of REACH analytical tools and methodologies

Status of streamlining of REACH analytical tools and methodologies into nutrition governance processes

REACH contribution to positioning of nutrition in the national development agenda

REACH contribution to the functionality of government multi-sector coordination structures with clear roles and responsibilities

Stakeholders interviews

REACH Secretariat REACH CC Regional nutrition advisors REACH facilitators UN agencies nutrition focal points SUN focal point Sector ministries (members of national multi-sector platforms) CSO alliance (Chair and co-chair) Donor network (Chair, co-chair) and Canada

Document review

National development and sector policies and strategies/action plans

Country progress reporting to Secretariat and CC

Baseline and end-line data

Minutes of country consultation workshops/meetings

Semi-structured individual interviews

Document review

In country debriefings

Triangulation of information obtained through different methods and from different sources

Validation of preliminary findings through debriefings

Comparing countries case studies findings

Page 64: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 56 | P a g e

Annex 3: Country Visit Schedule

Date Time Organization Name Function Location

28 June 08h-11h REACH Maimouna DOUDOU, Ousmane OUEDRAOGO

REACH facilitators WFP

12h-13h WFP Jean-Charles DEI WFP Representative WFP 15h-16h Ministry of Education Bamouni INNOCENT Department Head, Health

Sanitation Nutrition Laico Hotel

29 June 8h-9h WHO Fousséni DAO

WHO Nutrition Focal Point WHO

10h-11h Ministry of Water and Sanitation

Anissatou OUEDRAOGO Officer, Directorate Water and Sanitation

Laico Hotel

11h-12h WFP Olga NINON Project Officer Nutrition Focal Point

Laico Hotel

15h-16h Canada Mathieu RIOUX First Secretary (development) Canadian Embassy 30 June 9h-10h

10h-11h

14h-15h ACF (RESONUT) Hermann GOUMBRI

Advocacy Officer ACF

1 July 11h-12h UNICEF Denis GARNIER UNICEF Nutrition Focal Point UNICEF 2 July Power Point Presentation Preparation 3 July 9h-10h FAO Prosper SAWADOGO Project Officer/ Nutrition Focal

Point FAO

10h-11h FAO Aristide ONGONE OBAME FAO Representative FAO 11h30-12h30 WHO Alimata Jeanne DIARRA-NAMA

WHO Representative/UNN Chair

WHO

14h-15h Directorate, Nutrition Division

Bertine OUARO DABIRE SUN Focal Point/Director of Nutrition Division

Directorate, Nutrition Division

15h30-16h30 WHO Ferima COULIBALY-ZERBO Nutrition Focal Point, WHO Regional Office

WHO/IST

4 July 9h-10h Donor Network Fanta OUEDRAOGO Project Manager European Union

11h-12h30 UNICEF Anne VINCENT UNICEF Representative/ UNN Vice-Chair

UNICEF

14h-15h SP-CNPS Karime GANEMTORE Director of safety net monitoring

SP-CNPS

Page 65: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 57 | P a g e

15h30-16h30 Private Sector Network for Nutrition (RAPNUT)

Diédonné Alain HIEN President of RAPNUT WFP

5 July 9h-10h Parliamentarians Honorable GNOUMOU

Parliamentarian Network for Food Security, Coordinator (REPASEN)

10h30-11h30 Food Security Benedicta AKOTIONGO Executive Secretary SE CNSA 11h30-12h30 Debriefing with facilitators WFP

13h-14h Follow-up presentation preparation 6 July 9h-10h Follow-up WFP

REACH SEC (by TC)

Nicolas BIDAULT Deputy Global Coordinator

Tania GOOSSENS Programme Manager

Christine WENZEL M&E Officer

REACH Country Maimouna DOUDOU International Facilitator

Ousmane OUEDRAOGO National Facilitator

WFP Olga NINON WFP Nutrition Focal Point

WHO Fousséni DAO Nutrition Focal Point

UNICEF Kayari EMNA UNICEF Nutrition

FAO Prosper SAWADOGO FAO Nutrition Focal Point

ACF Talara DAYAMBA RESONUT

Assembly Honorable GNOUMOU Coordinator (REPASEN)

DN Estelle BAMBARA DN Officer

DN Céline ZONGO DN Officer

Ministry of Water and Sanitation

Anissatou OUEDRAOGO Nutrition Focal Point

Departure for return trip

Page 66: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 58 | P a g e

Annex 4: List of People Interviewed

Organization Name Function

External Stakeholders

REACH Secretariat Nancy WALTERS UNN/REACH Global Coordinator

REACH Maimouna DOUDOU International Facilitator

Ousmane OUEDRAOGO National Facilitator

Members of REACH Country Committee

WFP Jean-Charles DEI WFP Representative

FAO Aristide ONGONE OBAME

FAO Representative

WHO Alimata Jeanne DIARRA-NAMA

WHO Representative/ UNN Chair

UNICEF Anne VINCENT UNICEF Representative/ UNN Vice-Chair

United Nations Agency Nutrition Focal Points

FAO Prosper SAWADOGO Nutrition Officer

WFP Olga NINON Nutrition Officer

WHO Boubacar SIDIBE Neglected Tropical Diseases Officer

UNICEF Denis GARNIER Nutrition Programme Officer

Canadian Cooperation

Canadian Embassy Mathieu RIOUX First Secretary (development)

External Stakeholders

SUN Focal Points and Network

RESONUT (Civil Society) Hermann GOUMBRI Advocacy Officer ACF

REPASEN (Parliamentarians)

Honorable GNOUMOU Coordinator (REPASEN)

RAPNUT (Private sector) Diédonné Alain HIEN President of RAPNUT

Donor Network (EU) Fanta OUEDRAOGO Project Manager

SUN focal point Bertine OUARO DABIRE

Director of Nutrition

Ministry of Water and Sanitation

Anissatou OUEDRAOGO

Officer, Directorate Water and Sanitation

Food Security Benedicta AKOTIONGO Executive Secretary

SP-CNPS Karime GANEMTORE Director of safety net monitoring

Ministry of Education Bamouni INNOCENT Head of Health, Sanitation and Nutrition Section

Others

WHO Ferima COULIBALY-ZERBO

Nutrition Focal Point, WHO Regional Office

Page 67: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 59 | P a g e

Annex 5: Data Collection Tools

Background

1. What was the situation in like before REACH and what were the outstanding challenges?

2. What is your perception of REACH’s capacity to resolve/reduce these challenges?

3. Overall, has REACH reached or surpassed expectations?

Performance of REACH and Explanatory Factors (EQ1 and EQ2)

4. What key outcomes has REACH contributed to at country level? What were the key events and contributing organizations? Which actor(s) played a major role? In what way? What factors explain the achievement of the REACH outcomes at country level?

5. How has the performance of REACH been affected by the operational and policy environments at country level? Please elaborate.

6. What, if any, have been the unintended outcomes of REACH’s interventions at country and global level? (Please make sure we get examples/evidence)

7. Are there particular equity challenges? To what extent, and in what way, has REACH contributed to creating awareness and to putting in place approaches on equity and gender issues in nutrition at country level? (Please make sure we get examples/evidence)

8. How effective have REACH’s governance structures been in supporting the achievement of its objectives? How effectively have the Secretariat and the country level worked together?

9. In what ways if any has the coordination among UN agencies evolved over the past years? Has REACH contributed to this?

10. What are the lessons learned about REACH performance?

Sustainability of REACH (EQ3)

11. To what extent are the outcomes that REACH has contributed to sustainable and how have they encouraged national ownership?

12. Was it realistic to expect that REACH would make a significant difference in the time frame that it was given (3 years)?

Future of REACH

13. If you had to make recommendations for the future of REACH what would you recommend?

Page 68: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 60 | P a g e

Annex 6: Bibliography

Government of Burkina Faso. 2016. Plan National de Développement Economique et Social (PNDES, National Economic and Social Development Plan) 2016-2020.

Ministry of Health. 2016. Politique Nationale de Nutrition (PNN, National Nutrition Policy).

Ministry of Health. 2016. Preliminary results from the 2016 SMART nutrition survey.

OEV. 2015. Strategic Evaluation. Joint Evaluation of Renewed Efforts Against Child Hunger and Under-Nutrition (REACH) 2011-2015.

Prime Minister. 2012. Politique Nationale de Protection Sociale (National Social Welfare Policy) 2013-2022.

REACH Burkina. 2015, 2016, 2017. Minutes from Inter-Network Meetings.

REACH Burkina. 2015, 2016, 2017. Minutes from UN Nutrition Focal Points.

REACH Burkina. 2015, 2016, 2017. Annual Work Plans (AWPs).

REACH Burkina. 2015, 2016, 2017. Monthly activity reports.

REACH Burkina. 2014. Country Implementation Plan (CIP).

REACH Burkina. 2017. Stratégie Nationale De Plaidoyer, Mobilisation Sociale et Communication pour le Changement Social et de Comportement en faveur de la Nutrition Au Burkina Faso (Advocacy, Social Mobilization and Communication for Social and Behavioural Change in Favour of Nutrition in Burkina Faso).

REACH Burkina. 2017. Burkina Faso Transition Plan.

REACH Secretariat. 2016. Scaling-up Nutrition Planning and Monitoring Tool (SUN PMT) Overview.

REACH Secretariat. Scaling-up Nutrition Planning and Monitoring Tool (SUN PMT) Terms of Reference.

REACH Secretariat. 2013. REACH Country Facilitator Manual (2nd Edition).

RESONUT. 2016. Analysis of PTF contributions in favour of specific and sensitive nutrition interventions.

RESONUT. 2017. Les Mécanismes de Redevabilité Existants et leur Fonctionnalité Au Burkina Faso : Proposition d’un Modèle pour les Acteurs Clés dans le Domaine de la nutrition (Existing Accountability Mechanisms and their Functionality in Burkina Faso: Proposal for a Model for Key Actors in the Field of Nutrition).

SP-CNPS. 2016. Operational Action Plan 2016-2018 for Implementation of the National Social Welfare Policy.

UNN for SUN. Frequently Asked Questions about the UN Network for SUN.

UNN for SUN. 2016. UNN Strategy 2016-2020.

WFP. 2014. Burkina Faso — Analyse Globale de la Vulnérabilité, de la Sécurité Alimentaire et de la Nutrition (AGVSAN, Burkina Faso — Global Analysis of Vulnerability, Food Security and Nutrition).

WFP. 2015. The Cost of Hunger in Burkina Faso.

WFP. 2016. Amendment to the Subsidiary Arrangement No. 11-612.

Page 69: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 61 | P a g e

Annex 7: Table — CIP Planned Outcomes, Outputs and Activity Categories

versus AWP

Outputs and deliverables as planned in CIP Outputs and deliverables in annual work plans

Outcome 1: Increased awareness of the nutrition situation and the best strategies and priorities for improvement 1.1 Multi-sector & multi-stakeholder stocktaking

Nutrition analysis including Situation Analysis Dashboard 2015 Stakeholder and nutrition action mapping exercise 2015

Validation and dissemination meeting 2015 and 2016 and modified

“scaling up CRF dialogue workshop”

Carry out a Policy and Plan Overview (see 2.1.1 Review of policies and action plans) 2015

1.2 Consensus on Core Nutrition Actions (CNA)

Technical assistance and facilitation of CNA prioritization 2015

Facilitation of targeting by intervention 2015 and 2016 and modified -

Finalization/validation CRF 1.3: Cost-benefit analysis: Investment Case (IC)

Facilitation of IC recommendations integrated into the advocacy strategy

1.4: Joint Advocacy Strategy

Development of a national communication and advocacy strategy (NCAS)

2015, 2016, 2017 and two new activities (advocacy

workshop for new government and NCAS validation workshop)

Identification of dissemination opportunities 2015

Identification of nutrition champions 2016

Facilitation of NCAS implementation at central and sub-regional levels 2015, 2016,2017

Outcome 2: Strengthened national policies and programmes that operationalize and address nutrition through a multi-sectoral approach

2.1: Integration of nutrition in government and UN strategies

Review of policies and action plans 2015

Identify opportunities to integrate nutrition into framework documents 2015, 2016

Leverage opportunities to integrate nutrition into government and UN strategies 2015, 2016, 2017 Integration of nutrition as a transversal question in the UNDAF 2016-2020 2015, 2016, 2017

2.2: Review/update of multi-sector national nutrition policy/strategy/action plan

Support revision of PNN (not planned in the CIP) 2015

Development of PSMN (2016-2020) and CRF 2015, 2016 et 2017

One new activity added in 2017: Reproduction of PNN and PSMN

Action Plan Costing 2015, 2016, 2017 and 1 new

activity: Resource Mobilization Workshop

2.3: CNA integration into the annual work plans of relevant ministries/sectors Support identification of sectoral focal points 2015, 2016

Support integration of CNAs into AWPs 2016

2.4: CNA uptake at the regional and sub-regional levels

Analysis of regional development plans 2016

Support integration and/or implementation of CNAs into regional development plans

2016, 2017 Broken into: - Exploratory analysis sharing - Consultation on integration processes

- Guide development Outcome 3: Increased human and institutional capacity on nutrition actions at all levels 3.1 Coordination capacity

Analyse existing coordination mechanisms at national and regional levels 2015

Identify human and institutional capacity development needs

Support identification/creation of a coordination mechanism 2015, 2016, 2017

Contribute to developing TOR for proposed nutrition coordination mechanisms 2015, 2016

Develop costed functional capacity development plan 2017 Contribute to reinforcing multi-sectoral coordination mechanism/regional coordination capacity

Contribute to establishment of SUN Networks 2015, 2016, 2017

3.2: Capacity development Identity capacity development needs for relevant sectors and levels 2015, 2016, 2017 Develop costed capacity development plan 2015, 2017

3.3: Orientation and training material

Development of TORs to strengthen capacity at the community level 2016

Page 70: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 62 | P a g e

Outputs and deliverables as planned in CIP Outputs and deliverables in annual work plans

Recruitment of one or more experts to develop guides and carry out the capacity development

2016

Develop a training guide for stakeholders 2016

Organize briefings in identified fields 2016

3.4: Knowledge-sharing network

Ensure dissemination of experiences/studies/research 2015, 2016, 2017

Facilitate experience sharing between country stakeholders and between countries 2015, 2016, 2017

Facilitate case study documentation and exchange with interested parties 2015, 2016, 2017

Creation and implementation of a website (not planned in the CIP) 2016, 2017 Outcome 4: Increased stakeholder effectiveness and accountability in implementing and supporting nutrition actions 4.1: Effectiveness — Implementation of a multi-sectoral M&E system and process

Implement a M&E framework for the multi-sectoral action plan 2015, 2016 Advocate for and support the integration of nutrition indicators in sectoral M&E systems

2016 ,2017

Strengthen coordination mechanism’s capacity for evaluation monitoring 2017 4.2: Accountability: results disseminated to all involved stakeholders

Support development of coverage dashboard to monitor coverage and impact indicators at the national, regional, community and sectoral levels

2016, 2017

Support a performance review of nutrition indicators

4.3 Joint UN effectiveness

Integrate nutrition into UNDAF 2016-2020 (repetition)

Support establishment of UNN with TOR and work plan

2015, 2017 broken into 3 activities: - Sharing UN agency nutrition action inventory results

- Finalisation and reproduction of the shared agenda

- Development of 2017 work plan Support establishment and functioning of the UNN by putting nutrition on EPNU meeting agenda

2015, 2016, 2017

Support the development of a UN joint strategy on nutrition 2015, report 2016 Colour coding

Service Provider Connecting countries with specialised service providers

Facilitation

Page 71: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 63 | P a g e

Annex 8: UNN/REACH Secretariat Support

REACH Secretariat

support

Manuals, tools and models

Facilitators Manual

(2013)

MNO

PPO

SUN PMT

Coverage Dashboard

Nutrition Capacity

Assessment Guidance Package (2016)

Compendium of Actions for

Nutrition (2016)

Tools to support UNN

Inventory of UN nutrition actions

UN Nutrition Agenda/Strategy

UNN Dashboard

Orientation/ briefing/

coaching of facilitators

Country visits

Financial tracking of

donors funds and

consolidation

Monthly Tele-conferences

REACH Annual

Gatherings

Page 72: Decentralized Evaluation - docs.wfp.org

REACH Evaluation – Burkina Faso Case Study 64 | P a g e

Acronyms

ANSP Africa Nutrition Security Partnership AWP Annual Work Plan CIP Country Implementation Plan CNA Core Nutrition Action CNCN Conseil National de Concertation en Nutrition (National Council for

Nutrition Coordination) CNSA Conseil National de Sécurité Alimentaire (National Food Security

Council) CRF Common Results Framework DN Direction de la Nutrition (Directorate of Nutrition) EU European Union FAO Food and Agriculture Organization of the United Nations GAC Global Affairs Canada GRET Groupe de Recherche et d’Echange Technologique (Research and

Technology Exchange Group) HKI Helen Keller International IFAD International Fund for Agricultural Development IYCF Infant and Young Child Feeding MDN Médecins du Monde (Doctors Without Borders) M&E Monitoring & Evaluation NGO Non-Governmental Organization PCD Plan communal de développement (Community Development Plan) PNDES Plan National de Développement Economique et Social (National

Economic and Social Development Plan) PNN Politique National de Nutrition (PNN, National Nutrition Policy) PSMN Plan Stratégique Multisectoriel de Nutrition (Strategic Multi-Sectoral

Nutrition Plan) PTF Partenaires Techniques et Financiers (Technical and Financial Partners) RAPNUT Réseau des Acteurs Privés pour la Nutrition (Private Sector Network) REACH Renewed Efforts Against Child Hunger and undernutrition RESONUT Réseau de la Société Civile pour la Nutrition (Civil Society Network for

Nutrition) SCPN Stratégie de Communication et de Plaidoyer Nationale (SCPN, National

Advocacy and Communication Strategy SMART Standardized Monitoring and Assessment of Relief and Transition SP-CNPS Secrétariat Permanent du Conseil National de Protection Sociale

(Permanent Secretariat of the National Social Welfare Council) SUN Scaling Up Nutrition SWOT Strengths, Weaknesses, Opportunities, Threats TOR Terms of Reference UN United Nations UNDAF United Nations Development Assistance Framework UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund WFP World Food Programme WHO World Health Organization USAID United States Agency for International Development

Page 73: Decentralized Evaluation - docs.wfp.org

Renewed Efforts Against Child Hunger and undernutrition https://www.unnetworkforsun.org/reach

Ro

me, D

ecem

ber 2

01

7, D

E/

20

17

/R

EA

CH

02

1