inter-agency humanitarian evaluation of …...2019/10/23 · • the proposed outline of the...

INTER-AGENCY HUMANITARIAN EVALUATION OF THE RESPONSE TO CYCLONE IDAI IN MOZAMBIQUE

Inception Report

September 2019 Evaluation Team: Jock Baker (Team Leader)

Tristi Nichols

Felisberto Afonso

Pity Estajo

With contributions from: Dr. Luis Artur (Team Leader – Household Survey)

Emanuel Malai

Rogério Sithole

Evaluation Management: Ali Buzurukov (OCHA)

Jane Mwangi (UNICEF)

Robert Mccouch (WHO)

Sergio Lenci (WFP)

Elma Balic (IOM)

Inter-Agency Humanitarian Evaluation Steering Group

IAHE Mozambique – Inception Report | i

Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations

Contents

Contents ................................................................................................................................................ i

Acronyms ............................................................................................................................................. ii

1 Introduction ............................................................................................................................... 1

2 Purpose, Scope, Use, Timeframe & Target Audience ........................................................... 1

2.1. Purpose & Use ............................................................................................................... 1

2.2. Scope ............................................................................................................................. 1

2.3. Stakeholder Analysis ..................................................................................................... 3

2.4. IAHE Background & Strategy ........................................................................................ 5

2.5. Documents Consulted ................................................................................................... 6

2.6 Country Context ............................................................................................................. 7

3 Methodology .............................................................................................................................. 8

3.1. IAHE Design .................................................................................................................. 8

3.2. Overall Approach ........................................................................................................... 8

3.3. Evaluation Questions ................................................................................................... 10

3.4. Assessing Achievements and Gaps ............................................................................ 12

3.5. Overall Field Travel Schedule ..................................................................................... 18

3.6. Key Milestones for the Study ....................................................................................... 21

3.7. Potential risks for the study ......................................................................................... 21

3.8. Limitations .................................................................................................................... 22

4 Quality Assurance .................................................................................................................. 23

4.1. Quality Assurance ........................................................................................................ 23

4.2. Ethical Considerations ................................................................................................. 23

5 Deliverables ............................................................................................................................. 24

5.1. Report Outline .............................................................................................................. 24

6 Measures of Success for this evaluation ............................................................................. 25

Annex A – Aerial Maps of Affected Areas ...................................................................................... 26

Annex B – Evaluation Matrix ............................................................................................................ 28

Annex C - Instruments for the Review ............................................................................................ 35

Annex D – Community Focus Group Discussion Guide ............................................................... 39

Annex E – Evaluation Workplan ...................................................................................................... 52

Annex F – Letter of Introduction ...................................................................................................... 53

Annex G – Informed Consent Form................................................................................................. 54

Annex H –Data Analysis Plan .......................................................................................................... 55

IAHE Mozambique – Inception Report | ii


Acronyms AAP Accountability to Affected People

COSACA emergency and humanitarian consortium

DDR Demobilization, Disarmament and Reintegration

DPOs Disabled Peoples Organizations

ERC Emergency Relief Coordinator

ERIC Ethical Research Involving Children

FGDs Focus Group Discussions

GBV Gender-Based Violence

GoM Government of Mozambique

HR Human Resources

HQ Headquarters

HC Humanitarian Coordinator

HCT Humanitarian Country Team

HH Household

HPC Humanitarian Program Cycle

HRP Humanitarian Response Plan

IAHE Inter-Agency Humanitarian Evaluation

IAHE AG Inter-Agency Humanitarian Evaluation Advisory Group

IAHE MG Inter-Agency Humanitarian Evaluation Management Group

IAHE SG Inter-Agency Humanitarian Evaluation Steering Group

IASC Inter-Agency Standing Committee

IASC EDG Inter-Agency Standing Committee Emergency Directors Group

IASC WG Inter-Agency Standing Committee Working Group IT Information Technology

INGC Instituto Nacional de Gestão de Calamidades

KIIs Key Informant Interviews

M&E Monitoring and Evaluation

MIRA Multi-sector Initial Rapid Assessment

MRP Management Response Plan

NFI Non-Food Item

NGO Non-government Organization

OPR Operational Peer Review

OCHA United Nations Office for the Coordination of Humanitarian Affairs

PREPOC Post-Cyclone IDAI Recovery Program

RC Resident Coordinator

SMART Specific, Measurable, Attainable, Relevant and Timely.

TOR Terms of Reference

UN United Nations

UNEG United Nations Evaluation Group

WASH Water, sanitation, and hygiene

IAHE Mozambique – Inception Report | 1


Introduction 1 This inception report outlines the understanding of the Evaluation Team regarding the international humanitarian

response in 2019 the Republique of Mozambique based on the Terms of Reference (TOR). During the inception

phase the Evaluation Team undertook document reviews and preliminary discussions with key stakeholders.

This report presents:

• the purpose and scope;

• target audience;

• objectives of the evaluation;

• analytical framework;

• methodology (including instruments to be used for the community household survey);

• timeline; and

• the proposed outline of the evaluation report.

Purpose, Scope, Use, Timeframe & Target Audience

2.1. Purpose & Use

2 Based on the TOR, it is understood that the specific purpose of the Inter-agency Humanitarian Evaluation (IAHE)

is to:

• provide an independent assessment of the extent to which planned collective objectives set out in the

Humanitarian Response Plan (HRP)1 to respond to the needs and concerns of affected people in

Mozambique have been met; and

• assess the extent to which Inter-Agency Standing Committee (IASC) mechanisms, including the scale-up

activation and Humanitarian Programme Cycle (HPC), have successfully supported the response.

3 Under both objectives, the evaluation will identify lessons from the response and system-wide scale-up activation

and will generate recommendations for future responses and preparedness. This IAHE will emphasize both

learning and accountability. Given that it will assess the first inter-agency response involving a scale-up

activation, it will thus provide valuable lessons for future scale-up activations, in particular in response to the

sudden-onset of natural disasters.

2.2. Scope

4 Defining the substantive areas to be covered in the evaluation

The IAHE will primarily focus on the inter-agency response to Cyclone Idai, with Cyclone Kenneth considered as

a factor that significantly influenced the course of the system-wide scale-up response due to various

considerations including the relative scale of Kenneth, time/resource constraints and security conditions that

would make it difficult to obtain a representative perspective from affected communities. Cyclone Idai put a strain

on the capacity of the humanitarian system and Cyclone Kenneth’s added to the demands. This situation offers

a potential learning opportunity about how agencies could respond to two simultaneous emergencies. This part

of the scope would draw from secondary data, since time constraints and access to the affected communities

probably would not yield a representative sample. This evaluation’s recommendations will also be carefully

1 Along with other plans developed and used by the HCT during the course of the response.



framed – taking in consideration the complexity and multiple demands. This approach has been validated with

key stakeholders during inception interviews, the Advisory Group in Maputo, and with senior staff who had been

directly involved in both responses.

5 The IAHE will examine the effectiveness of the scale-up activation and the global support provided to the

humanitarian response of the Humanitarian Country Team (HCT) and GoM. The following references will be

used as the main references against which to ascertain achievements:

• Revised version of the HRP version of March 2019;

• The IASC Scale-Up Response Benchmarks and Transition Strategy for Mozambique – April 2019 (dated 16

April 2019);

• Cyclone Idai – Response Strategy – Updated Discussion Paper (dated 24 May 2019); and

• Review of the Humanitarian System-Wide Scale-Up by the Mozambique HCT Benchmarks & Transition Plan

– Two Month Update (dated 24 May 2019).2,3

6 Geographic Areas

The evaluation will cover the flood-affected areas of Mozambique, or Sofála, Maníca, Quelimane, Tête, and to a

limited extent, Cabo Delgado (see figure 1), and this forms the basis of the overall selection criteria.

Figure 1: Map of Affected Areas4

2 While these inter-agency frameworks and strategies were recently secured, there may be additional frameworks which will be provided to the Evaluation Team in the field. 3 The IAHE will not evaluate the impact of humanitarian response covered by the original HRP launched in November 2018, which focused primarily on drought. 4 Source: Humanitarian Response Plan (Revised following Cyclones Idai and Kenneth, May 2019): Nov 2018 – Jun 2019

https://www.unocha.org/southern-and-eastern-africa-rosea/cyclones-idai-and-kenneth



7 Project Selection

• The selection of projects, partners and sites for visits will be determined on the basis of the following factors

to obtain a representative view of the program:

• Volume, budget and scale of project activities;

• Possibility to observe both ongoing and completed projects (to provide an independent assessment of

activities, outputs and outcomes);

• Locations with a concentration of various activities for different affected populations;

• Examples of both successful and less successful interventions;

• Direct access to male and female representatives of beneficiary communities;

• Communities which have been assisted by multiple agencies; and

• Logistically reachable within the limited timeframe.

8 Period Covered

While the temporal scope of the evaluation starts on 14 March, when Cyclone Idai made landfall in Mozambique

until the time that the Evaluation Team visits the region (1 September). The IAHE will be carried out from August

through December 2019. The field visit for the evaluation will take place during the transition phase after the

emergency phase came to an end at the end of June and before large scale recovery programs have launched.

9 Target Groups

In line with the principle of accountability to affected people (AAP), the IAHE will assess how the targeting of

beneficiaries focused on the most vulnerable groups, as well the extent of the coverage in relation to the total

population in need.

2.3. Stakeholder Analysis

10 The primary intended users of the results of this evaluation include stakeholders from the international

humanitarian agencies (HQ, regional, and country, levels), GoM, donors (including the private sector), peer

agencies (e.g., civil society), and affected communities, including direct beneficiaries. The table below outlines

the roles in the response and assumed interests in the IAHE.



Table 1: Key Stakeholders of the IAHE, Their Role(s), and Assumed Interests in the Evaluation

Stakeholder Group Role(s) in the Response Assumed interest in/Benefits from the IAHE

International

humanitarian agencies

and their local partners

operating in country

Implementers and supporters of

GoM’s efforts

As the primary stakeholders, these agencies are

assumed to be interested in emerging lessons at

both the strategic and operational levels, as well

as mutual accountability and AAP. Will

participate in Advisory Group meetings as well.

Headquarters (HQ) of

international


Source of policy guidance,

administrative and regulatory

frameworks, guidance and

support, surge/standby capacity,

accountability frameworks and

oversight (varies by organization)

As another primary stakeholder group, this

group is assumed to be interested in learning so

as to better implement its role in this response

and in future responses, as well as in

accountability.

Regional offices of

international


Government5 Coordination and support to

national priorities

As another primary stakeholder, the GoM’s

interest is assumed to be focused on holding the

humanitarian agencies to account for their

results in the response – and in learning how it

might better exercise its role and execute its

response in the future. Will participate in

Advisory Group meetings as well.

Donors

Financial and in-kind support to

the response itself, and to these

same organizations’ humanitarian

functions at the regional and HQ

levels

Donors’ interest is assumed to be in ensuring

that funded agencies have exercised fiduciary

responsibility over funds received, achieved

results, and critically self-reflected on their

performance through this evaluation and other

exercises. Private Sector

In-kind support to the response

itself at the local level from

foundations.

Peer agencies (e.g.,

civil society)

Lesson learned in the

coordination, partnerships

established, among other areas

Dissemination of lessons learned that could

potentially be used to better establish its own

contributions to the response.

Affected communities,

including direct

beneficiaries

Ultimate beneficiaries targeted by

humanitarian operations

This group can be categorised into village-level

leadership and residents, and both are

assumed to be interested in better

understanding how the humanitarian system

works (e.g., criteria for receiving assistance and

needs met), what is expected from them during

and after the experience (e.g., returning to

normalcy as soon as possible), and how they

ought to be treated (with respect, dignity, no

sexual harassment).

5 This includes: INGC, Instituto Nacional de Estatística (INÉ) [Statistics Institute]; Instituto Nacional de Acçao Social (INAS); Ministério da Saúde (Min. Health); Governo Da Província De Sofála; Governo Da Província de Maníca; Governo Da Província de Tête; Governo Da Província de Quelimane; Governo Da Província de Cabo Delgado.



2.4. IAHE Background & Strategy

11 The two tropical cyclones resulted in loss of life, injury and illness, destruction of livelihoods and infrastructure.

Following the cyclones and flooding caused, an estimated 2.2 million people required urgent assistance (1.85

million people due to Cyclone Idai and 374,000 people due to Cyclone Kenneth), on top of 815,000 people

already in need, because of an ongoing drought. Cyclone Idai hit the central region of Mozambique (Sofála,

including its capital Beira; Maníca; Zambézia, and Tête), leaving an estimated 3,000 km2 of land submerged and

damaging more than 240,000 houses. Subsequently, Cyclone Kenneth struck the northern region, including

Cabo Delgado, destroying more than 45,300 houses.

12 On 22 March 2019, following consultations with the Inter-Agency Standing Committee (IASC) Principals and the

GoM, the Emergency Relief Coordinator (ERC) formally declared a Humanitarian System-Wide Scale-Up

Activation (henceforth referred to as ‘scale-up activation’) for a period of three months, or until 22 June. This was

the first time such a designation had been made following the IASC decision in November 2018 to replace

previous “L3” system-wide activation definitions and procedures.6

13 The scale-up activation is an inter-agency mobilization mechanism in response to a sudden onset and/or rapidly

deteriorating humanitarian situation in a given country, including at the subnational level, where capacity to lead,

coordinate and deliver humanitarian assistance does not match the scale, complexity and urgency of the crisis.

14 The procedure activates mechanisms and tools to: (a) ensure that the IASC system delivers effectively in support

of national authorities and existing capacities and monitors its own performance, (b) ensure that adequate

capacities and tools for empowered leadership and coordination of the humanitarian system are in place, and

(c) engage IASC member organizations and Global Cluster Lead Agencies to put in place the required systems

and to mobilize the required resources to contribute to the response as per their respective mandates.

15 In line with the new protocols, a scale-up activation requires that an Operational Peer Review (OPR) of the

response be undertaken within five months of the crisis, and that an IAHE be conducted 9-12 months after the

declaration. OPRs, designed to be brief and using a collaborative process, are undertaken by peers. They are

not intended to measure results or the impact of the response. IAHEs, conducted at a later stage of the

humanitarian response, are independent and have the purpose of not only promoting accountability to donors

and affected populations but also encouraging strategic learning for the humanitarian system. The OPR in

Mozambique was conducted from 26-31 May 2019 and the report released on 24 June 2019.7 For this IAHE, it

is anticipated that the final report will be made available approximately seven months after the scale-up

declaration.

6 For further information, the scale-up protocols are located here. (see link) 7 IASC (2019) Operational Peer Review, Mozambique Cyclone Idai Response.

https://interagencystandingcommittee.org/iasc-transformative-agenda/content/iasc-humanitarian-system-wide-scale-protocols-released



2.5. Documents Consulted

16 The team has so far reviewed around 80 documents from multiple sources, of which the main reference

documents for this Inception Report include those indicated in Table 2.

Table 2: List of Key Documents Consulted in the Inception Phase

Specific Documents with Hyperlinks Where Possible

Co

un

try

Level

1. The GoM Humanitarian Plan [Post Disaster Needs Assessment];8

2. HRP version November 2018 and HRP version of March 2019;

3. Relevant plans and reports;9 and

4. Please note that cluster-level planning documents and monitoring data are not yet available.

Asse

ss

-

men

ts

5. Integrated Food Security Classification, ACAPS (for Mozambique), OCHA10, REACH (for Mozambique), Multi-sector Initial Rapid Assessment (MIRA), and aerial maps (see annex A).

HQ

Level 6. TOR for this Evaluation;

7. IAHE Governance Documents, including Guidelines from the following website (see link);

8. 2016 United Nations Evaluation Group (UNEG) Norms and Standards.

Pre

vio

us E

va

luati

on

s

9. IASC (2019) Operational Peer Review (OPR), Mozambique Cyclone Idai Response;

10. Particip (2018) “Evaluation of DG ECHO's disaster preparedness and DRR actions in Southern Afric& Indian Ocean”, Contract n°: ECHO/ADM/BUD/2011/01205;

11. International Federation of Red Cross and Red Crescent Societies (IFRC) (2019) Real-Time aluation, Mozambique: Tropical Cyclones Idai and Kenneth;11

12. Mozambique Red Cross Society and the International Federation of Red Cross and Red Crescent Societies (IFRC) (2015) The Final Evaluation of MDRMZ011;

13. Independent Review of COSACA Management of cyclone response in Mozambique (2019) Forthcoming.12

14. Other relevant references.13

Oth

er

15. IASC Operational Peer Review of the Mozambique Cyclone Idai Response. June 2019

16. Active Learning Network for Accountability and Performance (ALNAP) (2006)Evaluatinghumanitarian action using the OECD-DAC criteria An ALNAP guide for humanitarian agencies;

17. Organisation for Economic Co-operation and Development’s (OECD) Development Assistance Committee (DAC) criteria; and

18. The SPHERE Handbook (2018 edition).

8 INGC (2019) Post disaster needs assessment. 9 República de Moçambique Conselho de Minístros (2019) Plano Anual de Contingência 2019; The IASC Scale-Up Response Benchmarks and Transition Strategy for Mozambique – April 2019 (dated 16 April 2019); Cyclone Idai – Response Strategy – Updated Discussion Paper (dated 24 May 2019); Review of the Humanitarian System-Wide Scale-Up by the Mozambique HCT Benchmarks & Transition Plan – Two Month Update (dated 24 May 2019). 10 UN OCHA Situation Report No. 14 as of 15 April 2019; UN OCHA, Mozambique: Cyclone Idai & Floods Situation Report No. 22 As of 20 May 2019 UN OCHA Flash Appeals #1, #2, #3, #4, among other documents. 11 The authors of this report are: Juergen Hoegl, team leader (external) Sammy Fwaga, IFRC Africa Regional Office, Marie Manrique, IFRC Country Cluster Support Team, and Lima Victoria Stodart, IFRC Secretariat. 12 COSACA consists of the international aid organizations CARE International, Oxfam, and Save the Children (link here). 13 OCHA has been compiling a list of other planned Idai-related evaluations.

https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/pdna_report_mozambique_cyclone_idai.pdf


http://www.ipcinfo.org/ipcinfo-website/events/events-details/en/c/1151944/

https://www.acaps.org/country/mozambique/crisis/complex-crisis

https://www.reach-initiative.org/where-we-work/mozambique/

https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/mozambique_mra_20190426_en_def.pdf

https://www.humanitarianresponse.info/en/programme-cycle/space/evaluation

http://www.unevaluation.org/document/detail/1914

https://ec.europa.eu/echo/files/evaluation/2012/DIPECHO_south_africa_indian_ocean.pdf

https://ec.europa.eu/echo/files/evaluation/2012/DIPECHO_south_africa_indian_ocean.pdf

https://www.alnap.org/jobs-opportunities/independent-review-of-cosaca-management-of-cyclone-response-in-mozambique?mc_cid=16c25a5215&mc_eid=fe55982dbe

https://www.alnap.org/system/files/content/resource/files/main/eha-2006.pdf



https://spherestandards.org/handbook-2018/

http://www.care.org.mz/cosaca.php



17 Based on an initial scoping exercise and after reviewing these key resources, the Evaluation Team has been

able to: (i) identify the areas that can realistically be examined during this study; and (ii) develop a detailed

methodology, taking into account what has already been proposed in the TOR.

2.6 Country Context

18 Elements of a Complex Emergency (2018–2019)

The pre-existing socio-economic conditions in Mozambique show that Cyclone Idai took place in a context of

high vulnerability, creating conditions that have exacerbated poverty levels among affected populations.

Mozambique ranks as a country with one of the lowest human development index (HDI) in the world (0.437), at

180th among 189 countries.14 The agricultural sector accounts for 25 percent of GDP and employs 71 percent

of the labor force. Almost 94 percent of the poor are primarily engaged in agriculture.15 Before Tropical Cyclone

Idai, the country was already facing high levels of food insecurity. In 2017 and 2018, Mozambique’s Technical

Secretariat for Food Security and Nutrition (SETSAN) had conducted several studies documenting severe food

insecurity throughout the country due to the onset of drought conditions.16

19 Political dynamics were also operating in the background, compounding the effects of Cyclone Idai. For example,

preparations for the general elections to be held on 15 October 201917, a peace process between the

Government and Renamo, and the process of Demobilization, Disarmament and Reintegration (DDR) are

currently taking place. The nature and national scope of these processes, as well as the time limits for achieving

them, make them especially sensitive in the post-disaster context left by Idai. These are all vital for the

consolidation of peace and democracy in Mozambique.

20 GoM-Led Immediate Response to Cyclones (2019)

In the immediate aftermath of the storm, rapid aerial assessments were carried out to gather a general overview

of the scale and severity of the crisis, information from which was used to inform immediate operational decision-

making regarding priority areas for response.18 The GoM immediately implemented a series of actions in

response to the unfolding disaster, including the:

• declaration of a National State of Emergency on 19th March 2019;

• implementation of immediate search-and-rescue operations, and provision of humanitarian aid;

• appeal for international assistance;

• establishment of a post-Cyclone Idai Post-Disaster Recovery Programme (PREPOC) on 26 March;

• approval the TOR of the PREPOC on April 2, 2019;

• creation of the Post-Cyclone Reconstruction Office on April 09, 2019;

• extension of the Scope Assessment for Cabo Delgado and Nampula Cyclone following Cyclone Kenneth, on

April 30, 2019;

• training of staff of ministries / sectors central, provincial, district and municipal levels on the post-disaster

needs assessment (PDNA) methodology and calculation of damage, loss and recovery needs;

• conduct of the PDNA, approved by the Council of Ministers on May 7th; and

• development of the Final Report of the PDNA, which was approved by the Council of Ministers.19

14 UNDP. Human Development Indices and Indicators 2018 Statistical Update. 2018. 15 World Bank. Mozambique Poverty Assessment. 2018, page 25. 16 Ministério da Agricultura e Segurança Alimentar: Relatório Final da Avaliação Sazonal De Nutrição De Março-Abríl de 2018; Acute Food Insecurity CPI Analysis Report April 2018: Resultados das análises de IPC conduzidas em 36 distritos no período de Abril à Maio de 2018; Relatório da Monitoria da Insegurança Alimentar Aguda de Outubro e Novembro de 2017. 17 The general elections in October will elect a new President, Members of Parliament and for the first time, and the governors of 10 provinces. 18 INGC, ACAPS, IFRC, MapAction, OCHA, REACH Initiative, United Nations Disaster Assessment and Coordination. Multi-Sectoral Rapid Assessment Post-Cyclone Idai: 14 Districts in Sofála and Maníca Provinces, Mozambique 1-17 April 2019. 19 Government of Mozambique (2019) Post-Disaster Needs Assessment (PDNA), page 41. It is not yet known the exact date when the Council of Ministers approved the Final PDNA report, but the Evaluation Team will seek this information in the field.

http://www.setsan.gov.mz/

http://www.setsan.gov.mz/

http://hdr.undp.org/sites/default/files/2018_human_development_statistical_update.pdf

http://documents.worldbank.org/curated/en/377881540320229995/pdf/131218-WP-P162550-PUBLIC-FRI-OCT-26-7AM-DC-DIGITAL-Mozambique-Poverty-Assessment-2018.pdf

http://www.setsan.gov.mz/wp-content/uploads/2018/07/Relat%C3%B3rio-da-Avalia%C3%A7%C3%A3o-Sazonal-de-Nutri%C3%A7%C3%A3o-2018.pdf

http://www.setsan.gov.mz/wp-content/uploads/2018/07/Relatorio-de-IPC-SA-Aguda.pdf

http://www.setsan.gov.mz/wp-content/uploads/2018/07/Relatorio-de-IPC-SA-Aguda.pdf

http://www.setsan.gov.mz/wp-content/uploads/2018/01/Relatorio_AVASAN_Nov2017_29Dez2017.pdf



An international appeal for assistance in the amount of USD$281.7 million and revised the country’s humanitarian

response plan. The HCT also commissioned a multi-sectoral rapid assessment to inform the humanitarian

response.

Methodology

3.1. IAHE Design

21 The methodology for this IAHE has been designed with the aim of answering three main questions within the

limits imposed by time, resources and other constraints:

1) To what extent were priority humanitarian needs of affected communities met?

2) Given this was a nationally led response, what was the contribution of the international humanitarian system?

3) How effectively did the international humanitarian system work together to support the response and meet

priority needs of affected populations?

22 The results of this evaluation should thus provide a useful overall perspective but should not be expected to

provide detailed guidance on operational performance of individual agencies,20 apart from where agencies have

significant lead coordination roles. The aim is to examine how the international humanitarian system delivered

priority needs to affected populations as one system. Looking at how to better harness the support that is

provided from all partners is one issue that has been raised as very important in inception interviews.

3.2. Overall Approach

23 As shown in figure 2 below, the quantitative and qualitative information sources of the IAHE include: (1) structured

interviews with village level leadership; (2) a community HH survey; (3) community focus group discussions

(FGDs); (4) semi-structured individual and group interviews with humanitarian actors; and (5) document and data

analysis. The green colored icons reflect the work of the Core Evaluation Team, and the other data collection

activities are to be undertaken by the Survey Team.21 Section 3.3 discusses team composition.

24 While the qualitative data from individual and group interviews with humanitarian actors22/village level leadership

will be purposively gathered, often times using a snowball sampling method,23 the quantitative sampling process

for the community HH survey is based on a methodological approach defined by Glenn Israel (2000).24

Specifically, probabilistic methods will ensure that findings are at the 95 percent of confidence level and five

percent of precision. A total of 400 HH will be sampled in the three regions affected by Idai Cyclone (Maníca,

Sofála, and Tête) during the evaluation.25

20 During the Mozambique HCT meeting on 02 July 2019, the Team Leader outlined the limitations of the IAHE noting that the IAHE will not a suitable instrument for capturing operational lessons learned for individual agencies. Agencies were encouraged to undertake After Action Reviews (AAR) or similar reflective exercises, so that they could use the IAHE results to understand their individual contributions and at the same time provide the IAHE team with useful secondary data. 21 The Survey Team will be led by Dr. Luis Artur from Universidade de Eduardo Mondlane. 22 These interviews would be in Maníca, Sofála, Tête, and Zambezia. To the extent possible, the Evaluation Team will conduct skype/telephone interviews with humanitarian actors based in Cabo Delgado. 23 Noy, C. (2008). Sampling knowledge: the hermeneutics of snowball sampling in qualitative research. International Journal of Social Research Methodology, 11(4), 327-344. The definition on page 330 is: “A sampling procedure may be defined as snowball sampling when the researcher accesses informants through contact information that is provided by other informants. This process is, by necessity, repetitive: informants refer the researcher to other informants, who are contacted by the researcher and then refer her or him to yet other informants, and so on.” 24 Israel, Glenn D. (2000). Determining Simple Size. University of Florida, IFAS Extension.

25 This should give a confidence level of 95% for an affected population in excess of 1 million

https://www.ssoar.info/ssoar/bitstream/handle/document/5386/ssoar-ijsrm-2008-4-noy-sampling_knowledge_the_hermeneutics_of.pdf?sequence=1&isAllowed=y&lnkname=ssoar-ijsrm-2008-4-noy-sampling_knowledge_the_hermeneutics_of.pdf

https://www.ssoar.info/ssoar/bitstream/handle/document/5386/ssoar-ijsrm-2008-4-noy-sampling_knowledge_the_hermeneutics_of.pdf?sequence=1&isAllowed=y&lnkname=ssoar-ijsrm-2008-4-noy-sampling_knowledge_the_hermeneutics_of.pdf



Figure 2: IAHE Methods

Quantitative and Qualitative Data Sources

Community Level Interviews Community HH Survey

Service delivery stakeholders at community levels. Interview take place where survey is conducted in Maníca, Sofála and Tête

A total of 400 face-to-face surveys from a random sample of HHs in Maníca, Sofála and Tête

Community Focus Group Discussions Agency & Partner Interviews

Multiple FDGs in each province which satisfy specific criteria. Take place in the same villages where HH survey is conducted

Service delivery stakeholders at provincial, district and regional levels (Government, UN, Donors, NGOs) in Maníca, Sofála and Tête, Zambezua and Cabo Delgado

Document Analysis Secondary Data Analysis

• Analysis of existing & other relevant evaluations

• Humanitarian requirements & standards

• Meeting minutes

• Cluster-Level Strategy Plans

• Data for needs assessments

• Monitoring data

25 Each team member will be assigned focal point responsibilities for specific themes and issues to ensure an

adequate coverage of documentation, analysis, documentation on key issues emerging from interviews and

FDGs, while also periodically taking advantage of opportunities for validation by key stakeholders throughout the

entire process. Although evaluative methods will be employed to ensure an appropriate level of rigor and

credibility, specific emphasis will be given to maximizing learning, sharing, and emphasizing the need to utilize

results. The final part of the approach undertaken by the Evaluation Team is the systematic comparison,

triangulation, and analysis of data collected, which ensures data integrity and factual accuracy.

26 Contextual and Cross-Cutting Issues to be Taken into Account

The 2030 Agenda of “Leaving no one behind” draws attention to review factors that contribute to inequality (e.g.,

gender, equity, human rights, inclusion, and disempowerment linked to climate change-related risks). 26 Part of

the analysis in this evaluation will be dedicated to better understanding the implementing mechanisms designed

(and used) to empower and actively encourage the participation of all vulnerable groups in relevant decision-

making processes.

27 The National Context

Although this was a government-led response, emphasis on capacity building and national ownership are still

prevailing issues which will also be assessed.

26 A/RES/70/1 of 21 October 2015. Transforming our world: the 2030 Agenda for Sustainable Development.



28 Donors

As Mozambique is heavily affected by climate related risks, the extent to which there are devoted resources to

address this permeating risk will be further explored.

3.3. Evaluation Questions27

29 According to IAHE guidelines (page 15), the evaluations draw upon evaluation criteria from:

• the UNEG Norms and Standards;

• OECD / DAC criteria for development program, including: (1) relevance/appropriateness; (2) efficiency;

(3) effectiveness; (4) impact; (5) sustainability; and

• the ALNAP criteria for the evaluation of humanitarian action, which are: (1)

coherence/coordination/partnerships; (2) coverage; and (3) connectedness.

30 Understandably, not all of these criteria are applicable in this IAHE. The following seven evaluation criteria will

be examined: (1) Appropriateness; (2) Effectiveness; (3) Connectedness; (4) Coverage; (5) Partnerships; (6)

Localization; and (7) Coordination. Sustainability will not be assessed. Table 3 outlines the evaluation

questions under each criterion as well as sub-questions for context and cross-cutting issues.28 Using these

standard evaluation criteria is also important to ensure comparability of the evaluation findings and

recommendations between different IAHEs and other evaluations.

Table 3: Evaluation Questions and Sub-Questions

Evaluation Criteria Evaluation Questions & Sub-questions

1. Appropriateness

To what extent have the objectives set out in the HRP, other strategic documents, and

other joint planning documents been based on identified needs, including those of the

most vulnerable groups affected by the crisis?

1.1. To what extent were efforts undertaken to support disaster preparedness before

the event?29

1.2. After the event, how were the needs assessments undertaken, and to what

extent was the information used for response planning?

1.3. To what extent were assessment processes coordinated?

1.4. To what extent do the planning documents reflect identified needs and priorities

of affected people, including those of vulnerable groups?

1.5. To what extent did the mechanisms actively encourage the participation of

vulnerable groups in decision‐making processes, including in environmental

matters?30

27 The assessment criteria is based on definitions from: ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria An ALNAP guide for humanitarian agencies; and the IAHE Guidelines from the following website (see link). 28 These sub-questions were designed to fall in line with the 2030 Agenda “Leave no one behind”. 29 In the discussion various members of the Advisory Group in Maputo highlighted the importance of reviewing and recognizing early actions (prior to landfall), as well as preparedness issues (pre-position of relief items), which played an important role in the response. As the evaluation team will review program participant participation into the design of response, it is important to understand that design of prepositioned kits, including verifications of which were undertaken during the immediate response. Moreover, was there targeting vs. blanket distributions needs considered? 30 Appropriateness of humanitarian assistance is linked to engaging affected populations and other vulnerable stakeholders in decision-making. Two official records were consulted from the Economic and Social Council (the Committee for Development Policy): (1) E/2018/33 (2018) Supplement No. 13; and (2) E/2019/33 (2019) Supplement No. 13



https://www.humanitarianresponse.info/en/programme-cycle/space/evaluation

https://undocs.org/en/E/2018/33

https://undocs.org/en/E/2019/33



Evaluation Criteria Evaluation Questions & Sub-questions

2. Effectiveness

2.1. To what extent were the targets articulated in the HRP achieved (in terms of

assistance delivery)?

2.2. To what extent were the targets effective in meeting the needs of the most

vulnerable? [protection question]

2.3. To what extent has the scale-up activation supported the response (as stated

under ‘Purpose’ above)?

2.4. Did the response have any unintended, positive or negative effects on affected

communities?

2.5. To what extent were strategies, approaches or methodologies executed to

protect vulnerable groups, particularly at the community level? Are any new

approaches needed?

3. Connectedness

How was the IASC humanitarian system’s emergency assistance for people affected by

the crisis linked to longer-term recovery, resilience and development efforts? What, if

any, were the challenges in implementing this linkage?

3.1. What are the interventions designed to support the transition from recovery to development?

3.2. To what extent has the response contributed to restoring livelihoods (and returning to normalcy) at the community level, and particularly for vulnerable groups? [protection question]

3.3. Is there any evidence that the response helped affected people cope better with subsequent or future cyclones?

4. Coverage

To what extent were different groups of affected people in all locations affected by

Cyclone Idai reached with humanitarian emergency aid?

4.1. To what extent was protection integrated throughout the response efforts for

vulnerable groups in all locations?

4.2. How did the funding and donor priorities influence the overall coverage?

5. Partnerships To what extent have adequate partnerships been established (with international,

national and local stakeholders) to deliver assistance to affected people?

6. Localization

To what extent and in what ways have national and local stakeholders (GoM, IINGOs,

NNGOs, private sector, military) been involved in international coordination

mechanisms, and have their capacities and systems to respond in the future been

strengthened through the response?

6.1. How did the response strengthen the capacities of partners to contribute in

cross-cutting areas?

7. Coordination

How well-coordinated was the assistance, avoiding duplication of assistance and gaps?

7.1. How effective were coordination mechanisms at the strategic, inter-cluster, and

cluster/sector levels?

7.2. Is there any evidence that coordination helped to avoid duplications?31

7.3. How effective was the resource mobilization effort in raising sufficient, timely

and long-term funding? How did the timing of the funding and donor priorities

influence the overall response, and in what form of assistance (financial and in-

kind assistance) did donors provide assistance? 32

7.4. What major gaps in human and financial resources to carry out activities in

cross-cutting areas were identified? [protection question]

31 What are the challenges in the surge management among UN agencies, including the occurrence of “double-hatting”? 32 How (well) did the CERF mechanism function and is this the most effective way to release funds only via UN agencies?



3.4. Assessing Achievements and Gaps

31 Assessing the UN’s Contribution to the Government-Led Response

Reflecting on assumptions that underlie the inter-agency response in the Mozambique and taking into account

the IAHE Impact Pathway from the Inter-Agency Humanitarian Evaluations Process Guidelines, the Evaluation

Team has reconstructed a Humanitarian Response Pathway which better reflects the elements of this IAHE.

(See Table 4.)

Table 4: Humanitarian Response Pathway for the Inter-Agency Response in Mozambique

OUTPUTS33 SUPPORT CORE

RESPONSIBILITIES

INTENDED RESULTS OF

SCALE-UP ACTIVATION

DESIRED LONGER-

TERM IMPACT

International inter agency humanitarian system functions effectively.

International response complements and supports the government response.

Ten IASC Clusters in 10 hardest-hit districts.34

Humanitarian access secured for all, including the most vulnerable groups within the population in need.

Affected people live in enhanced safety and dignity with better prospects of thriving as agents of their own destinies.

Coordination

The lead agency is the National Disasters Management Institute (Portuguese acronym is INGC)

CCCM-Camp Coordination & Camp Management

Livelihoods protected and restored

Logistics Appropriate

response

Education Connectedness and coordination between humanitarian actors

Accurate Assessment

ECT-Emergency Tele-communications

Planning Food Security

Relevant and effective coverage

Partnerships Health

Funding & Fundraising

Nutrition

Learning Protection/ Child Protection/ GBV

Adequate capacity strengthening partnership with INGC.

Cross-cutting issues

Shelter/NFI

WASH

32 International support to the GoM is realized through activities and outputs. These are the key components of the

contribution during an emergency. The core responsibilities are from the ten IASC Clusters in ten of the hardest-

hit districts. Assuming that IF the efforts of these clusters are appropriate, effectively delivered (effectiveness,

coordination, and partnerships) met longer term needs (connectedness), had adequate coverage, and

33 This Response Pathway was adapted from UN OCHA. 2018. Inter-Agency Humanitarian Evaluations Process Guidelines (page 16). 34 Source: Review of the Humanitarian System-Wide Scale-up By the Mozambique HCT Benchmarks & Transition Plan – Three Month Update.



strengthened capacity (localization), THEN the specific intended results and desired longer-term impact are likely

to have occurred. These assumptions are linear, which may not be realistic for a humanitarian effort. However,

this Humanitarian Response Pathway (model) serves to promote a shared understanding of the response logic

model and guide the Evaluation Team’s thinking throughout the process.

33 Assessing Appropriateness

According to the TOR, appropriateness focuses on better understanding how the contributions of the international

humanitarian system supported the national humanitarian response, and in particular, the extent to which

activities were adapted to needs.35 The Evaluation Team will, therefore, review the objectives set out in the HRP

and determine the extent to which (and how) objectives were based on identified needs. The findings

emerging from this exercise will depend on a degree of coherence and accuracy between the data collected from

official sources of assessed needs, planning documents, and monitoring data.36 Although all of the needs

assessments are not yet available, an attempt will be made to secure additional official reports, such as

estimations of those who were exposed to multiple hazards (i.e., loss of life, flooding and loss of food security

and livelihood, forced resettlement, etc.). Through a HH Community Survey, satisfaction levels of the assistance

received as well as perspectives about the usefulness of the support received will also be explored.

34 Assessing Effectiveness

A review of effectiveness entails the examination of the extent to which the overall international response

achieved its objectives. Implicit within the criteria of effectiveness is the timeliness of the delivery of assistance.

While IASC documents will provide the Evaluation Team with the general scope of the prioritized and most time-

critical life-saving activities undertaken, including the efforts of the ten IASC Clusters,37 learning about whether

(and which) targeted results were (and were not) met in the different sectors – and why – will also shed

light on the precise and tangible benefits that were made available

immediately after Cyclone Idai. The study’s community HH community

survey results, also used for triangulation, will give further insights about

what was and was not achieved at the community level. The data from

FGDs will illuminate the overall experience of receiving assistance,

particularly the extent to which the beneficiaries with disabilities or specific

vulnerabilities were given sufficient attention. Such groups include the

elderly with mobility challenges, small children and children with disabilities,

pregnant women, girls who may have been subject to sexual abuse,

isolated communities, among others.38

35 Addressing the Prevention of Sexual Exploitation and Abuse (PSEA) was a

top priority in this evaluation effort. In ensuring that effective systems were

in place for preventing and responding to acts of sexual exploitation and

abuse, the Evaluation Team will proactively pursue beneficiary experiences

and examine if strict adherence to Standard Operating Procedures took

place with identified cases, while respecting “do no harm” protection

principles. The cases from the Linha Verde hotline (Figure 3) will be an

initial data source to examine.

35 UN OCHA, MOZAMBIQUE: Cyclone Idai and Floods Situation Report No. 22 As of 20 May 2019. 36 The key planning documents to be used as a reference are: (1) INGC (2019) Post disaster needs assessment; (2) HCT plans; (3) Cluster-level strategies (not yet available); (3) HRP version of March 2019; and (4) Monitoring data (not yet available) for the re)assessment of needs (i.e., through on-going monitoring activities, sitrep). This also includes the efforts which took place for the Scale up which took place before the scale down (of the drought). The alignment of the PDM align with the PDR will also be taken into account. 37 Information sourced from: Review of the Humanitarian System-Wide Scale-up By the Mozambique HCT Benchmarks & Transition Plan – Three Month Update. The ten clusters are: 1. CCCM-Camp Coordination & Camp Management; 2. Logistics; 3. Education; 4. ECT-Emergency Tele-communications; 5. Food Security; 6. Health; 7. Nutrition; 8. Protection/ Child Protection/ GBV; 9. Shelter/NFI; and 10. WASH. 38 UN OCHA, Mozambique: Cyclone Idai & Floods Situation Report No. 22 As of 20 May 2019.

Figure 3: Linha Verde Hotline




36 Assessing Connectedness

Connectedness refers to the need to ensure that activities of a short-term emergency nature are carried out in a

context that takes longer-term problems into account.39 Although the general scope of activities undertaken by

the ten IASC Clusters will provide an overview of the UN’s contribution to the government-led response, the

Evaluation Team will also learn about how the response met longer-term needs (e.g., material changes in their

lives from assistance), if the support was perceived as timely, and what type of assistance aided the recovery

and reconstruction process.

37 Assessing Coverage

Coverage refers to the need to reach major populations facing life-threatening suffering wherever they are. This

IAHE is unique that it will examine an emergency response that involved two different tropical cyclones (above

Category 2) which made landfall in two different time periods and in two different regions of the country. As both

of the cyclones made landfall during the critical harvest period, one of the compounded effects was that families

who lost their livelihood (houses and crops) were left without the prospect of another full harvest until March

2020.40 The ensuing lessons will show when decisions to scale up were made and how the international

humanitarian system supported the GoM in mobilizing and extending its reach to deliver assistance in multiple

locations. Further, the community HH community survey results about the extent to which the assistance

received was timely in different sectors and by region will provide additional insight. Survey results will also

indicate the extent to which assistance provision was tailored to the needs of the most vulnerable (also termed

social coverage).

38 Assessing Partnerships and Localization

This crux of this criteria will be to understand how the contribution of international humanitarian agencies

supported the government to alleviate the suffering of affected populations. The partnership component will be

examined at multiple levels, including with international, national and local stakeholders. There is a need to

better understand the strength of these partnerships, including whether they were pre-existing with mutual trust

and respect, newly established, strategic, operational, and what the stakeholder perceptions are regarding their

effectiveness. At the community level, FGDs will examine: (a) how communities were consulted; (b) how families

who received assistance were selected and what role they had in this decision-making process; and (c) how

support affected different members of the family, including those with family members who are disabled or

vulnerable. All of this information is intended to inform whether the partnerships could adequately deliver

assistance. Similarly, understanding the capacity needs to effectively partner with the GoM is another area that

will be explored. The share of resources that civil society received (if any at all) to engage in the response,

disaggregating support to international and national NGOs, is another area that will be examined.

39 Assessing Coordination

The degree to which the coordination mechanisms work coherently and efficiently will be examined, notably

between the HCT and the GoM at country and provincial levels, in order to understand the extent to which

the international humanitarian system complemented and supported this government-led humanitarian

response. In addition, coordination will be assessed in the following other areas:

1. among the HCT members at country level;

2. between and among the HCT and non-HCT and non-GoM partners (e.g., national and International NGOs

operating within the clusters, representatives from the private sector with in kind donations and

individuals/groups who functioned within clusters);

39 The assessment criteria is based on definitions from: ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria An ALNAP guide for humanitarian agencies; and the IAHE Guidelines. 40 UN OCHA (2019) Humanitarian Response Plan: Mozambique 2018-2019 (Revised following Cyclones Idai and Kenneth, May 2019), page 5.





3. regional level coordination for HCT members and INGOs which have a regional presence (CARE, Save the

Children, IFRC), particularly as it relates to surge management;

4. coordination at the HQ level; and

5. between the IASC and Emergency Management Group and the HCT, particularly as it relates to raising funds

and reporting updates and results to higher decision-making structures (e.g., Emergency Response Task

Force, IASC Principals, Emergency Director’s Group, OPAG).

40 Assessing Cross-Cutting Issues

Gender, equity, human rights, inclusion, and environmental issues are integrated throughout the evaluation

criteria of appropriateness, effectiveness, coverage, localization, and coordination (see table 2). Nonetheless,

it is recognized that the impact of the cyclone has a differential impact on women and girls. One report noted

that there was an elevated risk of Gender-Based Violence (GBV) during the humanitarian emergency, as there

is “greater exposure of women and girls to distant and unsafe locations, such as water collection points, sanitation

facilities, and health centers.”41 Moreover, households with women and girls, and particularly women-headed

households, are being subjected to sexual exploitation by men in power or with access to relief products and

resources. The Evaluation Team plans to assess the extent to which communities perceived that the support

and assistance provided contributed to the medium- and longer-term needs of vulnerable groups (i.e., elderly,

orphaned children). INGC drafted and adopted a Gender Strategic Plan (2016-2020) which aims to reduce the

vulnerability of women in relation to disasters and mitigate impacts on sociocultural, economic and political rights

arising from gender inequalities. This plan provided for the establishment of a gender unit to ensure the

implementation and monitoring of the actions undertaken by the INGC. The team will consult with the gender

unit as well as the Ministry of Women and Social Action.

41 It cannot be emphasized more that protection issues will be thoroughly examined. Due to the movement of

populations to safer areas, many children were separated from their families. The number of orphaned children

is currently unknown.42

The Evaluation Team plans to analyze data by gender, inclusion, and other relevant drivers of inequality, in line

with: (1) the UNEG Guidance on Integrating Human Rights and Gender Equality in Evaluation; (2) the UN

System-Wide Action Plan (UN-SWAP) on gender equality; (3) the IASC Gender Equality Policy Statement; and

(4) the IASC Guidelines on the Inclusion of Persons with Disabilities in Humanitarian Action.43

Please see annex B for the Evaluation Matrix which summarizes the evaluation questions, indicators, data

sources, and reliability of information to be gathered.

42 Stakeholder Analysis

Interviews in the field (including Maputo), the region (Kenya) and at a HQ level will allow the Evaluation Team to

gather different perspectives from various stakeholders. (See the table 1 for a description of key stakeholders.)

Due to time constraints, there will be a need to carefully select key informants for this evaluation (both

stakeholders and non-stakeholders), so as to secure a balanced sample. Sampling techniques are described in

section 3.2.

43 Observations

It is anticipated that the Evaluation Team will engage in field observations when traveling to the provinces and

districts. While observations are an important part of any evaluation process, a structured observation

process/plan (with an established methodology) has not been integrated into this evaluation due to resource

constraints.

41 INGC (2019) Post disaster needs assessment, page 20. The need to collect water more frequently and from greater distances and the use of public toilets and shared latrines increases the risk for women and girls. With the destruction of health facilities, pregnant women have limited access to safe deliveries. 42 INGC (2019) Post disaster needs assessment, page 170. 43 The Convention on the Rights of Persons with Disabilities (CRPD) will be consulted.

https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html



44 Interviews

A proposed interview/focus group guides include questions that are aligned with the objectives in the TOR and

initial scoping discussions (see annex C and D). At provincial and district levels, consultations will focus on

understanding how the international humanitarian community and GoM worked together to select communities,

the degree of partnerships required to carry out the work, and perceptions about whether the assistance that

was delivered was indeed lifesaving. Any challenges throughout the process will be discussed with the intent to

encourage learning. Maputo-based stakeholders, and those located in Nairobi/South Africa, will provide the

Evaluation Team with an overview of the strategic planning process, the conditions under which the humanitarian

response was managed, how national leadership led the effort, and any challenges experienced in the process.

Interviews will take roughly one hour.

45 Community Focus Groups

In each selected district, a number of FGDs44 will be undertaken with community residents. Each group will be

composed of approximately 6-8 persons. The intent is to have:

1. Group conversations with people that despite being affected had no access to humanitarian aid;

2. Group conversations with displaced and resettled people;

3. Group conversations with those that remained in same place after the event; and

4. Group conversations with those that were resettled but then returned back.

For each group conversation, there will be five separate categories: a group of all males, a group of all females,

a group of elderly, a group of youth, and a group of persons with disabilities. In essence, these consultations will

focus on understanding how assistance was delivered, examining the "how" or the overall process of receiving

assistance. The focus guide is presented in annex C. The intent is to gather perceptions about the relevance,

effectiveness, and timeliness of the early warning system, the response to Idai, and assistance received. The

process of recovery and reconstruction has particular emphasis. Given the stratified purposive sample of

selected participants, the study will uncover variations in attitudes and behaviours, depending up on the level of

vulnerability (e.g., seniors, persons with disabilities), location (three different provinces), and severity of suffering

resulting from the cyclone. FGDs will take roughly 1.5 hours.

46 HH Community Survey

The Community HH survey will be conducted through direct interviews with the head of household or an adult

family member appointed as the head of the household. Similar to the FGDs, HHs will be selected from the

following most affected districts:

• in Sofála province (namely Beira, Buzi, Dondo and Nhamatanda);

• 2 in Maníca province (namely Sussundenga and Gondola); and

• 1 Tete province (Tsangano).

The Evaluation Report will include details about how HHs were selected at the district and village levels. The

steps undertaken to minimize selection bias, increase validity, and strengthen reliability will be highlighted to

maintain transparency. The community HH survey (see annex D) focuses on affected persons interaction with

the aid workers, timeliness/relevance of and satisfaction with assistance received, material changes, impartiality,

perceptions of how they were treated, and clarity about the mechanisms available for lodging complaints. The

survey will take between 30 to 45 minutes to administer.

Due to the election campaign process that began on September 1st, it was decided in consultation with the IAHE

Management Group that the survey team’s work has been postponed until after the national elections on October

15th.

44 The precise number of FGDs has not yet been determined but will be made available after the field visit.



47 Guidance and Protocols

To ensure that the FGDs and survey are systematically executed as planned, the steps included are:

• The instruments have been developed in Portuguese and translated into English.

• To ensure consistency throughout the data collection team members in each province, enumerators will be

trained by the Team Leader of the Survey Team in ethics and confidentiality, data collection, using the digital

platform (Kobo toolbox), household selection techniques, interview techniques, data collection and

submission.45 The training, which will last approximately one week in each affected district, follows the best

practice of experiential learning.

• Prior to implementation, the instruments and equipment (tablets) will be pre-tested. As part of the training,

the Quantitative Team will choose one community affected by the cyclones, located near the place of training.

After implementing the survey, the results will be downloaded from the platform on real time, and then the

team can check inconsistence(s), and engage in focused discussions about lessons learned.

• During interviews and FDGs, the Survey Team will clearly explain that participation is voluntary and that

there are no consequences for refusing to take part in the evaluation or to answer specific questions. This

will be particularly important for conversations with affected people. Informed consent will be sought from

each person. Please see annex G for the Informed Consent Form.

• The standard of confidentiality will be strictly followed. For example, for all team members, the primary data

collected, e.g. interview notes and survey forms, will not be shared with anyone outside the Evaluation Team.

48 Data Analysis

The following steps will be undertaken to analyze the data collected in the field:

• Data analysis and preliminary findings; establishment of timelines to identify key events and key decision-

making points.

• Triangulation of findings to determine high, medium and low levels of convergence.

• Before completing the field visit, the review team will meet to compare and triangulate findings and

conclusions.

• Reports, presentations, and briefings related to evaluation findings will not attribute specific findings or

opinions to one particular individual or a specific organization. Only aggregated analyses of the collected

information will be presented.

49 Reporting and Validation of Findings and Recommendations

The following steps will be taken to validate and report the evaluation findings and conclusions:

• Debriefing session for international agencies based in Beira during the data collection process, so that

Provincial-level stakeholders may preview to preliminary findings;

• Debriefing session for the IAHE Advisory Group, and other invited stakeholders as appropriate, held at the

end of the field visit to help validate main findings, support learning and participation;

• Rapid review of an initial draft evaluation report by a small number of reviewers from the IAHE MG and

Advisor Group;

• Circulation of a revised draft report after consideration of feedback;

• Validation workshop(s) in Mozambique when participants will have an opportunity of assessing the relevance

and achievability of draft recommendations;

• Consideration of consolidated feedback by the Evaluation Team; and

• Submission of final report (in English).

45 There are six enumerators for this project and two teams (of three enumerators each). All of these individuals also speak the local languages.



50 Document research

Document research will be carried out in three stages by the review team. The inception phase presented key

documents consulted as well as data from interviews from initial briefing discussions. During the implementation

phase, however, relevant data collected during interviews, field observations and desk research will be collated

and placed in an evidence matrix designed to organize the data along lines of questioning. The evidence matrix

tool will facilitate subsequent analysis of evidence collected to identify trends and areas of controversy.

Frequencies of identified themes will also be assessed using the evidence matrix. This will enable the

significance and weight of the issue to be determined. Issues identified as significantly linked to conclusions will

be correlated to the location and clusters from which these came. This will allow the review team to link people’s

perceptions of perceived success and hindering factors to specific areas in the organization that will help reduce

bias. Each team member has been assigned responsibility for specific themes based on their individual area of

expertise to help ensure that specific issues are not overlooked.

51 Triangulation of data

Triangulation is a core principle in

mixed-method data collection, as it

ensures that the results are linked up

into a coherent and credible evidence

base (see figure 4). This review will

mainly rely on:

• Source triangulation. Evaluation

team members will compare

information from different sources,

e.g. at different management levels

and different functional units.

• Method triangulation. Team

members will compare information

collected by different methods (e.g.,

quantitative, qualitative, secondary

data from documents).

• Research triangulation. Comparison

and collation of information

collected by different team

members during the course of their

research.

• Comparator agency triangulation. Contrast and compare the operations, technical support and cost

structures of selected agencies for comparison purposes.

• Data from each source can then be placed into the evaluation framework to assist in identifying key findings,

conclusions, and interpreting results.

• Annex H includes a Data Analysis Plan which outlines the precise steps planned for both quantitative and

qualitative data.

3.5. Overall Field Travel Schedule

52 Three weeks have been allocated in the workplan for the field visit. Based on this allocation, it is proposed to

devote most of the first two weeks to field travel, visiting project sites and conducting stakeholder interviews, and

then spending the last week in Maputo to carry out interviews with stakeholders operating at more strategic levels

(i.e., government, UN, donors, civil society, NGOs). Two days of meetings in Kenya and Johannesburg/Pretoria

Figure 4: Process of data interpretation and development of evaluation

findings.



will be dedicated to consultations followed by debriefing. The following itinerary is proposed to make the most

use of the limited time available to undertake this evaluation:

• Sun, Sep 1: Internal Evaluation Team members arrive in Maputo

• Mon, Sep 2: Internal Evaluation Team meeting in Maputo

• Tue, Sep 3: Orientation meetings in Maputo

• Wed, Sep 4: team members arrive in Beira (courtesy meetings, interviews)

• Thu, Sep 5 thru Sat, Sep 7: two sub-teams conduct informant interviews in Sofála

• Mon, Sep 9 thru Fri, Sep 13: two sub-teams key informant interviews in Maníca, Tête and Zambézia

• Fri, Sep 13: Debriefing in Beira of preliminary results from Sofála and Maníca

• Sat, Sep 14: travel to Maputo

• Mon, Sep 16 - Thu, Sep 19 (morning): key informant interviews in Maputo

• Thu, Sep (afternoon): internal team meeting to prepare debrief

• Fri, Sep 20: Debrief in Maputo and internal team review of preliminary results

• Mon through Wed, Sep 23-25: key informant interviews in regional offices in Nairobi and

Johannesburg/Pretoria

The team, comprised on a Core Team and a Survey Team, are responsible for the execution of this evaluation

as shown in the table below. The Team Leader has overall responsibility for coordination and management of

the evaluation. The Core Team includes a senior evaluator and two national evaluators. The Survey Team is

comprised of a Survey Team Leader, two team members, and six enumerators.

Table 5 Roles and Responsibilities of Evaluation Team Members

TE

AM

LE

AD

ER

Team Leader Jock Baker

The Team Leader will be primarily responsible for submitting all deliverables and in

charge of directing the process, including providing overall leadership in analysis and

drafting of the inception and evaluation reports. He coordinates the work of the

Evaluation Team, distributing tasks and ensuring that each member maximizes their

contributions to the evaluation in a timely fashion and will be the focal point for

communications between the management committee and the other team members. He

will also be the thematic focal point for assessing coordination, partnerships, coverage,

[part of] effectiveness, and connectedness.

CO

RE

TE

AM

Senior Evaluator Tristi Nichols

The primary roles of this team member will be to:

• Control the quality of the work;

• Assure the application of an adequate methodology;

• Assure the delivery of deliverables within target deadlines;

• Interact and coordinate with the Core Team members and Survey Team members

during the evaluation process;

• Support the Survey Team, providing technical assistance in instrument pretesting,

data collection and analysis, and reporting.

• Serve as the thematic focal point for assessing appropriateness, [part of]

effectiveness, localization, and cross-cutting issues. Data analysis and report

drafting to be undertaken in these areas, among others.



CO

RE

TE

AM

National Evaluator Pity Estajo

• Engage in all interviews;

• Provide insightful knowledge about the partnerships with and among GoM, NGOs,

UN, and donors, particularly in the areas of education, WASH, Health, Nutrition,

Protection/ Child Protection/ GBV;

• Notetaking and data analysis; and

• Data analysis and report drafting to complement sections on appropriateness, [part

of] effectiveness, localization, and cross-cutting issues.

National Evaluator Felisberto Afonso

• Engage in all interviews;

• Provide insightful knowledge about the partnerships with and among GoM, NGOs,

UN, and donors, particularly in the areas of CCCM-Camp Coordination (& Camp

Management, Logistics (Surge, HR, & Supply), ECT-Emergency Tele-

communications, Shelter/NFI (including seed distributions), Food Security and

livelihoods, among others.

• Notetaking and data analysis; and

• Data analysis and report drafting to complement sections on coordination,

partnerships, coverage, [part of] effectiveness, and connectedness.

SU

RV

EY

TE

AM

Survey Team Leader Dr. Luis Artur

• Lead all the necessary communication and coordination with local government and

relevant entities for the smooth realization of the work;

• Controls the quality of the work;

• Assures the utilization of an adequate methodology;

• Assures the quality and delivery of deliverables within target deadlines;

• Interact and coordinate with the Core Team members during all the process;

• Co-design methodology and research tools;

• Responsible for submitting survey deliverables, including the survey report, to the

Management Group for their review; and

• Leads the conduct of FGDs.

Emanuel Malai

• Co-Design methodology and research tools;

• Design of data collection instruments;

• Train enumerators;

• Monitoring and quality control of data collection;

• Implements the FGDs and manages the survey team in Tête;

• Supports the drafting/reviewing of the reports.

Rogério Sithole

• Co-Design methodology and research tools;

• Design of data collection instruments;

• Train enumerators;

• Monitoring and quality control of data collection;

• Implements the FGDs and manages the survey team in Beira and Sofála; and

• Supports the drafting/reviewing of all sections of the reports.

As the elections delayed the planned work of the survey team until after October 15 th, the Senior Evaluator will

return to Mozambique to participate in pretesting all tools. These tools include the survey and the FGDs.



3.6. Key Milestones for the Study46

Table 6 Key milestones for the study

Milestone Date(s) Responsibility Participants

Final draft version of Inception Report returned from management group

Thu, Aug 27th Team Leader Evaluation Team and management group

Final report including all inputs from the Management and Advisory groups

Fri, Sep 12th Team Leader Evaluation Team

Core Team Field Travel Sep 1-25 Evaluation Team

Presentation of preliminary findings and emerging conclusions to to the IAHE Advisory Group

Sep 20th Team Leader Evaluation Team

Senior Evaluator travel to provide technical assistance to the Survey Team for instrument pretesting and data collection

End of October Senior Evaluator Senior Evaluator and Survey Team

Household survey team fieldwork Oct 28 – Nov 10

Team Leader, HH survey

Evaluation Team

First draft of survey report from Survey Team Nov 26th Team Leader, HH survey

Evaluation Team

Final version of survey report from Survey Team

Nov 29th Team Leader, HH survey

Evaluation Team

Draft evaluation report for wide circulation December 6th Team Leader Evaluation Team and management group

After completion of draft report, workshop in Maputo takes place with IAHE Advisory Group, HCT, and the IAHE Management Group

December 12th Team Leader and OCHA

Team Leader

Deadline for written feedback December 20th Team Leader and OCHA

All levels

Final version of report Jan 22nd Team Leader Evaluation Team

Presentation for IASC Operations, Policy and Advocacy Group and to the IASC Emergency Directors Group in Geneva and/or New York.

TBA Team Leader and OCHA

Evaluation Team

The Evaluation Workplan (annex E) includes specific activities.

3.7. Potential risks for the study

53 The presidential and parliamentarian elections are the biggest risk identified thus far, since most if not all of the

GoM stakeholders will be heavily preoccupied with campaigning activities. The implication is that there will be

very little time dedicated for the IAHE consultations, and this very important stakeholder group could risk being

inadequately represented in the process. This would be an unfortunate limitation, as the response was

government-led. Mitigating this risk requires quick deployment of the Evaluation Team without delay, so that all

data queries and consultations are undertaken at least one month before the election (i.e. before 15 September).

46 See also the attached workplan in the Annex.



3.8. Limitations

54 The Evaluation Team foresees the following limitations that could affect the achievement of the TOR:

• Given the relatively broad scope of the activities and the remoteness of many intervention sites and time and

capacity constraints, the coverage of the evaluation will be limited. This includes the response to cyclone

Kenneth. Due to various constraints (time, resources, security) it will not be possible for either the evaluation

team or the survey team to visit the area and data will be collected exclusively via secondary data and remote

interviews.

• Similarly, after considering resource constraints, logistics challenges and the relatively small affected

population in Zambezia the preferred approach is to carry out the household survey remotely in this province.

• The national election process in Mozambique has already affected the evaluation timeline so that the

household survey will only take place after the election on October 15th. Although post-election violence is

not seen as likely, it is nevertheless a risk that could potentially cause further delays.

• Assessment of effectiveness and coverage will also be dependent on the availability of suitable data,

including the availability of comparative data from the GoM and peer agencies.

• The high turnover of senior staff, since the beginning of the current strategy, may limit the amount of relevant

information available to the team.

• As Cyclone Kenneth also influenced the response, depleting already limited human and financial resources,

the recommendations that must ensue from this evaluation ought to have this narrow scope in mind.



Quality Assurance

4.1. Quality Assurance

55 The Evaluation Team will aim to ensure that the evaluation is conducted according to principles are inspired by

the United Nations Evaluation Group (UNEG) ethical principles, guidelines, standards (specifically Norm 6 and

standard 3.2.) and the OCHA Quality Assurance System for Evaluations. The evaluation’s design, planning and

implementation processes are inherently quality oriented, covering appropriate methodologies for data collection,

analysis and interpretation. The Evaluation Team includes national evaluators and will seek the participation of

the GoM throughout the evaluation process, as appropriate and possible. National and sub-national level disaster

risk management institutions and local actors have been identified and will be interviewed. In addition, the core

team will observe some of the surveys and FDGs conducted by the Survey Team. This would have the dual

advantage of: (1) fulfilling the quality assurance role; and (2) giving the core team some direct insights from a

community level.

56 To avoid any misinterpretations of the evaluation objective, the team will obtain a letter from OCHA explaining

the evaluation (see annex G). This letter will be presented to provincial and district leaders and will also be used

by both the Core and Survey Teams. The Management Group and Advisory Group also add layers of quality

assurance.

4.2. Ethical Considerations

57 Compensating for potential biases

During the orientation phase at the start of the field work, any potential biases of Evaluation Team members will

be raised so that they could be compensated for when planning interviews, conducting analysis, developing

conclusions and recommendations to help ensure impartiality.

58 Special Considerations

Protection: In line with the ALNAP Guide: Evaluation of Protection in Humanitarian Action47 and the IAHE

Guidelines, the evaluation will consider the extent to which the inter-agency humanitarian response has

mainstreamed protection issues and has considered the protection risks particularly affecting the most vulnerable

people. In the event of interviews or discussions take place with children, the Ethical Research Involving Children

(ERIC) approach48 will for followed.

47 https://www.alnap.org/help-library/alnap-guide-evaluation-of-protection-in-humanitarian-action 48 From Ethical Research Involving Children (ERIC) online compendium, p. 18. Graham, A., Powell, M. Taylor, N., Anderson, D. and Fitzgerald,

R. (2013). Florence: UNICEF Office of Research - Innocenti.

https://www.alnap.org/help-library/alnap-guide-evaluation-of-protection-in-humanitarian-action



Deliverables

59 The evaluation will generate the following outputs:

• Reports: A draft and a final version of the Inception Report (with relevant annexes). This will be followed by

an inclusive feedback process on Draft Report; and

• The Draft and Final Evaluation Report: will be submitted by the Team Leader to the Evaluation Manager.49

60 Presentations: The Evaluation Team will conduct the following presentations:

• At the end of the field visit, the Evaluation Team will share preliminary findings and emerging conclusions

with the IAHE Advisory Group and the IAHE Management Group. The brief will help clarify issues and outline

any expected pending actions from any stakeholders, as relevant, as well as discuss next steps. This

workshop will include a lessons-learned session on the IAHE itself so that stakeholders at country-level can

provide their feedback on the evaluation process and how it can be improved in future. The evaluation Team

will, to the extent possible, present to a stakeholder audience that includes NGOs/INGOs representing

affected communities, private sector representatives, and donors. If possible, this oral briefing (or relevant

parts) can be video recorded and presented for alternative ways of dissemination, such as websites, social

media.

• Soon after circulation of the draft evaluation report, there will be a day’s validation workshop in Maputo that

will help to assess the relevance and achievability of the recommendations and provide an opportunity to

provide additional relevant inputs.

• Following the workshop, a national consultant on the team will visit the capital cities in each of the four

districts affected by cyclone Idai to facilitate an interactive session with local government authorities and

humanitarian agencies to share evaluation results and collect additional feedback. The Team Leader of the

survey team may also join depending on his availability.

• Once the evaluation is completed, presentations of the main findings and recommendations will be made

available to various fora as decided by OCHA and the IAHE Management and Steering Groups. It is

understood that members of the Evaluation Team may be requested to assist with these presentations.

5.1. Report Outline

61 To ensure that the Evaluation Report directly addresses the objectives defined in the TOR, it is planned that the

report will be structured according to the lines of questioning described in the Methodology Section, i.e.:

• Title page

Front Section

• Title page

• Table of contents

• List of acronyms

• Preface

• Acknowledgements

• Executive summary of 1000 to 1500 words

Main Report (it is anticipated that the entire report including the Front Section, will be approximately 40-50 pages of text (excluding annexes):

• Introduction and Background

o Purpose, scope, rationale, target audience

49 This group includes: IOM, OCHA (chair), UNICEF, WFP and WHO.



o and expected use of the evaluation

o Summary background of the Intervention Strategy

o “Mapping” to situate the response within the broader national and regional context

• Methodology, including a description of limitations and constraints

• Separate sections corresponding to specific question areas of the evaluation, with relevant

findings and conclusions as appropriate. Case studies may be used to illustrate specific

findings.

• Conclusions (overall analysis and conclusions based on findings)

• Recommendations targeted specifically at:

o The HCT and

o The IASC at a regional and HQ level.

Annexes (may add others, if relevant):

1. Household survey report

2. TOR for the IAHE

3. Field visit itinerary

4. Interview Guides

5. List of Persons Interviewed

6. List of Key Reference Documents

Measures of Success for this evaluation

62 The Evaluation Team proposes that the following criteria should be used to assess the overall quality and utility

of the study process:

• Engage with an adequate and gender-balanced number of key stakeholders during the data collection and

analysis process in order to promote ownership and utilization of evaluation results.

• Generate robust findings that can be clearly linked to evidence through the quality-assurance process

adopted, with appropriate detail provided where there appear to be divergent views.

• Based on specific questions outlined in the TOR, establish clear links between the study findings, conclusions

and “SMART” recommendations targeted at specific stakeholder groups.

• Execution of the above activities in an independent fashion, so as to ensure the credibility of the findings and

recommendations while respecting relevant professional codes of codes of conduct to gain the respect and

confidence of the commissioning agency.



Annex A – Aerial Maps of Affected Areas Figure 5: Extent of Flooding in Sofála: 21 March 2019

Source: UN OCHA Briefing PowerPoint dated 22/03/2019.

https://www.humanitarianresponse.info/en/operations/mozambique/document/mozambique-situation-briefing-cyclone-idai22-march-2019



Figure 6: Satellite Imagery as of 14 and 20 March – Sofála Province, Mozambique.

Source: UN OCHA Briefing PowerPoint dated 22/03/2019.

https://www.humanitarianresponse.info/en/operations/mozambique/document/mozambique-situation-briefing-cyclone-idai22-march-2019



Annex B – Evaluation Matrix

Evaluation Questions (ToR) Indicators Data Sources Availability and

Reliability of Data

1. Appropriateness

To what extent have the objectives set out in the HRP been based on identified needs, including those of the most vulnerable groups affected by the crisis?50

• Qualitative indicator: degree of coherence and accuracy between the data collected from official sources and planning documents51;

• Number of affected people stating that the response met their most urgent needs; and

• Number of persons from vulnerable groups surveyed stating that the response met their most urgent needs.

Secondary data collected: Official and relief actors’ databases of population in need assessments in three Provinces.52 Document reviews of UN humanitarian actor’s internal information.53 Household (HH) survey results regarding the satisfaction of support received and the usefulness of the support received.54, 55

Unknown.

1.1. To what extent were efforts undertaken to support disaster preparedness before the event? 1.2. After the event, how were the needs assessments undertaken, and to what extent was the information used for response planning? 1.3. To what extent were assessment processes coordinated? 1.4. To what extent do the planning documents reflect identified needs and priorities of affected people, including those of vulnerable groups? [protection question] 1.5. To what extent did the mechanisms actively encourage the

UNOCHA data is reliable.

High. Survey data is reliable.

50 The three strategic objectives from the HRP version of March 2019 are: (1) Provide immediate life-saving and life-sustaining assistance to the population affected by severe food insecurity; (2) Provide immediate life-saving assistance to the population affected by the impact and flooding caused by cyclones Idai and Kenneth; and (3) Support the restoration of livelihoods and strengthen resilience of climate-affected population. 51 The key planning documents to be used as a reference are: (1) the GoM plan; (2) HCT plans; (3) Cluster-based plans; (3) HRP version of March 2019; and (4) other relevant plans. 52 The three provinces are Manica, Nampula, and Sofala. Preliminary reviews suggest that needs assessment data may be limited or insufficient. 53 UNOCHA Mozambique: Cyclone Idai Flash Update Numbers 1, 2, 3, 4 among other documents. 54 The survey is undertaken by Quantitative Team led by Dr. Luis Artur from Universidade de Eduardo Mondlane. 55 This information will likely be disaggregated by gender, region/province, and type of vulnerability.






Reliability of Data

participation of vulnerable groups in decision‐making processes, including in environmental matters?

2. Effectiveness56

2.1 To what extent were the targets articulated in the HRP achieved (in terms of assistance delivery)? 2.2 To what extent were the targets effective in meeting the needs of the most vulnerable? [protection question]

• Qualitative indicator: degree of coherence and accuracy between the data collected from official sources and planning documents;

Document review and analysis of past evaluations, monitoring reports, internal information, & needs assessments; Community HH survey results used for triangulation; and Interview data from KIIs in Maputo and in the field also used for triangulation.

Moderate. This information will not likely be consistent across all of the agencies. It is for this reason that other data sources will be used for triangulation purposes.

56 This evaluation question has three parts, and so it was split up into three sections to illustrate exactly how each part of the question would be addressed.




Reliability of Data

• Number of documents emphasizing the need to consider cross-cutting issues related to the most vulnerable;

• Number of vulnerable persons supported, and cases from the Linha Verde hotline, reached at “1458”;

• Number of respondents stating positive satisfaction levels and positive perceptions of impartiality (includes treatment with respect);

• Qualitative indicator from FDGs about if and how needs of family members with disabilities (including the elderly with mobility problems and children with disabilities) were given sufficient attention.

• Qualitative indicator: stakeholder perceptions about utility and effectiveness of targets.

Document analysis; Community HH survey results; FGD data; and Data from KIIs in Maputo and in the field.

High.

2.3 To what extent has the scale-up activation supported the response (as stated under ‘Purpose’)?57 2.4 Did the response have any unintended, positive or negative effects on affected communities? 2.5 To what extent were strategies, approaches or methodologies executed to protect vulnerable groups, particularly at

• Proportion of respondents stating positive satisfaction with:

o the support received; and o the usefulness of the support received.

• Qualitative indicator: stakeholder perceptions about the effectiveness of the scale-up activation and Humanitarian Programme Cycle (HPC).

Community HH survey data; Document review & analysis; and KIIs in Maputo and in the field.

High.

57 Recalling that one of the purposes of the evaluation will be to assess the extent to which IASC mechanisms, including the scale-up activation and Humanitarian Programme Cycle (HPC), have successfully supported the response.




Reliability of Data

the community level? Are any new approaches needed? [protection question]

3. Connectedness

How was the IASC humanitarian system’s emergency assistance for people affected by the crisis linked to longer-term recovery, resilience and development efforts? What, if any, were the challenges in implementing this linkage?

• Proportion of surveyed respondents with positive statements for:

o The response met longer-term needs (e.g., material changes in their lives from assistance).

o The support received was timely;

• Qualitative indicator from FGDs: Perceptions of the process and challenges of returning to normalcy. 58

Community HH survey data; Data from FDGs; and KIIs in Maputo and in the field for triangulation.

High.

3.1 What are the interventions designed to support the transition from recovery to development?

3.2 To what extent has the response contributed to restoring livelihoods (and returning to normalcy) at the community level, and particularly for vulnerable groups? [protection question]

3.3 Is there any evidence that the response helped affected people cope better with subsequent or future cyclones?

4. Coverage

To what extent were different groups of affected people in all locations affected by Cyclone Idai reached with humanitarian emergency aid?

• Qualitative indicator: degree of coherence and accuracy between the data collected from official sources and planning documents in different regions;

• Proportion of surveyed respondents with positive results that:

Document review & analysis of meeting minutes;

High.

58 The focus group question is: To what extent were you assisted in the recovery and reconstruction process? Discuss the response based on the 5 W (What? Where? When? Why? Who does?) And how?




Reliability of Data

1.1 To what extent was protection integrated throughout the response efforts for vulnerable groups in all locations? [protection question]

1.2 How did the funding and donor priorities influence the overall coverage?

o The support received was timely; o The response met their longer-term

needs (e.g., material changes in their lives from assistance).

Community HH survey data: cross-tabulations to generate region-based findings;59 and KIIs in Maputo and in the field for triangulation.

5. Partnerships

To what extent have adequate partnerships been established (with international, national and local stakeholders) to deliver assistance to affected people?

• Qualitative indicator: Stakeholder reflections on: o the type of partnerships that were

established; o how relevant the partnership was; o what could be improved to solidify

partnerships in the future.

• Qualitative indicator: Positive perceptions about involvement in the response.

• Proportion of resources received from civil society to engage in the response.

Interview data from KIIs in Maputo and in the field regarding the strength of partnerships at all levels. Budget information from civil society (as compared to the HRP), disaggregating support to international and national NGOs.

Moderate. Perceptions are likely to differ by stakeholder group.

Qualitative Indicator: Positive responses about the level of participation experienced with aid delivery system, examining the process of.

• How community groups were consulted;

• How criteria for receiving assistance was explained;

• How the assistance was delivered and how often?

• How support affected different members of the family.

Data from FGDs. Moderate to high.

6. Localization

59 A crosstabulation of satisfaction levels in different sectors and by region will demonstrate what different groups received emergency assistance.




Reliability of Data

To what extent and in what ways have national and local stakeholders (GoM, IINGOs, NNGOs, private sector, military) been involved in international coordination mechanisms, and have their capacities and systems to respond in the future been strengthened through the response?

• Qualitative indicator: Stakeholder reflections on: o Type of training received; o How relevant their contributions were to

support the response; o What could be improved to strengthen

their capacities for future responses.

• See qualitative indicator above about the positive responses about the level of participation that the community may have experienced with aid delivery system.60

• Amount of resources received from civil society to engage in the response.

Interview data from KIIs in the field Community HH survey; and Budget information from civil society (as compared to the HRP).

High.

6.1 How did the response strengthen the capacities of partners to contribute in cross-cutting areas? [protection question]

7. Coordination

How well-coordinated was the assistance, avoiding duplication of assistance and gaps?

• Qualitative indicator: stakeholder perceptions about:

o the effectiveness of coordination (including examples where duplications were realized or and avoided);

o timeliness of funding by aid organizations;

o their experiences of coordination at all levels; and

o challenges faced and how they were mitigated (the negative impact of challenges reduced).

o Amount of resources dedicated to cross-cutting issues in the response.

KIIs in Maputo and in the field; Document review [meeting minutes] & analysis for triangulation purposes; and Budget information from civil society (as compared to the HRP).

Moderate. Perceptions are likely to differ by stakeholder group.

7.1 How effective were coordination mechanisms at the strategic, inter-cluster, and cluster/sector levels?

7.2 Is there any evidence that coordination helped to avoid duplications?61

7.3 How effective was the resource mobilization effort in raising sufficient, timely and long-term funding? How did the timing of the funding and

60 It is recognized that there may be some overlap examining partnerships and localization at the community level. This could be further examined during the data collection. 61 What are the challenges in the surge management among UN agencies, including the occurrence of “double-hatting”?




Reliability of Data

donor priorities influence the overall response, and in what form of assistance (financial and in-kind assistance) did donors provide assistance? 62

7.4 What major gaps in human and financial resources to carry out activities in cross-cutting areas were identified? [protection question]

62 How (well) did the CERF mechanism function and is this the most effective way to release funds only via UN agencies?



Annex C - Instruments for the Review A set of questions and sub-questions have been developed based on the TOR for the evaluation that will be used to

organize results from interviews, focus group discussions and document research. The questions for each instrument

below will provide the framework for the evidence matrix the team will use to collate and analyze data.

Interview Guide (Provincial and District-level INGC)

INFORMED CONSENT SOUGHT

Questions Sub-Questions

Appropriateness In what way have you participated in the response? How were the needs of the most vulnerable identified? What were the biggest vulnerabilities? (please tell us an example) Probe assessments undertaken,

Effectiveness

In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering assistance in this region? How timely do you think the support was? In your view, what is the most important change brought about by the project/response?

Coverage In your opinion, please give your thoughts about whether longer-term needs were met in this province/district (in first 6 months)?

Coordination

Were you in contact with international agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] Was comparative advantage maximized? Were there any instances that you can remember where efforts were duplicated? What aspects of the coordination could be improved?

Partnerships

Describe the different partnerships; What were the top three/five most relevant partnerships? What was the biggest success of your organization and your partners? Why? Give examples. What could be improved to solidify partnerships in the future? How could the international community assist INGC?

Localization

Did you receive any training from the UN? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues) To what extent was this training relevant and useful for your ability to contribute to the response?

Securing additional information

Are there any relevant documents that we should review?

Recommendations for the UN

Do you have any suggestions for improvement for international humanitarian agencies?

Misc. Is there anyone else that you think we should try and speak to?

Misc. Any other comments?

Interview Guide (Provincial and District-level NGOs/Civil Society Service Providers) INFORMED CONSENT SOUGHT




Appropriateness In what way have you participated in the response?

Effectiveness

In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering assistance in this region? How timely do you think the support was? In your view, what is the most important change brought about by the project that you worked on?

Coverage In your opinion, please give your thoughts about whether longer-term needs were met in this province/district?

Coordination Were you in contact with international humanitarian agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] Was comparative advantage maximized? Were there any instances that you can remember where efforts were duplicated? What aspects of the coordination could be improved?

Partnerships Describe the different partnerships; What were the top three most relevant partnerships? What was the biggest success of your organization and your partners? Why? What could be improved to solidify partnerships in the future? How can the UN assist your organization?

Localization Did you receive any training from the international humanitarian agencies? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues) To what extent was this training relevant and useful for your ability to contribute to the response?

Securing additional information Are there any relevant documents that we should review?


Do you have any suggestions for improvement for the international humanitarian agencies?



Interview Guides UN RC/HC and the Mozambique HCT INFORMED CONSENT SOUGHT



Effectiveness

In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering assistance in this region? How timely do you think that the support was? In your view, what is the most important change brought about by the project you worked on?

Coverage In your opinion, please give your thoughts about whether longer-term needs were met?




Coordination

Were you in contact with international humanitarian agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] Was comparative advantage maximized? Were there any instances that you can remember where efforts were duplicated? In what way do you think that the coordination mechanism was strengthened after this humanitarian response? What aspects of the coordination could be improved? Specific areas for probing include:

1. among the HCT members at country level;

1. between and among the HCT and non-HCT and non-GoM partners (e.g.,

national and International NGOs operating within the clusters, representatives

from the private sector with in kind donations and individuals/groups who

functioned within clusters);

2. regional level coordination for HCT members and INGOs which have a regional

presence (CARE, Save the Children, IFRC) , particularly as it relates to surge

management;

3. coordination at the HQ level; and

4. between the IASC and Emergency Management Group and the HCT,

particularly as it relates to raising funds and reporting updates and results to

higher decision-making structures (e.g., Emergency Response Task Force,

IASC Principals, Emergency Director’s Group, OPAG).

Partnerships

Describe the different partnerships; What were the top three most relevant partnerships for [agency]? What was the biggest success of your organization and your partners? Why? What could be improved to solidify partnerships in the future? How can international humanitarian agencies be of more assistance to the GoM?

Localization

Did you provide any training? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues] What are the key elements to making training more relevant and useful so that partners may effectively contribute to the response? (Probe other issues besides timeliness)




Do you have any suggestions for improvement for international humanitarian agencies? [Engage in some self-reflection here…]



Interview Guides (Regional Level ) The above interview guide will be used for Regional offices. However, the focus will be about the role that the regional offices ought to play in supporting the HCT and HC/RC to be more empowered decision-makers within the institutional context of different international humanitarian agencies. The surge management will be the main focus of interviews.

Interview Guides (Donors) INFORMED CONSENT SOUGHT





Effectiveness

In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering financial assistance in this region? How timely do you think that the support that your office provided was? In your view, what is the most important change brought about as a result of funding from this office?

Coverage In your opinion, please give your thoughts about whether longer-term needs were met with the financial assistance provided?

Coordination

Were you in contact with international humanitarian agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] What aspects of the coordination could be improved to support the efficient use of resources?

Partnerships

Describe the different partnerships; What were the top three most relevant partnerships for [your office]? What could be improved to solidify partnerships in the future? How can international humanitarian agencies be of more assistance to the GoM?

Localization

Did you provide any resources for training activities? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues] In your view, what are the key elements to making training more relevant and useful so that partners may effectively contribute to the response? (Probe other issues besides timeliness)




Do you have any suggestions for improvement for international humanitarian agencies? [Engage in some reflection here…]





Annex D – Community Focus Group Discussion Guide

HOUSEHOLD SURVEY QUESTIONNAIRE Good morning/Good afternoon. My name is________ and I am here as part of a survey research team from the Universidade Eduardo Mondlane that is gathering information on behalf of the United Nations and its partners to better understand the experience of people affected by Cyclone Idai in March of this year. Your inputs into this survey will be used to help the UN and its partners to improve their work here in Mozambique and in future humanitarian responses elsewhere. If you agree, I would like to ask of you for anywhere from a few minutes to 40 minutes of your time, depending on your experience after the cyclone. Your participation is this survey is voluntary: whether you choose to participate or not participate will not affect any future assistance to you or anyone else, and if there are any of the questions in the survey that you cannot or do not want to answer, you can choose not to answer them. You can also choose to finish the survey at any time. In addition, the information you provide will be strictly confidential: only our research team will have access to the survey data, and the final report on the survey results will not present information on individual survey participants but rather on the feedback provided by the community as a whole. Please feel free to speak to us openly. If you find something wrong with this interview, you can call this number… Give permission to continue [seek informed consent.]

CONFIDENTIALITY AND STATISTICAL AUTHORITY (Law 7/96 July)

ARTICLE 6 STATISTICAL AUTHORITY- The principle of statistical authority consists of the power conferred to the National Institute of Statistics to carry out, in the conduct of statistical activities, obligatory surveys within the time limits set, as well as to undertake steps to produce statistics. Article 14 STATISTICAL CONFIDENTIALITY - All individual statistical information collected by the official statistics production organs bodies are of a strictly confidential nature.

SECTION A: HOUSEHOLD IDENTIFICATION

A000 Questionnaire code

A001 Province

A002. District

A003 Administrative post

A004 Locale

A005 Community

A006 Household Head (AF) name

1 Sofala

2 Manica

3 Tete

4 Zamézia

5 Cabo Delgado



A007 AF Sex 1 Male 2 Female

Disability? 1. Yes 2. No 3. If yes, what kind?

A008 AF Head Contact

A009 Name of the respondent

A010 Name of the Ennumerator

A011 Supervisor Name

A012 Date of interview

A013 Start time

A014 End time A015 Geographic coordinates of the interviewee.



SECTION B: SOCIO-DEMOGRAPHIC CHARACTERISTICS OF THE HOUSEHOLD Before we get started, we would like to get some information about the people who usually live here. Once again, this information is not being collected to identify you; we are collecting this information only to ensure that we are reaching different types of households in our survey, and to enable us to compare the different types of experiences after the cyclone.

B100. AF member name

(Caution: Start filling out the

questionnaire by the head of the AF)

B110. AF member Sex

1 Male 2 Female

3 Other

B111. Relationship with the head of the

AF

1 Self (Head) 2 Spouse/Partner

3 Child 4 Brother/Sister

5 Father/Mother 6 Nephew/Niece 7 Grandson/daughter 8 Other Relative 9 No relation

B112. Age (full years)

B113. Marital status

1 1. Single 2 2. Married 3 4 The Union:

2a. marital 2b. Polygamous

4 5 3. Divorced

/ Separated 4. Widower

B. 114 Can you read and

write?

1 1 Yes

2 No

If no,

skip to

B117

B115. Have You

ever attended school?

1 Yes 2 No

B116. Highest level of schooling

you've reached?

0 No formal school

1 1 ª Classe 2 2 ª Classe 3 3 ª Classe 4 4 ª Classe 5 5 ª Classe 6 6 ª Classe 7 7 ª Classe 8 8 ª Classe 9 9 ª Classe 10 10 ª Classe 11 11 ª Classe 12 12 ª Classe 13 Upper level

19th Literacy 20. Other (Specify)

Questions intended for people aged 10 years and over

1 B.117 What are the sources of income?

2 1 trade

3 2 agriculture

4 3 cattle raising

5 4 other

6

7

8

9



SECTION C. HOUSING CHARACTERISTICS, ACCESS TO WATER, ENERGY AND TELECOMMUNICATIONS Enumerator will make observation of the respondent’s living situation for C100 through

C100. Ownership

status of dwelling

C111. What kind of material was used in the cover of the main

house?

1 Slab/Concrete 2 Tile 3 Zinc Plate

4th Lusalite 5 Grass/ plastic/canvas 6. Other (Specify)

C112. What kind of material was used on the walls of the main

house?

C113. What kind of material was used in building the floor of

the main house?

1 Beaten down earth 2 Adobe 3 The Cement 4 Parquet

5. Other (Specify)

C114. Is the residence of the AF connected to the public/private

electricity network?

1 Yes 2 No

C115. What is the main source of water for the consumption

used by the AF?

1 Public Network 2 Fountain/bore 3 Well 4 River/Lake/Lagoon

5 Other (Specify)

C116. Does the AF have

access to the

radio signal?

1 Yes 2 No


access to the mobile

phone network?

1 Yes 2 No


access to the TV signal?

1 Yes 2 No

1. Owner / Leased 2. Assistance 1 Adobe

2 Cement block 3 Burnt Brick 4 Sticks 5 Wood/Zinc plate

6 Bamboo/Reed/

Stakes 7 Other (Specify)



SECTION D. Early Warning In this section, we have a few questions for you about early warning -- that is, whether and when you were notified in advance of the Cyclone's arrival so that you could act to minimize its effects on your household."

D100. Were you

directly affected by

Cyclone Idai?

1 Yes 2 No

If not, complete inquiry

D101. In the days or hours prior to the cyclone, did you

receive any kind of

advance warning that

it was coming?

1 Yes 2 No

If not, pass to E100

D102. Through what information sources did you receive the warning or notice?

[Mark all that apply]

1 Radio 2 Television 3 Phone number 4 Local Risk 5 Management 6 Committee 7 Local leader 8 Local 9 Government 10 Family /friends

neighbors 11 SMS 12 No, Other

(Specify)

D103. How many days in advance

did you receive

the notice?

(Register the number of days, if it was on the same day place zero)

D104.

Was the warning or notice clear to you?

1 Yes 2 Yes but with some difficulties

3 No

➔ If yes pass to

D106

D105. If the warning was not clear, what

was the reason was?


1. Very technical language

2. Very vague/not clear

3. Other (Explain )

D106. In general, did you receive the warning or notice in time to minimize the

cyclone’s harm to your household?

1. Yes

2. No

➔ If yes, skip to

E100

D107. Explain what happened because you did not receive the early warning

on time.



SECTION E. Information about Assistance Received/response readiness In this section, we have a few questions for you about they typeof assistance that you have received. --

E100. Did you

need rescuing

after the

cyclone ?

1 Yes 2 No

E101. After approximately

how long were you eventually rescued? (Enter the number of days and if you do not

know to put the Code

98)

(Register the number of days, if it

was on the same day

place zero)

E102. Do you think

the rescue

operation was:

1 Early

2 On

time

3 Late

4 Other

Explain

E103. After the

cyclone's occurrence, what if any

of the following forms of

assistance did your

family need right away?


1 To be

rescued 2 Shelter 3 Food 4 Clothing 5 Medicine 6 Other

(specify)

E104. Did you

receive any

assistance after the cyclone's

occurrence (that is, within the first 72 hours?)

1 Yes 2 No

E115. What kind of assistance did you receive? [Mark all

that apply]

1 Food Goods 2 Shelter Materials

(ex: tents, blankets, etc.) 3. Clothing/Footwear 4. Health (ex: medicines, vaccination, etc.)

5. Education (ex: school material) 6. Agricultural inputs

(ex: seeds) 7. Money (voucher) 8. Building Material

9. Land for construction in a safe area

10. Other (specify)

E116. Who provided assistance/support? [Mark all that apply]

1 Government INGC 2. United Nations institution 3. Red Cross

4. NGOs 5. Religious institution 6. Family/friends/neighbors

7. My community besides

family 8.Do not know

9. Other (specify)

E117. Do you think

the assistance

you received

was 1 Early

2 On

time

3 Late

4 Other

[Each

response from E116

will be asked this question]

E118 Was the

assistance received

according to your

expectations ?

1 Yes 2 No

If yes, skip to F120

E119. IF

E118=No, insert “not met” here

E120. In what specific ways were your

needs met?

[Open Ended question]



SECTION F. Consultation Process In this section, we have a few questions for you about how it was explained to you about receiving the assistance and making any complaint about the overall process.

F100 Before receiving assistance , did anyone ever approach you to try to understand what your needs were?

1 Yes 2 No

F101.a Were the criteria, or means

of determining households' eligibility for assistance,

clearly explained to you?

1 Yes 2 No 3 Don’t recall

F102. Do you think

the criteria used to receive assistance were fair? →If so, go to F104.

1 Yes 2 No

F103. What aspects

of the eligibility

determination process do you think

could have been done

better?

OPEN ENDED

F104.a Was it very clear to you

about the assistance you would receive regarding food

aid? 1.yes 2.no 3. More or less

F105.a If no or more or less, please explain:

OPEN ENDED

F106. At the time, did those

who provided

Assistance to you

treat you with respect

and dignity?

1 Yes 2 No

F107. Did anyone at

any time explain to you

the process by which you

could complain about the

assistance?

1 Yes 2 No 3 Don’t know

F108. In thinking about the overall

process, do you think it was easy for you to get the assistance?

1 Yes 2 No

F109. (ONLY IF “no”) What specifically

was difficult about

accessing the

support?

F104.b When you would receive it?

1.yes 2.no 3. More or less

F105.b If no or more or less, please explain:

OPEN ENDED

F101.b If no, please explain:

OPEN ENDED

F104.c The quantity

You would receive?

1.yes 2.no 3. More or less

F105.c If no or more or less, please explain:

OPEN ENDED

F104.d How often would you receive? 1.sim 2.não 3. More or less

F105.d If no or more or less, please explain:

OPEN ENDED

a) a)

b) b)



c) c)

SECTION G. Impartiality In this section, we have a few questions for you about factors that may have influenced the assistance that you received.

G100. Do you think the assistance you received was distributed fairly?

1 Yes

2 No

If yes, go to G102

G101. (ONLY IF “no”) please tell

me why?

G102. Do you think your gender influenced the help you received? 1. Yes 2. No 3. More or

less

If not, go to G104

G103. If yes or more or less please tell me why?

G104. Do you think your age influenced the help you received? 1. Yes 2. No 3. More or less

If not, go to G106


G106. Do you think your ethnicity influenced the help you received? 1. Yes 2. No 3. More or less

If not, go to G108




G108. Do you think your level of schooling influenced the help you received? 1. Yes 2. No 3. More or less

If not, go to G110


G110. Do you think your physical status (with special needs or not) influenced the help you received? 1. Yes 2. No 3. More or less

If not, go G112


G112. Are there political influences that may have helped you to receive the assistance? 1. Yes 2. No 3. More or less

If not, go to G114


G114. Do you think your shelter area influenced the help you received? 1. Yes 2. No 3. More or less

If not, go to G116


G118. Do you think that the assistance benefited the people who needed it most (most vulnerable people, seniors, disabled people, widows, orphaned children, etc.)?

1 Yes 2 No

G119. In your opinion, do you think there were people who needed the most, but who were left out for the benefit of the less needy?

1 Yes 2 No

G120. If so, who were these people that were left out?

1 Elderly people 2 Orphaned Children 3 Women/Widows 4 People who have lost everything 5 5. Very poor people 6. Other (Specify)

G121. For the next storm, what do you think should be done so that people who need it the most do not go back out?

OPEN ENDED

G122. Should a cyclone like this happen again, what do you think humanitarian agencies should do differently?

OPEN ENDED



SECTION H. Relevance of assistance/support received In this section, we have a few questions for you the appropriateness of the help that you received.

H100. What kind of support did you receive (based on what has already been said)? (Tick all valid options for them)

1. Rescue 2. Power 3. Accommodation 4. Water

5. Medical assistance

6. Household utensils 7. Construction

Material

8. Psychosocial

Apoio 9. Agriculture

Material (e.g. seeds, hoes, etc.)

10. Others (Specify)

H101. Do you think that assistance you received included what you most needed most at that time?

1 Yes 2 No

If yes, go to H103

H102. If not, what kind of assistance would you like to have received?

(Listen to the answers and tick according to

the categories below)

1 Agriculture 2 Shelter 3. Health care 4.Education 5. Food 6. Clothing

7. Water 8. Protection

against threats (ex-infringement, etc.

9. Other (Specify)

H103. Can you tell us

after how long after the cyclone

did you start to receive

the assistance?

(Register

the number of days you received the

first assistance)

H104. Were you satisfied with the quality of

the assistance you

received?[ [Multiple replies] With respect to:

a) Food aid b) Rescue c) Education d) Accommodation e) Water f) Medical care g) Household utensils h) Construction Material i) Psychosocial Apoio j) Agriculture Material (e.g. seeds, hoes, etc.)

1 Yes 2 No 3 more or

less

Food aid 1.yes 2.no 3. More or less

105. If not or more

or less

can you tell us why?

106. Were you satisfied with

the quantities received?[multi ple

replies]

a) feeding b) Accommodation c) Water D) Medical assistance e) Household utensils f) Construction Material g) Psychosocial Apoio (h) Material of agriculture (e.g. seeds, hoes, etc.)

1 Yes 2 No 3 more or less

107. How useful was

the assistance/

support received in helping to

recover in the short term

(immediate)? [Multiple replies]

1. Very useful 2 More or less 3 Not useful

108. How useful was the assistance/

support received in helping to

recover in the long term?

1. Very useful 2 More or less 3 Not useful

Rescue 1.yes 2.no 3. More or less

Education 1.yes 2.no 3. More or less

Water 1.yes 2.no 3. More or less Medical assistance 1.yes 2.no 3. More or less Household utensils 1.yes 2.no 3. More or less Construction Material 1.yes 2.no 3. More or less



Psychosocial Apoio 1.yes 2.no 3. More or less

Materials for agriculture 1.yes 2.no 3. More or less

a) a) a)

b) b) b)

c) c) c)

d) d) d)

e) e) e)

f) f) f)

g) …. g) g)



SECTION I. Effectiveness of assistance In this section, we have a few questions for you about factors that may have influenced the impact of the assistance that you received.

I100. How have you been

affected by the cyclone?

1 Death of at least one

family member

2 Injuries to at least one

family member 3. Illness to at least one

family member 4 Destruction of housing 5. Crop loss 6. Loss of animals 7.

Loss of

materials/production

equipment 8 Family displacement 9.Other (specify)

I101. If you have moved, to what

location?

1 Temporary

shelters 2 Family/friends'

home 3. Resettlement Zone 4 Other (Specify)

I102. If you compare your current level of

living with the previous one

before the cyclone, would you say your situation

has…?

1. Worsened 2 Been normal 3. Improved

4. Stayed the

same

I103. If it got worse or better,

why.

I104. What would have happened to your family if you had not benefited from any assistance? [Multiple responses]

1. We would not be alive 2. We would be hungry /

suffering malnutrition 3. We would be sleeping in

the bush

4. Diseases 5. Children would not be

studying 6. It would be difficult to

resume life 7. Other (Specify)

I105. Do you think the assistance received had any positive effect on your family?

1st Yes 2nd No

If not, go to I107

I106. If Yes, please tell me what

were the positive effects?

I107. Do you think the

assistance received had any negative effect on your

family?

1st Yes 2nd No

If not, go to I109



I108. If so, please tell me what the effects were?

I109. Do you think Assistance has helped to

create some kind of conflict(s) in your

community?

1 Yes 2 No

If not, go to I112

I110. If so, what kind of

conflicts?

I111. What do you think should have

been done to avoid this kind of conflict?

I112. If the cyclone were to happen again, do you think your family is better prepared to deal with the

phenomenon?

I1113. Why?

I 1114 – do you have any recommendations for future assistance – what would you do differently?

I 1115 – would you like to be informed on the results of this survey?

We appreciate your time. End



Annex E – Evaluation Workplan

IAHE Ethiopia – Inception Report | 53


Annex F – Letter of Introduction

31 August, 2019

Dear Sir/Madam;

Humanitarian assistance is of the utmost importance for the victims of natural disasters and

other emergencies. As per General Assembly Resolution 46/182, the Inter-Agency Standing

Committee recognizes that Mozambique has the primary responsibility to care for victims

of disasters and other emergencies, including the coordination and implementation of

humanitarian assistance.

The United Nations Humanitarian Team is in the process of conducting an independent

assessment of the extent to which planned collective objectives set out in the Humanitarian

Response Plan responded to the needs and concerns of affected people during cyclones Idai

and Kenneth.

In my capacity as Humanitarian Coordinator, I have the pleasure of informing you that our

office has engaged an Evaluation Team to conduct this evaluation from 1 – 31 September

2019.

The Evaluation Team will have questions related to the relevance, effectiveness, efficiency,

sustainability, scalability, and lessons learned about the humanitarian response.

As we value greatly your contributions that you may provide, we appreciate your

cooperation with these individuals,

Should you have any questions about the evaluation or the Evaluation Team, please feel

free to contact me.

Sincerely,

Humanitarian Coordinator

https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/GA%20Resolution%2046-182.pdf



Annex G – Informed Consent Form Good morning/Good afternoon. My name is________ and I am here as part of the team that is gathering information to enable us to improve the humanitarian response in the future. You are being asked to take part in a research study. Before you decide to participate in this study, it is important that you understand why the research is being done and what it will involve. PURPOSE OF STUDY: The purpose of this study is to understand if your priority humanitarian needs of were met during the cyclone Idai. We want to know if you were respected; treated with dignity, no sexual harassment, or discrimination. CONFIDENTIALITY: According to the law of Confidentiality and Statistical Authority (Law 7/96 July), your comments will be anonymous [unless you participate in a focus group]. Every effort will be made by the researchers to preserve your confidentiality. Our final report that we will produce will not present any information of one person but of the community as a whole. If you agree, I would like to ask of you about a half an hour (30 min) to 45 minutes of your time to discuss the humanitarian aid that has been provided to you during cyclone Idai. Note that participant data will be kept confidential except in cases where the researcher is legally obligated to report specific incidents. These incidents include, but may not be limited to, incidents of abuse, violence, or emotional risk [the protection protocol must be followed]. NO BENEFITS FOR PARTICIPATION: Your participation is completely voluntary. Our study is not linked to any of these election. We are impartial and independent. We want to understand your opinion and your situation. If you partcipate in this study, it will not influence any opportunity to receive any future assistance. VOLUNTARY PARTICIPATION: It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form. I have read and I understand the provided information and have had the opportunity to ask questions. I understand that my participation is voluntary, and I agree to take part in this study. Participant's signature ______________________________ Date __________ Investigator's signature _____________________________ Date __________

190902_Informed_C

onsent IAHE.docx

INTER-AGENCY HUMANITARIAN EVALUATION OF THE RESPONSE TO CYCLONES IDAI AND KENNETH IN MOZAMBIQUE



Annex H –Data Analysis Plan

QUALITATIVE DATA QUANTITATIVE DATA

• Agency & Partner Interviews Collection includes:

1. Note-taking;

2. Summarizing information and separating by: i)

emergent theme; and ii) evaluation questions;

3. Data coding to take place as needed.

A total of 400 face-to-face surveys from a random sample of HHs in Maníca, Sofála, and Tête. The analysis includes the following steps:

1. Data cleaning to eliminate any errors from coding;

2. Frequency analysis of overall samples

(disaggregating data by gender and region);

4. Crafting main messages from this data source;

5. Use Provalis or other tools to ensure that main

themes are salient.

3. Frequency analysis of all categorical variables;

• Content analysis of existing & other relevant

evaluations to be focused on lessons and

recommendations to ensure consistency with this

evaluation’s recommendations.

Note that higher level analysis that the following will be attempted (but may not be possible):

• Cross tabulation of satisfaction levels in different

sectors;

• Cross tabulation of satisfaction levels in different

regions; and

• Community FDGs / Community Level Interviews

includes:

1. Note-taking;

2. Summarizing information and separating by: i)

emergent themes; and ii) evaluation questions;

Data coding to take place as needed.

• Cross tabulation of satisfaction levels by different

groups that did or did not receive emergency

assistance.

• Cross tabulation of other levels of stratification if

possible but not promised.

3. Crafting main messages from this data source;

Use Provalis or other tools to ensure that main themes are salient.

4. Crafting main messages from these data sources;

5. Main figures to be used to substantiate findings

emerging from qualitative data sources.

• Humanitarian requirements & standards among

other resources to be used for constructing the

Response Theory of Change Model.

• Cluster-Level Strategy Plans; Data for needs

assessments; and monitoring data review includes

the comparison of:

1. Original needs assessed (and quality of

information source);

2. Objective of number of persons to target

proposed;

3. Follow on reports to determine accountability (if

available)

4. If not available, additional assessment

undertaken to produce new target of assistance

to affected population;

5. Observe trends;

6. Crafting main messages from this data source.



Page purposefully left blank