Monitoring and Evaluation

4

5

• Objective of this session

• Monitoring and Evaluation (M&E) – what do we want to measure?

• International frameworks for Monitoring and Tracking Adaptation

Content

6

What you can expect to learn from this session:

• Get familiar with the challenges of

• Monitoring and Evaluation (M&E) frameworks for adaptation projects

• Application of this knowledge to the finance area through a) measuring the impact of adaptation project >> effective spending b) options of tracking and coding >> transparent spending

7

Monitoring and Evaluation (M&E) – what do we want to measure?

8

Monitoring and Evaluation (M&E) of adaptation impacts – what do we want to measure?

Purpose M&E approaches or methods

To monitor and evaluate the collective performance ofnumerous adaptation programmes and/or projects

In case of programmes/projects being managed by the same organisation: use of predefined standard indicators that can be aggregated across projects. Example: Adaptation indicators of the Adaptation Fund (AF, 2011)

To monitor whether a (sub)-national adaptation plan orstrategy is being implemented and achieving its results

Developing a (sub)-national M&E system and/or integrating adaptation into existing (sectoral) M&E systems in the country. No specific guidance currently exists, but the WRI/GIZ guidebook may provide general advice

To monitor whether an adaptation project is proceeding according to plan and achieving its results

Monitoring using a theory of change (Guidelines by GIZ, WRI/GIZ)

Source: GIZ 2013. A closer look at Monitoring and Evaluation (M&E) of Adaptation

9

Adaptation M&E on a project level

10

Adaptation M&E on a project level

Source: www.adaptationcommunity.netGIZ, 2014. Integrating climate change adaptation into development planning. Training module 6

5. Operationalise the M&E system

4. Define indicators and set a baseline

3. Developing a results framework

2. Identify the contribution to adaptation

1. Describe the adaptation context

GIZ’s Adaptation made to measure:Five-step model

The principal aim of M&E is to demonstrate the benefits of adaptation.

M&E needs to be already considered during the planning and design phase

Adaptation M&E at the project level does not require a completely different M&E system

Potential challenges for adaptation M&E at the project level: Designing meaningful adaptation

indicators beyond the output level. Dealing with the uncertainty in

climate and socio-economic development.

Evaluating project results after the project lifetime.

11

Indicators can be categorised by type of adaptation activity:

Source: OECD, 2011

Types of adaptation indicators

12

Adaptation M&E on a national level

13

Adaptation M&E on a national level

Source: www.adaptationcommunity.netGIZ, 2014. Integrating climate change adaptation into development

planning. Training Module 6.

Effectively managing adaptation to climate change at national level requires an understanding of whether adaptation is taking place and what the outcomes of adaptation are.

M&E provides evidence for assessing the results of adaptation efforts and tracking progress towards climate resilient development.

Setting up national M&E system: Five-step model

14

The MRV+ System under Kenya’s National Climate Change Action Plan

Policy context Kenya National Climate Change Action Plan 2013 - 2017

Purpose • Informing and guiding on implementation of climate change actions• Fulfilling international reporting obligations• Demonstrating Kenya’s climate finance readiness and attract international finance

Process Institutional set up • Integrated into existing institutional structures• Draws on information already gathered as part of standard M&E

Indicator-based approach

Institutional adaptive capacity

• 62 national-level, process-based indicators• 10 short-listed outcome-based indicators

Vulnerability • 62 national-level indicators on local-level outcomes• 10 short-listed outcome-based indicators

Reporting

Resources • ~100 people to set up and run the system• Up to 3 years to operationalize the system

M&E for adaptation

Kenya MRV+

MRV of GHG

Establishment process

1. Review of existing practices and literature2. Design of the system building on existing structures ~20 months

• Annual reports or Medium Term Plans for Ministries, Departments and Agencies��

• Vision 2030 progress reports��

• Biennial Update Report (BUR) to the UNFCCC• (every two years starting in December 2014)• National Communications to the UNFCCC��

Source: GIZ 2013. Monitoring and Evaluating Adaptation at Aggregated Levels: A Comparative Analysis of Ten SystemsGIZ 2014. Kenya: The MRV+ System under Kenya’s National Climate Change Action Plan

15

M&E of multiple adaptation projects – international approaches

16

• Complexity of the climate finance due to multitude of actors, mechanisms and rapidly changing environment

• Clear internationally agreed definitions of mitigation and adaptation finance needed

• Actions, which contribute to adaptation or mitigation co-benefits, are often not “coded” as climate relevant

• Double-counting adaptation and mitigation spending, when it involves multiple institutions

• Lack of tracking of private finance and complex methodological issues related to it

• Multiple and overlapping reporting systems on the international level

Defining, coding, and tracking climate finance

• No common metrics for tracking

• outcomes of adaptation projects are often intangible and difficult to quantify, and

• often cannot be directly compared across different locations

• Uncertainty inherent in climate projections and climate impacts.

• Close interconnection between climatic and non-climatic stressors

• Long time scales – in many cases, the ultimate success of adaptation can only be measured far into the future

Common challenges for climate finance Additional challenges for adaptation finance

Source: ODI 2012GIZ 2013. A closer look Monitoring and Evaluation (M&E) of Adaptation

17Source: Adaptation Fund Results Framework and Baseline Guidance

Adaptation Fund results management framework

Goal: “Assist developing-country Parties to the Kyoto Protocol that are particularly vulnerable to the adverse effects of climate change in meeting the costs of concrete adaptation projects … ”

Impact: Increased resiliency at the community, national, and regional levels to climate variability and change

Outcomes:1. Reduced exposure at national level to climate-related hazards and threats2. Strengthened institutional capacity to reduce risks associated with climate-induced socioeconomic

and environmental losses3. Strengthened awareness and ownership of adaptation and climate risk reduction processes at

local level4. Increased adaptive capacity within relevant development and natural resource sectors5. Increased ecosystem resilience in response to climate change and variability-induced stress6. Diversified and strengthened livelihoods and sources of income for vulnerable people in targeted

areas7. Improved policies and regulations that promote and enforce resilience measures

Each outcome includes 1-2 outputs with concrete set of indicators

18

Green Climate Fund: Initial results management framework for adaptation

Source: See detailed indicators in the GCF decision B.08/07, Annex VIII, Table 1Schalatek L. Moving Beyond “Business as Usual”. Heinrich Boell Foundation, 2015.

LEVEL EXPECTED RESULTSParadigm shift Increased climate resilient sustainable development

Fund level impacts

• Increased resilience and enhanced livelihoods of the most vulnerable people, communities, and regions

• Increased resilience of health and wellbeing, and food and water security

• Increased resilience of infrastructure and the built environment to climate change threats

• Improved resilience of ecosystems and ecosystem services

Project / programme outcomes

• Strengthened institutional and regulatory systems for climate responsive planning and development

• Increased generation and use of climate information in decision-making

• Strengthened adaptive capacity and reduced exposure to climate risks • Strengthened awareness of climate threats and risk reduction

processes

Core indicator: Total number of direct and indirect beneficiaries; number of beneficiaries relative to total population

19

Tracking climate finance spending: Performance-based climate resilient grants by LoCAL programme

Source: http://www.local-uncdf.org/performance-based-climate-resilient-grants.html

LoCAL promotes mainstreaming of climate change adaptation and resilience into local panning and budgeting systems with Performance-based climate-resilient grants (PBCRGs)

Grants consist of a financial top-up, covering additional costs of making investments climate resilient, complementing regular budget allocations

Grants are included in the municipal budget revenue (as a separate line) and expenses (if possible in one or separate lines)

Grants are disbursed in the context of local annual planning and budgeting cycles, and selected measures are implemented

Performance are appraised in terms of the degree to which additional resources have been used to build resilience and promote adaptation to climate change, and audits are undertaken as part of the regular national process

20Source: adapted from ODI, 2013

Monitoring and Tracking Adaptation Activities (II)

Approach Strength Challenges

Activity categorized as either

A) financing an adaptation-relevant sector or

B) supporting institutional capacity development.

• Focus on knowledge management and learning

• Quality of reporting limited to detail of project documents which form the only means of verification.

• Relies heavily on expert judgment.

PPCR results framework

21Source: adapted from ODI, 2013GIZ 2014. Measuring, Reporting and Verifying Climate Finance

Monitoring and Tracking Adaptation Activities (III)

Approach Strength Challenges

Stage 1: Adaptation focus is categorized: UNFCCC provisions as 1. ‘principle objectives’, 2. ‘significant objectives’, or3. ‘not targeting the objectives.’Stage 2: Adaptation intervention is categorized into enabling activities and sectoral activities.

• Integrates adaptation tracking into DAC

• Community aid tracking processes

• To avoid double-counting, includes assessment of overlap between different objectives

• New separate unit to develop methodologies and measure private finance flows, incl. those mobilized by public intervention

• Only tracks activities that identify climate change in project documentation;

• Data gaps; • Reliant on self-reporting; • Simplistic in capturing the range and

scope of adaptation finance• Applied mainly to bilateral ODA• More precise eligibility criteria and

activity descriptions needed for clear understanding of what qualifies and should be quantified

• Further harmonization needed to make reported data comparable with the reporting under UNFCCC

OECD DAC Adaptation Marker

22

Monitoring and Tracking Adaptation Activities (III)

Approach Strength Challenges

• National Communications as main financial information reporting document

• Common Tabular Format (CTF) for reporting on financial support, technology transfer and capacity-building

• Biennial Reports for developed countries

• Agreed single reporting format for developed countries that provides fairly detailed and comprehensive financial information

• Harmonized and coherent reporting practice needed in the longer run

• Joint understanding on terminology and reporting parameters is critical

• Climate-related private finance tracking and reporting needs to be enhanced

• MRV of climate finance under UNFCCC appears to be institutionally fragmented

UNFCCC level

Further guidelines for measuring and tracking climate finance to be developed Biennial assessment and overview of climate finance flows to be presented at COP20 in

Peru to give a “big picture” and include meta-information

Source: adapted from ODI, 2013GIZ 2014. Measuring, Reporting and Verifying Climate Finance

23

Key Questions

• What is experience with M&E frameworks in your country?

• What are key challenges to design M&E frameworks for adaptation projects in line with international requirements?

24

Thank you for your attention!!!

25

BACK UP

26

Methods of adaptation M&E

Adaptation M&E seeks evidence that adaptation has taken or is taking place.

Common methods of adaptation M&E are:

• Employing indicators (quantitative or qualitative)

• Surveying stakeholders (e.g. the local population)

• Eliciting expert opinions

Qualitative evidence may consist of stories or narratives from community members.

Indicators may refer to one aspect, e.g. number of policies considering climate change,

or may be composite or index indicators that are summing up multiple inputs.

27

Monitoring and Evaluation (M&E) of adaptation impacts – what do we want to measure?

without clim

ate change

with climate change

Time

Development

AdaptationVulnerability as a proxy

with enhanced adaptive capacity

28

The vulnerability function (IPCC 2001)

Exposure Sensitivity

Potential Impact Adaptive Capacity

Vulnerability

to wheather and climate induced hazards such as drought, heavy rain falls etc.

of a system (e.g. eco-system, watershed, household, village, city, country) to wheather and climate induced hazards

potential or capability of a system to adjust to climate variability and extremes

29

Baseline development

Develop impact chain from vulnerability functions for impact groups

Step1: Identify relevant exposure factors (weather and climate) for the

selected impacts.

Step 2: Identify natural, physical or land-management related factors which

make an area or a group of people sensitive towards an impact.

Step 3: Identify political, social, economic and cultural factors which contribute to a high or low adaptive capacity.

30

Baseline development

Development of adaptation measures:Example: land degradation, erosion and landslides

Erratic but intensive precipitation events Deforestation

Unsuitable cultivation of steep slopes

Land degradation

Erosion and landslides

Knowledge of farmers on proper land management

Vulnerabilityof village farmers to land degradation, erosion &

landslides

Exposure Sensitivity

Impact

Adaptive Capacity

Afforestation with indegenous species

Plant suitable crops

High dependency on natural resources

Provide skills training to diversify farmer incomes

Awareness raising & training on land management

Adaptation Measures:

31

Assessing and managing the risks of Climate Change (IPCC 2014)

IPCC, 2014 (p. 3): http://www.ipcc.ch/pdf/assessment-report/ar5/wg2/ar5_wgII_spm_en.pdf

32

Assessing and managing the risks of Climate Change (IPCC 2014)

Vulnerability

Hazard

Risk

Exposure

Example: Communities located at flood-prone river bank

Community settling at the river bank, living in shanty houses

Number of houses and people and the time they spent in the flood-prone area

Frequent floods

Loss of lives and assets of community due to floods

33

Assessing and managing the risks of Climate Change (IPCC 2014)

Vulnerability

Hazard

Risk

Exposure

Potential adaptation measures:Communities located at flood-prone river bank Improve houses to

sustain floods, educate community members to recognise early signs of flooding

Relocate community from the river bank to another area

Build flood breaks or establish early-warning system

Measures result in reduction of probability of occurence and negative impacts

34Source: adapted from ODI, 2013

Monitoring and Tracking Adaptation Activities (I)

Approach Strength Challenges

Adaptation interventions and Adaptation processes are categorised

Sector themes help determine

• actors, • roles,• responsibilities,• methods,• technologies and • results.

Difficult to represent cross-sectoractivities;

relies heavily on expertjudgment

UNDP M&E Adaptation framework