activity monitoring and evaluation plan (amep) · apl area penggunaan lain (other uses zones) ......
TRANSCRIPT
ACTIVITY MONITORING AND
EVALUATION PLAN (AMEP)
FEBRUARY 18, 2016
This publication was produced for review by the United States Agency for International Development. It was prepared
by Tetra Tech ARD.
This publication was prepared for review by the United States Agency for International
Development under Contract # AID-497-TO-15-00005. It was produced by Tetra Tech ARD.
The period of this contract is from July 2015 to July 2020.
Implemented by: Tetra Tech P.O. Box 1397 Burlington, VT 05402
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 1
ACTIVITY MONITORING
AND EVALUATION PLAN
(AMEP)
FEBRUARY 18, 2016
DISCLAIMER
This publication is made possible by the support of the American People through the United
States Agency for International Development (USAID). The contents of this publication are
the sole responsibility of Tetra Tech ARD and do not necessarily reflect the views of USAID
or the United States Government.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 2
TABLE OF CONTENTS
Acronyms ............................................................................................................ 3
1. Introduction .................................................................................................... 7
1.1 Project Background ................................................................................................... 7
1.2. AMEP Development ................................................................................................. 9
2. LESTARI M&E Frameworks ......................................................................... 10
2.1 Situation Model ....................................................................................................... 10
2.2. LESTARI Results Framework and Theories of Change .......................................... 14
2.3 Logical Framework and Performance Indicator Tracking Table (PITT) .................... 25
3. Plan for Activity Monitoring and Evaluation .............................................. 44
3.1 LESTARI Learning Loop: Data Collection, Storage, Quality Assurance, Analysis, and Utilization ............................................................................................................... 44
3.2 Internal Evaluations Plan ......................................................................................... 51
3.3 Schedule for Performance Monitoring Tasks ........................................................... 55
3.4. Monitoring and evaluation structures, functions, and capabilities ........................... 56
Annex I: Performance Indicator Reference Sheets (PIRS) ........................... 59
Annex II: Results Chains ............................................................................... 118
Annex III: Data Quality Assessment Checklists .......................................... 126
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 3
ACRONYMS ADS Automated Directives System
AMEP Activity Monitoring and Evaluation Plan
APBD Anggaran Pendapatan dan Belanja Daerah (Local Budget Revenues and
Expenditures)
APL Area Penggunaan Lain (Other Uses Zones)
BAPPEDA Badan Perencanaan Pembangunan Daerah (Regional Development
Planning Agency)
BAPPENAS Badan Perencanaan Pembangunan Nasional (National Development
Planning Agency)
BAU Business as Usual
BIG Badan Informasi Geospasial (Geospatial Information Agency)
BMPs Best Management Practices
C4J Changes for Justice
CAs Conservation Areas
CCAs Community Conservation Agreements
CIFOR Center for International Forestry Research
CMMPs Conservation Management and Monitoring Plans
CMP Conservation Measures Partnership
CO2 Carbon Dioxide
CO2e CO2 equivalent
COP Chief of Party
CSOs Civil Society Organizations
DAK Dana Alokasi Khusus (Special Allocation Fund)
DCOP Deputy Chief of Party
DNPI Dewan Nasional Perubahan Iklim (National Council on Climate Change)
DQ Data Quality
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 4
DQAs Data Quality Assessments
FACTS Foreign Assistance Coordination and Tracking Systems
FMU Forest Management Unit (or KPH)
FY Fiscal Year
GAP Good Agricultural Practice
GHG Greenhouse Gas
GIS Geographic Information System
GOI Government of Indonesia
GPS Global Positioning System
Ha Hectare
HCS High Carbon Stock
HCV High Conservation Value
HO Home Office
ICCTF Indonesia Climate Change Trust Fund
ICRAF International Center for Research in Agroforestry/World Agroforestry
Center
IFACS Indonesian Forest and Climate Support
IPCC Intergovernmental Panel on Climate Change
IR Intermediate Result
IUCN International Union for Conservation Nature
KR Key Result
LBA Landscape Baseline Analysis
LCPs Landscape Conservation Plans
LEDS Low Emissions Development Strategy
LOP Life of Project
LULUCF Land Use, Land Use Change, and Forestry
M&E Monitoring & Evaluation
ME & L Monitoring Evaluation & Learning
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 5
METT Management Effectiveness Tracking Tool
MIS Management Information System
MOEF Ministry of Environment and Forestry
MOU Memorandum of Understanding
MRV Measuring Reporting Verification
MSF Multi Stakeholder Forum
NGO Non-Governmental Organization
NRM Natural Resource Management
PA Protected Areas
PES Payment for Environmental Services
PIRS Performance Indicator Reference Sheets
PITT Performance Indicator Tracking Table
PPPs Public-Private Partnerships
QCA Qualitative Comparative Analysis
RAN GRK Rencana Aksi Nasional Gas Rumah Kaca (National Action Plan for
Greenhouse Gas Emission)
REDD+ Reducing Emissions from Deforestation and Forest Degradation
(including conservation, sustainable management of forests, and
enhancement of forest carbon stocks)
RF Results Framework
REL Reference Emission Level
RPJM Rencana Pembangunan Jangka Menengah (Mid-Term Development
Plan)
RTRWK Rencana Tata Ruang Wilayah Kabupaten (District Spatial Plan)
RTRWN Rencana Tata Ruang Wilayah Nasional (National Spatial Plan)
RTRWP Rencana Tata Ruang Wilayah Provinsi (Provincial Spatial Plan)
SEAs Strategic Environmental Assessments
SDI Spatial Data Infrastructure
SMART Spatial Monitoring and Reporting Tool
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 6
SOP Standard Operating Procedure
SOW Scope of Work
STI Sustainable Travel International
TFA Tropical Forest Alliance
ToC Theory of Change
UN-REDD United Nations-Reducing Emissions from Deforestation and Forest
Degradation
USAID United States Agency for International Development
USG United State Government
WCPA World Commission of Protected Area
WCS Wildlife Conservation Society
WWF World Wildlife Fund
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 7
1. INTRODUCTION 1.1 Project Background USAID LESTARI supports the Government of Indonesia to reduce greenhouse gas (GHG)
emissions and conserve biodiversity in carbon rich and biologically significant forest and
mangrove ecosystems. Built on the strong foundation of USAID’s IFACS project, LESTARI
applies a landscape approach to reduce GHG emissions, integrating forest and peatland
conservation with low emissions development (LEDS) on other, already degraded land. This
is achieved through improved land use governance, enhanced protected areas management
and protection of key species, sustainable private sector and industry practices, and
expanded constituencies for conservation among various stakeholders. LESTARI is
implemented under the leadership of Tetra Tech and a consortium of partners including
WWF-Indonesia, Winrock International, Wildlife Conservation Society (WCS), Blue Forests,
Yayasan Sahabat Cipta, PT Hydro South Pole Carbon, Sustainable Travel International
(STI), Michigan State University, and the FIELD Foundation. LESTARI runs from August
2015 through July 2020.
LESTARI activities are targeted in six strategic landscapes on three of Indonesia’s largest
islands, where primary forest cover remains most intact and carbon stocks are greatest. In
northern Sumatra, the Leuser Landscape comprises significant portions of Aceh Selatan,
Gayo Lues, Aceh Tenggara, and Aceh Barat Daya districts, and includes the Aceh portion of
Leuser National Park and Singkil Wildlife Sanctuary. In Central Kalimantan, LESTARI works
in the Katingan-Kahayan Landscape, comprising Pulang Pisau, Katingan, and Gunung Mas
districts; Palangkaraya municipality; and Sebangau and Bukit Baka Bukit Raya National
Parks. LESTARI also works in four landscapes in Papua. Sarmi and Cyclops Landscapes
are located along the northern coast and comprise Sarmi district as well as Jayapura district
and municipality. The Lorentz Lowlands Landscape, comprising Mimika and Asmat districts
plus a large portion of Lorentz National Park, and the Mappi-Bouven Digoel Landscape are
located along Papua’s southern coast. LESTARI is managed from its headquarters in
Jakarta, with offices in each landscape as well as the provincial capitals of Aceh, Central
Kalimantan, and Papua.
Overall Results of LESTARI are:
Goals:
Key Result 1: At least 41% of total CO2-equivalent emissions reduced from land use,
land use change and deforestation averaged across all landscapes within the project
scope;
Key Result 2: At least 8.42 Million hectares of primary or secondary forest (and
peatland), including orangutan habitat, under improved management;
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 8
IR 1: Improved Forest Management:
Key Result 3: Management of at least six Conservation Areas (CAs) improved,
resulting in the conservation of valuable orangutan and other key species habitat,
and the reduction in poaching of threatened and endemic species;
Key Result 4: At least ten Public-Private Partnerships (PPPs) promoting low-
emissions conservation oriented development established;
Key Result 5: Funding leveraged from public and private sources, representing co-
investment in project outcomes;
IR 2: Improved Land Use Governance:
Key Result 6: Increased commitment of key private sector, government, and
community stakeholders regarding the positive benefits of conservation and
sustainable use of forests and the species they encompass;
Key Result 7: Policies, laws, regulations, and procedures in support of low emission
development and forest conservation and management increased, promulgated, and
enforced at all levels; and
Key Result 8: Model(s) for successful integration of district, provincial, and national
low emissions development and forest conservation strategies developed and shared
at all levels of government and with other key stakeholders.
To achieve these results, LESTARI will implement 8 Strategic Approaches in three
complementary technical thematic areas:
1. Forest & Land Use Governance & Advocacy;
2. Conservation Co-Management; and
3. Private Sector Engagement
Following a Landscape Approach1 to implementation, each strategic approach will be
nuanced to address the biophysical, social, political, and cultural context of each landscape.
LESTARI will be implemented primarily with congressionally mandated Biodiversity
Conservation funds. All Biodiversity Earmark-funded activities must align with U.S. Agency
for International Development’s (USAID) Biodiversity Policy and Biodiversity Code. The
Code requires that funded activities must a) have an explicit biodiversity objective; b) be
identified based on an analysis of drivers and threats to biodiversity and a corresponding
theory of change; c) must have the intent to positively impact biodiversity in biologically
significant areas; and d) must monitor indicators associated with each stated theory of
change for biodiversity conservation results. To support learning, adaptive management, and
to comply with the Biodiversity Code and ADS 203, Tetra Tech has developed the LESTARI
Activity Monitoring and Evaluation Plan (AMEP) to provide standards and guidelines for
monitoring, evaluating, and communicating our progress toward achievement of the
biodiversity and emissions targets, project results, and higher level USAID and GOI goals.
1 Further information about Landscape Approach, please refer to LESTARI Landscape Baseline Analysis (LBA) document.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 9
1.2. AMEP Development The LESTARI AMEP is a living document, designed to ensure accountability and learning
which will be used to facilitate and promote evidence-based decision making. The document
outlines Monitoring & Evaluation (M&E) systems, protocols, and data collection
methodologies. Throughout implementation, this framework will ensure continuous learning
while improving coordination and collaboration to track, evaluate, and articulate results and
impacts. The LESTARI M&E approach also ensures the adaptive management of project
activities through the use of sound, reliable, and timely data. Furthermore, the approach to
performance management integrates the five Open Standards for the Practice of
Conservation steps (hereafter, Open Standards). This enables the LESTARI AMEP to be
used as not only a monitoring tool but also a management tool that will be referenced by
staff, and used for training and capacity building to ensure comprehension and compliance
with sound performance management.
The design of LESTARI AMEP is further founded upon lessons learned from the IFACS
M&E system, and key recommendations from the IFACS Final Assessment. A key
recommendation was the need to incorporate a Theory of Change (ToC) approach in M&E
design. The assessment found that key critical assumptions about the causal linkages
between activities and higher level results had not been considered due to limited use of
program theory.
Following this recommendation, the LESTARI AMEP incorporates a theory of change
approach to define all building blocks required to bring about positive change whereby multi-
stakeholders’ interests and actions are better aligned with sustainable outcomes. LESTARI’s
theories of change not only illustrate the causal linkages between project interventions, but
also how these interventions address drivers to deforestation to reduce high level threats to
ecosystems, species, and human well-being targets. In addition, performance indicators and
key impact evaluation questions have been selected to monitor progress toward
achievement of outcomes and impacts along each Results Chain. Theories of Change will
also serve as a project management tool to review critical assumptions driving LESTARI
strategic approaches, facilitate early identification of technical challenges, manage
stakeholder expectations, and serve as a framework for gathering evidence.
This document outlines the detailed blueprint designed to monitor and evaluate the project’s
performance. First, the LESTARI Situation Model, Results Framework, Strategic
Approaches, and Theories of Change are described. This is followed by outlines project
indicators and targets which further detailed in Performance Indicator Reference Sheets
(PIRS) formatted to USAID standards. PIRS will provide identification of staff responsible for
different tasks, as well as timelines. The document then describes the LESTARI Learning
Loop which explains how the LESTARI M&E system be integrated into project management
and how project impact will be routinely monitored and assessed. Finally, the
operationalization of M&E system through the establishment of M&E Team and the
development of Management Information System (MIS). The structure and content of this
AMEP is congruent with ADS 203 and includes both USAID Foreign Assistance
Coordination Tracking Systems (FACTS) and custom indicators for monitor performance and
evaluating the theories of change.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 10
2. LESTARI M&E FRAMEWORKS 2.1 Situation Model The LESTARI AMEP was developed following the Conservation Measures Partnership
(CMP) Open Standards for Adaptive Management. Design began with identification of the
project scope (geographic focus), conservation targets (key species and ecosystems, as well
as human well-being targets linked to these species and ecosystems), direct and indirect
threats, and drivers to be included in the LESTARI Situation Model. Upon completion of the
Situation Model, theories of change were developed by analyzing how LESTARI
interventions will address factors within each results chain to reduce key threats and lead to
the conservation of priority targets and emissions reductions. In September 2015, a
participatory M&E Workshop was held with LESTARI Staff, USAID representatives, and
representatives from USAID’s Measuring Impact project to review and further nuance the
Theories of Change for inclusion in the AMEP. This process resulted in the identification of
eight Strategic Approaches which will be implemented within three Technical Themes:
Forest & Land Use Governance & Advocacy, Conservation Co-Management, and Private
Sector Engagement. Throughout the course of the project, LESTARI will lead participatory
reviews of theories of change with partners, staff across all landscapes, and USAID to
assess the validity of initial assumptions and casual linkages, to refine project planning as
necessary, to address gaps and maximize results, and to update the AMEP to support
ongoing learning and adaptive management. Reviews will address key evaluation questions,
including:
To what extent have LESTARI interventions been effective in reducing biodiversity
threats and carbon emissions in targeted areas?
To what extent have LESTARI intervention been effective in improved land-use
governance?
To what extent has the project addressed barriers of political will and buy-in at the
national and regional levels?
To what extent has improved collaboration between government, private sector and
community contributed to improved forest management?
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 11
Figure 1. LESTARI Situation Model
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 12
Table 1 LESTARI Situation Analysis
Problem Statement
As one of the richest biological countries in the world, Indonesia is considered “mega diverse” country, contributing, along with six other Asian countries, up to 70 percent of the globe’s biological diversity. Indonesia has held Asia’s largest and continuous track of tropical rain forests and 10% of the world’s forests. Important terrestrial ecosystems include mangroves, lowland forests, montane forests, and peat/swamp forests, the protection of which is considered critical for conserving biodiversity and preventing or reducing land based carbon emissions. Healthy intact forest ecosystems are also an important safety net for local communities living nearby, presenting economic opportunities through forest dependent livelihoods that prevent their slide into extreme poverty and improving their resilience to climate and other external shocks.
While at least 30 million people depend directly on Indonesia’s forests and on the ecosystem services they provide, these ecosystems have been and continue to be under extreme threats. Indonesia’s deforestation rate is well known to be among the highest in the world. It was reported 15.79 million Ha of forest cover loss for period 2000-2012 (Margono, 2014). Deforestation is especially critical in carbon rich peat and swamp forests of Sumatra, Kalimantan, and Papua. These forests are also the primary habitat for key species of global importance, particularly those listed on CITES Appendix I (especially rhinos, elephants, tigers, and orangutans) and endemics found in Papua. These key species are threatened by large-scale destruction of tropical forests, habitat loss, unsustainable harvesting, and trafficking.
Deforestation, peatland degradation, and forest fire have put Indonesia among the top three largest emitters of greenhouse gases in the world. The emissions resulting from forest fires and degradation are five times greater than those ensuing from non-forestry emissions. A 2010 report suggests that 85% of Indonesia’s greenhouse gas (GHG) emissions stem from land use activities with 37% due to deforestation and 27% due to peat fires (National Council on Climate Change, 2010). The most recent research on forest fire and emission it has resulted demonstrates that since the beginning of September 2015, Indonesia’s fires have been emitting carbon at a rate of 15-20 million tons per day, or more than the 14 million tons emitted on a daily basis by the whole American economy.
Forest fires also destroy large areas that serve as habitat for biodiversity. Plants and animals are eliminated directly by the fires and those survive will diminish gradually. For example, the fires in 1997-1998 caused the population of orangutans to decline by 33 percent on the island of Borneo (Rijksen et al.,2006).
Threats, Drivers, and Opportunities
Over the last 20 years deforestation has been driven predominantly by agricultural expansion, especially of oil palm plantation. A history of weak governance combined with opportunistic private sector actors has created unsustainable economics based on natural resource exploitation. Land clearing for oil palm plantations is one of the key drivers of deforestation. Palm oil is also an important element of Indonesia’s national development strategy, as it is the world’s largest producer and exporter of crude palm oil. Oil palm plantation area has doubled in Indondesia in the past decade, and pressures for expansion continues as GOI has instituted targets for doubling CPO production to 40 million tons by 2020.
Other commodity-driven pressures on land include unsustainable industrial scale logging and conversion for pulp, rubber, and coffee/cacao plantations. An inequitable system of land use decision-making is threatening forests, as smallholders that cannot compete for access with large private sector actors are forced to encroach onto forested areas.
In regards with key species, expansion of oil palm and pulpwood plantations in
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 13
Sumatra was responsible for nearly two-thirds of the destruction of tiger habitat from 2009 to 2011. When rainforests are cleared, it becomes increasingly easier for poachers to track and kill tigers, which are considered prized trophies for any hunters. Used in traditional Chinese medicine and as ornaments, each Sumatran tiger can sell for up to $5,000, offering poachers a greater income than they would earn working in paddy fields or on rubber tree plantations.
The World Wildlife Fund (WWF) estimates that orangutan populations have significantly declined from 230,000 nearly a century ago to 7,500 in Sumatra and 55,000 in Borneo. A significant factor in the decline of orangutan numbers is the fragmentation of their habitat, which makes them vulnerable to wildlife traffickers.2
The LESTARI Situation Model and Analysis illustrate how deforestation and forest degradation in Indonesia is a result of a complex dynamic of political, economic and institutional drivers. Decision-making on land use involves a large array of laws and actors and the process is rarely transparent or consistent. Land use decisions are driven by economic incentives that prioritize extractive industries and agriculture over low emissions and conservation-oriented business models. However, enabling conditions for a shift to more equitable and sustainable forest and land use management are emerging after years of contradictory talk about forest clearing moratoriums while oil palm and industrial forest plantations rapidly expanded. The new President administration is taking clear, firm, and ambitious steps to improve forest and land use governance in a sustainable and equitable manner. Three new ministries have been formed to elevate and reinforce the goals of better governance of natural resources and the rural poor who depend upon them for most livelihoods: Environment and Forestry; Agrarian and Spatial Planning; and Villages, Development of Backward Regions, and Transmigration.
The private sector is also demonstrating substantial leadership toward reducing large-scale deforestation. The US Ambassador recently facilitated the Palm Oil Pledge, signed by CEOs of Asian Agri, Cargill, Golden Agri Recourses, Wilmar, and Indonesian Chamber of Commerce (KADIN). Reaching all the way to their third-party suppliers, the pledge aims to make palm oil supply chains more sustainable. Further, Sinar Mas’ Asia Pulp & Paper has committed to halt deforestation in their concessions and to conserve 1,000,000 hectares of High Conservation Value (HCV) and High Carbon Stock (HCS) landscapes in and around their operations across Sumatra and Kalimantan; it has undergone an independent audit of these areas by Rainforest Alliance. Increasingly, companies are demonstrating that good business integrates sustainable landscape and conservation management into their operations. Civil society, particularly forest-dependent communities, are increasingly advocating for more responsible land and forest governance. This is demonstrated in Central Kalimantan, where communities have become disenchanted with the empty promises of economic benefits from oil palm expansion while suffering from the omnipresent haze of land clearing and peatland fires. Forest-dependent communities across Indonesia are initiating actions to counter this situation—mapping customary (adat) forests, integrating these maps into the OneMap initiative, and negotiating boundaries with concession holders and spatial planners. LESTARI aims to catalyze such positive change, and is strategically designed to do so. LESTARI targets a 41% GHG emissions reduction in its landscapes by 2020. The Government of Indonesia has also committed to reducing GHG emissions by 29% below the business as usual condition (baseline) by 2030 (previously 26% by 2020) unilaterally, and up to 41% below the baseline with bi-
2 http://www.onegreenplanet.org/animalsandnature/deforestation-poaching-and-the-wildlife-trade-in-indonesia/
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 14
lateral or international assistance. LESTARI will work within the same time frame as the GoI commitments and has adopted the same targets of working towards 41% reduction of Baseline emissions by 2020. As such the project can effectively harmonize efforts with the GoI within the LESTARI landscapes. A thorough understanding of where LESTARI will contribute to the GoI targets is presented in the Landscape Baseline Analysis Plan (LBA). Historical deforestation and degradation in protected areas, production forest and development (non-forest estate zones), as well as on peat and non-peat areas was analysed to indicate where forest loss and degradation is likely to occur in the future, thus targeting LESTARI’s efforts. LESTARI targets of GHG emission reductions are based on the LBA.
Indonesia is also a Party to the Convention on Biological Diversity and as such, has made commitments to achieve the Aichi Biodiversity Targets by 2020. Indonesia has set specific targets for protection of key endangered species such as tigers, rhinos, and orangutan. This clear alignment in goals and interests is expected to give LESTARI strong buy-in and support from all stakeholders and contribute to the project’s successes in protecting Indonesia’s forests.
LESTARI Landscape Approach
LESTARI applies a landscape approach – an integrated land use management framework that seeks to integrate policies and actions across sectors in order to harmonize development and conservation objectives. In other words, the landscape approach aims to ensure that all the uses of land and all the users of that land are being addressed in an integrated way. LESTARI activities are targeted in six strategic landscapes on three of Indonesia’s largest islands, where primary forest cover remains most intact and carbon stocks are greatest. The LESTARI landscapes are largely defined by national parks and forests supporting mega diversity and key species. To achieve both climate change and conservation objectives, LESTARI has targeted area with HCV and HCS. To address key threats, key activities to be implemented under LESTARI will focus on improving spatial planning for better land management, improved collaborative management, alternative funding for forest conservation or payment for environmental services, preventing encroachment, reduce poaching, community conservation agreements, forest education initiatives as well as strengthening local partners’ landscape vision and commitment for low emissions development.
2.2. LESTARI Results Framework and Theories of Change An overarching Results Framework (RF) has been developed (Figure 2a and 2b) to explain
how LESTARI Strategic Approaches will address the drivers of deforestation and forest
degradation, and reduce emissions and threats to biodiversity presented in the Situation
Model. The RF demonstrates causality and the logical linkages between shorter-term,
intermediate, and longer-term results. The Results Framework is founded upon the
overarching theory that if there is increased awareness and advocacy of local
stakeholders (government officials, communities, CSOs and private sector) for
conservation and natural resource management in ways which contribute to their
long term interests; if Conservation Co-Management is actually enhanced to the
benefit of key stakeholders, and if the private sector is engaged, there will be
improved governance and natural resource management in biologically significant
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 15
and critical areas3, leading to a 41% reduction in GHG emissions by 2020. Our Results
Framework is then translated into a matrix showing how LESTARI plans to integrate
LESTARI’s Technical Components into 8 strategic approaches that can be nuanced
according to the context of each targeted landscape. This matrix also demonstrates our
theory of change for each Strategic Approach, in accordance with the Results Chains
provided in Annex II: Results Chains.
3 Critical areas: areas identified for conservation because of their high conservation value or their high carbon value
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 16
Figure 2a. LESTARI Results Framework with Key Results
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 17
Figure 2b. LESTARI diagrammatic Results Framework
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 18
Table 2 Theory of Change Matrix
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
Technical Theme I: Forest and Land Use Governance and Advocacy
Awareness and Advocacy
C1.1. Increasing government level awareness, appreciation, and advocacy to ensure that government decision-makers embrace a conservation and sustainable use vision for land use planning.
Improved local level awareness, appreciation, and advocacy to promote citizen action is one of enabling conditions for other LESTARI strategic approaches to run effectively. Through the implementation of this strategy, we expect that local stakeholders will adopt and continue desired behavior thus results in the formation of constituent for conservation. We assume that if the citizens demand for healthy ecosystems and have the capacities for advocacy and lobbying, the government will respond by embracing a conservation and sustainable development vision to better address demand from its citizens.
As outlined in the Result Chains for this strategy (Annex II), it is critical for LESTARI to first identify target audience, message, desired behavior, and appropriate media to deliver the message. LESTARI needs to ensure that target audiences receive the messages through a series of awareness and advocacy activities, retain the desire knowledge which will lead to improvement on attitudes and values. Draw on IFACS experience, it is critical to confirm that knowledge and attitude be translated into effective desired behavior. Therefore, during the implementation LESTARI will assess factors that impede change to occur and address them immediately.
C4.1. Improve local level awareness, appreciation, and advocacy. Improve key stakeholders’ awareness, appreciation and advocacy for healthy ecosystems, leveraging increased availability of mobile and internet based technology.
C4.2. Empower media coverage of environmental issues. Strengthen the ability of media and citizen journalists to monitor, document, and publicize the positive and negative consequences of land use decisions, including those made by private sector actors in the landscapes.
Operationalize SEAs & LCPs
C1.2. Operationalizing Strategic Environmental Assessments (SEAs) and Landscape Conservation Plans (LCPs) developed by USAID IFACS within land use plans
Improved land use governance is a cornerstone of achieving improved forest management and biodiversity conservation. Good governance is characterized by the respect for legal certainty, transparency and free flow of information, significant public participation (expression of voice and choice), equality, high accountability, effective and coordinated management of public resources.
LESTARI assumes that SEAs and LCPs initiated under IFACS are effective entry points to improve land use governance. The incorporation of those documents in government planning (notably spatial planning,
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 19
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
RTRW and related zonation as well as development plans, RPJMD/P) will involve significant public participation. As such a comprehensive and accurate SEA is crucial to developing a spatial plan that incorporates LEDS. In conjunction, LCPs is a tool to give local communities quality information about the conservation value of their land and ensure that government plans consider local conservation needs. Such data can also serve as valuable evidence for communities in their advocacy and lobbying work, which may be an important element in holding decision makers accountable.
LESTARI will operationalize both SEA-LEDS and LCPs through the following:
Facilitate the development or update the SEA-LEDS document that incorporate LCPs
Advocate for district to issue instruction for operationalizing SEA-LEDS
Work through local government and the MSF to integrate high quality SEA-LEDS in spatial plans
Expand impact at district levels by officially incorporating SEA-LEDS recommendations into RPJMs’ planning and budgeting as well as licensing and permitting process
Environmental Governance
C1.3. Improving licensing and permitting processes so they are public and accessible, subject to public oversight, compliant with laws and regulations, and violations are enforced
Land use planning and licensing of natural resource use is the single largest contributor to GHG emissions in the LESTARI landscapes. Environmental governance is a principle strategic approach that contributes to improved planning, permitting, and enforcement of natural resource use within landscapes. It is an interlinked approach that combines introducing citizen-based mechanisms for input on land use, as well as increasing the district willingness to adopt such input, with improved mechanisms for licensing, monitoring, and enforcing the use of natural resources. Multi Stakeholder Forum (MSF) built under IFACS will serve as a bridge to facilitate communication between government and citizen. The strategic approach also links to operationalizing SEA and LCP recommendations.
C1.4. Monitoring and enforcing land use, including increasing district government prosecution and adjudication capacity, building on USAID C4J activities, and capacity to use remote sensing and other geospatial tools, building on the spatial data infrastructure units established previously by USAID IFACS
C4.3. Strengthen citizen based mechanisms for public input on land use. Building on the multistakeholder forums established
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 20
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
under USAID IFACS, support citizen-based mechanisms for public input and monitoring of land use decisions by key groups, especially local communities, indigenous groups, and women. This task is closely aligned with the Component 1 task to encourage districts to solicit and adopt input from citizen based mechanisms
It is also important to note that of the numerous governance challenges, the limited availability of data which several stakeholder groups deem credible represents one of the most serious. Robust and accurate data is needed to inform well-measured action plans, especially in terms of reforming legal and policy frameworks and improving the capacities of government, civil society, indigenous communities and private sector stakeholders. Therefore, LESTARI will continue to build upon IFACS progress on developing Spatial Data Infrastructure (SDI) units, linked to OneMap, to improve the accuracy, consistency, and availability of land use information especially public transparency and accessibility of natural resource use processes and licenses.
C1.6. Increasing district governments’ willingness to adopt input from citizen-based mechanisms. This is connected to the work with multistakeholder forums under Component 4.
Technical Theme II: Conservation Co-Management
Co-Management
C2.3 Improve co-management adjacent to Conservation Areas by building staff knowledge and implementation of improved collaborative management strategies with adjacent districts and local communities. This task is closely linked to Component 4’s collaborative management task.
Key to project success is building capacity and commitment for co-management among relevant actors in areas adjacent to PAs and critical areas. This is essential for pressure reduction on the PA through more transparent, negotiated roles and responsibilities. Relevant actors will include Ministry of Environment and Forestry Park staff, provincial and district governments, relevant CSOs, and villages living adjacent to and dependent on PA and critical areas resources and services.
LESTARI will facilitate dialogue between key stakeholders and communities, develop the capacity of villages to be able to negotiate with district, provincial and national agreements, comply with, and benefit from co-management agreements to protect their local natural resources.
The strategic approach supported by awareness and advocacy so as to increase the willingness of key stakeholders to develop co-management agreements with communities. It also links to strategic approach green enterprise and innovative financing mechanism whereas the co-management agreements provide incentives for communities to participate in forest protection.
LESTARI will work with leaders that represent both men and women and
C3.2. Pilot innovative financing for critical areas. Mobilize financing opportunities, including payments for ecosystem services, to incentivize sustainable enterprises that meet required criteria for environment and social soundness
C4.4. Improve co-management adjacent to critical areas. Provide technical assistance to establish co-management arrangements between communities living in and near critical areas and the appropriate government entities. Includes defining acquiring clear tenure. Closely linked to Strategic Approach 2. SEAs. See IUCN definition of "co-management".
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 21
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
other disadvantaged groups to ensure differing gender considerations and viewpoints are addressed through co-management.
Protected Area Management
C2.1. Improve Conservation Area management by determining priority needs and actions for addressing threats specific to each CAs and the species they harbor in consultation with key stakeholders. Once these have been identified, support the implementation of priority actions to address threats and strengthen the protection and/or management of CA and the species they harbor. These may include management planning; improving data, monitoring and information systems; increasing the use of technology to monitor and enforce wildlife encroachment and poaching; enhancing patrols; improving public relations with districts and communities inside or adjacent to CAs; and other capacity building
Key to success in improving conservation areas management capacity is building political leadership; implementing improved management approaches including protection of core areas to combat poaching and deforestation; and adopting M&E systems to assess the effectiveness of management interventions and adapt approaches as needed.
LESTARI will focus on the socialization and application of the METT tool to assess management effectiveness in protected areas embedded in its landscapes. Baseline METT will be conducted in a participatory way so as to enable CA managers taking ownership of METT score and committed to improving their management weaknesses. Awareness and advocacy will be emphasized to increase public pressures on CA authority to improve their management weaknesses.
A training program –with a focus on technical skills and modern tools that managers and PA staff require for effective day-to-day duties –will be delivered based on the outcome of the METT assessments. LESTARI will also equip CAs with the capacity to undertake SMART (Spatial Monitoring and Reporting Tool) as a tool to monitor and enforce wildlife encroachment and poaching.
This strategic approach links to improve co-management adjacent to CAs, especially because threats are coming from area adjacent to CAs. Further assumption is CAs could improve and implement key aspects of management plan if they are adequate financing. Therefore, LESTARI will identify and develop effective innovative financing plans for CAs.
Furthermore, the success of LESTARI in implementing a landscape approach requires effective policy engagement at multiple levels across multiple stakeholders. LESTARI will coordinate with the national level government and provincial level to ensure local impact sustainability,
C2.2. Pilot innovative financing for Conservation Area by identifying and trailing CA sustainable financing and revenue-generating strategies
C1.7. Enhance national level policy coordination Component 2
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 22
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
particularly since Law 23 of 2014 recentralizes a significant amount of natural resource decision-making to the province.
Technical Theme III. Private Sector Engagement
Green Enterprises
C3.1. Develop public private partnerships for green enterprises. This includes reviewing existing or conducting new market and feasibility studies to explore the potential for ecotourism and other environment-friendly sustainable enterprises that will directly benefit conservation and local communities living within and adjacent to high conservation area forests. Equality important is cultivating relationships with key private sector actors in the project’s landscapes to identify sources of co-investment in the above conservation efforts. Finally, the Contractor should develop and pilot sustainable business models that can have broad impacts, including outside the specified landscapes, and disseminate the results of pilots
As with all other Technical Themes, Private Sector Engagement takes an integrated approach to the development of livelihoods, responsibilities of environmental governance, and shared roles in co-management of natural resources by reducing spatial uncertainty especially for local communities. The capacity of indigenous communities is naturally impacted by the actions of the business sector: when a business company follows good governance practices, this will have a positive impact on indigenous communities, and vice versa. Therefore, LESTARI will facilitate both parties to seek opportunities for cooperation and mutual reinforcement. Through establishing public-private partnerships for green enterprises, and coupled with commitments to safeguards for forest resources (such as through co-management) and improve environmental governance, LESTARI will support alternative livelihoods while reducing pressure on forest resources. C4.5. Improved sustainable livelihoods and alternative
livelihoods for communities adjacent to critical areas. Provide technical assistance to communities living in or near critical areas to improve sustainable natural resources management practices, including those that might lead to eventual third party certification, that enhance productivity and income or adopt locally appropriate alternative livelihoods or green enterprises in order to reduce encroachment. Communities receiving livelihood support must acknowledge and agree to reduce encroachment through formal agreements such as the Community Conservation Agreements (CCAs) developed under the USAID IFACS project
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 23
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
Private Sector BMP
C3.3. Industry certification, best management practices (BMPs) and Conservation Management and Mitigation Plans (CMMPs). Support private sector-led efforts to make explicit commitments towards zero deforestation and conservation of HCV and other critical areas including support for third party certification, implementation of BMPs and CMMPs
The private sector provides significant opportunities for conservation of biodiversity and reduction of deforestation and degradation within LESTARI landscapes. LESTARI will adapt a suite of tools, expanding the applicability of the existing Agricultural Sustainability Screening Protocol and Sustainability Roadmap developed under the USAID CIRCLE project that allow the private sector to identify key steps to increase their sustainability and transparently monitor and demonstrate the impact of its commodity production on emissions. Subsequently, LESTARI will expand on its work under IFACS to promote BMPs for natural forest concessions and fiber and oil palm plantations leading to HCV forest set-asides, as well as for value chains critical to improved livelihoods. Through co-management agreements, LESTARI will facilitate the establishment of LEDS-based community partnerships with villages adjacent to concessions area. The assumption then if companies apply BMPs, if companies enhance co-management with villages, and if both parties gain benefits from BMPs and co-management implementation, then unsustainable use of natural resource will be discontinued thus emissions will be reduced and biodiversity conserved. Public discourse will be encouraged towards the removal of policy constraints to increasing the value of natural resources to the private sector in ways that are both economically and environmentally sustainable.
PES and REDD Innovative Finance
C1.5.Enhance district readiness to access financing (DAK, REDD+, etc) by building necessary capacity, supporting MRV systems, and informing policies, rules, and systems for the equitable distribution of funds generated from payments for carbon or other ecosystem services
With the creation of Indonesia Climate Change Trust Fund (ICCTF), Global Environment Fund, and the Green Climate Fund, there is potentially increased opportunity for districts to access funds to reduce emissions from deforestation and degradation. An important and more omnipresent source of funding is from government budgets. LESTARI will provide technical assistance to local government to prepare project design documents and safeguards to access REDD+ funding. LESTARI will also work with central and local government officials to include LEDS, forest conservation, and climate change mitigation programs to meet reduced emission commitments, especially through Payment for Environmental Services (PES) and societal valuing of such services in
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 24
Strategic Approaches
Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;
2. CAs & Key Species; 3. Private Sector; 4. Constituencies)
Theory of Change
general. LESTARI will also identify and engage related ministries to gain more access and support additional financing sources and to explore potential collaboration with similar programs from the government in optimizing the benefits to the communities and environment. If funding can be secured, then impact on emissions reduction could be magnified.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 25
2.3 Logical Framework and Performance Indicator Tracking Table (PITT) Our Logical Framework presents an overview of LESTARI Goals, Strategic Objectives,
indicators, means of verification, critical assumptions, and annual and life of project (LOP)
targets for monitoring progress toward achievement of results and evaluating the validity of
LESTARI’s Theories of Change. Output, outcome, and impact level indicators that provide a
sound benchmark for performance management and reporting on project results have been
selected, including required standard indicators for biodiversity and sustainable landscapes.
Detailed information for each indicator is provided in Annex I: Performance Indicator
Reference Sheets (PIRS).
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 26
Table 3 Logical Framework and Targets
Note: Please refer to Performance Indicator Reference Sheets (PIRS) in Annex I for indicator disaggregation
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
LESTARI Goals
Reduced GHG emissions from land-use sector
1. Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (OUTCOME/IMPACT)
Note: for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent
Comparison of actual emissions to REL following GoI methodology for Land-Based GHG Emission and Sequestration Baseline Calculations. The REDD Abacus SP program is used to calculate land cover changes and related emissions.
Reductions in deforestation and degradation will lead to reductions in GHG emissions.
0% reduction
Not measure
Not measu
re
30% reducti
on
40% reducti
on
41% reducti
on
41% reducti
on
Not measure
Not measu
re
41 M tCO2-eq
68 M tCO2-eq
96 M tCO2-
eq
96 M tCO2-
eq
Key species protected
See #3 See #3 See #3 See #3
See #3 See #3 See #3 See #3 See #3 See #3
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 27
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
Carbon rich forest, peatland, mangrove ecosystems and the habitat of key species conserved
2. Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance (FACTS 4.8.1-26) (OUTCOME/IMPACT)
Sub-national: Co-
management plan in place and under implementation as evidenced by incorporation into government program and budget.
FMUs assisted by LESTARI operational.
Certain area set-aside for conservation as result of SEA-LEDS and/or LCP recommendations into local development plans.
CA: Increased
METT score
Concessions:
CMMP or other conservation plan in place and under implementation as evidenced by incorporation into SOP and company budget.
Improved management will lead to the conservation of key species habitats as well as contribute to reduction in GHG emissions.
0 0 1.7 M 2.5 M 3 M 1.5 M 8.7 M
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 28
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
Reduced rate of deforestation and degradation
See #1 See #1 See #1 See
#1 See #1 See #1 See #1 See #1 See #1 See #1
Reduced threat on key species
3. Percentage reduction in poaching in focus area (OUTCOME)
Analysis of Spatial Monitoring and Reporting Tool (SMART) data. Comparison of baseline and endline. LESTARI will undertake baseline in Q2 Year 1 to determine number of current poaching incident in focus area.
Reduction in poaching will lead to conservation of key species in focus area
TBD Not measure
10% reducti
on
20% reducti
on
30% reducti
on
40% reducti
on
40% reducti
on
Improved Land Use Governance
Forest & Land Use Governance & Advocacy
4. Number of public policies addressing climate change and/or biodiversity conservation introduced, changed or adopted consistent with citizen input (OUTCOME)
Review the process of policies development, minutes of public consultation meetings, comparison of policies introduced, changed or adopted with citizen input.
When stakeholder’s voices continue to be heard by their government, it will ensure sustainability of MSF and lead to improved land use regulatory framework as prerequisite of improved land use governance
0 3 5 8 8 4 28
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 29
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
5. Number of sub-national government with improved licensing and permitting mechanism (OUTCOME)
The term “improved” is taken to subsume the five core principles of good governance (transparency, participation, accountability, responsiveness, and timeliness). Standard Operational Procedures (SOP) will be developed to ensure that all licenses and permits follow the government regulation and procedures. Verification will be made through reviewing the implementation of SOP, licensing process, interview licensing officers, business enterprises and community to ensure good governance principles are adhered.
When licensing and permitting mechanism adopt five core principles of good governance, the sustainable use of natural resource can be secured.
0 0 4 4 4 2 14
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 30
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
6. Number of sub-national government incorporating high quality SEA-LEDS & LCPs into draft spatial plans, zonation regulation or draft mid-term development plan (OUTCOME)
Comparison of SEA-LEDS & LCPs recommendation to spatial plan, zonation regulation and mid-term development plan.
Explicit reference to adopting SEA recommendation of improved-case scenario (not BAU) in spatial plan, zonation regulation and mid-term development plan documents.
Incorporation of SEA-LEDS & LCPs recommendation into government planning documents will lead to improved land use policy, which contributes to improved land use governance.
0 2 4 4 2 2 14
7. Number of Multi Stakeholder Forum (MSF) operational as citizen based mechanisms for public input on land use (OUTCOME)
Review of the process of public consultation, review Bupati decree of MSF, qualitative assessment to learn the effectiveness of MSF
The success of MSF to bridge communication between local government and wider public in land use will lead to more transparent, participatory and accountable land use decisions. Those principles are prerequisite of good governance.
0 2 4 4 4 0 14
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 31
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
8. Number of community champions engaged in advocacy interventions (OUTPUT)
Review advocacy plans or strategies, recording of press conference, copy of testimony or press release, advocacy campaign materials, record of public consultations.
Advocacy should be understood as a means for individuals and constituencies to shape public agendas, change public policies, and influence other processes that impact their lives. Advocacy interventions are essential aspects of democratic policy making, citizen participation, and oversight of government.
0 50 150 150 150 0 500
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 32
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
9. Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation (OUTPUT)
Sign-in sheet from communication trainings and meetings. Review communication products developed by stakeholders. Monitoring social media.
Qualitative assessments (e.g., case study) of increase awareness and understanding.
Increase awareness and understanding, supported by continuous advocacy will lead to increased commitment of key stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass, which in turn lead to improved land use decision making, thus contributes to improved land use governance.
0 1,000 4,000 5,000 3,500 1,500 15,000
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 33
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
Improved Forest Management
Conservation Co-Management
10. Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes (OUTCOME)
METT baseline and endline assessment; enriched by qualitative analysis
To maximize the potential of protected areas, it is important to understand the strengths and weaknesses of their management and the threats that they face. Addressing the weaknesses and reducing threats will lead to increase effectiveness of CAs management.
See PIRS
0 1 2 2 1 6
11. Number of co-management agreements signed that secure community rights and benefits (OUTPUT)
Review the content of co-management agreements, evaluation of the effectiveness of their implementation
Co-management agreement that secure community rights and benefits will lead to reduction in land use conflicts and increase commitment to better forest management
0 0 10 13 12 5 40
12. Number of people receiving USG supported training in natural resources management and/or biodiversity conservation (FACTS 4.8.1-27/OUTPUT)
Sign-in sheets, training reports. Assessments of the effectiveness of trainings and the application of knowledge and skills gained.
Increased capacity to manage natural resources and/or biodiversity conservation will lead to improved land use governance and forest management
0 1,000 2,000 3,000 3,000 1,000 10,000
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 34
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
Private Sector Engagement
13. Amount of investment mobilized (in USD) for climate change as supported by USG assistance (FACTS 4.8.2-10/OUTPUT)
Finance documents (e.g., APBD, finance reports, etc). Interviews the financers. Assess the use of funds and the issues being addressed with the funds.
LESTARI programs should attract additional funds that are necessary to increase capacities for addressing land use and forest conservation. Such funds represent knowledge and commitment of local stakeholders to, and institutionalizati on of, the issues being addressed with the funds.
0 2 M 4 M 4 M 5 M 5 M 20 M
14. Number of people receiving livelihood co-benefits (monetary or non monetary) (OUTCOME)
Random survey and on-site observations
Economic benefits derived from sustainable use of forest resource will provide incentives for people to be committed to LEDS and forest conservation.
0 0 7,500 7,500 7,500 7,500 30,000
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 35
STRATEGIC OBJECTIVES
INDICATOR MEANS OF
VERIFICATION ASSUMPTION TARGETS
B Y1 Y2 Y3 Y4 Y5 LOP
15. Number of private sector firms that have improved management practices as a result of USG assistance (FACTS 4.6.2-9/OUTCOME)
Review the CMMP/conservation plan document, incorporation into SOP and budget line item allocated by concessionaires to implement the plan. Spot-check to observe field implementation
Implementation of CMMPs/conservation plan will ensure that HCV areas in concessions are managed in sustainable manner.
0 2 2 2 2 2 10
16. Number of new USG-supported public-private partnerships (PPPs) formed (FACTS PPP 5) (OUTPUT)
PPP document signed by both public and private entities
PPP will provide incentives both for governments, private sectors and communities to embrace LEDS and conservation oriented practices.
0 3 5 5 5 2 20
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 36
Table 4 LESTARI Targets per Landscape
Note: This is illustrative table. The target per landscape will be updated in LESTARI Annual Work Plans based on annual achievement.
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
1. Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (OUTCOME/IMPACT)
Note: for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent
41% reduction
Leuser 0% Not measure
Not measure
30% 40% 41% 41%
Not measure
Not measure
5,839,579 7,786,105 7,980,758 7,980,758 M tCO2-eq
Katingan-Kahayan
0% Not measure
Not measure
30% 40% 41% 41%
Not measure
Not measure
40,715,537 54,287,382 55,644,567 55,644,567 M tCO2-eq
Lorentz Lowlands
0% Not measure
Not measure
30% 40% 41% 41%
Not measure
Not measure
12,231,438 16,308,584 16,716,299 16,716,299M tCO2-eq
Mappi-Bouven Digoel
0% Not measure
Not measure
30% 40% 41% 41%
Not measure
Not measure
10,132,229 13,509,638 13,847,379 13,847,379 M tCO2-eq
Sarmi 0% Not measure
Not measure
30% 40% 41% 41%
Not measure
Not measure
1,001,325 1,335,100 1,368,478 1,368,478 M tCO2-eq
Cyclops 0% Not measure
Not measure
30% 40% 41% 41%
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 37
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
Not
measure Not
measure 81,488 108,651 111,367 111,367 M
tCO2-eq
2. Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance (FACTS 4.8.1-26) (OUTCOME/IMPACT)
8.7 Million
Leuser 0 0 178,271 262,163 314,596 157,298 912,328
Katingan-Kahayan
0 0 416,571 612,604 735,125 367,563 2,131,863
Lorentz Lowlands
0 0 642,592 944,988 1,133,986 566,993 3,288,559
Mappi-Bouven Digoel
0 0 317,644 467,123 560,547 280,274 1,625,588
Sarmi 0 0 139,811 205,605 246,726 123,363 715,505
Cyclops 0 0 5,111 7,517 9,020 4,509 26,157
3. Percentage reduction in poaching in focus area (OUTCOME)
40% reduction
Leuser TBD Not measure
10% 20% 30% 40% 40%
Katingan-Kahayan
TBD Not measure
10% 20% 30% 40% 40%
Lorentz Lowlands
TBD Not measure
10% 20% 30% 40% 40%
Mappi-Bouven Digoel
n/a n/a n/a n/a n/a n/a n/a
Sarmi n/a n/a n/a n/a n/a n/a n/a
Cyclops TBD Not measure
10% 20% 30% 40% 40%
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 38
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
4. Number of public policies addressing climate change and/or biodiversity conservation introduced, changed or adopted consistent with citizen input (OUTCOME)
28 policies
Leuser 0 1 1 2 3 1 8
Katingan-Kahayan
0 1 1 2 3 1 8
Lorentz Lowlands
0 0 1 1 1 1 4
Mappi-Bouven Digoel
0 0 1 1 1 1 4
Sarmi 0 0 1 1 0 0 2
Cyclops 0 1 0 1 0 0 2
5. Number of sub-national government with improved licensing and permitting mechanism (OUTCOME)
14 sub-national governments
Leuser 0 0 1 1 1 1 4
Katingan-Kahayan
0 0 1 1 1 1 4
Lorentz Lowlands
0 0 1 1 0 0 2
Mappi-Bouven Digoel
0 0 0 1 1 0 2
Sarmi 0 0 1 0 0 0 1
Cyclops 0 0 0 0 1 0 1
6. Number of sub-national government incorporating high quality SEA-LEDS & LCPs into draft spatial plans, zonation regulation or draft
14 sub-national governments
Leuser 0 0 1 1 1 1 4
Katingan-Kahayan
0 1 1 1 1 0 4
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 39
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
mid-term development plan (OUTCOME) Lorentz
Lowlands 0 0 1 1 0 0 2
Mappi-Bouven Digoel
0 0 0 1 0 1 2
Sarmi 0 1 0 0 0 0 1
Cyclops 0 0 1 0 0 0 1
7. Number of Multi Stakeholder Forum (MSF) operational as citizen based mechanisms for public input on land use (OUTCOME)
14 MSFs
Leuser 0 1 1 1 1 0 4
Katingan-Kahayan
0 1 1 1 1 0 4
Lorentz Lowlands
0 0 1 1 0 0 2
Mappi-Bouven Digoel
0 0 0 0 2 0 2
Sarmi 0 0 1 0 0 0 1
Cyclops 0 0 0 1 0 0 1
8. Number of community champions engaged in advocacy interventions (OUTPUT)
500 community champions
Leuser 0 15 40 40 40 0 135
Katingan-Kahayan
0 15 40 40 40 0 135
Lorentz Lowlands
0 5 20 20 20 0 65
Mappi-Bouven Digoel
0 5 20 20 20 0 65
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 40
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
Sarmi 0 5 10 10 10 0 35
Cyclops 0 5 20 20 20 0 65
9. Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation (OUTPUT)
15,000 people
Leuser 0 300 1,500 2,000 1,300 500 5,600
Katingan-Kahayan
0 300 1,500 2,000 1,300 500 5,600
Lorentz Lowlands
0 150 500 500 500 200 1,850
Mappi-Bouven Digoel
0 50 200 300 300 200 1,050
Sarmi 0 50 100 100 0 0 250
Cyclops 0 150 200 100 100 100 650
10. Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes (OUTCOME)
6 CAs
Leuser See PIRS
Not measure
0 1 0 1 2
Katingan-Kahayan
See PIRS
Not measure
1 0 1 0 2
Lorentz Lowlands
See PIRS
Not measure
0 0 1 0 1
Mappi-Bouven Digoel
n/a n/a n/a n/a n/a n/a n/a
Sarmi n/a n/a n/a n/a n/a n/a n/a
Cyclops See PIRS
Not measure
0 1 0 0 1
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 41
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
11. Number of co-management agreements signed that secure community rights and benefits (OUTPUT)
40 agreements
Leuser 0 0 2 4 3 2 11
Katingan-Kahayan
0 0 2 3 4 1 10
Lorentz Lowlands
0 0 2 2 2 1 7
Mappi-Bouven Digoel
0 0 1 1 1 0 3
Sarmi 0 0 1 1 0 0 2
Cyclops 0 0 2 2 2 1 7
12. Number of people receiving USG supported training in natural resources management and/or biodiversity conservation (FACTS 4.8.1-27/OUTPUT)
10,000 people
Leuser 0 350 700 1,100 1,200 400 3,750
Katingan-Kahayan
0 350 700 1,000 1,200 400 3,650
Lorentz Lowlands
0 100 300 600 500 100 1,600
Mappi-Bouven Digoel
0 100 100 100 50 50 400
Sarmi 0 50 100 100 0 0 250
Cyclops 0 50 100 100 50 50 350
13. Amount of investment mobilized (in USD) for climate change as supported by USG assistance (FACTS 4.8.2-10/OUTPUT)
USD 20 Million
Leuser 0 500,000 1,500,000 1,000,000 1,000,000 1,500,000 5,500,000
Katingan-Kahayan
0 1,000,000 1,500,000 1,000,000 1,500,000 1,500,000 6,500,000
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 42
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
Lorentz Lowlands
0 500,000 1,000,000 1,200,000 1,500,000 1,000,000 5,200,000
Mappi-Bouven Digoel
0 0 0 600,000 350,000 500,000 1,450,000
Sarmi 0 0 0 200,000 300,000 500,000 1,000,000
Cyclops 0 0 0 0 350,000 0 350,000
14. Number of people receiving livelihood co-benefits (monetary or non monetary) (OUTCOME)
30,000 people
Leuser 0 0 2,500 2,500 3,000 3,000 11,000
Katingan-Kahayan
0 0 2,500 2,500 3,000 3,000 11,000
Lorentz Lowlands
0 0 1,000 1,000 700 700 3,400
Mappi-Bouven Digoel
0 0 200 200 300 300 1,000
Sarmi 0 0 300 300 0 0 600
Cyclops 0 0 1,000 1,000 500 500 3,000
15. Number of private sector firms that have improved management practices as a result of USG assistance (FACTS 4.6.2-9/OUTCOME) 10 private sector
firms
Leuser 0 0 0 0 1 0 1
Katingan-Kahayan
0 2 1 1 0 0 4
Lorentz Lowlands
0 0 0 0 0 1 1
Mappi-Bouven Digoel
0 0 0 1 1 1 3
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 43
INDICATOR TARGET OVER THE LIFE OF
PROJECT LANDSCAPE
ANNUAL TARGETS
B Y1 Y2 Y3 Y4 Y5 TOTAL
Sarmi 0 0 0 1 0 0 1
Cyclops n/a n/a n/a n/a n/a n/a n/a
16. Number of new USG-supported public-private partnerships (PPPs) formed (FACTS PPP 5) (OUTPUT)
20 PPPs
Leuser 0 1 1 1 1 0 4
Katingan-Kahayan
0 1 1 1 2 1 6
Lorentz Lowlands
0 1 1 1 1 0 4
Mappi-Bouven Digoel
0 0 1 1 1 1 4
Sarmi 0 0 1 0 0 0 1
Cyclops 0 0 0 1 0 0 1
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 44
3. PLAN FOR ACTIVITY MONITORING AND EVALUATION 3.1 LESTARI Learning Loop: Data Collection, Storage, Quality Assurance, Analysis, and Utilization Monitoring and evaluation (M&E) and reporting are essential parts of the LESTARI project.
The ability to rapidly collect, review, analyze, and communicate monitoring and evaluation
information is essential for maximizing LESTARI results. To facilitate this process, LESTARI
will develop a tailored Management Information System (MIS). The MIS will serve as an
accessible and transparent reporting tool to guide future activities and learning. It will also
connect the development of activity scopes of work to the LESTARI approval process and
link the outcomes of these activities to results as outlined in this AMEP as well as annual
work plans. To increase the efficiency, the MIS will also include features that allow
coordination of technical activities with necessary administrative and operations support.
Variable levels of MIS access will be provided to USAID and LESTARI partners for reporting,
operations support, and invoicing purposes. Progress captured in the MIS will be used to
facilitate LESTARI Adaptive Management approach.
DESIGN: The first step of the
project’s Learning Loop is design,
which includes building
consensus amongst project staff,
USAID, and partners on the
project’s theory of change, expected results, and
indicators. As described in Section 3.1, this
AMEP is founded upon the results of a
Measuring Impact Workshop, where LESTARI
staff and USAID collaborated in the
development of theories of change and
indicators for each LESTARI Strategic
Approach. The outcomes of this workshop were
presented to regional staff and partners in
breakout sessions during the first Annual Work
Planning Meeting, so that theories of change
could be further refined and used to develop
Landscape Level Work Plans.
Figure 3 LESTARI Adaptive Management Data Cycle
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 45
Step 1: DESIGN
USAID/Indonesia Construct Situation Model to understand project context
Develop Theories of Change for each Strategic Approach
Develop a Results Framework to show linkages between
Strategic Approaches and higher-level results
Identify performance indicators and targets.
Identify key evaluation questions.
Measuring Impact
LESTARI Team
Home Office M&E Specialist
Local Partners
COLLECT : Most M&E data will be collected using electronic data
collection form. During project startup, LESTARI Monitoring, Evaluation &
Learning (ME&L) Coordinator, with support from Tetra Tech’s Home
Office M&E Specialist, will use mobile data collection tools to create
customized electronic data collection forms (based on the Year 1 Work
Plan and AMEP) whereas data collected will be put into LESTARI MIS.
These electronic forms will include required fields, skips, and ranges to
improve standardization and data quality, as well as open-ended fields to
capture qualitative data.
It is important to note that changes in the methodology for collecting data may compromise
LESTARI’s ability to compare results over time. However, as the project continues to learn
how to improve data quality, changes to data collection tools or methods may become
necessary. In these cases, the ME&L Coordinator will be able to use the web-based
interface to update data collection forms. However, the ME&L Coordinator must document
these changes over time in revisions to the AMEP. Additional details on our specific data
collections methods for LESTARI indicators are outlined in Annex I: Performance Indicator
Reference Sheets (PIRS).
Step 2: COLLECT
LESTARI Staff (include Landscape M&E Specialists)
Collects data and data documentation for all project activities using
approved data collection forms and following the requirements outlined
in the PIRS
Submits data to ME&L Coordinator as data available
ME&L Coordinator Develops data collection forms
Collects additional qualitative data as necessary
Trains Technical Staff on data collection methods, standards, and
requirements
Trains Technical Staff on how to collect data using data collection
forms
Documents changes to data collection tools/methods resulting from
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 46
Step 2: COLLECT
ongoing learning
COP Provides support to the ME&L Coordinator in ensuring that data are
collected and submitted in a timely manner.
HO M&E Specialist Provides initial support and training in data collection form and
development of data collection tools.
REVIEW: Mobile data collection tool will improve data quality through
facilitating real-time reviews by the ME&L Coordinator. The ME&L
Coordinator will receive a notification each time new data are added to MIS
and will then review the quality of data against ADS 203 data quality criteria
(validity, reliability, timeliness, precision and integrity), and reassign the
form to the field-based data collector if corrections are required. If ongoing data quality
issues persist, the ME&L Coordinator will work with the CoP, Tetra Tech’s home office M&E
Specialist as well as USAID/Indonesia to develop new data collection strategies to
strengthen data quality. In addition to this continuous review of data quality, at least two
internal data quality assessments (DQAs) will be conducted by the Home Office M&E
Specialist during the project following the format outlined in Annex III: Data Quality
Assessment Checklists. The internal DQA described here is an internal quality control
mechanism and does not substitute for any formalized, USAID-initiated DQA of project data.
Step 3: REVIEW
Technical Staff & Landscape M&E Specialists
Makes necessary changes to incorrect/incomplete data reviewed by
ME&L Coordinator.
ME&L Coordinator Reviews all data against ADS 203 data quality standards
COP Ensures that at least two (2) internal Data Quality Assessments
(DQAs) are conducted during the project.
TT ARD Home Office M&E Specialist
Conducts two (2) internal DQAs during the project.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 47
MANAGE: Following the quality review, the Jakarta M&E Specialist will assist
ME&L Coordinator to enter disaggregated data into the LESTARI Management
Information System (MIS). For each data point entered, supporting
documentation (e.g., sign in sheets, photographs, meeting agendas, etc.) must
be provided.
Step 4: MANAGE
Jakarta M&E Specialist Inputs data into MIS
Ensures each data point is supported by documentation
Files hard and electronic data documentation
Maintain MIS in day to day basis
ANALYZE: On a quarterly basis, LESTARI’s ME&L Coordinator will
review disaggregated data to assess progress toward targets and identify
implementation challenges. When analyzing LESTARI data, the ME&L
Coordinator will consider both INTERNAL and EXTERNAL factors that
may be influencing results. Internal factors include challenges or
opportunities related directly to project implementation (for example, staff
turnover) while external factors refer to those outside of the control of
LESTARI staff and USAID (for example, negative factors might include political instability or
natural disasters; positive factors might include new opportunities for collaboration). An
important part of the analysis step is not only providing data to explain the “what,” but also to
provide context to explain the “why.” This context is extremely important in informing how
LESTARI may need to adjust activities. It will also be beneficial to future USAID projects, as
they determine what worked in the past, and why.
Step 5: ANALYZE
Technical Staff & Landscape M&E Specialists
Work with the ME&L Coordinator to contextualize the results: What
other INTERNAL and EXTERNAL factors (risks and challenges, or
enabling events or opportunities) may be influencing our results?
ME&L Coordinator Assesses progress toward targets
Analyzes quantitative and qualitative data to tell the story of project
successes, challenges, and lessons learned.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 48
REPORT AND COMMUNICATE RESULTS: Once data have been analyzed,
they will be ready for reporting to USAID/Indonesia. The M&E Sections of reports
will include:
Any M&E-related activities (e.g., trainings, data collection activities, internal data quality assessments and findings) conducted that quarter;
An indicator table(s) showing quantitative results achieved that quarter (targets vs. actuals);
A narrative describing overall analysis of project achievements, lessons learned, and implications for implementation;
Project Success Stories, including photographs and quotes from beneficiaries (starting in year two);
Discussion of any data limitations, reporting challenges, and proposed changes to the AMEP.
The findings of any special studies/analyses performed to inform project learning.
While the minimum requirement is an indicator table showing disaggregated actuals against
targets for each indicator, it is sometimes difficult for a reader to easily assess what is/isn’t
working in this format. The goal of the report is to provide a clear picture of what the project
has achieved, how much the project has achieved, and why the project has or has not met
its goals. Whenever possible, the project will use visual representations of data (such as pie
charts, bar graphs, or line graphs) to report project results.
In addition to reporting results to USAID, the ME&L Coordinator will support the development
and dissemination of Success Stories and Lessons Learned with LESTARI partners and
stakeholders at the national and regional levels during annual meetings.
Step 6: REPORT
Technical Staff & Landscape M&E Specialists
Provides any additional clarification/information requested by ME&L
Coordinator for inclusion in M&E Sections of Quarterly/Annual Reports
ME&L Coordinator Drafts M&E sections of Quarterly/Annual reports
Assists with development of Success Stories (beginning in Year 2)
COP Reviews M&E Sections of Quarterly/Annual reports for quality prior to
reporting to USAID.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 49
LEARN: A critical step in the adaptive management process is learning.
While the process described above will promote learning throughout all
phases of the project, the ME&L Coordinator will facilitate a formal learning
process at Annual Strategic Reviews. Following annual internal evaluations
conducted by the ME&L Coordinator, Strategic Reviews will seek the
participation of project staff, partners, and USAID to review evaluation data and findings to
address the following questions:
Which activities were successful, and why?
Which activities fell short of their anticipated results? Why?
How could any activity be redesigned to increase efficiencies, effectiveness, scope,
and satisfaction?
How have activities had a positive effect on women and other
vulnerable/disadvantaged groups?
Team members will also review key aspects of the AMEP (theories of change, indicators and
targets) and consider whether these remain aligned with project activities. If the process
reveals misalignment between activities, indicators and expected results; changes in initial
assumptions (or causal logic) supporting the theory of change, unanticipated factors
influencing achievement of results, or changes in scope, elements of the AMEP may need to
be refined or redesigned.
Table 5 Annual Strategic Reviews
Question Response
Decisions Made/
Actions Required
Items Not Resolved from
previous period- why?
Which activities during previous quarter were successful, and why?
What activities were planned but didn’t occur?
If they did not occur, why?
For activities planned but did not occur, when will they be re-programmed?
Are we achieving our performance indicators targets for key performance indicators?
If we are not, why?
What activities can be planned for the following year to meet our performance indicator targets?
Are our target assumptions still valid? If not, why?
Which activities are falling short of their anticipated
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 50
Question Response
Decisions Made/
Actions Required
Items Not Resolved from
previous period- why?
results and why?
Is the causal logic of the theories of change still valid? If not, why?
Are our assumptions related to our ability to implement Strategic Approaches and meet our performance indicators targets still valid?
If not, what has changed?
What effects do these changed assumption(s) have on the project?
What is our strategy to address any changed assumption(s)?
Are there any “red-flags” if so; are there any corrective actions that need to be taken?
Step 7: LEARN
Technical Staff & Landscape M&E Specialists
Participate in Annual Strategic Reviews
Contribute useful contextual information that contributes to the
learning process
ME&L Coordinator Facilitate learning sessions at Annual Strategic Review
Document and disseminate the findings to project staff
Include the findings in the next Quarterly/Annual Report
COP Ensure that findings from learning forums are included in annual work
plans and are considered in the development of activities.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 51
REFINE: If the learning process results in recommended changes to the
AMEP, the ME&L Coordinator (under the guidance of the COP), will
document any major changes in the Executive Summary of the revised
AMEP along with the rationale for these adjustments. Major revisions could
include changes to frameworks, indicators, or the PIRS. If minor AMEP
elements change, the AMEP will be updated to reflect these changes. While
required changes to the AMEP may become apparent at any point in the project cycle, the
project must continue to follow the standards and methods outlined in the current AMEP until
the revised document is officially approved by USAID. Once approved, the project may then
follow the new AMEP standards and protocols throughout the adaptive management
process.
Step 8: REFINE
Technical Staff & Landscape M&E Specialists
Review AMEP elements
Recommend changes to AMEP based on project learning
Recommend changes to Work Plan and/or activity implementation
based on project learning
ME&L Coordinator Review AMEP elements
Recommend changes to AMEP based on project learning
Recommend changes to Work Plan and/or activity implementation
based on project learning
Make necessary changes to the AMEP
COP Review and approve recommended changes to AMEP prior to
submission to USAID
Submit revised AMEP/Work Plan to USAID.
3.2 Internal Evaluations Plan Evaluation is an applied inquiry process for collecting and synthesizing evidence that
culminates in conclusions about the state of affairs, value, merit, worth, significance, or
quality of a program, product, person, policy, proposal, or plan. Conclusions made in
evaluations encompass both an empirical aspect (that something is the case) and a
normative aspect (judgement about the value of something). It is the value feature that
distinguishes evaluation from other types of inquiry, such as basic science research, clinical
epidemiology, investigative journalism, or public polling (Fournier, 2005, p.140)
In accordance with the 2011 USAID Evaluation Policy, LESTARI will use evaluation
methodologies with the dual purposes of accountability to USAID, government partners and
other stakeholders; and learning about key results to improve effectiveness. In the simplest
terms, evaluations planned for LESTARI will answer three questions: What? So what? Now
what? The answers of these questions will not only aim to understand project effectiveness
but also to generate new knowledge for the wider biodiversity and climate change
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 52
community. It is worth noting that evaluation for LESTARI is not an end in itself. The purpose
is to inform thought and action. Moving from what, to so what, to now what means moving
from data to interpretation to action.
Regular monitoring of LESTARI performance indicators will provide the project with the
answers of what questions as well as demonstrate accountability. However, several key
impact questions will need to be assessed to answer the so what question and track
changes over time so as to LESTARI can determine its effectiveness. An example for this
case could be: a national park has increased its METT score to 70 point (what), but does this
score reflect on improved biodiversity condition? (so what). If yes, what can we learn to
replicate the success (now what). If not, how can we modify our approach? (now what).
Internal evaluations for LESTARI will be designed in a way that meet the principles of USAID
evaluation policy (i.e., unbiased, relevant, based on the best methods, transparent, high
utility) and address the most important questions about project impact, how LESTARI is
impacting the direct threats and the key species of interest. Tools, methods and design
for evaluating LESTARI performance indicators will be developed and agreed upon in a
consultation with LESTARI COR on a timeframe that allows for LESTARI interventions to
demonstrate impact. These evaluations will be performed during mid and end of project life,
and timed in order to ensure that they are completed before and thus can contribute to
external mid-term and final evaluations. The SOW for the mid-term internal evaluation will be
developed with the USAID COR by the end of the second quarter of Year 2, and the
evaluation will be completed and written-up by the end of Year 2. The SOW for the final
internal evaluation will be developed with the USAID COR by the end of the first quarter of
Year 5, and the evaluation will be completed and written-up by the end of the third quarter of
Year 5. In both cases, adjustments could be made to ensure these internal evaluation
documents are available in time for external evaluations.
Table 6 presents the illustrative evaluation tools, suggested methods and designs relate to
the evaluation questions posed. The proposed key evaluation questions (KEQs) are selected
to assess the validity of assumptions illustrated in LESTARI Results Chains (Annex II).
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 53
Table 6 Illustrative summary of tools, methods and design implications for proposed evaluation questions
Key Evaluation Question Suggested tools, methods, and designs Timing of Data Collection
To what extent have LESTARI interventions
been effective in reducing biodiversity threats
(e.g., poaching, fragmentation of key habitat)
and positively impacted populations of key
identified species?
Collecting data from population survey done by WCS,
WWF, or other organization (e.g. BOSF). Perform
statistical analysis to determine a causal relationship
between LESTARI interventions and changes in
populations. When data on population is not available,
LESTARI may want to consider funding population
surveys for targeted key and indicator species (e.g.,
tiger and orangutan).
Mid-term (2017) and End of Project (2020)
Note:
Population survey is available every three years
depend on the key species assessed (e.g. a
WCS population survey for tiger will be
conducted in 2016).
Impact evaluation – detailed land use and land use
change (LULUC) assessments to determine the impact
of LESTARI interventions on key habitat.
To what extent has improved collaboration
between government, private sector and
community contributed to improved forest
management?
Are economic benefits sustained by
LESTARI beneficiaries –government,
private sector and community?
Are economic benefits sufficiently
incentivizing those beneficiaries to
minimize their encroachment of forests?
Cost-benefit analysis Mid-term (2017) and End of Project (2020)
To what extent have LESTARI intervention
been effective in improved land-use
governance?
To what extent has the project addressed
barriers of political will and buy-in at the
national and regional levels?
A combination of tools such as:
Assessing land cover and policy changes
Stories of change – a case study method to determine pathways of success
Bellwether method – an interview method to determine an issue’s position on the policy agenda
Stakeholder analysis – method to determine which stakeholders are invested in the intervention
Stories of change, bellwether method, and
stakeholder analysis will be used in regular basis
(rolling as data available) to monitor the process
of policy influence.
The data collected from the tools will feed into
QCA and Contribution Analysis which will be
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 54
Key Evaluation Question Suggested tools, methods, and designs Timing of Data Collection
What changes have been made to land use policy as result of advocacy interventions and strengthening citizen based mechanism input?
Have beliefs, opinions, and behaviours of stakeholders changed as result of advocacy and campaign activities?
Have improved land use governance impacted on reduction in large scale and small scale deforestation and degradation?
Qualitative Comparative Analysis (QCA)4 and other theory-based evaluation design, e.g. contribution
analysis5 will be used to gain a better understanding of project attribution. QCA is a middle way that combines certain features of the qualitative approach (case orientation, interest in complexity) with those of quantitative research (interest in generalization). An example of QCA utilization can be seen further in CIFOR research that aims to compare national REDD+ policy processes in 12 countries
(http://www.cifor.org/library/4278/qualitative-comparative-analysis-qca-an-application-to-compare-national-redd-policy-processes/).
used in mid-term and end of project evaluation.
Knowledge generation: enhance general
understandings and identify generic principles
about effectiveness
Meta-analyses
Lessons learned
Case studies
Mid-term and end of project year
4 QCA is an alternative approach to examining contributory causes that uses a comparative framework perspective. QCA provides a credible causal claim that the intervention is a contributory
cause. QCA can answer the question ‘Did it make a difference?’, but not the question ‘Why has the result occured?’ 5 Contribution analysis is one of theory-based approaches to evaluation. Contribution analysis confirms: a) that the expected result occured; b) that the causal package is sufficient; c) that the
intervention is a necessary part of the causal package.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 55
3.3 Schedule for Performance Monitoring Tasks Table 7 Key Performance Monitoring and Evaluation Tasks and Timeline
Tasks FY 2015 FY 2016 FY 2017 FY 2018 FY 2019 FY 2020
Responsible
Quarter: Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3
Submit Draft/Final AMEP
● ● M&E
Develop MIS and train staff
● ● M&E/HO M&E
Draft M&E Section of Quarterly Report
● ● ● ● ● ● ● ● ● ● ● ● ● ● M&E
Support development of Success Stories
● ● ● ● ● ● ● ● M&E
Draft M&E Section of Annual Report
● ● ● ● M&E
Conduct Internal Evaluation
● ● ● ● ● M&E
Facilitate M&E Session of Strategic Annual Review
● ● ● ● ● M&E
Revise AMEP ● ● ● ● ● ●
M&E
Draft M&E Session of Final Project Report
● M&E
Assess Data Quality
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● M&E
Conduct internal DQA
● ● ● M&E/HO M&E
Review Internal Performance
● ● ● ● ● ● ● ● ● ● M&E
Hold Adaptive Management Meeting
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● M&E
LESTARI Collects Data
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ALL
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 56
3.4. Monitoring and evaluation structures, functions, and capabilities Monitoring, evaluation, and learning are core priorities across LESTARI’s technical and
administrative positions. While the ME&L Coordinator (under the direction of the COP) will
hold primary responsibility for developing and implementing the M&E system, all LESTARI
staff must participate in monitoring and evaluation in order for the system to be successful.
This section provides a general description of the M&E roles and responsibilities for
LESTARI staff. Specific M&E roles and responsibilities for each indicator can be found in the
PIRS in Annex I.
Table 8 M&E Roles and Responsibilities
M&E Contributor Roles and Responsibilities
ME&L Coordinator
Provide professional guidance and hands-on assistance to all LESTARI
staff on the LESTARI AMEP; MIS; USAID ADS 203 guidelines; and data
collection techniques, standards, and best practices.
Provide training and technical assistance to local partners in
performance monitoring and information sharing.
With assistance from the HO M&E Specialist, develop and maintain the
LESTARI MIS, including standardized data collection forms, cloud-based
M&E database, and a Performance Monitoring Dashboard.
Assess the quality of collected data and provide feedback to parties
responsible for data collection.
Draft and submit M&E sections of quarterly and final reports to the COP
for submission to USAID.
Conduct special studies and analyses as necessary to inform project
learning.
Lead Quarterly Results Reviews.
Support development of Success Stories and other communications and
outreach materials.
Work with the DCOP and regional and technical staff to ensure that M&E
data are informing LESTARI management and decision making.
Conduct regular data verification activities.
Conduct annual internal evaluations.
Share internal evaluation findings and facilitate review and feedback
sessions with stakeholders at Annual Strategic Reviews.
Facilitate internal impact evaluations, and dissemination of findings.
Jakarta M&E Specialist
Provide inputs to ME&L Coordinator in the development of LESTARI
AMEP; standardized data collection forms; and MIS.
Maintain MIS operation in daily basis and train staff on its use.
Supports Jakarta technical staff to perform their monitoring tasks and
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 57
M&E Contributor Roles and Responsibilities
responsibilities.
Maintain up-to-date hard and electronic files of all data and
documentation.
Work with Landscape M&E Specialists to ensure timely reporting on
LESTARI activities.
Landscape M&E Specialists
In the activity design level, provide hands-on assistance to regional team
to develop activity logic align with project level ToC.
Train LESTARI regional team, grantees, and sub-contractors to
accurately and reliability collect data, ensure that data collection forms
and templates are used correctly.
Verify and validate all data including reports from grantee and
subcontractors for submission to ME&L Coordinator.
Provide inputs to ME&L Coordinator for quarterly and final reports.
Under guidance of ME&L Coordinator, conduct special studies and
analyses as necessary to inform project learning.
Provide inputs to ME&L Coordinator for the development of Success
Stories and other communications and outreach materials.
Work with the regional team to ensure timely reporting on LESTARI
activities.
Under guidance of ME&L Coordinator, conduct regular data verification
activities.
Facilitate review and feedback sessions with regional team at quarterly
strategic reviews.
Regional Coordinators/
Technical Staff
Work with the M&E team to gain consensus on how LESTARI activities
will contribute to indicators and objectives.
Collect data and data documentation (photos, GPS coordinates,
agendas, sign in sheets, etc.) using data collection form.
Provide recommendations to the M&E Team on any needed changes to
the AMEP or M&E process, such as revisions to data collection forms,
methods, indicators, or targets.
Participate in forums (Quarterly Results Reviews; Annual Strategic
Reviews) intended to facilitate learning from project data.
Incorporate findings from learning forums into future activities and work
plans.
DCOP The DCOP will be responsible for ensuring that project activities contribute to indicator targets and expected results, are gender-responsive, and that M&E data and findings are used to inform planning and decision making.
Tetra Tech Home Office M&E Specialist (Robynne
In Year 1, Tetra Tech’s home office M&E Specialist, Ms. Robynne Locke, will be responsible for assisting with development of the AMEP and providing training to the M&E team on data collection form development and AMEP
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 58
M&E Contributor Roles and Responsibilities
Locke) operationalization. Thereafter, she will provide training and technical assistance to the M&E team as needed. In addition, she will be responsible for conducting at least one internal DQA within the first two years of the project.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 59
ANNEX I: PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS)
INDICATOR 1
GOAL Reduced GHG emissions from land-use sector
RESULT KR 1. At least 41% of total CO2-equivalent emissions reduced from land use, land use change and deforestation averaged across all landscapes within the project scope
INDICATOR TITLE
Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL
(For the purpose of FACTS reporting, quantitiy of GHG emissions will be reported in metric tons of CO2equivalent)
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator – contributes to FACTS 4.8-7
PRECISE DEFINITION
This indicator reports the percentage of GHG emissions reduced, sequestered, and/or avoided, as a result of USG activities, as compared to a baseline level of GHG emissions. The baseline is the “business-as-usual” reference for GHG emissions that would have occured during the reporting period if there had been no USG intervention. At least 41% of total CO2-equivalent emissions will be reduced from land use, land use change and deforestation averaged across all landscapes within the project scope, based on the use of the IPCC Good Practice Guidance for Land Use, Land-Use Change and Forestry, and taking into consideration appropriate relevant national or subnational methods.
Reduced emission will be achieved through LESTARI direct assistance, originating from reducing forest encroachment and improving peat land management with communities and protected area authorities, developing effective communities conservation agreements and better agricultural practices, engaging the private sector to implement best management practices in their concessions, and governance advocacy in changing licensing and embracing sustainable development visions that will stop deforestation.
It is noted that for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent.
UNIT OF MEASURE Percentage (and metric tons of CO2equivalent)
DISAGGREGATIONS Landscape
RATIONALE OR JUSTIFICATION FOR
Indicator used to track progress on reducing emissions from land use sector. The indicator is in line with GoI target to reduce emissions nationally as outlined in
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 60
INDICATOR 1
INDICATOR National Action Plan for Greenhouse Gas Emissions (RAN-GRK).
DESIRED DIRECTION
Decreasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Reductions in deforestation and degradation will lead to reduction in GHG emissions. Reducing, sequestering, or avoiding GHG emissions will slow the rate of climate change and reduce the impacts.
DATA COLLECTION APPROACH
DATA COLLECTION
METHOD
The data collection methodology for calculating reduced emissions is described in the Landscape Baseline Analysis (LBA) report. LESTARI has adopted the exact same methodology of GoI for calculating land-based emissions in an attempt to gain the maximum level of buy-in from the local stakeholders, especially the government. Detailed methodology can be found in the BAPPENAS Technical Guide for Land-Based GHG Emission and Sequestration Baseline Calculations6 that is followed by the RAN/D-GRK. The methodology applied to LESTARI that works in the forestry and land-use sector is based on:
Living biomass (above ground) GHG emissions
Below ground GHG emissions from the decomposition of organic soils (peat)
GHG emissions from organic soils through land fires
The REDD Abacus SP program developed by World Agroforestry Center (ICRAF) is used to calculate land cover changes and related emissions. This includes below ground emission estimates on peat due to land use.
It is acknowledged in the LBA that the methodology may require revision or adaptation going forward. Any changes in the calculation methods for land-based emissions across LESTARI landscapes will be presented in the LESTARI Landscape Baseline Analysis (LBA) for approval by USAID.
DATA SOURCE The current methodology uses land cover data from Ministry of Environment and Forestry and MODIS hotspot (subject to change).
FREQUENCY AND TIMING OF DATA ACQUISITION
Annual
ESTIMATED COST OF DATA ACQUISITION
Partial cost of remotely sensed imagery allocated to this activity, ground check, staff time for data analysis
INDIVIDUAL RESPONSIBLE
Spatial Planning and GIS Specialist; Deputy Chief of Party
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
6 F. Agus, I. Santosa, S. Dewi, P. Setyanto, S. Thamrin, Y. C. Wulan, F. Suryaningrum (eds.). 2013. Pedoman Teknis
Penghitungan Baseline Emisi dan Serapan Gas Rumah Kaca Sektor Berbasis Lahan: Buku I Landasan Ilmiah. Badan
Perencanaan Pembangunan Nasional, Republik Indonesia, Jakarta.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 61
INDICATOR 1
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
The methodology for estimating emissions from fire is not universally
accepted due to the reliability of data used in calculating emissions. The
intensity and duration of fires, depth of peat, water table depth, drought
conditions and vegetation cover all affect the potential emissions from peat.
With all of these variables, it is unlikely that a single method for calculating
emissions from fire will ever be available.
The baseline data used in the analysis of deforestation and degradation and
associated GHG emissions uses Ministry of Environment and Forestry
(MOEF) wall-to-wall land cover (penutupan lahan) data that has
questionable accuracy in some parts of LESTARI landscapes – especially
Papua. GIS operators have frequently misclassified land use types and this
is the most problematic.
Another issue that needs some effort in rectifying and aligning is the
difference between peatland data (sourced from Soil Research Centre,
Ministry of Agriculture) and land cover (MOEF). For instance, the incidence
of dry land forest types occuring on peat in questionable. It is hard to state
which data set is more accurate. At the scale of LESTARI landscape, this is
not a major issue for the project, but administrations wishing to report on
their site level emissions reductions should seek accurate data. Where
unlikely combinations of land cover types and soils do not align, estimation
of emissions should be treated with caution.
Emissions reductions are calculated using a tool that relies on data of
carbon stocks, averaged across the country. If the government figures for
carbon stocks change (very likely within LOP) or Tier 2 or 3 values are used
to calculate a more accurate REL (baseline) then the tons may go up or
down. So depending on the methodology used the actual amount (metric
tons of CO2e) may go up or down. This makes setting LOP target in exact
figure of metric tons of CO2e is not feasible.
ACTION TAKEN TO
ADDRESS
LIMITATIONS
To calculate emissions from fire, compromises must be made and average
emissions from areas using available monitoring data have been applied to
develop the LESTARI baseline. This is in line with provincial government
approaches. Forest and land fire is a significant issue in Central Kalimantan
and LESTARI has applied methodology adopted by the Central Kalimantan
Province.
Ground check and verification with stakeholders (especially established
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 62
INDICATOR 1
Spatial Data Infrastructure groups and networks that have been established
through USAID IFACS) will be undertaken to ensure accuracy of land cover
data.
LESTARI should seek to obtain accurate peat distribution data and redefine
estimated emissions.
LESTARI will use percentage of reduction as project target, but will provide
information in metric tons of CO2-equivalent to serve USAID global reporting
purpose.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS Data analysis will be done by Spatial Planning and GIS Specialist with assistance from Winrock following USAID approved methodology
PRESENTATION OF DATA
Tabular by landscape
Landscape Target Achievement % Completion
xx % xx %
xx tCO2-eq xx tCO2-eq
REVIEW OF DATA Annually by ME&L Coordinator
REPORTING OF DATA
Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
LESTARI Landscape Baseline Analysis report. Baseline calculated following GoI methodology for Land-Based GHG Emission and Sequestration Baseline Calculations.
BASELINE NOTES The methodology used to calculate the baseline for LESTARI is based on the national approach. It is expected that this national level methodology will be improved in the future. For the LESTARI project, methodology used must evolve for the simple reason of delivering achievement at the end of the project. The data used for this methodology (land cover based on Landsat Imagery and developed by the MoEF) is normally available to the public at least 2 years after it was captured. Therefore, a methodology that can utilize up-to-date data taken within the last year of the project must be sought.
BASELINE VALUE
LESTARI applied the base years of 2006 – 2013. The table below presents a summary of annual deforestation and degradation rates and land-based GHG emissions per landscape.
Landscape Annual Deforestation
(ha/yr)
Annual Degradation
(ha/yr)
Net Annual GHG emissions/REL
(tCO2-eq)
Leuser 8,039 0 6,531,315
Katingan- 34,136 5,236 42,630,563
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 63
INDICATOR 1
Kahayan
Lorentz Lowlands 2,551 15,513 11,407,378
Mappi – Bouven Digoel
1,118 29,275 9,915,365
Sarmi 353 4,560 996,222
Cyclops 144 11 83,249
BASELINE YEAR 2006 - 2013
TARGETS NOTES The target set for LESTARI averaged across the landscapes is a 41% reduction from the baseline emissions by 2020. LESTARI proposes a progressive approach—increasing the reductions of to reach this target by 2020. Annual emissions will be reduced progressively by 10%, 20%, 30%, 40% and 41% each year of the project. But emissions reductions will only start to be reported in the third year of the project.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 Not measured Not measured
FY 2017 Not measured Not measured
FY 2018 30% reduction 41 M tCO2-eq % reduction M tCO2-eq
FY 2019 40% reduction 68 M tCO2-eq % reduction M tCO2-eq
FY 2020 41% reduction 96 M tCO2-eq % reduction M tCO2-eq
Targets LOP 41% reduction 96 M tCO2-eq % reduction M tCO2-eq
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: November 30th, 2015
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 64
INDICATOR 2
GOAL Carbon rich forest, peatland, mangrove ecosystems and the habitat of key species maintained
RESULT KR 2. At least 8.42 million hectares of primary or secondary forest, including orangutan habitat, under improved management
INDICATOR TITLE Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE FACTS 4.8.1-26
PRECISE DEFINITION
“Improved natural resource management” includes activities that promote enhanced management of natural resources for one or more objectives, such as conserving biodiversity, sustaining soil or water resources, mitigating climate change, and/or promoting sustainable agriculture. Management should be guided by a stakeholder-endorsed process following principles of sustainable NRM and conservation, improved human and institutional capacity for sustainable NRM and conservation, access to better information for decision-making, and/or adoption of sustainable NRM and conservation practices.
For LESTARI, an area is considered under "improved management” when any one of the following occurs:
1. Co-management plan among Provincial and District Government Community, and Forest Management Unit (FMU) are in place and under implementation.
2. FMUs assisted by LESTARI operational.
3. Certain area set-aside for conservation as result of SEA-LEDS and/or LCP recommendations into local development plans.
4. An increased in the Management Effectiveness Tracking Tool (METT) score for Protected Areas (PAs).
5. Conservation Management and Monitoring Plan (CMMP) or other conservation plan within private sector natural resources concessions are in place and under implementation as evidenced by incorporation in business SOP, company budget and monitoring reports (or through sustained certification audits implemented by a third party).
6. Adaptive management is demonstrated; or on-the-ground management impacts are demonstrated (eg. illegal roads closed, snares removed, no-fishing zones demarcated).
Reported as total number of hectares improved during the fiscal year in question, which can include maintained improvement in previously reported hectares and/or new, additional hectares. Improved management should be reported for activities where the LESTARI supported program was plausibly linked to the improvements observed.
UNIT OF MEASURE Hectares
DISAGGREGATIONS
Areas under improved management will be disaggregated by landscape following further disaggregation below:
Biologically significant areas = areas identified as important for biodiversity through national, regional, or global priority-setting processes. For LESTARI, these
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 65
INDICATOR 2
areas will include:
Orangutan habitats (note: if orangutan habitats are inside conservation areas, the hectares will be excluded from hectares of conservation areas under improved management to avoid double counting)
HCV in concession
Conservation Areas (e.g., national parks, wildlife reserve)
All other areas = areas with natural resources which are outside of biologically significant areas and targeted for management interventions with non-biodiversity funds. These may include areas characterized by forest production, watersheds, areas with tree crop or agroforestry systems, peatlands (high carbon stock), forested areas within and adjacent to private sector concessions, etc.
Areas under IFACS and new areas under LESTARI.
RATIONALE OR JUSTIFICATION FOR INDICATOR
Measures of this indicator demonstrate progress towards sustainable natural resources governance and institutions, and can inform adaptive management of programs. This indicator is a reliable annual measure that demonstrates the magnitude of USG investments in biodiversity conservation and other natural resource sectors.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
A spatial indicator is an appropriate measure of the scale of impact of biodiversity conservation and/or NRM interventions. Good management of natural resources is a prerequisite for achieving improved biophysical condition of natural resources and contributes to reduction in deforestation and degradation.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Field staff will compile for review implementation of conservation management plans (PA and non-PA), review district and provincial spatial plans and land use policy process (to identify changes in legal status), site-observation. The GIS team will conduct geospatial analysis of remotely sensed imagery and GIS layers.
DATA SOURCE GIS, conservation management plans, site-visit reports, land cover maps from Ministry of Environment and Forestry (MoEF), spatial plans, procurement of remotely sensed imagery (when feasible).
FREQUENCY AND TIMING OF DATA ACQUISITION
Annual
ESTIMATED COST OF DATA ACQUISITION
Partial cost of remotely sensed imagery allocated to this activity, ground check, staff time for data analysis and interpretation, and cost of photocopies for conservation management plans and spatial plans.
INDIVIDUAL RESPONSIBLE
Spatial Planning and GIS Specialist; Component Team Leaders
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 66
INDICATOR 2
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Precision is low: “improved management” is a relative term, and narrative is required to explain the quality of this management improved. Equal weight is given to unequal improvements along a continuum: eg. creating, adopting and implementing management plans may each be an improvement over a baseline. Likewise, a small management improvement across a large area may be as important as a large improvement across a small area. Conservation plans may not be well described spatially.
ACTION TAKEN TO ADDRESS LIMITATIONS
Conservation plans will be interpreted by trained LESTARI staff. Locations of all activity areas carried out by partners will be located by LESTARI field staff using GPS and recorded into electronic data form. Benchmark that are being used within the project to gauge success will be articulated clearly and a short narrative to describe the benchmarks that have been reached in the past year will be provided.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Field staff will compile for review conservation management plans, maps and district spatial plans. The GIS team will conduct geospatial analysis of remotely sensed imagery and GIS layers. Landscape hectare totals will be aggregated into a tabular format, with accompanying narrative. The link to USG assistance will be established through periodic assessments conducted by LESTARI staff.
For those areas already improved under IFACS, LESTARI will report these areas as project achievement should there is new level of improvement made under LESTARI. Narratives will be provided to describe new level of improvement. Data presentation will be disaggregated between improved areas under IFACS (e.g., concession area, area under Community Conservation and Livelihood Agreements, etc) and new areas under LESTARI.
Analysis will also be made to present data on orangutan habitats. If orangutan habitats are inside conservation areas, the hectares will be excluded from hectares of conservation areas under improved management to avoid double counting.
PRESENTATION OF DATA
Tabular
Landscape Type of Area Forest Management Type
Location Target Achievement % Completion
Biologically significant areas (divided into areas under IFACS and new areas
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 67
INDICATOR 2
under LESTARI):
Orangutan habitat
HCV in concession
Conservation Areas
All other areas (divided into areas under IFACS and new areas under LESTARI):
REVIEW OF DATA Annually by ME&L Coordinator
REPORTING OF DATA
Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
LESTARI Landscape Baseline Analysis report, METT baseline report, existing spatial planning and project activity progress report to determine the level of current management.
BASELINE NOTES Project activities to improve current management will be started in FY 1, therefore baseline for this indicator is assumed as zero because no improved management yet observed. Analysis of existing condition will include business-as-usual scenario in order to be able to attribute to USG assistance.
BASELINE VALUE 0
BASELINE YEAR 2015
TARGETS NOTES Hectares under improved management will only start to be reported in the second year of the project with the assumption that sufficient time is allowed for impact to occur.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 Not measure
FY 2017 1.7 M
FY 2018 2.5 M
FY 2019 3 M
FY 2020 1.5 M
Targets LOP 8.7 M
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 68
INDICATOR 2
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: January 15, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 69
INDICATOR 3
GOAL Reduced threats on key species
RESULT IR 1. Improved forest management
INDICATOR TITLE Percentage reduction in poaching in focus area
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
Poaching is the illegal hunting, killing or capturing of animals, a practice that occurs in a variety of ways. Poaching can refer to the failure to comply with regulations for legal harvest, resulting in the illegal taking of wildlife that would otherwise be allowable. Examples include: hunting, killing or collecting wildlife that is listed as endangered by IUCN and protected by law such as CITES, taking without a license or permit, use of a prohibited weapon or trap, taking outside of the designated time of day or year, and taking of a prohibited sex or life stage. Poaching can also refer to the taking of animals from a gazzetted wildlife sanctuary, such as a national park, game reserve, or zoo. Wildlife biologists and conservationists consider paoching to have a detrimental effect on biodiversity both within and outside protected areas as wildlife populations decline, species are depleted locally, and the functionality of ecosystem is disturbed.
Focus area are defined as the area within protected areas targeted for regular patrols due to the high level of threats (e.g., accessibility). Patrols are targeted and not all areas need to be patrolled. To monitor a reduction in poaching, LESTARI will use SMART (Spatial Monitoring and Reporting Tool). SMART is designed to improve anti-poaching efforts and overall law enforcement effectiveness in established conservation area management zones. SMART enables the collection, storage, communication, and evaluation of data on: patrol effort (e.g. the time spent on patrols, effective area patrol and the distance of patroling), patrol results (e.g. number of snares removed, arrests made), and threat levels (e.g. the number of paths into the protected areas, areas deforested or number of tress felled illegally).
UNIT OF MEASURE Percentage
DISAGGREGATIONS Conservation Area
RATIONALE OR JUSTIFICATION FOR INDICATOR
Indicator used to track increased commitment of key stakeholders regarding the protection of key species and conservation of their habitat.
DESIRED DIRECTION
Decreasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
When effectively monitored and enforced, reduce in poaching can help to substantially improve protection of wildlife and their habitats.
DATA COLLECTION APPROACH
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 70
INDICATOR 3
DATA COLLECTION METHOD
Review regular patrol reports and patrol maps (monthly or quarterly); undertake in-depth data analysis with an evaluation of various trends in patrol perfromance and threat-levels, as well as an evaluation of the entire patrol management system (annually).
DATA SOURCE Patrol reports; direct observation; annual evaluation
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling as data available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; monitoring and evaluation cost
INDIVIDUAL RESPONSIBLE
Biodiversity Conservation Advisor
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Using the SMART software and establishing a patrol database will not, on its own, improved protection of a conservation area. In addition to the SMART software and database, basic enforcement capacity and infrastructure must be in place.
ACTION TAKEN TO ADDRESS LIMITATIONS
LESTARI will combine the utilization of SMART with introduction of adaptive patrol management practices through promotion of feedback mechanisms between managers and rangers to discuss patrol efforts and results and set new patrol targets. This intervention is closely linked with METT intervention as to support overall improvement of conservation area management capacity.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
SMART allows the calculation of poaching incidence, per unit of effort – such as the number of snares removed per kilometer patrolled in high risk areas, and therefore can be used to show a change in paoching incidence even if the number of patrols varies over time. During the first 6 months, SMART patrols and observation in the field will identify areas of high risk, and develop a baseline of the average number of incidents per kilometer traveled on foot patrols. The base
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 71
INDICATOR 3
unit will be no. incidents / km measured semi-annually.
Application of effort-related base units is due to the fact that the number of poaching incidents is correlated with patrol effort. The ore one patrols, the more incidents are found. With sustained patrol, however, the assumption is that the incidents will start to decrease as the law is enforced. A semi-annual evaluation to determine trends in poaching levels will be made, as well as semi-annual evaluations of the threat levels within conservation areas to maintain targeting of patrol effort in high risk areas.
The reduction in poaching will only be reported for areas where patroling is programed and implemented on a regular basis. The individual Poaching reduction will be reported individually for each conservation areas and for the project as a whole. If different protected areas are including in the anti-poaching effort facilitated by the project, overall effort in patroling, a weighted average will be applied to report the LESTARI project-wide poaching redcution effort.
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape Conservation Area
Target Achievement % of Completion
REVIEW OF DATA Semi-annually by ME&L Coordinator
REPORTING OF DATA
Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
SMART patrol result in Year 1
BASELINE NOTES LESTARI will undertake baseline in Year 1 to determine number of current poaching incident in targeted conservation areas.
BASELINE VALUE TBD
BASELINE YEAR 2015
TARGETS NOTES Numbers of poaching will be reduced progressively by 10%, 20%, 30%, and 40% each year of the project compared to baseline. But achievement will only start to be reported in the second year of the project.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 Not measure
FY 2017 10%
FY 2018 20%
FY 2019 30%
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 72
INDICATOR 3
FY 2020 40%
Targets LOP 40%
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 73
INDICATOR 4
GOAL Reduced rate of deforestation and degradation
RESULT
IR 2. Improved land use governance
KR 6. Increased commitment of key private sector, government, and community stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass
KR 7. Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels
INDICATOR TITLE Number of public policies addressing climate change and/or biodiversity conservation introduced, changed or adopted consistent with citizen input
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator – contributes to FACTS 2.4.1-12 and FACTS 4.8.2-28
PRECISE DEFINITION
Public policies include any law, regulation, policy or similar directive that is formally adopted by either the legislative branch or a unit of the executive branch at any level.
Citizen inputs means that the public, citizens and/or civil society organizations have proposed language used in, provided comments incorporated into, or monitored the implementation of the policy.
Introduced refers to draft legislation formally being presented and accepted for consideration by a legislative body.
Changed refers to an existing policy that has been substantively changed.
Adopted refers to the process by which a public policy, whether a completely new policy or one that has been changed, is formally adopted through decree, passage of law or other formal process.
Policies may be district-wide if they cover LESTARI landscapes, e.g., acceleration of community forestry licensing in support of the national goal of 12.7 million hectares. Policies may explicitly target LESTARI landscapes, e.g., collaborative management of Cyclops.
For interpretation of this indicator, a qualitative description will be provided to explain what the number represents, particularly:
What is the title of the measure?
At what stage is it? (e.g., officially introduced, changed, adopted, or implemented?)
How does the measure contribute to climate change mitigation or biodiversity conservation?
What is/are the institution(s) that will be implementing and/or enforcing the measure and at what scale (e.g., national, provincial, district)
UNIT OF MEASURE Number of public policies
DISAGGREGATIONS District; Provincial; and National
Climate change policy and biodiversity policy
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 74
INDICATOR 4
RATIONALE OR JUSTIFICATION FOR INDICATOR
The indicator is used by LESTARI to ensure that there is willingness from local government to adopt input from citizen-based mechanisms, or in other word, to ensure that development policies are responsive to the public.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Democratic governance concerns are integral to improving policy reform and implementation across a wide range of development sectors. More effective policy change across development sectors requires government policy makers (in both the executive and legislative branches) to consider and incorporate citizen input into the policy development process. Also, an improved enabling environment through legal and policy reform is essential for ensuring that efforts and investments in climate change and/or biodiversity conservation have legal and strategic backing and institutional ownership.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review project reports, policy documents, tracking policy development process, direct observation, analysis of secondary data and third party observation (i.e. monitoring of public policy by local NGO).
DATA SOURCE Activity reports, policy documentation, assessment reports, policy monitoring reports
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling, as data are available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; policy monitoring cost; photocopies of documents
INDIVIDUAL RESPONSIBLE
Decentralized Governance Specialist; Advocacy Campaign Specialist
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA There are some limitations to validity since the quality of citizen input is very subjective. Reliability is also of concern since policy reform is not always a linear
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 75
INDICATOR 4
LIMITATIONS process. Because the intended result is an improved enabling environment, the number of public policies provides only a partial measure of success, given that effective implementation and enforcement are also critical. Public policies might also not be well-designed or effective.
ACTION TAKEN TO ADDRESS LIMITATIONS
Narrative is critical for interpreting this indicator. Also if the policies may affect climate or biodiversity indirectly, LESTARI will provide narrative to explain the connection.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
A citation analysis method will be used to evaluate “uptake and use.” This method involves looking at the extent to which citizen recommendations are visibly “picked up” and used by local governments. Analysis will include triangulation of secondary data and third party observation (when available). Case study will be deployed to assess the progress made along the way in terms of convening stakeholders, drafting, approving, and implementing/enforcing policies.
It should be noted that the policy introduced, changed or adopted may take concern provincial and national regulations. In this case LESTARI will still count it as a single policy.
Data analysis for this indicator will involve assessing a series of milestones. We will count this indicator achieved once the policy is either introduced, changed or adopted. However, the project will still monitor and report the number of policies that are implemented or enforced. Key analytical questions include: how many (percentage) of policies get adopted? Of those that get adopted, what percentage are implemented? How many of the policies that were adopted and implemented had substantial participatory events as part of the development process?
Type of Policy
Introduced Changed Adopted Implemented
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District, Provincial, National
Target Achievement % Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For this indicator, the baseline is zero
BASELINE VALUE 0
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 76
INDICATOR 4
BASELINE YEAR N/A
TARGETS NOTES It is assumed that in each of 14 districts at least two policies (for instance one concerning climate change and one concerning biodiversity) will be influenced as a result of advocacy activities. However, it should be noted that the intended result is an improved enabling environment. Therefore the numbers provide only a partial measure of success. Effective implementation and enforcement are much more critical which in turn could be reflected through achievement of LESTARI key goals: reducing emissions and conserving biodiversity.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 3
FY 2017 5
FY 2018 8
FY 2019 8
FY 2020 4
Targets LOP 28
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 77
INDICATOR 5
GOAL Reduced rate of deforestation and degradation
RESULT
IR 2. Improved land use governance
KR 7. Policies, laws, regulations, and procedures in support of low emission
development and forest conservation and management increased,
promulgated, and enforced at all levels
INDICATOR TITLE Number of sub-national government with improved licensing and permitting mechanism
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
Sub-national government including but not limited to province, district, FMU.
The term “improved” is taken to subsume the five core principles of good governance. First and foremost transparency, inclusivity/participation (e.g., village voice in license approval), accountability (clear focal point of responsibility for legally-approved licenses and their consequences), responsiveness (responding to public or sectoral complaints) and timeliness.
For new licenses, improved also means that all new requests for licenses and permits for a new concession area for natural resources extraction activities (i.e., mining, oil palm plantation and logging/industrial timber plantation) should follow Standard Operational Procedures (SOP) that developed within the tool based on the government regulation and procedures for granting licenses and permits.
Improved licenses and permits including but not limited to Izin Prinsip, Izin Lokasi, AMDAL, and others according to the type of the private sectors that are operating within LESTARI landscapes.
UNIT OF MEASURE Number of sub-national government units
DISAGGREGATIONS Province, District
RATIONALE OR JUSTIFICATION FOR INDICATOR
The indicator is used by LESTARI to track the progress of strengthening governance process in the respective districts regarding the issuance of forest licenses and permits. The indicator also illustrates the extent to which spatial plans policy enforced.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Improved licensing and permitting process will ensure the certainty of land use and transparency, reduce potential conflicts between private sectors and communities, and address driver of deforestation and degradation.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review the maps indicates existing licenses, case study to assess adoption of good governance principles into licensing and permitting mechanism, review the licensing process and SOP, interview government licensing officers and business enterprises, analysis of secondary data and third party observation (i.e. monitoring of forest licensing by local NGO).
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 78
DATA SOURCE Maps, SOP of licensing and permitting process, activity reports, monitoring/ assessment reports
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling, as data are available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; case study cost; photocopies of documents
INDIVIDUAL RESPONSIBLE
Sustainability Screening Specialist; Private Sector Engagement Coordinator
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Incomplete regulatory rules and the lack of synergy between different plans (the National Spatial Land use Plan (RTRWN), the Provincial Spatial Land use Plans (RTRWP) and the District Spatial Land use Plans (RTRWK)), opens the door to weak implementation of the regulations and corruption. As a result land may be inappropriately zoned or zones may be changed in order to allow logging or other extraction. Without adequate plans or consultation with local communities, the current zoning of land does not provide a basis on which to make licensing decisions, and therefore such decision-making processes are open to manipulation and undue influence by loggers or extractors. Furthermore, there are no official institutions tasked with monitoring the zoning of land and in the absence of accurate data and maps on logging/extraction zones independent monitoring is difficult.
ACTION TAKEN TO ADDRESS LIMITATIONS
LESTARI will develop capacity of local CSOs to monitor the forestry license activities through research and analysis supported by advocacy and Geographic information System (GIS) analysis. GIS analysis is used to identify any distortion in the areas for which logging/extraction permits have been given or violations in these areas. Spatial Data Infrastructure (SDI) initiated under IFACS will be strengthened to improve the accuracy, consistency, and availaibility of land use information for public use.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS Technical review will be carried out to determine transparency and accessibility of licensing and permitting information. Also to review if new licenses have followed the SOP/screening process. The analysis will be enriched by qualitative
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 79
assessment through interview processes toward government officers and business enterprises. LESTARI will track intermediate milestones when counting progress toward meeting these targets.
Sub-national level
Milestones
Review and align licenses and permits with spatial plan
Screening tool developed - Protocol and procedures for public monitoring developed
Communication and consultation with local government and approval received for screening tool and monitoring protocol
Official public socialization & training for the screening tool
Issuance of licenses and permits follow the SOP and good governance principles
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape Sub-national level
Target Achievement % Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For this indicator, the baseline is zero
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Achievement of target will only start to be reported in the second year of the project because a considerable time is needed to develop SOP and enhance local officers capacity to implement SOP.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 Not measure
FY 2017 4
FY 2018 4
FY 2019 4
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 80
FY 2020 2
Targets LOP 14
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 81
INDICATOR 6
GOAL Reduced rate of deforestation and degradation
RESULT IR 2. Improved land use governance
INDICATOR TITLE Number of sub-national government incorporating high quality SEA-LEDS & LCPs into draft spatial plans, zonation regulation or draft of mid-term development plan
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
Sub-national government including province and district.
There are different level of RTRW status in each district working with LESTARI: approved or under development. For the districts with approved RTRW’s, they now enter the 5-year review phase in which zoning (inside Kawasan and APL) through Peraturan Zonasi as well as Kawasan Pedesaan are required by Law 26 of 2007 and PP15 of 2010. This provides a unique opportunity for the LESTARI Landscapes to be regulated not only inside Kawasan Hutan but also APL down to the village level. SEA and LCP indicate where this should be and what should be done but to be actionable zoning at a greater level of spatial detail both inside and outside Kawasan Hutan to include APL will be required, especially important in peatland areas.
Therefore, incorporating SEA-LEDS and LCP recommendations will be done through zonation process. LESTARI will assist local government in such zonation. The results will be zonation maps and rules about what can and cannot be done in the zones towards reducing GHGs and increasing biodiversity conservation. Spatial planning enforcement will be supported by the preparation of the zonation maps and raising public awareness of the zonation areas (where and what allowed / not allowed). The measurable impacts will be the formalization of zoning maps down to the village cluster level (by watershed and as far as possible by sub-district) along with budget and institutional allocations to realize this along with public awareness about the maps in support of greater participation and accountability.
For the districts in which the RTRW are under development, incorporation of SEA/LEDS and LCPs recommendations will be achieved when such recommendations explicitly referred in draft RTRW. This will also be the case for draft RPJMD.
High quality SEA-LEDS and LCP means that the recommendations provide improved-case scenario and not business-as-usual.
UNIT OF MEASURE Number of sub-national government
DISAGGREGATIONS Province, District
RATIONALE OR JUSTIFICATION FOR INDICATOR
The indicator is used to measure progress of improved land use governance. Since both documents were initiated under IFACS project and none of the SEA-LEDS supported by IFACS have been incorporated into district spatial plans due to long periods required for approval across the various levels of government, it is crucial to keep monitoring the process and find other potential entry point to maximize impact through incorporating SEA-LEDS & LCPs into mid-term development plan which will secure financing for implementation.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 82
INDICATOR 6
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Integrating environmental consideration into strategic decision-making can improve the quality of local planning, ensure sustainability, and change the land-use practices thus contributes to emission reduction.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Technical review of SEA-LEDS and LCPs recommendations and draft of spatial plans, draft of district mid-term development plans and zonation maps/policy. Interviews will be conducted with relevant government officers to evaluate recommendations “uptake and use”.
DATA SOURCE SEA document, LCP document, Spatial Plan document, Mid-term development Plan document, assessment report, zonation maps.
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling, as data are available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; assessment cost; photocopies of documents
INDIVIDUAL RESPONSIBLE
Sustainable Land Use Planning Coordinator; Forest Governance Advisor
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
This indicator measures SEA-LEDS & LCP recommendations incorporated into draft spatial plans and mid-term development plans but does not measure the number or success of recommendations that have been implemented.
ACTION TAKEN TO ADDRESS LIMITATIONS
Build sustainability within the process to ensure that the recommendations are accepted and then implemented. Building sustainability is increased as a result of specific and targeted public consultation by LESTARI to bring the community and private sector and local government together to arrive at consensus and
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 83
INDICATOR 6
commitment to implement the plan. Incorporationg the recommendations into zonation policy will increase the likelihood of implementation.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Citation analysis method will be used to evaluate “uptake and use.” This method involves looking at explicit reference to adopting SEA recommendation of improved-case scenario in draft RTRW and draft RPJM documents or for approved RTRW indicated by zonation maps. Because policy improvement involves a multi-step process, LESTARI will track intermediate milestones when counting progress toward meeting these targets.
Sub-national level
Milestones
High quality SEAs developed or updated
Issuance of Bupati/Head of Dinas instructions for incorporation
Incorporation of SEAs into spatial plans, illustrated by zonation maps, explicitly referred in draft RTRW or draft RPJMDs
Enforcement evidenced
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape Sub-national Target Achievement % Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For this indicator, the baseline is zero
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES N/A
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 2
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 84
INDICATOR 6
FY 2017 4
FY 2018 4
FY 2019 2
FY 2020 2
Targets LOP 14
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 85
INDICATOR 7
GOAL Reduced rate of deforestation and degradation
RESULT IR 2. Improved land use governance
INDICATOR TITLE Number of Multi Stakeholder Forum (MSF) operational as citizen based mechanisms for public input on land use
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
A multi-stakeholder forum (MSF) is a working group of stakeholders with a vested interest in the future of forests, land use, Low Emission Development Strategies (LEDS), improving the livelihoods and future of their constituency, and mitigating and adapting to climate change in the district. MSF contains key decision makers and stakeholders from the public, private, and civil societies.
Operational means MSF is institutionalized and the key stakeholder base broadened, managed by a secretariat with local government budgetary support.
MSF will serve as a bridge between citizens and local government and foster trust-based relationships built upon mutual respect. MSF will be institutionalized with the explicit purpose of tightening the relationship between RTRW and RPJM as well as annual budgeting processes.
UNIT OF MEASURE Number of MSF
DISAGGREGATIONS District
RATIONALE OR JUSTIFICATION FOR INDICATOR
The indicator is used to ensure that all interested people have an opportunity to be heard or to influence government decisions that affect improved forest conservation. Also, this indicator is used to measure the extent to which citizen-based mechanisms for public input successfully strengthened as a result of USG assistance.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
The success of MSF to bridge communication between local government and wider public in land use will lead to more transparent, participatory and accountable land use decisions. Those principles are prerequisite of good governance.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review reports from regional technical team on MSF engagement, review the process of public consultation, review Bupati decree of MSF (i.e., clear statement of MSF function), qualitative assessment/case study to learn MSF effectiveness
DATA SOURCE Activity reports, MSF plans, LCPs, Bupati Decree, assessment reports
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling, as data are available
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 86
INDICATOR 7
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; assessment cost; photocopies of documents
INDIVIDUAL RESPONSIBLE
Stakeholder Coordination Specialist; Advocacy Campaign Specialist
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Activity reports may not always be complete
ACTION TAKEN TO ADDRESS LIMITATIONS
Will require supporting documentation and observation to verify results
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Because operationalization of MSF involves a multi-step process, partial completion will be counted by tracking intermediate milestones.
MSF
Milestones
MSF institutionalized - issuance of Bupati Decree
MSF revitalized (key stakeholder base broadened)
MSF functional with secretariat and local budget support
MSF facilitate change in policy / regulation / Bupati Vision etc)
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District Target Achievement % Completion
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 87
INDICATOR 7
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES MSF became dormant after IFACS. While popular, they were not institutionalized. Part of the problem was that MSF had disproportionate government representation. During LESTARI, MSF will be institutionalized and the key stakeholder base broadened, managed by a secretariat with local government budgetary support. Due to absence of this condition, the baseline is assumed as zero
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES LESTARI will develop or strengthen MSF in 14 target districts.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 2
FY 2017 4
FY 2018 4
FY 2019 4
FY 2020 0
Targets LOP 14
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 88
INDICATOR 8
GOAL Reduced rate of deforestation and degradation
RESULT IR 2. Improved land use governance
INDICATOR TITLE Number of community champions engaged in advocacy interventions
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Output
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
Community champions are individuals chosen to speak on behalf of their communities. The indicator measures community champions in LESTARI program that initiate or participate in advocacy interventions. Advocacy should be understood as a means for individuals, constituencies, or organizations to shape public agendas, change public policies, and influence other processes that impact their lives.
Advocacy does not involve one march, meeting or poster, but a series of strategic, interconnected, integrated activities designed to achieve goal. For LESTARI, advocacy activities could include lobbying, public interest litigation, letter writing campaigns, monitoring licensing and permitting, etc. Advocacy interventions will be strategic, involve a set of actions that are sustained in order to build and direct pressure, be designed to persuade, be targeted and involve alliance building. Community champions in LESTARI could be involved in MSF and use MSF as channel for advocacy.
Successful advocacy efforts result in change. This indicator is highly connected with Indicator #4 (public policies) and Indicator #5 (more transparent and accountable licensing and permitting).
UNIT OF MEASURE Number of individuals
DISAGGREGATIONS Sex (female/male); District; Landscape
RATIONALE OR JUSTIFICATION FOR INDICATOR
The indicator is used to track the progress of strengthening citizen-based mechanisms for public input on land use. It is also used for measuring the extent to which the project is able to build constituency for conservation.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Advocacy interventions are essential aspects of democratic policy making, citizen participation, and oversight of all branches of government. These interventions play an important role in giving voice to citizens and historically marginalized groups. Successful advocacy will improve land use decisions.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review advocacy plans or strategies, recording of press conference, copy of testimony or press release, advocacy campaign materials, record of public consultation. Case study will be conducted to track the process of advocacy and policy change.
DATA SOURCE Advocacy plans or strategies, press conference, press release, advocacy
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 89
INDICATOR 8
campaign materials, minutes of public consultation, assessment report.
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling, as data are available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; assessment cost; photocopies of documents
INDIVIDUAL RESPONSIBLE
Communication and Advocacy Specialist
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Activity reports may not always be complete
ACTION TAKEN TO ADDRESS LIMITATIONS
Will require supporting documentation and observation to verify results
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS Counting number of individuals chosen as spokespersons and engaged in advocacy activities supported by documentation
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District Target Achievement (disaggregated by Female/Male)
% Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF Quarterly and Annual Report
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 90
INDICATOR 8
DATA
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For this indicator, the baseline is zero
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on potential CSOs in the district level, population, membership of MSF during IFACS project.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 50
FY 2017 150
FY 2018 150
FY 2019 150
FY 2020 0
Targets LOP 500
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 91
INDICATOR 9
GOAL Reduced rate of deforestation and degradation
RESULT IR 2. Improved land use governance
INDICATOR TITLE Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Output
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
“Reached by LESTARI communication programs” is defined as individuals who have been reached via LESTARI communication and outreach campaign, such as printed climate change and forest conservation materials, radio campaigns, influential religious leader raising climate change and forest conservation issues to their congregations, social media, video, communication trainings, and exhibitions. People could be reached through passive participation (act as “empty vessels” and receive information) or empowerment participation where primary stakeholders are capable and willing to become involved in the process and take part in advocacy and decision-making.
This indicator is an early step to measure the scale of communication program. While changes in people knowledge and behaviors could be assessed when those people reached turn into champions as indicated by Indicator #8.
UNIT OF MEASURE Number of individuals
DISAGGREGATIONS Sex (female/male); District; Landscape
RATIONALE OR JUSTIFICATION FOR INDICATOR
These results will help to estimate the coverage of LESTARI communication program.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Increase awareness and understanding of LEDS and biodiversity, supported by continous advocacy will lead to increase commitment of key stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass. This in turn leads to improved land use decision making contributes to improved land use governance.
DATA COLLECTION APPROACH
DATA COLLECTION
METHOD
Data collection will vary widely based on the particular activity that reaches people with LESTARI messages/information on climate change and biodiversity conservation issues. For communication training, sign-in sheets will validate the result. For radio campaign, LESTARI will assess the number of people that could potentially be “reached” to this message through population and geographical data. Using the data, the project will estimate the % of people (disaggregated by sex) within that geographical area that likely heard that message. A similar methodology will be used for mass media such as information in newspapers and other forms of print media. The project will also review communication products
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 92
INDICATOR 9
developed by stakeholders and deliver case study to assess increase awareness and changes in behaviours.
For social media, the process of gathering data, as well as what data we gather, differs for each channel. In Facebook, LESTARI focus on three main areas: likes, comments and links on posts, and shares. If someone “likes” LESTARI page, we will know that they’re listening to us. When followers add comments to our posts, we’ve made the conversation two-way by engaging them. And when they share our posts with their Facebook friends, we’ve expanded our audience. Gathering this information will help us understand what people are doing on Facebook, and how to tap into it. Twitter is similar—engagement is a more accurate measure of our influence than follower count. Do followers retweet our posts, or share information of relevance and interest to our organization and constituents? Measure engagement against the time we spend on the site to get a sense of the channel’s value. For LESTARI website, we will use Google Analytics to monitor traffic and referral data.
DATA SOURCE Sign-in sheet from trainings, secondary data (population and geographical), communication products (e.g., articles, shermon sheets, etc) developed by stakeholders, activity reports, assessment report, social media monitoring report.
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling as data available
ESTIMATED COST OF DATA ACQUISITION
Staff and labor cost, assessment cost, photocopies of activities documentation.
INDIVIDUAL RESPONSIBLE
Communication & Advocacy Specialist
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Calculating direct beneficiaries as a result of communication training is rather easy. However, when LESTARI issues a radio spot or puts information in a local newspaper, measuring the precise number is extremely difficult and imprecise.
ACTION TAKEN TO ADDRESS
LESTARI will provide a rational justification for data counted against this indicator that obtained through mass media such as radio and newspaper information dissemination. LESTARI will access the reach of the message (e.g., the
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 93
INDICATOR 9
LIMITATIONS newspaper’s geographic circulation or how far a radio signal can carry) and determine the population within that area. LESTARI will then apply a percentage of people that are likely reached to that message, using a rather conservative percentage (e.g., 30% of total population within that range).
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Counting number of individuals reached in LESTARI communication programs. LESTARI will assess the efficacy of outreach and other communication activities using the case study as well as assess the achievement of other related indicators that indicate changes in behaviors.
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District Target Achievement (disaggregated by Female/Male)
% Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For this indicator, the baseline is zero
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on potential population reached per landscape and IFACS lesson learned. LOP target is 15,000. Target for the fourth and fifth year of the project decreased following the activity declines.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 1,000
FY 2017 4,000
FY 2018 5,000
FY 2019 3,500
FY 2020 1,500
Targets LOP 15,000
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 94
INDICATOR 9
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 95
INDICATOR 10
GOAL Reduced rate of deforestation and degradation
RESULT
IR 1. Improved forest management
KR 3. Management of at least six conservation areas improved, resulting in the
conservation of valuable orangutan and other key species habitat, and the
reduction in poaching of threatened and endemic species
INDICATOR TITLE Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
Conservation Areas include National Parks, Wildlife sanctuaries, natural reserves, all areas of high biodiversity under the jurisdiction of the Ministry of Forestry.
The Management Effectiveness Tracking Tool (METT) is a tool developed by the World Wildlife Foundation and The World Bank to track and monitor the effectiveness of protected area management. It provides a rapid and replicable assessment designed to reveal trends, standardize assessment and reporting, and aid in adaptive management. METT consists of two primary sections – datasheets and an assessment form. The datasheets contain contextual information such as size and location, local designation, and budget, as well as a ranking of threats. The assessment form consists of a 30-question scorecard that quantifies performance based on a 4-point scale (0-3). Each question also requires an explanation for qualitative judgments, such as detailing the level of staff knowledge or results from external studies.
Overall METT encompasses 6 key elements (context, planning, inputs, process, outputs, and outcomes) crucial to effective protected area management. METT allows park managers to identify needs, constraints, and priority actions to improve protected area management.
UNIT OF MEASURE Number of CAs
DISAGGREGATIONS CAs, Landscape
RATIONALE OR JUSTIFICATION FOR INDICATOR
METT was selected, not only because it allows the LESTARI team to monitor a portfolio of sites with a single, cost-effective tool, but also because it has been adopted by GoI as a tool to assess the management effectiveness of protected areas across Indonesia.
The ability to manage protected areas effectively relies on a combination of good governance, sufficient capacity, well-trained staff and enough money to pay for essential management activities and equipment. Therefore, METT will provide a quick and simple picture of effectiveness in individual protected areas, through repeated application in sites over time.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR
Success in maintaining biodiversity is linked to a well-regulated and managed protected area, where staff are assessing progress and making changes as necessary. The strongest association is with law enforcement, control of access,
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 96
INDICATOR 10
IMPACT resource management, monitoring and evaluation, maintenance of equipment, budget management and existence of annual work plans. Increase effectiveness of CAs management will contribute to conservation goal and reduce GHG emissions.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
METT baseline and endline survey; monitoring the implementation of CAs management plan
DATA SOURCE Survey report; CAs management plan; direct observation
FREQUENCY AND TIMING OF DATA ACQUISITION
Annual
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; METT survey cost; photocopies of necessary documents
INDIVIDUAL RESPONSIBLE
Biodiversity Conservation Advisor
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
METT has been developed to provide a quick overview of progress in improving the effectiveness of management in individual protected areas, to be filled in by the protected area manager or other relevant site staff. As such it is clear that there are limitations on what it can achieve: it should not for example be regarded as an independent assessment, or as the sole basis for adaptive management.
The Tracking Tool is also too limited to allow a detailed evaluation of outcomes and is really aimed at providing a quick overview of the management steps identified in the WCPA (World Commission of Protected Area) Framework up to and including outputs. Clearly, however good management is, if biodiversity
continues to decline, the protected area objectives are not being met. Therefore the questions on condition assessment have disproportionate importance in the overall Tracking Tool.
ACTION TAKEN TO Additional questions may need to be added to suit local circumstances rather
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 97
INDICATOR 10
ADDRESS LIMITATIONS
than modify the Tracking Tool. The Tracking Tool contains a set of questions that have been designed to be easily answered by those managing the protected area without any additional research. However, it is useful to review the results of existing monitoring and to spend sufficient time discussing each aspect
of management being assessed to arrive at a considered judgement. A group
of protected area staff from the reserve, project staff or other agency staff should be involved in answering the questions in the Tracking Tool; where possible additional external experts, local community leaders or others with knowledge and interest in the area and its management should also be involved.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Analysis will be done through quantitative (scoring) and qualitative methods. METT assessment is made by assigning a simple score ranging between 0 (poor) to 3 (excellent). A series of four alternative answers are provided against each question to help assessors to make judgements as to the level of score given.
The maximum score of the 30 questions and supplementary questions is 99. A final total of the score from completing the assessment form will be calculated as a percentage of 99 or of the total score from those questions that were relevant to a particular protected area. Thus if a protected area scores 65 out of a maximum score of 87 the percentage can be calculated by dividing 65 by 87 and multiplying by 100 (i.e. 65 ÷ 87 x 100 = 75%). Qualitative judgements will be applied whereas the information come from local staff knowledge (in many cases, staff knowledge will be the most informed and reliable source of knowledge), a reference document, monitoring results or external studies and assessments.
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape Conservation Area
Target Achievement % Completion
REVIEW OF DATA Annually by ME&L Coordinator
REPORTING OF DATA
Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
METT baseline survey undertaken by Ministry of Forestry and Environment (MOFE) for six conservation areas
BASELINE NOTES None
BASELINE VALUE
Conservation Areas METT score
Gunung Leuser National Park 67 %
Sebangau National Park 62 %
Lorentz National Park 52 %
Cycloops Nature Reserve 43 %
Rawa Singkil 55 %
Bukit Baka Bukit Raya National Park 64 %
BASELINE YEAR 2015
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 98
INDICATOR 10
TARGETS NOTES LESTARI will assist six conservation areas to improve their METT scores.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 Not measure
FY 2017 1
FY 2018 2
FY 2019 2
FY 2020 1
Targets LOP 6
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 99
INDICATOR 11
GOAL Reduced rate of deforestation and degradation
RESULT IR 1. Improved forest management
INDICATOR TITLE Number of co-management agreements signed that secure community rights and benefits
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Output
INDICATOR TYPE Custom Indicator
PRECISE DEFINITION
LESTARI uses an IUCN definition of the term co-management which includes “a situation in which two or more social actors (parties) negotiate, define and guarantee amongst themselves a fair sharing of the management functions, entitlements, and responsibilities for a given territory, area, or set of natural resources” – in this case for conservation and sustainable NRM utilization.
The co-management agreements referred here will include community based forest management agreements covered in the existing policies, which include Hutan Kemasyarakatan (HKm), Hutan Desa (HD), Hutan Adat (HA), Hutan Tanaman Rakyat (HTR), and other forms of co-management agreements such as partnership agreements between communities with conservation area managers, private sectors, or stakeholders as well as co-management agreements involving multistakeholders (such as the format that is currently initiated in Cyclops). These co-management agreements will ensure communities’ secured access to forest resources.
Co-management agreements signed will cover the process of obtaining permits from social forestry initiatives (HKM, HD, HTR, HA) or the signing of partnership agreements between communities and other stakeholders (conservation area managers, private sectors, etc).
Co-management agreement implemented and monitored means that forest management action plan that will be included in the agreements are implemented and monitored.
UNIT OF MEASURE Number of agreements
DISAGGREGATIONS Landscape; Type of co-management agreement
RATIONALE OR JUSTIFICATION FOR INDICATOR
The indicator is used by LESTARI to assess the extent to which key stakeholders committed to sustainable use of forests. The indicator also illustrates reduction in land use conflicts.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Many critical forest areas within LESTARI landscapes are poorly managed and experience a high rate of deforestation and degradation as a result. An effective management, a high level of local social capital, including cohesive communities, is essential for conservation of these important resources.
DATA COLLECTION APPROACH
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 100
INDICATOR 11
DATA COLLECTION METHOD
Review the content of co-management agreements, assessment of the effectiveness of their implementation, spot-check to observe compliance.
DATA SOURCE Assessment reports; document of agreements; direct observation
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling, as data are available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; assessment cost; photocopies of documents
INDIVIDUAL RESPONSIBLE
Biodiversity Conservation Advisor
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Activity reports may not always be accurate. Community may live in remote sites that are difficult to visit to validate data.
ACTION TAKEN TO ADDRESS LIMITATIONS
LESTARI field staff to spot-check claims through field visits. Sampling techniques for remote sites. Review agreements to confirm that they include statements of commitment and secure community rights and benefits.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
The development of co-management agreements involves multiple steps. When counting progress toward meeting this target, LESTARI will track the intermediate milestones. We will count this indicator achieved once the agreement is signed; however, the project will still monitor and report the implementation of agreements.
Hectares of the area managed under co-management agreement will also be reported and contribute to Indicator #2.
Districts
Milestones
Stakeholders and areas for
Dialogue facilitated by
Co-management
Agreements implemented
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 101
INDICATOR 11
co-management identified
the project agreements signed
and monitored
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape Type of co-management agreement
Target Achievement Ha under co-management agreement
% Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For this indicator, the baseline is zero
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on locations that proposed to LESTARI by Ministry of Environment and Forestry.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 0
FY 2017 10
FY 2018 13
FY 2019 12
FY 2020 5
Targets LOP 40
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 102
INDICATOR 12
GOAL Reduced rate of deforestation and degradation
RESULT IR 1. Improved forest management
INDICATOR TITLE Number of people receiving USG supported training in natural resources management and/or biodiversity conservation
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Output
INDICATOR TYPE FACTS 4.8.1-27
PRECISE DEFINITION
This indicator will measure the number of individuals trained through learning
activities in natural resources management (NRM) and/or biodiversity
conservation. Training will primarily be traditional classroom training and
workshops (led by designated instructors or “lead persons,” with a learning
objective and defined curricula).
Training topics will include but are not limited to: Best Management Practices,
forest fire management, Good Agricultural Practice (GAP), etc.
This indicator reports against the Biodiversity Fund.
UNIT OF MEASURE Number of people
DISAGGREGATIONS Sex (male/female); District; Landscape; and Affiliation (government, non-governmental organizations, private sector, and community, as appropriate).
RATIONALE OR JUSTIFICATION FOR INDICATOR
Tracking training in NRM and biodiversity conservation provides information about the reach and scale of training and capacity building efforts.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Increased capacity to manage forest resources and adapt to climate change will lead to improved land use management and strengthened mitigation and adaptation strategies to overall ecosystem management.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review of training attendance records, with a proportion of the trainings attended by LESTARI staff. Pre and post survey to seek information about changes in knowledge resulting from the training. Monitor other indicators to justify changes in application of knowledge and skill.
DATA SOURCE Training records/reports, training grantee/subcontractor reports, monitoring reports.
FREQUENCY AND TIMING OF DATA ACQUISITION
At the end of each training, to be compiled quarterly and annually
ESTIMATED COST Staff labor costs including field staff and transportation to training sites to observe
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 103
INDICATOR 12
OF DATA ACQUISITION
and verify the training.
INDIVIDUAL RESPONSIBLE
Assigned field staff will report to Landscape M&E Specialist with cc: ME&L Coordinator
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Training record/report may not always be accurate.
ACTION TAKEN TO ADDRESS LIMITATIONS
Independent verification through attendance by LESTARI staff.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS Counting number of people participating in LESTARI NRM and/or biodiversity trainings
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District Target Achievement (disaggregated by sex and affiliation)
% Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES For training, the baseline is zero
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 104
INDICATOR 12
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on potential population and IFACS lesson learned
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 1,000
FY 2017 2,000
FY 2018 3,000
FY 2019 3,000
FY 2020 1,000
Targets LOP 10,000
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 105
INDICATOR 13
GOAL Reduced rate of deforestation and degradation
RESULT
IR 1. Improved forest management
KR 5. Funding leveraged from public and private sources, representing co-
investment in project outcomes
INDICATOR TITLE Amount of investment mobilized (in USD) for climate change as supported by USG assistance
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Output
INDICATOR TYPE FACTS 4.8.2-10
PRECISE DEFINITION
This indicator includes finance mobilized (or leveraged), enabled by USG assistance, for actions, activities, projects or programs that avoid, reduce, or sequester GHGs. Finance may be mobilized from the public sector (e.g., local government) or private sector (e.g., corporate investment) and should help to advance the objectives established by LESTARI program.
For LESTARI, finance mobilized under this indicator also include those allocated for sustainable forest management, spatial planning, and conservation activities in targeted landscapes.
For LESTARI, finance can be mobilized through a variety of instruments and vehicles, including common funding instruments, parallel investments, or in-kind support as a result of USG support. Examples of the types of U.S. assistance (under LESTARI) that could mobilize finance include:
Finance interventions, such as:
Grants (or in-kind support) for technical assistance
Equity or investment shares through PPP scheme
Investments made possible by policy interventions and technical assistance interventions, such as:
Facilitate organizations/local governments to access financing from ICCTF (Indonesia Climate Change Trust Fund)
Market value chains for cocoa that link farmers to wider market
Regulatory policy support for transparent and fair permitting and approvals
Regulatory policy support for Payment for Environmental Services (PES)
Co-management interventions
Examples of what mobilized funds may support include improving the enabling environment for mitigation actions, funding the costs of climate change and conservation activities advanced by the program, monitoring climate change progress or outcomes, or sensitizing stakeholders to climate risks and land use issues and opportunities addressed through the program.
USD amount will be determined on a quarterly basis for all Indonesian rupiah values within that quarter using a currency converter (http://www.oanda.com/currency/converter/). This will be printed and filed with the indicator documentation.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 106
INDICATOR 13
UNIT OF MEASURE US Dollars
DISAGGREGATIONS
Domain of interest: Sustainable Landscapes and Biodiversity
Source of funds (public, private) dissagregated further into domestic and international sources of investment
District; Landscape
RATIONALE OR JUSTIFICATION FOR INDICATOR
Indicator used to track amount of investment leveraged the project could reach under initiatives fitting together public and private entities
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
LESTARI programs are intended to be catalytic and to have sustainable benefits. The mobilization of additional financial resources can be an important indicator for assessing the success of LESTARI in catalyzing resources needed for transformational change. This indicator can also help to provide a baseline of data needed to test hypotheses as to the most effective strategies, techniques, or necessary capacities for mobilizing the funds required to address climate change and biodiversity conservation, leading to lessons learning over time.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
LESTARI will gather data about the amount of finance mobilized from all types of potential activities intervention (please see the examples under precise definition section).
Documentation should include a rationale for how LESTARI support has facilitated the mobilization of additional resources and include information such as source of funds and use of funds.
DATA SOURCE Financial records/statements verify amounts leveraged and documentation that prices out labor and material per period of time that was mobilized to enhance LESTARI objectives in climate change and biodiversity conservation.
FREQUENCY AND TIMING OF DATA ACQUISITION
Rolling as data available
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; photocopies of documentation
INDIVIDUAL RESPONSIBLE
Landscape M&E Specialists
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 107
INDICATOR 13
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Proprietary information – some organizations providing funding may consider some information on their funding support proprietary.
Precision – measuring funding leveraged does not necessarily indicate the magnitude of impact or results achieved.
ACTION TAKEN TO ADDRESS LIMITATIONS
Accept self-reporting (with narrative caveats) and validate with estimates of public or private sector investments using documented similar investments for which data are known.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Use standard audit methods on available data, calculate descriptive statistics - Disaggregation by domain of interest - Comparison with targets. When LESTARI intervention resulting on government allocating budget for certain program (e.g., fire prevention) in APBD, we will make a comparison between current APBD and previous APBD to allow justification of achievement.
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District Target Achievement (disaggregated by domain of interest and source of funds – public, private – domestic vs international)
% Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES The baseline is zero. The project is at a starting point and no activity has been developed by this time.
BASELINE VALUE 0
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 108
INDICATOR 13
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on potential investment that can be leveraged in each landscape.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 2,000,000
FY 2017 4,000,000
FY 2018 4,000,000
FY 2019 5,000,000
FY 2020 5,000,000
Targets LOP 20,000,000
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 109
INDICATOR 14
GOAL Reduced rate of deforestation and degradation
RESULT IR 1. Improved forest management
INDICATOR TITLE Number of people receiving livelihood co-benefits (monetary or non monetary)
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE Custom Indicator – contributes to FACTS 4.8.2-36
PRECISE DEFINITION
This indicator identifies the number of people in LESTARI landscapes where sustainable landscapes activities as well as conservation and sustainable use of forest resources activities are implemented, who have received livelihood co-benefits associated with these activities. People included in the metric should be part of populations or households identified by a project with a documented relationship to the project. Beneficiaries should be reasonably assumed to have received a documented benefit or service enabled by USG assistance.
Beneficiaries may include, but are not limited to: members of a household with an increased income or a newly secured land title, members of a cooperative who have increased sales due to increased market access, or even children attending a school renovated with payments for ecosystem services.
Examples of monetary benefits may include, but are not limited to: increased income due to government policies related to climate change mitigation, payments for avoided emissions or carbon sequestration, monetary benefits from ecosystem services, public-private partnership, etc, facilitated by LESTARI.
Examples of non-monetary benefits may include, but are not limited to: access to services, access to markets, land titling or registration, increased access to environmental services, protection of traditional livelihoods and customary rights, or increased productivity from climate-smart agricultural practices, etc, facilitated by LESTARI.
Number is specific to each year, not cumulative
UNIT OF MEASURE Number of people
DISAGGREGATIONS Sex (male/female); Landscape; District
Type of activity: Sustainable landscape; Biodiversity
RATIONALE OR JUSTIFICATION FOR INDICATOR
Reducing emission from deforestation and degradation can stimulate a win-win situation, by delivering carbon sinks and at the same time guaranteeing the conservation of biodiversity and livelihoods for forest-dependent communities. It is important to not only monitor emission reduced or avoided, but also to keep track of biodiversity and the livelihoods of local people. This measure demonstrates project reach.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
This indicator links sustainable natural resources management to economic growth objective. When people receive tangible monetary or non-monetary benefits from natural resource management or conservation, they are more likely
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 110
INDICATOR 14
to value and support these activities into the future, well after the project ends, creating a sustainable impact.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Beneficiary interview, random survey and on-sight observations to approximate impact across households or population
DATA SOURCE Project and subcontractor/grantee records; survey reports
FREQUENCY AND TIMING OF DATA ACQUISITION
Annually
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; survey costs
INDIVIDUAL RESPONSIBLE
Landscape M&E Specialist
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
This indicator is not intended to capture the quality of co-benefits received or the degree to which livelihoods are improved.
ACTION TAKEN TO ADDRESS LIMITATIONS
During the quantitative survey, the survey team will also solicit qualitative assessment to capture performance and other relevant assumptions into the calculations.
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
The analysis will involve quantitative (statistical) and qualitative analysis. Qualitative will enrich explanation of project impact. Analysis includes gender and location of those receiving benefits, which will more closely assess are those benefits equitably distributed between men and women.
Individuals receiving benefits from more than one sustainable landscapes activity, or receiving multiple benefits from a single activity, should be counted once per fiscal year.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 111
INDICATOR 14
PRESENTATION OF DATA
Tabular with supporting narrative, as appropriate
Landscape District Type of Activity (Sustainable Landscape; Biodiversity)
Target Achievement (disaggregated by sex)
% Completion
REVIEW OF DATA Annually by ME&L Coordinator
REPORTING OF DATA
Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES N/A
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target achievement for this indicator will only start to be reported in the second year of the project to allow impact to realize.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 0
FY 2017 7,500
FY 2018 7,500
FY 2019 7,500
FY 2020 7,500
Targets LOP 30,000
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 112
INDICATOR 15
GOAL Reduced rate of deforestation and degradation
RESULT IR 1. Improved forest management
INDICATOR TITLE Number of private sector firms that have improved management practices as a result of USG assistance
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE FACTS 4.6.2-9
PRECISE DEFINITION
This indicator measures the number of firms receiving USG assistance that improved their management practices. There are numerous private sector companies in LESTARI landscapes, ranging from large, multi-national, publicly traded Tropical Forest Alliance (TFA) members to smaller Indonesian privately held firms. Firms include forestry concenssions, oil palm and fiber plantations, mining and agroforestry/commodity producers, and their buyers. Companies, especially those involved in international trade, are under growing pressure to establish and meet sustainability and reduced emissions standards. Most companies lack the tools to assess potential GHG impacts, costs, and efficacy of these emerging goals, nor do they necessarily incorporate best management practices (BMPs) to meet these goals.
The project will facilitate development (or update) of Conservation Management and Monitoring Plans (CMMPs) and support their implementation. These plans are used as a means to guide the management and monitoring of HCV and HCS areas in concessions that include Best Management Practices (BMPs). Improved management practices is achieved when companies incorporate BMPs in their Standard Operating Procedures (SOP), allocate budget for implementation, and implement some or all the action plans.
UNIT OF MEASURE Number of firms
DISAGGREGATIONS Landscape
RATIONALE OR JUSTIFICATION FOR INDICATOR
Indicator used to track private sectors commitment regarding the positive benefits of conservation and sustainable use of forests.
DESIRED DIRECTION
Increasing
LINKAGE TO LONG TERM OUTCOME OR IMPACT
Adoption and implementation of BMPs will ensure that HCV and HCS areas in concessions are managed in perpetuity thus result on reduction in GHG emissions.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review the CMMP document, SOP, and budget line item allocated by concessionaires to implement the plan. Spot-check to observe field implementation.
DATA SOURCE CMMP document, SOP, direct observation
FREQUENCY AND Rolling as data available
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 113
INDICATOR 15
TIMING OF DATA ACQUISITION
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; photocopies of documentation; field visit cost
INDIVIDUAL RESPONSIBLE
Private Sector Engagement Coordinator
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
Concessionaires may refuse to release their financial reports
ACTION TAKEN TO ADDRESS LIMITATIONS
LESTARI field staff to spot-check concessionaires’ claims of implementation through field visits and site observations
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS
Triangulation of findings will be made through compilation of CMMP, SOP, budget documents review, direct observation, and interview concessionaires managers and staff. When counting progress toward meeting this target, LESTARI will track the intermediate milestones. LESTARI will also report the number of hectares under improved management practices.
Achievement of this indicator will contribute to Indicator #2.
Milestones
MoU with private sector
BMP package agreed upon
Adoption of BMP activities and SOP finalized
Budget for BMP activities incorporated into SOP
Training for BMP provided to company staff
BMP activities implemented
BMP monitoring and evaluation conducted
PRESENTATION OF Tabular with supporting narrative, as appropriate
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 114
INDICATOR 15
DATA Landscape Target Achievement
(number of private firms and hectare under improved management practices)
% Completion
REVIEW OF DATA Quarterly by ME&L Coordinator
REPORTING OF DATA
Quarterly and Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES The baseline is zero. The Project is at a starting point and no activity has been developed by this time.
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on potential private firms to collaborate in each landscape.
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 2
FY 2017 1
FY 2018 3
FY 2019 2
FY 2020 2
Targets LOP 10
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 115
INDICATOR 16
GOAL Reduced rate of deforestation and degradation
RESULT
IR 1. Improved forest management
KR 4. At least ten public-private partnerships (PPPs) promoting low-emissions
conservation oriented development established
INDICATOR TITLE Number of new USG-supported public-private partnerships (PPPs) formed
PERFORMANCE INDICATOR DESCRIPTION
INDICATOR LEVEL Outcome
INDICATOR TYPE FACTS PPP 5
PRECISE DEFINITION
Number of public-private partnerships (PPPs) formed during the reporting year. Partnerships can be long or short in duration (length is not a criteria for measurement). A partnership with multiple partners should only be counted as a single partnership. However, an institution may form more than one partnership with the same entity and each partnership should be counted separately.
A partnership is considered “formed” when there is a clear agreement, written and signed, to work together to achieve a common objective. Public-private partnership will be understood as a formal agreement between public and private entities with the purpose to provide and /or fulfill a public service in a schema defined to allow to the private enterprise to reach return on investment. Private partners could be for-profit enterprises, NGOs, private company, a community group, or a state-owned enterprise which seeks to make a profit (even if unsuccessfully).
Public-private partnership also include partnerships between LESTARI (on behalf of USAID) and the private sector entities in its landscape.
LESTARI will work directly with businesses and business associations from the onset in the creation of the PPPs to foster their support and to broaden impact to increase economic benefits while reducing GHG emissions and deforestation.
PPP scheme will require some amount of leveraged funding from the private sector that contributes to low emissions conservation oriented activities. The ratio of funding between two parties will be determined during the process of developing agreement. Though it is expected that there is one to one match of funding.
Investment mobilized through PPP scheme will contribute to Indicator #13 (investment mobilized).
UNIT OF MEASURE Number of new partnerships. The indicator should only measure the new partnerships in the reporting year.
DISAGGREGATIONS Landscape; Type of PPP
RATIONALE OR JUSTIFICATION FOR INDICATOR
Indicator used to track capacity of public and private entities to reinforce the legal frameworks for facilitating PPP, establish partnerships and support implementation of these kind of partnerships. The indicator is also used to measure sustainability of efforts and reinforce initiative for resilience and local benefits.
DESIRED DIRECTION
Increasing
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 116
INDICATOR 16
LINKAGE TO LONG TERM OUTCOME OR IMPACT
PPP will provide incentives for governments, private sectors and communities to embrace LEDS and conservation oriented practices.
DATA COLLECTION APPROACH
DATA COLLECTION METHOD
Review of partnership signed, implementation documentation, activities reports and proofs of disbursements
DATA SOURCE PPP agreement, activities reports and proofs of disbursements
FREQUENCY AND TIMING OF DATA ACQUISITION
Annually
ESTIMATED COST OF DATA ACQUISITION
Staff labor costs; photocopies of documentation
INDIVIDUAL RESPONSIBLE
Private Sector Engagement Coordinator
DATA STORAGE LOCATION
LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta
DATA QUALITY
INITIAL DQA TBD
DATE OF FUTURE DQA
Internally by the Tetra Tech M&E team in June 2016
PROCESS FOR FUTURE DQA
The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.
KNOWN DATA LIMITATIONS
None
ACTION TAKEN TO ADDRESS LIMITATIONS
None
DATA ANALYSIS REPORTING AND REVIEW
DATA ANALYSIS Compilation of data - Disaggregation by type of PPP - Comparison with targets - Analyze on improvements and impacts on climate change and biodiversity conservation aspects.
PRESENTATION OF Tabular with supporting narrative, as appropriate
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 117
INDICATOR 16
DATA Landscape Target Achievement
(disaggregated type of PPP)
% Completion
REVIEW OF DATA Annually by ME&L Coordinator
REPORTING OF DATA
Annual Report
BASELINE AND TARGETS
BASELINE DATA SOURCE
N/A
BASELINE NOTES N/A
BASELINE VALUE 0
BASELINE YEAR N/A
TARGETS NOTES Target is determined based on potential PPP in each landscape
YEAR TARGETS ACTUAL OBSERVATIONS
FY 2016 3
FY 2017 5
FY 2018 5
FY 2019 5
FY 2020 2
Targets LOP 20
NOTES ABOUT CHANGES TO THE INDICATOR
LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 126
ANNEX III: DATA QUALITY ASSESSMENT CHECKLISTS This Data Quality Assessment (DQA) Checklist is provided as a recommended tool that an
operating unit (OU) may use to complete its DQAs. If the OU prefers or has successfully
used a different tool for conducting and documenting its DQAs in the past, they are free to
continue the use of that tool instead. The checklist below is intended to assist in assessing
each of the five aspects of data quality and provide a convenient manner in which to
document the OU’s DQA findings.
USAID Mission or Operating Unit Name:
Title of Performance Indicator:
[Indicator should be copied directly from the Performance Indicator Reference Sheet]
Linkage to Foreign Assistance Standardized Project Structure, if applicable (i.e. Project Area,
Element, etc.):
Result This Indicator Measures [For USAID only] (i.e., Specify the Development Objective,
Intermediate Result, or Project Purpose, etc.):
Data Source(s):
[Information can be copied directly from the Performance Indicator Reference Sheet]
Partner or Contractor Who Provided the Data:
[It is recommended that this checklist is completed for each partner that contributes data to an
indicator– it should state in the contract or grant that it is the prime’s responsibility to ensure
the data quality of sub-contractors or sub grantees.]
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 127
Period for Which the Data Are Being Reported:
Is This Indicator a Standard or Custom
Indicator?
____ Standard Foreign Assistance
Indicator
____ Custom (created by the OU; not
standard)
Data Quality Assessment methodology:
[Describe here or attach to this checklist the methods and procedures for assessing the
quality of the indicator data. E.g. Reviewing data collection procedures and documentation,
interviewing those responsible for data analysis, checking a sample of the data for errors,
etc.]
Date(s) of Assessment:
Assessment Team Members:
USAID Mission/OU Verification of DQA
Team Leader Officer approval
X_______________________________________
YES NO COMMENTS
VALIDITY – Data should clearly and adequately represent the intended result.
1 Does the information collected measure what it is supposed to
measure? (E.g. A valid measure of overall nutrition is healthy
variation in diet; Age is not a valid measure of overall health.)
2 Do results collected fall within a plausible range?
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 128
3 Is there reasonable assurance that the data collection
methods being used do not produce systematically biased
data (e.g. consistently over- or under-counting)?
4 Are sound research methods being used to collect the data?
RELIABILITY – Data should reflect stable and consistent data collection processes and
analysis methods over time.
1 When the same data collection method is used to
measure/observe the same thing multiple times, is the same
result produced each time? (E.g. A ruler used over and over
always indicates the same length for an inch.)
2 Are data collection and analysis methods documented in
writing and being used to ensure the same procedures are
followed each time?
TIMELINESS – Data should be available at a useful frequency, should be current, and should
be timely enough to influence management decision making.
1 Are data available frequently enough to inform project
management decisions?
2 Are the data reported the most current practically available?
3 Are the data reported as soon as possible after collection?
PRECISION – Data have a sufficient level of detail to permit management decision making;
e.g. the margin of error is less than the anticipated change.
1 Is the margin of error less than the expected change being
measured? (E.g. If a change of only 2% is expected and the
margin of error in a survey used to collect the data is +/- 5%,
then the tool is not precise enough to detect the change.)
2 Has the margin of error been reported along with the data?
(Only applicable to results obtained through statistical
samples.)
3 Is the data collection method/tool being used to collect the
data fine-tuned or exact enough to register the expected
change? (E.g. A yardstick may not be a precise enough tool to
measure a change of a few millimeters.)
INTEGRITY – Data collected should have safeguards to minimize the risk of transcription
error or data manipulation.
1 Are procedures or safeguards in place to minimize data
transcription errors?
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 129
3 Is there independence in key data collection, management,
and assessment procedures?
3 Are mechanisms in place to prevent unauthorized changes to
the data?
SUMMARY
Based on the assessment relative to the five standards, what is the overall conclusion
regarding the quality of the data?
Significance of limitations (if any):
Actions needed to address limitations prior to the next DQA (given level of USG control
over data):
IF NO DATA ARE AVAILABLE FOR THE INDICATOR COMMENTS
If no recent relevant data are available for this indicator, why not?
What concrete actions are now being taken to collect and report these
data as soon as possible?
When will data be reported?
RECOMMENDATIONS FOR CONDUCTING DATA QUALITY ASSESSMENTS
1. Data Quality (DQ) assessor should make sure that they understand the precise definition of the indicator by checking the Performance Indicator Reference Sheet. Please address any issues of ambiguity before the DQA is conducted.
2. DQ assessor should have a copy of the methodology for data collection in hand before assessing the indicator. For USAID Missions, this information should be in the AMEP’s Performance Indicator Reference Sheets for each indicator. Each indicator should have a written description of how the data being assessed are supposed to be collected.
USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 130
3. Each implementing partner should have a copy of the method of data collection in their files and documented evidence that they are collecting the data according to the methodology.
4. DQ assessor should record the names and titles of all individuals involved in the assessment.
5. Does the implementing partner have documented evidence that they have verified the data that has been reported? Partners should be able to provided USAID with documents (process/person conducting the verification/field visit dates/persons met/activities visited, etc.) which demonstrates that they have verified the data that was reported. Note: Verification by the partners should be an ongoing process.
6. The DQ assessor should be able to review the implementing partner files/records against the methodology for data collection laid out in the AMEP (for USAID Missions only). Any data quality concerns should be documented.
7. The DQ should include a summary of significant limitations found. A plan of action, including timelines and responsibilities, for addressing the limitations should be made.
LESTARI
Wisma GKBI, 12th Floor, #1210
Jl. Jend. Sudirman No. 28, Jakarta 10210, Indonesia.
Phone: +62-21 574 0565
Fax: +62-21 574 0566
Email: [email protected]