evaluations activity 4th annual technical reportgovernment of san martín (goresam) and the regional...

April 2017

This document was prepared by Partners for Global Research and Development LLC (PGRD) for review by the United States

Agency for International Development (USAID). The Evaluations Project is made possible by the American people through

USAID.

EVALUATIONS ACTIVITY

4TH ANNUAL TECHNICAL REPORT

Contract Number: AID-527-C-13-00002

April 01, 2016 – March 31, 2017

Contract # AID-527-C-13-00002

ii

FOREWORD The purpose of this report is to present the activities performed by the USAID/Peru Evaluations Activity

during its 4th year of intervention. Section 1 presents the purpose of the project and its main

components, and briefly describes the results framework. Section 2 through Section 4 are dedicated

each to describe the objective, main activities and products of each of Evaluations’ intermediate results

or components: studies and evaluations are used for decision-making, USAID’s implementing partners

are able to manage per results, and local partners have strengthen capacities to perform evaluations.

Lessons learned in the year are described in Section 5, products and deliveries in Section 6, and Section

7 refers to next quarter activities. Annexes 1 and 2 briefly present the approach for conducting

evaluations and for building capacities, each. Annex 3 presents and discusses the monitoring indicators of

the Evaluations Activity.

In addition to this report, Partner for Global Research and Development (PGRD) presents a financial

and administrative report.


iii

EVALUATIONS ACTIVITY

4TH ANNUAL TECHNICAL REPORT APRIL 01, 2016 TO MARCH 31, 2017

Revised May 04, 2017

Contract Number: AID-527-C-13-00002

Prepared for: Miriam Choy, COR - USAID

Submitted by: Partners for Global Research and Development LLC (PGRD).

The contents of this report are the sole responsibility of Partners for Global Research and Development

LLC (PGRD) and do not necessarily reflect the views of USAID or the United States Government.


CONTENTS Acronyms and Abbreviations .......................................................................................................................................... 5

Executive Summary ........................................................................................................................................................... 7

1. Background ................................................................................................................................................................ 8

1.1. Results Framework ....................................................................................................................... 9

2. Evaluation studies are used to improve programming .................................................................................. 10

2.1. Performance evaluations ............................................................................................................ 11

2.2. Cross-cutting studies and Assessments ................................................................................. 14

2.3. Dissemination Activities ............................................................................................................. 17

3. Implementing partners are able to manage per results ................................................................................. 20

3.1. Facilitate the preparation of USAID/Peru PMPs................................................................... 20

3.2. M&E training and technical assistance for partners ............................................................. 21

3.3. Support to USAID/Peru Evaluation Team ............................................................................. 35

3.4. Other activities ............................................................................................................................. 36

4. Strengthening Local Capacity for Evaluation .................................................................................................... 37

4.1. Evaluation training and technical assistance for evaluation networks ............................. 37

4.2. Dissemination activities to strengthen capacities................................................................. 39

5. Lessons learned ....................................................................................................................................................... 40

5.1. Lessons learned from Y4 Evaluations ..................................................................................... 40

5.2. Lessons Learned Implementing the Capacity building model............................................ 41

6. Project management and operations ................................................................................................................. 43

6.1. Deliverables of Year 4 ................................................................................................................ 43

6.2. Products Prepared during Year 4 ............................................................................................ 44

7. Planned performance results for Year 5, 1st quarter ..................................................................................... 45

7.1. Evaluation studies are used to improve programming ....................................................... 45

7.2. Performance Monitoring and Evaluation Assistance ........................................................... 48

7.3. Strengthening Local Capacity for Evaluation ......................................................................... 49

Annexes ............................................................................................................................................................................. 50

Annex 1. Ten evaluation moments ......................................................................................................................... 50

Annex 2. Approach for capacity building in M&E ................................................................................................ 52


5

ACRONYMS AND ABBREVIATIONS

AD Alternative Development

ADS Automated Directives System

AIDER Asociación para la Investigación y Desarrollo Integral

AMI Amazon Malaria Initiative

AOR Agreement Officer's Representative

APIC Afro-Peruvian and indigenous communities

ARA Regional Environmental Authorities

CATIE Centro Agronómico Tropical de Investigación y Enseñanza

CDCS Country Development Cooperating Strategy

CEDRO Centro de Información y Educación para la Prevención del Abuso de Drogas

CEPCO Centro de Estudios y Promoción Comunal del Oriente

CLIN Contract Line Item Number

CMM Conflict Management and Mitigation

COP Chief of Party

COR Contracting Officer’s Representative

CPP Criminal Procedure Code

CSO Civil service organization

DAIS Integrated and Sustainable Alternative Development

DEC Development Experience Clearinghouse

DEVIDA Comisión Nacional para el Desarrollo y Vida sin Drogas

National Commission for Development and Life without Drugs

DO Development Objective

DRE Dirección Regional de Educación

DRESM Dirección Regional de Educación San Martín

EEL Enseñar es Liderar / Teaching is leading

EET Early Engagement Timeline

ELB Evaluation Learning Briefs

EPT Education For Work

EvalPerú Red Peruana de Evaluación / Peruvian Evaluation Network

GORESAM Gobierno Regional de San Martín /Regional Government of San Martin

GOREU Gobierno Regional de Ucayali / Regional Government of Ucayali

GRDS Gerencia Regional de Desarrollo Social

HCM Healthy Communities and Municipalities

HICD Human and Institutional Capacity Development

IDEA International Institute for Democracy and electoral assistance

INL International Narcotics Law Enforcement

IP Implementing partners

IR Intermediate result

LANN Liderando los Aprendizajes de Niñas y Niños


6

LOW Line of work

M (N°) Evaluation Moment (N°)

M&E Monitoring and Evaluation

MII Institutional Integrity Model

MMD Performance Improvement Methodology

MINAM Ministerio del Ambiente / Ministry of Environment

NGO Non-governmental organization

PAD Project appraisal document

PAHO Pan-American Health Organization

PCM P C Meetings

PE Performance Evaluation

PEI Plan Estratégico Institucional /Institutional Strategic Plan

PERUME Peruvian Network of Monitoring and Evaluation

PFSI Peruvian Forest Sector Initiative

PGRD Partners for Global Research and Development

PMP Performance Monitoring Plan

PO Program Office

POC Point of Contact

PPB Peru Bosques Project

PTPA Peru Trade Promotion Agreement

Q (N°) Quarterly (number of quarterly)

Q&A Questions and Answers

ROF Reglamento de Organización y Funciones

SER Asociación Servicios Educativos Rurales

SERFOR National Forestry and Wildlife Service

SOW Statements of work

SPDA Sociedad Peruana de Derecho Ambiental

RPAGV Plan Regional contra la Violencia de Género

TA Technical assistance

TBD To be determined

TMI The Mountain Institute

TNC The Nature Conservancy

UNICEF United Nations Children's Fund

UPCH Universidad Peruana Cayetano Heredia

USAID United States Agency for International Development

VRAEM Valley of the Apurimac, Ene and Mantaro Rivers

Y (N°) Year (number of year)


7

EXECUTIVE SUMMARY The USAID/Peru Evaluations Activity provides USAID with monitoring and evaluation (M&E) services

and assistance to implement the USAID Evaluation Policy. Its ultimate goal is to promote and foster

evidence-based programming through the implementation of studies and evaluations as well as

strengthening capacities of implementing partners (IP). The main objectives of the Evaluations Activity

correspond to the three components of this annual report:

1) Design and implement both impact and performance evaluation studies and conduct assessments

supporting USAID/Peru programs;

2) Provide performance monitoring and evaluation technical assistance, and capacity building for

USAID/Peru, its implementing partners, and host country government counterparts; and,

3) Strengthen the capacity of local evaluation institutions to design and implement evaluation

studies and assessments according to international standards.

Under the first component PGRD completed 24 studies or evaluations by the end of Year 4. The team

worked on 12 studies in all, including three newly designed in the year and ten studies that were

completed and approved, one of which was designed in Y4. In addition, the team also submitted for

approval eight learning briefs: six fact sheets and two learning briefs.

Year 4 saw the refinement of the PGRD evaluation approach, with the ‘moments’ adjusted from nine to

ten and the process incorporating the Automated Directives System (ADS) guideline changes emitted in

January 2016. The addition of one more evaluation moment allowed for a separate milestone regarding

the submission of an evaluation draft report to USAID.

PGRD also demonstrated high flexibility in responding to USAID’s requirements, both for developing

study designs and for presenting study results to different audiences. At the request of the Program

Office (PO), the CLIN 1 team made a presentation on Lessons from and for Activity Design that was

praised by the PO and other attendees, including the USAID sub-director, and generated new awareness

on specific issues regarding the 2017-2021 Country Development Cooperating Strategy (CDCS).

Under the second component, PGRD continued providing technical assistance to USAID implementing

partners to improve monitoring best practices. During the year, 285 people —142 men and 143

women— attended one or more of 33 M&E workshops. Eleven new M&E Plans, prepared with PGRD

assistance, were approved by the implementing partner and/or USAID. Forty-four participants obtained

the M&E Diploma and nine obtained postgraduate certificates in at least one module. At the

decentralized level, PGRD implemented four technical assistance plans in support of the Regional

Government of San Martín (GORESAM) and the Regional Government of Ucayali (GRU).

PGRD completed the transfer of all training materials to the Cayetano Heredia Peruvian University

(UPCH) Postgraduate School that implemented the Diploma, so that it will continue offering the course.

The diploma was launched publicly in December and the request for applications concludes in April.

Through component 3, PGRD provided technical assistance to the PERUME network in the elaboration

of their strategic plan. Fifteen participants from PERUME and EvalPerú networks obtained the M&E

Diploma and six received a postgraduate certification in at least one module. PGRD and UPCH hosted a

workshop during the Semana de la Evidencia 2016 presenting the tools on evidence management, taken

from the last module of the M&E Diploma.


8

1. BACKGROUND USAID Forward is an ambitious reform effort that changes the way USAID does business: with new

partnerships, an emphasis on innovation, and a relentless focus on results. This includes a renewed effort

to strengthen the use of monitoring and evaluation, to build local capacity to improve aid effectiveness,

and to strengthen collaboration and partnership with other donors. Towards this goal, in January of

2011, USAID released a new Evaluation Policy —updated in October 2016— aimed at improving

accountability, learning, and evidence-based decision-making. This policy requires increased rigor,

objectivity, and transparency in all of USAID’s evaluations.

The Evaluations Activity provides USAID/Peru with monitoring and evaluation services and assistance to

implement the USAID Evaluation Policy. Its ultimate goal is to promote and foster evidence-based

programming through the implementation of studies and evaluations as well as strengthening capacities

of implementing partners (IP).

The Activity main objectives are to:

• Design and implement both impact and performance evaluation studies and conduct assessments

supporting USAID programs;

• Provide performance monitoring and evaluation technical assistance, and capacity building for

USAID Peru, its implementing partners, and host country government counterparts; and

• Strengthen the capacity of local evaluation institutions to design and implement evaluation

studies and assessments according to international standards.

The Evaluations Activity has three components that address these objectives: 1) Evaluations, studies and

assessments; 2) Performance monitoring and evaluation assistance; and 3) Strengthening local capacity

for evaluation. The activity areas under each component include:

Component 1: In accordance with the USAID Evaluation Policy, to perform high quality evaluation studies

of the Peru bilateral and South America Regional portfolios. The evaluations produced by Partners for

Global Research and Development (PGRD) will measure and document project achievements and

limitations as well as make recommendations regarding future related activities.

Component 2: To a) provide technical assistance to USAID Peru to improve its performance monitoring;

and b) provide technical assistance and training to USAID Peru’s implementing partners —especially to

local and host country government counterparts— to strengthen their performance monitoring and

evaluation capacities.

Component 3: To strengthen the technical capacities of at least two local monitoring and evaluation

institutions so that they are able to design and conduct effective performance and impact development

evaluations according to international standards.


9

1.1. RESULTS FRAMEWORK

The Evaluations Activity Results Framework does not address a specific Development Objective (DO)

of the 2012 – 2016 USAID Country Development Cooperating Strategy (CDCS). Instead, it documents

how this activity will provide evidence to guide decisions for future programming in a context of

reduced resources, learn through evaluations, document USAID´s program results, and build local

capacity to monitor and evaluate programs.

The Results Framework links the activity’s tasks to one main objective, to foster evidence-based

programming, and three IRs, as illustrated in Figure 1.

Figure 1: USAID/Peru Evaluations Results Framework

Under IR1, end-users, including USAID representatives, will use the results of impact and performance

evaluations conducted by the activity to improve programs and projects.

Under IR2, implementing partners will use results-based management after improving their monitoring

capacity.

Under IR3, selected local partners will be able to conduct high quality evaluations after the project

strengthens their evaluation skills and capacities.

At the close of the fourth year of the Evaluations Activity, the project has achieved its objectives in

terms of delivering a wide range of M&E products that address the demands of USAID under the three

tasks mentioned before. The vast majority of these products seek to increase evidence-based

programming through the provision of evidence to USAID, whether through studies or improved

performance monitoring by partners. Additional efforts to promote evidence-based programming during

the last year include such activities as the diversification of products derived from studies (e.g.

supplemental presentations and fact sheets), the development of study recommendation plans to

facilitate the tracking of evidence-based programming, and the expansion of capacity building for M&E

professionals in Peruvian public institution counterparts, among others.


10

2. EVALUATION STUDIES ARE USED TO

IMPROVE PROGRAMMING The purpose of this component is to perform high quality evaluations and assessments of the Peru

bilateral and South America Regional portfolios, in accordance with the USAID Evaluation Policy.

Performance and impact evaluations assess the results of USAID programs and identify lessons learned

concerning which development interventions accomplishes the expected results and which do not.

Assessments support learning and planning by analyzing specific or crosscutting issues relevant to USAID

technical areas, such as gender. The Evaluations Activity has adopted an explicit approach geared

towards an effective delivery of its evaluation services and assistance to implement the USAID

Evaluation Policy.

Figure 2 presents the 12 ongoing evaluation studies and assessments for Y4. The status of evaluation

studies/assessments ongoing from Q12 is shown by the first stacked bar in dark grey with a white dot

highlighting the beginning point for Year 4. The quarter in which a final report received approval is

labeled on top of each completed study. For example, the performance evaluation of the Amazon

Malaria Initiative (AMI) began Q13 having started M3: selection of proposals. The submission of the final

report took place in Q15, receiving formal USAID approval in Q16. The Education Quality in AD

Communities (CEPCO), the Capacity Building Assessment, and the final evaluation of Decentralization

and Local Governance (ProDecentralization) were designed in Y4, with CEPCO being satisfactorily

completed in the same year. The Project has completed 24 evaluations and assessments by Y4 and

begins Y5 with only two ongoing studies - the Capacity Building Assessment scheduled to be completed

in the first quarter (May 2017) and ProDecentralization in Q19.

Figure 2: Cumulative Progress Report Year 4

Own elaboration, CLIN 1 April 2016.


11

Table 1 presents key milestones for evaluation studies and assessments. Draft and final report dates are

submissions to USAID.

Table 1 Milestones by Y3 & Y4 evaluations/assessments

# Study Type Status

SOW

Approved

Inception

Report

Internal

Findings

Draft

Report

Final

Report1

Report

Approved

M2 M6 M7 M8 M10

15 Healthy Communities and

Municipalities (HCM II)

PE Actual 3-Sep-15 17-Nov-15 18-Dec-15 21-Mar-15 11-May-16 10-Jun-15

16 Promoting Justice and

Integrity in Public Admin

(ProIntegridad)

PE Actual 04-Sep-15 23-Nov-15 18-Dec-15 15-feb-16 29-Aug-16 1-Sep-16

17 Education Sector Brief

Assessment

AS Actual 3-Dec-15 26-Dec-15 N/A 22-Mar-16 11-Aug-16 3-Nov-16

18 VRAEM Economic

Assessment

AS Actual 23-Oct-15 1-Dec-15 27-May-16 11-Apr-16 1-Jul-16 16-Aug-16

19 AD Ethnographic Study (AD

Ethno. Study)

AS Actual 3-Sep-15 19-Feb-16 29-Apr-16 12-Jul-16 27-Mar-17 31-Mar-17

20 DEVIDA Institutional

Capacity (DEVIDA)

AS Actual 14-Oct-15 18-Feb-16 5-May-16 1-Jul-16 26-Sep-16 28-Mar-17

21 Environmental Management

and Forest (Peru Bosques)

PE Actual 22-Mar-16 24-Jun-16 31-Aug-16 11-Oct-16 23-Nov-16 11-Jan-17

22 Education Quality in AD

Communities (CEPCO)

PE Actual 23-Sep-16 18-Nov-16 18-Jan-17 3-Mar-17 30-Mar-17 31-Mar-17

23 Amazon Malaria Initiative

Final Evaluation (AMI)

PE Actual 23-Mar-16 15-Jul-16- 30-Sep-16 28-Oct-16 5-Jan-17 31-Mar-17

24 Gender Analysis

AS Actual 31-May-16 26-Jul-16-- 29-Sep-16 14-Nov-16- 27-Jan-17 23-Mar-17

25 Capacity Building Activities

(Capacity Building Act.)

AS Planned 1- Sep-16 22-Nov-16 6-Feb-17 20-Feb-17 24-mar-17

Actual 1- Sep-16 25-Nov-16 21-Feb-17 08-Mar-17

26 Decentralization and Local

Governance

(ProDecentralization)

PE Planned N/A TBD2

Actual 28-Mar-17 TBD

AS= Assessment; PE= Performance Evaluation

1. USAID deliverables are highlighted blue.

2. Timeline to be determined (TBD) at the point of subcontract with evaluation team given USAID expected timeframes.

2.1. PERFORMANCE EVALUATIONS

# 15. Healthy Communities and Municipalities (HCM II)

Evaluation approved (M10); Document found at http://pdf.usaid.gov/pdf_docs/pa00m676.pdf.

This evaluation assessed the transferability and sustainability of the intervention model of the Healthy

Communities and Municipalities (HCM) II Activity, as well as its feasibility of adoption in sectors other

than health. It identified the HCM model as designed and observed in the studied localities, assessed the

behavioral change in each of the main involved actors - e.g. family and community - and the extent to

which governmental, non-governmental, local, regional and national organizations adopted the model,

analyses the adoption, transferability and sustainability of the HCM model in sectors of interest other

than health, and the potential contribution to USAID’s Development Objectives.

The Model was found to be versatile and adaptable in full or in part in different contexts such as

structured decentralization, alternative development strategies, corporate social responsibility practices,

among other processes. Leadership for the adoption of the Model can be assumed by governmental or

non-governmental organizations. In the first case, political sustainability tends to be higher than when

http://pdf.usaid.gov/pdf_docs/pa00m676.pdf


12

non-governmental actors assume it. However, if the leadership is provided by the private sector,

financial sustainability may be higher. The Model also serves as a joint public sector strategy to facilitate

decentralization processes. While it was conceived with a community approach, it was evident that one

of its strengths is focused at the institutional level. USAID areas can promote governance through the

adoption by the national and subnational governments linking with the communities for sectoral service

provision responding directly to population demands, or vice versa, through the generation of spaces

where the population expresses its needs and priorities to the governments.

# 16. Promoting Justice and Integrity in Public Administration (Pro-Integridad)

Evaluation approved (M10); document is found at http://pdf.usaid.gov/pdf_docs/pa00mfqr.pdf.

The purpose of this evaluation was to analyze the final performance of the Promotion of Justice and

Integrity in Public Administration (Pro-Integridad) Activity implemented between 2013 and 2016 in Lima

and Callao and selected judicial districts of the Peruvian Amazon. This evaluation assessed achievement

of the Activity’s objectives in terms of concrete and sustainable changes in Courtrooms, Prosecuting

Offices, and Specialized Public Attorneys for Corruption Cases; the skills and knowledge acquired

through training programs for judges, prosecutors, public attorneys, and administrative staff of the

different instances; the relevance, usefulness and degree of replicability of the Institutional Integrity

Model (MII) implemented in the Ministry of Justice and Human Rights; and its influence on civil society

organizations (CSOs) to perform citizen oversight of corruption cases under the new Criminal

Procedure Code (CPP).

The Activity addressed the fundamental organizational problems within the management of Courtrooms,

Prosecuting Offices and Public Attorney Offices and contributed to a noticeable improvement from the

initial state. Contributions were related to the six key processes identified in the Judiciary Power and

the Public Ministry; the target regarding the number of participants attending the Diplomatura was also

achieved. The implementation of the MII achieved expected progress in the establishment of basic

processes within the Ministry of Justice headquarters. Results from the MII could have been greater had

it be more customized to the Ministry of Justice institutional attributes. A more rigorous diagnostic at

the design stage would have facilitated the evaluation to justify and value some of the activities

undertaken. The execution period was too brief when compared to the institutional complexity of the

public system responsible for addressing corruption cases. There are outstanding challenges facing the

State in disseminating better norms and practices of internal organization and information regarding the

opportunities of civil society to participate in the CPP.

# 21. Environmental Management and Forest Governance Support Activity (Peru Bosques)

Evaluation approved (M10); document is found at http://pdf.usaid.gov/pdf_docs/pa00mhcr.pdf.

This evaluation analyzes the contribution of the Peru Bosques Project (PPB) to the progress made in

fulfilling the Environmental Chapter and the Forest Sector Governance Annex of the United States-Peru

Trade Promotion Agreement (PTPA). The evaluation focuses specifically on contributions to the

institutional strengthening of the forestry sector; capacity building in forest management; monitoring,

control, and supervision of illegal logging; and to the sustainable use of the forest by the private sector

and indigenous communities.

The PPB contributed substantially to the consolidation of the institutional reforms in the forestry sector

through three important results: support for the creation and implementation of the National Forestry

and Wildlife Service (SERFOR), improvement to the viability of the prior consultation process for the

new regulations for the Forestry and Wildlife Law, and in the customized design of the Regional

Environmental Authorities (ARAs) for the regional governments of Loreto, Madre de Dios, and Ucayali.

http://pdf.usaid.gov/pdf_docs/pa00mfqr.pdf

http://pdf.usaid.gov/pdf_docs/pa00mhcr.pdf


13

It intervened successfully with the private sector, which is a key ally in the fight against illegal logging and

for the sustainable use of forests. It also involved the participation of indigenous organizations in the first

regulatory consultation process and the training of community stakeholders to protect the forest.

However, there is a lack of capacity building at the community level. Capacity building efforts among civil

servants were hampered by a lack of political will and the slow implementation of the civil service law.

# 22. Methodology to Improve Educational Quality in Alternative Development Communities-

CEPCO

Evaluation approved (M10); document is found at http://pdf.usaid.gov/pdf_docs/pa00mnmr.pdf.

The main purpose of this study has been to evaluate the project "Support and expansion of the

methodology of Active Schools", implemented by the Centro de Estudios y Promoción Comunal del Oriente

(CEPCO), with USAID support, on the effects of the application of the entrepreneurship methodology

in the academic area of Education For Work (Educación para el Trabajo - EPT in Spanish) in high schools

in the Monzón district, located in Huanuco, Peru. The research questions address the results of the

implementation; the conditions, both in the context and in schools, which affect its progress; and the

motivating factors and ideal student profiles that maximize the possibility of generating licit ventures.

Likewise, this study also investigates the conditions that would allow this initiative to be sustainable and

replicable.

Findings reveal the significant contribution of the intervention to the improvement of educational

services, the development of technical skills, and changes in attitudes and values of students. However,

the local context limits the sustainability of the initiative, especially because of the short duration of the

intervention. Finally, the role and leadership of school principals is crucial to obtain the expected results

and any possible replication.

# 23. Amazon Malaria Initiative (AMI) Final Evaluation

Evaluation approved (M10); document is found at http://pdf.usaid.gov/pdf_docs/pa00mnmm.pdf.

The objectives of this final evaluation were to: (1) identify the contribution of the USAID supported

Amazon Malaria Initiative (AMI) to the ability of eleven partner countries’ National Malaria Control

Programs to control malaria from 2001-2015; (2) evaluate the capabilities of partner countries to

address the changing scenarios of malaria transmission; and, (3) identify the main challenges that

countries may face while confronting new scenarios of malaria transmission as well as the challenges that

should be addressed by international cooperation agencies.

AMI partner countries have shown a substantial but heterogeneous reduction of malaria cases since

2001, including some setbacks and slower progress in later years, especially the last four. Partner

countries have achieved substantial progress and developed important capabilities in all six of the AMI

Program technical focus areas. AMI supported capacity development that addressed country-specific

malaria control needs and scenarios. The program served as a platform for inter-country collaboration,

introducing innovations and strengthening information systems through national communication

strategies and alliances. Key capacities for malaria control have been built and enhanced, and these

continue to respond to changing epidemiological scenarios. However, there is a need to make

improvements, such as better planning and monitoring of activities including alignment to the Pan-

American Health Organization´s (PAHO) strategy, in order to consolidate progress and prevent the

reintroduction of malaria.

# 26. Decentralization and Local Governance Final Evaluation (Prodecentralization)

Ongoing evaluation, SOW approved in March 2017 (M2).

http://pdf.usaid.gov/pdf_docs/pa00mnmr.pdf

http://pdf.usaid.gov/pdf_docs/pa00mnmm.pdf


14

Identification of questions for the final evaluation of the Prodecentralization Program took place

between December and January 2017. PGRD submitted draft SOW for USAID comments on March 2,

with a final submission on March 16. SOW approved on March 28. Expressions of interest were sent

late January and the call for proposals mid-March. Q&A session to clarify feedback regarding the SOW

took place on March 29; proposals due April 5.

2.2. CROSS-CUTTING STUDIES AND ASSESSMENTS

# 17. Education sector brief assessment

Study approved (M10); document is found at http://pdf.usaid.gov/pdf_docs/pa00mhcs.pdf.

Significant progress has been made in educational outcomes in Peru, measured by learning achievement

in both reading and mathematics. However, inequities persist, especially when comparing urban and

rural areas, and schools with public and private management. This study sought to assess the current

situation of basic education (initial, primary and secondary), in order to identify the most important lines

of intervention and in which there is space for international technical cooperation.

The diagnosis address the educational performance issues: characteristics of the teacher, quality of the

infrastructure, availability of equipment and curricular grill in the classroom, and management, which in

turn affect educational performance. The analysis identifies that there is a need for support - through

political priority, scarce resources or little technical capacity - in the policy of accompanying teachers, in

the implementation of the Selva Plan, in the use of information and communication technologies in

teaching models, in the strengthening of school and decentralized management capacities, and the

development of new evidence for public policy.

# 18. VRAEM Economic Assessment

Study approved (M10); document is found at

http://www.cies.org.pe/sites/default/files/files/otrasinvestigaciones/archivos/01-vraem_final.pdf

The Peruvian Valley of the Apurimac, Ene and Mantaro Rivers (VRAEM) has an undiversified agricultural

economy dominated by the cultivation of coca. The production of coffee, cacao and coca occupy almost

all of the valley economy’s fixed factor, land, and share available labor. In this economy, where licit and

illicit activities coexist, options for promoting economic growth must be understood in the context of

the factors that determine the profitability of cacao and coffee relative to the profitability of coca.

The study identifies four barriers to economic growth: 1) lack of a clear and effective application of the

law to combat illicit drug trafficking; 2) inadequate transportation infrastructure; 3) market restrictions

and the absence of an adequate labor force; and, 4) weak property rights, particularly with regard to the

communal property rights of indigenous communities. Foremost among the recommendations is

reducing the profitability of illicit coca cultivation through the state’s adoption of an effective counter-

drug strategy for the VRAEM implemented by the institutions responsible for Peru’s counter-drug

program.

http://pdf.usaid.gov/pdf_docs/pa00mhcs.pdf

http://www.cies.org.pe/sites/default/files/files/otrasinvestigaciones/archivos/01-vraem_final.pdf


15

# 19. AD Ethnographic Studies in Huanuco and Ucayali

Two of three ethnographic studies approved (M10); documents can be found at

http://pdf.usaid.gov/pdf_docs/PA00MPD3.pdf and http://pdf.usaid.gov/pdf_docs/PA00MPCZ.pdf

These ethnographic studies aimed to investigate household decision making behavior with regards to

grow or not grow coca in the communities of Agua Blanca and Rio Espino, in the department of

Huánuco, and Shanantia, in the department of Ucayali.

The key findings lead to conclude that: (a) families who grow cacao can exit the cycle sowing-selling of

coca leaf, but also can return to it through their employment in coca harvesting; (b) the process of

economic autonomy, which begins in children and adolescents since age 13/14, enables them to grow

coca, cultivate it, invest and get benefits from it; (c) labor migration informs the adolescent males of

other economic activities within and outside the region, be they legal or illegal, which facilitates the

cultivation of coca outside the community; (d) the coca functions as a rescuer/useful crop to meet

specific and expensive needs, especially if there are no stable revenue sources during the growing of

cacao plants, which lasts three years, and only this year will yield their first harvest; and (e) the

installation of State control, the perception that this exists, and appreciation for State actions

discourages the replanting of coca.

# 20. Institutional Strengthening Needs Assessment of the National Commission for

Development and Life without Drugs (DEVIDA)

Study approved (M10), to be uploaded in DEC upon USAID instruction.

DEVIDA is in the stage of "expansion and consolidation", the third of four stages of USAID’s Institutional

Development Framework. Its personnel possess the technical capacity and are aware of its clear

mandate, the institution has adequate delegation of administrative decisions, use of work plans that guide

the availability of more financial resources, financial controls implemented and processes of periodic

external audit, as well as the capacity and experience in the work with farmers, communities and

subnational governments.

The support of international and especially USAID cooperation has been decisive. USAID's contribution

to capacity building has generated sustainable results such as political empowerment and budget increase

in the entity, the monitoring system, updating of the organizational structure, among others. USAID's

direct support for activities in the Integrated and Sustainable Alternative Development approach (DAIS)

has been important but does not produce sustainable results in DEVIDA but immediate results.

The most important factor to ensure the sustainability of the results in DAIS is to strengthen the

stewardship role of DEVIDA to convene and achieve that national and subnational government entities

articulate their own plans for territorial planning, multisectoral and multilevel approach and are

implemented in a coordinated manner. To this end, internal capacity building needs have been identified

to strengthen its leadership, define the responsibilities of institutions that are part of the (DAIS)

approach, develop their capacities and those of executing institutions and systematize the knowledge

generated for the continuous improvement of the DAIS model.

24. Gender Analysis

Study approved (M10); Spanish version is found at http://pdf.usaid.gov/pdf_docs/pa00mnmg.pdf.

Additional products are an Infographic and the English version of the study.

The purpose of this study is to analyze the gaps between men and women in access to and control over

economic, political and social resources; to examine gender-based restrictions in the sectors and region

where USAID operates; to analyze how gender-based violence affects women’s participation; and to

review the equal opportunity policies implemented by the Peruvian government.

http://pdf.usaid.gov/pdf_docs/PA00MPD3.pdf

http://pdf.usaid.gov/pdf_docs/PA00MPCZ.pdf

http://pdf.usaid.gov/pdf_docs/pa00mnmg.pdf


16

The analysis revealed that the primary gender-based restriction on equitable participation and access to

equal opportunities is the assignment of work roles. Women’s contributions are through

uncompensated domestic work, which represents 20% of Peru’s gross domestic product. These

contributions are not recognized, while at the same time they limit women’s involvement in paid, social

and political activities. Furthermore, gender-based violence affects 71% of Peruvian women, and physical

violence affects 32%. The costs of this violence on small businesswomen is 1% of the gross domestic

product. Although the policy and statutory framework for equal opportunity is quite advance, there

have only been small administrative changes to address gender mainstreaming, and no functions or

budget programs have been created to allocate specific resources to this goal. This study’s

recommendations focus on providing visibility on the effects of continuing the gaps and restrictions on

equal opportunity, the need to incorporate a focus on gender in project planning, and including

indicators for monitoring and allocating a budget.

25. Capacity Building Activities Study

Study ongoing (M8): Final report under elaboration

This study analyzes seven specific experiences of capacity development in selected USAID projects, and

presents conclusions and recommendations based on implemented models, methods, activities and

results of the identified cases.

The study started in November 2016, and the results were presented in March 15, 2017. The final

report will be resubmitted by the subcontractor in April 28.

Cancelled Studies

While discussing Y5 pipeline starting Q16, USAID AD team identified a list of studies that were

requested in support of the elaboration of the AD Project (DO1). Several studies were cancelled for a

variety of reasons. These are as follows:

AD ethnographic studies II: This study was scheduled at the beginning of Y4, then its priority in the

planned pipeline was lowered and then canceled per USAID decision.

Communities for AD: This study was one of the first communicated to PGRD as part of the AD Y5

pipeline which was discussed since mid-January. PGRD shared with USAID initial concept note which

was discussed on January 23 to define and receive approval of the research questions. PGRD submitted

a refined set of research questions on February 2 (M1). At this point, study was placed on hold as the

AD team was still defining its pipeline. The AD Municipal Diagnostic took precedence over the

communities for AD assessment, and was eventually canceled.

AD Municipalities Assessment/Municipal Diagnostic: Previously, PGRD was asked to provide information

on 11 targeted municipalities and their provincial local government from current public databases; a

matrix of indicators and a power point presentation were submitted on February 23. Following these

data requests, PGRD received formal request from PO to conduct a municipal diagnostic on March 3

accompanied by the concept note. CLIN 1 submitted a revised question matrix which was discussed in a

meeting with USAID AD team on March 9. Per USAID request, a Spanish version of the SOW was

submitted on March 16 (M2). After reviewing the concept note with PGRD management, PGRD

presented the Regional Contract Office a plan to mitigate possible conflict of interest in implementing

the assessment. USAID opted to not pursue the study through Evaluations.

AD Verification studies: USAID requested PGRD to undertake a data verification exercise of four AD

activities – Peru Cacao Alliance implemented by Palladium, New Alternatives Program implemented by

New Alternatives Venture, Economic Development Alliance implemented by Technoserve, and Digital

Inclusion implemented by CEDRO. PGRD met with DO1 and PO staff on March 9 to better understand


17

purpose and requirements regarding the results verification for four USAID activities. Based on a

potential conflict of interest, PGRD submitted a letter to USAID declining to implement the verification.

2.3. DISSEMINATION ACTIVITIES

Presentations of Studies’ Results

The presentation of evaluation findings during PCM meetings accounts as the single moment of most

visibility during the implementation of an evaluation study/assessment. To capitalize on these

opportunities, PGRD works closely with the evaluation teams to address both the content of the

presentation and the delivery. Once presented to PGRD, PO and COR, comments are incorporated

into the final version presented to a wider USAID audience during PCM meetings. Additional

presentations are delivered per USAID request, as presented in Table 2. In total, PGRD provided 20

presentations on study results and two Plan of Recommendations meetings over the year. Table 2 also

includes meetings to discuss the Plan of Recommendations with activity COR and M&E staff, essential to

identifying and committing to follow-up actions derived from evaluation recommendations.

Table 2: Presentations to USAID of Evaluation Studies and Assessments

N° Evaluation studies Date of

meeting

Audience

16 Promoting Justice and Integrity in Public

Administration

13-Apr-16 COR -Plan of Recommendations

11-Oct-16 PCM-Mission

18 VRAEM Economic Assessment 27-May-16 DO1

6-Jun-16 Implementing Partners

9-Sep-16 International Narcotics Law

Enforcement (INL) agency

15-Sep-16 PCM – Mission

19 AD Ethnographic Studies 29-Apr-16 PO

10-Aug-16 PCM - Mission

20 DEVIDA Institutional Capacity Assessment 5-May-16 PO

6-Jun-16 DO1 M&E

6-Jul-16 PCM - Mission

21 Peru Bosques 31-Aug-16 PO

12-Sep-16 IP

13-Oct-16 PCM - Mission

17-Oct-16 COR – Plan of Recommendations

22 Amazon Malaria Initiative 30-Sep-16 COR & PO

23 Methodology to Improve Educational Quality

in AD Communities (CEPCO)

18-Jan-17 AOR &PO

1-Mar-17 PCM - Mission

24 Gender Analysis 29-Sep-16 PO – Gender Team

8-Feb-17 PCM - Mission

25 Capacity Building 21-Feb-17 M&E Team – COR

15-Mar-17 PCM - Mission


18

Brief learning reports

Y4 has marked an important milestone in the iterative learning process from design to approval for the

three types of learning documents produced by Evaluations, described as follows:

Evaluation Fact Sheets: two-page documents that provide brief background on the evaluation,

summarize key findings, and present main conclusions and recommendations based on final reports of

evaluation studies/assessments.

In Y4, PGRD delivered six evaluation fact sheets, two of them in English and Spanish. To this end, PGRD

conducted an international selection and recruitment process (Q15) for a professional writer and

designer, opting to hire two local consultants after multiple samples from each consultant. Table 3

presents a summarized timeline presenting the iterative process of drafting these six evaluation fact

sheets, which are pending USAID approval. Two of these fact sheets, the VRAEM Inclusive Growth

Study and the AD Ethnographic Studies were submitted both in Spanish and English. All evaluation fact

sheets will be translated upon USAID approval of the Spanish version.

Table 3: Y4 Evaluation Fact Sheet Pipeline to USAID

Evaluation Study 1st submission 2nd submission Final submission1

VRAEM Inclusive Growth Study 9-Dec-16 15-Mar-17 30-Mar-17

AD Ethnographic Studies 9-Dec-16 15-Mar-17 30-Mar-17

DEVIDA Institutional Capacity Assessment 9-Dec-16 On hold by USAID On hold by USAID

Healthy Communities and Municipalities

(HCM II)

15-Mar-17 -- 30-Mar-17

Perú Bosques 15-Mar-17 -- 30-Mar-17

Peruvian Forest Sector Initiative (PFSI) 15-Mar-17 -- 30-Mar-17

1. Pending approval of all evaluation fact sheets with the exception of DEVIDA which is on hold per USAID decision.

A list of fact sheets to be delivered next quarter is presented in section 7 of this report.

Evaluation learning briefs: two-page analytical documents that examine diverse project development

topics such as results sustainability, development of theories of change (ToC), criticality and implications

related to knowing beneficiaries, and gender mainstreaming, among others. In contrast to the fact

sheets, evaluation learning briefs are based on CLIN 1´s experience from managing previous evaluation

studies and assessments.

PGRD delivered its first evaluation learning brief – Knowing Your Beneficiary. Feedback from USAID is

expected in the latter part of April. Planned Evaluation Learning Briefs (ELBs) for next quarter is shown

in section 7. Subject/ titles and/or priority order are subject to modification.

Evaluation practice briefs: a four to six page reflective analysis on how PGRD adopts and

implements USAID’s best practices and recommendations regarding evaluations – from design to

dissemination and learning.

PGRD also delivered its first evaluation practice brief – Evaluation Utilization: Best Practice. Feedback

from USAID is expected in the latter part of April. Planned ELBs for Y5 is shown in section 7. Subject/

titles and/or priority order are subject to modification.

To maximize learning from learning briefs and practice briefs and to incorporate USAID feedback,

PGRD suggests hosting discussion sessions with a broader USAID audience prior to publishing.


19

Other activities

In addition to the evaluation studies, assessments and brief learning reports aforementioned, CLIN I

hosted an effective and targeted brief presentation on lessons from and for activity design based on

evaluations conducted (see Table 4). A sample of eight activities were analyzed and scored on six design

topics included in the ADS 201, providing a snapshot on the average performance on each topic. Main

issues and related observed effect were presented, reinforced by positive and negative examples in each

case.

Feedback from the USAID audience, including USAID sub-director and Program Office Chief was very

positive, reinforcing the criticality of managing main design issues like assumptions and measurable

objectives, among others. The presentation was timely as the Mission was preparing its new CDCS and

corresponding project appraisal documents (PADs).

Table 4: Other Learning and Dissemination Activities

N° Work material

produced

Date Audience Purpose

1 PPT - Lessons from

and for Activity

Design

23-Feb-2017 USAID director,

office chiefs, and

technical teams.

Incite improvements in activity

design based on recurrent lessons

identified from PGRD evaluations.


20

3. IMPLEMENTING PARTNERS ARE ABLE TO

MANAGE PER RESULTS The purpose of this component is to support USAID and its implementing partners (IP) to complete the

following processes and management documents:

• USAID M&E plans for each of the three Development Objectives, and the Mission itself, are

included in the Country Development Cooperation Strategy (CDCS).

• Implementing partner M&E Plans align with the USAID results framework.

• Tools for tracking implementing partners and DO’s M&E Plans

• Strengthening M&E capacity of local IPs (training and technical assistance)

These processes and products are being achieved through a) technical assistance (TA) to USAID to

improve its performance management tasks; and b) technical assistance and training to USAID’s

implementing partners, especially to local and host country government counterparts, to strengthen

their performance M&E capacities.

The overall approach of this component is to promote an M&E culture during program implementation,

and the use and management of M&E evidence for early identification and resolution of issues before

they become problems.

In order to achieve these purposes, intermediate result 2 includes three main activities:

• Facilitate the preparation of USAID/Peru M&E Plans

• M&E training and technical assistance for partners, and

• Support to the USAID/Peru Evaluation Team

This section describes the results obtained during the second year of project’s implementation.

3.1. FACILITATE THE PREPARATION OF USAID/PERU PMPS

3.1.1. Assist USAID Peru Team in the elaboration of DO’s M&E Plans

During this last year USAID did not requested any support for the elaboration of DO M&E Plans

because 2016 was a year for planning the new mission strategy. However, PGRD was asked for TA in

updating the Mission’s results tracking table for reporting to Washington. PGRD staff reviewed and

updated the values of four performance indicators and of twelve context indicators for years 2010 to

2016, according to data availability. This information allowed USAID to determine to what degree its

development objectives are being achieved. Reports and updated tracking table were sent to the COR

on March 15 and 30.

3.1.2. Assist USAID Peru Team and partners in other issues related to the M&E Plan, as

required by USAID

In Y4, PGRD provided support to USAID in reviewing the M&E Plan of the “Capacity Development and

Engagement program” implemented by Cayetano Heredia Peruvian University (UPCH). This review

sought to ensure consistency and coherence between the M&E Plans from “Liderando los Aprendizajes

de Niñas y Niños” (LANN) and “Enseñar es Liderar” (EEL), both activities receiving technical assistance

from UPCH. The COR approved the M&E Plan on February 22, 2017. The new M&E Plan will measure

implementation progress and performance results and provide USAID with improved data in reporting.


21

3.2. M&E TRAINING AND TECHNICAL ASSISTANCE FOR PARTNERS

The Project continued the implementation of the Human and Institutional Capacity Development

(HICD) approach, the USAID model for sustainable performance improvement. The model consists of

two components: the first aimed at improving the performance of the institution based on best practices

for monitoring and evaluation, and the second to improve the competences of the professionals that

perform monitoring and evaluation functions.

Training and technical assistance were delivered through three major activities during the year: the M&E

Postgraduate Diploma, the implementation of ad-hoc technical assistance plans and the organization of

training workshops on M&E topics, based on the diagnosis of each organization. Annex 2 provides a

detailed description of Evaluations capacity building approach.

3.2.1. M&E Postgraduate Program (M&E training plan)

During Year 4 two groups of participants completed the M&E Postgraduate Program, the academic

instruction of which was administered through subcontract by the Cayetano Heredia Peruvian

University (UPCH). The Program was implemented as a Postgraduate Diploma for USAID implementing

partners and as individual modules to other participants according to their own interests.

The phases of the post-graduate diploma activity completed over the year include: i) implementation, ii)

validation, iii) evaluation and competencies certification, iv) institutionalization and v) preliminary design

of follow-up with graduates (see Figure 3).

Figure 4 presents the disaggregated activities in each phase.

First Group Second Group

M&E Postgraduate Program

Preparatory phaseImplementation

phase

Validation

Phase

Evaluation and Certification Phase

Institutionalization

Phase

Follow up of graduates Phase

Figure 3: Phases and Groups of the Postgraduate Program


22

Figure 4: Phases and Activities of the Postgraduate M&E Program

M&E Postgraduate Program

Implementation phase

First Group: Evidence Management Module

First Group: Integration Worshop

Second Group: Integration Worshop

Second gropup: Evidence Management Module

Second group: Monitoring Module

Second gropup: Evaluation Module

Validation phaseValidation of content and preparation

of final documents

Follow up of graduates Phase SOW design (preliminary)

Institutionalization

Phase

Call and admission

Approval of the new curricular plan

Transference of the Program

Evaluation and Certification Phase

Evaluation of compentencies (incl. competencies validation)

M&E Postgraduate Program closure

Academic certification

of competencies


23

3.2.1.1. Implementation phase

During the year, two groups of participants participated in the M&E diploma (see Table 5).

First Group

Having completed most their coursework and practicum in the previous year, the first group finished

two modules and completed the program.

Evidence Management Module: This module started on February 27 and ended on April 8, 2016.

Participants had as main product the elaboration of an evidence report for the evaluation of a project.

They also prepared a plan to communicate the results of an evaluation. Participants rated the module as

“very good”.

Integration Workshop: This one-day workshop was dedicated for the students to present selected

products of the four modules. A team of teachers and external invitees acted as a jury to provide

feedback to the presented products.

Second Group

The Second group had 27 participants who finished three modules and completed the whole program.

Monitoring Module: The Second Group completed the Monitoring Module on April 28, 2016. The

session started with twenty-eight participants (one participant from the first group, DRESM, joined the

module) and twenty-seven passed (TNC informed us of the withdrawl of one participant for personal

reasons). The products of the Module are the M&E Plan for the organization and monitoring reports.

Eight were new M&E plans, and four were an update of an existing M&E plan. Two were approved by

the organization and USAID, and are being implemented.

Evaluation Module: Prior to the start, the UPCH delivered an orientation workshop for the new

teacher team. The module started with twenty-six participants. Twelve groups of participants presented

their evaluation designs, data collection tools, SOW for the implementation of the evaluation designs

and critical analysis of an evaluation report and the design of an evaluation. Teachers and guest

specialists from USAID and PGRD provided recommendations for improving the designs.

Evidence Management Module: This module started on February 27 and ended on April 8, 2016.

Twenty-four participants completed the Module on Evidence Management. Participants prepared the

Evidences Report for the evaluation design of a project and a communications plan. Participants rated

the module as “very good”, mainly because of its communication component.

Integration Workshop: The integration workshop took place on September 10 with 26 participants.

Groups of participants made presentations of select products developed during the four modules. The

presentations were scored by a team of teachers from the Diploma and specialists from USAID.

Participants had to respond to questions and received feedback from these evaluators. All of them

passed the workshop, which was the last course of the M&E Program.


24

Table 5: Schedule of the Postgraduate Program- Activities during 2016

Module First Group Second Group

Period Attendance

moment

Period Attendance

moment

Opening and Introductory

Workshop

August 21 January 21

Key Competencies August 22 to

October 2, 2015

August 22- 23 January 22 to

March 4, 2016

January 22-23

October 1- 2 March 3- 4

Monitoring October 3 to

November 27, 2015

October 3- 4 March 5 to April

28, 2016

March 5-6

November 26-27 April 27-28

Evaluation November 28 2015

to February 26,

2016

November 28-29 April 29 to July 22,

2016

April 29-30

January 15-16 June 10-11

February 25-26 July 21-22

Evidence Management February 27 to April

8, 2016

February 27-28 July 23 to

September 9,2016

July 23-24

April 7-8 September 8-9

Integration Workshop April 9 April 9 September 10,

2016

September 10,

2016

3.2.1.2. Validation phase

Based on the experience of the first group, PGRD and UPCH reviewed and validated all training

materials of the Postgraduate Program: curriculum, syllabus, guidelines, texts and evaluation tools. The

improved materials were used with the second group of participants. PGRD technical team, UPCH

coordinators and the teachers of each module were involved in the validation of materials. PGRD

prepared validation guidelines used during this process.

Validated materials have been uploaded to the DRIVE shared with USAID:

https://drive.google.com/open?id=0B-hFtToW1ZYvRjhScGJyZEtNclU

3.2.1.3. Evaluation and Certification Phase

Fifty-three participants from 20 USAID partners attended the M&E Postgraduate Diploma, 26 in the first

group and 27 in the second one. Six implementing partners are public institutions (MINAM, GORESAM,

GOREU, DRESM, DREU and DEVIDA) and four are non-governmental organizations (NGO). The total

number of participants is shown in Table 6.

https://drive.google.com/open?id=0B-hFtToW1ZYvRjhScGJyZEtNclU


25

Table 6: Number of Participants with Diploma and Postgraduate Certificate

N° of scholarships

provided

Participants

with Diploma

Participants

with

Certificate

Total

number of

participants USAID Implementing

Partner DO

First

Group

Second

Group Total

MINAM DO3 3 6 9 9 9

GORESAM 1 3 4 4 4

GOREU 3 1 4 1 3 4

DRESM DO2 1 2 3 2 1 3

DREU DO2 2 2 2 2

DEVIDA DO1 3 5 8 8 8

CEDRO DO1 3 3 3 3

Technoserve DO1 2 2 1 1 2

SER DO2 1 1 1 1

CEPCO DO2 2 2 2 2

SPDA DO2 2 2 2 2

Grupo Propuesta

Ciudadana DO2 2 2 1 1 2

AIDER DO3 1 2* 3 3 3

TMI DO3 2 2 2 2

Lutheran World Relief DO3 1 1 1 1

CEDEP DO2 1 1 1 1

IDEA Internacional DO2 1 1 1 1

UPCH DO2 1 1 1 1

SERFOR DO3 1 1 1 1

TNC DO3 1 1 1 1

Total 26 27 53 44 9 53

*The NGO AIDER financed the participation of one of its employees.

Competencies evaluation: PGRD measured the level of M&E competencies of all participants prior

to beginning and following the completion of the diploma program. The baseline data was used to

identify what areas required strengthening through the training program. The final data was used to

measure the participants’ progress.

Figure 5 shows that the participants were had basic levels of technical competencies and required

improvement across the board. After participating in the M&E program, all technical competencies

improved from basic to medium level, although most will need additional strengthening to reach an

advanced level.

When measuring the change in key competencies, also known as ‘soft competencies’, there is a slight

improvement from 2.58 to 3.05, which indicates the need of reinforcement.


26

Figure 5: Measurement of M&E Competencies Before and After Training Program

Source: Self-diagnostic online survey.

Period: Group I: August 2015 – May 2016, Group II: December 2015 – October 2016

Base: Group I: 19, Group II: 22

Academic Certification and Closing Ceremony: Forty-four participants completed the Diploma

and achieved the competencies in M&E, 20 from the first group and 24 from the second. They received

the M&E Diploma, which certifies the achievement of the 24 academic credits of the Program, and the

competencies obtained.

The University Rector led the closing ceremony on September 21, 2016. The Deputy Director of the

Regional Program Office also presided over the ceremony. Other participants were the COR/AOR

from implementing partners, the executive directors, professors and relatives from the graduates. The

nine participants that passed three or less modules received only module certificates (rather than the

diploma).

PGRD prepared a video and eight posters regarding the participants’ achievements. In the video,

participants describe the contribution of the Diploma to their professional development and to their

institutions’ capabilities. These were also displayed publicly at USAID and uploaded to the USAID

website: https://www.usaid.gov/news-information/videos/monitoring-and-evaluation-diploma-program.

3.2.1.4. Institutionalization Phase

Transference of training materials: USAID approved the transfer of the Diploma Program to

UPCH. PGRD sent a letter to UPCH explaining the conditions of the transference, which the university

accepted. UPCH will have exclusive rights to the materials during 2017 and will elaborate an

implementation plan that will include periodic meetings with PGRD to report on the advances and

results. The physical transference of the curricula contents was held during the closing ceremony.

A complete list of the training materials has been delivered to UPCH, signed by the Director of the

Postgraduate School and the Project COP.

New curricular plan approval: On November 7, the steering committee of UPCH Postgraduate

School approved the validated curricula plan of the M&E Diploma, and the University Council approved

the Program: Resolution TRANS-SEGEN-UPCH-2016-CU-0807.

UPCH designed and implemented a Convalidation Plan and a methodology for the certification of

competencies. This plan has been approved, was tested during the evaluation of Networks’ participants

(see next section of this report) and will be part of the academic offer of the Program.

https://www.usaid.gov/news-information/videos/monitoring-and-evaluation-diploma-program


27

On December 6, UPCH presented the M&E Postgraduate Diploma in the Conference “M&E Trends and

Challenges in Development Programs and Projects”. Fifty people attended the conference.

Call and admission: The Unit of Academic Services will conduct the diploma, carry out marketing and

make the call for candidates to the M&E Diploma1.

UPCH launched the Diploma and sent the brochure to USAID partners who will have a discount of 15%

of the total cost. For dissemination purposes, UPCH also organized information sessions attended by

130 persons. Classes will start on April 22.

UPCH Subcontract Management: Contract ended on September 30, 2016. PGRD continues

attending coordination meetings, such as one with the docent team on February 15.

3.2.1.5. Graduates follow up phase

PGRD is elaborating a plan to follow-up the graduates to obtain information about how they are using

the competences acquired, to update the requirements of the M&E practice, to update the address book

and to determine the social mobility of the graduates, among others.

The implementation of UPCH Diploma has been constantly assessed and has received positive feedback.

Participants have evaluated the modules and teachers with satisfactory results and have provided

recommendations that have been implemented by the UPCH.

3.2.2. Regional Workshops

During the second part of the year, the PGRD team focused on training and technical activities for

USAID regional partners: Government of San Martin (GORESAM), Regional Office of Education San

Martin (DRESM), Government of Ucayali (GOREU) and Regional Office of Education Ucayali (DREU).

Activities started after PGRD conducted an initial baseline measuring institutional practices and

identified M&E capacity strengthening needs.

PGRD prepared a training module for this purpose named, “Designing a Results Oriented M&E Plan”.

The purpose of the training module is to provide the conceptual capacities and the skills and tools for

the elaboration of an M&E Plan that could be used for any development program or Project. It is

organized in three phases, each with an on-site and on-line component. The regional government sought

the participation of Universidad Nacional de Ucayali in order to obtain a certification for the training.

Participants prepared their M&E plans addressing the monitoring purposes shown in Table 7.

Table 7: Number of Participants in M&E Training per Partner and Monitoring Purpose

Partner # of

participants

Monitoring Purpose

GORESAM 27 Regional Plan against gender violence. San Martín. 2016-2021

GOREU 24 Strategic Objective #2 of the Regional Development and Concerted Plan

DRESM 14 Selected Indicators of the Institutional Strategic Plan (PEI) 2015 -2017

DREU 12 Education Management Indicators Annually Agreed with Ministry of Education

TOTAL 77

1 Dr. Violeta Pérez, Lilian Damian and Miriam Pérez are currently in charge of the M&E Diploma.


28

The training targets the establishment of the following competencies:

• Apply results oriented management in monitoring and evaluation,

• Draft the purpose and objectives of and M&E plan,

• Identify the components of the logical frameworks, horizontal and vertical logic,

• Identify information users and information management needs,

• Draft a matrix of indicators (base line, targets),

• Draft the technical description of indicators,

• Prepare the flow chart of required information

• Design the process for information management,

• Prepare a timeline, define roles and responsibilities.

3.2.3. Provide M&E Technical Assistance (TA) to USAID implementing partners

USAID implementing partners that have received M&E technical assistance from PGRD have changed

from year to year, mainly because being a partner is a temporal condition that ends with a project’s

completion. During the first year of Evaluations, USAID M&E team proposed twelve partners from the

National and Regional governments and from NGOs. This list was updated every year in the Evaluations

annual work plans, and to date, PGRD has worked with 33 partners.

On October 6, PGRD met with the USAID M&E team to analyze the progress made under this

component and to identify the partners to be prioritized over the next quarter and final year. The

partners selected were: GORESAM, GOREU, DRESM, DREU, Autoridad Nacional de Fauna Silvestre

(SERFOR), and the Ministry of Environment (MINAM).

Complementing the capacity building strategy, during Year 4 PGRD team provided TA in M&E to four

USAID IPs in addition to the training and TA provided through the M&E Postgraduate Diploma and the

Regional Workshops. Technical assistance activities focused on closing performance gaps2 related to

M&E best practices prioritized by each implementing partners.

Below we describe the results achieved during the year and the activities implemented with each

partner.

3.2.3.1 Regional Government of San Martin (GORESAM)

Within the Regional Government of San Martin, PGRD provided technical assistance to the Regional

Office for Social Development (GRDS). Products achieved in year 4 were:

• Draft M&E Plan of the Regional Plan Against Gender Violence.

• 27 people trained in M&E through regional workshops.

Figure 6 shows the sequence of TA activities and the products obtained as a result of each activity.

PGRD team facilitated the second performance assessment (April) and elaborated a preliminary

performance plan. Then, on August, the team presented the results to the Regional Manager of Social

Development and prioritized with his team the interventions to close gaps. They agreed on the

elaboration of the M&E Plan of the Regional Plan against Gender Violence (RPAGV), approved on

August 2016.

2 Performance gaps were identified jointly with the implementing partner, as part of the assessment sessions organize for this

purpose.


29

Based on this decision, PGRD prepared the technical assistance plan that was approved by the Manager

of Social Development3 on December 22. This TA plan established three products to achieve up to June

2017: i) the M&E Plan of the regional plan against gender violence, ii) an indicator’s dashboard, and iii) a

monitoring report.

Figure 6: Activities and Products of Year 4

2nd Performance Assessment (Apr 2016)

Regional Workshops (Jan – Mar 2017)

Diagnosis for Developing an Indicator’s Dashboard

27 Persons trainedPerformance Plan

Technical Plan Approved

Diagnosis Report

Draft M&E Plan of the

RPAGV

To initiate the elaboration of the M&E Plan, PGRD facilitated in December 2016, the technical meetings

to prioritize the indicators to be included in the M&E plan. The Deputy Governor, the new Manager of

Social Development and 26 representatives from 8 Local Governments attended to these meetings.

Participants prioritized the set of indicators and the staff who will participate in the regional training for

elaborating the M&E plan.

The regional training was delivered from January to march with the attendance of 27 persons as shown

in Table 8. The product of the workshops is the draft of the M&E Plan which will go under PGRD’s

revision and then submitted to GORESAM team for its approval and implementation.

Table 8: Number of Participants to the Regional Training Workshops

Date Participants Men Women

First attendance moment 18 6 12

Second attendance moment 20 10 10

Third attendance moment 17 7 10

Total 27 10 17

During the revision of the RPAGV, the GORESAM team identified the need to look more carefully

through the document and to update it. PGRD and the PERUME Network representatives helped the

team to obtain the technical assistance of the Ministry of Women for updating the plan up to mid-May.

The implementation of the RPAGV is now responsibility of the Regional Office of Social Inclusion

Before ending the regional training, PGRD met with regional authorities in March 16-17 to organize the

second phase of the TA plan. A result from these meetings was the diagnosis for the development of the

indicators’ dashboard, which was sent to GRDS authorities in March 24.

3 Letter N° 868-2016-GRSM/GRDS


30

During this year, the GRDS has had four general managers4. Fortunately, the activities implemented by

PGRD continued despite the high turnover.

3.2.3.2 GORESAM Regional Office of Education (DRESM)

During year 4, PGRD provided training and technical assistance to DRESM for elaborating two M&E

plans. PGRD’s regional advisor provided day-to-day technical assistance until August 2016. The following

is a list of the results and products of this TA:

• M&E Plan of “Teaching is leading” project elaborated and approved by USAID

o Flowcharts for collecting data.

o Data collection tools.

o Communication plan of the activity.

• Draft of the M&E Plan of the Regional Government’s Institutional Strategic Plan – Sector

education.

• 14 people trained in M&E through the regional workshops.

• Map of all DRESM’s projects and activities’ indicators.

Figure 7 presents the year 4 activities and deliverables resulting from PGRD support to DRESM.

Figure 7: Activities and Products of Year 4

During the first half of the year, PGRD provided training and day-to-day technical assistance for the

elaboration of the M&E Plan of the USAID’s activity “Teaching is leading”. The Regional Manager and

USAID approved the M&E Plan in June, 2016. The DRESM is implementing this M&E Plan and elaborating

quarterly reports.

In April 2016, PGRD facilitated baseline performance measurement and elaborated an improvement

plan. Based on these results, the DRESM prioritized the elaboration of the M&E Plan of the Regional

Education Sector Institutional Strategic Plan. DRESM also requested PGRD technical assistance in

4 Luis Rodriguez, Félix Segundo Rosales, Manuel Eduardo Vásquez Contreras (acting manager), Aurora Torrejón Riva (current

acting manager)


31

elaborating the information system “Teaching is leading”. PGRD elaborated a technical assistance plan

responding to these requests.

PGRD facilitated the identification of information users and their needs and prepared a workplan to

complete the web application of the M&E system, to be finished by October 2016. PGRD reviewed the

advances and provided feedback.

In October, PGRD presented the activities being performed with the DRESM and the results of the

baseline performance measurement to a new Manager5. He ratified the activities and requested the

elaboration of a map of all DRESM’s projects and activity indicators to serve as an input for the M&E

Plan. PGRD updated the M&E Plan and submitted it for approval. This map was presented in December

and the results showed the quantity of data that was being produced (mainly for reporting to other

institutions) and not being used by the DRESM authorities.

The training to DRESM was delivered from January to march with the attendance of 14 people, as

shown in Table 9. The training produced a draft of the M&E Plan, which PGRD will review for

adjustments by DRESM before its submission for approval and implementation.

Before ending the regional training, PGRD held meetings (March 16-17) to organize the second phase of

the TA plan. The result of these meetings was a diagnosis for the development of the ‘indicators’

dashboard’, which was sent to DRESM authorities (March 24). During this meeting, PGRD also reviewed

the advances of the implementation of the information system “Teaching is leading”. PGRD submitted a

target professional profile for the person who will design it.

Table 9: Participants attending to the Regional Workshops

During this year, the DRESM had two managers6. Fortunately, the activities implemented by PGRD

continued despite these changes.

3.2.3.3 Regional Government of Ucayali (GOREU)

In addition to receiving TA from the core PGRD team, PGRD has been providing day-to-day technical

assistance to GOREU from a PGRD regional advisor, Lenin Castillo, since January 2017.

Products achieved in year 4:

• Map of projects implemented by GOREU

• Draft M&E Plan of the PRDC’s education indicators

• 24 people trained in M&E through the regional workshops.

Technical assistance activities addressed to GOREU resumed in October 2016 with a technical meeting

between regional authorities and USAID. Mr. Manuel Gambini Rupay, Regional Governor, chaired this

meeting and PGRD presented the results of the GOREU performance M&E assessment. Participants

5 Former Director resigned to her position on September 2016. Appointed manager was Wilson Quevedo Ortiz 6 Pilar Saavedra (until September 2016) and Wilson Ricardo Quevedo Oritz (current director)





Total 14 11 3


32

agreed on a technical assistance package for the gradual implementation of a monitoring and evaluation

system through a participatory process that will be led by the deputy manager of Articulation,

Monitoring and Evaluation. Participants also agreed to elaborate the GOREU’s Project Map, which will

define how projects are expected to contribute to the achievement of the objectives of the Regional

Development Plan and the Strategic Plan of the GOREU. (Figure 8).

Figure 8: Activities and Products for Year 4

GOREU appointed a team to lead the process of implementing the M&E system7. This team and other

professionals from the different regional offices were summoned for a technical meeting (November 14)

to begin mapping the regional projects. In December, the Map was presented to authorities. These

authorities agreed to start the GOREU M&E System with the Strategic Objective 2 of the Regional

Development Plan addressed to improving access and the quality of education services. The M&E Plan

was approved on Feb. 2017.

PGRD delivered regional training from January to March with the attendance of 24 people, as shown in

Table 10. A product of the workshops is the draft of the M&E Plan. PGRD will review and return the

plan to GOREU for improvements before it is submitted for approval and implementation.

Table 10: Attendance to Regional Workshops





Total 24 12 12

Before ending the regional workshops, PGRD held meetings (March 23-24) to organize the second

phase of the TA plan. Result of these meetings was the diagnosis for the development of the “indicator

dashboard”, which was sent to GOREU authorities (March 29).

7 Regional Executive Resolution No. 0779-2016-GRU-GR dated November 10, 2016


33

3.2.3.4 GOREU Regional Office of Education (DREU)

During year 4, PGRD provided training and technical assistance to DREU to elaborate two M&E plans.

The PGRD regional advisor provided day-to-day technical assistance, and PGRD Lima’s team delivered

training. Below are results and products achieved (see Figure 9).

Products achieved in year 4:

• M&E Plan of “Liderando los aprendizajes de niñas y niños” (LANN) USAID’s activity:

o Flowcharts for collecting data.

o Data collection tools.

o Improvement in data bases

• Draft of the M&E Plan of Compromisos de Gestión Escolar 2017

• 12 people trained in M&E through the regional training.

• Map of all DREU’s projects and activities.

Figure 9: Products and Activities for Year 4

PGRD provided training and day-to-day technical assistance for the elaboration of the M&E Plan of the

USAID’s activity “Liderando los Aprendizajes de Niñas y Niños”. The M&E Plan was approved by the

Regional Manager and USAID on June 2016. The DREU team is implementing this M&E Plan and

producing M&E reports.

In October, PGRD presented the results of the first performance assessment. Based on these results,

the DRESM agreed on the following: i) to complement a map of DREU’s projects initiated by The United

Nations Children's Fund (UNICEF), ii) to prepare the M&E Plan of prioritized indicators and iii) to

provide support for the creation of an M&E office at the DREU. It was also agreed to facilitate a second

performance measurement, which was completed in October. According to the results of this second

assessment and the identification of gaps, the DREU team maintained the decision to elaborate the M&E

Plan of the Compromisos de Gestión Escolar. PGRD elaborated the technical assistance plan for this

purpose, approved on March 15.


34

The DREU appointed a team to lead the implementation of the TA Plan8.

PGRD delivered regional training from January to March with the attendance of 12 people, as shown in

Table 11. Product of the workshops is the draft of the M&E Plan. PGRD will review and resubmit the

plan to DREU for improvements before its approval and implementation.

Table 11: Attendance to Regional Workshops





Total 12 5 7

Before ending the regional training, PGRD held meetings in March 23-24 to organize the second phase

of the TA plan. A product of these meetings was the diagnosis for the development of the “indicator

dashboard”, which was sent to DREU authorities (March 29).

The main result of the technical assistance provided to DREU is the creation of an M&E area in the

DREU structure (ROF). This area will be hosted in the Office for Planning, Budget and Modernization

under the Direction of Management for Educational Quality named “M&E Unit and Educative

Investigation”.

It is important to mention that during this year, the DREU had three managers9. Fortunately, the

activities implemented by PGRD continued despite this leadership turnover.

3.2.3.5 Ministry of Environment (MINAM)

PGRD ended TA activities with MINAM before the appointment of the new national government.

However, in October 2016, PGRD and MINAM began coordinating on a new M&E training as part of

the annual SYNERGY event. In November, MINAM informed PGRD of the cancelation of the SYNERGY

event. No further activities have been implemented during the year. PGRD is looking forward to

reinitiating activities with new MINAM authorities.

3.2.3.6 Servicio Nacional Forestal y de Fauna Silvestre (SERFOR)

On February 22, Fernando Chávez, the USAID M&E POC, introduced PGRD to the SERFOR team10.

PGRD presented the methodology and experience in M&E capacity building with USAID partners.

SERFOR will send a request to USAID asking for PGRD technical assistance to initiate an M&E

improvement process. To date, no formal request has been received.

3.2.3.7 CEPCO, SER and CEDRO

The PGRD team held technical meetings with each partner in late March 2017 to carry out institutional

assessments of M&E performance. These new results permitted a comparison of results from baseline

8 Regional Resolution No. 000123-2017-DREU dated January 30, 2016 9 Víctor García, Manuel Vásquez Valera and Ángela Villacorta (current director and former director of pedagogical

management) 10 Walter Nalvarte (Director de Gestión Sostenible del Patrimonio Forestal y de Fauna Silvestre), Lucy Rocío Malleux

(Directora de Información y Ordenamiento Forestal y de Fauna Silvestre) and Álvaro Anicama (Director de Control de la

Gestión del Patrimonio Forestal y de Fauna Silvestre)


35

measurements from three years prior. PGRD also provided feedback on the results of the M&E

Diploma. All three partners have participated in the M&E Diploma and have received technical assistance

from PGRD.

Table 12 presents the results of the performance measurement, showing an important improvement:

SER moved from a minimum level to an intermediate, and CEDRO and CEPCO moved from

intermediate to adequate. These follow-up measurements mark the graduation of these organizations

from PGRD technical assistance.

Table 12: Institutional Performance Level Before and After Training and TA

Baseline 2nd assessment after the TA and M&E

Diploma

CEDRO 69.26 Intermediate November 2013 82.40 Adequate March 2017

SER 37.00 Minimum November 2013 70.80 Intermediate March 2017

CEPCO 62.68 Intermediate December 2013 80.00 Adequate March 2017

Level of performance: Minimum (Up to 60%); Intermediate (From 60% to 80%); Adequate (From 80% to 100%).

3.2.3.8 Other USAID implementing partners

Starting Year 4, on April 5, PGRD facilitated an M&E workshop for nine implementing mechanisms

working with USAID Environmental Office: Ministry of Environment (MINAM), Asociación para la

Investigación y Desarrollo Integral (AIDER), Centro Agronómico Tropical de Investigación y Enseñanza (CATIE),

Chemonics, Lutheran World Relief, The Mountain Institute (TMI), The Nature Conservancy (TNC),

Peru Forest Sector Initiative (PFSI), Conservación Internacional and USDA Forest. Participants shared their

experiences and critical nodes when collecting common indicators. They also analyzed and shared their

experiences applying the survey to measure the self-efficacy gender indicator to workshop participants.

One result of the workshop was adapting the survey to local common language. Participants also agreed

to validate the updated survey. A second result was a proposal for USAID CORs of a summary matrix

to be incorporated in final project reports.

3.3. SUPPORT TO USAID/PERU MONITORING & EVALUATION TEAM

M&E Point of Contacts (POC) Workshop

In April 2016, the team carried out a workshop addressed to the USAID M&E team. PGRD presented

the critical nodes identified in the projects’ design and in the M&E plans. Discussions addressed the

analysis of selected indicators using the SMART criteria, and the selection of indicators to measure

gender and capacity building to be included in the M&E Plan of the new CDCS.

Main conclusions included a summary of lessons learned to improve monitoring and a proposal of

objectives and contents for the workshop to be addressed to COR/AOR.

Workshop for COR/AOR

PGRD supported the M&E team in organizing a workshop addressed to COR/AORs. Sixteen

participants from DO2 and DO3 attended the workshop on July 2016. In a post-workshop online survey

13 participants provided feedback on the workshop. Results showed that: nine participants were

interested in participating in this kind of workshops; participants found it useful to interact with

colleagues from the others DOs; and, indicators were the most useful theme of the workshop.

Participants were also interested on attending this type of workshops for planning the new CDCS and

selecting good indicators.


36

Support to DO2

As requested by USAID, PGRD reviewed and provided comments, several times, to the M&E Plan of the

training activity implemented by UPCH. M&E plan was approved on February 2017.

Support to DO3

PGRD was asked to provide support to validate the survey to measure self-efficacy in women

participating in DO3 activities. Preliminary terms of reference were sent to M&E POC for comments on

July 4th. On September 6, PGRD was told to reschedule the consultancy until the Mission gender

analysis was completed.

Reviewing new ADS 201

In July 2016 a new version of ADS 201 was released, replacing January 2015 version and ADS 203

February 2012 version. PGRD reviewed and presented the changes related to monitoring and evaluation

to COR and AOR.

Workshop for USAID M&E Team “Use and communication of evidences”

During the last quarter of Y4, the team has been organizing the two-day workshop addressed to the

M&E team and preparing the training materials.

The workshop will be delivered on April 6-7. Topics to be addressed are: use of evidence in program

design, format and contents of final reports, dimensions and indicators in capacity building, and

communication and incidence.

3.4. OTHER ACTIVITIES

Internship

In October 2016, PGRD agreed with USAID to provide a three-month internship for Analy Rodriguez,

an intern in USAID’s Program for Afro-Peruvian and Indigenous Communities (APIC). She joined the

PGRD team on October 3 and has been trained in performance measurement based on M&E best

practices. She also participated in the design and validation of tools for the diploma graduates follow-up

study. The internship finished on December 23, 2016.


37

4. LOCAL CAPACITIES FOR EVALUATION

STRENGHTENED The purpose of Component 3 is to develop and strengthen the technical capacities of two local

evaluation institutions so that they are able to design and conduct effective performance and impact

evaluations according to international standards.

To this end, the project elaborated and implemented a plan to strengthen capacities —as defined by

competencies, skills and organizational structure—, based on the Human and Institutional Capacity

Development (HICD) approach, detailed in Annex 2. The following section describes the component

progress during Year 4.

4.1. EVALUATION TRAINING AND TECHNICAL ASSISTANCE FOR

EVALUATION NETWORKS

The PGRD team continued strengthening the capacities of the two evaluation networks identified in

Year 1: EvalPerú and PERUME. Both networks are affiliated to Latin American regional evaluation

networks.

4.1.1. Postgraduate Modules in Social Key competences, Evaluation and Evidence

Management

A total of 21 network affiliates attended the M&E postgraduate program organized with UPCH. Of

these, 13 participants attended threee modules subsidized by USAID and then opted to pay themselves

to attend a fourth module in order to obtain the M&E Diploma. The other participants that did not

attend all four modules obtained a certificate for each satisfactorily completed module (see Table 18).

Table 13: Number of Participants from Evaluation Networks

Evaluation Network Participants with

Diploma

Participants with

Certificate

Total number of

participants

PERUME 11 7 18

EvalPerú 2 1 3

Total 13 8 21

Two participants from the EvalPeru Network applied and obtained a certification of competences under

the validation option under the M&E program. They paid for the validation exam and after passing,

obtained the Diploma.

PGRD measured the level of competencies in M&E of more than half of the participants before the

program started and after it finished. Used as a sample, the baseline data was used to identify which

areas needed strengthening among the networks’ members. The final data was used to measure

progress made by the individual participants.

Figure 10 shows that network participants needed to develop their technical competences mainly in

evaluation and in evidence management, while their competences in monitoring only required marginal

strengthening. After participating in the M&E program, participants improved all of their technical

competencies and those related with monitoring and evaluation have reached an advanced level (a score

of 3+).


38

Figure 10: Progress in Technical Competences of Networks' Participants

Source: Self-diagnostic online survey. Period: Group I: August 2015 and May 2016, Group II: December 2015 and

October 2016. Base: Group I: 7, Group II: 5

Measurement of key competencies —soft skills such as communication, leadership, negotiation— rated

high in the first measurement and slightly decreased in the final measurement (see Figure 11). After

discussing these results with the participants and the training staff, we reached the conclusion that

participants were more critical of their abilities in the second self-assessment. This is a common result

under self-assessments, as participants gain a better understanding of the complexity of acquiring soft

competencies during the learning process.

Source: Self-diagnostic online survey. Period: Group I: August 2015 and May 2016, Group II: December 2015

and October 2016. Base: Group I: 7, Group II: 5

Figure 11: Progress in Key Competences of Networks’ Participants


39

4.1.2. Institutional Strategic Plan of PERUME Network

The Institutional Strategic Plan of the network was completed with PGRD assistance. The network

approved the document through an official communication.

4.2. DISSEMINATION ACTIVITIES TO STRENGTHEN CAPACITIES

PGRD participated in Session 3 of the V Encounter of PERUME Network (October 19):

Institucionalización del uso y generación de evidencias en el sector público.

PGRD and UPCH facilitated a workshop related to tools for evidence management during the “Week of

Evidence 2016 - Peru (October 19) Participants: 23. For this workshop, a Rapid Guide Herramientas para

la gestión de evidencias científicas was prepared and validated. The evaluation of the event was satisfactory.

The report and the workshop evaluation can be found at https://www.eventbrite.es/e/entradas-v-

encuentro-redperume-generacion-y-uso-de-evidencias-para-las-politicas-publicas-27570868209.

https://www.eventbrite.es/e/entradas-v-encuentro-redperume-generacion-y-uso-de-evidencias-para-las-politicas-publicas-27570868209

https://www.eventbrite.es/e/entradas-v-encuentro-redperume-generacion-y-uso-de-evidencias-para-las-politicas-publicas-27570868209


40

5. LESSONS LEARNED 5.1. LESSONS LEARNED FROM Y4 EVALUATIONS

CLIN 1 has a consolidated approach to managing evaluations that stems from the Performance

Improvement Plan of May 2015, following systematized evaluation management processes and timelines

for design and implementation phases. PGRD has established best practices that have allowed CLIN 1 to

ensure quality and timeliness of its evaluations.

As no two evaluations or studies are the same, CLIN 1 derives lessons learned from every evaluation or

study undertaking as well as from its findings. These lessons are recorded in the lessons learned matrix,

which is recorded quarterly and discussed among CLIN 1 and project staff.

5.1.1. Non-managerial Lessons Learned

1. Re-assess value added of the Plan of Recommendations: All seven evaluations since May 2015 – Conflict

Management and Mitigation, Peru Cacao Alliance phase 1, Healthy Communities and Municipalities II,

Pro Integridad, Peru Bosques, Amazon Malaria Initiative, and Education Quality in AD

communities/CEPCO - have been final evaluations. It should not be a wonder that the Plans of

Recommendations have had limited use and effect. In Y5, PGRD will discuss the relevance and/or

potential use and audience of the Plan of Recommendations at the beginning of a new evaluation

and, if necessary, validate any decision throughout its implementation.

When discussing contents with USAID PO and COR, and sometimes with IP representatives, it was

clear that identified actions need to be carefully analyzed in terms of context, timing and others

since they potentially result in initiating new engagements with people and/or activities that may

have not been planned and/or may cause unexpected outcomes.

2. Brief learning reports: PGRD began planning, internally and with USAID, and systematizing relevant

experiences in preparation of drafting at the beginning of Y4. However, more important than a

systematized and agreed upon timeline is the definition of a clear purpose for each learning report,

and it is not necessarily easy to reach consensus. The above is as relevant for evaluation learning

briefs and evaluation practice briefs, as for the fact sheets.

Each of the brief reports has had various revisions internally prior to USAID submission. This can

also demand significant time and effort since quality, style and extension of contents are essential to

produce a stimulating and friendly document. Once this is reached, a similar process of validation is

undertaken with USAID, including the PO and COR, and sometimes, other staff. The above requires

adequate planning and coordination of time and resources.

This process has demanded more time and effort from PGRD than expected. Therefore, it is

suggested that this rich experience be shared with USAID, IPs and others to maximize the learning

benefits. Early in Y5 these learning reports will be disseminated and discussion sessions for LoW 2

and 3 will take place.

3. Activity design: In an effort to support USAID in improving its Activity design, PGRD identified six

fundamental issues that were presented to USAID and that require greater attention: (i) clearer and

more precise sequence of events of Activity, (ii) well thought out and explicit consideration of

assumptions, (iii) more precise and relevant context analysis and better knowledge of the main

characteristics and demands of target population, (iv) a clearer and realistic Activity objective, (v)

better selection and definition of indicators, and (vi) clearer and more precise calculation of targets

consistent with Activity objective, timeframe and budget. USAID expressed commitment to follow

up and take action and PGRD is standing by to provide further support.


41

5.1.2. Management Lessons Learned from Y4

1. Engage implementing partners (IPs) throughout the evaluation: Drawing from the management of the last

two years’ studies, PGRD recognizes that active engagement of IP during the evaluation process

makes the study richer, particularly in terms of depth. This engagement also maximizes the

likelihood of adopting recommendations while serving as a learning experience to the organization.

Going into Y5, PGRD remains committed to increasing IP engagement to maximize ownership of

the study, both from USAID and the IP. At the same time, this will result in cumulative trust built

with the IP, facilitating the required revision of the draft report (ADS 201, 125). A more systematic

approach started with the Peru Bosques and Education Quality in AD communities/ CEPCO with

excellent results and will continue with the final evaluation of Prodecentralization and others.

2. Evaluation-specific timeline: PGRD has been preparing a more predictable and precise timeline for

each evaluation. Given several factors that affected the strict compliance of the programmed dates

as scheduled in the EET, PGRD began utilizing a specific timeline for each evaluation and/or study

prepared after signing of sub contracts (M4) with dates and evaluation milestones that highlight

required and expected availability from USAID for their timely feedback.

3. Later submission of preliminary findings: In Y4, the Project has requested submission of preliminary

findings and recommendations two weeks after fieldwork completion. This has proven to be a good

decision. First, the analytical and reporting quality submitted by the research team to PGRD has

improved markedly. Second, it helped PGRD to provide more adequate lead time to USAID. Third,

USAID has appreciated the better quality of the presentations.

4. Peer reviewers: Starting in Y4, the Project has contracted peer reviewers more systematically to

review data collection instruments more closely, help in their refinement, provide partial fieldwork,

and critically assess draft reports. Involvement of peer reviewers in fieldwork yielded an immediate

result: it enabled PGRD to provide immediate and specific feedback to subcontractors to improve

quality and maximize the collection and scope of quality data. Critical insights provided in the

revision of draft report have led to improved overall quality.

5.2. LESSONS LEARNED IMPLEMENTING THE CAPACITY BUILDING

MODEL

1. Engaging and involving decision makers among all stakeholders, including USAID implementing partner

senior management and regional and local authorities. Capacity development processes were optimized

when decision-makers (senior management of partner institutions, regional authorities and USAID

project representatives) participated in them. Their participation allowed the developing practice of

M&E to be strengthened and in some cases, progressively accepted and incorporated. Their

participation also generated institutional awareness regarding the need and benefits of consider M&E

functions in the organizational structure. This progress will contribute to the consolidation of the

processes planned for this year.

2. Identifying and working with a formal lead counterpart team responsible for keeping the counterpart

institution mobilized. PGRD made an impact with top management and achieved the formation of the

"Team responsible for the implementation of the technical assistance plan," which was formally

recognized with explicit responsibilities. This overcame some challenges in convoking participants

and overcoming non-compliance with the planned activities, since this official designation made

participants responsible for the follow-up and fulfillment of the technical assistance plan. It also


42

overcome difficulties caused by high staff turnover, since the official team required that an outgoing

person be immediately replaced by another worker.

3. Implementing technical assistance activities on the basis of a TA Plan that describes the responsibilities of all

parties to achieve specific results that are formally approved by the participating institution organization.

The preparation and joint implementation of the M&E technical assistance plan with the designated

team allowed for the clear delineation of the responsibilities of the institution and of PGRD. Once

assumed by both institutions, this delineation facilitated formal approval by managers, although in

some cases this took longer than optimal.

4. An integrated and complementary set of capacity strengthening activities that promotes the progressive

growth of institutional and individual M&E capacity. The strategy of integrating training activities

(diploma and workshops), technical assistance and exchange of experiences has favored the

development of capacities of the people and institutions. This has been reflected in the institutional

progress vis-a-vis good M&E practices among partners. Complementing this suite of approaches,

PGRD will incorporate a virtual course to provide mass access on priority topics, this year.

5. Incorporation of M&E specialists from USAID and the public sector into training activities. The

development of M&E graduate activities and regional workshops with the involvement of M&E

specialists from USAID and the public sector allowed participants to share experiences from

international cooperation and the Peruvian State. This was motivating and fomented collaborative

learning. This experience has been continued by UPCH and will continue to invite the participation

of specialists from both groups. PGRD will continue this strategy in the regional workshops.

6. Inclusion of graduates of the Postgraduate Program in training and technical assistance activities to

consolidate their M&E skills. The incorporation of graduates of the Diplomado in the regional

workshops has been a catalyst for the strengthening of their competencies. In this context, the

graduates have been recognized by their colleagues as regional facilitators in monitoring. Also, their

participation in the measurement of good practices in institutional M&E and formulation of the plan

of improvement was highly valued. The opportunity to share their learning and contribute to their

institutions has been a source of satisfaction among these graduates. This year, other graduates will

be recruited for training and technical assistance activities.

7. Establishment of mechanisms for feedback to partner institutions related to the progress of training and TA

activities. PGRD debriefing of partner leadership on the results of the M&E graduate was productive.

The managers corroborated the improved performance of their participants and showed their

appreciation. It is best if this feedback is provided during the process and not at the end. This year,

PGRD will have meetings with the IP managers to report on progress and receive suggestions for

improvement.

Agreement with the partner institutions for the creation of an M&E governing body and the allocation of an M&E

budget. The lack of M&E functions as part of partners’ organizational structure prohibits the

advancement and the positioning of these functions. To address this, PGRD lobbied at the management

level for the structural inclusion of M&E and the programming of funds to carry out this function. This

has generated reflection that may enable structural change, even among those that have not taken this

step. We will continue working on this in Year 5.


43

6. PROJECT MANAGEMENT AND

OPERATIONS 6.1. DELIVERABLES OF YEAR 4

N° Report Name Type of

Study

Date Link Language

15 Evaluación de la Transferencia,

Expansión y Sostenibilidad del Modelo

Municipios y Comunidades Saludables en

Perú

Performance

Evaluation

2016 http://pdf.usaid.gov/

pdf_docs/pa00m67

6.pdf

Spanish

16 Evaluación Final de Desempeño del

Proyecto Promoción de la Justicia y la

Integridad en la Administración Pública

Performance

Evaluation


pdf_docs/pa00mfqr.

pdf

Spanish

17 Estudio de la Situación Actual del Sector

Educación en el Perú

Sector

Assessment


pdf_docs/pa00mhcs

.pdf

Spanish

18 La economía del VRAEM. Diagnóstico y

opciones de política

Sector

Assessment

2016 http://www.cies.org

.pe/sites/default/files

/files/otrasinvestigac

iones/archivos/01-

vraem_final.pdf

Spanish

19 Toma de Decisiones en Hogares y

Cultivo de Coca en Shanantia

Ethnography 2017 http://pdf.usaid.gov/

pdf_docs/PA00MP

D3.pdf

Spanish

Toma de Decisiones del Antiguo

Productor de Coca en una Comunidad

del Monzón

Ethnography http://pdf.usaid.gov/

pdf_docs/PA00MP

CZ.pdf

Spanish

20 DEVIDA Institutional Capacity Sector

Assessment

2017 Spanish

21 Evaluación Final de Desempeño del

Proyecto Peru Bosques

Performance

Evaluation


pdf_docs/PA00MH

CR.pdf

Spanish

22 Evaluación Final del Proyecto Apoyo a la

Expansión de la Metodología de Escuelas

Activas en Peru

Performance

Evaluation


pdf_docs/PA00MN

MR.pdf

Spanish

23 Evaluación Final de Desempeño de la

Iniciativa Amazónica contra la Malaria

Performance

Evaluation


pdf_docs/PA00MN

MM.pdf

Spanish

24 Análisis de Género: Perú 2016 Sector

Assessment


pdf_docs/PA00MN

MG.pdf

Spanish


44

6.2. PRODUCTS PREPARED DURING YEAR 4

N° Documents produced Used for

Component 2

1 Improvement plans for partners Prioritize actions in order to close gaps.

2 Technical assistance plans for partners:

GOREU, GORESAM, DREU, DRESM

Establish the products to be elaborated through technical

assistance and training by each partner.

3 Diagnosis report for the development

of the indicators’ dashboard for

GOREU, DREU, GORESAM, DRESM

Identify the capacities (human and technological) of each

partner to design and implement an indicators’ dashboard.

4 Diagnosis report for the

implementation of PEELs information

system

Identify the capacities (human and technological) of DRESM

to design and implement an information system.

5 Map of DRESM’s indicators of projects

and activities

Provide information to DRESM of the data produced by the

institution and inputs to analyze the use of this information.

Provide information on how DRESM interventions are

aligned to the PEI.

6 Map of GOREU’s projects

Provide information to GOREU on how its interventions

are aligned to the regional and national policies.

7 Regional workshop syllabus and

training materials

Training material for elaborating M&E Plans.

Component 3

1 Rapid Tool for Management of

Scientific Evidence for Development

Plans, Projects and Programs

Evidence week I

Community in general


45

7. PLANNED PERFORMANCE RESULTS FOR

YEAR 5, 1ST QUARTER In this section, we present the activities programmed for the first quarter of the fifth year organized by

the three project components. These activities are widely explained in the fourth year work plan.

7.1. EVALUATION STUDIES ARE USED TO IMPROVE PROGRAMMING

Table 20 presents the evaluations and assessments ongoing at the beginning of Y5. Progress to be

completed in Q17 is shown in light blue. PGRD will approach USAID early next quarter to define the

implementation timeline for Y5 evaluations, presented as TBD.

Table 14: Implementation Timeline for Y5 Evaluation Studies and Assessments

# Study Type Status SOW Inception

Report

Initial

Findings

Draft

Report

Final

Report

19 AD Ethnographic Study AS Ongoing 3-Sep-15 19-Feb-16 29-Apr-16 12-Jul-16 Pending

submission11

25 Capacity Building Assessment AS Ongoing 1-Sep-16 25-Nov-16 21-Feb-17 08-Mar-17 28-Apr-17

26 Prodecentralization FE Ongoing 23-Mar-17 18-May-17 10-Aug-17 4-Sep-17 27-Sep-17

27 Rural Communities

Engagement Strategies (DO3)

AS Planned May-17 Q17 Q18 Q19 Q19

26 TBD AS

AS= Assessment; PE= Performance Evaluation

7.1.1 Performance evaluations

# 26. ProDecentralization

PGRD will proceed with selection of proposals (M3) and signing of contract (M4) by end of April 2017.

Expected start date of implementation is April 28. The level of effort of this final evaluation is planned in

18 weeks, as shown in Table 8 above, submission of final report (M10) is expected by the second half of

October (Q19).

7.1.2 Sector and crosscutting assessments

# 19. AD Ethnographic Study 1

Two of the three final reports – Shanantia and Agua Blanca – have been approved and uploaded in DEC

in Y4. The third – Rio Espino – will be submitted in May 2017 and will, immediately after, uploaded in

DEC.

# 25 Assessment of Capacity Building Activities

PGRD will review the final report, provide feedback to the consultant team for improving it. If necessary

a meeting will be arranged for talking about recommendations. It is expected to have the final report by

the end of April and submit it to USAID on May 2017.

11 Two of three already submitted, Rio Espino is pending.


46

# 27 Rural Communities Engagement Strategies – DO3

A meeting to discuss the purpose, context and research questions has been scheduled for April 2017

(M1).

PGRD will submit draft SOW during May 2017 and expects to obtain approval same month (M2) or

early June 2017. Signing of contract (M4) will also take place in Q17. Launching of the study (M5) and

approval of inception report (M6) are expected to take place in Q18. Specific dates will be provided in

early Q17 upon approval of SOW. This study is expected to be completed by Q19.

7.1.3 Post evaluation Activities

Brief Learning Reports

In Y4, six evaluation fact sheets, one evaluation learning brief and one evaluation practice brief have been

submitted to USAID in final and are pending approval.

In Q17, and thereafter every quarter, an additional five evaluation fact sheets will be submitted to

USAID. Please see table 21 below for the planned fact sheets.

Table 15: Evaluation Fact Sheets

Evaluation Study/ Assessment Year DO Final draft

PGRD

Submission

to USAID

Final evaluation of AMI 3 2 May May

Education Brief Assessment 3 2 May May

Final evaluation CEPCO 4 1 May May

MINAM Technical Assistance Program 2 3 May May

Final evaluation CMM 2 2 May May

Simultaneously, PGRD will deliver two additional learning briefs in Q17, and two every quarter

thereafter until Q19. See table 22 below for the proposed learning subjects.

Table 16: Evaluation Learning Briefs

Title Submission

to USAID

Discussion

w/USAID

Final to

USAID

2. Strengthened government institutions: Have we? May, 17 June, 17 June, 17

3. Knowledge transferred: Will it be sustained? May, 17 June, 17 June, 17


47

Finally, PGRD will submit up to two evaluation practice briefs before the end of the project.

Table 17: Evaluation Practice Briefs

Title Submission

to USAID

Discussion

w/USAID

Final to

USAID

2. What does it take for good project design? Q18 Q18 Q18

3. TBD Q19 Q19 Q19

7.1.4 Overall component activities

For this upcoming quarter, the following activities take priority:

• As discussed earlier in section 2, pursuing a plan of recommendations for every evaluation will

be discussed with USAID’s PO and COR at the beginning of every evaluation study/assessment

undertaken. Since the implementation of the recommendations is reported as evidence of the

use of evaluations, this will have to be adjusted accordingly and officially by USAID.

• Discuss dissemination plans and communication deliverables when undertaking an evaluation or

study at the time of the lunching workshop with USAID and the evaluation teams.

• Related to the use of the studies, PGRD will subcontract to conduct an external qualitative

study to examine the use of the evaluations and assessments among USAID, IP and other

stakeholders. Findings from this study will be reported in the final annual report.

• Validate with USAID the list of future fact sheets and learning reports.


48

7.2. IMPLEMENTING PARTNERS ARE ABLE TO MANAGE PER RESULTS

7.2.1 Facilitate the preparation of USAID/Peru M&E Plans

▪ Construction or refinement of progress indicators;

▪ Calculation of baseline measurement for progress indicators, including the identification of

secondary data sources;

▪ Support USAID in ensuring that all plans are compliant with the most recent version of ADS 201

and other relevant regulations, such as the ADS 205, Integrating Gender Equality and Female

Empowerment in USAID’s Program Cycle.

▪ Provide support as requested by USAID.

7.2.2 Monitoring and evaluation training and technical assistance for partners

Activities Period Expected products

Follow up M&E capacity assessment for implementing

partners, Diploma participants and Diploma graduates

May - July

TA for Six Implementing Partners GORESAM April - October - M&E Plan

- Indicators’

dashboard

DRESM

GOREU

DREU

MINAM May – October TBD

SERFOR

M&E Post-Graduate Diploma April - December

7.2.3 Support to USAID/Peru Evaluation Team

Activities Period

Annual Workshop with the USAID Mission M&E Team April


49

7.3. LOCAL CAPACITIES FOR EVALUATION STRENGTHENED

Activities Period

Evaluation training and technical assistance for evaluation networks

Follow up M&E capacity assessment for professional evaluation networks,

Diploma participants and Diploma graduates

May – July

TA for the PERUME network May – October

Wide dissemination of M&E Diploma Materials June – August

Design and implementation of an Online M&E Course May – August


50

ANNEXES ANNEX 1. TEN EVALUATION MOMENTS

1st moment –

Identification of Questions

Evaluation questions identified in consensus with client, responding

to client needs. Client participation is critical to ensure that the

client identifies and understands the genesis of the questions that

it wants to answer since they will drive the study design and

results.

2nd moment – SOW Timely SOW approval is critical to start the implementation

process. SOW provides USAID with a suggested implementation

design and timeline.

3rd and 4th moments –

Selection of Proposals and

Signing of Contract

As of Y3, call for proposals are sent via direct invitation to a

greater pool of established Peruvian firms and advertised to

qualified individual consultants through online evaluation networks.

Additionally, Project Evaluations also screens and recruits

international, highly qualified experts where local talent is scarce

and/or unavailable in USAID specific mechanisms, eg. global

development alliance.

5th moment – Launching

of Evaluation

Implemented in Y3, orientation workshops ensure alignment of

expectations among USAID, the evaluation team(s) and Project

Evaluations. Herein, the evaluation team hears firsthand client

expectations and context regarding the project, alongside a more

strategic and programmatic perspective. The presence of project

and Program Office CORs provides an institutional memory

irreplaceable by Project Evaluations.

6th moment – Inception

Report

This report serves to ensure that the evaluating team understands

the nuances of the evaluation questions and objectives, establishes

a detailed plan and is prepared to implement. The detailed context

and methodology allow the evaluating team and the Project to

align expectations on how and when fieldwork, analysis, and other

stages will be executed. Identifying the relationship between

evaluation questions, methodology and data sources is aimed at

ensuring the evaluation questions will be answered through

evidence-based findings.

7th moment –Fieldwork

Report and Findings

Presentation

Upon fieldwork completion, findings are presented to USAID in

order to receive comments and suggestions to address in the final

report. This presentation takes place approximately two weeks

after fieldwork is completed, allowing the team sufficient time to

process the data.

8th moment– Draft report

sent

Project Evaluations submits draft report of evaluation/assessment

study, which incorporates USAID feedback received during the

presentation of findings taking place upon fieldwork completion.

9th moment– Plan of

Recommendations

USAID, the evaluations team, and Project Evaluations discuss and

agree on a set of recommendations that will be evaluated and


51

hopefully implemented by USAID, implementing partners, and

other stakeholders. Recommendations relate to the design and or

implementation phases, either by USAID and/or by implementing

agencies.

10th moment – Final

report approved and

Uploaded in DEC

Refers to submission of final report and approval by USAID. This

final report incorporates all comments from USAID and Project

Evaluations. This final report must satisfy the Project´s quality

standards regarding the methodological implementation of the

evaluation to provide evidence-based findings. Lastly, the approved

final report is uploaded to the DEC.


52

ANNEX 2. APPROACH FOR CAPACITY BUILDING IN M&E

The Evaluations approach to building M&E capacities is holistic, beginning with activities that target

performance improvements followed by the implementation of a consensus-based action plan that

generates concrete results for the institution.

The model consists of two components: the first aimed at improving the performance of the institution

based on best practices for monitoring and evaluation, and the second to improve the competences of

the professionals that perform monitoring and evaluation functions. The model is presented in Figure 12.

Figure 12: Approach for capacity building in M&E

For the first component, the team used the Performance Improvement Methodology (MMD), which

standardizes and streamlines organizational processes through the adoption of best practices and

recognizing performance milestones. The process for improving performance consists of four stages: (i)

current measurement and identification of performance gaps, (ii) analysis of gaps and identification of

root causes, (iii) development of a performance improvement plan, and (iv) implementation of the

improvement plan and feedback actions involving recognition and reinforcement of progress.

The second component aims at strengthening and improving individuals’ skills, and is comprised of three

practicum-focused activities: (i) implementation of a postgraduate Diploma in M&E, (ii) regional training

workshops in M&E, and (iii) virtual M&E courses.

The activities of the two components are coordinated and combine to improve institutional

performance by closing M&E performance gaps and improving the use of evidence to inform program

management.


53

ANNEX 3. INDICATOR PROGRESS

In this annex we will present tables, graphics and narrative summarizing key progress indicators under each intermediate result and its activities.

The updated results tracking table is presented in table 18. The RTT in excel is attached as Annex 4.

Table 18. Evaluations activity results tracking table

N° Indicator Level Frequency

of report

Years

for

report

Baseline value Year 1 Year 2 Year 3 Year 4 Year 5

Year Value Goal Result Goal Result Goal Result Goal Qtr13 Qtr14 Qtr15 Qtr16 Result Goal

IR: 1. EVALUATION STUDIES ARE USED TO IMPROVE PROGRAMMING

1.1 % of evaluations/

studies used to

improve programs or

projects

Outcome Semi-annual Y3, Y4,

Y5

NA NA 100% 0% 100% 0 100% 63% 100% NA NA NA NA 67% 100%

1.1.1 # of recommendation

plans agreed for

implementation

Output Semi-annual Y3, Y4,

Y5

NA NA NA 0 NA 2 NA 2 TBD 1 0 0 4 5 TBD

1.2.1 # of evaluation/studies

reports approved

Output Quarterly Y1 to Y5 NA NA 8 2 9 4 3 8 4 1 2 1 6 10 4

1.2.2 # of evaluation/studies

final reports sent to

USAID for approval

Output Quarterly Y3, Y4,

Y5

NA NA NA 2 NA 11 20 17 24 1 2 3 2 25 28

IR: 2. IMPLEMENTING PARTNERS ARE ABLE TO MANAGE PER RESULTS

2.1 # of IP that improve

their institutional

performance in M&E

based on best

practices

Outcome Annual Y4, Y5 NA NA NA NA NA NA NA NA 9 NA NA NA NA 4 9

2.2. # of IP with PMP and

reporting results

Outcome Annual Y4, Y5 2013 4 NA NA NA NA NA NA 9 4 9

2.1.1 # of new or modified

M&E Plans approved


2.1.2 # of USAID's

monitoring processes

and/or activities

strengthened

Output Quarterly Y1 to Y5 NA NA NA 3 NA 3 NA 4 NA 3 1 1 0 3 NA

2.2.1 # of participants of the

M&E Postgraduate

Program who obtain

the academic

certification

Output Semi-

Annual

Y4 2015 13 have

taken any

postgrad

monitoring

course; ten

an

NA NA NA NA NA NA 52 NA NA NA NA 43 NA


54


of report

Years

for

report



evaluation

course.

2.2.2 # of IP with M&E tools

according to quality

standards

Output Semi-

Annual

Y4, Y5 2013 0 NA NA NA NA NA NA 20 NA NA NA NA 15 20


M&E Postgraduate

Program who improve

their competencies in

ME

Output Annual Y4 NA NA NA NA NA NA NA NA 52 NA NA NA NA NA NA

Competencias Clave NA NA NA NA NA NA NA NA NA NA NA NA NA 35 NA

Monitoreo NA NA NA NA NA NA NA NA NA NA NA NA NA 41 NA

Evaluación NA NA NA NA NA NA NA NA NA NA NA NA NA 42 NA

Gestión de evidencias NA NA NA NA NA NA NA NA NA NA NA NA NA 41 NA

2.2.4 # of IP participating in

M&E workshops

Output Quarterly Y1, to

Y5

NA NA 5 14 10 22 15 31 15 24 12 4 6 33 TBD

2.2.5 # of individuals trained

in M&E workshops


Number of men Output NA NA NA 76 NA 150 NA 254 NA 48 19 75 41 396 0

Number of women Output NA NA NA 58 NA 168 NA 249 NA 58 13 63 45 392 0

2.2.6 # of M&E workshops

delivered

Process Quarterly Y1 to Y5 NA NA NA 13 NA 46 NA 76 NA 11 5 8 9 109 NA

2.2.7 # of capacity building

hours in M&E

delivered

Input Quarterly Y1 to Y5 NA NA 400 258 1040 1683 1760 4211 1920 1379 1196 559 764 8108 0

IR: 3. LOCAL PARTNERS CONDUCT HIGH QUALITY EVALUATIONS

3.1 # of local evaluation

organizations that

improve their

performance

Annual Y4, Y5 NA NA NA NA NA NA NA NA 2 NA NA NA NA Not

measure

d yet

2


M&E Postgraduate

Modules who obtain

the academic

certification

Output Semi-

Annual

Y4 NA NA

Taller Introductorio NA NA NA NA NA NA NA NA 21 NA NA NA NA 21 NA

Módulo de

Competencias Clave

NA NA NA NA NA NA NA NA 21 NA NA NA NA 20 NA

Módulo de Monitoreo NA NA NA NA NA NA NA NA NA NA NA NA NA 13 NA

Módulo de Evaluación NA NA NA NA NA NA NA NA 18 NA NA NA NA 14 NA


55


of report

Years

for

report



Módulo de Gestión de

Evidencias

NA NA NA NA NA NA NA NA 21 NA NA NA NA 16 NA

Taller de Integración NA NA NA NA NA NA NA NA 21 NA NA NA NA 15 NA

Certificación Académica

(Diplomado)

NA NA NA NA NA NA NA NA NA NA NA NA NA 13 NA

3.1.2 # of local evaluation

organizations with

evaluation tools

according to quality

standards

Output Semi-

Annual

Y4, Y5 2013 0 NA NA NA NA NA NA 2 NA NA NA NA 2 NA

3.1.3 # of members of the

local evaluation

organizations who

improve their

competencies in

evaluation

Output Semi-

Annual

Y3, Y4,

Y5

NA NA NA NA NA NA NA NA 21 NA NA NA NA NA

Competencias Clave NA NA NA NA NA NA NA NA 21 NA NA NA NA 12 NA

Monitoreo NA NA NA NA NA NA NA NA 0 NA NA NA NA 14 NA

Evaluación NA NA NA NA NA NA NA NA 18 NA NA NA NA 15 NA

Gestión de evidencias NA NA NA NA NA NA NA NA 21 NA NA NA NA 14 NA

3.1.4 # of individuals trained

in M&E workshops

Output Quarterly Y1 to Y5 NA NA NA 50 20 89 40 126 60 24 11 15 0 165 0

Number of men NA NA NA 20 NA 39 NA 54 NA 9 5 6 0 70 NA

Number of women NA NA NA 30 NA 50 NA 72 NA 15 6 9 0 95 NA

3.1.5 # of M&E workshops

delivered

Process Quarterly Y1 to Y5 NA NA NA 2 NA 4 NA 18 NA 6 5 1 0 30 NA

3.1.6 # of capacity building

hours in M&E

delivered

Input Quarterly Y1 to Y5 NA NA 0 6 200 22 400 613 600 295 179 2 0 1089 600

Cumulative

Fee tied


56

IR 1. Evaluation studies are used to improve programming

Under IR 1, USAID approved ten evaluation/studies during Y4, twenty four in the life of the project, as we

present in graphic 5. In table 19 we present the evaluations/studies approved during the year of report along

with some relevant information of each one.

Graphic 5. Progress of Indicator 1.2.1. Number of evaluation/studies reports approved (cumulative)

Most evaluations conducted by PGRD have been performance evaluations as shown in Graphic 6.

Graphic 6. Number of evaluations/studies approved according to type of evaluation

Eight out of the ten approved studies (in year 4) have been already uploaded to USAID DEC. In addition,

one study have been uploaded in the Consorcio de Investigación Económica y Social website (see table 25)

2

6

14

24

0

5

10

15

20

25

30

Year 1 Year 2 Year 3 Year 4

N°

of

eval

uat

ion

s/st

ud

ies

app

rove

d (

cum

ula

tive

)

1

14

9

Impact Performance Sector/Crosscutting


57

Table 19. Number of evaluation/studies reports approved in year 4

Evaluation / Study Type of evaluation DO QTR of

approval

1 Evaluación de la Transferencia, Expansión y

Sostenibilidad del Modelo Municipios y

Comunidades Saludables en Perú

Performance DO1 QTR13

2 Evaluación Final de Desempeño del Proyecto

Promoción de la Justicia y la Integridad en la

Administración Pública, Pro-Integridad


3 La economía del VRAEM. Diagnóstico y opciones

de política

Sector/crosscutting DO1 QTR14

4 Rapid Assessment. Estudio de la Situación Actual

del Sector Educación en el Perú

Sector/crosscutting DO2 QTR15

5 Evaluación Final de Desempeño del Proyecto Peru

Bosques


6 Evaluación Final de Desempeño de la Iniciativa

Amazónica contra la Malaria


7 Evaluación Final del Proyecto Apoyo a la Expansión

de la Metodología de Escuelas Activas en Peru


8 Ethnographic Study Sector/crosscutting DO1 QTR16

9 DEVIDA Institutional Capacity Sector/crosscutting DO1 QTR16

10 Análisis de Género: Perú 2016 Sector/crosscutting Crosscutting QTR16

Main purpose of PGRD is to promote evidence-based programming, meaning that the evaluations and

studies elaborated by the Evaluations activity have to be used by USAID staff. To determine that, the

Evaluations team applied an online survey among 31 USAID representatives. Up to the day of report, we

received seven answers and could documented that eight out of twelve (67%) studies/evaluations have been

used for programmatic improvements, presented in Table 20. The list of studies used is presented in Table

27. We will encourage USAID representatives to answer the survey and will update the progress of this

indicator. Likewise, the PGRD team will conduct a qualitative assessment to deepen the factors that

influence the use or not of the evaluations.

Table 20. Progress of Indicator 1.1. % of evaluations/studies used to improve programs or projects

N° of evaluations/studies

evaluations that were

consulted in the survey

N° of

evaluations

studies used

%

Year 3 16 10 63%

Year 4 12 8 67%

When asked about the type of actions and/or decisions made upon each evaluation/study, the interviewed

ones indicated that:

• The conclusions and/or recommendations of eight out of the eight evaluations, were used in policy

dialogue with other entities, or to inform new proposals, or for reporting purposes.

• Four out of the eight Influenced programmatic or organizational changes at the institutional level

(strategies, priorities, etc.)

• One of the eight was used for modifying or incorporating new activities in the project/program

evaluated (reflected in new workplan, for example)

• One of the eight was used for modifying other projects/programs or contributed to the design of a

new project/program.


58

Table 21. Evaluations/studies used to improve programs or projects

Studies and Evaluations DO Type of

evaluation

Year of

approval

N° of people

who have read

the report

¿Evalution/Study

used?

Assessment of the transferability and

sustainability of the Healthy Communities

and Municipalities II model

DO1 Performance 2016 1 Not yet

Final performance evaluation of the Project

for the Promotion of Justice and Integrity in

Public Administration (Pro-Integridad)


Final performace evaluation of Peru Bosques

Project

DO3 Performance 2017 4 Yes

Final performance evaluation of the Amazon

Malaria Initiative (AMI)


Final evaluation of the project "Support and

expansion of the methodology of Active

Schools"

DO2 Performance 2017 3 Yes

Rapid assessment. Study of the current

situation of Peruvian education sector

DO2 Sector/Crosscutting 2017 1 Not yet

La economía del VRAEM. Diagnóstico y

opciones de política

DO1 Sector/Crosscutting 2016 2 Yes

Ethnographic Study I: Toma de Decisiones

en Hogares y Cultivo de Coca en Shanantia


Ethnographic Study I: Toma de Decisiones

del Antiguo Productor de Coca en una

Comunidad del Monzón


DEVIDA Institutional Capacity DO1 Sector/Crosscutting 2017 2 Yes

Análisis de Género: Perú 2016 Crosscutting Sector/Crosscutting 2017 4 Yes

Estudio: Desarrollo de capacidades en

proyectos seleccionados de USAID|Perú

Crosscutting Sector/Crosscutting 2017 4 Yes

Indicator 1.1.1 # of recommendation plans agreed for implementation

Even when the Plan of Recommendations is not a section of the Final Report of an evaluation and in order

to make feasible the use of the evaluations, PGRD included in some evaluations, specific actions to be

carried out according to the recommendations included in the report. These recommendations were

addressed to USAID. For the calculation of the indicator, we count the Recommendations Plans agreed with

USAID for implementation. As stated before, not all of the evaluations have recommendation plans.

During year 4, five recommendations plan have been agreed to be implemented, 9 in the life of the project.

Recommendations plan agreed belong to the following evaluations, as shown in Table 22.


59

Table 22 Evaluations/studies used to improve programs or projects

Recommendations plan agreed Year 1 – Year 3 Year 4

Retrospective Impact Evaluation of Alternative Development Program in

Huanuco, San Martin and Ucayali (2007-2012)

Yes

Midterm Evaluation of the Peru Decentralization and Local Governance Project -

Executive Report

Yes

Mid-term Evaluation of the Technical Assistance Program for the Ministry of

Environment - Peru / Evaluación de medio término del Programa de Asistencia

Técnica al Ministerio del Ambiente de Perú

Yes

Evaluación de las Actividades de Manejo y Mitigación de Conflictos en Perú Yes

Evaluación de la Transferencia, Expansión y Sostenibilidad del Modelo Municipios

y Comunidades Saludables en Perú

Yes

Evaluación Final de Desempeño del Proyecto Promoción de la Justicia y la

Integridad en la Administración Pública, Pro-Integridad

Yes

Evaluación Final de Desempeño del Proyecto Peru Bosques Yes

Evaluación Final de Desempeño de la Iniciativa Amazónica contra la Malaria Yes

DEVIDA Institutional Capacity Yes

TOTAL 4 5

IR 2. Implementing partners are able to manage per results

Over the year, under Intermediate Result 2, PGRD has conducted 33 M&E workshops (Indicator 2.2.6) for

implementing partners, as described below:

• Sixteen with the objective to prepare an M&E Plan

• Nine workshops as part of the M&E Postgraduate Diploma.

• Six addressed to Regional Management of Social Development of San Martin, Regional Education

Directorate of Education from Ucayali, SER and CEPCO for first and seconds performance-

based measurement of M&E best practices.

• One addressed to DO3 partners

• One directed to the DRE SM for the elaboration of the information system of PEEL.

Two hundred and eighty-five people (142 men and 143 women) (Indicator 2.2.5) from twenty four

implementing partners participated in at least one of these M&E workshops. Training and technical

assistance provided over the year added 3,897 staff hours for capacity development (Indicator 2.2.7), as

shown in Table 23.

Table 23. Indicator 2.2.7 Number of capacity building hours in M&E delivered (cumulative)

Hours


Goal Result Goal Result Goal Result Goal Result

400 258 1,040 1,683 1,760 4,211 1,920 8,108

The list of implementing partners attending to M&E workshops and the number of people trained is

presented in Table 24 and the number of individuals trained disaggregated by year and sex on Table 31.


60

Table 24. Implementing partners attending to M&E workshops and number of people trained

Implementing partner Type of

organization

N° of workshops

attended during

year 4

Number of people

trained

Men Women Total

1 Autoridad Nacional Forestal y de

Fauna Silvestre - SERFOR

National Government 7 0 3 3

2 Comisión Nacional para el Desarrollo

y Vida sin Drogas - DEVIDA

National Government 9 3 5 8

3 Ministerio del Ambiente National Government 9 5 5 10

4 Gobierno Regional San Martín Regional Government 15 27 36 63

5 Gobierno Regional San Martín -

Dirección Regional de Educación

Regional Government 15 19 4 23

6 Gobierno Regional Ucayali Regional Government 12 46 26 72

7 Gobierno Regional Ucayali - Dirección

Regional de Educación

Regional Government 17 28 35 63

8 Asociación para la Investigación y el

Desarrollo Integral - AIDER

Local NGO 10 0 3 3

9 Asociación Servicios Educativos

Rurales - SER

Local NGO 2 3 2 5

10 Centro de Estudios para el Desarrollo

y la Participación - CEDEP

Local NGO 4 1 0 1

11 Centro de Estudios y Promoción

Comunal del Oriente - CEPCO

Local NGO 2 0 5 5

12 Centro de Información y Educación

para la Prevención del Abuso de

Drogas - CEDRO

Local NGO 2 2 1 3

13 Conservación Internacional Perú Local NGO 1 0 1 1

14 Grupo Propuesta Ciudadana Local NGO 2 2 0 2

15 IDEA Internacional Local NGO 4 0 1 1

16 Sociedad Peruana de Derecho

Ambiental

Local NGO 2 0 2 2

17 Centro Agronómico Tropical de

Investigación y Enseñanza

International NGO 1 1 0 1

18 CHEMONICS International NGO 1 1 0 1

19 Lutheran World Relief International NGO 8 0 2 2

20 TECHNOSERVE International NGO 2 1 1 2

21 The Mountain Institute International NGO 3 1 1 2

22 The Nature Conservancy International NGO 4 0 1 1

23 Universidad Peruana Cayetano Heredia University 7 2 7 9

24 USDA Forest Other 1 0 1 1

Management Science for Health* 0 1 1

Total 142 143 285

* A representative from MSH participated in one of the workshops addressed to the Regional Government

of San Martín. However we do not consider MSH as an institution trained because the workshop was not

addressed to it and/or its needs.

By applying our data quality procedures, we have detected a fail in the excel-based M&E system related to

the number of people trained mainly in QTR9 (Year 3). Data reported in this annual report is the actual

data.


61

Table 25. Progress of Indicator 2.2.5 Number of individuals trained in M&E workshops

Year 1 Year 2 Year 3 Year 4 Cumulative

QTR13 QTR14 QTR15 QTR16 Result

Men 76 74 104 48 19 75 41 142 396

Women 58 110 81 58 13 63 45 143 392

Total 134 184 185 106 32 138 86 285 788

In table 26 and graphic 7, we present the number of people trained during the life of the project, counting

one person only once. For example, in year 1 participated 134 people, in year 2 participated 171 new

people, and so on. By year 4, PGRD have provided training to 654 different people.

Table 26 Number of people trained during the life of the project


Men 76 69 90 94 329

Women 58 102 67 98 325

Total 134 171 157 192 654

Cumulative --- 305 462 654 ---

Graphic 7. Progress of Number of people trained during the life of the project

One expected result of the Evaluation activity is that implementing partners have M&E competencies. One

of the indicators used and direct result of the training and technical assistance provided through the M&E

Diploma is the number of participants of the M&E Diploma who improve their competencies in M&E. This

improvement was determined through an auto diagnostic of training needs using an online survey. The

results are presented in Table 27.

134

305

462

654

0

100

200

300

400

500

600

700

Y1 Y2 Y3 Y4

Nu

mb

er

of

pe

op

le

Year of implementation


62

Table 27. Indicator 2.2.3. Number of participants of the M&E Postgraduate Program who improve their competencies in ME

Competencies

Year 4 Goal Group I Group II Result

52 26 26 NA

Key Competencies 52 17 18 35

Monitoring 52 19 22 41

Evaluation 52 19 23 42

Evidence management 52 18 23 41

It is important to mention that this results include those participants who obtained the academic

certification (Diploma) and those who did not.

In graphic 8 we present the initial and final level of needed training in technical competencies and in graphic

9, the level of development achieved in key competencies.

Graphic 8. Training required before and after the M&E diploma, according to technical functions

Source: Autodiagnostic online survey.



1.74

1.401.27

2.74 2.662.49

1

2

3

4

Monitoring Evaluation Evidence management

Leve

l of

trai

nin

g re

qu

ired

Before After

1 - 1.9 Needs to

be trained from

the basics

2 - 2.9

Reinforcement /

Update

3 - 3.9 Advanced

4: No studies

required


63

Graphic 9. Development of key competences before and after the M&E diploma

2.57

3.03

1

2

3

4

Key Competencies

Leve

l of

dev

elo

pm

ent

Before

After

1 - 1.9

Initial

development

2 - 2.9

Intermediate

3 - 3.9 Advanced

4: Maximum





64

A higher indicator used to determine the progress of the mention result is 2.2.1. Number of participants of the M&E Postgraduate Program who

obtain the academic certification. As described in section 3.2., we expected 52 participants to obtain the academic certification and 44 of them

achieved it as shown in table 28 and graphic 10. We include in the results, the participant from AIDER who self-financed her participation.

Table 28. Indicator 2.2.1. Number of participants of the M&E Postgraduate Program who obtain the academic certification in the M&E Postgraduate Diploma

Type of

Institution

Group I Group II Total

Goal Result Goal Result

Men Women Total Men Women Total Men Women Total Men Women Total Men Women Total

National

Government 2 4 6 2 4 6 6 6 12 6 6 12 8 10 18

Regional

Government 4 1 5 1 0 1 6 2 8 6 2 8 7 2 9

Local NGO 5 6 11 4 6 10 1 2 3 0 2 2 4 8 12

International NGO 2 2 4 2 1 3 0 2 2 0 1 1 2 2 4

University 0 0 0 0 0 0 1 0 1 1 0 1 1 0 1

Total 13 13 26 9 11 20 14 12 26 13 11 24 22 22 44


Graphic 10. Number of participants of the M&E Postgraduate Program who obtain the academic certification by type of Organization

As a result of the participation in the M&E Postgraduate Diploma, we expected that the 20 partners

could have five quality tools (products of the M&E Diploma modules with satisfactory grade) that allow

them to produce and use data quality information. The results obtain in indicator 2.2.2. are shown

below (Table 29 and Table 30).

Table 29. Indicator 2.2.2. Number of IP with M&E tools according to quality standards

Type of partner

Year 4 Goal

Code of

ethics for

M&E

M&E

Plan

Monitoring

report

Evaluation

design and

SOW

Communication

plan

N° of partner

that have 5

quality tools

20 19 19 19 17 17 15

National Government 3 3 3 3 3 3 3

Regional Government 4 4 3 4 4 4 3

Local NGO 8 7 8 8 5 6 5

International NGO 4 4 4 3 4 3 3

University 1 1 1 1 1 1 1

Table 30. Number of IP according the number of quality M&E tools as a result of the M&E Diploma

Type of partner Year 4

Goal

1 Product 2 products 3 products 4 products 5 products

20 19 19 19 17 17

National Government 3 3 3 3 3 3

Regional Government 4 4 3 4 4 4

Local NGO 8 7 8 8 5 6

International NGO 4 4 4 3 4 3

University 1 1 1 1 1 1

One important tool for generating quality information is the M&E Plan. During year 4, the project

provided training and technical assistance for developing M&E plans, achieving the elaboration and

approval of 11. During the life of the project, 25 M&E plans have been elaborated and approved by

USAID, implementing partner or the M&E Postgraduate Diploma. Results are shown in graphic 11.

18

9

12

4

1

0

2

4

6

8

10

12

14

16

18

20

National Government Regional Government Local ONG International ONG University

Nu

mb

er o

f p

eop

le


66

Graphic 11. Progress of indicator 2.1.1. Number of new or modified M&E Plans approved (cumulative)

As stated before, the importance of having quality tools is to have quality data that can be used for

evidence-based decision making. Expected result of the capacity building activities is that partners

manage for results and we have established two indicators that inform us if we are achieving this result

or not:

• Indicator 2.1 Number of implementing partners that improve their institutional performance in M&E

based on best practices

• Indicator 2.2 Number of implementing partners with M&E Plans and reporting results

As we present in the Table 31 and graphic 12, is that four out five partners which have a second

performance assessment have improved its institutional performance level up to date of report.

Table 31. Indicator 2.1 Number of implementing partners that improve their institutional performance in M&E based on

best practices

Partner Initial performance

level

Second peformance

level

Increased its

performance level?

1 GORESAM (GRDS) May 2014 44.71% Apr 2016 27.10% No

2 DREU Apr 2016 31.00% Oct 2016 42.70% Yes

3 DRESM Apr 2016 39.20% Not measured yet NA

4 GOREU Jul 2015 29.80% Not measured yet NA

5 MINAM

Not measured yet

NA

Oficina de Coordinación Internacional Nov 2013 39.82%

Oficina de Planificación y Presupuesto Dec 2013 72.29%

CGP Nov 2013 39.92%

DPNIGA Oct 2013 40.10%

6 SERFOR Not measured yet Not measured yet NA

7 CEPCO Dec 2013 62.68% Mar 2017 80.00% Yes

8 SER Nov 2013 37.00% Mar 2017 70.80% Yes

9 CEDRO Nov 2013 69.26% Apr 2017 82.40% Yes

In the graphic below we present the levels obtained by each partner.

3

0

10

7

15 14 15

25

0

5

10

15

20

25



Nu

mb

er o

f M

&E

Pla

ns

app

rove

d


67

Graphic 12. Institutional performance level by implementing partner approved

Indicator 2.2 “Number of implementing partners with M&E Plans and reporting results” is referred to those

partner which have an M&E Plan for each project implemented and elaborate monitoring reports

including recommendations. Results show that 4 out of the 5 meet this requirement as shown in table

32.

Table 32. Indicator 2.2 Number of implementing partners with M&E Plans and reporting results

Partner Elaborate M&E

Plans for all

Project?

Elaborate monitoring

reports including

recommendations?

Meet the

requirement of

indicator 2.2

1 GORESAM (GRDS) No Yes No

2 DREU Yes Yes Yes 3 CEPCO Yes Yes Yes 4 SER Yes Yes Yes 5 CEDRO Yes Yes Yes

IR 3. Local partners conduct high quality evaluations

Over the year, under Intermediate Result 3, PGRD has conducted 12 M&E workshops (Indicator 3.1.5)

for evaluation organizations, as described below:

• Nine workshops as part of the M&E Postgraduate Program.

• Two for the elaboration of the Institutional Strategic Plan of PERUME Network

• One as part of the evidence week.

Thirty-nine people (16 men and 23 women) (Indicator 3.1.4) from the two evaluation networks

participated in at least one of these M&E workshops. Training and technical assistance provided over the

year added 475.7 staff hours for capacity development (Indicator 3.1.6), as shown in Table 33.

45%

31%

63%

37%

69%

27%

43%

80%

71%

82%

0%

20%

40%

60%

80%

100%

GORESAM (GRDS) DREU CEPCO SER CEDRO

Per

form

ance

leve

l

Implementing partners

Initial Second

0 < 60%

Initial level

60 < 80%

Intermediate

80 - 100%

Adequate


68

Table 33. Indicator 3.1.6 Number of capacity building hours in M&E delivered (cumulative)

Hours



0 5.5 200 22 400 613.3 600 1089

The number of people trained in each network is presented in Table 34 and the number of individuals

trained disaggregated by year and sex on Table 35.

By applying our data quality procedures, we have detected a fail in the excel-based M&E system related

to the number of people trained mainly in QTR9. Data reported in this annual report is the actual data.

Table 34. Number of individuals trained in M&E workshops

Evaluation network N° of workshops

attended during year 4

Number of people

trained

Men Women Total

1 Red Peruana de Monitoreo y

Evaluación - PERUME

12 15 22 37

2 Red Peruana de Evaluación - EvalPerú 6 1 1 2

Total 16 23 39

Table 35. Progress of Indicator 3.1.4 Number of individuals trained in M&E workshops


QTR13 QTR14 QTR15 QTR16 Result

Men 20 19 15 9 5 6 0 16

Women 30 20 22 15 6 9 0 23

Total 50 39 37 24 11 15 0 39

In table 36 and graphic 13, we present the number of people trained during the life of the project,

counting one person only once. For example, in year 1 participated 50 people, in year 2 participated 29

new people, and so on. By year 4, PGRD have provided training to 118 different people.

Table 36 Number of people trained during the life of the project


Men 20 15 8 6 49

Women 30 14 14 11 69

Total 50 29 22 17 118

Cumulative --- 79 101 118 ---


69

Graphic 13. Progress of Number of people trained during the life of the project (cumulative)

One expected result of the Evaluation activity is that members of the local evaluation networks

evaluation competencies. One of the indicators used and direct result of the training and technical

assistance provided through the M&E Program is the number of participants of the M&E Diploma who

improve their competencies in M&E. This improvement was determined through an auto diagnostic of

training needs using an online survey. The results are presented in Table 37.

Table 37. Indicator 3.1.3. Number of members of the local evaluation organizations who improve their

competencies in evaluation

Competencies

Year 4

Goal Group I Group II Result

21 12 9 -----

Key Competencies 21 6 6 12

Monitoring 0 6 8 14

Evaluation 18 7 8 15

Evidence management 21 6 8 14

It is important to mention that this results include those participants who obtained the academic

certification and those who did not. It is also important to mention that an unexpected result is that a

group of participants self-financed the modules for which they not received a scholarship (monitoring

and evidence management in the case of EvalPeru network).

In graphic 14 we present the initial and final level of needed training of M&E Postgraduate Diploma in

technical competencies and in graphic 15, the level of development achieved in key competencies.

50

79

101

118

0

20

40

60

80

100

120

Y1 Y2 Y3 Y4

Nu

mb

er

of

pe

op

le

Year of implementation


70

Graphic 14. Training required before and after the M&E diploma, according to technical functions

Graphic 15. Development of key competences before and after the M&E diploma

A higher indicator used to determine the progress of the mention result is 3.1.1. Number of participants

of the M&E Postgraduate Program who obtain the academic certification. As described in section 3.3., we

expected 21 participants to obtain the academic certification in the modules of Key competences,

Evaluation and Evidence Management. Results are presented in Table 38.







2.12

1.76

1.45

3.03 2.962.70

1

2

3

4

Monitoring Evaluation Evidence Management

Leve

l of

trai

nin

g re

qu

ired

Before After

1 - 1.9 Needs to be

trained from the basics

2 - 2.9 Reinforcement /

Update

3 - 3.9 Advanced

4: No studies required

3.503.19

1

2

3

4

Key Competencies

Leve

l of

de

velo

pm

en

t

Before After

1 - 1.9

Initial

2 - 2.9

Intermediate

3 - 3.9

Advanced

4: Maximum


71

Table 38. Indicator 3.1.1. Number of participants of the M&E Postgraduate Program who obtain the academic

certification in the M&E Postgraduate Program (Modules)

As stated before, an unexpected result is that a group of participants self-financed the modules for

which they not received a scholarship (monitoring and evidence management in the case of EvalPeru

network), obtaining the Diploma. Thirteen participants obtained the Diploma as shown in table 39. The

participants from EvalPerú obtained their Diploma by convalidating their competencies.

Table 39. Number of participants of the M&E Postgraduate Program who obtain the academic certification

(Diploma)

Evaluation Organization Group I Group II Total

Men Women Total Men Women Total Men Women Total

Red Peruana de Monitoreo

y Evaluación - PERUME

2 5 7 1 3 4 3 8 11

Red Peruana de Evaluación -

EvalPerú

0 1 1 1 0 1 1 1 2

Total 2 6 8 2 3 5 4 9 13

As a result of the participation in the M&E Postgraduate Program, we expected that the two networks

could have two quality tools (products of the M&E Diploma modules with satisfactory grade) that allow

them to conduct quality evaluations. EvalPerú was expected to elaborate 2 tools (Code of Ethics for

M&E and Communicacion Plan); PERUME network was expected to produce one additional product

(TdR of an evaluation design). Both networks elaborated satisfactorily the expected products. The

results obtain in indicator 3.1.2. are shown below (Table 40).

Table 40. Indicator 3.1.2. Number of local evaluation organizations with evaluation tools according to quality

standards

Evaluation

organizations

Year 4

Goal Code of

ethics for

M & E

TdR of an

evaluation +

Design

Communication

plan

N° of organizations

with evaluations

tools

2 2 1 2 2

Red Peruana de

Evaluación - EvalPerú

1 1 Not

applicable

1 1

Red Peruana de

Monitoreo y Evaluación -

PERUME

1 1 1 1 1

Modules Year 4

Goal Result

Introductory Workshop 21 21

Module of Key competencies 21 20

Module of Monitoring NA 13

Module of Evaluation 18 14

Module of Evidence Management 21 16

Integration Workshop 21 15


72

The indicator 3.1. “Number of local evaluation organizations that improve their performance” has not

been measured during year 4, it is programmed to be measured in year 5.

U.S. Agency for International Development

Av. La Encalada s/n, Santiago de Surco

Lima 33, Perú

Email: [email protected]