aaron zazueta chief evaluation officer 2013 evaluation in the gef
TRANSCRIPT
Aaron Zazueta
Chief Evaluation Officer
2013
EVALUATION IN THE GEF
Page 2
Overview
1. What is M &E
2. Monitoring & Evaluation Policy
3. Terminal Evaluations in the GEF
4. GEF5 - GEFEO work program
5. Special Initiatives: Climate-Eval and CEPKE
Page 3
M&E Policy
Page 4
M&E and RBM
• Monitoring is one of the main instruments of Results Based Management
• Evaluation is a “reality check” on monitoring and RBM• Monitoring & RBM tell whether the organization is “on track”• Evaluation could tell whether the organization is “on the right
track”
Page 5
Monitoring <-> Evaluation Monitoring Evaluation
Page 6
The GEF M&E Policy
• Indicates the GEF minimum M&E requirements covering: project design, application of M&E at the project level, and project evaluation.
• Defines the concepts, role, and use of monitoring and evaluation within the GEF
• Defines the institutional framework and responsibilities.
• Current M&E Policy: Approved by GEF Council in November 2010
Page 7
M&E in the GEF
2 overarching objectives:
• Promote accountability for the achievement of GEF objectives through the assessment of results, effectiveness, processes, and performance of the partners involved in GEF activities.
• Promote learning, feedback, and knowledge sharing on results and lessons learned among the GEF and its partners as a basis for decision making on policies, strategies, program management, programs, and projects; and to improve knowledge and performance.
Page 8
Separate reporting lines for Monitoring (through Secretariat) and Evaluation (through Evaluation Office)
GEF Council
Project and Program Implementation Reports Agency Portfolio Reports Project documents with M&E plans
Corporate evaluations Project and Program Independent evaluations
Project and Program evaluations
GEF Secretariat GEF
Evaluation Office Agency
evaluation units
Agency GEF coordination units
GEF projects and programs
Project and Program Implementation Reports Project and Program monitoring documentation Terminal evaluations
Annual evaluation reports Overall Performance Study (to Assembly) Annual Work Program and Budget
Annual Monitoring Report Evaluation Management Response Programming documents and indicators Results Based Management
Page 9
M&E Key Responsibilities in GEF
Page 10
GEF Evaluation Office
Mission
“Enhance global environmental benefits through excellence, independence and partnership in monitoring and evaluation”
Principles• Impartiality• Professionalism• Transparency
Page 11
Types of Evaluations
Project Evaluations Program Evaluations Country Level Evaluations
Impact Evaluations Cross-cutting and thematic evaluations
Process and performance evaluations
Ad-hoc Reviews Overall Performance Studies (OPS)
Different TORs, scopes, frequency, audiences , methodologies, lead
Page 12
Follow up to Evaluations
• A management response is required for all evaluations and performance reports presented to the GEF Council by the GEF EO. It is coordinated by GEF CEO with inputs from other GEF stakeholders.
• GEF Council takes into account both the evaluation and the management response when taken a decision.
• GEF EO reports on implementation of decisions annually (Management Action Record)
Page 13
M&E: Minimum Requirement 1
Design of M&E PlansConcrete and fully budgeted M&E plan by CEO endorsement for FSP and CEO approval for MSP. Project logical frameworks should align with GEF focal area results frameworks. M&E Plan should include:
• SMART indicators• Baseline data for M&E by CEO endorsement• Mid Term Reviews (where required or foreseen) and Terminal
Evaluations included in plan• Organizational set up and budget for M&E
Page 14
M&E: Minimum Requirement 2
Implementation of M&E Plans Project/program monitoring and supervision will include execution of the M&E plan:
• Use of SMART indicators for process and implementation• Use of SMART indicators for results• Baseline for the project is fully established and data are compiled
to review progress• Organizational set up for M&E is operational and its budget is
spent as planned
Page 15
M&E: Minimum Requirement 3
Project/Program Evaluations: • All full sized projects and programs will be evaluated at the end of
implementation• Evaluations should:
Be independent of project management or reviewed by GEF Agency evaluation unit
Apply evaluation norms and standards of the GEF Agency Assess, as a minimum, outputs and outcomes, likelihood of sustainability,
compliance with Minimum Requirements 1 & 2 Contain basic project data and lessons on the evaluation itself (including TORs) Should be sent to GEF EO within 12 months of completion of project/program
Guidelines for evaluating MSPs/EAs will be developed
Page 16
M&E: New Minimum Requirement 4
Engagement of Operational Focal Points
• M&E plans should include how OFPs will be engaged• OFPs to be informed on M&E activities, including Mid Term
Reviews and Terminal Evaluations, receiving drafts for comments and final reports
• OFPs invited to contribute to the management response (where applicable)
• GEF Agencies keep track of the application of this requirement in their GEF financed projects and programs
Page 18
Terminal Evaluations
• Purpose of Terminal Evaluations Track performance of GEF portfolio (aggregation of project results) Provide feedback on performance Synthesize lessons that may help improve GEF functioning
Page 19
Terminal Evaluations
• Are carried out by Agencies at project completion• Follow GEF EO Terminal Evaluation Guidelines• Done 6 months (±) of project completion• Reviewed by Agency evaluation office• Submitted to the GEF EO within 12 months of evaluation
completion
Page 20
Terminal Evaluation Review
• GEF EO reviews Terminal evaluations not reviewed by the Agency Evaluation Offices Random “spot checking” for the terminal evaluations reviewed by the
Agency Evaluation Offices
• Primary reviewers prepare draft reviews• Peer reviewers provide feedback• If required senior reviewers confirm the findings
Page 21
Terminal Evaluation Review
• Draft reviews shared with Agencies• Agency feedback incorporated• TE verification on a sample basis• Ratings: outcomes, sustainability, supervision, M&E, and quality
of TE aggregated• Synthesis of the data presented in the APR
Page 22
GEFEO GEF-5 Work Program
• Implementation of GEF M&E Policy Annual Reporting to Council: Country Portfolio Evaluations (May-June) Performance and process issues (May-June) Impact (November)
• Thematic/Cross-sectoral evaluations (November) Knowledge Sharing dissemination of lessons;
• participation in country support program
• Overall Performance Studies – replenishment process (every 4 years)
• Active participation evaluation communities (UN/MDBs)
Page 23
Knowledge ProductsThematic Evaluations
Country Portfolio Evaluations Sub-Studies and Evaluations
Performance
Impa
ct
M & E Policy
Learning Products
Blog MultimediaSignposts WebinarsLearning Products
Page 24
Special Initiatives
Climate-Eval: Community of Practice on Evaluation of Climate Change and Development www.climate-eval.org
CEPKE: Comprehensive Evaluation Platform for Knowledge Exchangewww.cepke.net
Thank you