documents.wfp.orgdocuments.wfp.org/stellent/groups/public/documents/rep… · web viewassessment...
TRANSCRIPT
Decentralized evaluation for evidence-based decision makingWFP Office of Evaluation
Decentralized Evaluation Quality Assurance System (DEQAS)
Quality Checklist for Decentralized Evaluation ReportVersion November 2015
[title of the decentralized evaluation]Overall
General Comments/Status
Length: Report is concise, not exceeding 40 pages excluding Annexes and Executive SummaryAccessibility Written in a clear and accessible manner The structure of the findings allow readers to
understand evaluation findings Key messages are summarised and highlighted The report presents a good balance between descriptive
and analytical information Terminology is used correctly The report is free from grammar, spelling, or
punctuation errors The tone of the report is constructive The language used in the report is simple and clear and
the report is reader friendly
Credibility The report presents successes and failures, positive and
negative sides in a balanced way.
T O R Q C V e r s i o n O c t o b e r 2 0 1 5 P a g e 1 | 10
The report raises critical issues when necessary The report’s findings and conclusions are fully evidence-
based.Editing
WFP format and templates are applied Acronyms are spelt out the first time they are used Paragraphs and pages are numbered electronically Table of contents is included and lists tables, graphs,
figures and annexes Tables and diagrams are used as relevant and are
numbered List of acronyms is included When data or quotes are provided, the report indicates
the source (either in a footnote or directly below a table/graph).
Cover Page
Uses WFP DEQAS Report Template Title of the decentralized evaluation is identical to that
in the TOR Date and status of the report (draft/final) indicated on
the cover page
Executive SummaryOverall: The Executive Summary should provide a complete and balanced synthesis of the evaluation findings, conclusions and recommendations in not more than 4 pages.
Expected Content Assessment criteria Comments/status
Introduction: main features of the evaluation including evaluation purpose and objectives, main users/ intended audience, context, subject of the evaluation.
Executive Summary is coherent, self-explanatory and self-contained
The key findings form the largest part of the
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 2 | 10
Methodology Key findings on all the
evaluation questions Overall conclusions Recommendations
section All recommendations
are presented in brief The Executive
Summary could be used as a standalone aid to decision-making
Length does not exceed 4 pages
1. IntroductionOverall: Succinct overview of the evaluation subject, context and features, providing clarity on why and how the evaluation was carried out, in not more than 5 pages.
Expected Content Assessment criteria Comments/status
Clear reasons for the evaluation
Specific objectives Scope of the evaluation Stakeholders in the evaluation Intended users and how they
will use the evaluation findings
Introduction clearly sets the scene for the evaluation, including its rationale, scope, stakeholders and users.
Information is succinct and focused
1.1 Overview of the Evaluation SubjectFactual information regarding the evaluation subject including:
Type of intervention (operation, activity, thematic area, transfer modality, pilot project) Geographic scope of the evaluation subject Relevant dates: Approval date; start date; end date
Information is relevant and important to understanding the subject of the evaluation:o What it iso When it was designed o What are the key
inputs ($ value) o What are the planned
outputs? (beneficiaries, MT,
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 3 | 10
Planned outputs at design o Beneficiary numbers
(planned and revised) disaggregated by gender/activity
o Amount of transfers (food, cash, vouchers)
o Any other outputs Planned outcomes at design Key activities Main partners (Government; NGOs; Bilateral; Multilateral) Resources (% funded of total requirements) and key donors. If subject funded from pooled funds or is a component/activity of an operation, show resource allocated to the subject of the evaluation Assessment of the Logical Framework or similar tool from evaluation perspective Other relevant preceding/ concurrent activities/ interventions/operations Any amendments to initial design Include reference to:o Past evaluations/reviews related to the subjecto Maps/graphs for illustration
Cash &Voucher $)o What is the
target/scope?o What are the planned
outcomes? o Who is involved in its
implementation? Highlights relevant issues
from past evaluations and reviews that are relevant to the evaluation
Gender dimensions explained
Differences between original design and implementation are explained if appropriate
1.2 ContextOverview of the surrounding context directly relevant to the
Information is relevant and necessary to understand the
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 4 | 10
evaluation including:
Poverty, food and nutrition security
Government policies and priorities
Humanitarian issues Gender dimensions of the
context Key external events Features of international
assistance in the area Other WFP work in the area Work of other key actors
implications for the evaluation subject
Relevant indicators and latest information have been identified and trend data used
Gender context of country is described
Information draws from and is consistent with inception report, updated where appropriateInformation is explicitly geared to the evaluation subject, rather than being generically presented.
1.3 Evaluation Methodology and Limitations
Methodology used for the evaluation is summarized
Evaluation criteria applied to the evaluation are listed and justified (and if possible interpreted for the subject)
The main evaluation questions are set out
Summary of the overall methodological approach is provided
Specific methods and sampling used are provided
A statement on how validity and reliability was addressed is included
Limitations of methods or data
Information allows reader to understand how the evaluation was conducted
Methodological approach is comprehensive and presents a systematic approach that generates trust in the credibility of the evaluation
Methodological approach is coherent, logical and in line with the TOR and inception report
Specific methods and D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 5 | 10
availability encountered, and mitigation strategies applied/implications for the findings, are explained
Gender considerations for the data collection methods are explained
Full details in a specific Annex
sampling applied are relevant to the methodological approach and are feasible/appropriate
Limitations and their effects on the evidence base are included
Statement on validity and reliability generates confidence in the evidence base
Methods demonstrate consideration of gender equality and women’s empowerment
Full methodology provided in an Annex (updated from Inception Report in relation to its actual implementation).
2. Evaluation FindingsOverall: This section should be the most substantial of the evaluation report. It should present information that supports the conclusion(s) of the evaluation team against the key evaluation questions. It should not exceed 30 pages.
Expected Content Assessment criteria Comments/status
Presentation of clear data analysis against all evaluation questions, including triangulated information
Findings are substantiated
The findings have been formulated clearly and are based upon evidence collected
Findings are triangulated Findings use credible
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 6 | 10
by credible evidence that has been checked for accuracy, consistency and reliability
Limitations or gaps in evidence are presented
Indication of where evidence is inconclusive
Information sources are provided and accurately quoted
Findings are adequate to substantiate the conclusions and recommendations made by the evaluation team (next section)
Findings use gender (and equity) sensitive language throughout and data is disaggregate.
Findings are presented in a gender and culturally sensitive manner.
external sources of information
Findings address the evaluation criteria chosen
Information sources are provided, and where evidence is inconclusive this is clearly stated.
Findings address any limitations or gaps in evidence
Visual aids are used in a clear an accessible way to illustrate the argument
Information sources are quoted
The findings balance detail and synthesis
Findings reflect gender analysis and address gender sensitivity including use of gender disaggregated data Findings are presented by evaluation question
3. Conclusions and RecommendationsOverall: Conclusions and Recommendations are core components of the evaluation report; Lessons learned and Good Practices should be included if appropriate. The section should not exceed 5 pages.
Expected Content Assessment criteria Comments/status
3.1 Overall Assessment/Conclusion
Conclusions are logically derived from
Conclusions follow logically from the
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 7 | 10
findings and substantiated by evidence
There is clear referencing back to findings
Report contains a brief and balanced assessment of the evaluation findings related to the selected evaluation criteria and questions. This may be organised by criteria, theme and/or by evaluation question
findings, with clear and explicit linkage to them
A conclusive picture is presented, formed from the previous sections of the report
A summary against evaluation criteria is included
Conclusions are succinct, synthesising common findings
Conclusions include recognition of different stakeholder groups and gender aspects
No new evidence is introduced that has not been presented in the findings
Conclusions are impartial and free of bias
3.2 Lessons Learned and Good Practices (optional)
Lessons are presented that WFP and partners can use in future contexts to improve WFP projects and programmes.
Lessons include any positive and negative lessons
Good practices are presented which represent successful
Lessons contribute to wider organizational learning
Lessons specifically indicate what should be repeated or avoided in future contexts to guide future action
Good practice is potentially replicable
Good practices describe why things have
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 8 | 10
strategies or interventions that have performed well;
Emerging good practices can be presented (i.e. an intervention that has potential to perform well)
worked
3.3 Recommendations
Short paragraphs describing up to 10 recommendations, which are all relevant to the subject of the evaluation
Recommendations logically derive from the findings and conclusions and are clearly linked to the evidence base
Recommendations are realistic/feasible
Recommendations are prioritised, sequenced and addressed to specific target groups
Gender issues are addressed where appropriate in recommendations
Recommendations follow logically from the findings and conclusions
They are based on the analysis and evidence collected
They are realistic (implementable), presenting plausible options for improvement
They are clear and specific and clustered /presented in a logical order
They are prioritised and have a clear timeframe for implementation
The target group for each recommendation is clear and relevant to the intended users of the evaluation
Gender is integrated into the recommendations and where feasible,
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 9 | 10
recommendations for improving WFP’s performance on gender are provided.
Annexes Comment Comments/status
Annexes should support and expand on text in the main report. Including: Terms of Reference Documents Reviewed Stakeholders
Interviewed Evaluation matrix Data Collection tools Bibliography List of Acronyms
Annexes listed and numbered
Annexes referenced, where appropriate in the main report
Data collection tools are included
Not all working documents are included
D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 10 | 10