module 4 evaluation: general characteristics. 25/02/20162 overview of the module definitions...

Module 4 –Module 4 – Evaluation: Evaluation: General CharacteristicsGeneral Characteristics

23-05-06 2

Overview of the ModuleOverview of the Module

• Definitions• Purposes• Milestones• Stakeholders• A win-win exercise

23-05-06 3

Program evaluation consists in using scientific Program evaluation consists in using scientific research methods to check the design, the research methods to check the design, the implementation and the effectiveness of the implementation and the effectiveness of the program. program. (General Accounting Office, USA)(General Accounting Office, USA)

Definitions of EvaluationDefinitions of Evaluation

Evaluating the effectiveness of a program Evaluating the effectiveness of a program consists in using scientific research methods to consists in using scientific research methods to assess what part of the results, both intended assess what part of the results, both intended and not, can be attributed to program and not, can be attributed to program activities. The causal relationship is established activities. The causal relationship is established by comparing the actual results with an by comparing the actual results with an estimation of what the results would have been estimation of what the results would have been without this program. without this program. (General Accounting Office, (General Accounting Office, USA)USA)

23-05-06 4

Various Definitions (cont.)Various Definitions (cont.)

Evaluation is a project management tool which makes it possible to better program for the future based on the past and to inform the various stakeholders. It is also a control instrument that is part of the trust reinforcement process. (ONG Round Table, Jan. 94, France.)

23-05-06 5

Evaluation is a function that consists in making an appraisal, as systematically and objectively as possible, of an ongoing or finished project, program or series of courses of actions, its design, its implementation and its results. The purpose is to assess the relevance of the objectives and the extent to which they were attained, the efficiency in development terms, the efficacy and the viability. (Development Assistance Committee DAC/OECD)


23-05-06 6

To evaluate is to communicate. (UNICEF)The evaluation of a program is its periodical, unbiased and objective review to determine, in view of the circumstances, to what extent its objectives, its design and its results, both expected and unexpected, are satisfactory. The evaluation explores the purpose of the program, its effects and its cost-effectiveness compared to other approaches. (Treasury Board of Canada.)

Evaluation is the measurement of the interactions between a development project and its context. (IRFED)


23-05-06 7


• verifying the relevance and the coherence of the initial objectives;

• assessing how resources are put to use and how adequate they are in relation to the objectives;

• measuring if the objectives have been attained and if the results are sustainable. (Evaluation Guide (July 2001),Bureau de l'évaluation, DGCID, MAE, France)

The purpose of evaluation is to pass a judgement on the value of an action. It aims at:

23-05-06 8

For the World BankFor the World Bank• The purpose of evaluation is to measure

to what extent and with what degree of efficiency operational programs and activities produce the desired results and to reuse the information gathered to outline new directions, policies and procedures.

• Evaluation is an essential instrument that supports a result-based management approach aimed at providing a framework for strategic planning and management based on the principles of knowledge improvement and responsibility sharing.

What are the Purposes What are the Purposes of Evaluationof Evaluation ? ?

Discussion

23-05-06 10

Know,understand

and learn lessons

Manage, monitor

DecidePeople in charge

Manager,Operational

staff

Researcher

Know, be informed

Public, beneficiaries, contributors

4 mainpurposes

Purposes of an EvaluationPurposes of an Evaluation

23-05-06 11

Action/ measure

Action/ measure

Project Project Program

Policy

Action/ measure

Action/ measure

Action/ measure

Action/ measure

Projects, programs, policies

Scope of an EvaluationScope of an Evaluation

23-05-06 12

Identification Preparation Launch

Implementation (Phase 1)

Implementation (Phase 2)

End of Project Ongoing effects

Key milestones in the life of a project, a program or a policy

Feasibility Studies (ex ante evaluation) Ongoing Evaluation Mid-term Evaluation Ex Post Evaluation Post-Completion Evaluation retrospective

The “Key Milestones" of an The “Key Milestones" of an EvaluationEvaluation

23-05-06 13

Key Evaluation ProductsKey Evaluation Products

Before During At the end

Afterwards

To check the design

To improve the implementation

For accountability

purposes

To assessthe impact

Program

StartStart

23-05-06 14

Types of EvaluationsTypes of Evaluations• Formative Evaluation

– Aims at improving the functioning of an existing program

– Carried out during the implementation of the program

– Involves stakeholders and operational staff who will learn during the evaluation and then redirect the project, if needed.

• Summative Evaluation– Done to evaluate the results of a project or

program (its effects) – Carried out once the program is over– Makes it possible to grasp the totality of the

results (the ‘sum’).

23-05-06 15

Donor Partner country

Possible co-funders

Evaluating body

Project operational

staff

External experts

Stakeholders/beneficiaries

?

Who Participates in an Who Participates in an Evaluation? Evaluation?

23-05-06 16


Possible co-funders

Evaluating body

Project operational staff

External experts


Self-evaluation


23-05-06 17


Possible co-funders

Evaluating body


External experts


Internalevaluation


23-05-06 18


Possible co-funders

Evaluating body


External experts


Joint evaluation


23-05-06 19


Possible co-funders

Evaluating body


External experts


External evaluation


23-05-06 20


Possible co-funders

Evaluating body


External experts


External evaluationIncluding partners


23-05-06 21


Possible co-funders

Evaluating body


External experts


Participative Evaluation


23-05-06 22

… as long as norms and ethical principles are respected

Evaluation, a Win-Win Evaluation, a Win-Win Exercise…Exercise…Methods used in evaluations• Confrontation with facts and

standards, use of documents, suitable methods and credibility of findings

• Transparency and pluralism • Dialogue• Outside perspective on things • Logic, the ultimate weapon

module 4 evaluation: general characteristics. 25/02/20162 overview of the module definitions...

Documents