lecture 11 pmec

8/3/2019 Lecture 11 PMEC

1/30


2/30

M&E system

Theextent of progresstoward theprojects

objectives, and thereasons for success or

failure.This is what a monitoring andevaluation system provides.


3/30

Monitoring and Evaluation

Monitoring and evaluation is the process of collecting andanalyzing information about the project that tells youwhether you are on track to reach your objectives, andwhether or not the project achieved or contributed to the

desired impact. In order to know whether or not you are on track to

achieving your projects objectives, you must monitor theproject during implementation as well as evaluate itsimpact at the end of the project.

It is necessary to plan for monitoring and evaluationwhen you design your project; this will help you both todesign an effective program and ensure that you plan(and budget) for appropriate monitoring and evaluation

activities.


4/30

Project M&E

The process of monitoring and evaluation generally

helps you answer the following questions:

Inputs: Were program inputs available,

adequate, timely?

Activities: Were activities performed onschedule?

Outputs: Were outputs produced? Were they ofacceptable quality?

Effects: Were effects observed?

Impact: Was impact achieved?


5/30

What is the need for conductingEvaluation?

The evaluation of projects contributes towards

Decision-making,

Strategic Planning

Reporting or Modification

Organizational learning

Accountability

Impact


6/30

Project Evaluation

Project evaluation is a systematic and objectiveassessment of an ongoing or completed project.The aim is to determine the relevance and level ofachievement of project objectives, developmenteffectiveness, efficiency, impact andsustainability.Evaluations also feed lessons learned into thedecision-making process of the project

stakeholders.Evaluation is based on certain standards, criteria andmeasures


7/30

Project Evaluation

Evaluations provide an opportunity for in-depth reflection on the strategy andassumptions guiding the project.

They assess progress made towards theachievement of a projects objectives andmay recommend adjustments to itsstrategy.

They are also a means by which to assesshow well project-level actions link tohigher level strategies and objectives


8/30

Key Elements of ProjectEvaluation

The concept and principles of projectevaluation;

Policies for project evaluations and roles

and responsibilities;

The implementation of project evaluationand evaluation report;

Follow-up, dissemination andknowledge sharing of evaluationoutcomes


9/30

Why Organizations need tocarry out Project Evaluation?

Evaluation is an important tool that your organization canuse to demonstrate its accountability,

Improve its performance, increase its abilities for

obtaining funds or future planning, and fulfill theorganizational objectives

By communicating the results of the evaluation, yourorganization can inform its staff, board of directors,

service users, funders, the public, or other stakeholdersabout the benefits and effectiveness of yourorganizations services and programs


10/30

Project Evaluation is anAccountability Function

By evaluating a project, you monitor the process to

ensure that appropriate procedures are in place

forcompleting the project on time, and you identify

and

measure the outcomes to ensure the effectiveness

and achievements of the project. All these efforts

make you organization capable of reporting

answering all inquiries, and being accountable for

itslans


11/30

Evaluation

Identify ways to improve or shift your project activities

Prepare project reports (e.g., mid-term reports, final reports) Inform internal and external stakeholders about the project Plan for the sustainability of the project

learn more about the environment in which the project is beingor has been carried out

learn more about the target population of the project

Present the worth and value of the project to stakeholders andthe public

Plan for other projects

Compare projects to plan for their futures Make evidence-based organizational decisions Demonstrate your organizations concerns to be accountable

for implementing its plans, pursuing its goals, and measuringits outcomes.


12/30

Evaluation Plan

Creating an evaluation plan is the very first

stage of conducting an evaluation. Your plan

should answer questions such as:

What is being evaluated? Why is it being evaluated?

Who wants the evaluation?

Who will do it? How will it be done?

What results do you expect?


13/30

Purpose of Evaluation

To assess the degree to which projectobjectives were achieved.

To document the lessons learned.

To provide recommendations for projectdevelopment and improvement.

To examine the changes that resulted fromdoing the project.

To provide input to guide decision makingfor the upcoming renewal and extension of

project funding.

Id if i E l i


14/30

Identifying Evaluationstakeholders Stakeholders are the individuals or organizations

that have an interest in your project; they maymake decisions, participate in the projectactivities, or be affected by those activities.

Your project may have both primary and

secondary stakeholders. Primary stakeholders are those who are closelyand directly involved in or affected by the resultsof your project (e.g.the participants themselvesand an organization that has invested in your

project). Secondary stakeholders are those who are less

involved and less affected by your project but mayhave some benefits in your project (e.g., an

organization that is interested in knowing aboutthe results of your project).


15/30

Evaluation Types

Selecting an evaluation type provides direction foryour evaluation. It helps keep the evaluationprocess focused on its main purpose anddetermines the evaluation questions that should beanswered and the data that should be collected.The most common types of evaluation are: Formative

Process Summative Outcome


16/30

Types of Evaluation

Formative Evaluation is an ongoingevaluation that starts early in a project. Itassesses the nature of the project, the needs

the project addresses, and the progress andimplementation of the project. It can identifymajor gaps in the projects content and

operational aspects (i.e., what was done andhow) and suggest ways to improve them.


17/30

Types of Evaluation

Process evaluation is used to monitor activities to makesure a project is being implemented and completed asdesigned and on time.It can be complementary to formative evaluation.

Although formative evaluation has a larger scope thanprocess

evaluation, there are many similarities between them: bothfocus on the effectiveness and the operational aspect of a

project; both start at a very early stage of a project and can beperformed by internal staff; and both require a strongmonitoring mechanism to track operational activities andto collect information related to the process


18/30

Types of EvaluationSummative Evaluation is an overall assessment of the projects

effectiveness and achievements. To provide adequate information, asummative evaluation requires a set of well-defined goals and objectives

for the project and a plan that keeps the focus of evaluation on the end-

results.

It reveals whether the project did what it was designed to do.

It provides information for future planning and decisions and usually iscompleted when the project is over.

This type of evaluation usually does not directly affect the current

project, but it helps stakeholders decide the future of this or similarprojects.

This type of evaluation should be conducted by either an externalevaluator or by staff and practitioners who are well equipped with therequired knowledge, resources, and tools for this type of evaluation

T f E l i


19/30

Types of EvaluationOutcome Evaluation assesses the extent to which a project has

achieved its

intended effects, and other effects it could have had on the projectsparticipants or the environment. It focuses on immediate, intermediate, orultimate outcomes resulting from the completion of the project. The

resultsof this evaluation should identify both the desirable and undesirable

impacts

of the project.

To conduct this type of evaluation, need a good understanding ofthe project process, the outcomes, and the relationship betweenthe two

Then, you need to collect sufficient evidence during the project todemonstrate how certain outcomes are related to a specific set ofproject activities

In this type of evaluation, identifying evaluation indicators,collecting appropriate data, and interpreting the results are crucialas well as challenging


20/30

How to select and EvaluationType The objectives and priorities of your project The purpose of the project evaluation

The nature of the project (i.e., whether it is

process-oriented or outcome-oriented) The time frame for conducting the

evaluation (i.e., during or after the project)

How, and by whom, the results will be used

The time frame and budget for completingthe evaluation.


21/30

Evaluation Questions

Evaluation questions are the key questions that youneed to answer to ensure the successful completion ofyour project or to understand its impact, effectiveness,and achievements.

Asking and answering the right questions will lead to

useful evaluation results that can be easilycommunicated with external audiences or put to use inyour organization.

Evaluation questions play a crucial role in the analysis

and interpretation of the data you collect. Therefore, you should allow for enough time and

discussion with your key stakeholders to develop andselect them.

You may also need to revisit and revise these


22/30

Evaluation Tools Evaluation tools help you gather the information you

need to answer your evaluation questions. They can be different from the tools you use to

carry out the core activities of the project. Forexample, conducting workshops to train

organizations is a project tool; interviewing keyparticipants to ask their opinions about the contentand success of the training workshops is anevaluation tool.

Evaluation tools can use both formal and informalmethods for gathering information.

Formal evaluation tools include focus groups,interviews and survey questionnaires.

Informal evaluation tools include observations,informal conversations, and site visits.


23/30

Quantitative and QualitativeInformation

Depending on your evaluationquestions, you may need a tool thathelps you gather quantitative information

by numbering, rating and rankinginformation.

Or you may need a tool that helps you

gather qualitative information such asstories, anecdotes, opinions about aparticular aspect of the project


24/30


25/30

Evaluation Indicators

Indicators are measurable factors or evidence thatshows the extent of the projects progress, success,or achievements. Identifying indicators can help youin collecting useful data and in your search forrequired evaluation tools and information sources.

Indicators can be quantitative, such as the number ofparticipants, number of website visits, andrate or rank of opinions.

They can also be qualitative, such as positive or

negative feedback, problems, complaints, andcomments. You can also use some project outputs as indicators

if they show the projects progress toward anobjective


26/30

Types of Data

Quantitative data describe a situation by using numbers. Thesedata could be the actual numbers; for example, the number ofevents your organization held, the number of services youprovided, the number of materials you produced.

Also, the quantitative data could be some numbers that

represent the relationships among concepts; for example, thenumbers that express the rates (e.g., in a six-point scale) orranks of opinions, feelings, skills, and knowledge.

You may collect this type of data when you use a surveyquestionnaire, an evaluation form, a test (e.g., knowledge test),or some outputs of your project.

Once you complete your analysis, the most commonquantitative findings you look at and use for interpretationscould be total numbers, percentages, frequencies, averages,ratios, ranks, and orders.


27/30

Types of Data

Qualitative data take the form of words,texts, ideas, and stories that explain asituation.

You typically collect this data when projectparticipants describe a situation in theirown words, whether you use formal orinformal conversations, interviews, or focus

groups.

You can also use project-relateddocuments such as notes about direct

observations, minutes of meetings, andournal entries


28/30

Evaluation Budget

Project evaluations can be costly, particularly ifthey aim to capture various aspects of both theprocess and outcomes of the project.

You should plan your budget in a way thatmakes your evaluation realistic, manageable,efficient, and productive.

In some cases, projects have a fixed budgetand evaluators need to adjust their activities to

that budget. In other cases, evaluators need to develop a

budget.


29/30


30/30

When to use Evaluation

Any time there is an unexpected result orperformance outlier that requires furtherinvestigation

When resource or budget allocations are beingmade across projects, programs, or policies

When a decision is being made whether or not toexpand a pilot

When there is a long period with no improvement,and the reasons for this are not clear

When similar programs or policies are reportingdivergent outcomes

lecture 11 pmec

Documents