design an “evaluable project” - cosvdesign an “evaluable project” ... 9defines project logic...
TRANSCRIPT
In collaboration with
Development of integrated project and financial
management tools in conformity with the 2003 FPA
Design an “evaluable project”
ECHO and the evaluability
ECHO and the ECHO and the evaluabilityevaluability
CarryingCarrying out of the mission in out of the mission in coco--ordinatedordinated way way bybyECHO and the NGOECHO and the NGOThe The CommissionCommission shallshall makemake the the draftdraft of the report of the report availableavailable to the NGO to the NGO forfor eventualeventual commentscomments
IfIf agreedagreed on, the on, the commentscomments of the NGO of the NGO shallshall bebeinsertedinserted in the Final Reportin the Final Report
Include Include evaluationevaluation costcost in budgetin budget
InsertInsert genericgeneric ToRToR in in Single FormSingle Form
PresentPresent specificspecific ToRToR to ECHO to ECHO beforebefore the mission the mission withwith Curriculum VitaeCurriculum Vitae of of evaluatorsevaluators
The The Commission shall carry Commission shall carry out out evaluations evaluations of the of the humanitarian operations financedhumanitarian operations financed (art. 22.1. (art. 22.1.
ProvisionsProvisions; ; art. 8 CG)art. 8 CG)
The The NGOs must make NGOs must make ex ante/ex post ex ante/ex post evaluations evaluations wherewhere possiblepossible (art. 22.2 (art. 22.2 ProvisionsProvisions))
PresentPresent EvaluationEvaluation Report Report withwith Final ReportFinal Report
Keep Keep a a constant monitoringconstant monitoring
ECHO and the ECHO and the evaluationevaluation
Control of Control of results based results based on the promotion on the promotion of best of best practicepractice
Application Application of HOLIS program of HOLIS program –– Standard Standard Operative Operative ProceduresProcedures
Possibility Possibility of of ““dede--selectionselection”” of underof under--performing partnersperforming partners ((monitoringmonitoring & & evaluationevaluation))
Introduces Introduces the the principle principle of of responsability towards responsability towards beneficiarybeneficiary, , donors donors and and partnerspartners
Introduces Introduces the the principle principle of of measurability measurability and and monitorabilitymonitorability
Obligatory insertion Obligatory insertion PCM, LF and PCM, LF and indicators indicators (project (project formsforms))
Allocation of Allocation of aid aid in a in a balancedbalanced and and verifiableverifiable way way for beneficiaryfor beneficiary
ACCOUNTABILITY ACCOUNTABILITY ((FromFrom ECHO ECHO AidAid StrategyStrategy 20032003))
Allocation of Allocation of aid aid in in balanced balanced way way for beneficiaryfor beneficiary
ConceptualConceptual work on a work on a methodologymethodology to allocate to allocate aidaid in a more in a more balancedbalanced way per way per beneficiary has progressedbeneficiary has progressed in 2002, in 2002, highlightinghighlighting, , howeverhowever, , problems withproblems with data data qualityquality. Data . Data availabilityavailability and and quality will therefore be addressed asquality will therefore be addressed as a a prioritypriority in the in the 2003 HOLIS work 2003 HOLIS work programmeprogramme..
Control of Control of results using results using ““best best practicepractice”” approachapproach
AdoptionAdoption of of measures focusingmeasures focusing on control of on control of results usingresults using a a ““best best practicepractice”” approachapproach. . CapacityCapacity building through the building through the disseminationdissemination of best of best practicespractices..
Application Application of HOLIS program of HOLIS program –– Standard Operative Standard Operative ProceduresProcedures
The establishment of final Standard Operating The establishment of final Standard Operating Procedures for crisis responseProcedures for crisis response in in humanitarian emergencies will also contributehumanitarian emergencies will also contribute to ECHOto ECHO´́s s qualityquality--based approachbased approachand and itsits operationaloperational management management capacitycapacity..
Obligatory insertion Obligatory insertion LF and LF and indicatorsindicators
In In orderorder to facilitate the to facilitate the monitoringmonitoring of of resultsresults, the , the objectivesobjectives of of an operationan operation and and the performance the performance indicators applicableindicators applicable to to measure achievements will become measure achievements will become constituent partsconstituent parts of the new project of the new project proposalproposal standard standard formsforms..
Possibility Possibility of Partner of Partner ““dede--selectionselection””
The The conceptconcept of of qualityquality partnership partnership also involvesalso involves a regular a regular monitoringmonitoring of of performance, performance, withwith the concrete the concrete possibilitypossibility of of ““dede--selectionselection”” of underof under--performingperformingpartnerspartners..
ACCOUNTABILITY ACCOUNTABILITY ((FromFrom ECHO ECHO AidAid StrategyStrategy 20032003))
Project Project presentationpresentation: : mandatory mandatory LF of LF of maximum maximum 3 3 pagespages
Final Report: the LF Final Report: the LF is adapted is adapted in light of the in light of the completed operationcompleted operation
ObligationObligation to to draw draw up a Log up a Log frameframe
Interim ReportInterim Report: the LF : the LF is adapted is adapted in light of in light of what what has been donehas been done
The partner The partner shall guarantee an internal monitoring shall guarantee an internal monitoring of the project, of the project, informing informing ECHO through the ECHO through the Interim Interim ReportReportThe base of the The base of the monitoring monitoring system system is given by is given by the the Work PlanWork Plan
Evaluation Evaluation
Monitoring Monitoring
AccountabilityAccountability of ECHO on money of ECHO on money spentspentLessonsLessons learnedlearned
ACCOUNTABILITY ACCOUNTABILITY ((FromFrom ECHO ECHO ManualManual Project Project CycleCycle
Management 2003Management 2003))
The “evaluable”
quality
The The ““evaluableevaluable””
qualityquality
Key pointsKey points expressedexpressed byby ECHO in ECHO in ““Qualitative Qualitative ApproachApproach””
CLEARNESS
Precise identification of the beneficiaries and the objective
Identify needs, problems, priorities and risks
Detailed analysis – elements defined (better if supported by Report)
Expected results in relation to defined indicators (SMART)
Realistic planning – efficient expense organization
TRUSTWORTHINESS
The Project(the Operation)
RESPONSABILITY
Continuous monitoring, ability to adapt, no over budgeting
Transparency of procedure, correctness of work
Attention to priority themes (Gender, Human Rights)
Quality control
CROSSOVER
Visibility and General Protocols and safety details
Project phases
Identification Feasability … Evaluation Passages project Logic
PCM PCM phasesphases withwith emphasisemphasis on qualitative on qualitative aspectsaspects of the of the constructionconstruction of the of the proposalproposal::
•• ProblemProblem analysis, single out analysis, single out objectivesobjectives//strategiesstrategies
•• Work plan, Work plan, monitoringmonitoring
•• EvaluationEvaluation, , best best practicespractices
Approval Comunication … LiquidationPassages ECHO procedures
PhasesPhases relative to project iter relative to project iter foreseenforeseen byby GC and GC and documentsdocuments::
•• Responsability Responsability and and maturity towardsmaturity towards ECHOECHO
•• Procedure, Procedure, statements statements and and mandatory documentsmandatory documents
Real definition phases and project management:
• Predisposition of statements
• Reference system (sinergies)
• Supervision e monitoring
Preparation Start … ClosurePassages Internal Projects
The project The project cyclecycle shallshall keepkeep in account the in account the variousvariousneedsneeds accordingaccording to to pointspoints of of viewview whichwhich mustmust bebe
consideredconsidered::
RespectRespect the the orthodoxyorthodoxy of the projectof the project
RespectRespect the ECHO the ECHO proceduresprocedures
RespectRespect internalinternal HO HO proceduresprocedures
FocalFocal pointspoints
FocalFocal pointspoints in in practicepracticeThe project The project
implicatesimplicates
From the proposal to the end focus on method
Qualitative approach
Demonstrate know how and knowledge of procedures
Evaluationand Monitoring
Plan the work from the initialphase
Preventive approach
The basic idea The basic idea isis……
Qualitative approach
Evaluationand monitoring
Preventiveapproach
The The ““perfectperfect””ProjectProjectperh
aps
Quality of Reports
QualityQuality of of ReportsReports
Qualitative Qualitative approachapproachbasic basic elementselements
The The ““FiveFivepointspoints””
RelevanceRelevance
EffectivenessEffectiveness
EfficiencyEfficiency
Impact Impact
SustainabilitySustainability
Qualitative Qualitative approachapproachbasic basic elementselements
StakeholderStakeholderAnalysisAnalysis
PotentialPotential of of beneficiariesbeneficiaries ((numbernumber, , categorycategory, , typestypes), ), institutionsinstitutions, , locallocalgroupsgroups, , etcetc……
Limits (Limits (obstaclesobstacles))
InterestsInterests
PossiblePossible rolerole of stakeholders in the of stakeholders in the projectproject
StakeholdersStakeholders Analysis Analysis tooltool
Qualitative Qualitative approachapproachbasic basic elementselements
ProblemProblem and and needsneeds analysisanalysis
IdentifyIdentify generalgeneral problemsproblems
IdentifyIdentify particularparticular problemsproblems and and needsneeds
IdentifyIdentify needsneeds of of beneficiariesbeneficiaries ((byby categoriescategories))
CorrelationCorrelation and and articulationarticulation of of mainmain elementselements((logiclogic of analysis)of analysis)
ProblemProblem TreeTree
Macro contest Macro contest shownshownbyby ECHO/CEECHO/CE
InterventionInterventionfocusfocus
Qualitative Qualitative approachapproachbasic basic elementselements
Analysis of Analysis of objectivesobjectives
DefineDefine ((strategicstrategic) ) generalgeneral objectivesobjectives ((connectingconnectingstrategystrategy EU and HO)EU and HO)
IdentifyIdentify quantitative and qualitative quantitative and qualitative beneficiariesbeneficiaries
IdentifyIdentify objectiveobjective of the of the opertionopertion ((servicesservices givengiven to to target group)target group)
DefineDefine observtionobservtion criteriacriteria ((typetype of of needneed) and ) and selectionselection((inclusioninclusion--exclusionexclusion))
ObjectiveObjective TreeTree
Qualitative Qualitative approachapproachbasic basic elementselementsAnalysis of Analysis of
resultsresults and and activitiesactivities
DefineDefine generalgeneral frameworkframework of of servicesservices ((expectedexpectedresultsresults in line in line withwith budget)budget)
IdentifyIdentify activitiesactivities ((descriptiondescription, management, , management, humanhuman resourcesresources, , timestimes, , detaileddetailed descriptiondescription of of costscosts))
General General frameworkframework of of mainmain pointspoints relative to relative to implementationimplementation ((purchasepurchase/tender, /tender, safetysafety, , visibilityvisibility, , monitoringmonitoring plan)plan)
Qualitative Qualitative approachapproachbasic basic elementselementsAnalysis of Analysis of risksrisks
and and assumptionsassumptions
IdentifyIdentify ((evaluateevaluate) ) risksrisks inherentinherent to the to the actionaction
IdentifyIdentify ((evaluateevaluate) ) assumptionsassumptions
IdentifyIdentify mitigationmitigation meausuresmeausures
Intervention Logic
Objectively Verifiable Indicators
Sources of Verification
Risks and Assumptions
Principal Objective
Specific Objective
Results
Activities
Means
Costs
Pre-conditions
ReferRefer to the EC/ECHO to the EC/ECHO strategicstrategic frameworkframework
Log Log FrameFrame
ConsiderConsider costscosts and and organizationorganization of a of a monitoringmonitoring planplan
ReferRefer to the budget to the budget lineslines
CannotCannot bebe modifiedmodified, , referreferto ECHO decisionto ECHO decision
Imp!
Imp!
Imp!
Imp!
WorkflowWorkflow
Qualitative Qualitative approachapproachbasic basic elementselements
QualityQualityfactorsfactors
OwnershipOwnership byby beneficiariesbeneficiaries
Policy Policy supportsupport (positive/negative)(positive/negative)
Appropriate Appropriate technologytechnology
SocioSocio--cultural cultural issuesissues
EqualityEquality of of allall ((especiallyespecially the the mostmost vulnerablevulnerable group)group)
InstitutionalInstitutional and management and management capacitycapacity
CoCo--ordinationordination betweenbetween the the differentdifferent actorsactors
Qualitative Qualitative approachapproachbasic basic elementselements
Cross cutting Cross cutting issuesissues
LRRD LRRD ((towardstowards self self sufficiencysufficiency))
Gender (Gender (rolesroles, impact of project, , impact of project, womenwomen’’s s rolerole asas developmentdevelopment factorfactor))
HumanHuman RightsRights (impact on HR)(impact on HR)
SafetySafety
VisibilityVisibility
Qualitative Qualitative approachapproachbasic basic elementselementsIndicatorsIndicators and and
sourcessources of of verificationverification
ClearClear definitiondefinition
((quantityquantity, , relevancerelevance, , thresholdsthresholds, , minimum minimum standardsstandards))
FeasabilityFeasability of controlof control
RelevanceRelevance of project of project variablevariableobservedobserved
ReferenceReference DocumentsDocuments
Define and plan steps;
General indication of operative formalities and more critical points (personnel, tender, means, safety);
Experience, ideas, strategies.
Indication on UE/ECHO strategies on regional, national and local levels.
PCM synthesis (with attention to preliminaryanalysis and log frame construction);
Terms and synthesis of criteria for defining, monitoring and evaluating the projects.
PCM ECHO and PCM ECHO and Solint Solint manualmanual
Data Base and HO Data Base and HO praxispraxis
Country Country StrategyStrategyPaperPaper ECHO/UEECHO/UE
Defines procedures, times, and focal elements of work;
Elegible expences, purchases, visibility, responsability.
Indicates main variables for the defining, monitoring and evaluation of the activity(performance e benchmark);
Formation of log frame.
FPA FPA documentsdocuments
Sphere Sphere ManualManual
InstrumentsInstruments and and methodsmethods
Collects preliminary data;
Meetings with key actors;
Meetings (Focus Group) with privileged actors andstakeholders analysis.
RapidRapidsurveysurvey
Clear identification of beneficiaries and project components;Clear identification of critical points;Coherent base of project logic (attention to objectives, not activity)
Lists minimum elements for defining the proposal.
ProblemProblemTreeTree
ChekChek ListList
Defines project logic and analysis (deviation fromplan, hiring, indicators);
List key elements (objectives, results activity and indicators).
Log Log frameframe
Realistic analysis of times;
Initial Assessment and verification of responsabilityand focal moments of intervention.
WorkWork--flowflow((activityactivity plan)plan)
The end
The endThe end