1 ex-ante evaluations 2014-2020 of esf operational programmes budapest 26 th september 2013 kamil...
TRANSCRIPT
1
Ex-ante evaluations 2014-2020 ofESF operational programmes
Budapest26th September 2013
Kamil Valica
Unit A.3 Impact Assessment and EvaluationDG Employment, Social Affairs and Inclusion
2
Intervention logic
Indicators
Ex-ante evaluation
(EAE)
On-going evaluation
Ex-post evaluationImp
lem
en
tati
on
Mon
itori
ng
Result Orientation- important factors
Programme design
Interactive process
Iterative process
Ex-ante evaluation – a few major points
• Intervention logic
• Indicators
• Timing
Clear intervention logic (1)
• Challenges and needs
• Country-specific recommendations• National Reform Programme• European semester analyses
• Consistent translation into thematic objectives and investment priorities
Funding priorities
Linkage to Europe 2020 Strategy
Clear intervention logic (2)
• Specific objectives (SO)• Precise definition (of change)
- Should be more specific than the investment priority about target groups, problem area or structure, procedures, institutions, etc. it seeks to change
- Should logically link to the proposed actions
• Measurement (result indicators)
• Types of actions
• The most appropriate interventions to achieve SO (the right "action mix"?)
• The choice based on an analysis of problems
• Measurement of their outputs (output indicators)
Clear intervention logic (3)
• Logical framework / Theory of change
• Clarifies the intervention logic
• Demonstrates causal links between:
Types of actions -> outputs -> intended results / specific objective
Description of how planned actions will contribute to specific objectives is often lacking in first OP drafts reviewed.
• Helps identifying for which results/outputs, not captured by common indicators, programme-specific indicators should be established
Result indicators
Output indicators
Relevant and clear indicators
Ex-ante evaluators should consult carefully draft guidance on ESF Monitoring and Evaluation 2014-2020 and assess:• Relevance
• Result indicators should capture a change in the situation of supported participants or entities
• Clarity• Indicators should have a clear title and an
unequivocal and easy to understand definition
Indicators with realistic targets and identified data sources
Ex-ante evaluators should also assess:• Target values – are they realistic?
• Consult background papers for ESF target setting http://ec.europa.eu/esf/main.jsp?catId=67&langId=en&newsId=8174
• Data collection• Will data be collected and reported on time?• Have necessary arrangements been put in place
to collect and store micro-data of participants?• Are existing databases used as data sources?(NB: reduction of admin. burden for beneficiaries)• Are procedures in place to ensure quality of data?
Good timing is necessary to allow for interactive and iterative process
• To allow for an interactive process, contractors need to be found on time to accompany the design of programme from its first draft
• To allow for an iterative process (work in stages, revisiting of earlier stages on basis of evaluation recommendations), also programme design should start on time to allow for sufficient time space to evaluate and take on board recommendations
Ex-ante evaluations of ESF programmes – state of play
Based on a questionnaire of 20 MS (AT, BE, BG, CY, CZ, DK, EE, EL, ES, FI, HU, IR, LV, MT, NL, PL, PT, RO, SI, UK) – information from DG EMPL country geodesks received by 24/09/2013
Is ex-ante evaluation running?YES 11 MS Some EAEs started already in 2012 (4 MS)NO 6 MS 1) Contractors found, but EAEs have not started yet
2) Some regions preparing procurement, no information on other regions3) EAEs were planned to start in 1st half of Sept 2013, but not sure if they have started4) Winning bidder selected, but not yet awarded5) Invitation for tender published6) EAE awarded, but is expected to be initiated in Nov 2013 (after finalisation of draft PA)
YES/NO 3 MS 1) For 1 OP running, for 2 OPs expected start in Dec 20132) For the majority of OPs running, for 1 OP even finalised; for some regional OPs procurement launched in July/Aug 20133) Not running for national OP, procurement launched (some regional OPs) or contractors found and EAE running (other regional OPs)
Ex-ante evaluations of ESF programmes – other observations
Based on a questionnaire of 20 MS (AT, BE, BG, CY, CZ, DK, EL, EE, ES, FI, HU, IR, LV, MT, NL, PL, PT, RO, SI, UK) – information from DG EMPL country geodesks received by 24/09/2013
• In general, the EAEs carried out in an interactive way
• Challenges:• Time constraints
- Waiting for elaboration of other strategies• Parallel process of programming and EAE
- Frequently changing OP drafts make preparation of EAE complicated and difficult to keep relevance of EAE interim reports; nevertheless valid EAE recommendations taken on board.
• Delay in public procurement