Evaluation SeminarEvaluation SeminarCzech Republic CSF and OP Czech Republic CSF and OP

Managing AuthoritiesManaging Authorities

Session 3: Mid-Term Evaluation

Presentation Structure

• Organisation– Role of Planning Group and steering committees– Selection of Evaluators

• The Evaluation Process– Evaluation Issues– Methods

• Outcomes and Utilisation• Case Study

– Evaluation of EU horizontal priorities

Organisation

• Started preparation in early 2002• Planning Group established• Composition…. • Functions

– Agree timetable– Approved core TOR (June 2002)– Approve quality criteria– Acted as CSF steering committee

Steering Committees

• Evaluation Steering Committee established for each OP

• Composition– OP Managing Authority (MA)– Evaluation Unit, CSF MA and Commission – Implementing departments

• Functions– Adapt core TOR to OP situation– Selection of evaluators (see next slide)– Quality control function

Selection of Evaluators

• Each RFT advertised in EU Journal• Selection criteria

– TOR coverage and understanding– Methodology – Evaluation experience– Cost– Resources allocated

• Interviews held in some cases

Evaluation Process

• Key evaluation questions (see Session 1) were– Relevance

– Effectiveness

• Core analytical tasks specified in TOR– review of external developments

– progress to date (end 2002)

– Efficiency, project selection, indicators

– Performance reserve

– Horizontal priorities (see later)

Methods

• Analysis of financial and physical performance monitoring data

• Review of policy papers • Consultations

– Managing authorities– Implementing ministries and agencies– Other stakeholders

• More sophisticated approaches used at CSF level– Macro and labour market modelling

Outcomes and Utilisation

• Process worked reasonably well– Planning Group played useful coordination role– All reports completed on time– Met TOR and of acceptable quality

• Utilisation– Reports considered by MCs and submitted to

Commission – Influenced decisions on financial reallocations and

performance reserve– But other considerations (N+2) taken into account

Case Study: EU Horizontal Priorities

• 2 horizontal priorities/principles– Equal opportunity (gender)– Environment

• Apply to all measures • Evaluated in OP reports under several headings

– Reporting quality– Indicator coverage – Integration to project selection systems

• Sample of measures for more in-depth analysis • CSF evaluation: synthesis based on OP inputs

–

EU Horizontal Priorities• Conclusions

– major problems with integration of principles– principles poorly understood– little evidence of implementation influence– absence of indicators and poor quality reporting

• Recommendations – More focused approach going forward – Concentrate on small number of relevant measures

• Outcome– Recommendation accepted– Follow-up work carried out by Evaluation Unit– Future effort to concentrate on identified relevant measures

•