performance indicators for convergence in regional · performance indicators for convergence in...

Performance Indicators for Convergence in Regional

Development - How reliable are they to ensure Targeted and Result-Oriented spending?

Budgetary Affairs

Policy Department D for Budgetary Affairs Directorate General for Internal Policies of the Union

PE 621.807 - April 2019

STUDY requested by the CONT Committee

EN

DIRECTORATE GENERAL FOR INTERNAL POLICIES

POLICY DEPARTMENT D: BUDGETARY AFFAIRS

Performance Indicators for Convergence in RegionalDevelopment - How Reliable are they to Ensure

Targeted and Result-Oriented Spending?

STUDY

Abstract

This study examines how reliable Cohesion Policy indicators are in helping to ensuretargeted and result-oriented expenditure. Overall, substantial progress has been made inrecent years in developing an effective system. However, the challenge remains to shiftfrom a focus on financial absorption and outputs to assessing more meaningful results andimpacts of interventions, and to improve data collection systems. The paper includeseleven recommendations to address these and other challenges ahead of the new 2021–2027 period.

IP/D/ALL/FWC/2015-001/LOT2/C4 17/04/2019

PE 621.807 EN

This document was requested by the European Parliament's Committee on Budgetary Control. Itdesignated Mrs Claudia Schmidt to follow the study.

AUTHOR(S)

Jack Malan, Jayne Woolford, Sandra Ruiz-Fernandez, Michaela Brady, and Maria Kazakova from theCentre for Strategy & Evaluation Services LLP (CSES)

RESPONSIBLE ADMINISTRATOR

Mr Kaare Barslev (Seconded National Expert)

Policy Department on Budgetary Affairs

European Parliament

B-1047 Brussels

E-mail: [email protected]

LINGUISTIC VERSIONS

Original: EN

Translation of the Executive Summary: DE, FR

ABOUT THE EDITOR

Policy Departments provide in-house and external expertise to support European Parliament'scommittees and other parliamentary bodies in shaping legislation and exercising democratic scrutinyover EU policies.

To contact the Policy Department or to subscribe to its newsletter please write to:

Mr Niels FISCHER

E-mail: [email protected]

Manuscript completed in April 2019.

Brussels, © European Union, 2019.

This document is available on the Internet at:

http://www.europarl.europa.eu/supporting-analyses

DISCLAIMER

The opinions expressed in this document are the sole responsibility of the author and do notnecessarily represent the official position of the European Parliament.

Reproduction and translation for non-commercial purposes are authorised, provided the source isacknowledged and the publisher is given prior notice and sent a copy.

Performance Indicators for Convergence in Regional Development -How Reliable are they to Ensure Targeted and Result-Oriented Spending?

_

3

CONTENTS

CONTENTS 3

EXECUTIVE SUMMARY 9

SYNTHÈSE 20

ZUSAMMENFASSUNG 34

1 INTRODUCTION 48

1.1 Resume - study objectives 48

1.2 Overview of the research paper 49

2 ESIF PERFORMANCE MONITORING 50

2.1 Context and key issues 50

2.1.1 EU Cohesion Policy 502.1.2 Performance Framework 512.2 Evolution of ESIF monitoring and evaluation requirements 52

2.2.1 2000-2006 programming period 542.2.2 2007-2013 programming period 552.2.3 2014-2020 Programming Period 592.2.4 Performance Framework and Performance Reserve 602.2.5 ERDF and ESF Indicators 2014-2020 612.3 Conditionalities and the performance reserve 66

2.3.1 Ex ante conditionality 662.3.2 Performance Reserve 672.4 Data collection and monitoring 69

2.5 Proposals for the 2021-2027 period 70

3 ASSESSMENT OF KEY ISSUES 75

3.1 Use of common ERDF and ESF indicators in 2014-20 75

3.2 ERDF common indicators 78

3.3 ESF common indicators 80

3.4 Cohesion Policy conditionalities 83

3.5 Performance Framework 85

3.6 Performance reserve 88

3.7 Data collection 90

4 CONCLUSIONS & RECOMMENDATIONS 91

4.1 Evolution of the ESIF performance monitoring approach 91

4.2 Use of common and programme-specific indicators 93

4.3 ESIF monitoring systems 96

4.4 Performance reserve and contribution to wider EU policy goals 98

4.5 Regulatory Framework for 2021-2027 99

REFERENCES 101

ANNEX A: LIST OF INTERVIEWEES 104

DG IPOL | Policy Department D: Budgetary Affairs

4

ANNEX B: EUROPEAN SOCIAL FUND COMMON INDICATORS 106

ANNEX C: ERDF AND COHESION FUND COMMON INDICATORS 2014-2020 108

ANNEX D: ANALYSIS OF THE USE OF COMMON INDICATORS 2014-2020 111

ANNEX E: COMMON OUTPUT AND RESULT INDICATORS FOR THE ERDF AND THE CF 2021-2027

114


_

5

LIST OF TABLESTable 2.1: Summary – ERDF Key performance indicators ........................................................................................62

Table 2.2: Summary – ESF Common Indicators ...........................................................................................................63

Table 3.1: Use of ERDF and ESF common indicators in the 2014-2020 Programming Period....................75

Table 3.2: Average number of core/common indicators per OP (2007-2013 are those with target values

included only) ....................................................................................................................................................77

LIST OF FIGURESFigure 1.1: Evolution of the ESIF Regulatory Framework (2000-2027) ................................................................11

Figure 2.1: Overview – ESIF Results Orientation..........................................................................................................52

Figure 2.2: Evolution of the ESIF Regulatory Framework (2000-2027) ................................................................53

Figure 4.1: Evolution of the ESIF Regulatory Framework (2000-2027) ................................................................92

LIST OF BOXESBox 1.1: ‘Elements of interest’ ............................................................................................................................................48

Box 2.1: Key Features of the Results Orientation.........................................................................................................51

Box 2.2: Performance Indicators........................................................................................................................................51

Box 2.3: Use of ERDF Indicators 2007-2013 ...................................................................................................................56

Box 2.4: Examples of impact indicators 2007-2013....................................................................................................56

Box 2.5: Weaknesses in the Quality and Reliability of the ERDF data 2007-2013 ............................................57

Box 2.6: Case study (Spanish ERDF 2014-2020 Output Indicators Manual).......................................................60

Box 2.7: Use of Common and Specific Indicators........................................................................................................64

Box 2.8: Results Indicators ...................................................................................................................................................65

Box 2.9: Cohesion Policy Conditionalities......................................................................................................................66

Box 2.10: Performance Reserve .........................................................................................................................................68

Box 2.11: Data Collection Systems....................................................................................................................................69

Box 2.12: Performance Framework and Monitoring and Evaluation in the Common Provisions

Regulation for 2021-2027 .................................................................................................................................70

Box 2.13: Vision Post-2020...................................................................................................................................................73

Box 3.1: Most Used ERDF Common Indicators 2014-2020.......................................................................................77

Box 3.2: Most Used ESF Common Indicators 2014-2020 ..........................................................................................77

Box 3.3: Summary of Key Points - 2018 study on post-2020 ERDF and CF indicators...................................78

Box 3.4: Spain - Examples of use of programme specific indicators instead of common indicators .......79

Box 3.5: Analysis of ESF Annual Implementation Reports’ Shortcomings .........................................................81

Box 3.6: Key Research Findings on ESF ...........................................................................................................................83


6

Box 3.7: European Commission findings – Role of Ex ante Conditionalities in Promoting Results

Orientated Interventions...................................................................................................................................83

Box 3.8: Key evaluation findings on Performance Frameworks (2016)..............................................................85

Box 3.9: Project Selection Procedures and Monitoring Data ..................................................................................87

Box 3.10: Issues reported in relation to the 2014-2020 Performance Framework ..........................................88

Box 3.11: Italy – Assessing Performance.........................................................................................................................89

Box 4.1: Issues Raised in the Research Paper that are addressed in the Common Provisions Regulation

for 2021-2027 ........................................................................................................................................................99

Box 4.2: Issues Raised in the Research Paper that are not fully addressed in the Common Provisions

Regulation for 2021-2027 ................................................................................................................................................. 100


_

7

LIST OF ABBREVIATIONS

AA Audit Authorities

AIR Annual Implementation Report

CA Certifying Authority

CP Cohesion Policy

CSF Common Strategic Framework

EAC Ex-ante conditionality

EC European Commission

ECA European Court of Auditors

EP European Parliament

ERDF European Regional Development Fund

ESF European Social Fund

ESIF European Structural and Investment Funds

ETC European Territorial Cooperation

EU European Union

IB Intermediate Bodies

ICT Information and communication technology

IT Information Technology

MA Managing Authorities

MCS Management and control system

MFF Multiannual Financial Framework

MS Member State

OPs Operational Programmes

PACs Post Approval Commitments

RCE Regional Competitiveness and Employment

SMEs Small of Medium-sized Enterprise

TFEU Treaty on the Functioning of the European Union

YEI Youth Employment Initiative


8

COUNTRY ABBREVIATIONS

BE Belgium

BG Bulgaria

CZ Czechia

DK Denmark

DE Germany

EE Estonia

IE Ireland

EL Greece

ES Spain

FR France

HR Croatia

IT Italy

CY Cyprus

LV Latvia

LT Lithuania

LU Luxembourg

HU Hungary

MT Malta

NL Netherlands

AT Austria

PL Poland

PT Portugal

RO Romania

SI Slovenia

SK Slovakia

FI Finland

SE Sweden

UK United Kingdom


9

EXECUTIVE SUMMARY

OVERVIEW

During the current 2014-2020 programming period, some EUR 352 billion has been allocated

from the European Regional Development Fund (ERDF) and the European Social Fund (ESF) to

the EU’s Cohesion Policy. It is clearly important that interventions are monitored and evaluated in the

interests of accountability but also to help ensure the most efficient and effective deployment of EU

funding.

The specific objectives of this research paper for the EP was to investigate how reliable the key

performance indicators for convergence in regional development are in helping to ensure

targeted and result-oriented spending. To this end, the Research Paper seeks to analyse the present

system of common indicators within the EU’s Cohesion Policy, including their strengths, weaknesses

and other relevant factors. Looking ahead, the Research Paper makes suggestions on possible changes

to the current policies in order to improve the overall results of the EU’s Cohesion Policy. The

assignment was carried out in early 2019 for the EP’s Directorate D (Budgetary Affairs), Directorate-

General for Internal Policies of the Union, by the Centre for Strategy & Evaluation Services (CSES)1.

Overall, this Research Paper suggests that good progress has been made to develop an effective

system of performance indicators to monitor the contribution of Cohesion Policy to convergence

in regional development. There is now a well-designed set of output indicators in place for both the

ERDF and the ESF. However, the challenge remains to shift the focus more from measuring financial

inputs and outputs to being able to assess the more meaningful results and impacts of interventions.

There is also a lot still to be done to develop data collection systems that are capable of generating the

type of information required to assess results and impacts without imposing an unreasonable

administrative burden on national authorities and Cohesion Policy beneficiaries.

KEY CONCLUSIONS AND RECOMMENDATIONS

We first summarise key features of the framework for performance measurement for the ERDF and ESF,

and then assess how effective this has been in ensuring results-orientated interventions.

FRAMEWORK FOR ERDF AND ESF PERFORMANCE MEASUREMENT

The Common Provisions Regulation 1303/2013 for Cohesion Policy in the 2014-2020 programming

period introduced significant changes intended to increase focus on performance and promote a more

result-orientated approach, consisting of these main elements:

1 Consortium Partner Blomeyer & Sanz ensured quality control.


10

Key Features of the Results Orientation

Ex ante conditionalities – these have been negotiated to help ensure that the necessary

conditions are in place for a focus on key performance indicators and successful programme

delivery;

Intervention rationale and logic frameworks – ensuring the result orientation of

programmes through linking the needs to be addressed to the results to be achieved and the

inclusion of result indicators with definitions and measurable targets;

Performance Framework – the establishment of clear and measurable milestones and targets

to ensure progress is made as planned (performance framework) with a Performance Reserve

used to reward achievement of results.

This Research Paper is primarily concerned with the Performance Framework, in particular the common

and programme-specific indicators that are used to help ensure targeted and result-oriented

interventions and measure the effect and results of the policy on the ground. The research suggests

that with some further refinement, the system that now exists is capable of ensuring a targeted

and results-orientated approach to implementing ERDF and ESF programmes.

In implementing the Performance Framework, the Commission and Member States should work

together to define baselines (e.g. the situation in a region with regard to skills or in relation to business

start-ups) and targets (e.g. how ERDF and ESF interventions might improve the situation) with

movement against indicators then being used to assess progress in implementation.

The European Commission’s ‘Better Regulation’ guidelines (2015 and 2017) define three basic types

of indicators: output indicators which measure the specific deliverables of the interventions (e.g.

number of SMEs receiving ERDF grants, number of ESF training participants); results indicators, i.e. the

immediate effects of an intervention with the reference to the beneficiaries (e.g. improved

performance of SMEs, ESF participants’ qualifications); and impact indicators which relate to the

effects on the wider economy and society (e.g. job creation, improved competitiveness). Whereas

outputs are physical in nature and relatively easy to measure, results and impacts are socio-economic

and much more difficult to assess. Likewise, whereas the ESF interventions focus on individuals, the

ERDF has a more diverse target group with less homogeneous units of measurement.

Importantly, an effective Performance Framework depends not only on appropriate indicators but also

having the data collection mechanisms embedded in programme management systems that can

generate the required information on interventions and their effects.

EVOLUTION OF THE PERFORMANCE FRAMEWORK

There have been significant improvements to the ESIF monitoring and evaluation approach

across each programming period. The key developments in the ESIF regulatory framework that have

taken place over the years are summarised below.


11

Figure 0.1: Evolution of the ESIF Regulatory Framework (2000-2027)


12

In summary, before 2000, there was no common approach to defining indicators and targets with

Member States able to choose their own, leading to a wide range of indicators being used across ERDF

and ESF programmes with very limited harmonisation. Not only were different indicators used by

different Member States and regions but even where the same indicators were used, definitions varied

as did the methodologies used to implement them. As a consequence, it was difficult if not impossible

to aggregate information at EU level on the results of the policy.

USE OF COMMON AND PROGRAMME-SPECIFIC INDICATORS

The current 2014-2020 mainstream ERDF and ESF programmes are being monitored against a

set of 72 common output and results indicators. At the same time, Managing Authorities are

making use of a large number of programme-specific indicators. This reflects the fact that whilst

the common indicators should be mostly relevant to all Member States and regions, there may be a

need for others that more closely reflect regional specificities (e.g. there are instances where indicators

are not relevant - km of rail constructed is not relevant in Member States with no railway network such

as Cyprus and Malta). A notable trend over the years has been the effort to promote simplification with

the number of different indicators being progressively consolidated.

The ESF currently has 32 common indicators (23 for outputs and 9 for results) that represent the

minimum that should be included in each Operational Programme (OP). The most frequently-

selected indicators refer to the number of employed participants in ESF-funded programmes, followed

by the number of unemployed, disadvantaged individuals and young people. Taken together, a total

of 16,895 output and 3,556 result indicators are being used in the 180 ESF OPs during the current

programming period (it should be noted that some programmes are multi-fund and are covered under

both the ESF and ERDF figures). Whilst only some 15% of indicators used in ESF OPs are common

indicators, programme-specific indicators for the most part reflect sub-categories or combinations of

categories used within the common indicators. In addition to the output indicators, the ESF has defined

common indicators for ‘immediate’ and ‘longer-term’ results.

In the case of the mainstream ERDF programmes (i.e. excluding European Territorial

Cooperation (ETC)), the OPs currently have a total of 40 common output indicators for the

current 2014-2020 period. These have been used 5,757 times in the 201 OPs and account for roughly

55% of all indicators used and cover roughly the same amount of the investment. The most frequently-

used common indicators relate to the number of enterprises receiving support and the value of grants,

the employment increase in supported enterprises, and the reduction of greenhouse gas emissions. In

some cases, where OPs have not used common indicators they have been encouraged to use EU level

indicators (e.g. those derived from the innovation scoreboard) in order to improve measurability and

comparability. ERDF programme-specific indicators cannot usually be aggregated and have arguably

been over-utilised within the OPs’ performance frameworks. Unlike the ESF, the ERDF has not

developed a common set of results indicators and those that exist are programme-specific.


13

Overall, the use of obligatory common indicators for outputs (and also results for the ESF) based

on standardised definitions has contributed to strengthening programme intervention logic i.e.

how the planned spending contributes to achieving change in the context of the identified

development needs - and forms part of the programme negotiation process. The transparency,

visibility and accountability of the policy have all improved. Furthermore, defining clear and

measurable indicators and targets as part of the intervention logic at the outset of a programme

encourages an improved focus by Managing Authorities and regional / national stakeholders on results

and progress in performance.

The current set of ERDF and ESF common indicators is generally well-designed and appropriately

targeted in relation to the goals of the Cohesion policy but there are some deficiencies that need

addressing. Whilst ESF programme-specific indicators tend to reflect sub-categories or combinations

of categories used within the common indicators, those utilised by ERDF programmes are extensive in

number and cannot usually be aggregated. The system entails a considerable administrative burden

and cost for sometimes low levels of quality and accuracy. However, simplification and harmonisation,

whilst welcome in terms of reducing administrative burden and increasing accountability, should not

lead to inflexibility in programming or a one-size-fits-all approach.

Recommendation 1: To the extent possible, the process of simplifying the ESIF common

indicator system should continue. There is a need to reflect the principle of proportionality and

ensure a more streamlined approach to the use of indicators and performance/monitoring data

collection. Common and programme-specific indicators should only be used to monitor ESIF

programmes if they produce useful information on performance. The current total of 72 common

output and result indicators is probably about right and should not be increased unless absolutely

necessary. More generally, feedback from the research suggests that there is still potential to

improve the relevance, focus, coverage, measurability, accuracy, and usefulness of the indicators as

well as the data collection systems.

Recommendation 2: The current set of ERDF and ESF common indicators should be reviewed

to ensure comprehensive coverage of Cohesion Policy objectives. There are some gaps in terms

of coverage, for example, the current set of common indicators do not fully cover the types of

activity funded (e.g. ICT), as well as issues relating to measurability (e.g. GHG emissions and energy

savings), usefulness (e.g. number of projects, number of visitors), accuracy (double-counting) as well

as some discrepancy or inconsistency in interpretation (e.g. definitions of enterprises cooperating and

brownfield sites) and aggregation (e.g. when to report data, reporting non-cumulative figures,

reporting sensitive data).


14

Recommendation 3: In relation to the specific indicators being used for ERDF and ESF

programmes, the European Commission should work with national and regional authorities

to examine the scope for greater harmonisation. According to an estimate provided by the

European Court of Auditors (ECA), there are some 8,000 to 9,000 programme-specific indicators

being used in the current 2014-2020 programmes. Greater use of common rather than specific

indicators would make it easier to identify the results and impacts being achieved at the

programme and EU levels.

A significant trend over the years has been a shift away in the ERDF and ESF monitoring and

evaluation frameworks from a focus on the absorption of funding and outputs towards a results-

oriented approach. The ESF is arguably more advanced in developing common results indicators than

the ERDF, but this difference in being addressed in the regulatory proposals for 2021-2027. In relation

to impacts, neither the ERDF nor the ESF requires common impact indicators to be monitored in the

current programming period. Whilst there is European Commission guidance on the development of

indicators and methodologies that can be used to evaluate impacts, there is no requirement to use

them. Instead, impacts are assessed through ex-post evaluations, which are normally the responsibility

of the European Commission.

Recommendation 4: Achieving a greater focus on measuring ERDF and ESF ‘results’ and

‘impacts’ should be prioritised in order to ensure closer alignment with the ‘Better

Regulation’ principles. Impact indicators have not been required in the current and previous

programming periods and are again excluded from the post-2020 proposals for the ERDF and ESF.

As noted in the report, the ESF already has common results indicators for ‘immediate’ and ‘longer-

term’ effects. A similar set of common results indicators is needed for the ERDF and the fund

proposal for 2021-2027 makes a welcome step in this direction. In both cases, there is a need to

develop the capacity to assess impacts more effectively.

Member States have made considerable progress in embedding the results-orientation

approach in their project selection processes. However, as a 2018 ECA report indicated, the focus

upon results has yet to become fully embedded in these procedures. Relevant indicators that

projects should utilise and contribute to are included in the call for proposals documentation and

project selection criteria in a minority of Member States, but for the most part not formalised in grant

contracts. There are increasing efforts to link payments to results and these should continue to be

developed, although incorporating some flexibility to reflect programming complexity.

Recommendation 5: The effort to ensure that the common ERDF and ESF common indicators

focus more on ‘results’ and ‘impacts’ needs to be also reflected in project assessment and

selection procedures. The design of selection processes continues to emphasise outputs and

absorption rather than results with quantified results indicators rarely included in grant

agreements whilst project selection tends to be undertaken on a first-come first-served basis.


15

Overall, there is strong evidence that the ESIF Performance Framework and the system of

indicators have shifted the focus more towards performance, but mixed evidence of targets and

milestones being defined in a way that strikes an appropriate balance between being realistic

and challenging. Indicators, targets and milestones are defined by Member States and regions on the

basis of their previous and individual experiences of programming achievements and reflect historical

reference data. They are then negotiated with the European Commission in the context of ‘shared

management’ and partnership. Targets and milestones have tended to be set cautiously because of

the risks of under-performing against Performance Reserve targets and consequently triggering

financial penalties. At present, neither the ERDF and ESF guidance explains how appropriate milestones

and targets should be set beyond taking into account regional needs and resources. The proposed

Common Provisions Regulation for 2021-2027 sets out a requirement for the Performance Framework

methodology to include the criteria applied, and data/evidence used in the selection of indicators and

targets and assurance of quality as well as any additional factors that may influence the achievements

of the programme and is a welcome improvement. Additionally, it has removed the requirement for

the ex-ante evaluation which has always proved an important element in the process of establishing

indicators/the performance framework and ensuring their relevance, suitability, realism and

quantification.

Recommendation 6: Further consideration should be given to the role of ex-ante evaluation

in ensuring results-oriented ERDF and ESF programmes. The removal of the requirement for

ex-ante evaluation in the 2021-2027 period is arguably a mistake. Previous experience suggests

that ex-ante evaluations can be very helpful in ensuring that appropriate and consistent objectives,

results and targets are set, and that planned interventions are likely to contribute in an effective

and efficient way to Cohesion Policy objectives and ensure EU added value. Ex-ante evaluations

can also be useful in assessing aspects such as what works in what circumstances, the net effect or

impact of the policy and enabling an in-depth assessment of the quality of interventions.

Recommendation 7: More should be done to help Member States and regions to define ERDF

and ESF targets robustly with appropriate levels of ambition. The proposals for the 2021-2027

period develop methodological approaches further whilst also reflecting the ‘shared management’

approach and principle of subsidiarity. Further methodological guidance and support, as well as

suitable capacity to deal with this requirement, should be ensured, especially in the proposed

absence of an ex-ante evaluation.

ESIF MONITORING SYSTEMS

Good progress has been made in developing efficient ESIF monitoring systems with most EU

Member States providing centralised guidance and IT tools to ensure consistency, and built-in

plausibility checks introduced at EU level. In the current 2014-2020 period, there has been a greater

focus on trying to ensure that the statistics required for the ERDF and ESF common indicators are

comparable across EU Member States and regions. The introduction of ‘Ex Ante Conditionality 7’ has

been important in this respect, to help ensure that the ESIF performance framework is supported by an


16

appropriate statistical system and a common definition of indicators and the information needed for

them.

However, assessments undertaken by the European Commission and the European Court of

Auditors suggest a mixed picture with deficiencies remaining in data collection. A lot of the

information that is required to measure results and impacts at the ERDF and ESF programme level

comes from those involved in implementing projects. Feedback from the research in Member States

indicates that beneficiaries are often contractually obliged to provide results information to Managing

Authorities. However, this is not so universally. Additionally, the results of programmes do not always

materialise at the required data collection point for the common indicators (e.g. six months after a

beneficiary of ESF support ceases to participate in a programme), whilst there is a sensitivity with regard

to some of the monitoring data required for the ESIF common indicators in certain Member States (this

relates specifically to some categories of ESF beneficiaries in countries where there are strict privacy

laws). Other deficiencies noted relate to aspects such as simple human error, incorrect units of

measurement, double-counting and a lack of an audit trail. Similarly, there is a very mixed picture with

regard to other methods of collecting data for the assessment of results and impacts with, for example,

the practice of conducting surveys, quite common in some Member States but not in others.

Recommendation 8: Further steps need to be taken to improve the timeliness, accuracy and

feasibility of the data collection systems underpinning the Performance Framework for ERDF

and ESF programmes. The timeliness of ESIF performance data should be improved whilst

ensuring that this does not place an unreasonable additional workload on national authorities.

Performance data held by the European Commission can be two years out of date because of the

timing of the annual implementation report cycle. The proposals for 2021-2027 envisage a near

real-time data collection with monitoring data proposed for submission on a two-monthly basis.

Whilst annual reporting is not frequent enough, a two-monthly cycle risks going to the opposite

extreme and is likely to prove too frequent for Managing Authorities. As an alternative, 4 or 6

monthly submissions should be considered. The 2021-2027 regulatory proposal includes the

possibility to use national data sources to collect ESF achievements. Use of existing national

administrative registers / datasets should lead to an improvement in the quality of data (e.g. social

security registers) as well as improving the linkage and alignment of ESF+ with national

interventions.

Through the ‘shared management’ of ESIF programmes, the European Commission and Member

States have worked closely together to implement the 2014-2020 ERDF and ESF programmes.

Feedback from the research suggests that partnership working has generally performed well. The

development of improved IT tools has made it easier to transfer data to the Commission. There have

been many other initiatives to promote more efficient and effective programme management and

monitoring systems. Problems remain, partly because the legislative basis and other arrangements for

monitoring systems in the 2014-2020 period were not put in place early enough.


17

Recommendation 9: The supporting systems and rules for the 2021-2027 Performance

Framework (PF) should be put in place as early as possible and before the ERDF and ESF

programmes are adopted. The current consultation by the European Commission (EC) with the

Member State (MS) delegations around indicator sets and definitions through the Evaluation

Network is welcome and should be continued, with observer roles considered (if not already in

place) for the other relevant institutions such as the EP and ECA. The EP should schedule a research

report for early in the programming period to determine the extent to which the PF for 2021-2027

is focusing more on results and impacts.

PERFORMANCE RESERVE AND CONTRIBUTION TO WIDER EU POLICY GOALS

Whilst the improved results-orientation and indicator sets under ESIF programming means that the

focus is increasingly shifting towards performance, the Performance Framework and Performance

Reserve as programming instruments do not reward programme managers on the basis of policy

results but on speed of implementation which is already influenced by the N+2/N+3 automatic de-

commitment rule.2

Looking ahead, and given experience in 2014-2020 with the Performance Reserve, the decision

to discontinue it is correct and widely supported, according to our research. Through the

Performance Reserve, stronger incentives have been introduced to encourage the achievement of

outputs. However, the Performance Reserve becomes practically irrelevant in the context of

reprogramming possibilities and few Member States are likely to lose any of their allocation, which will

simply be reallocated to better performing priorities or programmes. The process to date is seen to be

highly political and administrative and lacking in any reflection of the complexity of programming on

the ground. This was emphasised in another ECA special report that noted that the Performance

Framework and Performance Reserve provide little incentive for a better result orientation of the OPs

since they are mostly based on spending and outputs. Whilst the new 5+2 programming approach3

could enable a more strategic approach, it is likely to still be overly bureaucratic and output-focused.

The introduction of a more qualitative review process is welcome but should be proportionate and

appropriately responsive, i.e. able to reflect new development needs in a timely fashion not as part of

a significant administrative exercise for all programmes at a specific point in time. Current re-

programming possibilities would seem therefore to be mostly adequate and could be enhanced by the

possibility of an interim review enabling regions to approach the process according to their specific

performance issues, domestic circumstances and timing requirements.4 It could be possible for

regions/Member States to determine flexibly a proportion of funding (up to a maximum specified

percentage) that would remain unallocated5 and could be then reprogrammed based on programme

2 The N+2/N+3 decommitment rule provides that any EU funding commitment against which a payment application has notbeen received by the end of the second/ third year following the year of commitment will be decommitted. Under the 2014-20 programming period the rule has been N+3; under the post-2020 regulatory proposals the rule will be N+2 for the years2021-2027, returning to pre-2013 requirements, (for the EU-15).3 The 5+2 approach under the post-2020 regulatory proposals refers to the initial programming for five years, and theprogramming of the final 2 years only subsequent to a Mid Term Review.4 For example, in the current programming period, the Scottish government bought forward the Performance Review processto 2017 in a so-called ‘early review’ in response to programming uncertainty following Brexit.5 The 2007-2013 programming period allowed for a voluntary performance reserve at the discretion of the Member States.


18

performance, new regional challenges or needs and relevant Country-Specific Recommendations.

Member States with a small number of programmes or small financial allocations could decide not to

implement a review and reprogramming process based upon the principle of proportionality.

Recommendation 10: The decision to discontinue the Performance Reserve is widely

supported and any future system that replaces it should be based on a more results-

orientated approach. Although programme outputs are of course important, it is results that

provide a more meaningful indication of performance and the contribution of cohesion policy to

EU objectives. According to our research, the majority of Managing Authorities considered the

process a narrow partial view of what the programmes actually achieve, and particularly rigid and

inflexible in relation to external factors and more innovative and experimental uses of the funds.

Whilst its potential value in terms of acting as a tool for performance-orientation is recognised, it is

considered an overly bureaucratic means by which to achieve this. These and other criticisms will

also need to be addressed in any future performance-based system, which should be proportionate

and appropriately responsive. The European Commission should reconsider other means by which

programmes and regions can adapt quickly to crises and demonstrate flexibility and

responsiveness. A balance equally needs to be struck with the need for stable strategic investment

planning, which the 7-year programming period facilitates.6 A minor variation on the current re-

programming provisions/possibilities could enable an interim review that takes into account

specific regional performance issues and progress, domestic circumstances and socio-economic

situation, as well as timing requirements, and allow for significant reprogramming on that basis.

Feedback from the research suggests that ESIF spending has been more targeted on Europe

2020 priorities than was the case in previous programming periods in relation to the then-

prevailing EU objectives. However, the ECA and others have argued that ex-ante conditionalities and

the Performance Reserve were innovative but not yet effective instruments for ensuring good results

in relation to these and other priorities.

Recommendation 11: The focus on results-orientated performance measurement should be

supported by a greater emphasis on establishing how ESIF programmes contribute to overall

EU policy goals. The Partnership Agreements being used as a basis for implementing ERDF and

ESF programmes include wider EU policy goals. It would be unrealistic to expect the indicator

system to be able to help determine the extent to which wider EU policy goals are being promoted

through ERDF and ESF interventions. However, a link between such interventions and EU policy

goals could be more clearly established and assessed in ex-ante and ex-post evaluations. A

constraint on being able to do this in the 2021-2027 period could be the absence of a wider EU

strategy and coherent set of overall EU policy goals.

REGULATORY FRAMEWORK FOR 2021-2027

The Common Provisions Regulation for Cohesion Policy in the 2021-2027 period addresses many of the

issues examined in this Research Paper. On the following page we provide a summary.

6 http://data.consilium.europa.eu/doc/document/ST-14216-2018-INIT/en/pdf


19

Issues Raised in the Research Paper that are addressed in the Common ProvisionsRegulation for 2021-2027

ESF+ indicators have not been significantly altered, allowing for continuity. This is in line withour view that unless absolutely necessary and where useful information on performance canbe provided, the number of indicators used should not increase.

The ERDF common indicators have doubled in number. However, the modification is asignificant enhancement providing a more comprehensive set to cover the range ofinterventions funded gaps. The inclusion of both output and result indicators demonstratesa clear improvement on the previous programming periods with some long-used indicatorsappropriately re-categorised from outputs to results.

Whilst programme-specific indicators are still possible, reflecting suitable flexibility inprogramming, the requirement for all indicators to form part of the performance frameworkcould potentially reduce their number.

The regulatory requirement for the Performance Framework methodology to include thecriteria applied, and data/evidence used in the selection of indicators and targets and assuranceof quality as well as any additional factors that may influence the achievements of theprogramme should improve target-setting and ensure appropriate ambition.

A more frequent data reporting requirement will improve the timeliness of data. However,two-monthly reporting is likely to prove too frequent for Managing Authorities, and a 4 or 6monthly submission could be considered.

The possibility to use national data sources to collect ESF achievements should lead to animprovement in the quality of data, a reduction in administrative burden and improve thelinkage and alignment of ESF+ with national interventions.

Discontinuing the Performance Reserve is appropriate.

Issues Raised in the Research Paper that are not fully addressed in theCommon Provisions Regulation for 2021-2027

Whilst the new 5+2 programming approach could enable a more strategic approach to beadopted through a mid-term review, it is likely to still be overly bureaucratic and output-focused as well as somewhat inflexible and disproportionate in terms of the administrativeburden. The proposals do not include the requirement for the use of impact indicators foreither fund. This goes against the Better Regulations guidance, and according to the ECA, isequally in contradiction to the Financial Regulation.

The lack of requirement for an ex-ante evaluation has removed an important element in theprocess of establishing indicators/the performance framework and ensuring their relevance,suitability, realism and quantification, as well as strategic assessment of the policy andintervention logic in relation to regional development needs.

Guidance and definitions related to the new indicator sets must be available well in advanceof 2021-2027 OP adoption.


20

SYNTHÈSE

Vue d’ensemble

Pour la période de programmation 2014-2020 en cours, quelque 352 milliards d’euros ont été

alloués à la politique de cohésion de l’Union européenne au titre du Fonds européen de

développement régional (FEDER) et du Fonds social européen (FSE). Il est de toute évidence

primordial que les interventions fassent l’objet d’un suivi et d’une évaluation dans un souci de reddition

de comptes, mais aussi pour aider à garantir une affectation des financements de l’Union la plus

efficiente et la plus efficace possible.

La présente étude, destinée au Parlement européen, vise à examiner dans quelle mesure les

indicateurs clés de performance pour la convergence en matière de développement régional

contribuent à des dépenses ciblées et orientées vers les résultats. À cette fin, il convient d’analyser

le système d’indicateurs communs actuellement utilisé dans la politique de cohésion de l’Union

européenne, y compris ses forces, ses faiblesses et d’autres facteurs pertinents. Tournée vers l’avenir,

cette étude contient des propositions de modifications éventuelles des stratégies appliquées à l’heure

actuelle, l’objectif étant d’améliorer les résultats globaux de la politique de cohésion de l’Union

européenne. Cette mission a été accomplie début 2019 par le Centre for Strategy & Evaluation Services

pour le compte du département thématique D (affaires budgétaires) et de la direction générale des

politiques internes de l’Union du Parlement européen7.

De manière générale, cette étude montre que des progrès satisfaisants ont été effectués dans le

développement d’un système efficace d’indicateurs de performance en vue d’assurer le suivi de

la contribution de la politique de cohésion à la convergence en matière de développement

régional. L’ensemble d’indicateurs de réalisation actuellement utilisé par le FEDER et le FSE est bien

conçu. Toutefois, le défi à relever demeure le même, à savoir parvenir à moins se concentrer sur la

mesure des contributions et rendements financiers pour se pencher davantage sur la capacité à évaluer

les résultats et les impacts des interventions, plus significatifs. Il reste également beaucoup à faire pour

développer des systèmes de collecte de données qui soient en mesure de produire le type

d’informations nécessaires à l’évaluation des résultats et des incidences, sans pour autant imposer de

charge administrative disproportionnée aux autorités nationales et aux bénéficiaires de la politique de

cohésion.

Conclusions et recommandations principales

Il s’agit tout d’abord de résumer les caractéristiques majeures du cadre d’évaluation de la performance

applicable au FEDER et au FSE, puis d’apprécier dans quelle mesure il a permis de garantir la réalisation

d’interventions orientées vers les résultats.

7 Blomeyer & Sanz, partenaire du consortium, a assuré le contrôle qualité.


21

Cadre d’évaluation de la performance applicable au FEDER et au FSE

Le règlement (UE) no 1303/2013 du Parlement européen et du Conseil du 17 décembre 2013 portant

dispositions communes relatives au Fonds européen de développement régional, au Fonds social

européen, au Fonds de cohésion, au Fonds européen agricole pour le développement rural et au Fonds

européen pour les affaires maritimes et la pêche, portant dispositions générales applicables au Fonds

européen de développement régional, au Fonds social européen, au Fonds de cohésion et au Fonds

européen pour les affaires maritimes et la pêche, et abrogeant le règlement (CE) no 1083/2006 du

Conseil (le règlement portant dispositions communes pour la période de programmation 2014-2020)

a introduit des modifications importantes visant à mettre davantage l’accent sur la performance et à

favoriser une approche plus orientée vers les résultats. Ces modifications sont composées des

principaux éléments exposés ci-après.

Principales caractéristiques de l’orientation vers les résultats

Conditions ex ante: elles ont été négociées dans le but de contribuer à garantir que les

conditions nécessaires sont réunies pour que l’accent soit mis sur les principaux indicateurs de

performance et sur le succès de la mise en œuvre du programme.

Cadres pour la justification et la logique des interventions: il s’agit de garantir l’orientation

des programmes vers les résultats en liant les besoins à satisfaire aux résultats à atteindre et en

élaborant des définitions et objectifs mesurables pour les indicateurs de résultat.

Cadre de performance: la mise en place d’étapes et d’objectifs clairs et mesurables vise à

garantir la conformité des progrès accomplis avec les prévisions (cadre de performance), une

réserve de performance permettant de récompenser les résultats obtenus.

La présente étude porte principalement sur le cadre de performance, notamment sur les indicateurs

communs et sur les indicateurs spécifiques par programme utilisés pour contribuer à garantir des

interventions ciblées et orientées vers les résultats, d’une part, et à mesurer les effets et les résultats de

la politique de cohésion sur le terrain, d’autre part. D’après les recherches, quelques améliorations

permettront au système existant de veiller à une approche ciblée et orientée vers les résultats

de la mise en œuvre des programmes du FEDER et du FSE.

Pour appliquer le cadre de performance, la Commission et les États membres devraient coopérer en

vue de définir des valeurs de référence (par exemple, la situation d’une région en matière de

compétences ou de création d’entreprises) et des objectifs (par exemple, comment les interventions

du FEDER et du FSE pourraient améliorer la situation). La confrontation de ces données avec les

indicateurs permettrait alors d’évaluer les progrès accomplis dans la mise en œuvre.

Les lignes directrices pour une meilleure réglementation de la Commission européenne (2015

et 2017) fixent trois types d’indicateurs de base: les indicateurs de réalisation, qui mesurent les

résultats spécifiques des interventions [par exemple, le nombre de petites et moyennes entreprises

(PME) bénéficiant de subventions du FEDER ou le nombre de participants aux formations du FSE]; les

indicateurs de résultat, qui portent sur les effets immédiats d’une intervention sur les bénéficiaires

(par exemple, l’amélioration des performances des PME ou des qualifications des participants aux

formations du FSE); les indicateurs d’impact, qui concernent les effets sur l’ensemble de l’économie


22

et de la société (par exemple, la création d’emplois ou l’amélioration de la compétitivité). Si les

réalisations sont de nature matérielle et, partant, relativement faciles à mesurer, les résultats et les

impacts sont d’ordre socio-économique et nettement plus difficiles à évaluer. De la même manière, les

interventions du FSE sont axées sur les personnes, tandis que le FEDER cible des groupes plus diversifiés

et ses unités de mesure sont moins homogènes.

Il importe de souligner que l’efficacité du cadre de performance dépend non seulement de

l’adéquation des indicateurs utilisés, mais aussi de l’intégration de mécanismes de collecte de

données dans des systèmes de gestion de programmes capables de produire les informations

nécessaires sur les interventions et sur leurs effets.

Évolution du cadre de performance

Le suivi et l’évaluation des fonds structurels et d’investissement européens (fonds ESI) ont

connu de grandes améliorations à chaque période de programmation. Les principales

modifications apportées au fil des années au cadre réglementaire applicable aux fonds ESI sont

résumées ci-dessous.


23

Figure 0.1 — Évolution du cadre réglementaire des fonds ESI (2000-2027)


24

En résumé, avant l’an 2000, il n’existait pas d’approche commune permettant de définir des indicateurs

et des objectifs: chaque État membre pouvait choisir son approche, ce qui donnait lieu à une

multiplicité d’indicateurs appliqués sur l’ensemble des programmes du FEDER et du FSE et à une

harmonisation très limitée. Non seulement les indicateurs variaient selon les États membres et les

régions mais, y compris en cas d’utilisation des mêmes indicateurs, les définitions changeaient, tout

comme les méthodes de mise en œuvre. Par conséquent, il était difficile, voire impossible, de regrouper

des informations au niveau de l’Union concernant les résultats de la politique de cohésion.

Utilisation d’indicateurs communs et d’indicateurs spécifiques par programme

Les principaux programmes du FEDER et du FSE actuellement menés sur la période 2014-2020

font l’objet d’un suivi effectué grâce à un ensemble de 72 indicateurs communs de réalisation et

de résultat. Parallèlement, les autorités de gestion utilisent un grand nombre d’indicateurs

spécifiques par programme. Cette situation montre que, s’il est vrai que les indicateurs communs

devraient être en grande partie pertinents pour tous les États membres et régions, d’autres indicateurs

reflétant plus précisément les spécificités régionales peuvent se révéler nécessaires (certains

indicateurs ne sont pas pertinents dans certains cas de figure, par exemple, les kilomètres de chemins

de fer construits n’ont pas de sens pour les États membres sans réseau ferroviaire, comme Chypre et

Malte). Il ressort qu’au fil des années, l’effort de simplification consenti afin de réduire le nombre

d’indicateurs employés s’est progressivement consolidé.

Le FSE compte actuellement 32 indicateurs communs (23 indicateurs de réalisation et

9 indicateurs de résultat), qui constituent le minimum à intégrer à chaque programme

opérationnel. Les indicateurs les plus fréquemment sélectionnés visent le nombre d’actifs participant

à des programmes financés par le FSE, suivi du nombre de chômeurs, de personnes défavorisées et de

jeunes. Dans l’ensemble, 16 895 indicateurs de réalisation et 3 556 indicateurs de résultat sont utilisés

pour les 180 programmes opérationnels du FSE durant la période de programmation actuelle (sachant

que certains programmes sont plurifonds et relèvent à la fois du FSE et du FEDER). Seulement 15 % des

indicateurs déployés dans les programmes opérationnels du FSE sont des indicateurs communs, alors

que la plupart des indicateurs spécifiques par programme correspondent à des sous-catégories ou à

des combinaisons de catégories utilisées dans les indicateurs communs. Outre les indicateurs de

réalisation, le FSE a défini des indicateurs communs pour les résultats «immédiats» et «à plus long

terme».

En ce qui concerne les principaux programmes du FEDER [c’est-à-dire sans tenir compte de la

coopération territoriale européenne (CTE)], un total de 40 indicateurs communs de réalisation

sont actuellement employés dans les programmes opérationnels de la période 2014-2020. Ces

indicateurs ont été utilisés 5 757 fois dans les 201 programmes opérationnels. Ils représentent environ

55 % de l’ensemble des indicateurs usités et portent plus ou moins sur le même montant

d’investissement. Les indicateurs communs les plus fréquemment utilisés signalent le nombre

d’entreprises bénéficiant d’un soutien ainsi que la valeur des subventions, la croissance de l’emploi

dans les entreprises bénéficiant d’une aide et la réduction des émissions de gaz à effet de serre. Les

programmes opérationnels n’exploitent parfois aucun indicateur commun. Le cas échéant, il est


____________________________________________________________________________________________

25

recommandé de recourir à des indicateurs définis au niveau de l’Union (tels que des indicateurs tirés

du tableau de bord de l’innovation) afin d’améliorer la mesurabilité et la comparabilité. Les résultats

des indicateurs propres à chaque programme du FEDER ne peuvent généralement pas être agrégés et

ces outils ont sans doute été employés de manière excessive dans les cadres de performance des

programmes opérationnels. Contrairement au FSE, le FEDER n’a pas été doté d’un ensemble commun

d’indicateurs de résultat: ceux qui existent sont propres à chaque programme.

En règle générale, l’emploi d’indicateurs communs obligatoires de réalisation (et de résultat

pour ce qui est du FSE) fondés sur des définitions normalisées a permis de renforcer la logique

d’intervention des programmes, c’est-à-dire la manière dont les financements prévus contribuent à

réaliser les changements attendus au regard des besoins recensés en matière de développement.

L’utilisation de ces indicateurs obligatoires fait partie du processus de négociation des programmes.

Pour ce qui est de la politique de cohésion, la transparence, la visibilité et l’obligation de rendre compte

ont enregistré des améliorations. En outre, le fait de définir des indicateurs et des objectifs clairs et

mesurables dans le cadre de la logique d’intervention dès le début d’un programme encourage les

autorités de gestion et les acteurs régionaux/nationaux à accorder une attention accrue aux résultats

et progrès de la performance.

L’ensemble d’indicateurs communs actuellement exploité pour le FEDER et le FSE est

globalement bien conçu et adéquatement ciblé au regard des objectifs de la politique de

cohésion, mais il est nécessaire de remédier à certaines lacunes. Si les indicateurs spécifiques par

programme du FSE correspondent généralement à des sous-catégories ou à des combinaisons de

catégories utilisées dans les indicateurs communs, ceux auxquels recourent les programmes du FEDER

sont très nombreux et, souvent, ne peuvent pas être cumulés. Ce système impose une charge

administrative et des coûts considérables, et la qualité ainsi que la précision de ses résultats sont parfois

faibles. Toutefois, si elles présentent l’avantage de réduire la charge administrative et de renforcer le

respect de l’obligation de rendre compte, la simplification et l’harmonisation ne devraient pas donner

lieu à une programmation rigide ou à une approche uniforme.

Recommandation no 1: dans la mesure du possible, il y a lieu de poursuivre la simplification

du système d’indicateurs communs utilisé pour les fonds ESI. Il est nécessaire de concrétiser le

principe de proportionnalité et de garantir une approche plus rationnelle dans l’utilisation des

indicateurs et dans la collecte de données sur la performance et le suivi. Pour assurer le suivi des

programmes des fonds ESI, il convient d’utiliser les indicateurs communs et les indicateurs

spécifiques par programme uniquement lorsqu’ils produisent des informations utiles sur la

performance. Le total actuel de 72 indicateurs communs de réalisation et de résultat est

probablement approprié et ne devrait pas être augmenté, sauf en cas de nécessité absolue. Plus

généralement, les recherches ont montré qu’il restait une marge d’amélioration concernant la

pertinence, le ciblage, la portée, la mesurabilité, la précision et l’utilité des indicateurs ainsi que des

systèmes de collecte de données.


26

Recommandation no 2: il convient de réexaminer l’ensemble d’indicateurs communs

actuellement utilisé pour le FEDER et le FSE afin de garantir une prise en compte complète

des objectifs de la politique de cohésion. Il existe des lacunes à cet égard: à titre d’exemple,

l’ensemble actuel d’indicateurs communs ne couvre pas tous les types d’activités financées

(comme les technologies de l’information et de la communication). Certains problèmes se posent

également sur le plan de la mesurabilité (y compris des émissions de gaz à effet de serre et des

économies d’énergie), de l’utilité (le nombre de projets ou le nombre de visiteurs, par exemple) et

de la précision (double comptage). Ont également été révélées des divergences ou incohérences en

matière d’interprétation (par exemple, les définitions des entreprises coopérant et des friches

industrielles) et d’agrégation (par exemple, le calendrier de communication des informations, la

transmission de chiffres non cumulés ou de données sensibles).

Recommandation no 3: en ce qui concerne les indicateurs spécifiques utilisés pour les

programmes du FEDER et du FSE, la Commission européenne devrait coopérer avec les

autorités nationales et régionales pour déterminer la marge de manœuvre dont elles

disposent pour accroître l’harmonisation. Selon une estimation de la Cour des comptes

européenne, 8 000 à 9 000 indicateurs spécifiques par programme sont employés dans les

programmes en cours pour la période 2014-2020. Il serait plus facile de recenser les résultats et les

impacts obtenus aux niveaux des programmes et de l’Union si le recours aux indicateurs communs

était plus fréquent que le recours aux indicateurs spécifiques par programme.

Il ressort clairement qu’au fil des années, les cadres de suivi et d’évaluation du FEDER et du FSE

ont de moins en moins mis l’accent sur l’absorption des fonds et sur les réalisations pour se

concentrer davantage sur une approche orientée vers les résultats. Le FSE est sans doute plus

avancé que le FEDER en matière d’élaboration d’indicateurs communs de résultats, mais les

propositions de règlements établies pour la période 2021-2027 tentent de résorber cet écart. En ce qui

concerne les incidences, ni le FEDER ni le FSE n’imposent d’assurer le suivi d’indicateurs communs

d’impact pour la période de programmation actuelle. Bien que la Commission européenne ait publié

des orientations pour l’élaboration des indicateurs et des méthodes permettant d’évaluer les

répercussions des programmes, leur utilisation n’est pas obligatoire. Les impacts sont plutôt évalués

au moyen d’évaluations ex post, qui relèvent généralement de la responsabilité de la Commission

européenne.

Recommandation no 4: il convient d’accorder la priorité à la mesure des «résultats» et des

«impacts» du FEDER et du FSE afin de garantir un alignement plus précis sur les principes

pour une meilleure réglementation. Les indicateurs d’impact ne sont pas obligatoires dans la

période de programmation actuelle, ne l’étaient pas non plus dans les périodes précédentes, et

sont à nouveau exclus des propositions relatives au FEDER et au FSE pour l’après-2020. Comme

indiqué précédemment, le FSE dispose déjà d’indicateurs communs de résultat pour les effets

«immédiats» et «à plus long terme». Le FEDER a besoin d’un ensemble d’indicateurs communs de

résultat de ce type, et la proposition relative à ce fonds pour la période 2021-2027 fait un pas


____________________________________________________________________________________________

27

encourageant dans cette direction. Dans les deux cas, il est nécessaire de développer la capacité à

évaluer plus efficacement les impacts.

Les États membres ont accompli des progrès considérables en intégrant l’approche orientée vers

les résultats dans leur processus de sélection des projets. Toutefois, selon un rapport de la Cour

des comptes européenne de 2018, l’orientation vers les résultats doit encore être pleinement

intégrée à ces procédures. Les indicateurs concernés que les projets devraient utiliser et auxquels ces

derniers devraient contribuer figurent dans les documents d’appel à propositions et dans les critères

de sélection des projets dans une minorité d’États membres, mais ne sont pas, dans la plupart des cas,

formalisés dans les contrats de subvention. Les efforts se multiplient pour subordonner les paiements

aux résultats et il convient de poursuivre sur cette voie, bien qu’une certaine souplesse doive être

apportée afin de tenir compte de la complexité de la programmation.

Recommandation no 5: les efforts consentis aux fins d’une orientation accrue des indicateurs

communs du FEDER et du FSE vers les «résultats» et les «impacts» doivent aussi se refléter

dans l’évaluation des projets et les procédures de sélection. La conception des processus de

sélection continue de mettre l’accent sur les réalisations et sur l’absorption plutôt que sur les

résultats: les conventions de subvention comprennent rarement des indicateurs de résultats

chiffrés, tandis que les projets sont souvent sélectionnés selon le principe du «premier arrivé,

premier servi».

De manière générale, de nombreux éléments indiquent que le cadre de performance et le

système d’indicateurs des fonds ESI sont davantage axés sur la performance, tandis que le

tableau est mitigé concernant la définition des cibles et étapes de façon à ménager un juste

équilibre entre réalisme et exigence. Les indicateurs, les valeurs cibles et les valeurs intermédiaires

sont définis par les États membres et les régions sur la base de leur propre expérience acquise en

fonction des résultats des programmes précédents, et sont le reflet de leurs données de référence. Ils

sont ensuite négociés avec la Commission européenne dans le cadre d’une «gestion partagée» et d’un

partenariat. Les objectifs et les étapes sont généralement fixés avec prudence en raison des risques

d’enregistrer une performance inférieure aux objectifs de la réserve de performance et, partant, de se

voir imposer des sanctions financières. À l’heure actuelle, ni les orientations du FEDER ni celles du FSE

n’expliquent comment il convient d’établir les valeurs intermédiaires et les valeurs cibles, outre la prise

en considération des besoins et des ressources des régions. La proposition de règlement portant

dispositions communes pour la période de programmation 2021-2027 prévoit que la méthode

d’élaboration du cadre de performance comprenne obligatoirement les critères appliqués, les données

et les éléments probants utilisés aux fins de la sélection des indicateurs et des objectifs, l’assurance

qualité, ainsi que tout autre facteur susceptible d’influer sur la réalisation du programme, ce qui

constitue un progrès appréciable. La proposition supprime par ailleurs l’obligation d’évaluation

ex ante, laquelle a toujours constitué un élément essentiel dans la mise au point des indicateurs et du

cadre de performance ainsi que dans la garantie de leur pertinence, de leur adéquation, de leur

réalisme et de leur évaluation chiffrée.


28

Recommandation no 6: il convient de considérer avec plus d’égards l’importance du rôle que

joue l’évaluation ex ante dans la garantie que les programmes du FEDER et du FSE sont

orientés vers les résultats. La suppression de l’obligation d’effectuer une évaluation ex ante pour

la période 2021-2027 constitue vraisemblablement une erreur. L’expérience montre que les

évaluations ex ante peuvent se révéler très utiles pour veiller à ce que des valeurs cibles, des

résultats et des objectifs appropriés et cohérents aient été définis, et que les interventions prévues

peuvent contribuer de manière efficace et efficiente à la concrétisation des objectifs de la politique

de cohésion et à la création de la valeur ajoutée de l’Union européenne. Les évaluations ex ante

peuvent également servir, entre autres, à sélectionner les travaux à accomplir selon les

circonstances, à mesurer l’effet ou l’impact net de la stratégie adoptée et à faciliter l’évaluation

approfondie de la qualité des interventions.

Recommandation no 7: des efforts supplémentaires doivent être mobilisés afin d’aider les

États membres et les régions à fixer des valeurs cibles solides reflétant un niveau d’ambition

adéquat. Les propositions pour la période 2021-2027 contiennent des approches

méthodologiques et rendent également compte de la démarche de «gestion partagée» et du

principe de subsidiarité. Il y a lieu de fournir de nouvelles orientations méthodologiques et un

soutien supplémentaire, ainsi que des ressources adaptées pour respecter cette exigence,

notamment en l’absence d’évaluations ex ante.

Systèmes de suivi des fonds ESI

Des progrès satisfaisants ont été réalisés dans la conception de systèmes efficients de suivi des

fonds ESI: la plupart des États membres de l’Union fournissent des orientations et des outils

informatiques centralisés permettant de garantir la cohérence, et des contrôles de

vraisemblance intégrés ont été établis à l’échelle de l’Union. Pour la période 2014-2020, les

statistiques obligatoires pour les indicateurs communs du FEDER et du FSE ont fait l’objet de mesures

visant à les rendre comparables d’un État membre et d’une région de l’Union à l’autre. L’introduction

de la 7e condition ex ante joue un rôle important à cet égard, afin que le cadre de performance des

fonds ESI soit soutenu par un système statistique approprié, par une définition commune des

indicateurs et par les informations nécessaires à cette fin.

Les évaluations réalisées par la Commission européenne et par la Cour des comptes européenne

ont toutefois dressé un bilan mitigé, signalant des insuffisances persistantes en matière de

collecte de données. Nombreuses sont les informations nécessaires pour mesurer les résultats et les

impacts des programmes du FEDER et du FSE qui émanent des acteurs participant à la réalisation des

projets. Les recherches menées dans les États membres montrent que les bénéficiaires sont souvent

contractuellement tenus de transmettre les résultats aux autorités de gestion, mais ce n’est pas

toujours le cas. En outre, les résultats des programmes ne sont pas toujours disponibles au moment

requis pour la collecte des données des indicateurs communs (par exemple, six mois après que le

bénéficiaire du soutien du FSE a cessé de participer à un programme). Certaines données de suivi

obligatoires pour les indicateurs communs des fonds ESI constituent par ailleurs des données sensibles


____________________________________________________________________________________________

29

dans certains États membres (en particulier pour certaines catégories de bénéficiaires du FSE dans les

pays dotés d’une législation stricte en matière de protection de la vie privée). Parmi les autres lacunes

constatées figurent de simples erreurs humaines, des unités de mesure incorrectes, le double

comptage et l’absence de piste de contrôle. De la même manière, il existe une grande hétérogénéité

dans les autres méthodes de collecte de données utilisées pour évaluer les résultats et les impacts: à

titre d’exemple, la réalisation d’enquêtes est très courante dans certains États membres, mais pas dans

d’autres.

Recommandation no 8: il est nécessaire de prendre des mesures supplémentaires en vue

d’améliorer l’opportunité, l’exactitude et la faisabilité des systèmes de collecte de données

qui sous-tendent le cadre de performance des programmes du FEDER et du FSE. Il convient

d’améliorer le respect des délais applicables aux données relatives à la performance des fonds ESI,

tout en veillant à ne pas imposer de charge de travail supplémentaire déraisonnable aux autorités

nationales. Les données relatives à la performance dont dispose la Commission européenne

accusent parfois deux ans de retard en raison du calendrier du cycle annuel des rapports de mise

en œuvre. Les propositions pour la période 2021-2027 prévoient une collecte des données en

temps quasi réel, les données de suivi devant être présentées sur une base bimestrielle. Si

l’établissement de rapports sur une base annuelle est trop espacé, les cycles de deux mois risquent

de verser dans l’extrême inverse et d’être trop fréquents pour les autorités de gestion. Une autre

possibilité serait de présenter les rapports tous les quatre à six mois. La proposition de règlement

pour la période 2021-2027 prévoit la possibilité de puiser les informations relatives aux réalisations

du FSE dans des sources de données nationales. L’exploitation de registres administratifs et de jeux

de données nationaux existants (par exemple, les registres de sécurité sociale) devrait permettre

d’améliorer la qualité des données ainsi que le lien et la concordance du FSE+ avec les interventions

nationales.

Grâce à la «gestion partagée» des programmes des fonds ESI, la Commission européenne et les

États membres collaborent étroitement pour mettre en œuvre les programmes du FEDER et du

FSE de la période 2014-2020. Les recherches montrent que le travail en partenariat produit

généralement de bons résultats. L’amélioration des outils informatiques facilite le transfert de données

à la Commission. De nombreuses autres initiatives ont été lancées pour promouvoir des systèmes de

suivi et de gestion des programmes plus efficaces et plus efficients. Certains problèmes persistent

toutefois, notamment parce que la base juridique et les autres dispositions régissant les systèmes de

suivi de la période 2014-2020 n’ont pas été mises en place suffisamment tôt.

Recommandation no 9: il convient de mettre en place le plus rapidement possible les

systèmes de soutien et les règles applicables au cadre de performance pour la période 2021-

2027, avant l’adoption des programmes du FEDER et du FSE. La consultation que mène

actuellement la Commission européenne auprès des délégations des États membres par

l’intermédiaire du réseau d’évaluation en ce qui concerne les ensembles et les définitions des

indicateurs constitue une bonne initiative, qu’il convient de poursuivre. Il serait envisageable

d’accorder (si ce n’est pas déjà le cas) aux autres institutions concernées, comme le Parlement


30

européen et la Cour des comptes européenne, un rôle d’observatrices de ce processus. Le

Parlement européen devrait prévoir l’établissement d’un rapport de recherche pour le début de la

prochaine période de programmation afin de déterminer la mesure dans laquelle le cadre de

performance 2021-2027 est orienté vers les résultats et les impacts.

Réserve de performance et contribution aux objectifs plus vastes de l’Union

L’amélioration de l’orientation vers les résultats et des ensembles d’indicateurs dans le cadre de la

programmation des fonds ESI se traduit par un déplacement progressif de l’attention vers la

performance. Cependant, en tant qu’instruments de programmation, le cadre de performance et la

réserve de performance ne récompensent pas les gestionnaires de programmes sur la base des

résultats de la politique, mais selon la rapidité de la mise en œuvre, qui est déjà influencée par la règle

du dégagement automatique «N+2/N+3»8.

En ce qui concerne l’avenir et au vu de l’expérience acquise pendant la période 2014-2020, nos

recherches montrent que la décision de supprimer la réserve de performance est adéquate et

reçoit une large approbation. La réserve de performance a permis de renforcer les incitations à la

production de réalisations. Cependant, dans la pratique, elle perd son intérêt en cas de possibilité de

reprogrammation: il est rare que les États membres perdent leur dotation, laquelle est simplement

réaffectée à des priorités ou à des programmes plus performants. La procédure est à ce jour jugée

essentiellement politique et administrative, et elle ne reflète pas la complexité de la programmation

sur le terrain. Ce point est mis en évidence dans un rapport spécial de la Cour des comptes européenne,

qui relève que le cadre de performance et la réserve de performance n’incitent que peu à améliorer

l’orientation des programmes opérationnels vers les résultats, car ils sont le plus souvent fondés sur les

dépenses et sur les réalisations. La nouvelle approche de programmation «5+2»9 pourrait se révéler

plus stratégique, mais elle reste probablement trop bureaucratique et axée sur les réalisations.

L’introduction d’un processus d’évaluation plus qualitatif constitue une bonne nouvelle, mais il devrait

être proportionné et adéquatement réactif, c’est-à-dire capable de tenir compte des nouveaux besoins

de développement en temps opportun, sans être intégré à un grand exercice administratif que tous les

programmes devraient effectuer à un moment précis. Il semblerait, par conséquent, que les possibilités

de reprogrammation existantes soient généralement appropriées, et pourraient être améliorées par

des bilans intermédiaires permettant aux régions d’appréhender le processus en fonction de leurs

propres problèmes de performance, de leur situation intérieure et des exigences en matière de

calendrier10. Les régions et les États membres devraient pouvoir déterminer avec souplesse une

8 La règle de dégagement «N+2/N+3», prévoit que tout engagement de financement de l’Union n’ayant pas fait l’objet d’unedemande de paiement avant la fin de la deuxième ou de la troisième année suivant l’année d’engagement sera dégagé.Pendant la période de programmation 2014-2020, la règle applicable est «N+3». Les propositions réglementaires relatives àl’après-2020 prévoient d’appliquer la règle «N+2» pour la période 2021-2027, soit un retour aux exigences en vigueuravant 2013 (dans l’Europe des quinze).9 Dans les propositions réglementaires pour l’après-2020, l’approche «5+2» désigne la programmation initiale sur cinq ans,ainsi que la programmation des deux dernières années seulement à la suite d’un examen à mi-parcours.10 À titre d’exemple, au cours de la période de programmation actuelle, le gouvernement écossais a avancé le processusd’examen des performances à 2017 sous la forme d’un «examen anticipé» en raison des incertitudes relatives à laprogrammation à la suite de la sortie du Royaume-Uni de l’Union européenne.


____________________________________________________________________________________________

31

proportion du financement (jusqu’à un pourcentage maximal) susceptible de ne pas être attribuée11,

qui pourrait ensuite être reprogrammée en fonction de la performance des programmes, des nouveaux

défis ou besoins régionaux et des recommandations par pays. Les États membres dont les programmes

sont peu nombreux ou qui bénéficient de dotations financières réduites pourraient décider de ne pas

mettre en œuvre de processus d’examen et de reprogrammation en se fondant sur le principe de

proportionnalité.

Recommandation no 10: la décision de supprimer la réserve de performance est largement

approuvée, et tout futur système de remplacement devra se fonder sur une approche

davantage orientée vers les résultats. Si les réalisations des programmes sont bien sûr

essentielles, les résultats fournissent une indication plus claire sur la performance et la contribution

de la politique de cohésion à la concrétisation des objectifs de l’Union. D’après nos recherches, la

plupart des autorités de gestion estiment que ce processus donne une vision à court terme et

incomplète des résultats réels des programmes, et qu’il manque cruellement de souplesse en ce

qui concerne les facteurs extérieurs et les utilisations innovantes et expérimentales des fonds. Si sa

valeur potentielle d’outil favorisant l’orientation vers la performance est reconnue, la réserve de

performance est considérée comme un moyen trop bureaucratique pour y parvenir. Il conviendra

de tenir compte de ces critiques et d’autres écueils dans tout futur système fondé sur la

performance, lequel devra être proportionné et adéquatement réactif. La Commission européenne

devrait envisager d’autres moyens permettant aux programmes et aux régions de s’adapter

rapidement aux crises avec souplesse et réactivité. Il sera également nécessaire de ménager un

équilibre avec le besoin de stabilité en matière de planification des investissements stratégiques,

que facilite la période de programmation de sept ans12. Les dispositions et les possibilités de

reprogrammation actuelles pourraient être légèrement modifiées afin de permettre

l’établissement de bilans intermédiaires tenant compte des problèmes et progrès de performance

régionaux, des circonstances intérieures et de la situation socio-économique, ainsi que des

exigences en matière de calendrier. Ces bilans pourraient ainsi jeter les fondements de

reprogrammations d’envergure.

Les recherches montrent que les dépenses des fonds ESI sont davantage axées sur les priorités

de la stratégie Europe 2020 que pendant les périodes de programmation précédentes en ce qui

concerne les objectifs de l’Union alors fixés. Toutefois, la Cour des comptes européenne et d’autres

acteurs estiment que les conditions ex ante et la réserve de performance sont des instruments

innovants, mais qu’ils ne permettent pas encore de garantir efficacement de bons résultats à l’égard

de ces priorités et d’autres.

Recommandation no 11: il convient de soutenir l’évaluation des performances orientée vers

les résultats en s’attachant davantage à déterminer comment les programmes des fonds ESI

contribuent aux objectifs globaux de l’Union. Les accords de partenariat utilisés comme base de

11 La période de programmation 2007-2013 permettait aux États membres d’établir volontairement une réserve deperformance.12 http://data.consilium.europa.eu/doc/document/ST-14216-2018-INIT/fr/pdf


32

la mise en œuvre des programmes du FEDER et du FSE comprennent des objectifs plus vastes de

l’Union. Il ne serait pas réaliste de penser que le système d’indicateurs peut contribuer à déterminer

la mesure dans laquelle les interventions du FEDER et du FSE servent les objectifs de l’Union. Il est

toutefois possible d’établir un lien plus clair entre ces interventions et les objectifs de l’Union, et de

l’examiner au moyen d’évaluations ex ante et ex post. L’absence d’une stratégie plus complète et

d’un ensemble cohérent d’objectifs globaux au niveau de l’Union pourrait constituer un obstacle à

l’établissement de ce lien pour la période 2021-2027.

Cadre réglementaire pour la période 2021-2027

Le règlement portant dispositions communes de la politique de cohésion pour la période 2021-2027

répond à nombre des questions examinées dans la présente étude. Nous en fournissons un résumé ci-

après.

Questions soulevées dans la présente étude dont il est tenu compte dans le règlementportant dispositions communes pour la période de programmation 2021-2027

Les indicateurs du FSE+ n’ont pas fait l’objet de modifications majeures, ce qui permet d’assurerla continuité. Cette décision est conforme à notre point de vue selon lequel le nombred’indicateurs utilisés ne doit pas augmenter, sauf en cas de nécessité absolue et de possibilitéde recueillir des informations utiles sur les performances.

Les indicateurs communs du FEDER sont deux fois plus nombreux. Il s’agit toutefois d’uneamélioration considérable, le nouvel ensemble permettant d’indiquer de manière pluscomplète les interventions présentant des lacunes de financement. L’ajout d’indicateurs deréalisation et de résultat constitue une nette amélioration par rapport aux périodes deprogrammation précédentes, quelques indicateurs déployés depuis longtemps ayantjudicieusement changé de catégorie, passant de «réalisation» à «résultat».

Les indicateurs spécifiques par programme sont encore autorisés, ce qui témoigne d’unesouplesse adéquate en matière de programmation, mais l’obligation d’intégrer tous lesindicateurs au cadre de performance pourrait réduire leur nombre.

L’obligation réglementaire d’intégrer à la méthode d’élaboration du cadre de performanceles critères appliqués, les données et les informations consultées en vue de la sélection desindicateurs et des objectifs, l’assurance qualité, ainsi que tout autre facteur susceptible d’influersur les réalisations du programme, devrait améliorer la fixation des objectifs et garantir unniveau d’ambition adéquat.

L’exigence d’augmenter la fréquence de la communication des données permettrad’améliorer l’actualité des données. Toutefois, la communication sur une base bimestriellerisque d’être trop fréquente pour les autorités de gestion, et il pourrait être envisagé deprésenter les données tous les quatre ou six mois.

La possibilité d’utiliser des sources de données nationales aux fins de la collecte desinformations relatives aux réalisations du FSE devrait permettre d’accroître la qualité desdonnées, de réduire la charge administrative et d’améliorer le lien ainsi que la concordance duFSE+ avec les interventions nationales.

La suppression de la réserve de performance est judicieuse.

Questions soulevées dans la présente étude dont il n’est pas pleinement tenu compte dansle règlement portant dispositions communes pour la période

de programmation 2021-2027


____________________________________________________________________________________________

33

La nouvelle approche de programmation «5+2» pourrait constituer une démarche plusstratégique grâce aux examens à mi-parcours, mais elle est probablement encore tropbureaucratique et axée sur la réalisation. Elle pourrait également être quelque peu rigide etdisproportionnée au regard de la charge administrative qu’elle pourrait générer. Dans lespropositions, aucun fonds ne se voit associer l’obligation d’utiliser des indicateurs d’impact.Cette absence va à l’encontre des lignes directrices pour une meilleure réglementation et, selonla Cour des comptes européenne, elle est également contraire au règlement financier.

L’absence d’obligation d’effectuer des évaluations ex ante supprime un élément essentielde l’élaboration des indicateurs et du cadre de performance, ainsi que de la garantie de leurpertinence, de leur adéquation, de leur réalisme et de leur évaluation chiffrée. Cette absence setraduit également par l’impossibilité d’effectuer une évaluation stratégique de la logique despolitiques et des interventions au regard des besoins régionaux en matière de développement.

Les orientations et les définitions relatives aux nouveaux ensembles d’indicateurs devrontêtre disponibles bien avant l’adoption des programmes opérationnels de la période 2021-2027.


34

ZUSAMMENFASSUNG

Überblick

Im Programmplanungszeitraum 2014-2020 wurden etwa 352 Mrd. EUR aus dem Europäischen

Fonds für die regionale Entwicklung (EFRE) und dem Europäischen Sozialfonds (ESF) für die

Kohäsionspolitik der EU bereitgestellt. Es ist zweifellos wichtig, die Maßnahmen im Interesse der

Rechenschaftspflicht zu überwachen und zu bewerten, aber auch einen möglichst effizienten und

wirksamen Einsatz der EU-Mittel sicherzustellen.

Mit dieser Forschungsarbeit für das EP sollte vor allem untersucht werden, wie verlässlich sich

mit den wichtigsten Leistungsindikatoren für Konvergenz in der regionalen Entwicklung ein

gezielter und ergebnisorientierter Mitteleinsatz sicherstellen lässt. Zu diesem Zweck sollen in der

Forschungsarbeit das derzeitige System gemeinsamer Indikatoren innerhalb der Kohäsionspolitik der

EU einschließlich seiner Stärken, Schwächen und sonstiger relevanter Faktoren analysiert werden. Mit

Blick auf die Zukunft enthält das Forschungspapier Vorschläge für mögliche Änderungen an der

derzeitigen Politik, um die Gesamtergebnisse der Kohäsionspolitik der EU zu verbessern. Der Auftrag

wurde zu Beginn des Jahres 2019 für die Direktion D (Haushaltsangelegenheiten) des EP,

Generaldirektion Interne Politikbereiche der Union, vom Centre for Strategy & Evaluation Services

(CSES) ausgeführt13.

Insgesamt wurden dem Forschungspapier zufolge gute Fortschritte bei der Entwicklung eines

wirksamen Systems von Leistungsindikatoren erreicht, mit denen der Beitrag der

Kohäsionspolitik zur Konvergenz bei der regionalen Entwicklung überwacht wird. Es liegen

nunmehr sorgfältig ausgearbeitete Outputindikatoren für den EFRE und den ESF vor. Allerdings stellt

sich weiterhin die Herausforderung, den Schwerpunkt von der Messung der finanziellen Inputs und

Outputs auf die Bewertung der aussagekräftigeren Ergebnisse und Auswirkungen von Interventionen

zu verlagern. Es bleibt noch viel zu tun, um Datenerfassungssysteme zu entwickeln, mit denen die Art

von Informationen gewonnen werden kann, die für die Bewertung von Ergebnissen und Auswirkungen

benötigt werden, ohne dass ein unverhältnismäßiger Verwaltungsaufwand für nationale Behörden

und die Begünstigten der Kohäsionspolitik entsteht.

Wichtigste Schlussfolgerungen und Empfehlungen

Zunächst werden die Hauptmerkmale des Rahmens für die Messung der Leistung von EFRE und ESF

zusammengefasst, anschließend wird bewertet, wie wirksam dies für das Erreichen

ergebnisorientierter Maßnahmen war.

Rahmen für die Messung der Leistung von EFRE und ESF

Mit der Verordnung (EU) Nr. 1303/2013 mit gemeinsamen Bestimmungen für die Kohäsionspolitik im

Programmplanungszeitraum 2014-2020 wurden umfangreiche Änderungen eingeführt, um die

13 Consortium Partner Blomeyer & Sanz übernahm die Qualitätskontrolle.


____________________________________________________________________________________________

35

Ausrichtung auf die Leistung zu erhöhen und einen stärker ergebnisorientierten Ansatz zu fördern, der

aus den folgenden Hauptkomponenten besteht:

Hauptmerkmale der Ergebnisorientierung

Ex-ante-Konditionalitäten – sie wurden ausgehandelt, um sicherzustellen, dass die

notwendigen Bedingungen für eine Schwerpunktlegung auf wichtige Leistungsindikatoren

und eine erfolgreiche Durchführung des Programms gegeben sind;

Begründung für die Intervention und logischer Rahmen – dadurch wird die

Ergebnisorientierung von Programmen durch die Verknüpfung der zu berücksichtigenden

Bedürfnisse mit den zu erreichenden Ergebnissen und die Einbeziehung von

Ergebnisindikatoren mit Definitionen und messbaren Zielen sichergestellt;

Leistungsrahmen – Festlegung klarer und messbarer Etappenziele und Zielwerte, um

sicherzustellen, dass die geplanten Fortschritte erzielt werden (Leistungsrahmen), mit einer

leistungsgebundenen Reserve, die zur Belohnung für das Erreichen von Ergebnissen

verwendet wird.

In diesem Forschungspapier wird in erster Linie der Leistungsrahmen untersucht, insbesondere die

gemeinsamen und programmspezifischen Indikatoren, die verwendet werden, damit gezielte und

ergebnisorientierte Maßnahmen sichergestellt und die Folgen und Ergebnisse der Politik vor Ort

bewertet werden können. Untersuchungen zeigen, dass mit dem derzeit bestehenden System

durch weitere Verfeinerung ein zielgerichteter und ergebnisorientierter Ansatz für die

Umsetzung der EFRE- und ESF-Programme erreicht werden kann.

Bei der Umsetzung des Leistungsrahmens sollten die Kommission und die Mitgliedstaaten

zusammenarbeiten, um Ausgangspositionen (beispielsweise die Lage in einer Region im Hinblick auf

Kompetenzen oder im Zusammenhang mit Unternehmensgründungen) und Ziele (z. B. wie die Lage

mit EFRE- und ESF-Maßnahmen verbessert werden könnte) festzulegen; dabei werden Bewegungen

anhand von Indikatoren gemessen und zur Bewertung der Fortschritte bei der Umsetzung

herangezogen.

In den Leitlinien „Bessere Rechtsetzung“ (2015 und 2017) der Kommission sind drei grundlegende

Arten von Indikatoren festgelegt: Outputindikatoren, mit denen die spezifischen Leistungen der

Interventionen gemessen werden (z. B. Zahl der KMU, die EFRE-Zuschüsse erhalten, Zahl der

Teilnehmer an ESF-Schulungen); Ergebnisindikatoren, d. h. unmittelbare Auswirkungen einer

Maßnahme unter Berücksichtigung der Begünstigten (z. B. bessere Leistung von KMU, Qualifikationen

von Teilnehmern am ESF) und Wirkungsindikatoren, die sich auf die Auswirkungen auf die gesamte

Wirtschaft und Gesellschaft beziehen (z. B. Schaffung von Arbeitsplätzen, bessere

Wettbewerbsfähigkeit). Outputs haben physischen Charakter und sind relativ leicht zu messen,

Ergebnisse hingegen haben sozioökonomischen Charakter und sind schwieriger zu bewerten. Ebenso

gilt, dass ESF-Maßnahmen auf Einzelpersonen ausgerichtet sind, wohingegen der EFRE eine

heterogenere Zielgruppe mit weniger homogenen Messeinheiten hat.

Wichtig ist, dass ein wirksamer Leistungsrahmen nicht nur von geeigneten Indikatoren, sondern auch

davon abhängt, dass die Datenerfassungsmechanismen in Programmverwaltungssysteme


36

eingebettet sind, mit denen die erforderlichen Informationen über Interventionen und ihre

Auswirkungen gewonnen werden können.

Entwicklung des Leistungsrahmens

In jedem Programmplanungszeitraum gab es deutliche Verbesserungen beim Ansatz für die

Überwachung und Bewertung der ESI-Fonds. Die wichtigsten Entwicklungen des Rechtsrahmens

der ESI-Fonds, die im Laufe der Jahre stattgefunden haben, sind nachstehend zusammengefasst.


____________________________________________________________________________________________

37

Abbildung 0.1: Entwicklung des Rechtsrahmens der ESI-Fonds (2000-2027)


38

Zusammenfassend lässt sich sagen, dass es vor 2000 keinen gemeinsamen Ansatz für die Festlegung

von Indikatoren und Zielen gab und die Mitgliedstaaten eigene auswählen konnten, was dazu führte,

dass in den EFRE- und ESF-Programmen eine Vielzahl von Indikatoren mit sehr geringer

Harmonisierung verwendet wurde. Es wurden nicht nur verschiedene Indikatoren von verschiedenen

Mitgliedstaaten und Regionen verwendet, sondern auch bei Verwendung derselben Indikatoren

variierten die Definitionen und die Methoden zu ihrer Umsetzung. Daher war es schwierig, wenn nicht

gar unmöglich, die Informationen zu den Ergebnissen der Politik auf EU-Ebene zusammenzufassen.

Verwendung gemeinsamer und programmspezifischer Indikatoren

Die laufenden Mainstream-Programme 2014-2020 des EFRE und ESF werden mit

72 gemeinsamen Output- und Ergebnisindikatoren überwacht. Gleichzeitig verwenden die

Verwaltungsbehörden eine große Zahl programmspezifischer Indikatoren. Dies zeigt, dass die

gemeinsamen Indikatoren zwar in den meisten Fällen für alle Mitgliedstaaten und Regionen relevant

sein dürften, es aber möglicherweise anderer Indikatoren bedarf, in denen die regionalen

Besonderheiten deutlicher zum Ausdruck kommen (es gibt beispielsweise Fälle, in denen Indikatoren

nicht relevant sind – die Länge der Eisenbahnstrecke in km ist in Mitgliedstaaten ohne Eisenbahnnetz

wie Zypern und Malta nicht von Belang). Als deutlicher Trend zeichneten sich im Laufe der Jahre die

Bemühungen um eine Förderung der Vereinfachung ab, wobei die Zahl der verschiedenen Indikatoren

schrittweise konsolidiert wurde.

Beim ESF gibt es derzeit 32 gemeinsame Indikatoren (23 für Outputs und 9 für Ergebnisse), die

das Minimum darstellen, das in jedes operationelle Programm (OP) aufgenommen werden

sollte. Die am häufigsten verwendeten Indikatoren beziehen sich auf die Zahl der erwerbstätigen

Teilnehmer in aus dem ESF-finanzierten Programmen, gefolgt von der Zahl der arbeitslosen,

benachteiligten Einzelpersonen und Jugendlichen. Insgesamt werden 16 895 Output- und

3 556 Ergebnisindikatoren in den 180 OP des ESF im laufenden Programmplanungszeitraum

verwendet (es sei darauf hingewiesen, dass einige Programme Multifonds-Programme sind und unter

den Zahlen sowohl des ESF als auch des EFRE erfasst sind). Während nur etwa 15 % der in den OP des

ESF verwendeten Indikatoren gemeinsame Indikatoren sind, entsprechen die programmspezifischen

Indikatoren größtenteils Unterkategorien oder Kombinationen von Kategorien, die in den

gemeinsamen Indikatoren verwendet werden. Neben den Outputindikatoren wurden für den ESF

gemeinsame Indikatoren für „unmittelbare“ und „langfristigere“ Ergebnisse festgelegt.

Im Falle der EFRE-Mainstream-Programme (d. h. ausgenommen der Europäischen territorialen

Zusammenarbeit) bestehen für die OP derzeit insgesamt 40 gemeinsame Outputindikatoren für

den laufenden Zeitraum 2014-2020. Diese wurden 5 757 Mal in den 201 OP verwendet, machen

etwa 55 % aller verwendeten Indikatoren aus und decken etwa den gleichen Betrag der Investition ab.

Die am häufigsten verwendeten Indikatoren beziehen sich auf die Zahl der unterstützten

Unternehmen und den Wert von Zuschüssen, den Beschäftigungszuwachs in unterstützten

Unternehmen und die Verminderung von Treibhausgasemissionen. In einigen Fällen, in denen bei den

OP keine gemeinsamen Indikatoren verwendet wurden, wurde darauf hingewirkt, Indikatoren auf EU-

Ebene zu verwenden (d. h. Indikatoren, die sich aus dem Innovationsanzeiger ergeben), um die


____________________________________________________________________________________________

39

Messbarkeit und Vergleichbarkeit zu verbessern. Die programmspezifischen Indikatoren des EFRE

können in der Regel nicht aggregiert werden und wurden innerhalb der Leistungsrahmen der OP wohl

zu stark genutzt. Im Gegensatz zum ESF wurden beim EFRE keine gemeinsamen Indikatoren

ausgearbeitet, die bestehenden Indikatoren sind programmspezifisch.

Insgesamt hat die Verwendung verbindlicher gemeinsamer Indikatoren für Outputs (und auch

für Ergebnisse beim ESF) auf der Grundlage standardisierter Definitionen dazu beigetragen, die

Interventionslogik des Programms zu stärken, d. h. wie die geplanten Ausgaben zum Erreichen von

Veränderungen im Rahmen des festgestellten Entwicklungsbedarfs beigetragen haben, und ist Teil des

Verhandlungsprozesses für Programme. Die Transparenz, Sichtbarkeit und Rechenschaftspflicht der

Politik haben sich verbessert. Darüber hinaus fördert die Festlegung klarer und messbarer Indikatoren

und Ziele als Teil der Interventionslogik zu Beginn eines Programms eine bessere Ausrichtung von

Verwaltungsbehörden und regionalen/nationalen Akteuren auf Ergebnisse und Fortschritte bei der

Leistung.

Die derzeitigen gemeinsamen Indikatoren des EFRE und des ESF sind in der Regel gut konzipiert

und den Zielen der Kohäsionspolitik angemessen, es bestehen jedoch Mängel, die behoben

werden müssen. Während in den programmspezifischen Indikatoren des ESF in der Regel

Unterkategorien oder Kombinationen von Kategorien zum Ausdruck kommen, die im Rahmen der

gemeinsamen Indikatoren verwendet werden, wird bei den EFRE-Programmen auf sehr viele

Indikatoren zurückgegriffen, die in der Regel nicht aggregiert werden können. Das System bringt einen

beträchtlichen Verwaltungsaufwand und Kosten bei zuweilen geringer Qualität und Korrektheit mit

sich. Eine Vereinfachung und Harmonisierung ist zwar im Hinblick auf eine Verringerung des

Verwaltungsaufwands und eine Verbesserung der Rechenschaftspflicht zu begrüßen, sollte jedoch

nicht zu einer unflexiblen Programmplanung oder einem Einheitsansatz führen.

Empfehlung 1: Die Vereinfachung des Systems gemeinsamer Indikatoren für die ESI-Fonds

sollte soweit wie möglich fortgesetzt werden. Der Grundsatz der Verhältnismäßigkeit muss

berücksichtigt werden; notwendig ist ein strafferer Ansatz bei der Verwendung von Indikatoren

und der Erfassung von Leistungs-/Überwachungsdaten. Gemeinsame und programmspezifische

Indikatoren sollten zur Überwachung von ESI-Fonds-Programmen nur dann genutzt werden, wenn

mit ihnen zweckdienliche Informationen zur Leistung gewonnen werden können. Die derzeitige

Gesamtzahl von 72 gemeinsamen Output- und Ergebnisindikatoren dürfte angemessen sein und

sollte nicht erhöht werden, sofern dies nicht unbedingt notwendig ist. Allgemein deuten die

Rückmeldungen aus der Forschung darauf hin, dass die Relevanz, die Ausrichtung, der

Erfassungsbereich, die Messbarkeit, die Genauigkeit und die Zweckmäßigkeit der Indikatoren und

der Datenerhebungssysteme noch weiter verbessert werden können.

Empfehlung 2: Die bestehenden gemeinsamen Indikatoren für ERFE und ESF sollten

überprüft werden, damit eine umfassende Abdeckung der kohäsionspolitischen Ziele

erreicht werden kann. Beim Erfassungsbereich bestehen einige Lücken, beispielsweise decken die

bestehenden gemeinsamen Indikatoren nicht vollständig die Arten der unterstützten Tätigkeiten


40

(z. B. IKT) ab; ferner gibt es Probleme im Zusammenhang mit der Messbarkeit (z. B.

Treibhausgasemissionen und Energieeinsparung), Zweckmäßigkeit (z. B. Anzahl der Projekte,

Anzahl der Besucher), Genauigkeit (Doppelzählung) sowie Widersprüchlichkeiten oder

Unstimmigkeiten bei der Auslegung (z. B. Definitionen zusammenarbeitender Unternehmen und

Industriebrachen) und Aggregierung (z. B. Zeitpunkt der Meldung von Daten, Meldung nicht

kumulierbarer Zahlen, Meldung sensibler Daten).

Empfehlung 3: Bei den spezifischen Indikatoren, die für die EFRE- und ESF-Programme

verwendet werden, sollte die Kommission mit den nationalen und regionalen Behörden an

einer Überprüfung des Geltungsbereichs arbeiten, um eine stärkere Harmonisierung zu

erreichen. Einer Schätzung des Europäischen Rechnungshofes (EuRH) zufolge werden bei den

laufenden Programmen im Zeitraum 2014-2020 zwischen 8 000 und 9 000 programmspezifische

Indikatoren verwendet. Eine umfassendere Verwendung gemeinsamer statt spezifischer

Indikatoren würde es leichter machen, die auf Programm- und EU-Ebene erreichten Ergebnisse und

Auswirkungen zu ermitteln.

Als deutlicher Trend zeichnete sich bei den Überwachungs- und Evaluierungsrahmen des EFRE

und des ESF im Laufe der Jahre eine Verlagerung des Schwerpunkts von der Mittelabsorption

und den Outputs hin zu einem ergebnisorientierten Ansatz ab. Die Entwicklung gemeinsamer

Ergebnisindikatoren ist beim ESF wohl weiter vorangeschritten als beim EFRE, dieser Unterschied wird

jedoch in den Rechtsetzungsvorschlägen für den Zeitraum 2021-2027 berücksichtigt. Im Hinblick auf

die Wirkung ist weder im EFRE noch im ESF vorgesehen, dass gemeinsame Wirkungsindikatoren im

laufenden Programmplanungszeitraum überwacht werden müssen. Es bestehen zwar Leitlinien der

Kommission zur Ausarbeitung von Indikatoren und Methoden, die zur Bewertung der Auswirkungen

verwendet werden können, ihre Verwendung ist aber nicht vorgeschrieben. Stattdessen werden die

Auswirkungen durch Ex-Post-Evaluierungen bewertet, für die üblicherweise die Kommission zuständig

ist.

Empfehlung 4: Vorrang sollte ein stärkerer Schwerpunkt auf der Bewertung der

„Ergebnisse“ und „Auswirkungen“ haben, um eine stärkere Anpassung an die Grundsätze

der „besseren Rechtsetzung“ zu erreichen. Wirkungsindikatoren sind im laufenden und waren

in vorangegangenen Programmplanungszeiträumen nicht vorgesehen und werden auch in den

Vorschlägen für den EFRE und den ESF für die Zeit nach 2020 wieder nicht berücksichtigt. Wie im

Bericht festgestellt, bestehen für den ESF bereits gemeinsame Ergebnisindikatoren für

„unmittelbare“ und „längerfristige“ Folgen. Ähnliche gemeinsame Ergebnisindikatoren werden für

den EFRE benötigt, im Vorschlag zum Fonds für die Jahre 2021-2027 wird ein begrüßenswerter

Schritt in diese Richtung unternommen. In beiden Fällen muss die Fähigkeit entwickelt werden,

Auswirkungen wirksamer zu bewerten.

Die Mitgliedstaaten haben bei der Einbeziehung des ergebnisorientierten Ansatzes in ihre

Verfahren der Projektauswahl beträchtliche Fortschritte erreicht. Wie es in einem Bericht des

EuRH von 2018 heißt, muss die Ausrichtung auf Ergebnisse jedoch noch vollständig in diese


____________________________________________________________________________________________

41

Verfahren integriert werden. Relevante Indikatoren, die für die Projekte verwendet und zu denen sie

beitragen sollten, sind in der Dokumentation der Aufforderung zur Einreichung von Vorschlägen und

den Auswahlkriterien für Projekte in einer Minderheit der Mitgliedstaaten enthalten, werden aber

größtenteils nicht in Zuschussverträgen festgelegt. Es gibt zunehmend Bemühungen, Zahlungen an

Ergebnisse zu koppeln, dies sollte weiterentwickelt werden, wobei allerdings Flexibilität notwendig ist,

um der Komplexität der Programmplanung Rechnung zu tragen.

Empfehlung 5: Die Bemühungen darum, die gemeinsamen EFRE- und die gemeinsamen ESF-

Indikatoren stärker auf „Ergebnisse“ und „Auswirkungen“ auszurichten, müssen auch in den

Verfahren zur Bewertung und Auswahl der Projekte zum Ausdruck kommen. Bei der

Konzipierung der Auswahlverfahren stehen Outputs und Absorption weiterhin eher im

Vordergrund als Ergebnisse: Quantifizierte Ergebnisindikatoren werden selten in

Finanzhilfevereinbarungen aufgenommen, während die Projektauswahl in der Regel nach dem

Windhundverfahren erfolgt.

Insgesamt gibt es deutliche Hinweise darauf, dass beim Leistungsrahmen für die ESI-Fonds und

beim Indikatorensystem der Schwerpunkt stärker auf die Leistung verlagert wurde, aber

uneinheitliche Hinweise darauf, dass Zielwerte und Etappenziele so festgelegt sind, dass ein

angemessenes Gleichgewicht zwischen realistischen und anspruchsvollen Vorgaben erreicht

wird. Indikatoren, Zielwerte und Etappenziele werden von den Mitgliedstaaten und Regionen anhand

ihrer früheren und individuellen Erfahrungen mit Ergebnissen der Programmplanung festgelegt und

geben historische Referenzdaten wieder. Sie werden anschließend mit der Kommission im Rahmen der

„geteilten Mittelverwaltung“ und Partnerschaft ausgehandelt. Ziele und Etappenziele wurden

tendenziell vorsichtig festgelegt, da die Gefahr besteht, dass die Ziele der leistungsgebundenen

Reserve nicht erreicht werden und somit finanzielle Sanktionen verhängt werden. Derzeit wird weder

in den EFRE- noch in den ESF-Leitlinien erläutert, wie geeignete Etappenziele und Zielwerte abgesehen

von der Berücksichtigung des Bedarfs und der Mittel in den Regionen festgesetzt werden sollten. Die

vorgeschlagene Dachverordnung für den Zeitraum 2021-2027 sieht vor, dass die Methodik des

Leistungsrahmens die angewandten Kriterien und die bei der Auswahl der Indikatoren und Ziele und

der Qualitätssicherung verwendeten Daten/Erkenntnisse sowie alle zusätzlichen Faktoren, die die

Ergebnisse des Programms beeinflussen können, umfassen muss; dies stellt eine begrüßenswerte

Verbesserung dar. Darüber hinaus wurde die vorgeschriebene Ex-ante-Bewertung gestrichen, die

immer eine wichtige Rolle bei der Festlegung von Indikatoren/des Leistungsrahmens und der

Sicherstellung ihrer Relevanz, Eignung, ihrer Realitätsnähe und ihrer Quantifizierung gespielt hat.

Empfehlung 6: Zur Rolle der Ex-ante-Bewertung bei der Sicherstellung ergebnisorientierter

EFRE- und ESF-Programme sollten weitere Überlegungen angestellt werden. Die Streichung

der Verpflichtung zur Durchführung einer Ex-ante-Bewertung im Zeitraum 2021-2027 ist wohl ein

Fehler. Frühere Erfahrungen zeigen, dass Ex-ante-Bewertungen insofern äußerst nützlich sein

können, dass kohärente Ziele und Ergebnisse festgelegt werden und die geplanten Maßnahmen

voraussichtlich wirksam und effizient zu den Zielen der Kohäsionspolitik beitragen und einen EU-

Mehrwert sicherstellen. Ex-ante-Bewertungen können sich auch als hilfreich bei der Bewertung


42

verschiedener Aspekte, z. B. der Frage, was unter welchen Umständen funktioniert, sowie bei der

Bewertung der Nettowirkung oder der Auswirkungen der Politik erweisen und ermöglichen eine

eingehende Bewertung der Qualität von Maßnahmen.

Empfehlung 7: Es sollte mehr unternommen werden, um die Mitgliedstaaten und Regionen

dabei zu unterstützen, solide und angemessen anspruchsvolle Ziele des EFRE und des ESF

festzulegen. Bei den Vorschlägen für den Zeitraum 2021-2027 werden methodische Ansätze

weiterentwickelt und auch dem Ansatz der „geteilten Mittelverwaltung“ und dem

Subsidiaritätsprinzip Rechnung getragen. Weitere methodische Leitlinien und Unterstützung

sowie geeignete Kapazitäten zur Bewältigung dieser Anforderung sollten sichergestellt sein,

insbesondere in Anbetracht des vorgeschlagenen Wegfalls einer Ex-ante-Bewertung.

Überwachungssysteme für die ESI-Fonds

Gute Fortschritte wurden bei der Entwicklung wirksamer Überwachungssysteme für die ESI-

Fonds erreicht: Die meisten Mitgliedstaaten stellen zentrale Orientierungs- und IT-Instrumente

zur Sicherstellung der Kohärenz bereit und auf EU-Ebene wurden integrierte

Plausibilitätsprüfungen eingeführt. Im laufenden Programmplanungszeitraum 2014-2020 wurde

stärker darauf geachtet, dass die für die gemeinsamen EFRE- und ESF-Indikatoren erforderlichen

Statistiken in den EU-Mitgliedstaaten und -Regionen vergleichbar sind. Die Einführung der „Ex-ante-

Konditionalität 7“ leistete einen wichtigen Beitrag dazu, dass der Leistungsrahmen der ESI-Fonds durch

ein angemessenes statistisches System und eine gemeinsame Definition von Indikatoren sowie der

dafür benötigten Angaben unterstützt wird.

Aus den Bewertungen der Kommission und des Europäischen Rechnungshofes ergibt sich

jedoch ein gemischtes Bild mit weiterhin bestehenden Mängeln bei der Datenerfassung. Viele

der Informationen, die zur Bewertung von Ergebnissen und Auswirkungen auf EFRE- und ESF-

Programmebene benötigt werden, stammen von den an der Projektdurchführung Beteiligten.

Rückmeldungen aus der Forschung in den Mitgliedstaaten zeigen, dass die Begünstigten häufig

vertraglich verpflichtet sind, den Verwaltungsbehörden Informationen zu Ergebnissen vorzulegen.

Dies gilt jedoch nicht in allen Fällen. Zudem liegen die Ergebnisse der Programme nicht immer der für

die gemeinsamen Indikatoren erforderlichen Datenerhebungsstelle vor (z. B. sechs Monate nach

Beendigung der Programmteilnahme eines Empfängers von ESF-Förderung); bei einigen der für die

gemeinsamen ESIF-Indikatoren in bestimmten Mitgliedstaaten erforderlichen Überwachungsdaten

handelt es sich dagegen um sensible Daten (dies betrifft insbesondere einige Kategorien von ESF-

Begünstigten in Ländern, in denen es strenge Datenschutzgesetze gibt). Andere festgestellte Mängel

betreffen Aspekte wie schlichtes menschliches Versagen, falsche Messeinheiten, Doppelzählung oder

nicht vorliegende Prüfberichte. Ein sehr uneinheitliches Bild ergibt sich auch im Hinblick auf andere

Methoden der Datenerfassung für die Bewertung von Ergebnissen und Auswirkungen, beispielsweise

bei der Praxis der Durchführung von Umfragen, die in einigen Mitgliedstaaten sehr verbreitet ist, in

anderen jedoch nicht.


____________________________________________________________________________________________

43

Empfehlung 8: Es müssen weitere Schritte unternommen werden, um die Aktualität,

Genauigkeit und Durchführbarkeit der Datenerhebungssysteme, die dem Leistungsrahmen

für EFRE- und ESF-Programme zugrunde liegen, zu verbessern. Die Aktualität der

Leistungsdaten zu den ESI-Fonds sollte verbessert werden, dabei muss sichergestellt werden, dass

dadurch kein unangemessener zusätzlicher Arbeitsaufwand für die nationalen Behörden entsteht.

Die von der Kommission gespeicherten Leistungsdaten können aufgrund des Zeitrahmens für den

jährlichen Umsetzungsbericht zwei Jahre alt sein. Die Vorschläge für den Zeitraum 2021-2027

sehen eine Datenerfassung nahezu in Echtzeit vor, die Überwachungsdaten sollen alle zwei Monate

eingereicht werden. Eine jährliche Berichterstattung ist zwar nicht häufig genug, ein

zweimonatiger Zyklus stellt jedoch das andere Extrem dar und könnte sich für die

Verwaltungsbehörden als zu häufig erweisen. Als Alternative sollte eine Einreichung alle vier oder

sechs Monate erwogen werden. In den Rechtsetzungsvorschlägen für den Zeitraum 2021-2027 ist

die Möglichkeit vorgesehen, nationale Datenquellen zur Erfassung von Leistungen des ESF zu

verwenden. Die Nutzung bestehender nationaler Verwaltungsregister/Datenreihen (z. B.

Sozialversicherungsregister) dürfte zu einer Verbesserung der Datenqualität sowie zur

Verbesserung der Verknüpfung und Angleichung des ESF+ an nationale Maßnahmen führen.

Im Rahmen der „geteilten Mittelverwaltung“ der ESI-Fonds-Programme haben die Kommission

und die Mitgliedstaaten bei der Umsetzung der EFRE- und ESF-Programme in den Jahren 2014-

2020 eng zusammengearbeitet. Rückmeldungen aus der Forschung zeigen, dass die

partnerschaftliche Arbeit im Allgemeinen gut funktioniert hat. Die Entwicklung verbesserter IT-

Instrumente hat die Datenübermittlung an die Kommission vereinfacht. Es gab zahlreiche weitere

Initiativen zur Förderung effizienterer und wirksamerer Programmverwaltungs- und -

überwachungssysteme. Es bestehen jedoch weiterhin Probleme, die teilweise damit

zusammenhängen, dass die Rechtsgrundlage und andere Vereinbarungen für Überwachungssysteme

im Zeitraum 2014-2020 nicht früh genug geschaffen wurden.

Empfehlung 9: Die unterstützenden Systeme und Bestimmungen für den

Leistungsrahmen 2021-2027 sollten so früh wie möglich und vor der Annahme der EFRE- und

ESF-Programme eingeführt werden. Die derzeitige Abstimmung der Kommission mit den

Delegationen der Mitgliedstaaten (MS) zu Indikatoren und deren Festlegung über das

Evaluierungsnetzwerk ist zu begrüßen und sollte fortgesetzt werden; ein Beobachterstatus für die

anderen betreffenden Institutionen wie das EP und den EuRH ist zu erwägen (wenn nicht bereits

vorhanden). Das EP sollte einen Forschungsbericht zu einem frühen Zeitpunkt im

Programmplanungszeitraum vorsehen, um festzustellen, inwieweit der Leistungsrahmen für die

Jahre 2021-2027 stärker auf Ergebnisse und Auswirkungen ausgerichtet ist.

Leistungsgebundene Reserve und Beitrag zu allgemeineren politischen Zielen der EU

Durch die verbesserte Ergebnisorientierung und die verbesserten Indikatoren bei der

Programmplanung der ESI-Fonds verlagert sich zwar der Schwerpunkt zunehmend auf die Leistung,

mit dem Leistungsrahmen und der leistungsgebundenen Reserve als Planungsinstrumenten werden


44

Programmverwalter jedoch nicht aufgrund der Ergebnisse von Maßnahmen, sondern aufgrund der

Geschwindigkeit der Umsetzung belohnt; dies wird bereits von der N+2/N+3-Regel zur automatischen

Aufhebung der Mittelbindung beeinflusst.14

Im Hinblick auf die Zukunft und angesichts der Erfahrungen mit der leistungsgebundenen

Reserve in den Jahren 2014-2020 ist die Entscheidung, diese einzustellen, richtig und wird

unseren Untersuchungen zufolge weitgehend unterstützt. Durch die leistungsgebundene Reserve

wurden stärkere Anreize eingeführt, um das Erreichen von Outputs zu fördern. Die

leistungsgebundene Reserve wird vor dem Hintergrund der Möglichkeiten der Programmneuplanung

jedoch praktisch bedeutungslos, nur wenige Mitgliedstaaten dürften ihre Mittel verlieren, die lediglich

für Prioritäten oder Programme mit besserer Leistung neu zugewiesen werden. Der bisherige Prozess

gilt als äußerst politischer und administrativer Prozess und berücksichtigt nicht die Komplexität der

Programmplanung vor Ort. Dies wurde in einem weiteren Sonderbericht des EuRH hervorgehoben, in

dem festgestellt wurde, dass Leistungsrahmen und leistungsgebundene Reserve wenig Anreize für

eine bessere Ergebnisorientierung der OP schaffen, da sie hauptsächlich auf Ausgaben und Outputs

ausgelegt sind. Mit dem neuen 5+2-Ansatz für die Programmplanung15 könnte zwar ein strategischerer

Ansatz erreicht werden, der jedoch wahrscheinlich weiterhin übermäßig bürokratisch und output-

orientiert ist.

Die Einführung eines stärker qualitativen Überprüfungsverfahrens ist zu begrüßen, es sollte jedoch

verhältnismäßig und angemessen flexibel sein, d. h. einem neuen Entwicklungsbedarf rechtzeitig und

nicht als Teil eines umfangreichen Verwaltungsverfahrens für alle Programme zu einem bestimmten

Zeitpunkt Rechnung tragen. Die derzeitigen Möglichkeiten der Programmneuplanung werden daher

als weitgehend ausreichend betrachtet und könnten durch die Möglichkeit einer

Zwischenüberprüfung ergänzt werden, durch die Regionen das Verfahren entsprechend ihren

speziellen Leistungsproblemen, den Gegebenheiten vor Ort und zeitlichen Anforderungen angehen

könnten.16 Regionen/Mitgliedstaaten könnten einen Anteil der Fördermittel (bis zu einem bestimmten

Höchstanteil) flexibel festlegen, der nicht zugewiesen würde17 und danach aufgrund der

Programmleistung, neuer regionaler Herausforderungen oder eines neuen regionalen Bedarfs und

entsprechenden länderspezifischen Empfehlungen neu programmiert werden könnte.

Mitgliedstaaten mit wenigen Programmen oder geringen Mittelzuweisungen könnten entscheiden,

14 Die N+2/N+3-Regel zur automatischen Aufhebung der Mittelbindung sieht vor, dass alle Finanzierungszusagen der EU, fürdie bis Ende des zweiten/dritten Jahres nach dem Jahr der Mittelbindung kein Auszahlungsantrag vorgelegt wurde,aufgehoben werden. Im Programmplanungszeitraum 2014-2020 galt die N+3-Regel, bei den Rechtsetzungsvorschlägen fürdie Zeit nach 2020 wird die N+2-Regel für die Jahre 2021-2027 gelten, dies ist eine Rückkehr zu den Bestimmungen in der Zeitvor 2013 (für die EU-15).15 Der 5+2-Ansatz im Rahmen der Rechtsetzungsvorschläge für die Zeit nach 2020 bezieht sich auf eine ersteProgrammplanung für fünf Jahre, die Programmplanung der letzten zwei Jahre findet erst nach einer Halbzeitüberprüfungstatt.16 Beispielsweise hat die schottische Regierung im laufenden Programmplanungszeitraum den Prozess derLeistungsüberprüfung in einer sogenannten „frühzeitigen Überprüfung“ auf 2017 vorgezogen, um auf Unsicherheiten bei derProgrammplanung infolge des Brexit zu reagieren.17 Im Programmplanungszeitraum 2007-2013 war eine freiwillige leistungsbezogene Reserve im Ermessen derMitgliedstaaten vorgesehen.


____________________________________________________________________________________________

45

aufgrund des Grundsatzes der Verhältnismäßigkeit auf die Durchführung der Überprüfung und

Programmneuplanung zu verzichten.

Empfehlung 10: Die Entscheidung, die leistungsgebundene Reserve einzustellen, findet

breite Unterstützung, jedem künftigen System, das sie ersetzt, sollte ein stärker

ergebnisorientierter Ansatz zugrundeliegen. Die Outputs der Programme sind zweifellos

wichtig, aussagekräftigere Hinweise auf die Leistung und den Beitrag der Kohäsionspolitik zu den

Zielen der EU liefern jedoch die Ergebnisse. Den durchgeführten Untersuchungen zufolge

betrachtete die Mehrheit der Verwaltungsbehörden den Prozess als einen engen Teilaspekt

dessen, was die Programme tatsächlich erreichen, und als besonders starr und unflexibel in Bezug

auf externe Faktoren und eine innovativere und experimentellere Verwendung der Mittel. Sein

potenzieller Wert als Instrument zur Leistungsorientierung wird zwar anerkannt, er gilt jedoch als

ein zu bürokratisches Mittel, um dies zu erreichen. Diese und weitere Kritikpunkte müssen in einem

künftigen leistungsbasierten System ebenfalls berücksichtigt werden, das verhältnismäßig und

angemessen flexibel sein sollte. Die Kommission sollte andere Mittel prüfen, mit denen Programme

und Regionen rasch an Krisen angepasst werden und Flexibilität und Reaktionsfähigkeit zeigen

können. Ein Gleichgewicht muss auch bei einer notwendigen stabilen strategischen

Investitionsplanung angestrebt werden, die der siebenjährige Programmplanungszeitraum

erleichtert.18 Eine geringfügige Abweichung von den derzeitigen Bestimmungen/Möglichkeiten

für die Programmneuplanung könnte eine Zwischenüberprüfung ermöglichen, die spezifische

regionale Leistungsprobleme und Fortschritte, die Gegebenheiten vor Ort und die

sozioökonomische Lage sowie zeitliche Anforderungen berücksichtigt und auf dieser Grundlage

eine umfassende Programmneuplanung möglich macht.

Rückmeldungen aus der Forschung zeigen, dass die Ausgaben stärker auf die Europa-2020-

Prioritäten ausgerichtet waren, als dies in früheren Programmplanungszeiträumen im Hinblick

auf die damals bestehenden Ziele der EU der Fall war. Der EuRH und andere haben jedoch

argumentiert, dass die Ex-ante-Konditionalitäten und die leistungsgebundene Reserve zwar

innovative, jedoch keine wirksamen Instrumente waren, um gute Ergebnisse im Hinblick auf diese und

andere Prioritäten sicherzustellen.

Empfehlung 11: Die Ausrichtung auf eine ergebnisorientierte Leitungsmessung sollte durch

eine stärkere Betonung der Frage unterstützt werden, wie ESI-Fonds-Programme zu den

allgemeinen politischen Zielen der EU beitragen. In den Partnerschaftsvereinbarungen, die als

Grundlage für die Durchführung von EFRE- und ESF-Programmen verwendet werden, sind auch

allgemeinere politische Ziele der EU vorgesehen. Es wäre unrealistisch zu erwarten, dass mit dem

Indikatorensystem festgestellt werden kann, inwieweit allgemeinere politische Ziele der EU durch

EFRE- und ESF-Maßnahmen gefördert werden. Eine Verbindung zwischen diesen Maßnahmen und

den politischen Zielen der EU könnte jedoch deutlicher hergestellt und in Ex-ante- und Ex-post-

Evaluierungen bewertet werden. Dies im Zeitraum 2021-2027 zu erreichen, könnte durch das

18 http://data.consilium.europa.eu/doc/document/ST-14216-2018-INIT/de/pdf


46

Fehlen einer umfassenderen EU-Strategie und kohärenter, allgemeiner politischer Ziele der EU

verhindert werden.

Rechtsrahmen für die Jahre 2021-2027

In der Verordnung mit gemeinsamen Bestimmungen für die Kohäsionspolitik für den Zeitraum 2021-

2027 werden viele der in dieser Forschungsarbeit untersuchten Fragen behandelt. Auf der folgenden

Seite ist eine Zusammenfassung wiedergegeben.

In der Forschungsarbeit angesprochene Fragen, die in der Verordnung über gemeinsameBestimmungen für den Zeitraum 2021-2027 behandelt werden

ESF+-Indikatoren wurden nicht wesentlich verändert, dies ermöglicht Kontinuität. Dies decktsich mit der Auffassung der Autoren, dass die Zahl der verwendeten Indikatoren nicht erhöhtwerden sollte, sofern dies nicht unbedingt notwendig ist und zweckdienliche Informationenzur Leistung vorgelegt werden können.

Die Zahl der gemeinsamen EFRE-Indikatoren hat sich verdoppelt. Die Änderung stellt jedocheine wesentliche Verbesserung dar, da mehr Indikatoren zur Verfügung stehen, um dieBandbreite der geförderten Interventionen abzudecken. Die Aufnahme von Output- undErgebnisindikatoren stellt eine deutliche Verbesserung gegenüber den früherenProgrammplanungszeiträumen dar; dabei wurden einige seit Langem verwendete Indikatorenentsprechend von Output- zu Ergebnisindikatoren umkategorisiert.

Programmspezifische Indikatoren sind zwar weiterhin möglich und Ausdruck einerangemessenen Flexibilität bei der Programmplanung, durch die Anforderung, dass alleIndikatoren Teil des Leistungsrahmens sein müssen, könnte sich ihre Zahl jedochmöglicherweise verringern.

Durch die rechtliche Anforderung, dass die angewendeten Kriterien in die Methodik desLeistungsrahmens aufgenommen und die bei der Auswahl von Indikatoren und Zielen undder Qualitätssicherung verwendeten Daten/Erkenntnisse sowie alle zusätzlichen Faktoren, diedie Ergebnisse des Programms beeinflussen können, einbezogen werden müssen, dürften dieZielsetzung verbessert und angemessene Ziele sichergestellt werden.

Durch die Auflage einer häufigeren Meldung von Daten wird die Aktualität der Datenverbessert. Eine zweimonatige Berichterstattung dürfte für die Verwaltungsbehörden zu häufigsein, eine Einreichung alle vier oder sechs Monate könnte erwogen werden.

Die Möglichkeit, nationale Datenquellen zur Erfassung der ESF-Ergebnisse zu nutzen, solltezu einer Verbesserung der Datenqualität, zu einer Verringerung des Verwaltungsaufwands undzu einer besseren Verknüpfung und Angleichung des ESF+ mit nationalen Maßnahmen führen.

Die Einstellung der leistungsgebundenen Reserve ist sinnvoll.

In der Forschungsarbeit angesprochene Themen, die in der

Verordnung mit gemeinsamen Bestimmungen für den Zeitraum 2021-2027 nichtvollständig behandelt werden

Der neue 5+2-Programmplanungsansatz könnte zwar einen strategischeren Ansatz imRahmen einer Halbzeitüberprüfung ermöglichen, dürfte aber dennoch zu bürokratisch undoutput-orientiert sowie nicht ausreichend flexibel und unverhältnismäßig im Hinblick auf denVerwaltungsaufwand sein. In den Vorschlägen ist nicht vorgesehen, dass dieWirkungsindikatoren für jeden Fonds verwendet werden müssen. Dies läuft den Leitlinien füreine bessere Rechtsetzung zuwider und widerspricht nach Ansicht des EuRH ebenfalls der


____________________________________________________________________________________________

47

Haushaltsordnung.

Durch das Fehlen einer Ex-ante-Evaluierung entfällt ein wichtiger Aspekt bei der Festlegungvon Indikatoren/des Leistungsrahmens, der ihre Relevanz, Eignung, Realitätsnähe undQuantifizierung sowie die strategische Bewertung der Politik und der Interventionslogik inBezug auf den Bedarf an regionaler Entwicklung sicherstellt.

Leitlinien und Definitionen im Zusammenhang mit neuen Indikatoren müssen deutlich vorder Annahme der OP für den Zeitraum 2021-2027 zur Verfügung stehen.


48

1 INTRODUCTION

This document contains the Research Paper ‘Performance Indicators for Convergence in

Regional Development - How Reliable are they to Ensure Targeted and Result-Oriented

Spending?’ The Research Paper was prepared for the European Parliament (Directorate D -

Budgetary Affairs, Directorate-General for Internal Policies of the Union, European Parliament)

by CSES (Centre for Strategy & Evaluation Services LLP).19

1.1 RESUME - STUDY OBJECTIVES

The purpose of this study for the European Parliament (EP) was to prepare a Research Paper on how

reliable the Performance Indicators for convergence in regional development are in helping to ensure

targeted and result-oriented spending20. The specific objectives included in the Terms of Reference

(ToR) are to prepare:

An analysis of the present system of Performance Indicators within the European Union’s (EU)

Cohesion Policy, including strengths, weaknesses and other relevant factors;

Suggestions on how to learn from the findings in the form of changes to the current policies in

order to improve the overall results of the EU’s Cohesion Policy.

The assignment involved analysing various issues relating to the Performance Framework, notably the

common and programme-specific indicators being used in the current ESIF programming period. The

(non-exhaustive) list of elements of interest that was included in the EP’s ToR is outlined below:

Box 1.1: ‘Elements of interest’

Do the Performance Framework and the Key Performance Indicators within the Structural andCohesion Policy have the right focus?

What is the relationship between the KPI’s and the programme indicators - are they consistentand relevant for all regions?

Are the statistics comparable across regions and countries and are indicators and milestonesdefined with comparable levels of ambition given also the difference in who is defining these?

Have the Performance Framework and the system of indicators managed to shift focus moretowards performance?

Do the results of Structural and Cohesion Funds Policies as measured by the Key PerformanceIndicators have any consequence for the spending priorities?

Has the possible financial consequence of the Performance Reserve increased the focus ondelivering good results?

19 Consortium Partner Blomeyer & Sanz ensured quality control.20 The EP’s ToR refer to ‘Key Performance Indicators (KPIs) to measure ERDF and ESF performance. ‘KPIs’ are, in fact, used in theEuropean Commission Annual Management Reports and Annual Activity Reports and represent a small selection of thecommon ERDF and ESF indicators. The full set of common indicators has been examined in this Research Paper. To avoidconfusion, we only refer to ‘common’ and ‘specific’ indicators in the Research Paper rather than KPIs.


____________________________________________________________________________________________

49

It is envisaged that the Research Paper will contribute to the debate on the future design of the EU’s

Cohesion Policy post-2020 and help ensure an enhanced focus on result-oriented spending by

identifying potential improvements to the legal and regulatory framework.

The research for this assignment involved desk research to examine existing material on the use of

performance indicators and related issues (a bibliography is included in Appendix A)21, and an interview

programme with officials from the EP, European Commission (EC) DG EMPL, DG REGIO), the European

Court of Auditors (ECA), and a sample of 10 Member States and regions in them (a list of interviews is

provided in Appendix B). Following a kick-off meeting with the EP on 17 December 2018, an interim

report was then submitted on 25 January 2019 with the remaining research being carried out in

February 2019.

1.2 OVERVIEW OF THE RESEARCH PAPER

The Research Paper is structured as follows:

Section 2: ESIF Performance Monitoring – explains how the European Structural and Investment

Funds (ESIF) framework for performance monitoring has evolved over the years and examines the

key components;

Section 3: Assessment of Key Issues – examines key issues from the terms of reference with

regard to the functioning of the ESIF framework for performance monitoring and its capacity to

ensure targeted and result-oriented spending;

Section 4: Conclusions and Recommendations – sets out the overall conclusions of the research

and how the ESIF framework for performance monitoring could be improved. We also assess the

extent to which the Common Provisions Regulation for 2021-2027 addresses issues raised in the

Research Paper.

The main sections of the Research Paper are supported by several appendices including a list of

interviews and secondary sources, lists of European Regional Development Fund (ERDF) and European

Social Fund (ESF) indicators being used in the current 2014-2020 programming period, and the

indicators proposed for 2021-2027.

21 The literature review has involved an analysis of the following types of documentation: ESIF regulatory provisions andguidance documents at EU level in relation to the Performance Framework, Common Indicators and Monitoring andEvaluation; evaluations and studies of the effectiveness of EU Cohesion Policy, ESIF indicators and Performance Framework,including previous programming periods where relevant, i.e. the 2000-06 performance reserve experience, the core indicatorsunder the 2007-2013 period; regulatory and policy proposals and positions in relation to the 2021-2027 programming periodin relation to result-orientation and performance; Partnership Agreements and relevant programming documents from thesample of Member States and regions as well as ex-ante evaluations and any additional methodological material provided bythe relevant authorities.


50

2 ESIF PERFORMANCE MONITORING

In this section, we examine the framework for ESIF performance monitoring, examining the key

elements for the ERDF and ESF. We first provide a brief overview of the context.

2.1 CONTEXT AND KEY ISSUES

2.1.1 EU Cohesion Policy

The EU promotes balanced economic development across its territory through regional policy, also

known as Cohesion Policy. Articles 174 to 178 of the Treaty on the Functioning of the European

Union (TFEU) provide the legal basis for EU action aimed at promoting ‘harmonious development’,

strengthening economic, social and territorial cohesion and reducing disparities in levels of regional

development. The Multiannual Financial Framework (MFF) for 2014-2020 was implemented in the

context of the financial and economic crisis and strained public finances and dropping investment

levels in many EU Member States. Amongst other things this has emphasised the need to make the

most effective and efficient use of resources.

Regulatory requirements for the implementation of the ESIF in this period introduced new provisions

to improve their effectiveness and European added value. This was achieved by concentrating

resources on key Europe 2020 objectives, establishing a Performance Framework based on

measurable indicators and targets linked to the release of a Performance Reserve, introducing ex-

ante conditionalities as well as creating closer linkages with the EU economic governance and the

European Semester process22. These measures were complemented by the Investment Plan for

Europe supported by the European Fund for Strategic Investments (EFSI), and the Commission’s

‘Budget Focused on Results’ (BFOR) initiative23.

Under Cohesion Policy a system of ‘shared management’ means that whilst the European Commission

has overall responsibility for the EU budget, implementation of its three funds – the ERDF, the ESF and

the Cohesion Fund is delegated to the Member States24. Common provisions in management and

control approaches have been established although specific fund rules also apply to each.

Approximately 32.5 % of the EU’s budget for 2014-2020 (equivalent to some EUR 351.8 billion over

seven years at 2014 prices) is allocated to Cohesion Policy. A total of 11 thematic objectives have been

identified for the current programming period25. Investment from the ERDF supports all 11 objectives,

22 Mid-term review/revision of the multiannual financial framework 2014-2020: An EU budget focused on results (SWD(2016)299 final). Communication from the European Commission to the European Parliament and Council.23 BFOR was designed to address four key questions: Where to spend EU money – how to achieve high EU added value? Howto spend EU money – how to get more out of each Euro spent by the EU budget? How can the performance of the EU budgetbe adequately assessed? How to communicate about the EU budget – how to explain better to citizens what the EU budgetachieves?24 The ESIF include also the European Agricultural Fund for Rural Development (EAFRD) and the European Maritime andFisheries Fund (EMFF).25 1. Strengthening research, technological development and innovation; 2. Enhancing access to, and use and quality of,information and communication technologies; 3. Enhancing the competitiveness of SMEs; 4. Supporting the shift towards alow-carbon economy; 5. Promoting climate change adaptation, risk prevention and management; 6. Preserving andprotecting the environment and promoting resource efficiency; 7. Promoting sustainable transport and improving network


____________________________________________________________________________________________

51

but objectives 1-4 are the main priorities for investment. The main priorities for the ESF are 8-11, though

the Fund also supports 1-4 while the Cohesion Fund supports objectives 4-7 and 11.

2.1.2 Performance Framework

The backdrop to this study is the move in the current 2014-2020 programming period away from

a focus on the absorption of funding towards a results-oriented policy with indicators to

measure outcomes in terms of people’s well-being and progress. (The evolution of ESIF

performance monitoring and evaluation since 2000 is described in Section 2.2 below.) Within Cohesion

Policy, a greater focus on results in the 2014-2020 period is being encouraged through three main

components26:

Box 2.1: Key Features of the Results Orientation Ex ante conditionalities – these have been negotiated to help ensure that the necessary

conditions are in place for a focus on key performance indicators and successful programmedelivery;

Intervention rationale and logic frameworks – ensuring the result orientation ofprogrammes through linking the needs to be addressed to the results to be achieved and theinclusion of result indicators with definitions and measurable targets;

Performance Framework – the establishment of clear and measurable milestones and targetsto ensure progress is made as planned with a Performance Reserve used to reward achievementof results.

In implementing the Performance Framework, the European Commission and Member States are

required to work together to define baselines (e.g. the situation in a region with regard to skills or in

relation to business start-ups) and targets (e.g. how ERDF and ESF interventions might improve the

situation) with performance indicators then being used to assess progress from the baseline towards

the target. The EC’s ‘Better Regulation’ guidelines (2015 and 2017) define three basic types of

performance indicators:

Box 2.2: Performance Indicators Outputs - the specific deliverables of the interventions – in the case of ESIF, they refer to direct

products of spending programme resources on implementing projects (e.g. number ofpersons receiving training, number of SMEs assisted);

Results - the benefits arising directly from the intervention, they capture a change in thesituation: under ERDF, in relation to the change in wider society/economy and under ESFrelated to the situation of a supported entities or participants upon leaving a project (e.g. newqualifications);

Impacts - the intended outcome of the intervention on the wider economy and societybeyond those directly affected by the intervention (e.g. reduced unemployment rates in aregion, increased GDP).

infrastructures; 8. Promoting sustainable and quality employment and supporting labour mobility; 9. Promoting socialinclusion, combating poverty and any discrimination; 10. Investing in education, training and lifelong learning; and 11.Improving the efficiency of public administration.26 Adapted from EPRC, The monitoring and evaluation of the 2014-2020 EU Cohesion Policy Programmes’, IQ-Net ThematicPapers 36(2), May 2016.


52

The ‘Better Regulation’ guidelines define good indicators as those corresponding to the RACER

acronym – R – relevant (appropriate thematic coverage and a direct and close link with the objective it

is measuring), A - accepted (understood by those in charge of data monitoring), C – credible

(unambiguous and easy to interpret), E - easy to monitor (data collections is feasible in terms of costs

and time) and R – robust (clearly defined).

There are potentially many complications in applying and using indicators for the ESIF. Whereas

outputs are physical in nature and relatively easy to measure, results and impacts are socio-economic

and much more difficult to identify and assess. Likewise, whereas the ESF interventions focus on

individuals, the ERDF has a more diverse target group with less homogeneous units of measurement.

The overall framework is outlined below:

Figure 2.1: Overview – ESIF Results Orientation

2.2 EVOLUTION OF ESIF MONITORING AND EVALUATION REQUIREMENTS

In this section, we provide an overview of the evolution of the regulatory provisions for monitoring and

evaluation of ESI Funding from 2000 onwards. The following chart provides an overall summary:


____________________________________________________________________________________________

53


z


54

2.2.1 2000-2006 programming period

The Structural Funds’ regulations for the 2000-2006 programming period were the first to

envisage a move away from purely financial monitoring to include monitoring of physical

performance. Supporting this, there was a clearer definition of monitoring and evaluation

responsibilities at the EU, national and regional levels 27. While the N+2 automatic de-commitment

rule28 introduced an incentive to ensure that programmes advanced to schedule, it was complemented

by initiatives which aimed to ensure that resources were spent as effectively as possible, with

monitoring and evaluation arrangements to reflect the nature and content of the assistance. The

European Commission supported a gradual, ‘pragmatic and flexible’ approach that took account of

specific circumstances and needs as well as resources available to undertake the activity.

Shortly before the programming period started, the European Commission published an indicative

methodological guidance for 2000-2006 setting out indicator terminology and frame of reference.

The guidance proposed a list of indicators to promote consistency and enable comparisons across

programmes and measures as well as aggregation at a higher level in ‘priority areas of Community-

wide interest’29. Programmes were expected to use a small number of these core indicators, with target

values, to ensure appropriateness and manageability. It was envisaged they would be applied from the

lowest operational level (project) upwards and reported in Annual Implementation Reports (AIRs).

The indicators were subject to ex-ante, mid-term and ex-post evaluation, with the evaluators

expected to play an active role in improving the quality of the indicator system and assessing the

quality and relevance of the indicators. Whilst impact indicators were included, the complexity of

measuring impacts that occur after a certain lapse of time and without clear causal relationships was

recognised, as well as the fact that programmes could produce unexpected results that the indicators

were unable to detect. In recognition that none of the requirements around core indicators was

obligatory, the European Commission guidance also provided wider examples of monitoring and

evaluation indicators. Programme-specific indicators were used to allocate the 4% Performance

Reserve at the mid-term ‘in close consultation with the Commission’30 to programmes or priorities

‘considered to be successful’.

The ERDF ex-post evaluation for the 2000-2006 programming period found that Cohesion Policy

management and implementation systems were efficient but not significantly focussed on

delivering strategic objectives and performance. Whilst more attention was paid to monitoring than in

previous programming periods, including through the development of integrated monitoring systems

27 Article 36 of Council Regulation 1260/1999 laying down general provisions on the Structural Funds28 The N+2/N+3 decommitment rule provides that any EU funding commitment against which a payment application has notbeen received by the end of the second/ third year following the year of commitment will be decommitted. Under the 2014-20 programming period the rule has been N+3; under the post-2020 regulatory proposals the rule will be N+2 for the years2021-2027, returning to pre-2013 requirements, (for the EU-15).29 DG REGIO, Working paper 3, Indicators for Monitoring and Evaluation: An indicative methodology, available at:https://ec.europa.eu/regional_policy/sources/docoffic/working/doc/indic_en.pdf30 The performance reserve was set at 4% of the commitment appropriations included in the indicative breakdowns perMember State. Articles 7 and 44 of Council Regulation 1260/1999 laying down general provisions on the Structural Funds; DGREGIO, Working Paper 4, Implementation of the performance reserve available at:https://ec.europa.eu/regional_policy/sources/docoffic/working/doc/reserve_en.pdf


55

and the inclusion of physical indicators, the utility of the systems and information provided was

considered to be sometimes questionable. Regulatory compliance and maintaining the pace of

spending tended to dominate programme management activity. A reassessment of the application of

the de-commitment rule and financial control and audit requirements, alongside a stronger focus on

policy outcomes, was recommended31. In the case of the ESF, the main challenge was the diversity and

lack of comparability of the data across Member States which meant there was only limited scope for

aggregation at the EU level. Key recommendations for 2007-2013 included ensuring more comparable

data as well embedding the evaluation process from the beginning within ESIF programming32.

2.2.2 2007-2013 programming period

In the 2007-2013 programming period, core indicators were proposed by the European Commission

for the ERDF, the ESF and the Cohesion Fund (CF)33. The indicators were seen as particularly important

in demonstrating accountability at the EU level, enabling the aggregation of information to show what

Cohesion Policy resources were being spent on and what they were achieving. However, the core

indicators were not mandatory and were developed after the adoption of many programmes and

hence retro-fitted in many cases. Unless core indicators were used and additionally reported on

through the electronic system for the transfer of information on indicators, aggregation of Cohesion

Policy achievements remained elusive.

Core indicators (the predecessor to common indicators) were used to varying degrees in the 2007-2013

ERDF programmes with those with a Lisbon focus used most. Convergence OPs tended to use core

indicators more frequently in Transport, Environment and Risk Prevention whilst Regional

Competitiveness and Employment (RCE) OPs34 used core indicators for Climate Change, Renewable

Energy and R&TD more frequently. The focus on basic infrastructure investments in Convergence

programmes and R&TD, entrepreneurship and renewable energy in RCE regions reflects eligible

activities and the priority focus of the two categories of region. The most frequently used core

indicators are shown below.

31 EPRC, 2009, EX POST EVALUATION OF COHESION POLICY PROGRAMMES 2000-2006 CO-FINANCED BY THE ERDF (OBJECTIVE1 AND 2) WORK PACKAGE 11: MANAGEMENT AND IMPLEMENTATION SYSTEMS FOR COHESION POLICY available at:https://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wp11_en.htm32 https://slideplayer.com/slide/6589159/33EUROPEAN COMMISSION, DG REGIO, The New Programming Period 2007-2013 INDICATIVE GUIDELINES ON EVALUATIONMETHODS: EVALUATION DURING THE PROGRAMMING PERIOD Working Document No.534 Under the 2007-13 regulations, regions were categorised as Convergence, Competitiveness or phasing in/out. The regionstargeted by the Convergence objective were those whose per capita gross domestic product (GDP) measured in purchasingpower parities was less than 75 % of the Community average. The Regional Competitiveness and Employment categorycovered all other regions of the EU with the exception of those eligible for transitional aid as phasing in / phasing out regions.


56

Box 2.3: Use of ERDF Indicators 2007-2013

“Number of gross direct jobs created” (in 64.3% (74) of the Competitiveness programmes and

44.3% (58) of the Convergence programmes)

“Investment induced [in EUR] to SMEs” (in 34.8% (40) of the Competitiveness programmes and

of the 25.2% (33) Convergence programmes)

“Number of research jobs created” (in 30.4% (35) of Competitiveness programmes and 22.1%

(29) of Convergence programmes)

“Reduction of greenhouse emissions in CO2 or CO2 equivalents” (in 30.4% (35) of

Competitiveness programmes and 14.5% (19) of Convergence programmes)

“Number of cooperation projects between enterprises and research institutions” (in 53% (61) of

the Competitiveness programmes and 26.7% (35) of the Convergence programmes).

Source: The Potential for Regional Policy Instruments, 2007-2013, to contribute to the Lisbon and Göteborgobjectives for growth, jobs and sustainable development, 2009

The average number of core indicators used in the EU27 programmes was 10, ranging from an average

of seven indicators per Competitiveness OP to an average of 12 core indicators per Convergence

programme. Poland was the Member State that made the most extensive use of the core indicator set,

using an average of 29 core indicators per programme (and using each core indicator at least once in

every programme). However, in general the largest EU15 Member States with both Convergence and

Competitiveness objectives (France, Germany, Greece, Italy, Spain, and the UK) made the most

significant use of the core indicator set, largely since they were eligible for many different priority

themes. Sweden, Denmark, Luxembourg and Belgium had the lowest use of the core indicator set with

only 3-4 per programme.

In 2007-2013 all of the programmes used result indicators, with half (132 in total) of the programmes

using the result indicator “number of gross jobs created”. Meanwhile, 23 of the 27 Member States

defined impact indicators despite the fact that impact indicators were not a legal requirement; the

exceptions were Austria, Finland, Greece and Ireland35. However, many of the programmes did not

clarify how they would collect measurable data against their impact indicators.

Box 2.4: Examples of impact indicators 2007-2013

The average rate of ICT usage in Danish businesses is at least 75% by 2013 (Baseline: 56% in2005) (Denmark)

Target for 2013 employment rate (15-64 year cohort) in the OP region 55% (2005 baseline:53.8%) (Hungary)

Targets for employment in creative sector industries; 2003 baseline 3%, 2010 target 3.7%, 2015target 4.2% (Estonia)

Source: The Potential for Regional Policy Instruments, 2007-2013, to contribute to the Lisbon and Göteborgobjectives for growth, jobs and sustainable development, 2009

35 The Potential for Regional Policy Instruments, 2007-2013, to contribute to the Lisbon and Göteborg objectives for growth,jobs and sustainable development, 2009, Final Report to the European Commission, Directorate- General for Regional Policy,Evaluation Unit No 2007.CE.16.0.AT.041


57

The evaluations of ERDF data collection practices in the 2007-2013 programming period36 noted that

whilst reporting against core indicators was not a legal obligation, both the Member States and the

European Commission undertook to report against them, albeit without compulsory requirements as

to how they were defined and the likelihood of undercounting. The use of core indicators meant it

was the first programming period that indicator data could be aggregated and analysed across

the programmes. The most frequently used indicator was ‘Jobs created’ which was used in 178

programmes, followed by ‘Number of cooperation projects enterprise-research institutes’ and ‘Number

of start-ups supported’. Highest achievements against targets were reported under the ‘Number of

start-ups supported’ (105%) and the lowest under ‘additional capacity of renewable energy

production’ (1%). Most Managing Authorities (MA) had put in place measures to ensure data quality

such as guidance documents, standardised quality control procedures and manual checks.

Some weaknesses in the quality and reliability of the ERDF data were however reported. Of the

total of more than 1,700 core indicators used across the EU-27, 7% were not consistent with EU

definitions, with the worst offending Member States (MS) being France, Germany and Poland. Other

minor issues related to inconsistent units being used for indicators. The use of other indicators tended

to reflect the lack of a relevant core indicator and none of these could be aggregated at EU level to

establish reliable information. Further findings under the ex-post evaluations for the 2007-2013

programming period included:

Box 2.5: Weaknesses in the Quality and Reliability of the ERDF data 2007-2013

Monitoring systems for the most part performed adequately in collecting information relating to

implementation and a sufficient aggregation of data on financial progress and outputs at EU level

was possible. These improvements mostly resulted from improved IT systems.

Improved monitoring led to improved programme decision-making and responsiveness to socio-

economic circumstances and enabled strategic adjustments and OPs revisions. They did not

however lead to improved strategic follow-up that improved implementation / solved

implementation problems or consideration of the contribution of the use of the funds to overall

strategy. The role of Monitoring Committees was weak in this regard.

Indicators were overly focused on financial achievements and absorption. A tendency to set low

targets prevailed. A number of the indicators used were unsuitable for measuring the results of

implementation.

MS that had low financial allocations under ERDF/CF tended to monitor achievements parallel to

national or regional monitoring systems and as an exercise purely to meet regulatory

requirements. Real added value was perceived to occur where monitoring of EU funding was fully

integrated with or integrated into national and regional monitoring processes (one example is

the ESF monitoring system in North Rhine-Westphalia).

Despite improvements, a clear picture of the overall achievements of Cohesion Policy at MS and

36 EC, DG REGIO, 2015, Ex-post evaluation of Cohesion Policy programmes 2007-2013 Work Package 0 – Data collection andassessment, submitted by t33, OIR and Spatial Foresight, and Work Package 12 Delivery System, KPMG and Prognos.


58

EU level could not be produced. The high number of indicators (over 8,000 over the OPs) led to

difficulties in relation to validity, consistency and comparability of the indicators.

Very few programmes had a “focus on results” setting clear goals and selecting projects

accordingly and a cultural shift would be required in most cases.

Cohesion Policy achievements tend to be understated as final achievements are only reported

after formal programme closure (3 years after the programme end date). Programme-end date

achievements tend to only represent achievements at the point on average of 77% of the

programme financial allocation having been spent.

Weaknesses in the ERDF indicator set include over-counting around jobs created, including those

that were only temporary and also jobs safeguarded rather than created.

In 2007-2013, the European Commission additionally placed emphasis on the measurement of results

throughout the programming period. Regulation 1083/2006 identified three types of evaluation

according to their timing: before (ex-ante), on-going, and after (ex-post) the programming period

(Article 47[2]). The Regulations promoted a shift from a concept of mid-term evaluation driven by

regulatory imperatives (i.e. that meant evaluations in the previous programming period were launched

too early where interventions were delayed), towards a more flexible, demand-driven approach to

evaluation during the programming period. The introduction of on-going evaluation enabled this

type of assessment to be adapted to address the specific needs of the Member State and to be driven

by national and regional decision-makers.

A priority of the new approach to evaluation was also to assess the contribution of Cohesion Policy to

the achievement of the Lisbon goals and to make that contribution more visible. In the 2007-2013

programming period, Member States had to ensure a minimum allocation of expenditure to Lisbon-

related categories, and to report the achievements of Cohesion Policy in relation to a set of

earmarking codes37. The EU15 were obliged to achieve a set of targets in relation to ‘earmarking’ at the

level of the Convergence and the Regional Competitiveness and Employment (RCE) objectives (i.e. by

category of region) for the entire programming period38. Targets for each Member State were

established based on data from the 2000-2006 period. Twice during the programming period (in 2009

and in 2012), Member States were required to provide a strategic report on the contribution of the

programmes towards the Lisbon earmarking targets. These were then incorporated into a synthesis

report by the European Commission in 2010 and 2013 for the Council.

The ex-post evaluation of the ERDF, ESF and Cohesion Fund Delivery Systems for the 2007-2013

programming period39 concluded that a sufficient aggregation of data had been possible, partly as a

result of the significant improvement in national IT systems and the quality of data in nearly all Funds

and all countries. However, there were numerous examples of incorrect use of measurement units and

37 "Earmarking" is governed by the provisions of the General Regulation, see Annex IV of the General Regulation which liststhe earmarked categories by Objective, as well as by the Fund-specific regulations (Regulation (EC) No 1080/2006 andRegulation (EC) No 1081/2006. See: EUROPEAN COMMISSION, DG REGIO, Information Note no 1, available at:https://ec.europa.eu/regional_policy/sources/docoffic/cocof/2007/cocof_07_0012_00_en.pdf38 Member States that acceded to the European Union on or after 1 May 2004 were only invited to apply these provisions, notobliged to.39 Ex-post evaluation of Cohesion Policy 2007-13, Work package 12 Delivery Systems


59

inconsistencies between EU and national/regional indicator definitions. Evaluators also noted a

tendency for under-ambitious target-setting, albeit in the case of new Member States and often

resulting from a lack of information about similar interventions on which to base their targets.

Monitoring and reporting systems continued to have an excessive focus upon financial indicators

and absorption, and projects were often selected for their ability to absorb funds as much as their

contribution to programme objectives. Programme evaluations tended to focus upon programme

implementation in relation to process rather than impact, whilst strategic reporting tended to be

treated as a compliance exercise with little strategic or policy learning application.

2.2.3 2014-2020 Programming Period

The legislative package for Cohesion Policy in the 2014-2020 programming period introduced

significant changes intended to increase the focus on performance and promote a more result-

orientated approach. According to the EC:

“In the context of tighter budgets and more public attention to the effectiveness of EU policy

instruments in general, the demand for demonstrating the performance, impact and added value

of ESF-supported initiatives has grown. In this respect, monitoring and evaluation play a key role

in providing the necessary evidence.”40

To ensure programmes delivered, i.e. that priorities were implemented as planned and programmes

were on course to achieve their objectives, OPs were required to set out the:

‘Expected results’ for the specific objectives and corresponding results indicators, with a baseline

value and target value, ‘where appropriate quantified’,

Output indicators, including quantified values which should contribute to the results,

Implementation steps and financial and output indicators and where appropriate result

indicators to be used as milestones and targets for the performance reserve41.

Guidance on the performance indicators for ERDF and ESF programmes has been provided by national

authorities in many if not most Member States. An example is outlined below.

40 ‘Programming period 2014-2020: Monitoring and Evaluation of European Cohesion Policy and European Social FundGuidance document’ (September 2014 and August 2018).41 Article 96.2b. Regulation 1303/2013, the Common Provisions Regulation


60

Box 2.6: Case study (Spanish ERDF 2014-2020 Output Indicators Manual)

Spain defined output indicators at national level in collaboration with the EC and the Spanish

Autonomous Communities with the purpose of having a harmonised system that makes it

possible to measure the achievements obtained with ERDF support for each of the investment

priorities set in the 2014-2020 Spanish Partnership Agreement42.

This manual includes an individual descriptive sheet for each common and specific output

indicator (in total 44 common indicators and 83 specific indicators including technical

assistance indicators) with a detailed definition of the indicator, explanation about how to

calculate the indicator, when to load the data into the system, and other user information.

The list of indicators is grouped by thematic objectives, investment priorities, specific

objectives and fields of actions.

The Spanish output indicator system is flexible. With the progress on the different actions it has

been necessary to better adapt the definition of some specific indicators to a concrete action

and some definitions have been extended or even some new specific indicators have been

defined, although this as the last option.

2.2.4 Performance Framework and Performance Reserve

The Performance Framework consists of a set of milestones and targets defined for each priority in a

programme43. The output indicators included in the Framework should represent the majority of the

allocation to a Priority and are appraised in terms of their relevance, suitability, realism and

quantification as part of the ex-ante evaluation. Progress against the Performance Framework is

reviewed formally twice during the programming period: milestones are intermediate targets to be

achieved by 31 December 2018 and assessed in 2019. The Performance Reserve (representing 6% of

the resources allocated to the ERDF and ESF expenditure) will, following assessment in 2019, be

definitively allocated to priorities which achieve their milestones based on a Performance Review in

2019 (and hence following the end of this research project)44.

The targets themselves are set to be achieved by 31 December 2023 and assessed at programme

closure in 2025. Baselines for common and programme-specific output indicators are set at zero, and

each OP should also set cumulative quantified targets for output and result indicators.

42 A Partnership Agreement (PA) is a reference document for ESIF programming interventions linking them to the Europe 2020growth strategy. It defines the strategy and investment priorities chosen by the relevant Member State and presents a list ofnational and regional operational programmes (OPs) which it is seeking to implement, as well as an indicative annual financialallocation for each OP.43 The performance framework is established at the level of priority, with the exception of priorities concerning technicalassistance or financial instruments in favour of SMEs or complex priorities including the Youth Employment Initiative (the YEI);priorities covering more than one Fund or more than one category of region44 Within two months from the receipt of the annual implementation report for 2018, the European Commission will adopt adecision to determine for each Member State and ESI Fund, the programmes and priorities which have attained theirmilestones


61

Milestones and targets have to be realistic and achievable (i.e. neither too high nor too low, based on

evidence of past experience); relevant and reflect the objectives and operations of the priority;

consistent with the nature and character of the specific objectives of the priority (i.e. in line with the

intervention logic of the priority and resources allocated); transparent, with objectively verifiable

values, source data identified and publicly available; verifiable without imposing a disproportionate

administrative burden; and consistent across programmes (i.e. no significant, unjustified difference in

methods applied for comparable priorities in the same Member State)45.

2.2.5 ERDF and ESF Indicators 2014-2020

Whilst OPs may include programme-specific indicators, a set of common indicators is provided

in the fund-specific rules:

Regulation 1304/2013 establishes 23 ESF common output and 9 common result indicators. (see

Appendix C)

Annex I of the ERDF regulation 1301/2013 establishes a set of 40 ERDF (excluding ETC) common

output indicators that correspond to the Fund’s investment priorities (see Appendix D).

European Regional Development Fund

The 40 common output indicators for mainstream ERDF programmes (six are specific to ETC

programmes) are outlined in summary form in the table below (see also Appendix D).

Common indicators for the ERDF only cover outputs. The ERDF Regulation establishes that

programme-specific output and result indicators may be used46. Impact indicators are policy result

indicators specific to the individual programme. In the 2014-2020 period, they are very diverse and, as

with programme-specific result indicators, cannot be aggregated. They will be assessed through

evaluation which was an aspect of the policy that has been strengthened for 2014-2020, with the

requirement for Member States to submit evaluation plans.

45 Annex II Common Provisions Regulation.46 Where used, ERDF programme-specific result indicators should be: responsive to policy; closely linked to the policyinterventions supported; normative, i.e. having a clear and accepted normative interpretation (i.e. there must be agreementthat a movement in a particular direction is a favourable or an unfavourable result); robust: reliable, statistically validated; andsuitable for timely collection: data is available when needed.


62

Table 2.1: Summary – ERDF Key performance indicatorsTheme Indicators

Productive

investment Number of enterprises receiving support (subsets: receiving grants, receiving

financial support other than grants, receiving non-financial support, newenterprises supported);

Private investment in enterprises, Employment increase Increase in expected number of visits to supported sites of cultural or natural

heritage and attractionsICT Additional households with broadband access of at least 30 Mbps

Transport Total length of new railway line, of which: TEN-T Total length of reconstructed orupgraded railway line, of which: TEN-T

Total length of newly built roads, of which: TEN-T Total length of reconstructedor upgraded roads, of which: TEN-T

Total length of new or improved tram and metro lines Total length of new or improved inland waterway

Environment Additional waste recycling capacity,

Additional population served by improved water supply,

Additional population served by improved wastewater treatment, Population benefiting from flood protection measures, Population benefiting from forest fire protection measures, Total surface area of rehabilitated land, Surface area of habitats supported in order to attain a better conservation status

Research and

Innovation Number of new researchers in supported entities Number of researchers working in improved research infrastructure facilities Number of enterprises cooperating with research institutions Private investment matching public support in innovation or R&D projects Number of enterprises supported to introduce new to the market products Number of enterprises supported to introduce new to the firm products

Energy and

Climate

change

Additional capacity of renewable energy production Number of households with improved energy consumption classification Decrease of annual primary energy consumption of public buildings Number of additional energy users connected to smart grids Estimated annual decrease of GHG

Social

infrastructure Capacity of supported childcare or education infrastructure Population covered by improved health services

Urban

Development Population living in areas with integrated urban development strategies Open space created or rehabilitated in urban areas Public or commercial buildings newly built or renovated in urban areas Rehabilitated housing in urban areas


63

European Social Fund

Common ESF indicators are listed in Annex I and II of the ESF Regulation and represent the minimum

set of indicators for each OP. In total, there are 32 ESF common indicators (23 outputs and 9 results

indicators). Table 2.2 provides a summary and a full list is provided in Appendix B). European

Commission guidance for the ESF requires that common ‘outputs’ and ‘immediate’ and ‘intermediate’

‘results’ indicators are used and reported on for each ESF investment priority for subsequent

aggregation at the EU level47.

Common result indicators relate only to persons, not to entities, and are reported annually. The

definitions of each common ESF and Youth Employment Initiative (YEI) indicators are mostly based

upon common international definitions48. A distinction is made in the EC’s guidance between

‘immediate’ and ‘longer-term’ ESF result indicators. The immediate results indicators are designed to

capture the effects that occur when the participants or entities finish participating in a project and

should be captured four weeks after leaving the ESF-supported programme and reported annually in

the AIR49. Longer-term result indicators seek to capture similar data on participants but six months

after the exit date and are only reported twice throughout the programming period: in the 2019 AIR

and in the final report in 2025 on the basis of a representative sample under each investment priority.

The ESF Regulations do not require the use of impact indicators in the sense of results on broader

groups of society beyond the participants or entities directly benefiting from the support. However,

impact should be assessed through periodic evaluations according to the guidance.

Table 2.2: Summary – ESF Common Indicators

Effects Common indicators

Outputs (i) Number of ESF training ‘participants’ broken down by: employment status, age

(5 indicators); age (3); educational attainment (3); disadvantaged participants (4).

(ii) For ‘entities’: number of projects fully or partially implemented by social

partners or non-governmental organisations; number of projects dedicated to

sustainable participation and progress of women in employment; number of

projects targeting public administrations or public services at national, regional or

local level; and number of supported micro, small and medium-sized enterprises

(including cooperative enterprises, enterprises of the social economy)

Intermediate

results

Inactive participants engaged in job searching upon leaving; participants in

education/training upon leaving; participants gaining a qualification upon leaving;

participants in employment, including self-employment, upon leaving; and

47 An EU synthesis report is produced every year on the basis of the results reported in the OP’s AIRs; the latest available is for2017. European Commission, ‘Synthesis Report of ESF Annual Implementation Reports, Final report’ (Fondazione G.Brodolini,November 2017).48 In some cases, there is no EU-wide definition e.g. migrants, minorities, participants with disabilities and other disadvantagedgroups such as the homeless. Common international definitions are therefore used such as those from the Labour MarketPolicy database and the Labour Force Survey at Eurostat.


64

Effects Common indicators

disadvantaged participants engaged in job searching, education/training, gaining

a qualification, or in employment, including self-employment, upon leaving.

Longer

terms results

Participants in employment, including self-employment, six months after leaving;

participants with an improved labour market situation six months after leaving;

participants above 54 years of age in employment, including self-employment, six

months after leaving; and disadvantaged participants in employment, including

self-employment, six months after leaving.

Some of the feedback on the use of the common and specific indicators from the Member States used

as a sample for this Research Paper is provided below.

Box 2.7: Use of Common and Specific Indicators

BG: Bulgaria has limited experience in EU programme indicators design and implementation,

although its capacity has improved significantly since the previous programming period. All

Cohesion Policy OPs in Bulgaria use a mix of common and specific indicators. Common

indicators are used where possible. However, as in other countries, the specific needs of each

OP can require specific indicators and BG has had the flexibility to create these when necessary

as long as they are justified. Reflecting this, the OP Innovation and Competitiveness uses mostly

common indicators but the OP Environment and OP Regions in Growth have high numbers of

specific objectives as they are tailor-made to the Bulgarian context. Under the OP Science and

Education for Smart Growth, many indicators are based on the European Innovation

Scoreboard rather than the common indicators.

DE: The Germany ESF and ERDF programmes rely mostly on the common indicators with less

use of programme-specific indicators than in many other Member States. This is largely due to

the fact that there is a quite standardised approach across the 16 German regions in their

approach to ERDF and ESF indicators with a working group helping to ensure that common

indicators and methodologies are used.

EE: Estonia has only one OP funded by the ESF, ERDF and CF combined, and some 70% of the

indicators are common indicators. Because these do not cover all the areas that warrant

support, Estonia has implemented specific indicators to supplement these gaps.

ES (Andalucía): The list of common indicators is not considered sufficiently complete or well

defined in relation to the size of the programme and range of activity funded in Andalucía

under ERDF. Deficiencies in indicator definitions and doubts about possible interpretations and

way of measuring them has led to the definition and use of programme-specific indicators.

These specific indicators were harmonised at national level. In the case of the ESF, only three

specific indicators have been defined.

IT: Italy has 12 national and 39 regional programmes and wanted to avoid the use of excessively

high numbers of indicators. MAs were encouraged by the central evaluation and analysis unit

(Nucleo di valutazione e analisi per la programmazione, Dipartimento per le politiche di coesione)


65

to use common indicators as much as possible and then choose from a standardised list of

national common indicators based upon a unique national classification of types of investment

outlined in a national guidance document. This approach reduced the number of indicators

utilised, but the Italian programmes still use significant numbers of indicators and additionally

propose programme or project-specific indicators. Sometimes there are only subtle differences

from the common indicators and so they are undertaking a process to further streamline them.

HR: has used almost all of the ERDF common indicators in its OP, and these account for about

a third of the total number of indicators, with the other two-thirds being programme-specific

and reflecting areas where interventions were not adequately covered by the common set. A

‘Project Indicator Sheet’ has been developed for each indicator alongside the development of

the OP and through an extensive consultation process with stakeholders, and strong input from

the EC. These indicator sheets are regularly updated as the MA gains an increasing

understanding of the requirements and their appropriate interpretation on the ground in

Croatia.

In all Member States, the development of results indicators has proved challenging. Some further

feedback on this issue from the Member States used as a sample for this Research Paper is provided

below.

Box 2.8: Results Indicators

ES (Andalucía): The national harmonisation of ERDF result indicators was not approved by theEC, and although Andalucía and the other Spanish regions have developed their own lists,many of them are quite similar. The MA, together with the intermediate bodies, has produceda set of descriptive sheets with the definition of result indicators and these files are publishedvia a forum that all the intermediary bodies have access to. Although it is not obligatory to usethese indicators, most of the Autonomous Communities have done so. In general, resultindicators seem to be of poor quality. One reason is that there are too many requirements andnot much flexibility for setting result indicators (such as the need that the information comesfrom public sources).

FR: there was a general problem with ERDF result indicators during 2014-2020. The definitionsgiven by the EC were not clear. It was argued that result indicators were in fact impactindicators. This created confusion at national level. Given that in France there is a relativelycentralised mechanism for debating the indicators’ design, this was subject to severaldiscussions among the French Regions.

IT: Italian programmes have used context indicators for result indicators based upon officialstatistical data at the relevant territorial level. These are expected to define the ambition ofprogrammes at the outset and then drive discussion during implementation and are the resultof considerable investment by the Italian government in improving the system for timelyprovision of statistical data.


66

2.3 CONDITIONALITIES AND THE PERFORMANCE RESERVE

There are two further instruments to make ERDF and ESF spending more results-oriented that are

relevant to this research – the Conditionalities and the Performance Reserve.

Conditionality in the Cohesion Policy context means that Member States are obligated to comply with

certain requirements or loose the entitlement to financial assistance.

2.3.1 Ex ante conditionality

In 2007-2013 period, there were positive incentives to develop specific programmes focussing on key

European priorities, such as employment, social inclusion, innovation and energy efficiency. As noted

in a recent study, Cohesion Policy was reoriented from focussing only on the goal of achieving

economic and social cohesion but also to promote the objectives of the Lisbon agenda. The

conditionalities evolved from the European Commission proposals that called for "new conditionality

provisions […] to ensure that EU funding is focused on results and creates strong incentives for

Member States to ensure the effective delivery of Europe 2020 objectives and targets through

Cohesion policy".

In the 2014-2020 period, all five ESIF instruments became conditional in light of economic governance

objectives and procedures. The Common Strategic Framework (CSF) implemented conditionality

through three instruments: ex-ante conditionality, thematic ex ante conditionality and macroeconomic

conditionality.

Box 2.9: Cohesion Policy Conditionalities

Ex-ante conditionalities (7) linked to horizontal aspects of programme implementation,

applicable to all ESIF (public procurement, state aid, anti-discrimination, gender equality,

disability, environmental legislation and statistical systems).

Thematic ex ante conditionalities (29) setting out sector-specific conditions for investment

under Cohesion Policy. They cover, for instance, sectoral bottlenecks in the areas of transport,

digital economy, energy, SME support, labour market institutions, education, etc.

Macroeconomic conditionalities envisage partial or total suspension of ESIF in case of failure

by a MS to comply with one of the EU’s economic governance procedures because the

existence of fiscal or macroeconomic imbalances.

Performance measurement in relation to the various conditionalities occurs at the outset when the

Partnership Agreements are being prepared and then later during the programme implementation

period (by the end of 2016).

Of particular relevance to this Research Paper are the ex-ante conditionalities50. The ex-ante

conditionalities are defined in Regulation 1303/2013 and help strengthen the contribution of ESIF

50 Macroeconomic conditionality has always existed in Cohesion Policy. The gross (regional) domestic product per capita ispresently the only criterion for eligibility among the less developed or intermediate regions in the framework of the ESI Funds.According to Article 121 of the TFEU, Member States have the obligation to regard their economic policies as a matter ofcommon concern and to coordinate them within the Council, with a view to contributing to the achievement of the objectivesof the EU.


67

interventions to Europe 2020 objectives, in line with national and regional development needs.

Ex-ante conditionalities are conditions which have to be in place before funds are disbursed in order

to facilitate successful performance. They aim at better targeted public investment and regulatory,

strategic and administrative weaknesses that can hinder the effectiveness of public investments thanks

to improved and more strategic policy frameworks, prioritisation of projects, and complementarities

with other sources of funding. In 2014, ex-ante conditionality was especially important for the

establishment of the institutional framework that was required to be in place in order to pursue the

specific priorities defined for the period 2014-2020 and as a system for assessing progress in meeting

targets. In addition, ex-ante conditionalities help set baselines for the KPIs that can subsequently be

used to help assess the results achieved by ESIF interventions and to quantify the extent to which such

an improvement has come about.

General ex-ante conditionalities apply to all programmes and consist of requirements on

procurement or the use of reliable statistics, especially set out in Ex-ante Conditionality No. 7.51

Through a process of self-assessment by Member States and subsequent assessment and approval by

the EC, ex-ante conditionalities that are deemed applicable are selected and included in every OP. They

enable the European Commission to condition the disbursement of ESIF funds i.e. to re-programme or

suspend the disbursement of ESI Funds when certain pre-conditions are not met.

2.3.2 Performance Reserve

Ex-post’ conditionality lies in the conditions which focus on policy outcomes and have already been in

place for several programming periods. A Performance Reserve to reward prompt implementation of

programmes and linking achievement of results to additional funding was previously introduced in the

2000-06 programming period.

The 2014-2020 Performance Reserve (representing 6% of the resources allocated to the ERDF and ESF)

will be allocated to the Member States and regions whose programmes have met the objectives related

to Europe 2020 following a Performance Review in 2019 to determine the extent to which Priorities

achieve their milestones. The timing of this Research Paper will potentially not allow for lessons from

the current programming period to be drawn on the usefulness of a Performance Reserve.

Some of the feedback on the Performance Reserve from the Member States used as a sample for this

Research Paper is provided below.

51 General ex-ante conditionality 7 concerns the existence of a statistical basis necessary to undertake evaluations to assessthe effectiveness and impact of the programmes. See page 28 for further information.


68

Box 2.10: Performance Reserve

BG: Funds have been transferred from Bulgaria’s OP Science and Education for Smart Growth to

another OP and as a result, one of the main indicators included in the former OP has been

discarded from the Performance framework. This means that currently the OP consists of only

one output and one financial indicator for the purposes of the Performance Reserve. There was

a notable improvement under one OP priority axis as a direct result of the Performance Reserve

where implementation was accelerated to ensure the target was met, but these were relatively

straight-forward interventions aimed at SMEs.

DE: The approach adopted in Berlin to the Performance Reserve has been to simplify the

approach as far as possible by focusing on the smallest number of (output) indicators possible,

i.e. one financial indicator and one output indicator for each Priority Axis. This approach still

complied with the Performance Reserve rules which do not specify how many indicators should

be used but rather focus on ensuring more than 50% coverage of programme measures.

EE: Estonia and the other Baltic States see the Performance Reserve as being too ‘mechanical’

and not corresponding to the needs and activities on the ground. Social measures, which are

high priority at the national level, are not considered “winners” of the Performance Reserve

simply because their implementation was delayed.

ES (Andalucía): the Performance Reserve is not seen as having provided an incentive or led to

better results. It is suggested that ESIF outcomes may be better measured but this does not

mean that there is improvement in getting the best results. In addition, it is argued that when

an axis is penalised, all the implementing partners within the axis are penalised regardless of

whether they have reached their targets or not.

HR: Croatia has been fairly cautious about setting targets and milestones in relation to the

Performance Framework and Reserve, except where there has been political pressure

domestically. Performance issues in relation to meeting targets reflect significant complexity in

terms of delivering large infrastructure projects on the ground, issues upon which the financial

implication of a Performance Reserve has little effect. A number of targets and methodologies

had to be revised when it became clear there were misunderstandings or inconsistencies across

government departments when the targets were originally set. Croatia had an extra year to

implement its 2007-2013 programme on account of their 2013 EU accession and hence this also

delayed the implementation of programmes in 2014-2020.

IT: The Performance Reserve has increased awareness and interest around performance due to

the potential financial consequences. However, the focus has shifted on to indicators and

performance monitoring, not on to results. The Performance Reserve is generally not perceived

to be a fair, easy or clear process.

MT: In general, the financial consequence of the Performance Reserve has helped Malta take a

more holistic approach to its OPs. Instead of focusing too much on certain areas, MAs work to

ensure all projects are progressing well towards achieving their 2020 targets.


69

2.4 DATA COLLECTION AND MONITORING

The regulatory framework for the 2014-2020 programming period has introduced several changes

concerning data collection and monitoring arrangements to ensure that performance information is

reliable, complete and timely and can produce meaningful aggregated results-oriented data.

The requirements included the establishment of a management and control system (MCS) covering

adequate systems for reporting and monitoring, a computerised system to record all the data, and an

electronic system for exchange of information between beneficiaries and the MA, Certifying Authority

(CA), Audit Authorities (AA) and Intermediate Bodies (IB). Under Ex-ante conditionality (EAC) 7 (see

section on ex-ante conditionalities), “a system of result indicators necessary to select actions which

most effectively contribute to desired results, to monitor progress towards [those] results” should be

in place by 31 December 2016. These systems must be assessed against a common methodology and

an assurance on the quality of the monitoring data and adequacy of the system given. This data is

incorporated into AIRs sent to the European Commission with the 2018 report providing the basis for

the performance review and the allocation of the performance reserve. Performance monitoring is

additionally subject to audit and covered under the annual management and control assurance, whilst

evaluation requirements are strengthened through obligatory OP evaluation plans and impact

evaluations.

Further measures were introduced for the 2014-2020 programme period to ensure results-orientation

is embedded throughout programme implementation. OPs must set out “guiding principles” for the

selection of projects for each priority axis and design selection procedures and criteria “that ensure the

contribution of operations to the achievement of the specific objectives and results of the relevant

priority”.

Feedback on ERDF and ESF data collection systems from the Member States used as a sample for this

Research Paper is provided below.

Box 2.11: Data Collection Systems

BG: In order to improve the indicators’ reporting, the National Institute of Statistics is currently

developing a new system specifically for the purposes of the ESIF. The name of the project is

"Building a statistical basis and an information system for monitoring of European and national

strategies and regional policy". The project is funded by the OP "Good Governance" 2014-2020,

co-funded by the ESF. The main objective is to develop a statistical framework and procedures

for collecting and disseminating information in order to optimize the processes of data

exchange and accessibility of the information for monitoring and evaluation purposes.

EE: Estonia and the other Baltic States have very good information systems for ESIF

programmes with an estimated 98% of the monitoring data being considered reliable. This

reflects the fact that it is easier to develop good information systems for smaller countries and

programmes than for larger ones.

FR: In France, there is an issue with the discrepancy between the project reporting cycles at the

national and EU levels. The statistical data, which is necessary for most of the result indicators,


70

is usually published after the deadline for indicators’ reporting. For instance, in 2018, the latest

data on some result indicators was from 2016-2017.

ES (Andalucía): Taking into account the complex management structure in Spain, the

existence of a centralised management tool at the national level to monitor and control ERDF

and ESF operations is especially important in providing information on the situation in different

regions in terms of physical and financial realisation of the OPs.

HR: The most significant lesson learnt in Croatia has been to ensure that stakeholders and

beneficiaries understand the monitoring and evaluation system and that the methodology,

assumptions, definitions and evidence required are clear when setting targets and choosing

indicators. Without early clarity in the programming period issues emerge later in the reporting

processes. The most difficult ERDF common indicators to report against are those that measure

the number of enterprises (e.g. SMEs receiving support, collaborating with research

institutions) and avoiding double counting of enterprises that receive support under multiple

measures of the OP. It is difficult to apply some of the indicators to specific projects and apply

an appropriate methodology.

2.5 PROPOSALS FOR THE 2021-2027 PERIOD

Regulatory proposals for 2021-2027 continue the trend of adopting common rules for the ESIF

shared management funds52 with the aim of enhancing coherence and synergies across the

funds. The allocation method for the funds is still largely based on GDP per capita, although new

criteria are added (youth unemployment, low education level, climate change, and the reception and

integration of migrants). This is in line with the European Parliament resolution of 13 June 201753 which

supported the development of an additional set of indicators to complement the use of GDP in

allocating funding such as, for example, a Social Progress Index or a demographic indicator that could

better respond to the new types of inequalities between EU regions54.

Regulatory proposals include measures that aim to increase the results-orientation of the ERDF

and ESF within the wider context of simplification.

Box 2.12: Performance Framework and Monitoring and Evaluation in the Common ProvisionsRegulation for 2021-2027

All indicators used in programmes will be part of the Performance Framework, with a higher

proportion of common output and result indicators rather than programme-specific

indicators.

Outputs and direct results indicators are included but linked to the specific objectives level only

52 with the exception of EAFRD53 http://www.europarl.europa.eu/sides/getDoc.do?type=TA&reference=P8-TA-2017-0254&language=EN&ring=A8-2017-020254 An EP Research Report Indicators in Cohesion Policy also suggested the replacement of the use of Gross (Regional) DomesticProduct in allocating funding by total Final Consumption Expenses (FCE) to include four main dimensions: physical wellness;social vulnerability; educational and technological development; and regional attractiveness:http://www.europarl.europa.eu/thinktank/en/document.html?reference=IPOL_STU(2017)601976


71

(set in the Fund-specific regulations) with milestones to be achieved by the end of 2024 for

outputs and 2029 for outputs and results. Common output and result indicators are contained

in the annexes of the fund-specific regulations.

For the ERDF and the CF the output and result indicators are explicitly linked to the five policy

objectives and substantially increased in number. Neither makes reference to the use of impact

indicators55.

In terms of monitoring data, it is envisaged that results will be transmitted electronically and

every two months to the EC, meaning that the open data platform will be updated almost in

real time56. This could be too onerous on MAs (as we suggest later, a more optimal timeframe

would be 4-6 monthly updates).

Payments will be conditional on the achievement of pre-agreed results/outputs or completion

of policy actions or processes57.

Post 2020 performance monitoring will also make greater use of national administrative

registers and datasets to improve the quality of data (e.g. in the case of the ESF, social security

registers).

The Performance Reserve is not being continued. Instead, only the first five years will be

programmed initially, with the last two years’ allocations made on the basis of an in-depth mid-

term review and reprogramming process in 2025 based on progress in achieving objectives,

the socio-economic situation and new challenges identified in the country specific

recommendations, hence reinforcing the performance focus of the programming process.

Ex-ante conditionalities are replaced by “enabling conditions”, which will be fewer in number

and more focused on the Fund-specific goals as well as subject to application and monitoring

through the programming period. The conditions will also be strengthened with MS unable to

declare expenditure until the relevant enabling condition is fulfilled, as well as simplified in

relation to the European Semester and Country-Specific Recommendations (CSRs).

Suspensions will be linked to commitments only, not payments.

The proposal offers a range of simplification measures, such as simplified cost options, and

financing not linked to costs, which have the potential to shift focus from spending to results.

All indicators used in programmes will be part of the Performance Framework, with a higher coverage

of interventions with common output and result indicators rather than programme-specific indicators.

Outputs and direct results indicators are included linked to the specific objectives level only (set in the

55 1. A smarter Europe by promoting innovative and smart economic transformation; 2. A greener, low-carbon Europe bypromoting clean and fair energy transition, green and blue investment, the circular economy, climate adaptation and riskprevention and management; 3. A more connected Europe by enhancing mobility and regional ICT connectivity; 4. A moresocial Europe implementing the European Pillar of Social Rights; 5. A Europe closer to citizens by fostering the sustainable andintegrated development of urban, rural and coastal areas and local initiatives56 Proposal for a regulation of the EP and of the Council laying down common provisions on the ERDF, ESF+, the CohesionFund, the EMFF, Strasbourg 29.5.2018 COM(2018)375, 2018/0196 (COD)57 This option is the continuation of the «payments based on conditions» introduced in the Omnibus.


72

Fund specific regulations) with milestones to be achieved by the end of 2024 for outputs and 2029 for

outputs and results.

Common output and result indicators are set out in the annexes of the fund-specific regulations. For

the ERDF and the CF the output and result indicators are explicitly linked to the 5 policy objectives58

established and substantially increased in number. For the ERDF the number of indicators has more

than doubled, with the aim of covering about 80% of all investments made (see Appendix F for the full

ERDF list), whilst for the ESF there has been marginal change in the form of some minor streamlining.

New indicators for the ESF were not encouraged unless there were new categories of activity e.g.

migrants as a target group, and there is significant stability between the current and future

programming period which should enable targets to be set more easily. The ECA considers that

indicators can be too generic, and their relevance to policy objectives is not always evident. It therefore

proposes a restriction on the use of programme-specific output and result indicators in order to reduce

administrative burden59.

However, post 2020 proposals do not include impact indicators for either the ERDF or the ESF, despite

having originally been considered. The requirements for ex-ante evaluations and impact

evaluations – which are significant steps in determining development needs, the change that a

programme wants to achieve and its contribution to socio-economic development as well as enabling

synthesis at EU level - have equally been removed. The evaluation of what works in what circumstances

and the net effect or impact of the policy is relevant not only for ESIF investment but wider policy and

national funding schemes and would be a better investment than expensive monitoring of micro-level

data that is impossible to extrapolate. Simplification leads to an increased emphasis on performance

management and reporting but not on results and the strategic contribution of the policy. Embedding

greater levels of evaluation in the policy design would enable in-depth assessment of the quality of

interventions. The ECA are equally critical:

“The proposed removal of some current procedures, such as ex ante assessments of programmes,

appraisals of major projects, strategic reporting and the performance reserve, may offer simplification

but, in our view, their removal weakens the mechanisms in place to deliver results.”

The increased timeframe for submitting monitoring data (every two-months) is likely to be too

onerous on MAs; a more optimal timeframe would be 4-6 month updates. Post 2020 performance

monitoring should make greater use of existing national administrative registers/datasets in order to

improve the quality of data under ESF e.g. social security registers, as well as the linkage and alignment

with national interventions.

Feedback from the sample of Member States we used for this Research Paper on the development of

the ESIF for the 2021-2027 period is summarised below.

58 1. A smarter Europe by promoting innovative and smart economic transformation; 2. A greener, low-carbon Europe bypromoting clean and fair energy transition, green and blue investment, the circular economy, climate adaptation and riskprevention and management; 3. A more connected Europe by enhancing mobility and regional ICT connectivity; 4. A moresocial Europe implementing the European Pillar of Social Rights; 5. A Europe closer to citizens by fostering the sustainable andintegrated development of urban, rural and coastal areas and local initiatives59 https://www.eca.europa.eu/Lists/ECADocuments/OP18_06/OP18_06_EN.pdf


73

Box 2.13: Vision Post-2020

BG: The common indicators are considered to be a very helpful tool in Bulgaria, and a priority

is to further develop them in the post-2020 period. Bulgaria also supports the development of

a joint cross-fund monitoring system that covers both the ERDF and the ESF. This will facilitate

the aggregation of indicator information during the next programming period. There is also a

need for further data collection deadlines harmonisation during 2021-2027.

EE: Estonia is currently carrying out a mid-term review and is in the process of developing

indicators for the 2021-2027 programming period. A priority is to have everything in place

before the new programming period starts. Estonia needed to amend its 2014-2020 OP twice

which was highly time-consuming and had adverse consequences for the Performance Reserve

targets.

ES (Andalucía): The common indicators should be better defined in order to enhance their use,

to avoid different interpretations and to make it easier to aggregate data. It is also seen as

important to include an obligation for the beneficiaries to provide information for the result

indicators. Likewise, the use of representative samples for the immediate result indicators for

programmes with many participants is desirable.

FR: The list of indicators suggested for post-2020 is seen as being too long with concerns that

it will create too much of an administrative burden. At the same time, the suggested list does

not capture all the project outputs and results, and some of the indicators are regarded as being

repetitive while others are seen as too restricted.

HR: It is acknowledged that the EC has supported a substantial exercise to improve the ERDF

indicators for 2021-2027 and to address issues regarding clarity and methodology. MS have

been able to put forward suggestions and participate in the process. Croatia would like to see

the removal of indicators based on percentages rather than absolute values as they struggle to

translate this into project level targets.

IT: the new list of indicators is very long but still not considered comprehensive by the Italian

authorities who will continue to also use programme-specific indicators to better reflect the

programme activity and to fill in the gaps. Definitions need to be less broad and more legally

precise as beneficiaries and organisations implementing programmes on the ground are not

experts in data collection. The measurement of policy results rather than programme

performance against indicators should be the focus.

The incorporation of a “5+2” programming approach60 based on a mid-term review arguably

enables a more strategic qualitative approach to programme performance and greater flexibility to

respond to changing circumstances. However, the ECA considers that this represents an administrative

burden and a complication rather than a simplification, whilst the proposed timing of the review means

60 The 5+2 approach under the post-2020 regulatory proposals refers to the initial programming for five years, and theprogramming of the final 2 years only subsequent to a Mid Term Review.


74

it will “mostly be limited to the reported values of output indicators, and not result indicators in any

form, as there will be no milestones available for the result indicators in the performance framework.”

The ECA also judges that the methodology for the Mid-Term Review lacks clarity. The timing of the mid-

term review and reprogramming is also likely to coincide with post-2027 MFF negotiations and hence

be a significant administrative burden on both the European Commission and Member State

authorities, especially as all programmes will be seeking to re-programme and allocate the additional

funding at the same point in time. In terms of simplification the proposal does not demonstrate

proportionality or reduced complexity.

A key criticism of the ECA concerns the lack of EU-wide strategy or targets used to support the CPR

and with which to align funding and hence performance. Whilst the CPR does propose five high-level

policy objectives these are not translated into measurable, quantified results at the EU level. The

criticism builds on the commentary by the ECA that whilst the European Commission applies the

concept of EU value added as a guiding principle of its spending review and the allocation of resources

there is no robust concept or definition which is necessary also for the design and evaluation of

spending programmes61.

The EP has suggested amending Recital 18: “Member States should establish a performance framework

for each programme covering all indicators, milestones and targets to monitor, report on and evaluate

programme performance” adding the following text: “This should allow project selection and

evaluation to be result-driven.”62

61 https://www.eca.europa.eu/Lists/ECADocuments/BRP_MFF2/BRP_MFF2_EN.pdf62 Amendments adopted by the European Parliament on 13 February 2019 on the proposal for a regulation of the EuropeanParliament and of the Council laying down common provisions on the European Regional Development Fund, the EuropeanSocial Fund Plus, the Cohesion Fund, and the European Maritime and Fisheries Fund and financial rules for those and for theAsylum and Migration Fund, the Internal Security Fund and the Border Management and Visa Instrument (COM(2018)0375 –C8-0230/2018 – 2018/0196(COD))1(Ordinary legislative procedure: first reading)


75

3 ASSESSMENT OF KEY ISSUES

Having described the ESIF Performance orientation in the previous section, we now consider the

key issues set out in the EP’s terms of reference on how well the Performance Framework and

system of indicators are working and the extent to which they lead to targeted and result-

oriented spending.

3.1 USE OF COMMON ERDF AND ESF INDICATORS IN 2014-20

Table 3.1 below provides a summary of the data available from the European Commission on the extent

to which the various ERDF and ESF indicators are being used in the 2014-2020 programming period (a

more detailed breakdown of this information is provided in Appendix E).

Table 3.1: Use of ERDF and ESF common indicators in the 2014-2020 Programming Period

ERDF Indicator theme (Commonindicator numbers)

Number of timesindicators used across

EU-28 ERDFprogrammes

Number of timesindicators used acrossEU-15 CF programmes

Productive investment (CO1-CO9) 2729 -

ICT (CO10) 48 -

Transport (CO11-CO16) 199 72

Environment (CO17-CO23) 310 63

Research and Innovation (CO24-CO29) 816 -

Energy and Climate change (CO30 - CO34) 697 45

Social infrastructure (CO35-36) 150 -

Urban development (CO37-CO40) 252 -

Total 5201 180

ESF indicator dimension (Common indicator numbers) Number of times indicators usedacross EU-28 ESF programmes

Output indicators – Participants (CO01-CO19)

Employment status (CO1-CO5) 3,654

Age (CO6-CO8) 3,089

Educational Attainment (CO09-CO11) 3,135

Disadvantaged Participants (CO12-19) 5,778

Output indicators – Entities (CO20-23) 1,239

Immediate result indicators on participants (CR01-CR05) 2,838

Long Term Result Indicators (CR06-CR09) 718

Total 20,451


76

Source: Own elaborations based on European Commission Common indicator guidance and data from the ESIF

open data portal at: https://cohesiondata.ec.europa.eu/countries

Common indicators are not explicitly linked to different thematic objectives and their thematic

coverage varies, hence an analysis of indicator use in relation to available financial allocation per

thematic objective and without consideration of programme specific indicators (that may be used

where there is a gap or lower coverage in the indicator set) is problematic. Generally, the larger Member

State with the largest national allocations and number of programmes demonstrate greater usage of

each of the common indicators (e.g. Italy uses common indicators 788 times within its programmes,

Poland 734 times, and France 693 times).

The overview of common indicator usage across the EU-28 Member States shows the highest usage

under ERDF for indicators related to productive investment63. These represent more than half of

the common indicator usage and relate to support to enterprises that can be funded/relevant under

multiple thematic objectives and investment priorities and hence appear more than once in individual

operational programmes. Other thematic categories of indicators are more straightforward or

demonstrate clearer linkages to thematic objectives (e.g. transport indicators). The one ICT indicator64

is used the least, with only France, Spain and Italy showing any significant usage nationally. Transport

indicators (e.g. new or improved rail, road, metro/tram and inland waterway) are also only significant

in Spain, Greece, Italy, Poland and to a lesser extent Romania under ERDF, although use of transport

indictors is more even under the CF EU-15 Member States. Environmental indicators, relating to

waste, water, land rehabilitation and habitats as well as risk prevention, appear more frequently under

the OPs in Spain, France, Greece, Italy, and Poland. Urban development indicators are most significant

in France, Portugal and Germany.

Common ESF output indicators are used far more frequently than common result indicators.

Indicators related to age (under 25) and employment status (unemployed) are by far the most

utilised indicators in line with the target groups of much ESF activity. The disadvantaged participants

dimension contains the most utilised indicators (although they are greater in number; each one is used

on average 722 times). Immediate result indicators are more popular than the long-term result

indicators, with specifically the last three indicators65 the least used of the entire set of common

indicators (used on average 137 times). The Polish and Spanish programmes have by far the most

extensive usage of ESF common indicators, followed by the Italian and German programmes. This data

analysis has limitations and could benefit from further exploration in relation to financial allocations to

the different themes, i.e. ICT and transport infrastructure may not be relevant in significant numbers of

programmes either due to their reduced funding allocations, regional development needs, eligibility

criteria of different categories of region as well as number of programmes per Member State. This is

problematic considering the lack of explicit link between thematic objectives and common indicators.

63 See Table 2.1 for a summary overview and Appendix D.64 Additional households with broadband access of at least 30 Mbps65 The indicators relate to participants over 54 years of age or disadvantaged that are in employment after 6 months orparticipants with an improved labour market situation after 6 months of leaving.


77

Table 3.2: Average number of core/common indicators per OP (2007-2013 are those with targetvalues included only)

EU Member States

ERDF

2007-13

ERDF

2014-20

ESF

2014-20

ERDF

2007-13

ERDF

2014-20

ESF

2014-20

Austria 15 24 196 Italy 6 28 185

Belgium 4 26 120 Luxembourg 4 8 81

Bulgaria 5 9 91 Latvia 4 33 131

Croatia n/a 30 149 Lithuania 17 48 191

Cyprus 8 24 52 Malta 11 33 125

Czech

Republic

10 13 126 Netherlands 6 17 54

Denmark 4 17 104 Poland 29 37 240

Estonia 8 23 182 Portugal 5 35 55

Finland 7 14 107 Romania 5 15 16

France 7 21 63 Slovakia 10 25 88

Germany 13 23 136 Slovenia 24 26 139

Greece 22 34 70 Spain 6 24 165

Hungary 5 22 87 Sweden 3 17 54

Ireland 14 15 137 UK 6 32 80

Source: 2014-2020 data: own elaboration from Open Data Platform; 2007-2013 data from The Potential forRegional Policy Instruments, 2007-2013, to contribute to the Lisbon and Göteborg objectives for growth, jobsand sustainable development, 2009.

The following boxes provide an analysis of the common indicators for ERDF and ESF interventions thathave been most frequently used during the 2014-2020 programming period.

Box 3.1: Most Used ERDF Common Indicators 2014-2020

Number of enterprises receiving support – used 749 times across mainstream ERDF

programmes in EU-28, in 90.5% of OPs and on average 4 times per OP.

Employment increase in supported enterprises – used 324 times across mainstream ERDF

programmes in EU-28, in 72.6% of OPs and used on average 2 times per OP.

Estimated annual decrease of greenhouse gases – used 327 times, in 73.6% of OPs and an

average of 2 times per OP.

Source: own elaboration from Open Data Platform

Box 3.2: Most Used ESF Common Indicators 2014-2020

Outputs

Employment status - used 2,390 times, in 98.3% of OPs, used on average 13.5 times per OP

Age – used 2513 times, average of 13.5 times per OP, in 97.2% of OPs, used on average 14

times per OP.


78

Educational attainment – used 3135 times, average of 18 times per OP. In 96.6% of OPS.

Results

Participants in employment, including self-employment, upon leaving - used 607 times, in

82.7% of OPs, used on average 4 times per OP.

Disadvantaged participants engaged in job searching, education/ training, gaining a

qualification, or in employment, including self-employment, upon leaving – used 637 times,

on average 4 times per OP. In 86.6% of OPs.

3.2 ERDF COMMON INDICATORS

The ECA special report number 2/2017 calculates that on average each ERDF OP used 13.2 different

programme specific output indicators and 14.8 common output indicators, and stressed the benefits

of a more streamlined, simplified and harmonised framework. A 2018 study on post- 2020 ERDF and

CF indicators66 produced a number of relevant findings:

Box 3.3: Summary of Key Points - 2018 study on post-2020 ERDF and CF indicators

The 2014-20 indicators are not explicitly linked to the Thematic Objectives (TO) but can be

used with more than one TO;

Thematic coverage varies - some TOs lack sufficient coverage of common indicators (TO1 and

TO3 have the greatest number or coverage of relevant indicators and TO11 the lowest).

Programme-specific output indicators can be introduced where common indicators are not

considered representative or relevant to programme activity.

Only 59% of ERDF and 40% of CF output indicators (from a sample of programmes) are

common output indicators. The use of common output indicators is uneven across TOs,

Investment Priorities (IPs), funds and MS. Programme specific indicators have been introduced

in 70% of cases. Under the CF, 4 countries use more common than programme-specific

indicators.

In the case of the ERDF, Croatia, Czechia, Estonia and Slovakia use more programme-specific

than common output indicators. Additional common indicators are needed to cover gaps in

the current indicator set.

Result indicators measure changes in the programme area but are programme specific.

Whilst other ESI funds have direct result indicators, the lack of ERDF / CF common direct result

indicators is an obstacle to further harmonisation and simplification;

Clearer definition of indicators is also required to reduce the need for programme specific

indicators that are of potentially limited use and have lower levels of accountability and

visibility for EU funding;

In terms of reporting against the indicators, those that relied on reporting directly from

projects and were based on previous similar monitoring experiences were considered to have

66 T33 and SWECO, 2018, Development of a system of common indicators for ERDF and CF interventions after 2020


79

the highest level of feasibility. Indicators that rely on input from external sources and of which

there is limited previous experience in measurement and reporting and challenging

definitions have lower levels of feasibility;

Introducing a common methodology for indicators which have previously proved

challenging should be a pre-condition for their inclusion in the post-2020 regulatory package,

e.g. those relating to estimated greenhouse gas (GHG) emissions;

Improved monitoring needs to be accompanied by evaluation to determine whether

changes can be attributed to project and programme implementation.

Box 3.4: Spain - Examples of use of programme specific indicators instead of commonindicators

The MA had doubts over whether Clean Points could be included under the definition of

indicator C017 – Additional waste recycling capacity: “Annual capacity in tonnes of newly

built waste recycling facilities, including extension of existing facilities”, and there was no

clarification from the EC. Fearful that in a subsequent audit Clean Points would be deemed

unacceptable, the MA created a programme-specific indicator with a suitably broad definition.

Indicator E022 – Additional capacity for collection and transport of urban solid waste

covers: “Additional capacity for collecting and transporting urban Solid Waste, which is the co-

financed operation. This Indicator will cover the realization of all types of operations that

produce an additional capacity to collect and transport urban Solid Waste, such as: realisation

and implementation of Plans and Management improvement studies, Plans and studies of

organizational improvements, creation of new facilities, acquisition of equipment, expansion

of facilities, etc”.

Another example where a specific indicator was created because none of the common

indicators were considered appropriate is the indicator E050 that was chosen to measure the

number of contractors in the context of the creation or renovation of big R&D

infrastructures. Whilst common Indicators CO01-05, CO26, CO28 and CO29 are close they do

not specifically enable measurement of enterprises being hired as contractors (only enterprises

as beneficiaries of grants, advisory support or financial instruments).

The Regulations allow for financial sanction in the case where there are issues in the monitoring

systems. A recent DG REGIO audit found serious issues in only 8 out of 26 programmes audited and the

European Commission has initiated a procedure where necessary that ultimately could result in

payment suspension67. The annual control reports due in February of each year also contain a

requirement to report on data reliability.

67 Information about this audit is not publicly available but an overview of issues can be found here:https://ec.europa.eu/regional_policy/sources/docgener/evaluation/doc/20062018/reliabilityindicators_june2018.pdf


80

3.3 ESF COMMON INDICATORS

The 2017 synthesis of the ESF Annual Implementation Reports (AIRs) reported on the findings from

an analysis of 61 accepted and 97 Managing Authority (MA) admissible reports, as well as another 10

AIRs that were deemed non-admissible.

It highlighted the strengths and weaknesses of the ESF common indicators - missing values, ’zero’

values, extreme values, coherence between output and result values, unit costs, identification

of measurement of units68. The report made a useful distinction between: (i) the detection of trivial

errors (gaps, inconsistencies and format errors); and (ii) the detection of performance peculiarities

(under- or over performance, etc.) and concluded that considerable efforts had been undertaken to

improve the use and operation of the indicators, especially in the previous year:

Monitoring data from AIR 2015 showed that 89% of the ex-ante conditionality No. 769 criteria still

needed to be fulfilled, up from 15% the year before;

1,373 targets had been added to result indicators, meaning that 88.7% of programme-specific

results indicators had a target as at the end of 2017.

However, some countries (DE, FR, GR, HU and UK) still did not fulfil certain ex-ante conditionality

No. 7 criteria.

The successful use of the ESF common indicators depends on having systems in place in the

Member State to collect the necessary data and to analyse it. Ex-ante Conditionality No. 7 sets a

number of detailed sub-criteria for ESIF data collection and result indicators for the 2014-2020 period

(e.g. arrangements for timely collection and aggregation of data, and an effective system of results

indicators). A report produced by the European Commission is, overall, quite positive about the

reporting by Member States70. Based on an analysis of the 2015 AIRs, the report established that most

Member States had ensured that appropriate ESF statistical systems were in place. It highlighted a small

number of Member States as lagging in this respect (FR, GR, HU, and IT). However, it noted that the

largest number of actions had been taken to ensure that each ESF programme had specific result

indicators, albeit that in some cases not much remained to be done (e.g. a final assessment to confirm

that the objectives and indicators are in line with the guidance).

A more recent assessment in May 201871 based on information extracted in November 2017 from the

AIRs for 2016, concluded that there were 15 ESF OPs in five Member States (DE, GR, FR, HU and the UK)

in which at least one of the six criteria had not been fulfilled. Criterion 7.4, dealing with ESF results

indicators and targets, remained the most problematic, remaining unfulfilled for all but one of

the OPs.

68 European Commission, ‘Synthesis Report of ESF Annual Implementation Reports, Final report’ (Fondazione G.Brodolini,November 2017).69 General ex-ante conditionality 7 concerns the existence of a statistical basis necessary to undertake evaluations to assessthe effectiveness and impact of the programmes. See page 28 for further information.70 European Commission, Directorate-General for Employment Social Affairs and Inclusion, Synthesis Report of ESF 2016Annual Implementation Reports (December 2016)71 European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Study on the Monitoring andEvaluation Systems of the ESF (May 2016), prepared by Applica, WIIW and Tarki.


81

Based on a survey of MA, this report identifies a number of the difficulties faced by national authorities.

The first of these related to the complexity of the requirements: two-thirds of the MAs surveyed

reported some/significant difficulties in setting up monitoring systems for the 2014-2020

programming period due to difficulties understanding the reporting requirements (67%) and/or

coordinating with beneficiaries to establish compliant procedures (65%). It was argued that the

Regulations were too detailed, and that the reporting requirements for the AIRs are too complicated

and time-consuming.

The second factor related to the complexity of some common indicators. According to the MAs

surveyed, the use of complex common indicators places excessive demands on data collection and

data processing, adding to the already high administrative burden72. In fact in response to the requests

from Member States, three indicators relating to households were removed during a revision of the

regulations in 2018 as data collection against them was too complex and onerous73. Data on

participants should include personal non-sensitive data on gender, employment status, age and

education attainment levels. Where this cannot be recorded, no data at all on that particular participant

should be reported. This can lead to under-reporting of the actual number of participants under a

programme74.

Moreover, the EC’s May 2018 report argued that some common ESF indicators are not particularly

relevant. Some of the current output indicators on participants are not necessarily a reflection of

disadvantage, which could lead to misleading interpretation and some common result indicators

focusing on labour market status and transitions are not always relevant to the objectives of all

operations. This, it was argued, is particularly true in regard to measuring the progress towards social

inclusion (TO 9) and reflecting the objectives and expected results of interventions targeting entities

(e.g. SMEs) and institutions (TO 11). Additionally, results do not always materialise at the required

data collection point for the common ESF indicators (on exit for immediate results and 6 months

after exit for longer-term results). Finally, the fact that common indicators are not always suitable

for target setting limited their use and created a need to establish many programme-specific

indicators which cannot be aggregated at the EU level and adds further complexity to data collection

processes. Other shortcomings identified were:

Box 3.5: Analysis of ESF Annual Implementation Reports’ Shortcomings

The lack of common definitions for certain common output indicators referring to “sensitive

data” (e.g. those related to disability, and to migrant or minority status).

72 Examples cited in the report include combining different variables such as the common output indicator “above 54 years ofage who are unemployed, including long-term unemployed, or inactive not in education or training” or the common longer-term result indicator “disadvantaged participants in employment, including self-employment, six months after leaving”).73 These were: participants who live in jobless households, participants who live in jobless households with dependentchildren and participants who live in a single adult household with dependent children. See:http://data.consilium.europa.eu/doc/document/PE-13-2018-INIT/en/pdf, Article 273.74 Under-reporting, the difference between the grand total and the total of participants, enables the number of eligibleparticipants for whom not all non-sensitive data has been collected to be identified. DG EMPL's audit methodology uses 10%as a benchmark for under-reporting, above which an in-depth analysis about its causes, as well as additional efforts by MAs toreduce it, are necessary. A similar approach is applied in case of a gap between totals of participants by employment statusand educational attainment levels.


82

Problems with the ‘single participation record’ requirement’75;

Confusion over the unit of measurement (notably the use of absolute values vspercentages);

Difficulty in following the EC’s guidance on drawing-up of representative samples (thisoption was introduced as an alternative to 100% target group coverage as a way of making iteasier to satisfy reporting requirements).

A lack of resources (53% of MAs) and/or relevant expertise or technical capacity (51% of MAs)to support the establishment of ESF monitoring systems for the 2014-2020 period.

Under the Youth Employment Initiative (YEI)76, a small number of output indicators tend to be used:

seven OPs specify one output indicator (BE-BXL, CY, FR, IE, SE, SK, RO), while 13 OPs specify a small

number of output indicators (BE-WL, CZ, EL, ES, HR, HU, IT, LT, LV, PL, PT, SI, UK-SC). The focus on one or

a small number of output indicators in most Member States reflects the limited focus on specific sub-

groups, e.g. the low skilled, those with disabilities, in the OPs. More detailed output indicators for

specific sub-groups can ensure that those – often more disadvantaged – sub-groups are targeted

specifically.

In the same DG EMPL 2016 report, it was noted that anticipated results varied significantly across

OPs. While the result targets followed the indicators set in Annex II of the ESF Regulation across

Member States, there was significant variation in how ambitious targets are, varying between relatively

low numbers or proportions (>20%) to high (<80%) in respect of proportions gaining employment,

qualifications etc. Of all participants who joined a YEI measure, it was expected that approximately 75%

would complete; of those who completed the intervention, typical expected proportions for those who

will achieve qualification, training or employment outcomes range between 25 and 50%.

An additional issue with regard to monitoring and reporting is the availability of up-to-date output

data which is collected and submitted to the European Commission on an annual basis. In the absence

of real time monitoring systems at MA level, data cannot be extracted readily77. Data held by the

European Commission on performance is not timely but can date from up to 2 years previously due to

the timing of the annual implementation report cycle and may also be limited in scope/marginal in

nature.

The results of our research backed up these findings. Where there is no specific unit of measure

provided in the regulations or the guidance, a percentage can be used instead of an absolute number

which means aggregation is not possible. So for example under ESF, only 62% of result indicators for

75 For example, individuals entering more than once in the same operation will have only one result (situation after the finalparticipation), however, in reality, they will have had multiple spells of participation, each of them potentially with differentoutcomes.76 European Commission, First results of the Youth Employment Initiative, pp 11-12 (Ecorys and PPMI, June 2016).77 Spain, as one of the major recipients of YEI funding for example, delivered the YEI through regional and local authoritiesand was – at the end of 2015 - yet to finalise a comprehensive monitoring system which would bundle the information fromall its implementing partners and enable it to distinguish participants of YEI-funded activities from those young people whohad registered in the national Youth Guarantee scheme. The conclusion of the study was that this could lead to futurechallenges on the feasibility of such an undertaking, where data will have to be gathered and entered into the consolidatedsystem retrospectively


83

which a target was set can be aggregated. MAs reported difficulties in applying indicators with

percentages at project level also and hence these should be avoided in future programming provisions.

Box 3.6: Key Research Findings on ESF

ES – Spain has chosen not to fund certain activity under ESF due to the onerous or non-

proportionate nature of reporting and instead funds certain activities only through national

funds. A key difficulty is in aggregating the data across the wide number of interventions,

structures and lengths of intervention.

MT – The final implementation report for 2007-2013 concluded that the indicators and results

were unclear, programme implementation had been delayed, and the evaluation culture and

capacity was weak. In 2014-2020 Malta have strengthened their capacity/expertise through

working on long term indicators with the national statistical agency. The MA also provides

comprehensive training for all applicants around data reporting requirements and

performance indicators.

PL – is the only example under ESF where payments to projects are linked to achieving results

i.e. that the MA can reduce the amount of funding awarded if certain thresholds and results

are not achieved. This requirement is established in the grant contract with the applicant,

although the application is flexible in light of changing circumstances and applied in light of

the principle of proportionality.

A “creaming effect” is a significant risk under ESF: in trying to meet targets, interventions

can be directed to the easiest and least disadvantaged participants and beneficiaries i.e. those

that are already closest to the labour market rather than harder to reach groups that display

the greatest need.

3.4 COHESION POLICY CONDITIONALITIES

A recent study for the European Commission (2016) provides a useful initial assessment of the ex-ante

conditionalities. The research findings highlight the value of ex-ante conditionalities, in

encouraging the fulfilment of EU regulatory requirements faster than might have been the case

in their absence and reinforcing effectiveness through associated strategies in the policy areas

supported by the ERDF and ESF. According to the study, around 75% of the applicable general ex-ante

conditionalities and 58% of the thematic ones were considered to be fulfilled at the time the ESIF

programmes were adopted.

The European Commission has identified several ways in which ex-ante conditionalities help to

improve the effectiveness of public investment and stimulate structural changes in EU Member

States.

Box 3.7: European Commission findings – Role of Ex ante Conditionalities in PromotingResults Orientated Interventions

Of particular relevance to this Research Paper is the argument that is the ex-ante


84

conditionalities encourage the more effective targeting of ERDF and ESF interventions.

Moreover, many ex-ante conditionalities require that support from the ESIF forms part of policy

or strategic frameworks which meet certain quality criteria. Some of them require

prioritisation of investments based on a needs analysis, including national and regional public

investments. For instance, transport-related ex ante conditionalities require comprehensive

national or regional transport plans to be in place, accompanied by a well-developed project

pipeline.

It is also argued that the ex-ante conditionalities have encouraged the development of

effective institutional structures and implementation mechanisms for ESIF funding. This is

important because insufficient capacity and efficiency of public administrations in some MS

and regions have a negative impact on the implementation of the ESIF and the competitiveness

of EU regions.

Overall, the European Commission study concludes that the ex-ante conditionalities have proved to

be an important incentive for Member States and regions to improve their investment

frameworks. Looking ahead, it identifies several challenges: firstly, the complexity of the ex-ante

conditionalities process which requires an additional workload and costs, particularly for those

Member States with a large number of IPs and TOs in relation to the allocated EU funds.

Secondly, the scope of the system is seen as involving too many ex- ante conditionalities, and there

are also some inconsistencies related to the fact that the conditionalities do not apply to the EU funds

outside the ESIF. Thirdly, the fact that the rules do not foresee monitoring of the ex-ante

conditionalities by the Commission once they are considered to be fulfilled; and last but not least,

there is a criticism that under the current rules, Member States were expected to fulfil the

conditionalities by the end of 2016 but ESIF programmes could have been launched and payment

claims submitted to the Commission before the fulfilment of the conditionalities. A further assessment

has been undertaken by Swedish Institute for European Policy Studies (SIEPS). According to this:

“Most ex-ante conditionality criteria have been relevant for all Member States’ specific objectives

and the fulfilment rate is very high. Ex-ante conditionality resulted in a more effective and

structured deployment of ESI Funds; and provided incentives for Member States to implement

policy reforms. Nevertheless, there are also important difficulties. The main critiques on ex-ante

conditionality have been in the complexity of the process and the administrative burden”.

SIEPs also questions whether the very broad spectrum of criteria (e.g. from waste water management

to reforms in the pension system) could be undermining the trend toward more transparency and

simplification. “A clear link between the ex-ante conditions and financial support could overcome this

situation”, it is argued. As an alternative to reducing ESI funding if ex-ante conditionalities are not met,

the SIEPS paper argues in favour of ‘positive’ conditionality, i.e. the strengthening of the link between


85

incentives and fulfilment of specific ex-ante conditionality (e.g. in line with the YEI, which may

concentrate on the specific budgetary burden related to migration)78.

Overall, the SIEPS paper concludes that conditionality is still a controversial mechanism with legal

and political implications that have yet to be clarified. It argues that, looking ahead, new criteria

could be added, for instance criteria linked to the European challenges relating to demographics,

education, social inclusion and migration. In addition, there could be a stronger focus on results

through specific payment based on outputs and results. More flexibility to re-adjust the budgetary

allocation could help to react to unforeseen challenges and to create positive incentives. It is also

argued that the macroeconomic conditionalities should be more closely linked to specific goals. Last

but not least, there should be a thematic concentration in Cohesion Policy programmes on a small

number of priorities so there is greater coherence among EU instruments and they all support common

EU objectives. It is argued that an increased conditionality would require the EP to have a stronger say

to address the criticism that conditionality has no ‘democratic’ basis.

3.5 PERFORMANCE FRAMEWORK

The European Court of Auditors (ECA) has welcomed improvements implemented by the European

Commission in the 2014-20 programming period to improve the focus of Cohesion Policy upon results.

The ECA Special Report No 2/2017 concluded that the European Commission and Member States

had been successful in developing OPs with a more robust intervention logic i.e. with a clear link

between development needs, specific objectives and result indicators. Nevertheless, they

highlighted weaknesses in the effectiveness of the measures and the quality of the monitoring

information available.

A 2016 evaluation79 of the implementation of Performance Frameworks under ESIF found that the

development of the performance framework, selection of indicators and definition of milestones and

targets was largely based upon past experience and documents such as AIRs, previous programming

documents and internal monitoring and evaluation reports. Processes were generally robust and

established to ensure consistency and coordination at the national level, through the creation of

national guidelines and harmonised procedures. Key findings included:

Box 3.8: Key evaluation findings on Performance Frameworks (2016)

Nearly 70% of MS apply harmonised procedures, 57% have national guidelines, 54% held

seminars and workshops and c. a third introduced regulations to ensure consistency.

Financial and output indicators are most frequently used in performance frameworks (88% of

the total). Approximately 50% of all output indicators are common ones with high numbers found

78 With the macroeconomic conditionalities, experience with the implementation of the new provisions is limited so far.According to the EC’s assessment, there has so far been only one case, namely the decision of the Council to suspend the CFin 2012 in Hungary because of its failure to take effective action to address its excessive deficit (this suspension wasnevertheless lifted before having an actual impact because Hungary adopted the required effective action).79 EC, DG REGIO, 2016, The implementation of the performance frameworks in 2014-2020 ESI Funds, evaluation by SWECO,OIR and Spatial Foresight


86

in AT, DK, LU, and SE. Furthermore, in BG, ES, EL, HR, IE, NL, SE and the UK, more than 60% of PF

output indicators are common.

The advantage of programme-specific indicators is their relevance to the specific programme

context and the reflection of individual characteristics of each region and programme.

More than 65% of Partnership Agreements have identified more than two indicators for their

performance framework and the remaining 35% have only two indicators.

Key implementation steps are used to a lesser extent and mostly where no measurable output

is expected by the end of 2018. 75% of all PAs do not contain implementation steps. The highest

numbers of key implementation steps are used in EL and BG, and under TO7 (Promoting

sustainable transport and removing bottlenecks in key network infrastructure).

The quality of indicator milestones and targets is assessed as high overall, however consistency

across programmes was determined difficult to assess.

MA and stakeholders generally had a positive opinion on the performance framework and felt

them to be beneficial in relation to enhanced result orientation, thematic focus and the definition

of realistic targets and expectations. A few concerns related to targets being set too low.

While the evaluation found that the performance reserve had acted as an incentive for a more effective

and efficient use of funds, ECA special report 15/2017 notes that the performance framework and

reserve provide little incentive for a better result orientation of the OPs since they are mostly based on

spending and outputs80.

Ex-ante conditionalities were considered an innovation in Cohesion policy and provide a consistent

framework for assessing the Member States readiness to implement EU funds at the start of the 2014-

2020 programme period. Nevertheless, it is unclear to what extent this has effectively led to changes

on the ground, with the European Commission not making use of the possibility to suspend payments

on the basis of unfulfilled ex-ante conditionalities. Around half of the more than 700 action plans

adopted by Member States to fulfil all ex-ante conditionalities were not reported as completed by the

end of 2016, covering c. 27 % of the ERDF, CF, and ESF spending.

A 2018 report determined that the focus on results has yet to become embedded in the project

selection process despite the fact that a results-oriented approach must be followed through

during implementation (project selection, monitoring and reporting). The design of selection

processes continues to emphasise outputs and absorption rather than results and be undertaken on a

first-come first-served basis with results indicators rarely included in grant agreements.

Whilst selection procedures are at the discretion of the Member States (with PMC approval),

some comparison between project applications should take place to ensure that the most

results-oriented projects are selected. The ECA found that only 20% of selection procedures

examined included criteria requiring the quantification of result indicators at project level and in 75%

of cases, these result indicators did not correspond directly to those defined at OP level. In terms of

80 ECA special report 15/2017 Ex-ante conditionalities and performance reserve in cohesion: innovative but not yet effectiveinstruments. Available at: http://publications.europa.eu/webpub/eca/special-reports/partnership-agreements-15-2017/en/


87

approved projects only 41% of those reviewed had quantitative information on expected result

indicators in their application and grant agreement, and in more than 40% of cases, the result indicators

did not correspond directly to the OP indicators.

Box 3.9: Project Selection Procedures and Monitoring Data

From our research Bulgaria and Croatia are examples of where calls for proposals outline the

programme indicators that projects must contribute towards.

Further work is being undertaken in Croatia to link payments more directly to project

achievements and hence ensure their result-orientation.

In Estonia and the other Baltic States, SMEs and other beneficiaries are contractually obliged

to provide the authorities with the data required for the common ERDF indicators.

Shortcomings in monitoring systems mean that assessing the actual contribution of EU funding

to EU and Member State objectives is problematic. The late approval at EU level, in December 2013,

of the legislative framework for the period 2014-2020 delayed the development of Member States’ IT

monitoring systems. For the ESF indicators, added complexity concerning the sensitivity of the data

and the definition of the indicators meant that discussions went into 2014 and further delay. At the end

of 2016, 50 action plans in relation to the level of implementation of a system of results indicators under

EAC 7 were incomplete, i.e. around 11% of all OPs still did not fulfil their regulatory requirement81. The

ECA found that in the first half of 2017, the fourth year of the programming period, some IT systems

were still not fully functional, posing risks to the completeness and accuracy of the data collected. At

the end of 2017, 26 AIRs had still not been accepted by the Commission due to missing data for the

indicators used in the performance framework, inconsistencies in the data reported with the

information provided on the implementation of the OP, or a lack of explanation for some of the data

reported.

Approximately 40% of ERDF OPs did not report any values for their performance framework

indicators in 2017. The 2018 AIRs of course form the basis for the allocation of the Performance

Reserve and it is unclear to what extent the systems in place at MA level have fulfilled the criteria

relating to monitoring. The ECA noted that “shortcomings are particularly susceptible to being

identified late and there may not be sufficient time to apply the necessary adjustments, (which)

jeopardises the implementation of the performance review in 2019, which will be based on data

reported by Member States in the 2018 AIRs by 30 June 2019”. The EC’s strategic report for 2017 mainly

presents progress in implementation and achievements against the main output indicators by the end

of 2016, but not on the achievement of results with the exception of: the number of participants that

found a job immediately after the training, and the number of participants that gained a qualification

immediately after training. For the ERDF, this reflects the fact that there are no common result

indicators and hence it is not possible to aggregate those used at Member State level. In the case of the

81 It should be noted that all action plans related to EAC 7 were completed by the end of February 2018.


88

ESF, as the measurement of the values for the common result indicators is still taking place these are

anyway captured in the MA’s IT systems.

The ECA recommended that the European Commission define common result indicators for the ERDF.

Issues also occur around ERDF result indicators at OP level not being directly related to the

funded interventions but being ‘national indicators’ which take into account the influence of

external factors and are unable to isolate the immediate results or impact attributable to the

ERDF interventions. These indicators tend to rely on national statistics bodies rather than project

beneficiaries. Under ESF it is difficult to verify the completeness and correctness of the data concerning

participants (migrants, foreign background, minorities) as the data is classified as sensitive in national

legislation and at the discretion of the participants to decide whether to declare it.

Box 3.10: Issues reported in relation to the 2014-2020 Performance Framework

Misunderstanding regarding the definition of common indicators by either the MA or

beneficiaries.

Inadequate or incorrect collection of data against the indicators e.g. beneficiaries providing

data to MAs late, using different definitions, double counting achievements.

The high number of different performance indicators poses a real challenge to the collection

and reporting of performance information.

OPs reporting having achieved the targets set for 2023 even as early as in the 2015 AIRs, without

requests for amendments to more realistic target values or milestones for the performance

framework having been submitted to the EC (only at the MS initiative) in a timely manner.

The vast majority of the indicators used in the performance framework are related to output

(57%), key implementation steps (9%) and financial indicators (33%). As the use of result

indicators is marginal, the release of the performance reserve will to a large extent remain input-

and outputs-oriented, and not focused on results.

The 2018 ESIF Summary Report82 notes that the intervention rationale focus of the 2014-2020

programmes with the emphasis on the wider use of common indicators is leading to more robust and

coherent performance reporting. Reporting inconsistencies detected in the annual reports during the

quality check process are queried with the MA. The large majority of the audits carried out to date by

the European Commission have arrived at an overall positive assessment of the reliability of the

systems, with only a few programmes identified as having system deficiencies or serious data

inaccuracies.

3.6 PERFORMANCE RESERVE

A Performance Reserve as a means of rewarding prompt implementation and linking achievement of

results to additional funding was previously employed in the 2000-2006 programming period. The

82 EC, ESIF 2014-2020, 2018 Summary Report of the programme annual implementation reports covering implementation2014-2017


89

European Commission allocated 4% of the Structural Funds' budget in 2004 to programmes, on the

basis of efficiency, management and financial implementation indicators.

A DG REGIO report on the performance reserve in 2004 noted that one of the strengths of the approach

was the incentive it provided in terms of good programme management practices to ensure timely and

adequate expenditure monitoring and evaluation, control processes and project selection. This was

seen as particularly important in the context of the enlarged EU and the need for capacity building in

management and administrative capacity in new Member States. Whilst the N+2 rule was the main

impetus for improved levels of expenditure, the performance reserve process encouraged capacity

building around identifying appropriate indicators and setting targets, and improved transparency.

Effectiveness indicators were more complex and the majority of the weaknesses in the system

occurred here. The EC’s proposal was that they should comprise a subset of the output and result

indicators of the programme, representing at least 50% of expenditure. All Member States chose both

output and result indicators, except for Greece, Italy Objective 2 and Denmark, which used only

outputs, and Sweden, which used only results. It was recommended that more guidance and

appropriate benchmarks should be proposed by the European Commission for any future process 83.

Box 3.11: Italy – Assessing Performance

Italy demonstrates good practice in appointing a group of experts in an advisory role to assess

performance under Objectives 1 and 2 in relation to the 2004 Performance Reserve award. The

working groups were chaired by and included representatives from the Italian Evaluation Unit

alongside regional evaluation units and two independent experts nominated by the EC and

produced in-depth reports on the performance of each programme against the agreed criteria84.

In the 2007-2013 programming period a voluntary PR was included in the regulations and its use at the

discretion of the Member State. Either 3% could be set aside as a national PR, or 1-3% as a National

Contingency Reserve to cover unforeseen local/sectoral crises. The reserve was not widely taken up.

In the current programming period, the Performance Reserve is considered conceptually very good but

practically irrelevant in the context of reprogramming possibilities and a constraint on performance,

encouraging resources to be directed towards areas where easier results can be obtained rather than

to harder to reach groups for example and adding administrative burden to the programming process.

Few Member States will lose Performance Reserve funds allocated as a result of underperformance, as

they will be simply reallocated to other priority axes or programmes (and could theoretically be

reallocated back in a subsequent reprogramming).

In most cases underperformance has reflected real change, external factors or difficulty in

implementing on the ground e.g. the complexity of delivering large infrastructure projects, delay in OP

approval, outmigration leaving reduced target groups for specific interventions. These are issues which

the PR cannot mitigate against and which the European Commission considers justified reasons for

83 https://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/performance_midterm_2004.pdf84 Ibid


90

reprogramming, including target reduction, and a number of programmes sought modifications in

2018 in advance of the 2019 review. Only two examples were provided of a specific push to

implementation as a result of Performance Reserve milestones, in Bulgaria and Malta, and in specific

cases where acceleration was possible e.g. non-infrastructure related. As the Performance Reserve

measures progress in and speed of implementation it is not appropriate that the methodology is

consistent across all sectors, as this encourages allocations to be moved away from areas and sectors

where delivery is more problematic or complex such as research. Its focus is still on absorption: the

financial targets reflect n+3 targets and hence are largely insignificant. There is significant political

pressure around the award of the Performance Reserve, it is not a neutral mechanism nor an additional

incentive.

3.7 DATA COLLECTION

Member States have embedded the results-orientation approach to varying degrees in their project

selection processes. Relevant indicators that projects should utilise and contribute to are included in

the call for proposals documentation and project selection criteria in a number of Member States.

Performance milestones and targets need to be embedded in grant contract conditions.

Whilst there are increasing efforts to link payments to results and these should continue to be

developed, drawing broader conclusions that link costs and outputs should be avoided. Few Member

States directly link project results to payments, although the Polish ESF programme explicitly requires

achievements against project results in order for full payment as part of the grant contract. There is

flexibility built into the process however in line with the conclusions of the ESF ex-post for 2007-2013,

and so projects are able to justify not meeting them at times. Programming instruments such as

simplified cost options enable this approach and pilots are currently being undertaken by DG EMPL in

order to demonstrate their usage.


91

4 CONCLUSIONS & RECOMMENDATIONS

Overall, this Research Paper suggests that good progress has been made to develop an effective

system of performance indicators to monitor the contribution of Cohesion Policy to convergence

in regional development.

There is now a well-designed set of output indicators in place for both the ERDF and the ESF. However,

the challenge remains to shift the focus more from measuring financial inputs and outputs to being

able to assess the more meaningful results and impacts of interventions. There is also a lot still to be

done to develop data collection systems that are capable of generating the type of information

required to assess results and impacts without imposing an unreasonable administrative burden on

national authorities and Cohesion Policy beneficiaries.

4.1 EVOLUTION OF THE ESIF PERFORMANCE MONITORING APPROACH

There have been significant improvements to the ESIF monitoring and evaluation approach

across each programming period. In summary, before 2000, there was no common approach to

defining indicators and targets with Member States able to choose their own, leading to a wide range

of indicators being used across ERDF and ESF programmes with very limited harmonisation. Not only

were different indicators used by different Member States and regions but even where the same

indicators were used, definitions varied as did the methodologies used to implement them. As a

consequence, it was difficult if not impossible to aggregate information at EU level on the results of the

policy85.

The key developments in the ESIF regulatory framework that have taken place over the years are

summarised below.

85 Overall, it is estimated that there are some 10,000 users across EU Member States of the data generated from the commonindicators.


92



93

4.2 USE OF COMMON AND PROGRAMME-SPECIFIC INDICATORS

The current 2014-2020 mainstream ERDF and ESF programmes are being monitored against a

set of 72 common output and results indicators. At the same time, MAs are making use of a large

number of programme-specific indicators. This reflects the fact that whilst the common indicators

should be mostly relevant to all Member States and regions, there may be a need for others that more

closely reflect regional specificities (e.g. there are instances where indicators are not relevant - km of

rail constructed is not relevant in Member States with no railway network). A notable trend over the

years has been the effort to promote simplification with the number of different indicators being

progressively consolidated.

The ESF currently has 32 common indicators (23 for outputs and 9 for results) that represent the

minimum that should be included in each OP. The most frequently-selected indicators refer to the

number of employed participants in ESF-funded programmes, followed by the number of unemployed,

disadvantaged individuals and young people. Taken together, a total of 16,895 output and 3,556 result

indicators are being used in the 180 ESF OPs during the current programming period (it should be

noted that some programmes are multi-fund and are covered under both the ESF and ERDF figures).

Whilst only some 15% of indicators used in ESF OPs are common indicators, programme-specific

indicators for the most part reflect sub-categories or combinations of categories used within the

common indicators. In addition to the output indicators, the ESF has defined common indicators for

‘immediate’ and ‘longer-term’ results.

In the case of the mainstream ERDF programmes (i.e. excluding ETC), the OPs currently have a

total of 40 common output indicators for the current 2014-2020 period. These have been used

5,757 times in the 276 OPs and account for roughly 55% of all indicators used and cover roughly the

same amount of the investment. The most frequently-used common indicators relate to the number

of enterprises receiving support and the value of grants, the employment increase in supported

enterprises, and the reduction of greenhouse gas emissions. In some cases, where OPs have not used

common indicators they have been encouraged to use EU level indicators (e.g. those derived from the

innovation scoreboard) in order to improve measurability and comparability. ERDF programme-

specific indicators cannot usually be aggregated and have arguably been over-utilised within the OPs’

performance frameworks. Unlike the ESF, the ERDF has not developed a common set of results

indicators and those that exist are programme-specific.

Overall, the use of obligatory common indicators for outputs (and also results for the ESF) based

on standardised definitions has contributed to strengthening programme intervention logic i.e.

how the planned spending contributes to achieving change in the context of the identified

development needs - and forms part of the programme negotiation process. The transparency,

visibility and accountability of the policy have all improved. Furthermore, defining clear and

measurable indicators and targets as part of the intervention logic at the outset of a programme

encourages an improved focus by MAs and regional/national stakeholders on results and progress in

performance.


94

The current set of ERDF and ESF common indicators is generally well-designed and appropriately

targeted in relation to the goals of Cohesion Policy but there are some deficiencies that need

addressing. Whilst ESF programme specific indicators tend to reflect sub-categories or combinations

of categories used within the common indicators, those utilised by ERDF programmes are extensive in

number and cannot usually be aggregated. The system entails a considerable administrative burden

and cost for sometimes low levels of quality and accuracy. However, simplification and harmonisation,

whilst welcome in terms of reducing administrative burden and increasing accountability, should not

lead to inflexibility in programming or a one-size-fits-all approach.

Recommendation 1: To the extent possible, the process of simplifying the ESIF common

indicator system should continue. There is a need to reflect the principle of proportionality and

ensure a more streamlined approach to the use of indicators and performance/monitoring data

collection. Common and programme-specific indicators should only be used to monitor ESIF

programmes if they produce useful information on performance. The current total of 72 common

output and result indicators is probably about right and should not be increased unless absolutely

necessary. More generally, feedback from the research suggests that there is still potential to

improve the relevance, focus, coverage, measurability, accuracy, and usefulness of the indicators as

well as the data collection systems.

Recommendation 2: The current set of ERDF and ESF common indicators should be reviewed

to ensure comprehensive coverage of Cohesion Policy objectives. There are some gaps in terms

of coverage, for example, the current set of common indicators do not fully cover the types of

activity funded (e.g. ICT), as well as issues relating to measurability (e.g. GHG emissions and energy

savings), usefulness (e.g. number of projects, number of visitors), accuracy (double-counting) as well

as some discrepancy or inconsistency in interpretation (e.g. definitions of enterprises cooperating and

brownfield sites) and aggregation (e.g. when to report data, reporting non-cumulative figures,

reporting sensitive data).

Recommendation 3: In relation to the specific indicators being used for ERDF and ESF

programmes, the EC should work with national and regional authorities to examine the

scope for greater harmonisation. According to an estimate provided by the ECA, there are some

8,000 to 9,000 programme specific indicators being used in the current 2014-2020 programmes.

Greater use of common indicators would make it easier to identify the results and impacts being

achieved at the programme and EU levels.

A significant trend over the years has been a shift away in the ERDF and ESF monitoring and

evaluation frameworks from a focus on the absorption of funding and outputs towards a results-

oriented approach. The ESF is arguably more advanced in developing common results indicators than

the ERDF but this difference in being addressed in the regulatory proposals for 2021-2027. In relation


95

to impacts, neither the ERDF nor the ESF requires common impact indicators to be monitored in the

current programming period. Whilst there is European Commission guidance on the development of

indicators and methodologies that can be used to evaluate impacts, there is no requirement to use

them. Instead, impacts are assessed through ex-post evaluations, which are normally the responsibility

of the European Commission.

Recommendation 4: Achieving a greater focus on measuring ERDF and ESF ‘results’ and

‘impacts’ should be prioritised in order to ensure closer alignment with the ‘Better

Regulation’ principles. Impact indicators have not been required in the current and previous

programming periods and are again excluded from the post-2020 proposals for the ERDF and ESF.

As noted in the report, the ESF already has common results indicators for ‘immediate’ and ‘longer-

term’ effects. A similar set of common results indicators is needed for the ERDF and the fund

proposal for 2021-2027 makes a welcome step in this direction. In both cases, there is a need to

develop the capacity to assess impacts more effectively.

Member States have made considerable progress in embedding the results-orientation

approach in their project selection processes. However, as a 2018 ECA report indicated, the focus

upon results has yet to become fully embedded in these procedures. Relevant indicators that

projects should utilise and contribute to are included in the call for proposals documentation and

project selection criteria in some Member States, but for the most part these requirements are not

formalised in grant contracts. There are increasing efforts to link payments to results and these should

continue to be developed, although incorporating some flexibility to reflect programming complexity.

Recommendation 5: The effort to ensure that the common ERDF and ESF common indicators

focus more on ‘results’ and ‘impacts’ needs to be also reflected in project assessment and

selection procedures. The design of selection processes continues to emphasise outputs and

absorption rather than results and be undertaken on a first-come first-served basis with quantified

results indicators rarely included in grant agreements.

Overall, there is strong evidence that the ESIF Performance Framework and the system of

indicators have shifted the focus more towards performance but mixed evidence of targets and

milestones being defined in a way that strikes an appropriate balance between being realistic

and challenging. Indicators, targets and milestones are defined by Member States and regions on the

basis of their previous and individual experiences of programming achievements and reflect historical

reference data. They are then negotiated with the European Commission in the context of ‘shared

management’ and partnership. Targets and milestones have tended to be set cautiously because of

the risks of under-performing against Performance Reserve targets and consequently triggering

financial penalties.

At present, neither the ERDF and ESF guidance explains how appropriate milestones and targets should

be set beyond taking into account regional needs and resources. The proposed Common Provision

Regulation for 2021-2027 sets out a requirement for the Performance Framework methodology to


96

include the criteria applied, and data/evidence used in the selection of indicators and targets and

assurance of quality as well as any additional factors that may influence the achievements of the

programme and is a welcome improvement. Additionally, it has removed the requirement for the ex-

ante evaluation which has always proved an important element in the process of establishing

indicators/the performance framework and ensuring their relevance, suitability, realism and

quantification.

Recommendation 6: Further consideration should be given to the role of ex-ante evaluation

in ensuring results-oriented ERDF and ESF programmes. The removal of the requirement for ex-

ante evaluation in the 2021-2027 period is arguably a mistake. Previous experience suggests that

ex-ante evaluations can be very helpful in ensuring that appropriate and consistent objectives,

results and targets are set, and that planned interventions are likely to contribute in an effective

and efficient way to Cohesion Policy objectives and ensure EU added value. Ex-ante evaluations can

also be useful in assessing aspects such as what works in what circumstances, the net effect or

impact of the policy and enabling an in-depth assessment of the quality of interventions.

Recommendation 7: More should be done to help MS and regions to define ERDF and ESF

targets robustly with appropriate levels of ambition. The proposals for the 2021-2027 period

develop methodological approaches further whilst also reflecting the ‘shared management’

approach and principle of subsidiarity. Further methodological guidance and support, as well as

suitable capacity to deal with this requirement, should be ensured, especially in the proposed

absence of an ex-ante evaluation.

4.3 ESIF MONITORING SYSTEMS

Good progress has been made in developing efficient ESIF monitoring systems with most EU

Member States providing centralised guidance and IT tools to ensure consistency and built in

plausibility checks introduced at EU level. In the current 2014-2020 period, there has been a greater

focus on trying to ensure that the statistics required for the ERDF and ESF common indicators are

comparable across EU Member States and regions. The introduction of ‘Ex Ante Conditionality 7’86 has

been important in this respect, to help ensure that the ESIF performance framework is supported by an

appropriate statistical system and a common definition of indicators and the information needed for

them.

However, assessments undertaken by the European Commission and the European Court of

Auditors suggest a mixed picture with deficiencies remaining in data collection. A lot of the

information that is required to measure results and impacts at the ERDF and ESF programme level

86 General ex-ante conditionality 7 concerns the existence of a statistical basis necessary to undertake evaluations to assessthe effectiveness and impact of the programmes.


97

comes from those involved in implementing projects. Feedback from the research in Member States

indicates that beneficiaries are often contractually obliged to provide results information to MA.

However, this is not so universally. Additionally, the results of programmes do not always materialise

at the required data collection point for the common indicators (e.g. six months after a beneficiary of

ESF support ceases to participate in a programme), whilst there is a sensitivity with regard to some of

the monitoring data required for the ESIF common indicators in certain Member States (this relates

specifically to some categories of ESF beneficiaries in countries where there are strict privacy laws).

Other deficiencies noted relate to aspects such as simple human error, incorrect units of measurement,

double-counting and a lack of an audit trail. Similarly, there is a very mixed picture with regard to other

methods of collecting data for the assessment of results and impacts with, for example, the practice of

conducting surveys quite common in some Member States but not in others.

Recommendation 8: Further steps need to be taken to improve the timeliness, accuracy and

feasibility of the data collection systems underpinning the Performance Framework for ERDF

and ESF programmes. The timeliness of ESIF performance data should be improved whilst

ensuring that this does not place an unreasonable additional workload on national authorities.

Performance data held by the EC can be two years out of date because of the timing of the annual

implementation report cycle. The proposals for 2021-2027 envisage a near real-time data collection

with monitoring data proposed for submission on a two-monthly basis. Whilst annual reporting is

not frequent enough, a two-monthly cycle risks going to the opposite extreme and is likely to prove

too frequent for MA. As an alternative, 4 or 6 monthly submissions should be considered. The 2021-

2027 regulatory proposal includes the possibility to use national data sources to collect ESF

achievements. Use of existing national administrative registers/datasets should lead to an

improvement in the quality of data (e.g. social security registers) as well as improving the linkage

and alignment of ESF+ with national interventions.

Through the ‘shared management’ of ESIF programmes, the European Commission and Member

States have worked closely together to implement the 2014-2020 ERDF and ESF programmes.

Feedback from the research suggests that partnership working has generally worked well. The

development of improved IT tools has made it easier to transfer data to the EC. There have been many

other initiatives to promote more efficient and effective programme management and monitoring

systems. Problems remain, partly because the legislative basis and other arrangements for monitoring

systems in the 2014-2020 period were not put in place early enough.

Recommendation 9: The supporting systems and rules for the 2021-2027 Performance

Framework should be put in place as early as possible and before the ERDF and ESF

programmes are adopted. The current consultation by the EC with the MS delegations around

indicator sets and definitions through the Evaluation Network is welcome and should be continued,

with observer roles considered (if not already in place) for the other relevant institutions such as

the EP and ECA. The EP should schedule a research report for early in the programming period to


98

determine the extent to which the performance framework for 2021-2027 is focusing more on

results and impacts.

4.4 PERFORMANCE RESERVE AND CONTRIBUTION TO WIDER EU POLICY GOALS

Whilst the improved results-orientation and indicator sets under ESIF programming means that the

focus is increasingly shifting towards performance, the Performance Framework and Performance

Reserve as programming instruments do not reward programme managers on the basis of policy

results but on speed of implementation which is already influenced by the N+2/N+3 automatic de-

commitment rule.

Looking ahead, and given experience in 2014-2020 with the Performance Reserve, the decision

to discontinue it is correct and widely supported, according to our research. Through the

Performance Reserve, stronger incentives have been introduced to encourage the achievement of

outputs. However, the Performance Reserve becomes practically irrelevant in the context of

reprogramming possibilities and few Member States are likely to lose any of their allocation, which will

simply be reallocated to better performing priorities or programmes. The process to date is seen to be

highly political and administrative and lacking in any reflection of the complexity of programming on

the ground. This was emphasised in another ECA special report that noted that the Performance

Framework and Performance Reserve provide little incentive for a better result orientation of the OPs

since they are mostly based on spending and outputs. Whilst the new 5+2 programming approach

could enable a more strategic approach, it is likely to still be overly bureaucratic and output-focused.

The introduction of a more qualitative review process is welcome but should be proportionate and

appropriately responsive i.e. able to reflect new development needs in a timely fashion not as part of a

significant administrative exercise for all programmes at a specific point in time. Current re-

programming possibilities would seem therefore to be mostly adequate and could be enhanced by the

possibility of an interim review enabling regions to approach the process according to their specific

performance issues, domestic circumstances and timing requirements87. It could be possible for

regions/Member States to determine flexibly a proportion of funding (up to a maximum specified

percentage) that would remain unallocated88 and could be then reprogrammed based on programme

performance, new regional challenges or needs and relevant Country-Specific Recommendations.

Member States with a small number of programmes or small financial allocations could decide not to

implement a review and reprogramming process based upon the principle of proportionality.

Recommendation 10: The decision to discontinue the Performance Reserve is widely

supported and any future system that replaces it should be based on a more results-

orientated approach. Although programme outputs are of course important, it is results that

provide a more meaningful indication of performance and the contribution of cohesion policy to

EU objectives. According to our research, the majority of MA considered the process a narrow

87 For example, in the current programming period, the Scottish government bought forward the Performance Review processto 2017 in a so-called ‘early review’ in response to programming uncertainty following Brexit.88 NB the 2007-2013 programming period allowed for a voluntary performance reserve at the discretion of the MS.


99

partial view of what the programmes actually achieve, and particularly rigid and inflexible in

relation to external factors and more innovative and experimental uses of the funds. Whilst its

potential value in terms of acting as a tool for performance-orientation is recognised, it is

considered an overly bureaucratic means by which to achieve this. These and other criticisms will

also need to be addressed in any future performance-based system which should be proportionate

and appropriately responsive. The EC should reconsider other means by which the programmes

and regions of the EU can adapt quickly to crises and demonstrate flexibility and responsiveness. A

balance equally needs to be struck with the need for stable strategic investment planning, which

the 7-year programming period facilitates89. A minor variation on the current re-programming

provisions/possibilities could enable an interim review that takes into account specific regional

performance issues and progress, domestic circumstances and socio-economic situation, as well as

timing requirements, and allow for significant reprogramming on that basis.

Feedback from the research suggests that ESIF spending has been more targeted on Europe

2020 priorities than was the case in previous programming periods in relation to the then-

prevailing EU objectives. However, the ECA and others have argued that ex-ante conditionalities and

the Performance Reserve were innovative but not yet effective instruments for ensuring good results

in relation to these and other priorities.

Recommendation 11: The focus on results-orientated performance measurement should be

supported by a greater emphasis on establishing how ESIF programmes contribute to overall

EU policy goals. The Partnership Agreements being used as a basis for implementing ERDF and ESF

programmes include wider EU policy goals. It would be unrealistic to expect the indicator system to

be able to help determine the extent to which wider EU policy goals are being promoted through

ERDF and ESF interventions. However, a link between such interventions and EU policy goals could

be more clearly established and assessed in ex-ante and ex-post evaluations. A constraint on being

able to do this in the 2021-2027 period could be the absence of a wider EU strategy and coherent set

of overall EU policy goals.

4.5 REGULATORY FRAMEWORK FOR 2021-2027

The Common Provisions Regulation for Cohesion Policy in the 2021-2027 period addresses many of the

issues examined in this Research Paper. On the following page we provide a summary.

Box 4.1: Issues Raised in the Research Paper that are addressed in the Common ProvisionsRegulation for 2021-2027

ESF+ indicators have not been significantly altered, allowing for continuity. This is in line with ourview that unless absolutely necessary and where useful information on performance can beprovided, the number of indicators used should not increase.

The ERDF common indicators have doubled in number. However, the modification is a significant

89 http://data.consilium.europa.eu/doc/document/ST-14216-2018-INIT/en/pdf


100

enhancement providing a more comprehensive set to cover the range of interventions funded gaps.The inclusion of both output and result indicators demonstrates a clear improvement on theprevious programming periods with some long-used indicators appropriately re-categorised fromoutputs to results.

Whilst programme-specific indicators are still possible, reflecting suitable flexibility inprogramming, the requirement for all indicators to form part of the performance framework couldpotentially reduce their number.

The regulatory requirement for the Performance Framework methodology to include the criteriaapplied, and data/evidence used in the selection of indicators and targets and assurance of qualityas well as any additional factors that may influence the achievements of the programme shouldimprove target-setting and ensure appropriate ambition.

A more frequent data reporting requirement will improve the timeliness of data. However, two-monthly reporting is likely to prove too frequent for Managing Authorities, and a 4 or 6 monthlysubmission could be considered.

The possibility to use national data sources to collect ESF achievements should lead to animprovement in the quality of data, a reduction in administrative burden and improve the linkageand alignment of ESF+ with national interventions.

Discontinuing the Performance Reserve is appropriate.

Box 4.2: Issues Raised in the Research Paper that are not fully addressed in the CommonProvisions Regulation for 2021-2027

Whilst the new 5+2 programming approach could enable a more strategic approach through a

mid-term review, it is likely to still be overly bureaucratic and output-focused as well as somewhat

inflexible and unproportionate. The proposals do not include the requirement for the use of impact

indicators for either fund. This goes against the Better Regulations guidance, and according to the

ECA, is equally in contradiction to the Financial Regulation.

The lack of requirement for an ex-ante evaluation has removed an important element in the

process of establishing indicators/the performance framework and ensuring their relevance,

suitability, realism and quantification, as well as strategic assessment of the policy and intervention

logic in relation to regional development needs.

Guidance and definitions related to the new indicator sets must be available well in advance of

2021-2027 OP adoption.


101

REFERENCES

EU LEVEL

ECA Critical Review of the Proposed ERDF/CF Post-2020 Approach:

https://www.eca.europa.eu/Lists/ECADocuments/OP18_06/OP18_06_EN.pdf

European Commission, DG REGIO, Common indicator guidance and data from the ESIF open

data portal at: https://cohesiondata.ec.europa.eu/countries

European Commission, DG EMPL, Programming period 2014-2020: Monitoring and Evaluation

of European Cohesion Policy and European Social Fund Guidance document’ (September 2014

and August 2018).

European Commission, DG REGIO, Working paper 3, Indicators for Monitoring and Evaluation:

An indicative methodology, available at:

https://ec.europa.eu/regional_policy/sources/docoffic/working/doc/indic_en.pdf

European Commission, DG XVI, The New Programming Period 2000-2006: methodological

working papers, Working Paper 3: Indicators for Monitoring and Evaluation, An indicative

methodology

European Commission, Internal Guidance on Ex Ante Conditionalities for the European

Structural and Investment Funds PART I.

European Commission, DG REGIO, Working Paper 4, Implementation of the performance reserve

available at:

https://ec.europa.eu/regional_policy/sources/docoffic/working/doc/reserve_en.pdf

European Commission, DG REGIO, 2016, The implementation of the performance frameworks in

2014-2020 ESI Funds, evaluation by SWECO, OIR and Spatial Foresight

EUROPEAN COMMISSION, DG REGIO, New Programming Period 2007-2013 INDICATIVE

GUIDELINES ON EVALUATION METHODS: EVALUATION DURING THE PROGRAMMING PERIOD

Working Document No.5

European Commission, ESIF 2014-2020, 2018 Summary Report of the programme annual

implementation reports covering implementation 2014-2017

European Commission, ‘Synthesis Report of ESF Annual Implementation Reports, Final report’

(Fondazione G.Brodolini, November 2017).

European Commission, Directorate-General for Employment Social Affairs and Inclusion,

Synthesis Report of ESF 2016 Annual Implementation Reports (December 2016)

European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Study

on the Monitoring and Evaluation Systems of the ESF (May 2016), prepared by Applica, WIIW and

Tarki.

European Commission, First results of the Youth Employment Initiative, pp 11-12 (Ecorys and

PPMI, June 2016).

Mid-term review/revision of the multiannual financial framework 2014-2020: An EU budget

focused on results (SWD(2016) 299 final. Communication from the Commission to the European


102

Parliament and the Council.

European Court of Auditors, Special Report 15/2017 Ex-ante conditionalities and performance

reserve in cohesion: innovative but not yet effective instruments. Available at:

http://publications.europa.eu/webpub/eca/special-reports/partnership-agreements-15-

2017/en/

European Commission, ‘Synthesis Report of ESF Annual Implementation Reports, Final report’

(Fondazione G.Brodolini, November 2017).

EPRC, The monitoring and evaluation of the 2014-2020 EU Cohesion Policy Programmes’, IQ-Net

Thematic Papers 36(2), May 2016.

EPRC, 2009, Ex post evaluation of Cohesion Policy programmes 2000-2006 co-financed by the

ERDF (Objective 1 and 2), work package 11, available at:

https://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wp11_en.htm

Mario Kölling, Swedish Institute for European Policy Studies, “Policy conditionality – a new

instrument in the EU budget post-2020?”, 2017

Peter Berkowitz, The European Union’s experiences with policy conditionalities, EC-OECD

Seminar Series, April 2017.

Proposal for a regulation of the EP and of the Council laying down common provisions on the

ERDF, ESF+, the Cohesion Fund, the EMFF, Strasbourg 29.5.2018 COM(2018)375, 2018/0196

(COD)

Metis GmbH (2016) The Implementation of the Provisions in Relation to the Ex Ante

Conditionalities during the Programming Phase of the European Structural and Investment (ESI)

Funds (July 2016).

Swedish Institute for European Policy Studies, “Policy conditionality – a new instrument in the

EU budget post-2020? 2017

T33 and SWECO, 2018, Development of a system of common indicators for ERDF and CF

interventions after 2020

Research for REGI Committee – Future links between structural reforms and EU cohesion policy,

2018

Research for REGI Committee – Conditionalities in Cohesion Policy, 2018

Research for REGI Committee -Indicators in Cohesion Policy, 2017

Research for REGI Committee - Integrated use of ESI funds to address social challenges, 2017

Research for REGI Committee - Building Blocks for a Future Cohesion Policy – First Reflections,

2018

Research for REGI Committee – Review of the Adopted Partnership Agreements, 2015

NATIONAL LEVEL

Programa Operativo FEDER de Andalucía 2014-2020" July 2015

Evaluación Exante del Programa Operativo FEDER de Andalucía 2014-2020" Regio Plus

Consulting


103

Plan Específico de Evaluación FEDER 2014-2020 de Andalucía", Dirección General de Fondos

Europeos. Junta de Andalucía. July 2016

Informe de Evaluación del Programa Operativo FEDER de Andalucía 2014-2020. 2017

Acuerdo de Asociación de España 2014-2020

Evaluación ex ante del Acuerdo de Asociación de España 2014-2020", May 2014

Informe de Evolución del Acuerdo de Asociación de España 2014-2020, August 2017

Annual Implementation Reports 2017 ESF Performance Reports and Thematic Reports Country

Summary Tables — Estonia, 2017

Summary of the Partnership Agreement for Estonia, 2014-2020, 2014

Ninvestu fir-riżorsi umani sabiex noħolqu aktar opportunitajiet filwaqt li nippromwovu t-tisħiħ

tas-soċjetà, 2018

Annual Implementation Reports 2017 ESF Performance Reports and Thematic Reports -

VC20160064 Country Summary Tables – Malta, 2017

Partnership Agreement of Malta, 2014-2020, 2014


104

ANNEX A: LIST OF INTERVIEWEES

EU Level

John Walsh DG REGIO, Evaluation Unit

Egbert Holthius DG Employment, Head of Unit, Germany

Filippa Azevedo European Parliament Regi Committee, Administrator

Kai Stryczynski DG REGIO, Head of Unit, Germany

Przemyslaw Kalinka DG REGIO, Poland

Andreas von Busch DG REGIO, Head of Unit, Czech Republic

Jeannette Monier DG Employment, Deputy Head of Evaluation Unit

Gábor Toth DG Employment, Evaluation Officer

Linda Sproge DG REGIO, Deputy Head of Unit, Estonia

Gerhard Ross ECA

Olivia Jordan DG REGIO, Head of Unit, Audit

Isabelle Maquet Deputy Head of Unit

Ana Carraro DG Employment, MS Coordinator

Constantin-Viorel Mihai DG Employment, MS Coordinator

Nevena Bisevac DG REGIO, Desk officer, Croatia & Bulgaria

Simeon Shener DG REGIO, Desk officer, Croatia & Bulgaria

Cornelia Grosser DG Employment, Deputy Head of Unit, France

Paola Bertolissi DG Employment, Desk officer, France

Mario Gerhartl DG Employment, Desk officer, France

Carole Mancel-Blanchard DG REGIO, Head of Unit, Spain

Enrico Marescotti DG REGIO, Desk officer, Spain

Member States

María Luisa Asensio Pardo Directorate General of European Funds, Ministry of Economy,

Finance and Public Administration of the Junta de Andalucía;

Spain

María Muñoz Martínez Directorate General of European Funds, Ministry of Finance.

Subdirectorate General for Programming and Evaluation; Spain


105

Yolanda Alcalá García Directorate General of European Funds, Ministry of Finance.

Subdirectorate General for Programming and Evaluation; Spain

Esther Mª Pérez Quintana Ministry of Labour, Migrations and Social Security. Subdirectorate

General for Programming and Evaluation of ESF; Spain

Kairi Nisamedtinov Ministry of Finance, Estonia

Galina Vassileva Ministry of Transport; Bulgaria

Velina Popova OP Innovation; Bulgaria

Laurent Caillaud Nouvelle Aquitaine Region; France

Christian Biral Rhone-Alpes Region; France

Claire Hallegouet Commissariat General for Territorial Equality, France

Anne-Laure Vallauri Brittany Region; France

Marta Raljević Ministry of Regional Development and EU Funds; Croatia

Nataša Filipović Ministry of Regional Development and EU Funds; Croatia

Paola Casavola Evaluation and Analysis Unit, Department of Cohesion Policy, Italy

Simona de Luca Evaluation and Analysis Unit, Department of Cohesion Policy, Italy

Jonathan Vassallo Ministry of European Affairs and Equality, Malta

Kristina Landrieux Ministry of European Affairs and Equality, Malta

Diane Camilleri Ministry of European Affairs and Equality, Malta

Marilou Micaleff Ministry of European Affairs and Equality, Malta

Oliver Schwab IfS Institut für Stadtforschung und Strukturpolitik GmbH, Berlin


106

ANNEX B: EUROPEAN SOCIAL FUND COMMON INDICATORS

OUTPUT INDICATORS FOR ESF INTERVENTIONS – PARTICIPANTS

Indicatordimension

Name of indicator Reportingfrequency

Referencepopulation

Employmentstatus

unemployed, including long-term unemployed* annually allparticipants

long-term unemployed* annually allparticipants

inactive* annually allparticipants

inactive, not in education or training* annually allparticipants

employed, including self-employed* annually allparticipants

Age below 25 years of age* annually allparticipants

above 54 years of age * annually allparticipants

above 54 years of age who are unemployed, includinglong-term unemployed, or inactive not in education ortraining*

annually allparticipants

Educationalattainment

with primary (ISCED 1) or lower secondary education(ISCED 2)*


with upper secondary (ISCED 3) or post-secondaryeducation (ISCED 4)*


with tertiary education (ISCED 5 to 8)* annually allparticipants

Disadvantagedparticipants

migrants, people with a foreign background,minorities (including marginalised communities suchas the Roma)**


participants with disabilities** annually allparticipants

other disadvantaged** annually allparticipants

homeless or affected by housing exclusion* once in2017

allparticipants

OUTPUT INDICATORS FOR ESF INTERVENTIONS – ENTITIES


Referencepopulation

number of projects fully or partially implemented by social partners ornon-governmental organisations

annually all projects

number of projects dedicated to sustainable participation andprogress of women in employment


number of projects targeting public administrations or public servicesat national, regional or local level


number of supported micro, small and medium-sized enterprises(including cooperative enterprises, enterprises of the social economy)

annually all SMEsdirectlysupported


107

INTERMEDIATE ESF RESULTS INDICATORS


Reference population

inactive participants engaged injob searching upon leaving*

annually inactive participants

participants ineducation/training uponleaving*

annually all participants, except participants who were engagedin education/training upon entering

participants gaining aqualification upon leaving*

annually all participants

participants in employment,including self-employment,upon leaving*

annually - unemployed- inactive participants

disadvantaged participantsengaged in job searching,education/ training, gaining aqualification, or in employment,including self-employment, uponleaving**

annually Disadvantaged participants who achieve a resultunderstood as a change in the situation upon leaving,compared to the situation when entering the ESFoperation.Disadvantaged participants are:- migrants, people with a foreign background,minorities (including marginalised communities such asthe Roma)**- participants with disabilities**

LONGER TERM ESF RESULTS INDICATORS


Reference population Representativeness ofsample regarding

participants in employment,including self-employment, sixmonths after leaving*

2019 and2025

unemployed inactive participants

1a) unemployed1b) inactive2) age group3) educationalattainment

participants with an improvedlabour market situation sixmonths after leaving*

2019 and2025

employed participants

1) age group2) educationalattainment

participants above 54 years ofage in employment, includingself-employment, six monthsafter leaving*

2019 and2025

unemployed inactive participants above 54 years of age

1a) unemployed1b) inactive2) educationalattainment

disadvantaged participants inemployment, including self-employment, six months afterleaving**

2019 and2025

unemployed inactive participants with any of the following

disadvantage(s) (not mutually exclusive): migrants, people with a

foreign background,minorities (includingmarginalised communitiessuch as the Roma)**

participants withdisabilities**

other disadvantaged**

1a) unemployed1b) inactive2) age group3) educationalattainment


108

ANNEX C: ERDF AND COHESION FUND COMMON INDICATORS 2014-2020

ERDF AND COHESION FUND COMMON INDICATORS 2014-2020

UNIT NAME Definition/Comments1 enterprises Number of enterprises

receiving supportNumber of enterprises receiving support in any formfrom ERDF (whether the support represents state aidor not).Enterprise: Organisation producing products orservices to satisfy market needs in order to reachprofit. The legal form of enterprise may be various(self-employed persons, partnerships, etc.).

2 Number of enterprisesreceiving grants

Number of enterprises receiving support in forms ofnon-refundable direct financial support conditionalonly to completion of project (grants).Subset of 'Number of enterprises receiving support'

3 Number of enterprisesreceiving financial supportother than grants

Number of enterprises receiving non-grant typefinancial support, in forms of loan, interest subsidy,credit guarantee, venture capital or other financialinstrument.Subset of 'Number of enterprises receiving support'

4 Number of enterprisesreceiving non-financialsupport

Number of enterprises receiving support that doesnot involve direct financial transfer (guidance,consultancy, enterprise incubators, etc.). Venturecapital is considered as financial support.Subset of 'Number of enterprises receiving support'

5 Number of new enterprisessupported

Number of enterprises created receiving financial aidor support (consultancy, guidance, etc.) from ERDFor ERDF financed facility. The created enterprise didnot exist three years before the project started butthe Managing Authority or national legislation mayset lower the time criterion. An enterprise will notbecome new if only its legal form changes.Subset of 'Number of enterprises receiving support'

6 EUR Private investment matching(grants) public support toenterprises

Total value of private contribution in supportedproject that qualifies as state aid where the form ofsupport is grant (see Common Indicator 2 'Numberof enterprises receiving grants'), including non-eligible parts of the project.

7 EUR Private investment matchingpublic support to enterprises(non-grants)

Total value of private contribution in supportedproject that qualifies as state aid where the form ofsupport is other than grant (see Common Indicator 3'Number of enterprises receiving financial supportother than grants'), including non-eligible parts ofthe project.

8 FTE Employment increase insupported enterprises

Gross new working positions in supportedenterprises in full time equivalents (FTE).

9 Visits/year Increase in expected numberof visits to supported sites ofcultural or natural heritageand attractions

The ex ante estimated increase in number of visits toa site in the year following project completion.

10 households Additional households withbroadband access of at least30 Mbps

Number of households with internet access with adownload speed of at least 30 Mb/sec and whobefore only had more limited access or did not haveaccess at all.


109

UNIT NAME Definition/Comments30 Mbps is in line with EU2020, see COM(2010)245 "Adigital agenda for Europe"

11 Km Total length of new railwayline,11a: of which: TEN-T

Length of railroads constructed by the project whereno railroad existed beforeTotal length of new railway line within TEN-T

12 Km Total length of reconstructedor upgraded railway line,12a: of which: TEN-T

Length of railroads of which quality or capacity havebeen improved.Total length of reconstructed or upgraded railwayline within TEN-T

13 Km Total length of newly builtroads,

13a: of which: TEN-T

Length of roads (in kilometres) constructed by theproject where no road existed before or the capacityand quality of the previously existinglocal/secondary road is significantly improved toreach a higher classificationTotal length of newly built roads within TEN-T

14 Km Total length of reconstructedor upgraded roads,

14a: of which: TEN-T

Length of roads where the capacity or quality of theroad (including safety standards) was improved.Total length of reconstructed or upgraded roadswithin TEN-T

15 Km Total length of new orimproved tram and metrolines

Length of metro, tram or suburban train linesconstructed or upgraded.

16 Km Total length of new orimproved inland waterway

Length of inland waterway with new or improvednavigation capacity.

17 tonnes/year Additional waste recyclingcapacity

Annual capacity of waste recycling facilities.

18 persons Additional population servedby improved water supply

Number of persons provided with drinking waterthrough drinking water supply network as aconsequence of increased drinking waterproduction/transportation capacity built by theproject, and who were previously not connected orwere served by sub-standard water supply.

19 populationequivalent

Additional population servedby improved wastewatertreatment

Number of persons whose wastewater is transportedto wastewater treatment plants through wastewatertransportation network as a result of increased wastewater treatment/transportation capacity built by theproject, and who were previously not connected orwere served by sub-standard wastewater treatment.

20 persons Population benefiting fromflood protection measures

Number of people exposed to flood risk wherevulnerability decreased as a direct consequence of asupported project.

21 persons Population benefiting fromforest fire protection measures

Number of people exposed to forest fire hazardswhere vulnerability decreased as a directconsequence of a supported project.

22 Hectares Total surface area ofrehabilitated land

Surface of remediated or regenerated contaminatedor derelict land made available for economic (exceptnon-eligible, e.g. agriculture or forestry) orcommunity activities.

23 Hectares Surface area of habitatssupported in order to attain abetter conservation status

Surface of restored or created areas aimed toimprove the conservation status of threatenedspecies.

24 FTE Number of new researchers insupported entities

Gross new working positions (that did not existbefore) to directly perform R&D activities, in full timeequivalents.


110

UNIT NAME Definition/Comments25 FTE Number of researchers

working in improved researchinfrastructure facilities

Existing working positions in research infrastructurefacilities that (1) directly perform R&D activities and(2) are directly affected by the project.

26 enterprises Number of enterprisescooperating with researchinstitutions

Number of enterprises that cooperate with researchinstitutions in R&D projects.

27 EUR Private investment matchingpublic support in innovationor R&D projects

Total value of private contribution in supportedinnovation or R&D projects, including non-eligibleparts of the project.

28 enterprises Number of enterprisessupported to introduce new tothe market products

The indicator measures if an enterprise receivessupport to develop a 'new to the market' product inany of its markets.

29 enterprises Number of enterprisessupported to introduce new tothe firm products

The indicator measure if an enterprise is supportedto develop a 'new to the firm' product.

30 MW Additional capacity ofrenewable energy production

Increase in energy production capacity of facilitiesusing renewable energy resources, built/equippedby the project. Includes electricity and heat energy.

31 households Number of households withimproved energyconsumption classification

Number of households in improved energy class –see Directive 2010/31/EU. Improved class must bethe direct consequence of the project completion.

32 kWh/year Decrease of annual primaryenergy consumption of publicbuildings

Calculations are based on the energy certificate ofbuildings (see Art.12.1.b of Directive 2010/31/EU.Value will be calculated from the energy certificatesissued before and after the reconstruction. Theindicator will show the total decrease of annualconsumption, not the total saved consumption.

33 Users Number of additional energyusers connected to smart grids

Smart grid: Electricity network that integrate theactions of energy users by exchanging digitalinformation with the network operator or supplier.

34 tons of CO2equivalent

Estimated annual decrease ofGHG

This indicator is calculated for interventions directlyaiming to increase renewable energy production(see indicator 30) or to decrease energyconsumption through energy saving measures (seeindicators 31 and 32), thus its use is mandatory onlywhere these indicators are relevant.

35 persons Capacity of supportedchildcare or educationinfrastructure

Number of users who can use newly built orimproved childcare or education facilities.

36 persons Population covered byimproved health services

Population of a certain area expected to benefit fromthe health services supported by the project.

37 persons Population living in areas withintegrated urbandevelopment strategies

Population living in areas with integrated urbandevelopment strategies within the meaning ofArticle 7 of Regulation 1301/2013 (ERDF).

38 Squaremeters

Open space created orrehabilitated in urban areas

Size of renovated / newly developed publiclyaccessible open-air areas.

39 Squaremeters

Public or commercialbuildings newly built orrenovated in urban areas

Size of renovated/newly developed public andcommercial areas

40 Housingunits

Rehabilitated housing inurbanareas

Number of renovated/newly developed housingunits in residential areas, as part of urbanrehabilitation.


111

ANNEX D: ANALYSIS OF THE USE OF COMMON INDICATORS 2014-2020ESF – ALL MS

AT BE BG CY CZ DE DK EE ES FI FR GR HR HU IE IT LT LU LV MT NL PL PT RO SE SI SK UKGrandTotal

CO01-03-05 22 42 29 3 41 254 15 17 437 22 192 158 19 36 15 392 25 12 12 15 6 462 61 4 11 15 22 51 2390

CO02 7 15 7 3 10 77 5 4 120 7 67 38 7 9 5 90 6 4 3 3 2 133 28 1 4 4 4 13 676

CO04 6 11 5 12 79 2 6 96 7 55 38 4 13 5 65 7 3 3 3 2 125 14 1 4 3 4 15 588

CO06-06a-07 23 52 40 8 45 263 14 18 465 23 213 126 20 40 14 377 27 11 21 16 6 477 95 5 10 19 28 57 2513

CO08 6 12 6 1 8 62 4 4 120 6 58 30 5 7 4 67 7 2 3 3 2 121 19 3 3 4 9 576CO09-10-11-11a 28 64 42 8 60 300 15 22 577 29 291 164 21 46 20 456 33 12 22 17 5 634

109 3 13 23 35 86 3135

CO12 8 16 10 2 12 87 6 132 7 71 40 6 12 4 80 7 4 4 4 2 162 35 1 5 6 16 739

CO13 7 13 9 2 11 83 6 122 7 71 40 4 12 4 66 7 4 4 3 2 160 34 1 5 5 16 698

CO14 7 17 15 2 15 89 6 127 7 76 46 6 15 5 78 8 4 6 5 2 175 34 1 7 7 20 780

CO15 7 19 15 2 11 92 5 6 147 8 77 54 6 14 5 112 9 4 5 5 2 152 4 6 8 16 791

CO16 7 19 15 3 10 90 1 6 145 7 74 41 6 12 5 82 9 4 5 6 2 174 2 1 4 6 10 21 767

CO17 8 14 10 3 13 69 5 6 136 7 73 42 6 10 5 102 9 4 5 4 1 174 22 1 4 5 8 21 767

CO18 4 1 11 8 67 4 3 83 5 65 18 3 12 5 30 4 3 4 2 1 145 4 2 5 16 505

CO19 5 13 15 3 13 78 5 6 118 8 65 21 6 15 4 91 7 4 6 2 176 34 2 4 7 9 14 731

CO20 8 25 4 1 14 54 5 6 21 7 27 14 4 10 48 1 3 3 6 115 10 5 5 3 1 4 404

CO21 5 8 10 50 6 18 7 27 17 1 4 22 2 80 2 5 1 2 267

CO22 17 4 2 15 3 4 6 15 5 25 27 2 13 111 8 7 50 1 2 5 2 6 330

CO23 4 17 2 9 27 1 2 16 5 21 15 1 5 52 3 2 1 40 3 1 4 7 238

CR01 6 14 2 3 74 4 3 86 7 52 30 2 9 5 37 2 2 4 2 77 3 3 12 439

CR02 7 18 12 2 8 77 4 6 102 7 65 30 2 10 5 47 4 3 6 1 88 20 3 2 3 16 548

CR03 7 15 4 11 84 3 5 134 7 73 24 3 14 5 58 6 2 5 2 106 7 1 4 4 1 22 607

CR04 7 16 6 3 8 74 4 6 145 7 69 34 5 12 5 66 3 3 4 2 108 23 4 4 5 14 637

CR05 7 15 11 2 10 76 4 6 117 8 70 33 2 12 5 51 6 3 6 2 129 4 3 4 4 17 607

CR06 10 7 30 6 52 2 24 28 3 2 4 103 2 2 17 4 3 8 307

CR07 2 1 4 27 6 33 1 12 8 2 2 33 2 16 1 1 151

CR08 7 4 16 4 29 14 1 1 3 19 1 2 2 1 3 107

CR09 8 5 28 4 32 16 2 2 3 41 1 2 3 1 5 153

Grand Total 196 480 273 52 377 2310 104 182 3625 213 1943 1116 149 347 137 2776 191 81 131 125 54 4096 558 31 108 139 175 482 20451


112

ERDF – ALL MS

AT BE BG CY CZ DE DK EE ES FI FR GR HR HU IE IT LT LU LV MT NL PL PT RO SE SI SK UK GrandTotal

CO01 4 12 5 5 14 56 4 3 71 8 92 71 5 18 5 133 5 4 6 9 92 45 7 41 3 13 26 757

CO02 4 5 4 4 12 29 4 1 59 3 36 58 2 10 2 92 5 3 4 9 62 33 3 7 3 9 14 477

CO03 1 2 2 1 4 21 1 22 5 35 16 1 8 55 4 1 3 4 30 18 3 10 2 7 12 268

CO04 9 1 1 1 9 2 28 5 47 15 2 6 4 15 4 1 2 6 21 2 27 2 6 14 230

CO05 4 1 3 16 1 33 2 21 21 2 4 2 48 2 1 9 8 1 9 1 8 17 214

CO06 3 1 2 1 11 10 3 8 3 30 5 1 6 2 26 3 1 9 52 25 2 7 1 3 7 222

CO07 2 3 14 3 3 32 1 7 11 4 2 4 26 2 10 1 3 12 140

CO08 2 4 1 2 4 23 2 38 5 23 45 2 3 2 50 2 2 2 1 29 35 24 1 8 16 326

CO09 1 1 1 5 11 10 15 1 4 17 1 1 2 16 7 2 95

CO10 1 12 11 2 1 1 2 9 1 1 1 1 1 1 3 48

CO11 5 1 2 1 9

CO11a 1 1 2

CO12 1 2 3 1 5 16 2 2 1 2 35

CO12a 1 3 1 1 1 1 1 9

CO13 2 3 1 12 1 2 1 17 3 1 3 46

CO13a 4 1 1 1 7

CO14 1 1 1 3 1 16 3 5 2 1 20 2 2 2 3 63

CO14a 1 4 1 1 1 8

CO15 1 2 9 2 2 1 17

CO16 1 1 1 3

CO17 1 1 3 2 10 5 14 36

CO18 4 5 13 5 11 1 2 41

CO19 10 5 8 5 15 1 1 45

CO20 5 2 9 10 1 10 1 13 1 1 53

CO21 1 7 2 5 15

CO22 2 1 1 8 9 6 2 4 7 1 1 1 1 1 45

CO23 1 1 1 3 9 18 13 1 8 15 1 1 1 2 75

CO24 3 2 1 1 2 14 11 15 7 1 4 2 13 1 3 2 1 4 2 89

CO25 2 1 1 16 1 16 18 10 1 2 10 1 1 1 1 18 8 1 1 1 4 3 118


113

AT BE BG CY CZ DE DK EE ES FI FR GR HR HU IE IT LT LU LV MT NL PL PT RO SE SI SK UK GrandTotal

CO26 2 3 2 1 3 17 1 1 15 33 14 2 2 2 29 1 1 1 3 20 8 1 10 1 3 8 184

CO27 1 1 1 1 14 1 18 4 30 4 2 2 1 18 1 1 6 16 2 5 4 3 136

CO28 3 1 1 2 11 1 1 13 3 12 5 2 2 22 1 1 6 17 7 1 7 1 7 6 133

CO29 3 1 2 12 1 1 20 3 14 7 2 2 31 1 1 6 16 8 2 7 16 156

CO30 3 1 8 20 25 6 7 22 1 1 1 3 2 23 1 1 3 4 132

CO31 1 2 1 1 10 28 11 1 2 2 3 1 1 1 1 15 5 1 2 1 4 94

CO32 3 2 1 7 13 15 15 1 2 26 1 1 1 1 17 7 1 1 5 2 122

CO33 2 2 2 3 1 8 1 1 20

CO34 3 7 4 1 4 32 3 29 64 24 7 2 48 1 2 1 4 58 8 3 3 9 12 329

CO35 3 2 3 10 5 18 1 1 12 1 1 22 7 2 2 90

CO36 1 5 2 14 4 6 1 1 1 16 7 1 1 60

CO37 2 1 11 3 1 24 5 5 2 4 1 1 7 1 1 1 70

CO38 2 1 1 10 2 4 8 11 1 6 1 1 1 5 17 4 1 1 77

CO39 7 2 1 6 2 3 9 3 6 8 1 1 1 1 4 11 2 1 3 72

CO40 1 6 2 3 8 2 3 7 1 33GrandTotal 24 78 45 24 79 363 17 23 535 44 693 514 35 135 30 788 48 8 33 33 67 734 276 60 172 26 125 192 5201


114

ANNEX E: COMMON OUTPUT AND RESULT INDICATORS FOR THE ERDF AND THE CF 2021-2027

COMMON OUTPUT AND RESULT INDICATORS FOR ERDF (NON-ETC)

Policy objective Outputs Results

(1) (2) (3)

1. A smarter Europe by

Promoting innovative

and smart economic

transformation

RCO90 01 - Enterprises supported (of which: micro, small, medium,

large)*

RCO 02 - Enterprises supported by grants*

RCO 03 - Enterprises supported by financial instruments*

RCO 04 - Enterprises with non-financial support*

RCO 05 - Start-ups supported*

RCO 06 - Researchers working in supported research facilities

RCO 07 - Research institutions participating in joint research projects

RCO 08 - Nominal value of research and innovation equipment

RCO 10 - Enterprises cooperating with research institutions

RCO 96 – Interregional investments in EU projects*

RCR91 01 - Jobs created in supported entities*

RCR 02 - Private investments matching public support (of which:

grants, financial instruments)*

RCR 03 – SMEs introducing product or process innovation*

RCR 04 - SMEs introducing marketing or organisational innovation*

RCR 05 - SMEs innovating in-house*

RCR 06 - Patent applications submitted to European Patent Office*

RCR 07 - Trademark and design applications*

RCR 08 - Public-private co-publications

RCO 12 - Enterprises supported to digitise their products and

services

RCO 13 - Digital services and products developed for enterprises

RCO 14 - Public institutions supported to develop digital services

and applications

RCR 11 - Users of new public digital services and applications*

RCR 12 - Users of new digital products, services and applications

developed by enterprises*

RCR 13 - Enterprises reaching high digital intensity*

RCR 14 - Enterprises using public digital services*

RCO 15 - Capacity of incubation created* RCR 16 - High growth enterprises supported*

RCR 17 - 3-year-old enterprises surviving in the market*

RCR 18 - SMEs using incubator services one year after the incubator

creation

90 RCO: Regional Policy Common Output Indicator.91 RCR: Regional Policy Common Result Indicator.


115


(1) (2) (3)

RCR 19 - Enterprises with higher turnover

RCR 25 - Value added per employee in supported SMEs*

RCO 16 - Stakeholders participating in entrepreneurial discovery

process

RCO 17 - Investments in regional/ local ecosystems for skills

development

RCO 101 – SMEs investing in skills development

RCO 102 - SMEs investing in training management systems*

RCR 24 - SMEs benefiting from activities for skills development

delivered by a local/regional ecosystem

RCR 97 – Apprenticeships supported in SMEs

RCR 98 – SMEs staff completing Continuing Vocational Education and

Training (CVET) (by type of skill: technical, management,

entrepreneurship, green, other)

RCR 99 – SMEs staff completing alternative training for knowledge

intensive service activities (KISA) (by type of skills: technical,

management, entrepreneurship, green,

other)

RCR 100 – SMEs staff completing formal training for skills

development (KISA) (by type of skills: technical, management,

entrepreneurship, green, other)*

2. A greener, low-carbon

Europe by promoting

clean and fair energy

transition, green and blue

investment, the circular

economy, climate

adaptation and

risk prevention and

management

RCO 18 - Households supported to improve energy performance of

their dwelling

RCO 19 - Public buildings supported to improve energy

performance

RCO 20 - District heating network lines newly constructed or

improved

RCR 26 - Annual final energy consumption (of which: residential,

private non-residential, public non-residential)

RCR 27 – Households with improved energy performance of their

dwellings

RCR 28 – Buildings with improved energy classification (of which:

residential, private non-residential, public non-residential)

RCR 29 – Estimated greenhouse gas emissions*

RCR 30 - Enterprises with improved energy performance

RCO 22 - Additional production capacity for renewable energy (of

which: electricity, thermal)

RCR 31 - Total renewable energy produced (of which: electricity,

thermal)


116


(1) (2) (3)

RCO 97 – Number of energy communities and renewable energy

communities supported*

RCR 32 – Renewable energy: Capacity connected to the grid

(operational)*

RCO 23 - Digital management systems for smart grids

RCO 98 – Households supported to use smart energy grids

RCO 24 - New or upgraded disaster monitoring, preparedness,

warning and response systems*

RCO 25 - Coastal strip, river banks and lakeshores, and landslide

protection newly built or consolidated to protect people, assets and

the natural environment

RCO 26 - Green infrastructure built for adaptation to climate change

RCO 27 - National/ regional/ local strategies addressing climate

change adaptation

RCO 28 - Areas covered by protection measures against forest fires

RCR 33 - Users connected to smart grids

RCR 34 - Roll-out of projects for smart grids

RCR 35 - Population benefiting from flood protection measures

RCR 36 - Population benefiting from forest fires protection measures

RCR 37 - Population benefiting from protection measures against

climate related natural disasters (other than floods and forest fires)

RCR 96 – Population benefiting from protection measures against

non-climate related natural risks and risks related to human activities*

RCR 38 - Estimated average response time to disaster situations*

RCO 30 - Length of new or consolidated pipes for household water

connections

RCO 31 - Length of sewage collection networks newly constructed

or consolidated

RCO 32 - New or upgraded capacity for waste water treatment

RCR 41 - Population connected to improved water supply

RCR 42 - Population connected to at least secondary waste water

treatment

RCR 43 - Water losses

RCR 44 - Waste water properly treated

RCO 34 - Additional capacity for waste recycling RCR 46 - Population served by waste recycling facilities and small

waste management systems

RCR 47 - Waste recycled

RCR 48 - Recycled waste used as raw materials

RCR 49 - Waste recovered

RCO 36 - Surface area of green infrastructure supported in urban

areas

RCR 50 - Population benefiting from measures for air quality

RCR 95 -Population having access to new or upgraded green

infrastructure in urban areas


117


(1) (2) (3)

RCO 37 - Surface of Natura 2000 sites covered by protection and

restoration measures in accordance with the prioritised action

framework

RCO 99 - Surface area outside Natura 2000 sites covered by

protection and restoration measures

RCO 38 - Surface area of rehabilitated land supported

RCO 39 - Systems for monitoring air pollution installed

RCR 51 - Population benefiting from measures for noise reduction

RCR 52 - Rehabilitated land used for green areas, social housing,

economic or community activities

3. A more connected

Europe by enhancing

mobility and regional ICT

connectivity

RCO 41 - Additional households with broadband access of very high

capacity

RCO 42 - Additional enterprises with broadband access of very high

capacity

RCR 53 - Households with broadband subscriptions to a very high

capacity network

RCR 54 - Enterprises with broadband subscriptions to a very high

capacity network

RCO 43 - Length of new roads supported - TEN-T

RCO 44 - Length of new roads supported - other

RCO 45 - Length of roads reconstructed or upgraded - TEN-T

RCO 46 - Length of roads reconstructed or upgraded - other

RCR 55 - Users of newly built, reconstructed or upgraded roads

RCR 56 - Time savings due to improved road infrastructure

RCR 101 – Time savings due to improved rail infrastructure

RCO 47 - Length of new rail supported - TEN-T

RCO 48 - Length of new rail supported - other

RCO 49 - Length of rail reconstructed or upgraded - TEN-T

RCO 50 - Length of rail reconstructed or upgraded - other

RCO 51 - Length of new or upgraded inland waterways - TEN-T

RCO 52 - Length of new or upgraded inland waterways - other

RCO 53 - Railways stations and facilities - new or upgraded

RCO 54 - Intermodal connections - new or upgraded

RCO 100 – Number of ports supported

RCR 57 - Length of European Rail Traffic Management System

equipped railways in operation

RCR 58 - Annual number of passengers on supported railways

RCR 59 - Freight transport on rail

RCR 60 - Freight transport on inland waterways

RCO 55 - Length of tram and metro lines- new RCR 62 - Annual passengers of public transport


118


(1) (2) (3)

RCO 56 - Length of tram and metro lines- reconstructed/ upgraded

RCO 57 - Environmentally friendly rolling stock for public transport

RCO 58 - Dedicated cycling infrastructure supported

RCO 59 - Alternative fuels infrastructure (refuelling/ recharging

points) supported

RCO 60 - Cities and towns with new or upgraded digitised urban

transport systems

RCR 63 - Annual users of new/ upgraded tram and metro lines

RCR 64 - Annual users of dedicated cycling infrastructure

RCO 41 - Additional households with broadband access of very high

capacity

RCO 42 - Additional enterprises with broadband access of very high

capacity

RCR 53 - Households with broadband subscriptions to a very high

capacity network

RCR 54 - Enterprises with broadband subscriptions to a very high

capacity network

RCO 43 - Length of new roads supported - TEN-T92

RCO 44 - Length of new roads supported - other

RCO 45 - Length of roads reconstructed or upgraded - TEN-T

RCO 46 - Length of roads reconstructed or upgraded - other

RCR 55 - Users of newly built, reconstructed or upgraded roads

RCR 56 - Time savings due to improved road infrastructure

RCR 101 – Time savings due to improved rail infrastructure

RCO 47 - Length of new rail supported - TEN-T

RCO 48 - Length of new rail supported - other

RCO 49 - Length of rail reconstructed or upgraded - TEN-T

RCO 50 - Length of rail reconstructed or upgraded - other

RCO 51 - Length of new or upgraded inland waterways - TEN-T

RCO 52 - Length of new or upgraded inland waterways - other

RCO 53 - Railways stations and facilities - new or upgraded

RCO 54 - Intermodal connections - new or upgraded

RCR 57 - Length of European Rail Traffic Management System

equipped railways in operation

RCR 58 - Annual number of passengers on supported railways

RCR 59 - Freight transport on rail

RCR 60 - Freight transport on inland waterways

92 Regulation (EU) No 1315/2013 of the European Parliament and of the Council of 11 December 2013 on Union guidelines for the development of the trans-European transport network andrepealing Decision No 661/2010/EU (OJ L 348, 20.12.2013, p. 1).


119


(1) (2) (3)

RCO 100 – Number of ports supported

RCO 55 - Length of tram and metro lines- new

RCO 56 - Length of tram and metro lines- reconstructed/ upgraded

RCO 57 - Environmentally friendly rolling stock for public transport

RCO 58 - Dedicated cycling infrastructure supported

RCO 59 - Alternative fuels infrastructure (refuelling/ recharging

points) supported

RCO 60 - Cities and towns with new or upgraded digitised urban

transport systems

RCR 62 - Annual passengers of public transport

RCR 63 - Annual users of new/ upgraded tram and metro lines

RCR 64 - Annual users of dedicated cycling infrastructure

4. A more social Europe

Implementing the

European Pillar of Social

Rights

RCO 61 - Annual unemployed persons served by enhanced facilities

for employment services (capacity)

RCR 65 - Job seekers using annually the services of the employment

services supported

RCO 63 - Capacity of temporary reception infrastructure created

RCO 64 - Capacity of rehabilitated housing – migrants, refugees and

persons under or applying for international protection

RCO 65 - Capacity of rehabilitated housing - other

RCR 66 - Occupancy of temporary reception infrastructure built or

renovated

RCR 67 - Occupancy of rehabilitated housing – migrants, refugees and

persons under or applying for international protection

RCR 68 - Occupancy of rehabilitated housing - other

RCO 66 - Classroom capacity of supported childcare infrastructure

(new or upgraded)

RCO 67 - Classroom capacity of supported education infrastructure

(new or upgraded)

RCR 70 - Annual number of children using childcare infrastructure

supported

RCR 71 - Annual number of students using education infrastructure

supported

RCO 69 - Capacity of supported health care infrastructure

RCO 70 - Capacity of supported social infrastructure (other than

housing)

RCR 72 - People with access to improved health care services

RCR 73 - Annual number of persons using the health care facilities

supported

RCR 74 - Annual number of persons using the social care facilities

supported


120


(1) (2) (3)

RCR 75 - Average response time for medical emergencies in the area

supported

5. A Europe closer to

citizens by fostering the

sustainable and

integrated development

of urban, rural and

coastal areas and local

initiatives

RCO 74 - Population covered by strategies for integrated urban

development

RCO 75 - Integrated strategies for urban development

RCO 76 - Collaborative projects

RCO 77 - Capacity of cultural and tourism infrastructure supported

RCR 76 - Stakeholders involved in the preparation and

implementation of strategies of urban development

RCR 77 - Tourists/ visits to supported sites*

RCR 78 - Users benefiting from cultural infrastructure supported

RCO 80 – Community-led local development strategies for local

development

Horizontal -

Implementation

RCO 95 - Staff financed by ERDF and Cohesion Fund RCR 91 - Average time for launch of calls, selection of projects and

signature of contracts*

RCR 92 - Average time for tendering (from launch of procurement

until signature of contract) *

RCR 93 - Average time for project implementation (from signature of

contract to last payment)*

RCR 94 - Single bidding for ERDF and Cohesion Fund interventions*

** For presentational reasons, indicators are grouped under, but not limited to, a policy objective. In particular, under policy objective 5, specific objectives from policy objectives 1-4may be used with the relevant indicators. In addition, in order to develop a full picture of the expected and actual performance of the programmes, the indicators marked with (*)may be used by specific objectives under more than one of the policy objectives 1 to 4, when relevant.

This study examines how reliable Cohesion Policy indicators are in helping to ensure targeted and result-oriented expenditure. Overall, substantial progress has been made in recent years in developing an effective system. However, the challenge remains to shift from a focus on financial absorption and outputs to assessing more meaningful results and impacts of interventions, and to improve data collection systems. The paper includes eleven recommendations to address these and other challenges ahead of the new 2020-2027 period.

DISCLAIMER

This document is prepared for, and addressed to, the Members and staff of the European

Parliament as background material to assist them in their parliamentary work. The content of the

document is the sole responsibility of its author(s) and any opinions expressed herein should not

Print ISBN 978-92-846-4726-2| doi: 10.2861/66132| QA-03-19-243-EN-C PDF ISBN 978-92-846-4700-2 | doi:10.2861/847200 | QA-03-19-243-EN-N