The Commission's

Impact Assessment system

21 January 2015

Michał Narożny

Impact Assessment unit

Secretariat General 1

Structure of the presentation

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. How do we carry out an IA?

4. How can stakeholders contribute?

5. How do we control the quality of IAs?

6. Latest developments

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. How do we carry out an IA?

4. How can stakeholders contribute?

5. How do we control the quality of IAs?

6. Latest developments

EU legislation is of high quality if it …

• Respects the principles of subsidiarity and proportionality

• Results in a simple, clear, stable and predictable regulatory framework

• Delivers most effectively on EU policy objectives at minimum costs

Smart Regulation is about ensuring high quality EU legislation

See Commission Communications on EU Regulatory Fitness and Smart Regulation

Smart Regulation concerns the whole policy cycle

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. How do we carry out an IA?

4. How can stakeholders contribute?

5. How do we control the quality of IAs?

6. Latest developments

• A set of logical steps to help Commission services structure preparation of a proposal

• Provide a balanced evidence base to support, not replace, political decision-making

• Integrated approach: all benefits and costs; economic, social and environmental impacts

• Independent centralised quality control: the IA Board

• Transparency: consultations, publication of IAs and IAB opinions

• An IA needs to be carried out for all initiatives expected to have significant direct impacts. These can be:

• Legislative proposals

• Non-legislative proposals (eg policy defining white papers, action plans)

• Implementing measures and delegated acts

• Identification by Commission SG, after consulting services

8

IA planning process

Roadmap Consultation

request

Consultation:

report on findings Proposal,

Impact

Assessment

reports, Board

Opinion

Initiate stage: the process inside the Commission

Initiate stage: outputs seen by outside stakeholders

IA process: consult,

get data,

analyse, draft report

Idea

Screening:

proceed?

is IA needed?

Impact Assessment Board

Revise IA

and proposal

Internal inter-

service consultation

College

(political approval)

Publish

Legislate stage

Proposal development

9

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. How do we carry out an IA?

4. How can stakeholders contribute?

5. How do we control the quality of IAs?

6. Latest developments

What are the key analytical steps of IA?

1. Identify the problem

2. Assess need for EU-level intervention

3. Define the objectives

4. Develop policy options

5. Analyse the impacts of the options

6. Compare the options

7. Outline policy monitoring and evaluation

The Key Steps – 1. Problem Definition (A)

• What should the problem statement do?

• Present the scope/size of the problem to different actors

• Identify the risks

• Identify who is affected and how

• Start off the ‘thread’ of the analysis

• Show the links and drivers, the underlying pressures

12

The Key Steps – Problem Definition (B)

Possible causes (and reasons for public intervention):

Market failures –market by itself cannot achieve the

desired outcomes : • Weakness of competition/market power/barriers to entry

- Is it hard for new firms to compete?

• Information asymmetry - Does the buyer lack crucial information which the seller has?

• Public goods - Is there no way to exclude people from consumption of certain goods or services?

• Externalities - Does an action impose a cost or deliver a benefit to others?

13

The Key Steps – Problem Definition (C)

Possible causes (and reasons for public intervention):

Regulatory failures – existing regulation does not lead to desired outcomes :

• Inadequately defined property rights

• Poorly defined targets and objectives

• Unintended consequences

• “Regulatory capture” following rent-seeking by interest groups

• Implementation and enforcement failures

14

The Key Steps – Problem Definition (D)

Possible causes (and reasons for public intervention):

Discrepancy between the fundamental goals of the

Union and existing situation – e.g.:

o fundamental rights,

o discrimination,

o security of citizens

15

The Key Steps – 1. Problem Definition (F)

• Baseline scenario

• What can be expected to happen if current trends and policies continue?

• Are developments subject to significant uncertainty (risk analysis)?

• Who is likely to be affected?

• What actions have been taken and/or are planned by EU, Member States, others?

Strong factual basis and expressed in quantitative terms

17

• Example

• Source: Reference scenario used for the 2030 Climate Energy Framework

The Key Steps – 1. Problem Definition (G)

18

The Key Steps – 2. Need for EU action (A)

• Justify why EU action is required - is the EU level the most appropriate level for action?

• Subsidiarity analysis:

Why can the objectives of the proposed action not be

achieved sufficiently by Member States? (Necessity Test)

Can objectives be better achieved by action at EU level? (test of EU Value Added)

19

The Key Steps – 2. Need for EU action (B)

• Subsidiarity analysis:

Is the problem you aim to address linked to at least one

article of the Treaties and the objectives contained therein?

Does the legal basis (action under consideration) fall within one of the areas where the Treaty gives the Union exclusive competence (as defined by Art.3 of the Treaty on the Functioning of the European Union)?

Can the objectives of the proposed action be achieved sufficiently by Member States acting alone?

Can the objectives of the proposed action be better achieved at Union level?

20

The Key Steps – 3. Objectives (A)

• Linked to the problem and give direction to options

• Consistent with other existing policy objectives

• Overall goals of policy, immediate objectives and deliverables

• Specific, measurable and realistic

• Criteria for assessing success/failure of proposed policy options

• Come back to them later in the analysis!

21

The Key Steps – 3. Objectives (B)

• General objective (-> impact indicators) • An objective in a Treaty

• E.g. to improve the quality of environmental decision-making

• Specific objective (-> result indicators) • Take into account specific nature of the policy domain

• E.g. to increase the availability of quality geographical information, to reduce inefficiencies with the collection, handling, storing and distribution of geographical information

• Operational objectives (-> output indicators) • Indicators are defined subsequently for the preferred option, in some

cases delegated to working groups.

• In some cases explicit targets mentioned in the IA, e.g. REACH - to have 100% testing of certain chemicals classes before a certain date

22

The Key Steps – 4. Policy Options (A)

• An option is a possible way forward in the policy field at hand

• Options have to be on the content level, i.e. not only the question regulation vs. directive

• Options should take into account positions from stakeholders and experts

23

The Key Steps – 4. Policy Options (B)

• Examine wide range of possible options Relevant questions for “good” options

• What is the scope?

• Who has to comply?

• What is the level of ambition?

• What is the timescale?

• Who decides what is to be done (subsidiarity)?

• Always include option ‘No EU action’ or ‘No policy change’

• Recommended to consider alternatives to ‘classic’ regulation

• Look at possibility of doing less – streamlining, simplifying, pruning existing legislation

24

The Key Steps – 4. Policy Options (C)

• Micro-entities to be excluded from the scope of proposed legislation unless this impairs achievement of objectives [COM(2011)803]

• When included, IA should demonstrate the need for this and assess possible adapted solutions

=> Ensure you collect the necessary evidence and stakeholders views during the IA process

25

The Key Steps – 4. Policy Options (D)

• Not possible to analyse all options • Some “easy” to discard • Iterative process • Document all your steps

Screen possible options to see which can best meet the objectives

Measure against:

– Effectiveness: best placed to achieve the objectives

– Efficiency: cost-effectiveness

– Coherence: limiting trade-offs across the economic, social and environmental domains

Draw-up a ‘short-list’ of options (3-5) for further analysis

26

The Key Steps – 4. Policy Options (E)

• Generating policy options

• First, consider your objective What do you want to achieve?

Improve supervision of cross-border banks

• Think about content How might you do this?

1. Select a lead national supervisor

2. Create a European supervisory agency working with groups of national supervisors

3. Create a single European supervisor

• Think about the policy instrument What different ways could you introduce this to MS?

Communication, Recommendation, Voluntary Agreement, Directive, Regulation, Implementing legislation 27

The Key Steps – 4. Policy Options (F)

• Where can it go wrong?

• Too many do nothing or do everything or something outrageous

• Trying to solve everything with legislation

• Limited detail and ‘refinement’

• Policy options not linked to Problem Statement or Objectives

28

The Key Steps – 5. Analysing Impacts (A)

• Examine for all short-listed options

• Use tables listed in Guidelines to identify direct/indirect impacts across social, environmental, economic dimensions (including impacts on Fundamental Rights)

• Use available operational guidance for specific impacts (social, fundamental rights and competitiveness)

• Identify who is affected (including outside the EU)

• Consider compliance and implementation issues

30

Using Impact Tables in Guidelines

• http://ec.europa.eu/smart-regulation/impact/commission_guidelines/docs/iag_2009_en.pdf

• (p. 33-38)

31

The Key Steps – 5. Analysing Impacts (B)

• Assess impacts in qualitative, quantitative, and monetary terms where possible

• See what tools are available to undertake quantitative analysis

• Results need to be transparent, reproducible, robust

• Before you start quantifying/modelling – make sure you understand the process and the causality

• Back analysis with examples/stakeholders' views

32

The Key Steps – 5. Analysing Impacts (C)

What about quantification of impacts?

In principle, attempt to quantify costs and benefits where possible (subject to principle of proportionate analysis):

• Based on/using good historical figures, where available (EUROSTAT, industry organisations, etc.)

• Avoid exclusive reliance on stakeholder-provided data, double-check against other sources

• If extrapolation, make sure that well explained for the baseline, incl. assumptions etc., and provide sensitivity analysis

• Make sure that assumptions and model choice are transparent for non-expert readers

• Cross-checking results with alternative model enhance credibility 33

The Key Steps – 6. Comparing the Options (A)

Two (complementing) approaches to comparing options in Guidelines:

• Choose option with the largest quantified net benefit (full Cost Benefit Approach)

• Rate and rank options qualitatively on key criteria, to show trade-offs and inform choices. These criteria are a recommended starting point: Effectiveness: will the policy achieve its objective?

(benefits)

Efficiency: are results achieved at least cost? (benefit cost ratio or, if unknown, cost)

Coherence with other policies 34

The Key Steps – 6. Comparing the Options (B)

Assessment of policy options

Effectiveness Efficiency Coherence

Option 1: no policy

change

0 0 0

Option 2:

recommendation on

transparency

+/? + +

Option 3: binding

rules on transparency

+ ++ ++

Option 4: detailed

regulatory framework

++ - -

Magnitude of impact as compared with the baseline scenario (the baseline is indicated as 0):

++ strongly positive; + positive; – – strongly negative; – negative; ≈ marginal/neutral; ?

uncertain; n.a. not applicable

35

The Key Steps – 6. Comparing the Options (C)

Assessment of policy options by stakeholders group

Companies Investors Proxy

advisors

Regulators

Option 1: no policy

change

0 0 0 0

Option 2:

recommendation on

transparency

+ + +/- +

Option 3: binding

rules on transparency

++ ++ +/- +

Option 4:

introducing detailed

regulatory

framework

+ + -- ++

Magnitude of impact as compared with the baseline scenario (the baseline is indicated as 0):

++ strongly positive; + positive; – – strongly negative; – negative; ≈ marginal/neutral; ?

uncertain; n.a. not applicable

36

The Key Steps – Monitoring & Evaluation (D)

Level Policy objectives Monitoring indicators

General

Create/improve the functioning of the

internal market for product X

Cross-border trade between MS in

product X

Promote healthy diet Obesity

Increase employment of X Unemployment of X

Specific

Increase mutual recognition of

nationally approved products X

Number of requests for placing on the

market of products X coming from

another MS. Number of rejections.

Number of court cases.

Improve the nutritional value of people's

diet

Consumption of low fat, low sugar

food

Provide qualifications required on the

market for X

Number of X employed

Operational

Increase convergence regarding the

assessment criteria for the certification

of product X

Standards developed

Increase information on nutrition

content

Labelling requirements designed and

implemented

Create a re-training system for X Number of X retrained

Example: Link between monitoring indicators and objectives

37

The Key Steps – Monitoring & Evaluation (E)

Objectives Indicator Examples of indicators

Operational Indicators relate to results of

implementation and enforcement of an

intervention – i.e. deliverables that need

to be generated in order to achieve its

main objective(s).

Kilometres of roads built, scholarships

awarded, tax declarations filed,

permissions/derogations granted, bans

introduced, e-invoices exchanged cross

border, uptake/satisfaction with

provided consultancy services, etc.

Specific Indicators aim at monitoring what

concretely the policy intervention

intends to achieve, i.e. raison d'être of

your policy. These indicators should

allow monitoring direct, as well as any

significant indirect or unintended

impacts of an intervention.

Safety incidents at EU level, tax

compliance, innovations/new products

generated in the sector,

requests/complaints dealt with, time

saved by users of a road, survival rate

of businesses etc.

General Given the interaction with external

factors and other policies with the same

or similar Treaty-based objectives, these

indicators often serve the purpose of

monitoring whether the observed

changes move in the right direction,

rather than the individual contribution of

your policy intervention.

Net employment gains or losses in

sector concerned, gender pay gap,

expansion of a specific market,

consumer trust in a given technology

or product, etc.

38

How long does it all take?

39

Example: Regulation on requirements relating to emission limits and type-approval for internal combustion engines for non-road mobile machinery [COM(2014)581]

Roadmap (2011)

Stakeholder consultations (Jan-Apr 2013)

Impact assessment (2013)

IAB Opinion (Nov 2013)

Final proposal (25 Sept 2014)

40

Responsibilities & sharing tasks

Lead Commission service(s)

responsible for work and final IA report

external expertise

Key role of IA support unit/function in DGs

Work guided by

IA Steering Groups:

Early coordination

Input (incl. co-drafting)

Help with ToRs or consultation docs

How deep should the analysis be?

Proportionate analysis, which depends on:

Significance of impacts

Type of initiative

Refers to whole IA process: depth and scope of

analysis, data collection efforts, types of impacts

covered, etc.

42

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. How do we carry out an IA?

4. How can stakeholders contribute?

5. How do we control the quality of IAs?

6. Latest developments

Roadmaps Possibility for early reaction on Commission's plans

Stakeholder consultations

• Opportunity to provide views on policy ideas and input to impact assessment work; may also take place in context of other Commission analytical work (e.g. REFIT/ex-post evaluations, cumulative cost studies)

Continuously

• Continuous exchanges on policy impacts on a sector; sharing studies/analysis/methodology etc. with Commission

During legislative process

• Reactions on Commission's proposal once adopted

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. How do we carry out an IA?

4. How can stakeholders contribute?

5. How do we control the quality of IAs?

6. Latest developments

The Impact Assessment Board

• Independent body, members nominated on their personal capacity

• Established by President Barroso in 2006

• A 'positive' opinion in principle necessary to table proposal before College

• Assesses draft IAs against the IA guidelines

• Gives opinion on their quality and recommendations for improvement

• Changes announced for the Juncker Commission

The IAB's work in figures

2007 2008 2009 2010 2011 2012 2013

Total impact

assessments examined 102 135 79 66 104 97 97

Number of meetings 22 26 21 23 25 20 19

Number of opinions

issued 112 182 106 83 138 144 142

Opinions requesting

resubmission

(first submissions)

9 44 28 27 37 46 40

Resubmission rate 9 % 33 % 37 % 42 % 36 % 47% 41%

IAB's recommendations main focus (I)

IAB's recommendations main focus (II)

1. Why do we do impacts assessments (IAs)?

2. What is an IA?

3. When do we conduct an IA?

4. How do we carry out an IA?

5. How do we control the quality of IAs?

6. Latest developments

On-going review of the guidelines for IA, evaluations and consultations: due to be out as one package of BR guidelines.

Planned revision of the IIA

Changes to the IAB RSB with two external experts

New Working Methods

51

Latest developments in Better Regulation system

Useful information sources

• Commission work programme http://ec.europa.eu/atwork/programmes/index_en.htm

• Detailed information about forthcoming initiatives - roadmaps http://ec.europa.eu/smart-regulation/impact/planned_ia/planned_ia_en.htm

• IA reports and IAB opinions

http://ec.europa.eu/smart-regulation/impact/ia_carried_out/cia_2014_en.htm

• Impact Assessment guidelines

http://ec.europa.eu/smart-regulation/impact/commission_guidelines/commission_guidelines_en.htm

• EC Smart Regulation website

http://ec.europa.eu/smart-regulation/index_en.htm

• IAB 2013 annual report

http://ec.europa.eu/smart-regulation/impact/key_docs/docs/iab_report_2013_en.pdf