Download - 1 INDICATORS OF REGULATORY QUALITY Project website Claudio M. Radaelli Brussels, 25 January 2005
1
INDICATORS OF INDICATORS OF REGULATORY QUALITYREGULATORY QUALITY
Project website
Claudio M. RadaelliBrussels, 25 January 2005
2
Description of the projectDescription of the project
Regulatory quality is a complex concept. In our project we acknowledge the role of different stakeholders and institutional contexts
Quality measured in terms of design, activity-output, and real-world outcome
Assessment of indicators and indexes proposed in the literature
Analysis of better regulation programmes: focus on four tools (IA, Simplif., Access, Consultation)
Questionnaire sent to support network in May 2004 The project is about the DESIGN of indicators. We have
not calculated indicators, neither have we scored Member States
3
Why quality?Why quality?
Quality versus quantity Quality and efficiency of regulations Better regulation as a public policy (actors,
resources, instruments, and decision-making structure)
Quality and the Lisbon strategy Quality and indicators
4
Measuring regulatory quality in EU Measuring regulatory quality in EU Member States (questionnaire)Member States (questionnaire)
• Diffusion of the Mandelkern report principles• Most countries have opted for an experimental approach
to indicators – one based on the gradual introduction of simple measures
• Setting quantitative targets is an effective way to encourage public administration to implement measurement systems (as shown by the case of B, DK, NL and S)
• No systematic use of cost-benefit analysis in the EU Member States. Rather, the emphasis is on how stakeholders are affected by proposed regs
5
Diffusion of principles but different practice: Diffusion of principles but different practice: three clustersthree clusters
1. Tools of better regulation are still at the stage of pilot project. Measurement of quality has been considered, but limited activity.
2. Basic systems of quality assurance and focus on administrative burdens.
3. Sophisticated quality assurance, well-embedded impact assessment, initiatives for the measurement of quality which target the whole regulatory process
6
Combining diversity of approaches and Combining diversity of approaches and common progress in the EUcommon progress in the EU
• Different practice is compatible with common progress
• A common approach, shared beliefs about regulatory quality and its components, and sense of direction towards the Lisbon goals
• Three specific sets of indicatorsThree specific sets of indicators
7
The hard questionsThe hard questions
FROM INSTRUMENTS TO POLICY: How do we know that a good IA produces better regulation?
FROM POLICY TO REGULATORY OUTPUT: How do we know that ‘better regulation policy’ produces better regulations?
FROM REGULATORY OUTPUT TO FINAL ECONOMIC OUTCOME: What is the causal chain going from good regulation to real-world economic outcomes? How robust is it?
8
How we answer these questionsHow we answer these questions
Focus on how regulation changes the behaviour or stakeholders
Selection of five indicators that measure the quality of better regulation policy without going too far in the causal chain
We use Lisbon indicators, but we do not test the quality of better regulation policy directly on economic indicators like the growth of the knowledge-based economy
9
The three systemsThe three systems
1. Design of better regulation policy and tools. Macro and ex-ante
2. Ex-post and micro
3. Bridge between measurement of regulatory quality and the systematic evaluation of better regulation as public policy
10
Where do the indicators come Where do the indicators come from?from?
Operationalisation of the principles of better regulation
Questionnaire and consultation with support network
Experience of MS and other countries Criteria of “usable knowledge” = indicators
that can be embedded in policy processes
11
Indicators – characteristics of system 1: quality of the process
• an ex-ante macro-system - does not go down to the level of individual impact assessments or consultations
• based on objective indicators, it covers the design of better regulation policies, instruments (contents of formal guidance), and supporting and quality assurance activities
• can be managed by coordinating units in MSs and the SG in the Commission
12
Quality assuranceQuality assuranceIndicatorIndicator What is measuredWhat is measured InterpretationInterpretation Data collectionData collection
Quantitative Quantitative targetstargets
Existence or absence
Targets facilitate implem. of BR programmes
IRQ dataset
Reporting on Reporting on progressprogress
Existence or absence
Accountability and transparency
IRQ dataset
Performance of Performance of quality assurance quality assurance unitunit
Existence or absence
Establishing quantitative criteria to assess success or failure
IRQ dataset
Civil society – Civil society – public official public official documentdocument
Existence or absence
Legitimacy of better regulation programmes
Official guidance
Monitoring and Monitoring and evaluationevaluation
Existence or absence
Culture of monitoring and evaluation
IRQ database plus official guidance
13
Indicators – characteristics of system Indicators – characteristics of system 2: internal evaluation2: internal evaluation
• An ex-post micro-level system focused on what the tools of better regulation deliver in a given time-period
• We look at guidance as designed in system 1 and measure what has been achieved
14
System 2 contains the following:System 2 contains the following:
1. A simple checklist on the quality of individual impact assessments and major consultations
2. Real-world measures of the impact of better regulation policies (subjective: surveys over time)
3. Five real-world measures of the impact of better regulation policies (objective)
4. Limited number of important indicators of economic outcome – role of causal chains
15
Survey of regulators Survey of regulators Percentage of regulators who think that IA has the Percentage of regulators who think that IA has the
following effect:following effect:
[1] Makes policy formulation more transparent[2] Speeds up / delays policy implementation[3] Reduces conflict at the stage of policy formulation[4] Reduces conflict at the stage of policy implementation[5] Be more likely to represent the preferences of citizens[6] Final policy decision will be more likely to achieve goals[7] Final policy decision be more likely to represent a
consensus among affected entities[8] Final policy decision be more likely to create open
markets
16
Survey of citizens and firmsSurvey of citizens and firms[1] % of citizens who think that their views are taken on board in
the development of single market policy in the EU [2] % of firms who think that the quality of regulation in the EU
(including both EU and domestic rules) has improved over the last three years
[3] % of firms who think that participation in EU-level consultation makes an impact on the final decision
[4] % of firms who think that EU policy has an impact in opening markets, making it easier to compete, and generating competition in the professions
[5] % of firms who think that in last three years restrictions and obstacles to their business have disappeared altogether or significantly reduced
[6] % of firms who think that regulatory changes at the EU level are predictable
17
The Five Real-World measures of The Five Real-World measures of better regulation policybetter regulation policy
[1] Estimate of total regulatory costs delivered by regulations for which impact assessments (IAs) were prepared in year xxxx / Estimate of total regulatory benefits delivered by regulations for which IAs were prepared in year xxxx
[2] Net benefits delivered by regulations for which IAs were prepared in year t / Net benefits delivered by regulations for which IAs were prepared in year t-1
[3] Total cost reduction resulting from simplification [4] Annual rate of reduction in the total administrative
burdens stemming from EU legislation [5] Cost of administrative procedures eliminated in year t /
Cost of administrative procedures eliminated in year t-1
18
System 3: external evaluation “All policies can be evaluated, and better regulation is no exception”
• no matter how sophisticated, indicators are just a component of quality assurance
• the latter requires strategic and operational management, specific structures, and dedicated tools
• one important element is external evaluation• indicators must be complemented by interviews, case
studies, etc. • external evaluation should be done by think tanks,
academics, and stakeholders - notion of pluralistic evaluation
19
Indicators – which system is suitable for whom?
system 1 could be used by the cluster of Member States with basic approaches to regulatory quality and impact assessment in experimental-pilot stage. They should move as soon as possible to the second system
system 2 could be used by the cluster in which consultation, simplification, and the assessment of administrative burdens are already well embedded - however, some simple indicators of this system can be adopted by all EU governments now, no need to wait
20
Indicators – which system is suitable for whom?
finally, system 3 is applicable to countries with sophisticated quality assurance mechanisms (as well as to the Commission)
21
How to use indicatorsHow to use indicators
• Open method of coordination• High-level group on Competitiveness –
Competitiveness Council, this can be a
body that can adopt indicators• DBER can be the forum where data are
collected and technical discussions on
indicators take place
22
Why use open method of Why use open method of coordination?coordination?
• Better regulation policy is an essential component of the Lisbon agenda.
• It involves different levels of governance. • A responsibility for the EU and the
Member States