rod bogaards productivity commission australian government

Productivity Commission

Capacity Building On Regulatory Review

System Design Module‘Designing a RIA system that works effectively’

21-25 March 2011Rod Bogaards

Productivity CommissionAustralian Government

Productivity Commission 2

Some disclaimers!

• These are my own personal views although reference is made to Productivity Commission work

• While the Australian RIA system is highly regarded by international standards, I will focus on some of its pitfalls – no RIA system is perfect!

• How can we improve or enhance RIA institutional and analytical regulatory settings - that are already reasonably robust - to generate better and more consistent regulatory outcomes?

Productivity Commission

‘Every political party believes in the idea of better regulation. And yet every political

party, once in government, fails to achieve better regulation.’

Boyfield (2007)


Foundations of an effective RIA system

• Adequacy criteria

• Gate-keeper

• Transparency and accountability


Objectives of Regulatory Impact Analysis

• RIA aims to informs political decision makers on whether to regulate and how to regulate

• It aims to communicate information about the expected effects of regulatory proposals that can be used by decision makers


Good regulation addresses market failures …

• While allowing for:

• greater consumer choice, convenience and lower prices

• enhanced flexibility and competitiveness

• increased productivity

• greater transparency, accountability and public confidence in governance


‘Good regulation’ checklist

• Minimum necessary to achieve objectives

• Not unduly prescriptive

• Accessible, transparent and accountable

• Integrated and consistent with other regulations


‘Good regulation’ checklist

• Communicated effectively

• Mindful of the compliance burden imposed and

• Enforceable


What is Regulatory Impact Analysis?

• RIA processes follow a formalised sequence of steps to establish whether particular regulatory proposals would be in the community’s interest - by delivering a net benefit to the community

• RIA processes do this by taking a community-wide perspective of their effects, to ensure that the benefits to society of a regulation (broadly conceived) are greater than the costs (also broadly conceived)


7 Steps in a Regulation Impact Statement

1. Identifies the problem the regulation seeks to address

2. Outlines the objectives of government action3. Identifies a range of feasible options for addressing

the problem4. Assesses the costs and benefits of the feasible

options5. Documents community consultation6. Proposes a recommended option7. Outlines implementation and review mechanisms


Practical operation principles

• All seven elements of RIS adequately addressed

• Benefits should outweigh costs and (usually) net benefits maximised

• Analysis commensurate with size of likely impacts

• Objective, balanced public statement, not advocate’s document


The adoption of RIA processes is now widespread

• RIA have proved popular with governments trying to improve the quality of regulation:

• in 1980, only 2 or 3 countries were using RIA

• in 2000, 14 of 28 OECD countries had adopted RIA programs

• in 2010, all 31 OECD countries and many non-OECD countries had mandated the use of RIA


Trend in RIA adoption across OECD countries (1974-2010)

0

5

10

15

20

25

30

1974

1976

1978

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

Num

ber o

f jur

isdi

ction

s

31


Australia was an early adopter of RIA

• In 1985, Australia was one of only eight OECD countries with formal requirements for regulatory impact analysis

• In 1995, RIA was extended to cover the development of all national standards

• In 2009, all Australian states and territories had their own individual RIA processes to cover state government regulations


What should a good RIA process deliver?

• A well designed and implemented RIA process can improve the quality of new regulations:

• by demonstrating that a non-regulatory option is a better solution to the problem (or that the status quo is preferable)

• by identifying the most effective and efficient design elements to build into the new regulation, thereby increasing the benefits and/or reducing the costs


What should a good RIA process deliver?

• A well designed and implemented RIA process can improve the quality of new regulations:

• by building stakeholder support for proposals

• by reducing the risk of over-regulation

• by influencing over time the ‘regulatory culture’ within government agencies


Is RIA quality very good? OECD view

• The OECD (2009) has identified a number of issues with the performance of RIA:

• not well integrated in the decision-making process

• not applied to the most significant regulatory proposals

• fails to fully assess the costs and benefits of proposals

• no serious consideration of feasible alternatives


• The reviews revealed major weaknesses across most countries

• Relative lack of integration of RIA in the policy process, assessments are done too late, consultation is often not robust and oversight needs more teeth – methodological challenges

• Late timing of impact assessments is a widespread issue

• Setting up an oversight function in a single central unit has been difficult for many countries

• In order to force change a few countries are making RIA a legal requirement – with sanctions for non compliance

Findings of the OECD EU 15 Country Reviews


What is the quality of RIA? Has it improved regulatory outcomes? US experience

• A US study said most agencies failed to quantify costs and benefits, discuss alternatives, use consistent analytical assumptions, report their results clearly or make them accessible (Hahn 2000)

• A more recent US study said ‘… although there is some evidence economic analysis can improve the benefit-cost ratio of regulations, there is insufficient evidence that economic analysis of regulatory decisions has actually had any substantial impact’ (Hahn and Tetlock 2007)


What is the quality of RIA? Has it improved regulatory outcomes? UK experience

• A study of the UK RIA process also found the standard of RIA to be poor, a ‘bureaucratic sham’, treated ‘as a bolt-on extra designed to justify a regulation’ rather than being used to shape and inform policy formulation (Boyfield 2007)

• A study of UK Government RIAs by the British Chamber of Commerce concluded, ‘… we question whether RIAs have changed the political reality that ministers introduce regulations because they want them as distinct from being able to justify them’ (Ambler et al 2007)


What is the quality of RIA? Has it improved regulatory outcomes? EU experience

• A study of RIA processes and practices in the European Union and all Member States found:

‘In almost all cases we have examined, there is a large gap between requirements set out in official documents and actual Impact Assessment in practice … typically assessments are narrow, partial and done at a late stage. In many countries, a large share of proposals is not formally assessed or is assessed with a ‘tick box mentality’’ (Jacob et al 2008)


How effective has RIA been in improving regulation? Australian experience

• In Australia the results of RIA requirements have also been somewhat mixed

• 21 years after the RIA process was established, the 2006 Taskforce on Reducing Regulatory Burdens on Business pointed out the dramatic growth in the volume of regulation

• Quality of regulation was also at issue as complaints about growing regulatory burdens led to the Regulation Taskforce inquiry


Findings of the 2006 Regulation Taskforce

• Too much regulation imposing an unnecessary cost on business due to:• unclear or questionable objectives• failure to target the regulation – too blunt or

disproportionate to the size of the problem• undue prescription• excessive reporting or paperwork requirements• overlap, duplication or inconsistency with other

regulation, either within or between jurisdictions• poorly expressed and confusing terms• unwarranted differentiation from international

standards


How did this happen?

• Rising phenomenon of risk aversion in society

• Over reliance by governments on the development of regulatory solutions – ‘regulate first, ask questions later’ culture

• The requirements for good regulatory process had not been effectively discharged

• Taskforce said all of the above


Beginning of a new phase of regulatory reform

• Unless the underlying reasons for regulatory failures are addressed the regulatory problems will simply re-emerge

• The Regulation Taskforce recommended six principles of good regulatory process which were endorsed by the Australian Government


6 Principles of good regulatory process

1. Governments should not act to address problems through regulation unless a case for action has been clearly established

2. A range of feasible policy options – including self-regulatory and co-regulatory approaches – need to be assessed within a cost-benefit framework



3. Only the option that generates the greatest net benefit for the community (taking into account economic, social, environmental and equity impacts) should be adopted

4. Effective guidance should be provided to regulators and regulated parties to ensure that the policy intent of the regulation is clear, as well as what is needed to be compliant



5. Mechanisms such as sunset clauses or periodic reviews need to be built in to legislation to ensure that regulation remains relevant and effective over time

6. There needs to be effective consultation with regulated parties at the key stages of regulation making and administration


Changes to the RIA process in response to Regulation Taskforce recommendations

• ‘Preliminary assessments’ for all regulatory proposals to determine the level of regulatory impact analysis required

• A formal assessment of business compliance costs, using the Business Cost Calculator (or an approved equivalent), and greater use of cost-benefit analysis generally



• A clear statement in the RIS adequacy criteria that the RIS should demonstrate that:

• the benefits of the proposal to the community outweigh the costs

• the preferred option has the greatest net benefit for the community, taking into account all the impacts

• A whole-of-government consultation policy



• Improved gate-keeping arrangements that prevent a regulatory proposal from proceeding to Cabinet (or other decision maker) unless an adequate RIS has been prepared

• Post-implementation reviews are required within two years of implementation when a proposal proceeds to a decision maker without an adequate RIS



• The existing Office of Regulation Review was renamed the Office of Best Practice Regulation (OBPR) – to reflect a new focus to assist agencies to develop regulatory best practice

• A specialised cost-benefit analysis unit was created in the OBPR to provide advice and support to agencies preparing RIS


Regulation Taskforce had high expectations that it would create a better RIA system

“In the Taskforce’s view, the single most important way of strengthening compliance with the principles of good process would be for government to adhere to a rule that regulatory proposals that fail to meet the RIS requirements will not be permitted to proceed to consideration by Cabinet or other relevant decision maker, except in specially defined circumstances. Together with stronger analytical requirements, this would ensure the processes were taken more seriously and at an earlier stage”


RIA compliance rate following Regulation Taskforce reforms

• Compliance with Australian Government best practice regulation requirements is measured by compliance with the requirement to prepare adequate RIS (and BCC reports)

• To be assessed as adequate, a RIS must contain a degree of detail and depth of analysis that is commensurate with the size of the potential impacts of the proposal

• The OBPR used the criteria in the BPR Handbook (2007) to assess whether each element of a RIS is adequate


RIA compliance rate following Regulation Taskforce reforms

Type of RIA 2007-08 2008-09 2009-10

RIS 43/48 (90%) 45/53 (85%) 63/75 (84%)

BCC 7/7 (100%) 4/4 (100%) 2/2 (100%)

Exceptional Circumstances

3 6 4

Preliminary Assessments

753 662 823


What does a high RIS compliance rate mean?

• Does it mean the gate-keeper is doing a good job?

• A low compliance rate from a failure to conduct RISs indicates the RIS process is being evaded

• But a high compliance rate does not necessarily tell us much about the quality of the RIS supporting a regulatory proposal


Role of the Australian gate-keeper

• Independent (but within the Department of Finance & Deregulation)

• Advises on whether a RIS is required

• Examine/advise on adequacy of RISs

• Provide training and advice on RIS process

• Reports annually on RIS compliance


Are gate-keeper’s prone to Type I errors?

Pass Fail

Adequate RIS Correct Decision Type II Error or ‘false negative’

Inadequate RIS Type I Error or ‘false positive’

Correct Decision


Reasons why gate-keepers could be prone to Type I errors

• Political pressure - from government departments or Minister’s offices to pass a RIS can be intense

• And as the penalties for non-compliance were raised in 2007 – since the gate-keeper could effectively veto regulatory proposals, preventing them from going to decision makers - the pressure to pass RISs may have increased even more


Reasons why gate-keepers could be prone to Type I errors

• Insufficient transparency – publication of RIA (and the OBPR’s adequacy assessment) potentially occurred up to 18 months after a decision to proceed with a regulatory proposal had been made by government

• this may have provided the gate-keeper with poor incentives to perform its role and enforce the RIA process properly


How does Australia’s 2007 RIA model rate with respect to the key drivers?

• Strong adequacy criteria

• Strong gate-keeper

• Weak transparency and accountability


But did the 2007 changes lead to better RIA system?

• The existence of robust adequacy criteria and gate-keeper are necessary criteria for an effective RIA system – but they are not sufficient criteria

• The ultimate effectiveness of any RIA system depends on the level of transparency and accountability in the system


A case study of a proposed regulation

• Let’s examine a regulatory proposal and see how the RIS for the proposal engages with the policy development process


Productivity Commission’s 2009 Annual Review of Regulatory Burdens on Business

• The Productivity Commission proposed a strengthening of the best practice regulation requirements and made a number of key recommendations

• Industry submissions commented on the ongoing failure of government to properly assess the risks, costs, and benefits of regulation and called for further scrutiny of the process by which regulation is developed


• The concerns raised suggested there remains scope for the Australian Government to do more to improve regulatory outcomes, through:

• more systematic and rigorous application of the existing best practice regulatory process

• refinements and enhancements to aspects of the design and implementation of this process

• increasing the transparency and accountability of the current regulatory process




• The Commission recommended the Australian Government should develop a central online register of regulatory impact analysis that would include:

• RISs (and importantly, the OBPR’s adequacy assessments) published at the time government decisions are made public

• post-implementation reviews (and the OBPR’s adequacy assessments) published at the time these reviews are made public



• The Commission also recommended the Australian Government:

• incorporate a ‘consultation’ RIS in its regulation making process

• establish an independent public review of the current best practice regulation requirements


OECD’s 2010 Review of Regulatory Reform in Australia

• The OECD said Australia rates highly among OECD countries on the design and performance of its RIS process

• But said there are some potential improvements that could strengthen the Australian Government RIS process


Requirements for RIA processes used by central governments

2005 and 2008 (RIA 1)

Note: This figure summarises information about the existence of key elements of RIA processes in OECD member countries. It does not offer information on the quality of specific RIAs.Questions: Weights:a) Is regulatory impact analysis (RIA) carried out before new regulation is adopted? a) if no=0, in some cases=1, always=2c) Is a government body outside the ministry sponsoring the regulation responsible for reviewing the quality of the RIA? c) if yes, weight=3

e) Is there a clear "threshold“for applying RIA to new regulatory proposals? e) if yes, weight=2h(i) Is RIA required by law or by a similarly strictly binding administrative instrument? h(i) to h(vii):

if no=0, in other selected cases=1, only for major regulation=1, always=2

h(ii) Is RIA required for draft primary laws?h(iii) Is RIA required for draft subordinate regulations?h(iv) Are regulators required to identify the costs of new regulation?If yes: Is the impact analysis required to include the quantification of the costs?h(v) Are regulators required to identify the benefits of new regulation?If yes:Is the impact analysis required to include the quantification of the benefits?h(vi) Does the RIA require regulators to demonstrate that the benefits of new regulation justify the costs?h(vii) Are RIA documents required to be publicly released for consultation with the general public?k) Are ex post comparisons of the actual vs predicted impacts of regulations made? k) if yes, weight=1l) Is there an assessment of the effectiveness of RIA in leading to modifications of initial regulatory proposals undertaken? l) if yes, weight=1

Source: Question 10 / OECD Regulatory Management Systems’ Indicators Survey 2005 and 2008. www.oecd.org/regreform/indicators

0

5

10

15

20

25ES

PSV

KPR

TFR

ATU

RICE

NOR

NLD

HUN

AUT

BEL

EU LUX

ITA IRL

USA

SWE

GRE CHE

DNK

DEU POL

FIN

NZL

KOR

MEX JPN

CZE

AUS

CAN UK

20052008

27


Extent of RIA processes

2005 and 2008 (RIA 2)

Note: This graph summarises information about the extent of RIA processes in OECD member countries. It does not gauge whether these processes have been effective.Questions: Weights:d(ix) Is the RIA required to include assessments of other specific impacts: Impacts on the budget, impacts on competition, impacts on market openness, impacts on small businesses, impact on specific regional areas, impact on specific social groups, impact on other groups (not for profit sector including charities), impact on the public sector

d(ix), Impacts on the budget, competition, market openness, small businesses, specific regional areas, specific social groups, the public sector: if no=0, in other selected cases=1, only for major regulation=1, always=2 d(ix), Impact on other groups (charities, not for profit sector): if no=0, in other selected cases=0.5, only for major regulation=0.5, always=1

e) Is risk assessment required when preparing a RIA? e) if no=0, in other selected cases=1, only for major regulation=1always=1

For all regulation, for health and safety regulation, for environmental regulation? if yes, weight=1If “yes”: Does the risk assessment require quantitative modelling?f(i) Does the RIA require regulators to explicitly consider compliance and enforcement issues when preparing new regulation?

f(i) if yes, weight=1

f(ii) Are reports prepared on the level of compliance with the above RIA requirements? f(ii) if no=0, ad hoc basis=1, regularly=2f(iii) Are these reports published? f(iii) if yes, weight=2

Source: Question 10 / OECD Regulatory Management Systems’ Indicators Survey 2005 and 2008. www.oecd.org/regreform/indicators

0

5

10

15

20

PRT

ICE

LUX

TUR

ESP

HUN

FRA

KOR

NLD SVK

NOR EU G

RE POL

ITA

SWE

DEU BEL

AUT

CHE

JPN

MEX IRL

DNK

FIN

AUS

CZE

NZL

CAN

USA UK

20052008

24


OECD’s 2010 recommendations to improve Australia’s RIS process

• Ministerial certification of each RIS – to establish greater accountability at Ministerial level

• RIS training could be extended to Ministerial offices

• OBPR should report on compliance by government agencies with the obligation to quantify costs and benefits of regulatory proposals

• the RIS should be published in draft format for public consultation – a two stage approach

• the Australian National Audit Office should periodically review the quality of RISs – ‘Sed quis custodiet ipsos custodes’?


Recent changes to the Best Practice Regulation Handbook: Some positive

• Development of a central online RIS register (but only includes a ‘Yes’ adequacy assessment by OBPR – but no explanation of why ‘Yes’)

• Earlier signalling of non-compliance with the RIS process (previously took up to 18 months for the public to find out)

• Post-implementation reviews to be published on the central online RIS register (but adequacy assessment by OBPR not reported online)


Recent changes to the Best Practice Regulation Handbook: Some questionable

• That no explicit comparison of options will be required in a RIS

• That Cabinet may direct government agencies regarding which options to examine in a RIS

• That election commitments to introduce regulations will only be subject to a ‘modified’ RIS


Recent changes to the Best Practice Regulation Handbook: Some questionable

• That a RIS no longer has to demonstrate that the preferred option has the greatest net benefit for the community, taking into account all the impacts (economic, social, environmental and equity)

• That a RIS can be modified after the decision maker’s consideration but prior to publication to allow the inclusion of an option not considered in the original RIS – but without the public knowing


Recent comments by the Business Council of Australia on 2010 RIA model

• ‘ … changes to the RIS process may inhibit the independence and robustness of the policy analysis and advice received by Ministers from the bureaucracy, and this will have major implications for the outcomes of policy and its effectiveness’

• ‘These reforms may also reduce the effectiveness of the OBPR as an assessor of RISs … narrowing the requirements for analysis in RISs means that the OBPR will have less ability to assess them as being non-compliant with the process ... further diminishing the power of the OBPR to reject RISs’


How does Australia’s 2010 RIA model rate compared to the previous model?

2007 RIA model 2010 RIA model

Adequacy criteria Strong Weaker?

Gate-keeper Strong Strong

Transparency and accountability

Weak Weaker?


A perverse outcome of Australia’s ‘strong’ gate-keeper model?

• Giving the gate-keeper the power to veto regulatory proposals in 2007 - if the RIS process was not followed – may have led to increased pressure on the gate-keeper to pass RISs

• Gate-keepers do not need a ‘veto power’ to be effective so long as their adequacy assessments are made transparent in a timely manner


How can we improve the effectiveness of the RIA system?

• Transparency and accountability is the most important driver of an effective RIA system

• A RIA system must take account of, and improve, the incentives that face Ministers, departments and gate-keepers

• This requires a significant effort to improve transparency and accountability – the ‘inconvenient’ driver of an effective RIA system!


Foundations of an effective RIA system

Effective RIA System

Adequacy criteria( )

Gate-keeper()

Transparencyand

accountability ( )


Future work by the Commission that could impact on Australian RIS processes

• In 2012, the Commission will undertake a benchmarking study of the efficiency and quality of both COAG and all Australian jurisdictional RIS processes


• Design opportunities – would extending the requirements for RIA to Malaysia improve the regulation-making system?

• Governance opportunities – how to build oversight and quality control within the administration?

• Training opportunities – providing a strong support function within government – help desks

• Experience suggests that RIA is always a work in progress, governments must always return to strengthen and support the process

Opportunities for Malaysia ...