evaluating the quality and use of impact assessments the role and approach of the nao
DESCRIPTION
Evaluating the quality and use of Impact Assessments The role and approach of the NAO. To cover. The role of the National Audit Office (NAO) The potential benefits of Impact Assessments Our examinations of Impact Assessments Lessons learned. The role of the NAO. - PowerPoint PPT PresentationTRANSCRIPT
HELPING THE NATION SPEND WISELY
Evaluating the quality and use of Impact Assessments
The role and approach of the NAO
HELPING THE NATION SPEND WISELY
To cover
The role of the National Audit Office (NAO)
The potential benefits of Impact Assessments
Our examinations of Impact Assessments
Lessons learned
HELPING THE NATION SPEND WISELY
The role of the NAO
The NAO scrutinises public spending on behalf of Parliament through:
Financial audit of the accounts of all central government departments and agencies
Value for money audit of the economy, efficiency and effectiveness with which government spends public resources
Hold Government to account for spending taxpayers’ money
Independent analysis of the evidence
HELPING THE NATION SPEND WISELY
The Regulation teamWe cover:
The economic regulators – the design of regulators and risk-based approaches
Competition bodies – the introduction of competition and prevention of anti-competitive behaviour
Consumer issues – the impact or regulation on consumers and adequacy of consumer protection
The regulatory reform agenda – the effectiveness of the system and outcomes
5/6 reports each year.
HELPING THE NATION SPEND WISELY
Value for money
Hold regulators and departments to account but also seek improvements in delivery
Focus on effectiveness – low spend but big impact
Do not question the merits of policy decisions
Parliamentary scrutiny
HELPING THE NATION SPEND WISELY
Our Regulatory Reform work
Impact Assessments – since 2001
Reducing the burdens of regulation on business – since 2007
Reviewing regulators’ approach to inspecting and enforcing regulations – since 2008
HELPING THE NATION SPEND WISELY
Our objectives – regulatory reform
To hold the BRE and departments to account for delivering the regulatory reform agenda, and to help improve working practices
– Are ‘tools’ being used effectively?
– Are initiatives delivering their intended objectives?
HELPING THE NATION SPEND WISELY
The benefits of Impact Assessments
Increase transparency and strengthen accountability
Help encourage evidence-based policy formation
WHICH MEANS:
The need for regulation is challenged
A better chance of achieving good policy outcomes
HELPING THE NATION SPEND WISELY
Some challenges
Embedding IAs in policy development
Process and governance
HELPING THE NATION SPEND WISELY
Impact Assessments – NAO work
We began our work in 2001 and have published an annual report since 2003-04
Originally, we examined the quality of IAs
From 2006, we examined quality but also the use of IAs:• A review of departmental processes• Considered how depts have used IAs to inform policy development• Post-implementation review
2008 – our broadest review of quality
HELPING THE NATION SPEND WISELY
NAO criteria for assessment
Criteria NAO tests
A clear statement of the conclusion
Did the IA make comparisons against the preferred option?
Did the IA state the recommendation clearly?
Consideration of appropriate options
Did the IA consider a range of options, including the do-nothing option?
The use of impact tests and consultation
Did the IA consider social, environmental small business and competition tests?
Was consultation undertaken in accordance with guidance?
Standard of cost benefit analysis
Were costs and benefits of different options quantified? If not, was there a qualitative discussion?
What economic methods / expertise were used? Was the data source referenced?
The existence of and quality of implementation and enforcement plans
Is compliance considered? Does the IA detail who will enforce / monitor? Are there estimates of enforcement costs? Is there a plan?
Is there a statement on post-implementation review?
HELPING THE NATION SPEND WISELY
Results of NAO reviews
Generally ‘competent’ – few areas of significant weakness – policy officials understand the requirements
HELPING THE NATION SPEND WISELY
2007 Report
Scope and Purpose Consultation Costs and BenefitsCompliance and
Enforcement
Implementation, Monitoring and
Evaluation
Competition Assessment
GREEN 15 16 5 4 8 12AMBER 4 1 10 12 10 5RED 0 2 4 0 1 0N/A 0 0 0 3 0 2
HELPING THE NATION SPEND WISELY
2008 Report
Tests
Trend Overall assessment
Conclusion Improvement The strongest element of both samples – with a small improvement in 2008
Implementation and enforcement
Improvement The weakest area in 2006 but a considerable improvement in 2008
Analysis of costs and benefits
Improvement A weak area in 2006 but a number of improvements in 2008.
Impact tests and consultation
No change Scored well in both samples – although no improvement
The consideration of options Deterioration In 2008 fewer IAs presented an analysis of a range of options.
HELPING THE NATION SPEND WISELY
Results of NAO reviews - quality
COST-BENEFIT ANALYSIS
Improving but still persistent weaknesses
– greater incidence of quantification• 67% estimated costs (up from 56%)• 60% estimated benefits (up from 40%)
– type of analysis• 24% used sensitivity analysis (up from 13%)• 22% used specialists (up from 14%)
HELPING THE NATION SPEND WISELY
Results of NAO reviews - quality
But wide variation between the best and worst IAs
– Superficial evidence base– Basic analysis– Lack of justification for an absence of monetisation
A NEED TO FOCUS ON THE EVIDENCE BASE
A need to consider a range of options, including non-regulatory
HELPING THE NATION SPEND WISELY
Results of NAO reviews - quality
IMPLEMENTATION AND ENFORCEMENT
Practical enforcement considerations often missing:
– 80% did not include an implementation plan– 66% did not estimate enforcement costs
Impact tests need to be applied with more vigour – need to consider the full range of potential impacts
CONSIDER THE PRACTICAL IMPLCATIONS OF NEW REGULATIONS
HELPING THE NATION SPEND WISELY
Results of NAO reviews - governance
Departments are strengthening scrutiny arrangements
– increased frequency of review; – greater use of economists in review
IAs more embedded in the policy formation process
Challenge still lacking in places
A need to make scrutiny bite – inc external scrutiny
HELPING THE NATION SPEND WISELY
Lessons learned
Improve content of IAs
Establish robust processes to integrate into policy formation
Provide support for policy officials
Culture? Embed principles into a way of thinking
HELPING THE NATION SPEND WISELY
Lessons learned
A need to make IAs an integral part of the policy making process
How to create the right incentives?
– Senior level commitment– Parliamentary use in the passage of legislation– Scrutiny / review – Government / SAI / Parliament– Show benefits to policy officials / make easier to use