3rd party independant reviews

40
The gateway review process a quite success story of using a quality assurance process to improve the delivery of high risk projects …….and more generally, some lessons from around the globe Nick Pelham

Upload: nick-pelham

Post on 25-Jun-2015

84 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 3rd party independant reviews

The gateway review process

a quite success story of using a quality assurance process to improve the delivery of high risk projects

…….and more generally, some lessons from around the globe

Nick Pelham

Page 2: 3rd party independant reviews

But first…..a case study in asset management : project vs operations

Page 3: 3rd party independant reviews

The iron triangle of project management

TimeCost

Quality

Page 4: 3rd party independant reviews

Sydney Opera House, NSW (1952)

Page 5: 3rd party independant reviews

• Time planning began in the 1940s

• The need - “a suitable venue for large theatrical productions”

architectural design competition awarded in 1957 (Jorn Utzon).

construction started in 2 March 1959

Original completion date – 26 Jan 1963

Opened in 20 October 1973• Stage 1 (upper podium –> „59-63)• Stage 2 (roof -> „63-67)

Construction of the Opera HouseV

Page 6: 3rd party independant reviews

original estimate – £3.5m (~$7m) (1957)

final cost ~ $98.8 - $102m (1973)

since 1973... the first upgrade - $120m (1988-98) the second upgrade - $ 66.59m (2002)

total construction and refurbishment – (1957-2002)

$374.4m

+ „Venue Improvement Program‟ (2001-09) $69m

Page 7: 3rd party independant reviews

You all have your views……

Page 8: 3rd party independant reviews

Quality

TimeCost

1400% + 3½yrs ->14+ yrs

Page 9: 3rd party independant reviews

• Average cost per year over past 5 years

• $121m pa• Bldg dev and expenses $ 31.5m• Expenses $ 89.5m

• + „Venue Improvement Program‟ (2001-2009) - $69m($7.7m per year)

• More likely average cost per year

$128.7m pa

Running of the Opera HouseV

Page 10: 3rd party independant reviews

2011 NSW Audit

SOH needs a $1.1b upgrade to its facilities to remain viable and to address access and safety concerns:

• flawed acoustics• limited accessibility• limited seating and restricted spaces• shortcomings in OHS

But………

Page 11: 3rd party independant reviews

Construction

• $102m (1973) $102m

Upgrades (1988-2009)

• Upgrade 1 (1988-1998) $ 120m• Upgrade 2 (2002) $ 66.59m• Venue Improvement Program (2001-2009) $ 69m

$ 255.59m

• Upgrade 3 (2014- 2023) $1,100m

Running the opera house

over next 10 years $1, 326.6m

Summary

Page 12: 3rd party independant reviews

So what…….

Rule of thumb

1 - 4 - 20

Page 13: 3rd party independant reviews

1. Gate review processes

Page 14: 3rd party independant reviews

Origins of the ‘gate’ concept

underlying idea behind Gate Reviews can be traced to research and development in the private sector in the 1980s and is widely attributed to work Dr R. G. Cooper (Canada).

The terms used to describe such reviews include ‘Stage‐Gates’, ‘Gateway Reviews’ etc….

The approach remains strongly identified with product innovation and the introduction of new products to the market by private companies.

Page 15: 3rd party independant reviews

In public sector organisations

The Gateway Review Process in Australia began in 2003 and is modelled on the UK’s Office of Government Commerce (OGC) Gateway™ Process that was originally designed to address failures in information technologyprojects

Developed by Sir Peter Gershon in the UK in Feb 2001

Page 16: 3rd party independant reviews

Project performance-cost overrun by percentage (based on Flyvbjerg, 2003)

Page 17: 3rd party independant reviews

The Gateway Review Process

Gate 1: Strategic Assessment- Confirm business strategy and need

Gate 2: Business Case- Confirm business case is robust

Gate 3: Readiness for Market- Confirm procurement method and sources of supply

Gate 4: Tender Decision- Confirm investment decision

Gate 5: Readiness for Service- Confirm commissioning plans are in line with business strategy and

industry best practice

Gate 6: Benefits Evaluation- Assess project delivery effectiveness

Page 18: 3rd party independant reviews

Opportunity to influence project outcomes

Strategic

assessment

Procurement

Strategy

Competitive

Procurement

Award and

implement

contract

Manage Contract/facility

to deliver services

Develop

Business Case

Benefits

realisationConstruction Readiness

for Service

Needs

analysis

Business

justification

Procurement

strategy

Investment

decision

Whole

of Life

Expenditure

0%

100%

Time

Page 19: 3rd party independant reviews
Page 20: 3rd party independant reviews

What we’ve learnt.........

Page 21: 3rd party independant reviews

BENEFIT STRATEGIC

INTERVENTIONSPROBLEM

Improved liveability

around the market

KPI 1: Reduction in CO2 and

noise emissions

KPI 2: Improved health of

residents

Restrict truck

movements to daylight

hours

Remove trucks from

residential streets

Relocate residences

outside market area

Increasing road

freight in residential

streets is causing

declining health in

people who live near

the market

Starting at the start…..defining the Problem

Page 22: 3rd party independant reviews

Focus on a Case for the Change (business case)

• No executive summary

• No genuine link to strategic priorities

• Lack of specificity as to the ‘need’ or proposed solution –aka solutions looking for problems

• Lack of clarity of timing, cost and benefits

• Use of diagrams or tables which are unrelated to the text or hard to interpret

• Obvious inconsistency both numerical and with the detail of the business case

Traps for new players or poor past efforts:

Page 23: 3rd party independant reviews

The gateway review process ‘adds value’

UK –>

•2008 2-4% cost avoidance *

Australia

• 2009 3-5% cost avoidance

• 2010 22.4% time benefit, 29.3% cost benefit (Uni Melb)

* If all recommendations are implemented as outlined in the gateway review report

Page 24: 3rd party independant reviews

Assurance provided from the Review Process

Page 25: 3rd party independant reviews

Judging success

• Large projects often best judged with the benefit of hindsight

• Time and cost are useful yardsticks for success- ………. as is function and stakeholder acceptance

• Large projects are inherently delivered in a high risk environment

• Performance of capital projects are largely determined by time and cost

- Remember Sydney Opera House– Project outcomes

– Upgrade costs

Page 26: 3rd party independant reviews

Other jurisdictions

Page 27: 3rd party independant reviews

Whole of Life analysis – individual projects

Page 28: 3rd party independant reviews

Australia

1. Managing complex governance arrangements across agencies and jurisdictions

2. Project inception, approvals and funding

3. Risks associated with implementing projects in tight timeframes

4. Establishing a benefits lead approach

New Zealand

1. Business case

2. Risk and issues management

3. Program/project management

4. Stakeholder management

5. Resourcing

6. Governance

7. Program/Project planning

8. Management of change

9. Sourcing strategies

10. Dependency management

Page 29: 3rd party independant reviews

2. Structural deficiencies

Page 30: 3rd party independant reviews

‘Reviews’ of the reviews

Page 31: 3rd party independant reviews

Audits in the UK and Australia

2010

Audit of the Efficiency and Reform Group’s role in improving public sector value for money

Ombudsman investigation in to ICT enabled projects

Assurance for high risk projects

Assurance for major projects

Gate Reviews for Defence Capital Acquisition Projects

Administration of the Gateway Review Process

Parliamentary enquiries

2011 2012

Page 32: 3rd party independant reviews

What was found (+)

+…..a central quality assurance tool which assists in the delivery of high risk and complex projects…..

+…..valued by stakeholders for the benefits that result from preparing for a review as well as the impact of their findings on reducing project risks and avoiding costs.

however……

Page 33: 3rd party independant reviews

Improvements can be made (-)• Central assurance arrangements :

• were inadequate

• can be effective in isolation, but are more valuable when designed and operated as an integrated system.

• A good assurance system should highlight any breach of time, cost and quality control limits (as agreed when approving the business case) and trigger appropriate intervention.

•Improved visibility of project performance, tracked at the portfolio level, should lead to improved decisions across projects and departments.

•Assurance should transfer lessons across the portfolio.

Page 34: 3rd party independant reviews

effective integrated assurance systems

Four core components :

1. Plan assurance requirements across the portfolio.

2. Perform assurance reviews.

3. Report the findings and lessons.

4. Control and improve performance by using information in the system.

Page 35: 3rd party independant reviews

SO what was missing

•Focus on individual project success and not on the portfolio

•Reliance on ‘the project’ to rectify their issues and not that of the portfolio

Page 36: 3rd party independant reviews

Responses and Actions going forward

• centralisation and coordinated • “Major Projects Authorities”

• stricter approvals – new projects not considered unless assurance processes have been followed.

• Increased governance and oversight:

– actively reviewing the top x projects.

– databases of the riskiest projects/ programmes (technical risk, lifetime spend and priority); and

– increased powers to stop or re-scope failing projects, or additional help to boost management capability.

Page 37: 3rd party independant reviews

Summary (1 more slide to go)

• A number of benefits have emerged from implementing gate reviews

-Increased assurance that the expected service delivery outcomes will result in budget/costs are more likely to be achieved

-Project Sponsor and Senior Responsible Owners are important

-Lessons learnt systematically captured and analysed to track hotspots/ weaknesses

• Integrated, systematic and holistic management approaches are required to deliver high risk projects

-time and cost no longer provide decision makers with adequate decision evidence

-In modern project management, the importance of multi dimensional performance is being realised

Page 38: 3rd party independant reviews

Emerging issues

1. Convergence of peer reviews vs performance audit

- points of difference and ‘value’ offered by each

-assurance : documentation and interviews = assurance snapshot

-audit : standards and performance + ‘deep dives’ = governance check

2. Capital + Sustaining Infrastructure

- time and cost yardstick

- functionality and fit-for-purpose

- definitions of success + horizons changing

Page 39: 3rd party independant reviews