3rd party independant reviews
TRANSCRIPT
The gateway review process
a quite success story of using a quality assurance process to improve the delivery of high risk projects
…….and more generally, some lessons from around the globe
Nick Pelham
But first…..a case study in asset management : project vs operations
The iron triangle of project management
TimeCost
Quality
Sydney Opera House, NSW (1952)
• Time planning began in the 1940s
• The need - “a suitable venue for large theatrical productions”
architectural design competition awarded in 1957 (Jorn Utzon).
construction started in 2 March 1959
Original completion date – 26 Jan 1963
Opened in 20 October 1973• Stage 1 (upper podium –> „59-63)• Stage 2 (roof -> „63-67)
Construction of the Opera HouseV
original estimate – £3.5m (~$7m) (1957)
final cost ~ $98.8 - $102m (1973)
since 1973... the first upgrade - $120m (1988-98) the second upgrade - $ 66.59m (2002)
total construction and refurbishment – (1957-2002)
$374.4m
+ „Venue Improvement Program‟ (2001-09) $69m
You all have your views……
Quality
TimeCost
1400% + 3½yrs ->14+ yrs
• Average cost per year over past 5 years
• $121m pa• Bldg dev and expenses $ 31.5m• Expenses $ 89.5m
• + „Venue Improvement Program‟ (2001-2009) - $69m($7.7m per year)
• More likely average cost per year
$128.7m pa
Running of the Opera HouseV
2011 NSW Audit
SOH needs a $1.1b upgrade to its facilities to remain viable and to address access and safety concerns:
• flawed acoustics• limited accessibility• limited seating and restricted spaces• shortcomings in OHS
But………
Construction
• $102m (1973) $102m
Upgrades (1988-2009)
• Upgrade 1 (1988-1998) $ 120m• Upgrade 2 (2002) $ 66.59m• Venue Improvement Program (2001-2009) $ 69m
$ 255.59m
• Upgrade 3 (2014- 2023) $1,100m
Running the opera house
over next 10 years $1, 326.6m
Summary
So what…….
Rule of thumb
1 - 4 - 20
1. Gate review processes
Origins of the ‘gate’ concept
underlying idea behind Gate Reviews can be traced to research and development in the private sector in the 1980s and is widely attributed to work Dr R. G. Cooper (Canada).
The terms used to describe such reviews include ‘Stage‐Gates’, ‘Gateway Reviews’ etc….
The approach remains strongly identified with product innovation and the introduction of new products to the market by private companies.
In public sector organisations
The Gateway Review Process in Australia began in 2003 and is modelled on the UK’s Office of Government Commerce (OGC) Gateway™ Process that was originally designed to address failures in information technologyprojects
Developed by Sir Peter Gershon in the UK in Feb 2001
Project performance-cost overrun by percentage (based on Flyvbjerg, 2003)
The Gateway Review Process
Gate 1: Strategic Assessment- Confirm business strategy and need
Gate 2: Business Case- Confirm business case is robust
Gate 3: Readiness for Market- Confirm procurement method and sources of supply
Gate 4: Tender Decision- Confirm investment decision
Gate 5: Readiness for Service- Confirm commissioning plans are in line with business strategy and
industry best practice
Gate 6: Benefits Evaluation- Assess project delivery effectiveness
Opportunity to influence project outcomes
Strategic
assessment
Procurement
Strategy
Competitive
Procurement
Award and
implement
contract
Manage Contract/facility
to deliver services
Develop
Business Case
Benefits
realisationConstruction Readiness
for Service
Needs
analysis
Business
justification
Procurement
strategy
Investment
decision
Whole
of Life
Expenditure
0%
100%
Time
What we’ve learnt.........
BENEFIT STRATEGIC
INTERVENTIONSPROBLEM
Improved liveability
around the market
KPI 1: Reduction in CO2 and
noise emissions
KPI 2: Improved health of
residents
Restrict truck
movements to daylight
hours
Remove trucks from
residential streets
Relocate residences
outside market area
Increasing road
freight in residential
streets is causing
declining health in
people who live near
the market
Starting at the start…..defining the Problem
Focus on a Case for the Change (business case)
• No executive summary
• No genuine link to strategic priorities
• Lack of specificity as to the ‘need’ or proposed solution –aka solutions looking for problems
• Lack of clarity of timing, cost and benefits
• Use of diagrams or tables which are unrelated to the text or hard to interpret
• Obvious inconsistency both numerical and with the detail of the business case
Traps for new players or poor past efforts:
The gateway review process ‘adds value’
UK –>
•2008 2-4% cost avoidance *
Australia
• 2009 3-5% cost avoidance
• 2010 22.4% time benefit, 29.3% cost benefit (Uni Melb)
* If all recommendations are implemented as outlined in the gateway review report
Assurance provided from the Review Process
Judging success
• Large projects often best judged with the benefit of hindsight
• Time and cost are useful yardsticks for success- ………. as is function and stakeholder acceptance
• Large projects are inherently delivered in a high risk environment
• Performance of capital projects are largely determined by time and cost
- Remember Sydney Opera House– Project outcomes
– Upgrade costs
Other jurisdictions
Whole of Life analysis – individual projects
Australia
1. Managing complex governance arrangements across agencies and jurisdictions
2. Project inception, approvals and funding
3. Risks associated with implementing projects in tight timeframes
4. Establishing a benefits lead approach
New Zealand
1. Business case
2. Risk and issues management
3. Program/project management
4. Stakeholder management
5. Resourcing
6. Governance
7. Program/Project planning
8. Management of change
9. Sourcing strategies
10. Dependency management
2. Structural deficiencies
‘Reviews’ of the reviews
Audits in the UK and Australia
2010
Audit of the Efficiency and Reform Group’s role in improving public sector value for money
Ombudsman investigation in to ICT enabled projects
Assurance for high risk projects
Assurance for major projects
Gate Reviews for Defence Capital Acquisition Projects
Administration of the Gateway Review Process
Parliamentary enquiries
2011 2012
What was found (+)
+…..a central quality assurance tool which assists in the delivery of high risk and complex projects…..
+…..valued by stakeholders for the benefits that result from preparing for a review as well as the impact of their findings on reducing project risks and avoiding costs.
however……
Improvements can be made (-)• Central assurance arrangements :
• were inadequate
• can be effective in isolation, but are more valuable when designed and operated as an integrated system.
• A good assurance system should highlight any breach of time, cost and quality control limits (as agreed when approving the business case) and trigger appropriate intervention.
•Improved visibility of project performance, tracked at the portfolio level, should lead to improved decisions across projects and departments.
•Assurance should transfer lessons across the portfolio.
effective integrated assurance systems
Four core components :
1. Plan assurance requirements across the portfolio.
2. Perform assurance reviews.
3. Report the findings and lessons.
4. Control and improve performance by using information in the system.
SO what was missing
•Focus on individual project success and not on the portfolio
•Reliance on ‘the project’ to rectify their issues and not that of the portfolio
Responses and Actions going forward
• centralisation and coordinated • “Major Projects Authorities”
• stricter approvals – new projects not considered unless assurance processes have been followed.
• Increased governance and oversight:
– actively reviewing the top x projects.
– databases of the riskiest projects/ programmes (technical risk, lifetime spend and priority); and
– increased powers to stop or re-scope failing projects, or additional help to boost management capability.
Summary (1 more slide to go)
• A number of benefits have emerged from implementing gate reviews
-Increased assurance that the expected service delivery outcomes will result in budget/costs are more likely to be achieved
-Project Sponsor and Senior Responsible Owners are important
-Lessons learnt systematically captured and analysed to track hotspots/ weaknesses
• Integrated, systematic and holistic management approaches are required to deliver high risk projects
-time and cost no longer provide decision makers with adequate decision evidence
-In modern project management, the importance of multi dimensional performance is being realised
Emerging issues
1. Convergence of peer reviews vs performance audit
- points of difference and ‘value’ offered by each
-assurance : documentation and interviews = assurance snapshot
-audit : standards and performance + ‘deep dives’ = governance check
2. Capital + Sustaining Infrastructure
- time and cost yardstick
- functionality and fit-for-purpose
- definitions of success + horizons changing