large public sector projects – what determines failure or success? scrum gathering atlanta 2012
TRANSCRIPT
25.09.2014 • © PROMIS AS 1
Large public sector projects
– what determines failure or success?Remi Hansen, PROMIS AS
25.09.2014 • © PROMIS AS 2
Benefits from attending
• Learn what factors are important for succeeding with large government Scrum
projects and what pitfalls to look out for.
25.09.2014 • © PROMIS AS 3
The Presenter – Remi Hansen
• B.Sc. In SW Engineering, M. Sc. in Industrial Economics
• More than 20 years experience from
the IT Consulting business
• A few years as a system developer
• Primarily project manager and advisor / Business Consultant
on strategic projects in both private and public sector
• Some years as line manager – incl. Head of the Project Management Community in
one of Scandinavia’s largest IT consultancy groups, with more than 150 PMs with
extensive use of Scrum in their projects
• Certified Project Management Professional (PMP), PRINCE2 Practitioner,
IT Project Professional (ITPP), CSPO, ISTQB SW Testing Foundation and ITIL.
• Experience with agile projects started in 2002.
• Currently Senior PM in PROMIS, a leading provider of agile project
management services in Norway (www.promis.no)
25.09.2014 • © PROMIS AS 4
Background
• The Nordic model – mixed market economy, welfare states
• “It is distinguished from other welfare states with similar goals by its emphasis
on maximizing labor force participation, promoting gender equality, egalitarian
and extensive benefit levels, large magnitude of redistribution, and liberal use
of expansionary fiscal policy.” (Wikipedia)
• Public sector > 45 % of GNP
• Huge public sector spending large and complex IT systems
• Our fair share of bad projects
• Agile (Scrum) has become the standard approach for public sector projects
• Same motivation as in private sector – a fixed and stable requirements specification
will not give the desired outcome
25.09.2014 • © PROMIS AS 5
Agenda
Introduction 10 min
• Brief presentation of the three projects: 10 min
• Six main recommendations: 60 min
• Lessons learned summary: 10 min
• Q&A
25.09.2014 • © PROMIS AS 7
Case study
• Lessons learned from 3 federal
government projects
• Over 1,000,000 hours of effort
• Not a scientific study
• 1st and 2nd hand information used
• Factual information is from public sources,
the rest is partly my personal opinion and
partly from sources listed
Photo (Flickr):
Okko Pyykkö
25.09.2014 • © PROMIS AS 8
Norwegian Public Service Pension Fund (SPK)
PERFORM project
• SPK
• Administers pension entitlements of $ 59 billion on behalf of the Norwegian state
for almost 1 million former and existing employees in the public sector
• $ 3,2 billion in pension payments (2009)
• Purpose / benefit of the project
• The pension reform
• Outdated technology platform
• New benefit type introduced
• Status: Just completed – very successful, seen as best practice in agile
implementation
• Size:
• 13 teams, 2+1 main suppliers, 175 people
• 300 epics 2500 user stories
• 12 releases over 4 years
• $ 175 Million
25.09.2014 • © PROMIS AS 9
Norwegian Public Service Pension Fund
PERFORM project
• Interesting feature
• Huge project – needed to start development before legislation was passed
• Contract and sourcing setup
• Multisourced
• T & M on Solution Description phase – the customer retained responsibility for
analysis and conceptual design
• Suppliers responsible for construction of their deliveries
• Each delivery (release) negotiated with a separate target price – i.e. target price on
user story level
• Target price 50 / 50 (no cap)
25.09.2014 • © PROMIS AS 10
Statnett
LARM Project
Statnett
• State Enterprise responsible for all high
voltage electricity transmission
and distribution in Norway
• Owns, operates and regulates
the national main grid
• Statnett co-ordinates supply and demand, and owns the main Norwegian power grid.
Photo: Statnett.no
25.09.2014 • © PROMIS AS 11
Statnett
LARM Project
LARM
• Community-critical system for ensuring the responsiveness of
the national power grid
• Status:
• Ongoing (midway)
• Challenged (overruns and conflicting interests)
• Size:
• 2 mid-sized Scrum teams initially – gradually increased staffing
• $ 20 million (supplier cost only), 3 year project
• Interesting feature
• Very specialized domain – few users, but community critical operation
• Contract and sourcing setup
• Target price with customer friendly profile (very close to fixed price contract)
• All activities and phases included in target price
• Price commitment on total scope Photo: Statnett.no
25.09.2014 • © PROMIS AS 12
The Norwegian Public Roads Administration
AUTOSYS project
The NPRA
• Responsible for the planning, construction and operation of the national and
county road networks, vehicle inspection and requirements, driver training and
licensing
Autosys
• Replacing 30 year old mainframe system
• More self-service, fewer manual processes
• 20 000 users, 100 million transactions / year
8 million vehicles and 3,5 million driver licenses
• 5 releases in 3 years (overall 5 year project)
• SOA platform
• Status: Ongoing, challenged
• Size:
• 65 on supplier side, 35 on customer side
• 4 Scrum teams
• $ 100-150 million project budget ($70 M supplier cost)
Photo: Vegvesen.no
25.09.2014 • © PROMIS AS 13
The Norwegian Public Roads Administration
AUTOSYS project
• Interesting feature
• Previous attempt to run the project terminated – generally seen as a failure outside
the NPRA (called a “scandal” by the press)
• High visibility to the public
• Contract and sourcing setup
• Target price with customer friendly profile (close to fixed price contract)
• All activities and phases included in target price
• Price commitment on total scope
25.09.2014 • © PROMIS AS 14
Case study projects summarized
• Large to very large projects in public sector
• Very different domains and parts of the state administration
• All Scrum based projects
• One successfully completed, two ongoing challenged
• All rely on IT suppliers governed by contractual obligations
• All target price contracts (PS2000 based), but with very different customer /
supplier balance
25.09.2014 • © PROMIS AS 15
Agenda
Introduction 10 min
Brief presentation of the three projects: 10 min
• Six main recommendations: 60 min
• Lessons learned summary: 10 min
• Q&A
25.09.2014 • © PROMIS AS 16
Main recommendation areas
Six main recommendations:
1. Enterprise scale Scrum implementation
2. Soft skills and cultural aspects
3. Business value and product backlog issues
4. Integration of test and development tasks in sprint teams
5. Decision making and team interaction
6. Specific public sector challenges and contractual implications
25.09.2014 • © PROMIS AS 18
Findings
Enterprise scale Scrum implementation
What is needed to run a large Scrum project?
Wanting to be agile, but also maintaining predictability and control – different levels
of agility seen
Different approaches to scaling the projects’ Scrum implementation regarding
a) Organization, roles and responsibilities
b) Processes
c) Planning and controlling
d) Testing and approval
25.09.2014 • © PROMIS AS 19
Findings
Enterprise scale Scrum implementation
Contracts
• Add complexity – A large supporting organization is generally not need with two
teams, but a large contract requires administrative overhead
• Customers (also in private sector) want supplier commitment to cost /
productivity / buy-in
• A large project under a contract is different from just adding some extra Scrum
teams to your in-house Scrum development
Revisited later!
25.09.2014 • © PROMIS AS 20
Recommendations
a) Organization, roles and responsibilities
• Everybody cannot overview every aspect of the project when they get large –
division of responsibility needed
More people narrower field of responsibility and more interfaces to other
roles bigger chance of miscommunication
• More roles added needs clearly defined responsibilities
25.09.2014 • © PROMIS AS 21
Scrum Team
Scrum master
Subject Matter
Expert (Cust.)
Test specialist
(Cust.)
Technical
ArchitectDeveloper / tester
Developer / tester
Developer / tester
Developer / tester
Developer / tester
Scrum team
25.09.2014 • © PROMIS AS 22
Example organizationJoint
Steering Committee
Customer’s
Programme
Director
Supplier
Programme
Manager
Customer
Deliverables and
testing
Sub-project
Development
Scrum
team 4
Scrum
team 2
Scrum
team 1
Scrum
team 3
Supplier
Project Owner
Test
Manager
Technical
Architect
Usability
experts
Solution
Architect
Config. Mgr.
Tech. support
Test specialist
Technical tests
Test specialist
Functional tests
Training & impl.
Co-ordinator
Oracle Enterprise
Architect
Business Process
Expert
Supplier Internal SC
Customer
Internal SC
Customer
Implementation
project
Maintenance
coordinator
(from Delivery 1)
25.09.2014 • © PROMIS AS 23
Recommendations
a) Organization, roles and responsibilities
Some relevant roles
• Project management – large financial impact, …
• Product team – instead of single Product Owner (covered later)
• Architecture / technical environments team
25.09.2014 • © PROMIS AS 24
Recommendations
b) Processes
• More defined processes are necessary in large projects
• More roles and people, more fine grained division of work
• Not the level of documentation that counts,
it’s the level of common understanding!
Photo (Flickr):
IvanWalsh.com Photo (Flickr):
Samuel Mann
25.09.2014 • © PROMIS AS 25
Recommendations
c) Planning and controlling
• A release master plan (solution roadmap) – not just a product backlog
• Control gates: Book keeping for each sprint (earned value calculations etc) –
covered later
25.09.2014 • © PROMIS AS 26
Recommendations
c) Planning and controlling – Release Master Plan
• The Release Master Plan is a tool for planning handover and implementation of
new functionality
• A release demands capacity in the organization – the must be prepared
• The Release Master Plan is a tool for the project team and forms the foundation for
all sprints together to form a predictable and acceptable delivery
• The Release Master Plan identifies the number of releases, their size and
timing
• Orchestrates deliveries and pinpoints internal and external dependencies
• Shows milestones and phases toward go-live
• Basis for budgeting
• Guiding principle: Realization of business value!
25.09.2014 • © PROMIS AS 27
Recommendations
c) Planning and controlling – Release Master Plan
Content:
• Functional target
• What epics (functional and technical) are included?
• High level priorities
• Team capacities
• The number and duration of sprints
• Planned dates for sprint demos, control gates, start and finish for acceptance
test, go-live
25.09.2014 • © PROMIS AS 28
Recommendations
d) Testing and approval
• Control gates following each sprint
• Dedicated test phase (addressed later)
25.09.2014 • © PROMIS AS 29
Recommendations
d) Testing and approval – Control gate at sprint completion
• To check if the user stories meet Definition of done, the project performs a
«control gate»
• Administrative «tail» - aftermath of one sprint spills over into the next sprint –
Should be kept as short as possible to avoid discontinuity
• With some training, this may be performed in 2-3 working days following the
sprint demo
• During these days, it is ‘all customer representatives to the pumps’
• Functional verification by Product Owner and his / her team
• Checking code quality
• Checking architectural guidelines
• Checking test documentation
• Checking other documentation
• Contractual milestone
25.09.2014 • © PROMIS AS 30
The Anatomy of the Sprint
Sprint demo Sprint X
Sprint planning Sprint X+1 starts with decomposition
Sprint backlog for Sprint X+1 ready
Control gate:
1. Evaluation of Sprint X
2. Approval of sprint plan for Sprint X+1
3. Updating the risk matrix
4. Change control
5. Repatriation of agreed outstanding issues to the
product backlog
25.09.2014 • © PROMIS AS 31
Construction phase: The control gates
Blue line: Sprint team
Red line: Product Owner
Yellow line: Verification, Control gate and Definition of Done
Construction phaseSolution description
phase
25.09.2014 • © PROMIS AS 32
Planning and controlling considerations in CG
What was the productivity of the last sprint?
How was the product quality?
How is the risk picture evolving?
What can we improve – emphasis on learning and continous improvements
25.09.2014 • © PROMIS AS 33
Recommendations
d) Testing and approval – Commercial reconciliation
• Payments are sometimes linked to progress in development
• Our clear recommendation is to avoid commercial conditions linked to the
Control Gates – commercial reconciliation should wait until Acceptance testing
is completed
Photo (Flickr):
Images_of_Money
25.09.2014 • © PROMIS AS 34
Enterprise scale Scrum implementation
Key Learning Points
Size and contractual obligations drives the need for a
supporting project organization
Keep organization and processes on a practical level
to maintain agility
Use a Release Master Plan for planning and
communication
Execute a Control Gate process after each sprint for
control (progress / velocity, productivity, risk, etc.)
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 36
Findings
Soft skills and cultural aspects
• The projects differed significantly in their ability to build a common project
culture
• Contract parties not bonding together is very counterproductive – more focus
on handovers than the actual work to be done
• Project initiation takes time and should be given sufficient time
• Depending on both customer and supplier – T&M pricing is a good idea in this
phase, to ensure enough time and effort is spent on initiation
• Different cultures collided – washed away the basis for mutual understanding
and trust impeding effective interaction
• Customers adapting bad practice based on lack of experience
– Knowing when to seek external help is very useful
• Lacking the courage to take decisions severely hampers progress
25.09.2014 • © PROMIS AS 37
A clash of organizational cultures
Stereotypical public sector officer
behavior
Anticipated contribution in an agile
system development project
Work according to regulations Design future work processes based on
business value
Working with individual cases See different types of processes in
relation to simplify and standardize
Concrete work patterns "as is" – few
thoughts about future processes
Conceptual modeling of future processes –
limited knowledge of existing processes
Exact rules and all exceptions Good enough and main processes first
Collective / Agency Individual / user
Long-term production perspective Project delivery now
Don’t forget: What we expect from customer representatives
may be very different from what they do on a daily basis
– different jobs attract different people!
25.09.2014 • © PROMIS AS 38
Recommendations
Project culture and soft skills
• Invest sufficient time in creating a common project culture - focused on creating
business value
• Customer representatives in the Scrum teams is a good start!
• Take into account that cultural mismatches affect productivity
25.09.2014 • © PROMIS AS 39
Recommendations
Avoiding bad practice
• Contracts emphasis the supplier vs. customer obligations, less on the project
as one group
Avoid contractual thinking from team members on all levels – confine that to
the management level
• Appoint managers who are not afraid to take decisions
• Make sure to get external help to promote better practice – a good way to
resolve conflicts
• Use advisors to assist in resolution, not for gathering ammunition for a possible
blame game
25.09.2014 • © PROMIS AS 40
Soft skills and cultural aspects
Key Learning Points
Organizational culture aspects may influence your
project’s ability to succeed – act accordingly!
Scrum is relying on effective decision making – create
a culture to promote that
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 42
Findings
Business value and product backlog issues
• Poor quality of the product backlog is devastating for the project
• Too large user stories impossible to remove anything from the scope
(every user story contains something critical)
• Poorly defined user stories many discussions late completion of user story
limited time for testing
• Ambiguities low productivity and many assumptions (risk)
• Decision making: real (and lasting!) prioritization is difficult for inexperienced
P.O.s
• A lot of accumulated needs in the organization
• A tendency to behave like kids in a candy store, if there are no mechanisms to
control scope
• Responsibilities for different aspects of the product backlog is a source of
misunderstanding and controversy
25.09.2014 • © PROMIS AS 43
Findings
The vicious circle of poor backlog quality
Poor quality of the sprint backlog (capacity and/or expertise)
Abundant clarifications needed - takes more time from the P.O. and Scrum
teams
Late clarifications late completion of user stories
Poor tests and low productivity (developers losing focus, multitasking unclear
tasks)
More effort needed from both the customer and the supplier into the next sprint
to sort out whether user stories are “done”
Prolonged Control Gate period
Reduced opportunity for P.O. team to contribute to the planning and launching
of the next sprint (busy doing backward-facing activities) – introducing
uncertainty in the sprint plan
Lack of commitment from the teams – not possible to deliver the sprint contract
anyway, because of the poor sprint backlog quality
25.09.2014 • © PROMIS AS 44
Findings
Business value and product backlog issues
• The P.O. team did not work effectively – the Chief P.O. had to take all
decisions
• The customer did not meet their obligations regarding Business Analysts and
Test Specialist – partly having the wrong profiles
• The resulting lack of clarifications and scoping (acceptance criteria) gave extra
load on the P.O.
25.09.2014 • © PROMIS AS 45
Recommendations
Good product backlog practice
• The customer must control the scope!
• A well organized P.O. Team supporting the Chief P.O. is needed
• Individual P.O.s must co-ordinate across domains – creating one coherent
backlog
• Appoint a Business Analyst to each sprint team = P.O.’s delegates to the
teams will relieve the load on the P.O.
– Preferably Customer Business Rep., but in a long project even a hired
Business Analysts can add value
25.09.2014 • © PROMIS AS 46
Recommendations
Good product backlog practice
• Continuously work with grooming the backlog – invest enough capacity to
clearly define User stories, and give priority
• Setting priorities takes a lot more effort than one should expect – working
towards the line organization and avoiding rematches
• Put in place a method for assessing business value – to avoid somebody’s pet
requirements sneak past prioritization
• View the User story estimate as the budget for the task – don’t estimate the
task again with high level of freedom and creativity
• Challenge the P.O. to find a simpler solution if the scope doesn’t match the estimate
25.09.2014 • © PROMIS AS 47
Product owner team
Development team 1 Development team 2 Development team 3
Product owner team
Product owner
25.09.2014 • © PROMIS AS 48
Business value and product backlog issues
Key Learning Points
• The quality of the backlog cannot be stressed enough –
the project success completely relies on it!
Ensure enough effort is spent grooming the backlog
• Use business value as basis for prioritization
• Set up a P.O. team as described and appoint a
Business Analyst to each scrum team
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 50
Findings
Integration of test and development tasks
• Different system test approaches taken – integration of test into development
seems to give the best result
• The customer did not participate with test specialist in the Scrum teams as
dictated by contract
• Absence of testing capacity in the teams was compensated with extra effort
from the test team
• Role confusion (became part of the development team and not a gatekeeper to dev.)
• Capacity problems in the Test team not well prepared for system test
• Poor testing in the sprints the customer had to test more in Control Gate
(after sprint demo)
• Delayed CG and sprint plan for next sprint
• More book keeping and duplication of tests at the expense of other important work
• Aftermath stealing attention from planning
• Test data issues
25.09.2014 • © PROMIS AS 51
System Hardening sprints
• 2-3 sprints after the development sprints to perform system test and complete
outstanding work (typically documentation and clean-up) – done in parallel with
solution description for the next delivery
• Is that a good idea?
There’s always some activities not completed. May give better preparation for and
smoother execution of Acceptance test
Real integration test will not be done along the way becomes a waterfall
approach, where testing is postponed (long time from programming to testing)
Risk mitigation is delayed
Very demanding for the project to do both system testing and solution design in
parallel
Probably not a good idea – include system testing in the sprints instead to
minimize time from programming to testing
25.09.2014 • © PROMIS AS 52
Recommendations
Integration of test and development tasks
• The Test Manager should state quality requirements to development teams –
i.e. Definition of Done related to testing
• The Test Manager should have the authority to label user stories «Done» - i.e.
accepting what user stories will be demonstrated at sprint completion
• The Test Manager must have the motivation to withhold user stories not ready
for demo
• Allocate time in the coming sprint to do system tests of previous sprints delivery
– rather than waiting and doing all system testing in the end (hardening phase)
25.09.2014 • © PROMIS AS 53
Recommendations
Integration of test and development tasks
• There should be very limited need for the customer to perform separate tests –
rather verifying the test activities undertaken by the supplier
• In traditional waterfall environments «acceptance» is a very important decision
– such customer’s need reassurance that approving a user story does not
mean they have accepted the quality of the system (just that this part of the job
is done and in accordance with expectations) – i.e. commitment from the
customer on the content
25.09.2014 • © PROMIS AS 54
Recommendations
Integration of test and development tasks
• Integrate test into Scrum teams
• Better flow and continuity
• Close cooperation between business, developers and testers
• Appoint one in each Scrum team as Team Test Responsible
• No formal responsibilty for the product quality (still team responsibilty)
• Assignment to keep a close look at the testing activities and remind the others to do
their testing
• Planning, design, test conditions, development
and test performed collectively by the team
• Testers are no longer quality police
– instead valued members of the team
• Unit tests, integration tests, functional system
test and system integration tests
in construction sprints
25.09.2014 • © PROMIS AS 55
Test in agile development
• Tests are run continuously through the period
• Acceptance criteria written in parallel with
description of needs and solution design
• Tests written in advance of programing and
run in the sprints for completed user stories
• What can be approved, should be approved
progressively (in Control Gates)
25.09.2014 • © PROMIS AS 56
Integration of test and development tasks in the sprint teams
Key Learning Points
Maximize the testing done as part of the development
sprints
Appoint a test responsible in each sprint team
Empower the Test Manager to set process
requirements
Avoid pointless replication of tests on the Customer
side
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 58
Findings
Decision making and team interaction
• It takes courage to manage an agile project – No collective decision process to
hide behind
• E.g. as Product Owner you put your head on the block – will public sector employees
feel they can gain benefits making that worthwhile?
• Demanding to say “we don’t need that documented”
• Blame game mindset introduce waste and is self-perpetuating – fight it!
• The relentless will to learn and improve is not evident in every project – those
who learn fast seems to succeed
25.09.2014 • © PROMIS AS 59
Findings
Decision making and team interaction
• With larger projects the teams need guidelines – don’t leave it to the teams to
sort out how the project should work
• Delegate the task, but make a central decision
• All projects used self-organizing teams – with success
• Able to manage their own work and maintain good motivation
• Retrospectives worked well in all projects
• The teams used a wide range of techniques in the retrospectives – the process
worked well
• Important to summarize recommendations and actions across teams (incl. the
customer)
• With slippage and scope challenges the teams started to take shortcuts
• Measurement primarily on velocity – pressure to take on more new functionality
• Started building technical debt and accumulated risk
• Stopped taking responsibility for cross-team issues
25.09.2014 • © PROMIS AS 60
Recommendations
Decision making and team interaction
• Defer decisions
• Let those best qualified take the
decision – create a culture for
delegation and trust
• Use “mini demo” to reduce risk
• Introduce friendly competition between
the teams
Photo (Flickr):
Budzlife
25.09.2014 • © PROMIS AS 61
Recommendations
Communication boosters
• Separate daily Scrums are not sufficient
for coordination
• Scrum of Scrums
• Meta Scrum
• «Town hall meetings»
• Communities of practice
• Use visualization actively
to communicate concepts, plans,
and progress – the master release plan
is a good starting point
Photo (Flickr):
jardenberg
25.09.2014 • © PROMIS AS 62
Decision making and team interaction
Key Learning Points
Key stakeholders must be courageous - Strive for a
culture based on trust
Stress the importance of retrospectives and pursue
improvements
Use communication boosters to ensure everybody is
on the same page
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 64
Findings
Specific public sector challenges and contractual implications
• Public sector is engineered for cost control – «must» have a contract with
supplier obligations (a T&M project is difficult to fund in public sector budget
processes)
• Little personal benefit from taking risk and accountability
• No culture for accountability for project benefits – without financial
measurements the planned benefits may be too vague
• Two of the projects seem to be systematically underestimated – this
undermines the target price mechanism
• Sticking to unrealistic estimates is counterproductive – the supplier will not deliver as
good as it can and the customer will probably not get a very good system
• Two of the projects included the initiation phase in the target price, with
penalties on “initiation complete” milestone. The supplier thus defined itself as
finished on time, even if the delivery was not complete. That was paid dearly in
later stages.
25.09.2014 • © PROMIS AS 65
Findings
Specific public sector challenges and contractual implications
• In PERFORM the customer chose a balanced contract and took responsibility
for scope control
• Contract assignment per supplier per delivery at the end of each Solution
Description phase
• The challenged projects were more focused on cost control than having a
balanced contract
• Close to fixed price (cap on risk sharing)
• Including initiation and solution description in the fixed price
• Commitment on price for the complete scope, without any means of adjustment to
actual productivity
• This seemed to also reduce the customer’s focus on scope control – defining
that as supplier responsibility
• When overruns appeared (effectively making the price model = fixed price) we
saw a “culture of blame”
25.09.2014 • © PROMIS AS 66
Contracting price models:
• Time&Material:
• The Customer pays for all worked hours, disregarding
productivity and quality – all risk is on the Customer
• Fixed Price:
• The Customer pays the same amount, disregarding the hours
needed to produce the scope with agreed quality – all risk is
on the Supplier
• Target Price:
• Customer and Supplier share benefits or losses in an agreed
ratio – if this ratio is 50%, the risk is evenly distributed
between the parties
25.09.2014 • © PROMIS AS 68
What price model is beneficiary in agile projects?
Target price is best suited because:
• Agile is a close and mutual co-operation
between customer and supplier
• Target price with 50/50 sharing of
incentives and sanctions ensure the
customer and supplier works towards the
same goal
• Target price is suitable when requirements
are not very detailed
• Target price is suitable for projects with risk
• Target price can be based on a less
detailed basis than traditional fixed price
Price model Average overrun
Time & Material based
(4 projects)
55%
Fixed price (5 projects) 33%
Target price (7 projects) 10%
Others (2 projects) 13%
Total (18 projects) 27%
• Fixed price assumes
requirements are described in
detail and that uncertainty is
low.
• T & M is suitable when scope is
not well defined and uncertainty
is high.
25.09.2014 • © PROMIS AS 69
Why Target Price Contracts with Risk Sharing?
• The Supplier is invited to commit to estimates with a higher degree of
uncertainty than in traditional fixed price contracts
• This is paramount in agile system development projects, where one of the main
principles is to avoid waste
• In particular, effort can be reduced on initial specification, detailed design and
planning
• The Customer may produce tender documents with high level requirements
• The Suppliers produce solution descriptions, estimated with a significant
degree of uncertainty
• In this way, both parties reduce work load in the initial bidding
phases of the project
25.09.2014 • © PROMIS AS 70
Recommendations
Specific public sector challenges and contractual implications
A balanced contract is necessary to ensure commitment and quality
• Implies 50 / 50 risk sharing with no cap
• Not agile to commit to up-front estimates for several years based on very
limited knowledge – risk for the supplier and ultimately the customer
• Include mechanisms to avoid commitment to estimates over a very long time
• Negotiating target price per delivery (release)
• Other mechanisms to adjust to actual velocity and productivity
• Consider T&M for the initiation phase, to ensure no corners are cut
25.09.2014 • © PROMIS AS 71
PS2000 – a Target Price Contracting standard
• A Norwegian contracting standard in the IT industry
• A result of a research program during the years 1997 – 2000
• By 2012, used in a number of large system development projects in the
Norwegian public sector
• Also used in private sector and internationally
• Referred to in Mary and Tom Poppendieck:
’Implementing Lean Software Development:
From Concept to Cash’ Addison-Wesley 2007
25.09.2014 • © PROMIS AS 72
PS2000 Standard contract - English version
http://www.dataforeningen.no/it-contract-standards.146223.no.html
Requirement
Analysis Acceptance and -
Completion phase
Detailed planning Analysis and design
Testing Development
Progression
Iterative
construction phase
CG1
CGn C
CG2
PMS 0 Contract Signing
Solution
Description
PMS 1 Approved Solution
Description
PMS 2 Delivery ready
for Acceptance
PMS 3 Accepted
Delivery
25.09.2014 • © PROMIS AS 73
In summary - Advantages with PS2000 Agile
• Corresponding to the principles in Agile in a better way than alternative
contracting models
• Not unlike agreements to deliver competence and capacity (as in
Time&Material contracts), but framing in the scope, time, cost and risk
• Suitable for repeating releases in a frame agreement for new development or
application maintenance
• An incentive for the supplier to deliver more functionality within agreed time
limits, but with satisfactory quality
• The main success factor is that the parties agree on the process manging the
Product Backlog, Sprints and Control Gates, with the corresponding roles
• Change control becomes lean – a signed Product Backlog after each sprint
documents the agreed changes
25.09.2014 • © PROMIS AS 74
Specific public sector challenges and contractual implications
Key Learning Points
Agile projects can be run under a contract, given that
the contract
Is balanced
Is engineered for agile execution
Lets scope be defined as late as possible
(preferably contracting each release as a separate target
price delivery)
PS2000 Agile is such a contract standard
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 75
Agenda
Introduction 10 min
Brief presentation of the three projects: 10 min
Six main recommendations: 60 min
• Lessons learned summary: 10 min
• Q&A
25.09.2014 • © PROMIS AS 77
Summarizing some major issues
• Impression that all the projects would be agile – only one of them were
• Contractual constraints (~ fixed price)
• Cultural challenges – still a fixed price mindset on the customer side
• Customer not participating as actively as anticipated – both staffing and decision
making
Scrum does not necessarily make a project agile!
• Truly agile execution is possible within a contract constraint, with the right
contract
• Naive to expect Scrum / agile to solve every problem
• Even the most successful project had challenges in the first stage – but a
willingness to learn & adapt brought the project on the path of good practice
25.09.2014 • © PROMIS AS 78
Some smart moves in the PERFORM project
paving the way for success
• Gained experience with Scrum before starting the project
• Understood the value of a balanced contract
• Initiation phase on T&M price model
• Had own Scrum teams in parallel with
supplier teams
• Invested in good procedures and tools for migration
and configuration control
• Test integrated part of development
• Hired external help for agile project management
coupled with active top management involvement
• The customer took overall responsibility for all processes – the main suppliers
had responsibility for construction of their parts for every release
• A continuous focus on improvement
Truly agile execution!
Photo (Flickr):
apdk
25.09.2014 • © PROMIS AS 79
8 defining features of agile projects
1. Business value is the most important criteria for quality and direction
2. Continuous prioritization of functionality based on cost / benefit
3. Close communication between the business side and developers
4. Short iterations to delivery of executable code
5. Frequent releases to production (integrated and tested)
6. Binding decisions taken as late as possible (‘Rolling Wave Planning’)
7. Evaluation, learning and improvement along the way
8. Autonomy: Self-organizing cross-functional teams
25.09.2014 • © PROMIS AS 80
Defining features of agile projects
Feature Degree of
divergence in case
study
Covariance
between good agile
practice and
success
Business value focus High
Continuous prioritization High
Communication with business Medium-High
Short iteration – executable code Low
Frequent releases Low
Deferring decisions High
Learning Medium-High
Autonomy Low
25.09.2014 • © PROMIS AS 81
Other reflections
• Using agile methodology does not replace good craftsmanship in software
engineering
• Emphasis on architecture
• A good process to establish the product backlog
• Best practice in conceptual design / modelling and code standards
• Configuration control and build (CI)
• Estimation
• Test strategy and plans
• Agile projects facing non-agile environments (e.g. interfacing systems with
decided release plans for the coming year) is high risk – must be solved
through governance
25.09.2014 • © PROMIS AS 82
Public sector specifics?
• Wide variety of organizations, management and culture in public sector – not
possible to see the whole sector as one homogeneous group
• The recommendations given here applies equally to private sector projects
25.09.2014 • © PROMIS AS 83
References
• IT Project Professional certification curriculum - http://smidigeprosjekter.no/itpp
• Upcoming book – working title ‘Agile Contracting’
• PS2000 Standard Contract
• The Norwegian Computer Society offer a complete set of standard contracts for
software development and maintenance and management.
• The PS2000 Standard Contract (standard version) is based on iterative
development. An agile version exists with alternative set of annexes, expanded and
adjusted according to agile methodology. The terminology is mainly based on Scrum
• http://www.dataforeningen.no/it-contract-standards.146223.no.html
• Complete english version - PS2000 Agile (PS2000 Agile, Maintenance, Framework,
Service Operations) ~ 1000 USD
• PRINCE2