end-to-end deployment pipelines - avoiding pipeline fragmentation
DESCRIPTION
The deployment pipeline is a key feature of continuous delivery. Avoiding fragmentation of deployment pipelines is crucial if the maximum benefit is to be derived from pipelines. This presentation shows why pipeline fragmentation is bad, examines some of the reasons why pipeline fragmentation may occur, and explores some of the challenges involved in implementing end-to-end deployment pipelines.TRANSCRIPT
End-to-end Deployment Pipelines:
Avoiding Pipeline Fragmentation
Matthew Skelton :: 13 June 2012
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
2End-to-end Deployment Pipelines :: Matthew Skelton
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
3End-to-end Deployment Pipelines :: Matthew Skelton
Continuous Delivery
End-to-end Deployment Pipelines :: Matthew Skelton4
Empowering product owners & the business to
choose when to deploy
Software always ready to go live
„One-click‟ deployments
Visibility of work-in-progress
Continuous Delivery
End-to-end Deployment Pipelines :: Matthew Skelton5
Context
http://continuousdelivery.com/
Jez Humble & David Farley
(ex-)ThoughtWorks
Chapter 5, Anatomy of the Deployment Pipeline, is free to download:
http://www.informit.com/articles/article.aspx?p=1621865
Read the book!
+ screen casts on InfoQ
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
6End-to-end Deployment Pipelines :: Matthew Skelton
Value Stream Maps
End-to-end Deployment Pipelines :: Matthew Skelton7
Identify stages in the release cycle
„Value-added‟ time
Elapsed time
Remove bottlenecks, eliminate waste
Further reading:
Lean Software Development, Mary & Tom Poppendieck
“How long would it take your organization to deploy a change that involves just one single line of code?”
Value Stream Maps
End-to-end Deployment Pipelines :: Matthew Skelton8
Example (from Continuous Delivery)
Where is this visible within the organisation?
Is the information up-to-date?
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
9End-to-end Deployment Pipelines :: Matthew Skelton
What are deployment pipelines?
End-to-end Deployment Pipelines :: Matthew Skelton10
Automated manifestations of value stream maps
The Deployment Pipeline tells us...
End-to-end Deployment Pipelines :: Matthew Skelton11
At what stage is version X of the software?
Which version is in environment Y?
Which versions made it to Production?
What version should I test next?
...and so on.
The Deployment Pipeline tells us...
End-to-end Deployment Pipelines :: Matthew Skelton12
At what stage is version X of the software?
Which version is in environment Y?
Which versions made it to Production?
What version should I test next?
The Deployment Pipeline
End-to-end Deployment Pipelines :: Matthew Skelton13
http://www.thoughtworks-studios.com/go-agile-release-management/features-benefits
GO from ThoughtWorks Studios:
The Deployment Pipeline
End-to-end Deployment Pipelines :: Matthew Skelton14
http://www.thoughtworks-studios.com/go-agile-release-management/features-benefits
At a glance:
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
15End-to-end Deployment Pipelines :: Matthew Skelton
Pipeline Fragmentation
End-to-end Deployment Pipelines :: Matthew Skelton16
Deployment pipeline is broken into small chunks
End-to-end view is missing
http://blog.mikeobrien.net/2010/09/setting-up-thoughtworks-go-build-on.html
?
Pipeline Fragmentation
End-to-end Pipeline Fragmented Pipeline
End-to-end Deployment Pipelines :: Matthew Skelton17
From Dev to Production Multiple separate isolated steps
Pipeline Fragmentation
End-to-end Deployment Pipelines :: Matthew Skelton18
Questions (again): At what stage is version X of the software?
Which version is in environment Y?
Which versions made it to Production?
What version should I test next? ?
Problems with Pipeline Fragmentation
End-to-end Deployment Pipelines :: Matthew Skelton19
Difficult to get the overall picture
Multiple places to look
Limited traceability
Difficult to answer the common questions: At what stage is version X of the software?
Which version is in environment Y?
Which versions made it to Production?
What version should I test next?
What should be automated becomes manual
Causes of Pipeline Fragmentation
End-to-end Deployment Pipelines :: Matthew Skelton20
Teams and Roles Each team responsible for a single GO server
GO servers organised around responsibilities
Location Teams in different locations
Tricky to achieve inter-site view
Performance Slow-downs when > 200 pipelines
„Misuse‟ of pipelines
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
21End-to-end Deployment Pipelines :: Matthew Skelton
Deriving value from end-to-end pipelines
End-to-end Deployment Pipelines :: Matthew Skelton22
Challenges:
Versioning and dependency management
Separating subsystems can be tricky
A product- or service-centric view makes this easier Goal is one pipeline per product or subsystem
Design of deployment pipelines Triggers, build dependencies, re-using components as test harnesses
Multiple test environments complicate pipelines One environment per product better than one per team?
New „diamond dependencies‟ in GO 12.2 might help here
Optimising the automated test suites is essential
Deriving value from end-to-end pipelines
End-to-end Deployment Pipelines :: Matthew Skelton23
Choose small, simple products to begin with Limited or no dependency on other components
Involve the business sponsor
Aim for a “walking skeleton”, end-to-end Forces some tricky problems to be solved
For example: Change the text on a button
Commit, test, deploy to Production within a day
More complex scenarios can follow later
Deriving value from end-to-end pipelines
End-to-end Deployment Pipelines :: Matthew Skelton24
Answer questions from: Product owners
The Business
QA team
Developers
Visible progress
Take more informed decisions, more rapidly
Reduce cycle time
Introduction
Context: Continuous Delivery
Value stream mapping
What are deployment pipelines?
Pipeline fragmentation
Deriving value from end-to-end pipelines
Q&A
End-to-end Deployment PipelinesA
GE
ND
A
25End-to-end Deployment Pipelines :: Matthew Skelton
