the data on devops: making the case for awesome
TRANSCRIPT
Nicole Forsgren, PhD
Director of Organizational Performance & Analytics, ChefResearch Affiliate, Clemson University
The Data on DevOps:Making the Case for Awesome
“IT doesn’t matter.”-- Nicholas Carr, 2003
@nicolefv
IT Does Matter
• Times – and IT – have changed• DevOps is good for Organizations• DevOps is good for IT• And then some detail: What drives this change?
• Technical practices (hint: Continuous Delivery)• Management practices (hint: from Lean Management)• Culture and identity
@nicolefv
@nicolefv
@nicolefv
DevOps isTechnical practices
seen in Continuous Delivery,Management practices
seen in Lean Management principles, and Organizational Culture and Identity
@nicolefv
Research shows that these drive IT Performanceand Organizational Performance
DevOps is good for organizations
@nicolefv
High Performing IT organizations
2xMore likely to exceed Profitability, Market share, andProductivity goals
50%Higher market capgrowth over 3 years*
The 2014 and 2015 State of DevOps Report
@nicolefv
Devops is good for IT
Measuring DevOps and IT Performance- Deploy frequency (Note: NOT delivery)- Lead Time for Changes- Mean Time to Recover (MTTR)- Change Fail Rate
@nicolefv
High Performing DevOps teams
More agile
The 2016 State of DevOps Report
@nicolefv
High Performing DevOps teams
More reliable
The 2016 State of DevOps Report
@nicolefv
IT Performance over the years
The 2016 State of DevOps Report
@nicolefv
Deploy Frequency Change Lead Time Mean Time to Recover
DevOps promises – and delivers More throughput
More stability
In tandem. Without the tradeoffs that ITIL calls for.
Let’s talk about what this means for us@nicolefv
High Performing DevOps teams
More agile What does this mean for:200xMore frequent deployments
The 2015 DevOps Survey of Practice and its resulting database are the property of Puppet Labs, Inc. and Gene Kim and Associates, LLC. All rights reserved.
New content deliveryValue/savings around A/B testingValue around speed to marketCompliance / regulatorySecurity2,555x
Faster lead times@nicolefv
Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3were successful at improving the key metric!
Online Experimentation at Microsoft, Kohavi et al http://stanford.io/130uW6X@nicolefv
High Performing DevOps teams
More reliable What does this mean for:3xFewer deployfailures
The 2016 State of DevOps Report
Value/savings around reliabilityValue/savings around uptimeComplianceSecurityReputation around uptime,
compliance & security24xFaster MTTR
@nicolefv
@nicolefv
"Fixing it as soon as possible or having compensating controls in
place days before could have saved this entire breach from occurring in the first place."
We know:
• IT Performance is comprised of throughput and stability, and both are possible without tradeoffs
• IT Performance contributes to org performance ($$$)
So:What drives IT and Organizational Performance?
@nicolefv
IT Performance
Org Performance???
@nicolefv
IT Performance
Org Performance???
1.
@nicolefv
IT Performance
Org Performance???
2. 1.
@nicolefv
IT Performance
Org Performance???
2. $$$1.
@nicolefv
Continuous Delivery makes our work better…
The 2016 State of DevOps Report
@nicolefv
“We never had testability before. We have it now. We have this experience and know this stuff is working, and working with controls.”
– Product Owner for Yahoo Chef implementation
Automated configuration and deployment of 250,000 nodes
Can deploy up to 140k node configurations in 8 hours.
Can patch entire infrastructure within 6 hours of a patch being made available
@nicolefv
Continuous Delivery makes our work better… and makes us feel better!
@nicolefv
https://devops-research.comForsgren, N., J. Humble (2016). "The Role of Continuous Delivery in IT and Organizational Performance." In the Proceedings of the Western Decision Sciences Institute (WDSI) 2016, Las Vegas, NV. Available at SSRN: http://ssrn.com/abstract=2681909
Microsoft Engineering: DevOps LessonsThiago Almeida -- DevOps Days London, 2016
@nicolefv
Work/Life ScoresBefore CD: 38%After CD: 75%
Source: https://vimeo.com/165184757
Continuous Delivery makes our work better…
The 2016 State of DevOps Report
@nicolefv
But what else drives IT Performance?
The 2015 DevOps Survey of Practice and its resulting database are the property of Puppet Labs, Inc. and Gene Kim and Associates, LLC. All rights reserved.
@nicolefv
Lean Management practices
The 2015 State of DevOps Report
@nicolefv
“If it moves, graph it.” - Michael Rembetsy, Vice President Operations, Etsy
@nicolefv
“I was trying to figure out why my team was working themselves to death but not getting anything done… By implementing WIP limits, we were able to focus on our work. Finishing work feels better than sprinting and feeling like a hero in the moment, because that’s only a moment.”
- Julia Wester, Development Manager for Turner Sports, Turner Broadcasting
@nicolefv
Lean Product Management practices
The 2016 State of DevOps Report
@nicolefv
Organizational Culture
@nicolefv
Pathological Power-oriented
BureaucraticRule-oriented
GenerativePerformance-oriented
Low cooperation Modest cooperation High cooperation
Messengers shot Messengers neglected Messengers trained
Responsibilities shirked Narrow responsibilities Risks are shared
Bridging discouraged Bridging tolerated Bridging encouraged
Failure leads to scapegoating
Failure leads to justice Failure leads to inquiry
Novelty crushed Novelty leads to problems
Novelty implemented
• I am glad I chose to work for this organization rather than another company.
• I talk of this organization to my friends as a great company to work for.• I am willing to put in a great deal of effort beyond what is normally
expected to help my organization to be successful.• I find that my values and my organization's values are very similar.• In general, the people employed by my organization are working toward
the same goal.• I feel that my organization cares about me.
Adapted from adapted from Atreyi Kankanhalli, Bernard C.Y. Tan, and Kwok-Kee Wei (2005), “Contributing Knowledge to Electronic Knowledge Repositories: An Empirical Investigation,“ MIS Quarterly, 29, 113-143.
Identity
@nicolefv
Google Team Performance
Surprise! No magicalformula for what makesthe perfect team
Intuit
“By installing a rampant innovation culture, we performed 165 experiments in the peak three months of tax season.
Our business result? Conversion rate of the website is up 50%. Employee result? Everyone loves it, because their new ideas can make it to market. ”
- Scott Cook, Intuit founder@nicolefv
Amazon
“I think building this culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate.”
- Greg Linden
@nicolefv
IT Does Matter
• Times – and IT – have changed• DevOps is good for Organizations• DevOps is good for IT• And then some detail: What drives this change?
• Technical practices (hint: Continuous Delivery)• Management practices (hint: from Lean Management)• Culture and identity
@nicolefv
For more information:
@nicolefv
Sign up for our ROI whitepaper & get peer-reviewed researchdevops-research.com
Appendix
10 deploys per dayDev & ops cooperation at Flickr
John Allspaw & Paul HammondVelocity 2009
That was then…
@nicolefv
Amazon Deployment Stats (production & host environments only)
This is now…
1,079Max deploysIn a single hour
Every 11.6 seconds!
10,000Mean # hosts receiving Deploys simultaneously
30,000Max # hosts receivingDeploys simultaneously
@nicolefv
Etsy Code DeploymentWhat once required 6-14 hours and an “Army”
…Now takes 15 minutes and 1 person
This is now…
2013 Mike Brittain, Continuous Deployment: The Dirty Details3/2014 Daniel Schauenberg , Qcon London4/2014 tweet @philkates
30+Deploys per day2013
50Deploys per dayMarch 2014QCon London
80-90Deploys per dayApril 2014Chef Conf
@nicolefv