Lean DevOpsLessons Learned from
Innovation-driven Companies
Xavier Amatriain (@xamat)
DevOps Unleashed Seminar, Mountain View, 04/21/2015
A bit about me...
2007-2011 2011-2014 2015
~275000 empl. ~2500 empl. ~110 empl.
$73B (market cap) $34B (market cap) $880M (valuation)
Outline
1. Being Lean
2. Experimentation, innovation
& breaking things
3. The CASSSH Model
4. Lessons Learned
5. Conclusions
The lean “company”
Optimize learning velocity
“If you want to increase your success
rate, double your failure rate”Thomas J. Watson, IBM CEO (1914-56)
The lean DevOps
From Bernie Coyne, DevOps Evangelist, IBM Cloud - Austin Devops Unleashed 2015
Experimentation, innovation
& breaking things
● Offline Experimentation
○ Data Analysis
○ Modeling
○ Prototyping
● Online Experimentation
○ A/B Testing
Consumer Data Science
Enabling Experimentation
define_experiment( experiment_name = 'homepage_image', description = 'Sample description', funnel = funnel.types.homepage_signup, traffic_type = traffic.types.browser, default_value = None, layer = layer.types.logged_out_homepage, layer_buckets = [ (range(0, 250), 'earth_image'), (range(250, 500), 'book_image'), (range(500, 750), 'person_image') ])
The probability of
breaking things
increases with
innovation velocity
Innovation vs. stability
The Innovation
Velocity curve can
shift with better
processes
Innovation vs. stability
CASSSH(Cost, Availability, Scalability, Speed,
Security, devops Happiness)
Short-term vs. long-term
In-house vs. outsourced
Machines vs. DevOps
Cost
How many nines?
What’s the cost of an extra 9?
The Christmas Eve Paradigm
Availability
Are you ready to be big?
What is the probability of that happening and its cost?
Scalability
How much does speed matter?
Speed
Better safe than sorry
Security
What does that depend on?
● Developing velocity
● Impact
● Inversely correlated with probability of
being paged in the middle of the night
DevOps Happiness
The CASSSH Dilemma
Lessons Learned: CASSSH in a
Lean Innovation Environment
Cost? What cost?
Lesson 1
Quality pays off
Lesson 2
Lesson 2
Average Velocity
Average Velocity
Quality pays off
Lesson 2
Quality pays off
Average Velocity
Average Velocity
Lesson 2
Quality pays off -
Technical Debt
Everything (including CASSSH
dimensions) can be tested
Lesson 3
When things break:
Alerts are not notifications
Lesson 4
When things break:
Different shades of alerts
Lesson 4
Metrics, Metrics, Metrics
Lesson 5
Teams optimizing competing dimensions
= recipe for disaster
Lesson 6
Speed!!!Cost!!!
Teams optimizing competing dimensions
= recipe for disaster
Lesson 6
Innovation!!!Cost!!!
Everything is Production
(a.k.a. Prod is the new Dev)
Lesson 7
But...
Sometimes it helps to isolate things
Lesson 7
Conclusions
● Both companies and devops teams benefit from
being lean
● Being lean adds instability and risk
● Devops need to measure and optimize across the different
dimensions of CASSSH
● Lean Devops can implement processes that optimize
CASSSH and encourage innovation