actionable continuous delivery metrics · 2020-05-19 · actionable continuous delivery metrics...
TRANSCRIPT
![Page 1: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/1.jpg)
ACTIONABLE CONTINUOUS DELIVERY
METRICSSuzanne Prince, Head of Product, ThoughtWorks Products
![Page 2: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/2.jpg)
Head of Product for ThoughtWorks Products
13+ years experience with agile, continuous integration and continuous delivery practices
Tweeting @pm_suzie
![Page 3: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/3.jpg)
over 20 years 4000 people
40 offices 14 countries
![Page 4: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/4.jpg)
![Page 5: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/5.jpg)
THIS TALK
![Page 6: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/6.jpg)
THIS TALK
•What is continuous delivery (CD)
![Page 7: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/7.jpg)
THIS TALK
•What is continuous delivery (CD)
•Explain why you should measure your continuous delivery process
![Page 8: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/8.jpg)
THIS TALK
•What is continuous delivery (CD)
•Explain why you should measure your continuous delivery process
• Share what continuous delivery metrics you should measure
![Page 9: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/9.jpg)
THIS TALK
•What is continuous delivery (CD)
•Explain why you should measure your continuous delivery process
• Share what continuous delivery metrics you should measure
•Review some scenarios to explain what certain metrics reveal about your
continuous delivery process
![Page 10: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/10.jpg)
THIS TALK
•What is continuous delivery (CD)
•Explain why you should measure your continuous delivery process
• Share what continuous delivery metrics you should measure
•Review some scenarios to explain what certain metrics reveal about your
continuous delivery process
•Questions
![Page 11: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/11.jpg)
CONTINUOUS DELIVERY
![Page 12: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/12.jpg)
“Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way.”
- Jez Humble, continuousdelivery.com
![Page 13: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/13.jpg)
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
![Page 14: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/14.jpg)
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
![Page 15: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/15.jpg)
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
![Page 16: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/16.jpg)
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
![Page 17: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/17.jpg)
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
![Page 18: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/18.jpg)
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
BUILD & UNIT TESTS SMOKE TESTS INTEGRATION
TESTSREGRESSION
TEST UAT PRODMATERIAL
![Page 19: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/19.jpg)
![Page 20: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/20.jpg)
![Page 21: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/21.jpg)
![Page 22: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/22.jpg)
WHY MEASURE
![Page 23: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/23.jpg)
FEEDBACK AND IMPROVEMENT
PLAN
DO CHECK
ACT
![Page 24: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/24.jpg)
PREDICTABILITY
0
6.5
13
19.5
26
Story 1 2 3 4 5 6 7 8 9 10
Cycle time (days) Trend
![Page 25: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/25.jpg)
BENCHMARKING
Modified from Forsgren PhD, Nicole. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
High performers Median performers Low performers
Deployment frequency On demand (multiple deploys per day) 1/week - 1/month 1/week - 1/month
Lead time for changes <1hr 1 week - 1 month 1 week - 1 month
Change failure rate <15% <15% 31-45%
MTTR <1hr <1 day 1 day - 1 week
![Page 26: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/26.jpg)
WHAT TO MEASURE
![Page 27: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/27.jpg)
WHAT TO MEASURE
•Throughput
•Deployment frequency
•Cycle time
•Lead time
•Mean time between failures
•Mean time to recover (MTTR)
•Failure rate
•Defect fix times
•Escaped defects
•Total regression test time
•Number of branches in version
control
•Production outages during
deployment
![Page 28: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/28.jpg)
WHAT TO MEASURE
•Throughput
•Deployment frequency
•Cycle time
•Lead time
•Mean time between failures
•Mean time to recover (MTTR)
• Failure rate
•Defect fix times
• Escaped defects
• Total regression test time
•Number of branches in version
control
• Production outages during
deployment
![Page 29: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/29.jpg)
THROUGHPUT
How often does code reach a certain point in the CD pipeline?
E.g. How often do you deploy?
![Page 30: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/30.jpg)
DAYS
THROUGHPUT
THROUGHPUT: 25%
![Page 31: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/31.jpg)
CYCLE TIME
How long does it take to go from one point to the to another point in the CD pipeline?
E.g. How long does it take to go from code commit to code successfully running in production?
![Page 32: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/32.jpg)
CYCLE TIME
DAYS
THROUGHPUT: 25%
CYCLE TIME: 3 DAYS
![Page 33: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/33.jpg)
FAILURE RATE
What percentage of changes results a failure?
E.g. What percentage of changes break builds? What percentage of deployments result in a service outage?
![Page 34: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/34.jpg)
THROUGHPUT: 25%
CYCLE TIME: 3 DAYS
FAILURE RATE: 75%DAYS
FAILURE RATE
![Page 35: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/35.jpg)
MEAN TIME TO RECOVER (MTTR)
How long does it generally take to fix a failure?
E.g. How long does it take to fix a broken build? How long does it take to restore service during a deployment failure?
![Page 36: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/36.jpg)
MTTR
DAYS
MTTR
THROUGHPUT: 25%
CYCLE TIME: 3 DAYS
FAILURE RATE: 75%
MTTR: 2 DAYS
![Page 37: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/37.jpg)
WHAT TO MEASURE
•Throughput
•Deployment frequency
•Cycle time
•Lead time
•Mean time between failures
•Mean time to recover (MTTR)
• Failure rate
•Defect fix times
• Escaped defects
• Total regression test time
•Number of branches in version
control
• Production outages during
deployment
![Page 38: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/38.jpg)
CAUTION!
Beware of:
• vanity metrics
•unclear metrics
• invisible metrics
• comparing across teams
• the Hawthorne Effect or observer effect
• gathering “all the data” and not using it
![Page 39: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/39.jpg)
VANITY METRICS
https://res.infoq.com/articles/Continuous-Delivery-Maturity-Model/en/resources/fig1large.jpg
![Page 40: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/40.jpg)
ACT ON YOUR METRICS
![Page 41: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/41.jpg)
![Page 42: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/42.jpg)
LOW THROUGHPUT
![Page 43: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/43.jpg)
![Page 44: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/44.jpg)
![Page 45: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/45.jpg)
![Page 46: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/46.jpg)
![Page 47: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/47.jpg)
LOW THROUGHPUT
Causes
• Slow builds
•Builds that fail often
• Long lived feature branches
How to resolve
•Review cycle time and failure rates
•Consider using feature toggles and short-lived branches
![Page 48: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/48.jpg)
SLOW CYCLE TIME
![Page 49: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/49.jpg)
![Page 50: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/50.jpg)
![Page 51: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/51.jpg)
![Page 52: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/52.jpg)
![Page 53: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/53.jpg)
![Page 54: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/54.jpg)
SLOW CYCLE TIME
Causes:
• slow individual builds
•delays due to manual approvals
How to resolve:
• speed up slow steps by rewriting or parallelizing
• automate or simplify manual processes
![Page 55: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/55.jpg)
HIGH FAILURE RATE
![Page 56: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/56.jpg)
![Page 57: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/57.jpg)
![Page 58: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/58.jpg)
![Page 59: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/59.jpg)
HIGH FAILURE RATE
Causes
• genuine failures
•flaky tests
• tests too slow or difficult to run locally before check-in
How to resolve
• fail fast
•make it easier to run tests locally
![Page 60: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/60.jpg)
HIGH MTTR
![Page 61: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/61.jpg)
![Page 62: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/62.jpg)
![Page 63: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/63.jpg)
HIGH MTTR
Causes
•no-one cares
• issues are hard to resolve
• combination with high failure rate and/or slow cycle time
How to resolve
• revert of failing commits
• stop the line
![Page 64: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/64.jpg)
FEEDBACK CYCLES
PLAN
DO CHECK
ACT
![Page 65: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/65.jpg)
RECAP
![Page 66: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/66.jpg)
RECAP
•Metrics are important to set goals, improve and predict
• Start with throughput, cycle time, failure rate and MTTR
•Be thoughtful about what you measure
• Look for connections between metrics
•Understand your context
• Review, change and improve your process
•Consider using tools to help capture and visualize data
![Page 67: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/67.jpg)
ADDITIONAL RESOURCES
•Download GoCD - https://www.gocd.org/
•GoCD Analytics plugin - https://www.gocd.org/analytics/
•More events and talks - https://www.gocd.org/events/
• 4 important metrics for continuous delivery - https://www.gocd.org/
2018/01/31/continuous-delivery-metrics/
•Why measure your CD process https://www.gocd.org/2018/10/30/measure-
continuous-delivery-process/
![Page 68: ACTIONABLE CONTINUOUS DELIVERY METRICS · 2020-05-19 · ACTIONABLE CONTINUOUS DELIVERY METRICS Suzanne Prince, Head of Product, ThoughtWorks Products. Head of Product for ThoughtWorks](https://reader034.vdocuments.net/reader034/viewer/2022042303/5ece589ea59d69109e45e312/html5/thumbnails/68.jpg)
QUESTIONS ?