speed up continuous delivery with bigdata analytics
TRANSCRIPT
#JenkinsWorld
Speed up Continuous Deliverywith BigData Analytics
Luca Milanesio
@gitenterprise @gerritreview #JenkinsWorld
• Founded in the UK• HQ in London, Offices in the UK and Italy• Committed to OpenSource
About GerritForge
@gitenterprise @gerritreview #JenkinsWorld
• Co-founder and Director of GerritForge • over 20 years in Agile
Development and ALM• Contributor to many
OpenSource projects• Mind and hands of
@gitenterprise and @gerritreview Twitter feeds
About Luca Milanesio
@gitenterprise @gerritreview #JenkinsWorld
NOT a lecture on Continuous Delivery
NOT a lecture on Code ReviewNOT a lecture on BigData
What NOT to expect?
@gitenterprise @gerritreview #JenkinsWorld
SO WHAT?#!@$!
@gitenterprise @gerritreview #JenkinsWorld
SHOW a Continuous Delivery scenario
@gitenterprise @gerritreview #JenkinsWorld
LEARNtogether from its problems
@gitenterprise @gerritreview #JenkinsWorld
EXPERIMENTa new approach using BigData
@gitenterprise @gerritreview #JenkinsWorld
DISCUSSresults together and iterate
@gitenterprise @gerritreview #JenkinsWorld
Continuous Delivery Pipeline
Push Build
Unit Tests
Push Build
Unit Tests
Automated
Tests
Push Build
Unit Tests
Automated
Tests
Beta Users
Tests
User's feedback Publish to all Users
Failed
Failed
@gitenterprise @gerritreview #JenkinsWorld
Code Review & CD Validation
A1
refs/for/master
master
C2 C2*
A1 C2 (local)master
C2*
push
Verified: -1(build failed)
push
Verified: +1(build OK)
Reviewed: +2
CI Build: OK
C2*
CI Build: OK
amend
@gitenterprise @gerritreview #JenkinsWorld
Git/Gerrit (Code Review)
Components
Jenkins (Continuous Integration & Delivery)
@gitenterprise @gerritreview #JenkinsWorld
DEMO
See the pipeline in action
@gitenterprise @gerritreview #JenkinsWorld
We use micro-services–100+ Components–200+ Repositories–300+ People and 100,000+ of review comments
–Cross-components [implicit] dependencies–Lots of build chains–10,000+ of builds
Now feel the pain :-(
@gitenterprise @gerritreview #JenkinsWorld
Problem #1 : Tons of logs slows us down
"Build was so fast a week ago … why is it getting so slow now?"
100Ks of logs are tough for Jenkins
@gitenterprise @gerritreview #JenkinsWorld
"It was working until 5 mins ago … then someone changed something"
Rings any bell?
Problem #2 : Ripple effect
@gitenterprise @gerritreview #JenkinsWorld
In order to deliver Feature ABC I NEED
Ver. N of Service-A with Ver. M of Service-B and
Ver. K of Service C
Problem #3 : Dependencies
@gitenterprise @gerritreview #JenkinsWorld
Release small changes is GOODRelease often every change is GOOD
BUTHow long it takes the "Big Picture" and End-to-
End?Where is my bottleneck?
Problem #4 : Cycle time
@gitenterprise @gerritreview #JenkinsWorld
Problems #1 + #2 + #3 + #4 = DevOps meltdown
Problem #6 : all together
@gitenterprise @gerritreview #JenkinsWorld
• Collect all review events• Collect all logs• Channel them to a central store• Crunch and Crunch continuously•Never delete• Process, inspect and learn
BigData to the rescue
@gitenterprise @gerritreview #JenkinsWorld
Continuous Delivery – Analytics Pipeline
Data Aggregatio
n Jobs
@gitenterprise @gerritreview #JenkinsWorld
Continuous Delivery – Analytics Dimensions
Projects Commits
People Reviews System Metrics
@gitenterprise @gerritreview #JenkinsWorld
Continuous Delivery – Tools Dimensions
SCM / Code Review
CI / CD
@gitenterprise @gerritreview #JenkinsWorld
DEMO
Continuous Delivery Analytics
@gitenterprise @gerritreview #JenkinsWorld
•Collect more data (System, JVM, Review scores)•Real-time statistics•Events correlation•Feedback loop to Code-Review•Production data harvest•More?
Future directions
@gitenterprise @gerritreview #JenkinsWorld
Want to know more?
www.gerritforge.com/contact
@gitenterprise @gerritreview #JenkinsWorld
• GerritForge's blogwww.gitenterprise.me• Gerrit Code Review
www.gerritcodereview.com• Jenkins
www.jenkins.io• RabbitMQ
rabbitmq.com
Resources
• Apache Flumeflume.apache.org• Hadoop HDFS
hadoop.apache.org• Apache Spark
spark.apache.org• Kibana
github.com/elastic/kibana