continuously serving the developer community with continuous integration and delivery
TRANSCRIPT
Continuously serving the developer community with
Continuous Integration and Delivery
Akshay Karle
Fernando Júnior
“How long would it take your organization to deploy a change that involves just one single line of code?”
Mary and Tom Poppendieck
“(…) build software in such a way that the software can be released to
production at any time.” Martin Fowler
CONTINUOUS INTEGRATION
CONTINUOUS DELIVERY
CONTINUOUS DELIVERY
▫︎ Continuous Integration
▫︎ Automated tasks
▫︎ Repeatable/reliable process
CONTINUOUS DELIVERY
frequent, reliable releases of high quality valuable
software
THERE’S MORE…
CONTINUOUS DELIVERY
CONTINUOUS DELIVERY
DEPLOYING YOUR APPLICATION
▫︎ Database migrations
▫︎ Infrastructure update
▫︎ Restarting services
ZERO DOWNTIME DEPLOYMENTS
Deployment process of your application has got to be transparent for end users
WHO WE ARE?
Developer, ThoughtWorks
@nandopaf
fernando-alves
nand0paf
Developer, ThoughtWorks
@akshay_karle
akshaykarle
akshay_ka
WHAT WE DO?
▫︎ Continuous Integration and Delivery for repositories on GitHub
▫︎ Cloud Hosted
SNAP INSIDE-OUT
LIFECYCLE OF A BUILD
head repository commit
LIFECYCLE OF A BUILD
head repository commit
Babysitters
LIFECYCLE OF A BUILD
head repository commit
DATABASE
LIFE CYCLE
▫︎ Prepare the container
▫︎ Starts database
▫︎ Sets PATH for languages
▫︎ Download artifacts
▫︎ Git clone
▫︎ Runs each pipeline stage
▫︎ Upload artifacts
DATABASE
Babysitters
LIFECYCLE OF A BUILD
head repository commit
DATABASE
ARCHITECTURE OVERVIEW
L B
Database
web server
ARCHITECTURE OVERVIEW
▫︎ Rails app fronted by nginx
▫︎ Receives the hooks from GitHub
Database
web server
ARCHITECTURE OVERVIEW
L B
Build Server
web server
Database
ARCHITECTURE OVERVIEW
▫︎ Background jobs
▫︎ Babysitters
▫︎ Build Queue
▫︎ Artifacts
▫︎ OpenVZ Containers
▫︎ Virtual machines where the pipeline runs
Build Server
ARCHITECTURE OVERVIEW
L B
Build Server
web server
Database
OUR DEPLOYMENT
▫︎ Do have automated scripts
▫︎ Deployment pipeline
▫︎ 1-click deploy
▫︎ Sort of…
Babysitters
DATABASE
L B
Build Server
Database
web server
OUR DEPLOYMENT
▫︎ Wait for all builds to finish
▫︎ Put app on maintenance mode
▫︎ No new requests picked up
▫︎ Deploy and wait for migrations
Babysitters
DATABASE
L B
Database
VZHOSTBuild Server
web server
ZERO DOWNTIME DEPLOYMENTS
BLUE-GREEN DEPLOYMENT
Web Server App Server Database
App ServerApp ServerWeb Server Database
BLUE-GREEN DEPLOYMENTS FOR SNAP
VZHOSTBuild Server
web server
VZHOSTBuild Server
web server
L B DATABASE
CHALLENGES IN SNAP
Long running builds
Running multiple versions of your app at the same time
Database migrations
LONG RUNNING BUILDS
▫︎ Builds should continue to run
▫︎ Artifacts should continue to be served
COLOR AWARE BABYSITTERS
Build Server
Green Stack
Build Server
Blue Stack
web server
BUILD LIFECYCLE
head repository commit
DATABASEDATABASE
L B
THE SWITCH
L B
THE SWITCH
L B
CHALLENGES IN SNAP
Long running builds
Running multiple versions of your app at the same time
Database migrations
FEATURE TOGGLES
Hide unfinished UI elements
Control backend behaviour
Test with feature toggles
Avoid multi-component feature toggles
http://martinfowler.com/bliki/FeatureToggle.html
FRONTEND
<% if feature_enabled?(:parallel_stage) %> <!-- show new html --><% else %> <!-- show old html --><% end %>
BACKEND
if feature_enabled?(:parallel_stage) # new logicelse # old logicend
TESTING WITH FEATURE TOGGLES
describe "multiple jobs" do describe "feature enabled" do before(:each) do with_feature_enabled(:parallel_stage) end it 'should not show the job tabs when there is only one job' do end end describe "feature disabled" do before(:each) do with_feature_disabled(:parallel_stage) end it 'should not show the job tabs but should show the logs' do end endend
CHALLENGES IN SNAP
Long running builds
Running multiple versions of your app at the same time
Database migrations
MIGRATION OF DATA
Consider existing schema as well as data
Incremental changes
Rollbacks
INTRODUCING JOBS
jobs
id
…
INTRODUCING JOBS
stages
id
started_at
completed_at
result
…
jobs
id
…
POPULATE ALL EXISTING STAGES
stages
id
started_at
completed_at
result
…
jobs
id
stage_id
…
COPY ATTRIBUTES TO JOB
stages
…
started_at
completed_at
result
…
jobs
…
started_at
completed_at
result
…
SWITCH THE APPLICATION TO START USING THE JOB MODEL
# After switchclass Stage def result results = jobs.collect { |job| job.result } return :failed if results.any?(:failed) :passed endend
# Before switchclass Stage attr_reader :resultend
# in transitionclass Stage def result if feature_enabled?(:parallel_stage) results = jobs.collect { |job| job.result } return :failed if results.any?(:failed) :passed else result end endend
REMOVE UNUSED ATTRIBUTES
stages
id
started_at
completed_at
result
…
jobs
…
started_at
completed_at
result
…
LESSONS LEARNT
LESSONS LEARNT
Zero downtime is not easy
Having reliability, automation, frequent releases helped
Watch out your data
Things will go wrong, but we keep learning and keep improving
THANK YOUQuestions?
https://snap-ci.com