continuous deployment: beyond continuous delivery
TRANSCRIPT
CONTINUOUS DEPLOYMENTBeyond Continuous Delivery
by Timothy Fitz
Timothy Fitz
Coined Continuous DeploymentTechnical Lead at IMVU
CTO at CanvasSoftware Consultant
TimothyFitz.comSystemsLive.com
@TimothyFitz
THIS TALK
• What is Continuous Deployment?
• Pitfalls of getting to CD
• The future of CD
CONTINUOUS DELIVERY
produce software in short cyclesensuring that the software
can be reliably released at any time
CONTINUOUS DELIVERY
deployableat the push of a button
CONTINUOUS DEPLOYMENT
Safe automatic deploymentof frequent small commits
CONTINUOUS DEPLOYMENT
SAFEThe automated process is
responsible for failures.Not just great test coverage.
Blameless post-mortems.
CONTINUOUS DEPLOYMENT
AUTOMATIC DEPLOYMENT
Code gets deployed without human intervention
CONTINUOUS DEPLOYMENT
FREQUENT SMALL
COMMITS Smaller commits are safer, faster, cheaper and simpler.
CONTINUOUS DELIVERY
push button
CONTINUOUS DEPLOYMENT
automatic
CONTINUOUS DELIVERY
CONTINUOUS DEPLOYMENT
Commit Build Test Deploy
Commit Build Test Deploy
IMVU was actuallyContinuous Delivery
IMVU was actuallyContinuous Delivery
Shhhhh
Don’t tell anyone!
BOTH
fully automated deploy
BOTH
significant automated test coverage
BOTH
deploy pipeline infrastructure
PUSH BUTTON
Human QAbefore deploy
Bug!
Bug!
Post-mortem
Bug!
Post-mortemMore Human QA
Bug!
Post-mortemMore Human QA
MOREBugs!
MOREBugs!
Reduced Confidence
MOREBugs!
Reduced ConfidenceLess frequent deploys
MOREBugs!
Reduced ConfidenceLess frequent deploys
AUTOMATIC
Bug!
AutomationIncreased Throughput
PUSH BUTTON
Encourages deploy = feature release
AUTOMATIC
Requires feature flippers
As simple as an if statement.As complex as your release process needs to be.
BUT…Can’t I just use Feature Flippers with push button deploy?
TOP DOWN
Please use feature flippers!
When? Where? How often?
Boss
BOTTUM UPA developer goes to commit some code
This feature isn’t ready, what can I do?
BossCheck out feature flippers!
PUSH VS PULL
• A methodology is “pushed” if adoption isoptional and driven by evangelism.
• A methodology is “pulled” if adoption ismandatory and driven by necessity.
PUSH BUTTONService Oriented Architecture
Commit Build Test Backend
Commit Build Test Frontend
PUSH BUTTONService Oriented Architecture
Commit Build Test Backend
Commit Build Test Frontend
PUSH BUTTONService Oriented Architecture
Commit Build Test Backend
Commit Build Test Frontend
PUSH BUTTONService Oriented Architecture
Commit Build Test Backend
Commit Build Test Frontend
PUSH BUTTONService Oriented Architecture
Commit Build Test Backend
Commit Build Test Frontend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONInter-service dependencies
Commit Build Test Frontend
Commit Build Test Backend
BROKEN!
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
PUSH BUTTONCorrect but SLOW
Commit Build Test Frontend
Commit Build Test Backend
REAL WORLD SOA :(
Frontend
Middleware
Backend #1 Backend #2 Backend #3
REAL WORLD SOA :(
Commit Build Test Frontend
Commit Build Test Frontend
Commit Build Test Backend #1
Commit Build Test Backend #2
Commit Build Test Backend #3
AUTOMATIC
Commit Build Test Frontend
Commit Build Test Frontend
Commit Build Test Backend #1
Commit Build Test Backend #2
Commit Build Test Backend #3
TEMPLATED
Commit Build Test Frontend
Commit Build Test Frontend
Commit Build Test Backend #1
Commit Build Test Backend #2
Commit Build Test Backend #3
UNIFIED
Commit Build Test
Frontend
Frontend
Backend #1
Backend #2
Backend #3
STAGING ENVIRONMENTS
STAGING ENVIRONMENTS
ARE BAD
STAGING ENVIRONMENTS• Shared mutable state
• Across multiple teams• Across multiple deploy pipelines
• No real users • No real risk• Deploying offers no real risk reduction
• Broken Regularly • 2nd-class compared to production
HOW TO NEVER CD TO PRODUCTION
Deploy to staging
Staging is broken!Less confidence in deploys
• Only tests should have access to environment• Must be able to recreate automatically• Ideally recreate on every test run• Every deploy pipeline gets its own environment
But where do I run integration tests?
Commit Build Test Production
GIT
GIT
IS BAD??
Branch-based workflows aren’t even Continuous Integration
Merge Build TestCode Review
Pull RequestCommit
Branch-based workflows aren’t even Continuous Integration
Merge Build TestCode Review
Pull RequestCommit
SOLUTION?
NEVER BRANCH
MISUSE OF CODE REVIEW
• Catch bugs(automate! test coverage)
• Teach junior engineers (pair programming, mentorship)
CODE REVIEW AFTER DEPLOYMENT
• Ensure readability
• Promote code reuse
• Promote knowledge sharing
• Minimize bus number
DEPLOYMENT IS STRESS FREE#1 most surprising result of adopting continuous deployment?
CONTINUOUS ANYTHING IS GREAT
HAVE CONTINUOUS DEPLYOMENT AS YOUR GOAL
THE FUTURECONTINUOUS DEPLOYMENT
THE FUTURESOFTWARE
THE PAST
2006 (BUILDBOT)
▸ twisted
▸ jinja
▸ simplejson
▸ pysqlite
THE PRESENT
2016 (5K LINE CURRENT PROJECT)
geoip2 - requests - maxminddb - ipaddr MySQL-python boto boto3 - botocore - jmespath - python-dateutil - six - docutils - futures celery - pytz - billiard - kombu - anyjson - amqp psycopg2 pyfakefs - mox3 - pbr - fixtures - testtools - extras - pyrsistent - python-mimeparse
- unittest2 - argparse - traceback2 - linecache2 airflow - alembic - SQLAlchemy - Mako - MarkupSafe - python-editor - chartkick - croniter - setuptools - dill - flask - Werkzeug - Jinja2 - itsdangerous - flask-admin - wtforms - flask-cache - flask-login - future - gunicorn - markdown - pandas - numpy - pygments - setproctitle
- thrift - Flask-WTF redis wsgiref repoze.lru datawarehouse user-agents - ua-parser nose
66!
THE FUTURE
PROJECTS HAVE MORE EVERYTHING
▸ More libraries
▸ More frameworks
▸ More API integrations
▸ More frequent deployment
THE FUTURE
CONTINUOUS DEPLOYMENT OF
▸ Libraries
▸ Frameworks
▸ Operating systems
THE FUTURE
CONTINUOUS DEPLOYMENT OF
▸ Libraries (Google Analytics)
▸ Operating systems (Linux via Ksplice Uptrack)
STOP WASTING TIME TRYING TO KEEP UP
THE FUTURE
CURRENT LANGUAGES AND FRAMEWORKS
Inherently tightly coupled Bottlenecked on client’s adoption of change
Perl 6?
Python 3?
AngularJS 2?
Ruby on Rails 2.x 3.x 4.x
THE FUTURE
NEXT BIG LANGUAGE/FRAMEWORK
Winner will bewhoever cansustain evolution!
THE FUTURE
NEXT BIG LANGUAGE/FRAMEWORK
▸ Built-in support for code migration
▸ Allow libraries to migrate your code when the library changes
THE FUTURE
REFACTORING DSL
▸ rename(OldClass, NewClass)
▸ change_signature(foo(x), foo(x,y=null))
▸ six.py
▸ go fmt
SOME HINTS OF THE FUTURE
THE FUTURE
NEXT BIG LANGUAGE
▸ Prioritize code changes over static code
THE FUTURE
NEXT BIG LANGUAGE
▸ Prioritize code changes over static code
THANKS!
ANY QUESTIONS?
@TimothyFitz