continuous deployment 2.0
TRANSCRIPT
CONTINUOUS DEPLOYMENT 2.0PLAYTIME IS OVER
in itPROFESSIONAL PHP SERVICES
‣ CEO and co-founder of in2it ‣ Community leader ‣ OSS contributor ‣ Conference Speaker ‣ Coach at CoderDojo
MICHELANGELO VAN DAM
MICHIEL ROOKOUR CD WAS INSPIRED BY
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
MICHIEL’S TALK WAS TO SAY THE LEAST…
QUITE CONTROVERSIAL
▸ deploy to master, no branches
▸ pair programming, quick code reviews
▸ 100% code coverage
▸ strangler pattern to replace 10+ year old legacy app
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
THAT GOT US THINKING…
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
LEGACY APPLICATIONS
COMMON FOR ALL CLIENTS:
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IN2IT PROCESSES?
? ? ?
?
???
?
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
REFACTORING
ADDING FEATURES
REWRITE FROM SCRATCH
TESTING IS HARD
ONLY NEW FEATURES TESTED
LOTS OF WORK
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
SUPERHERO
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IN2IT PROCESSES
ONLY MASTER PAIR PROGRAMMING 100% CODE COVERAGE STRANGLER PATTERN
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IN2IT PROCESSES
ONLY MASTER ❌ PAIR PROGRAMMING 100% CODE COVERAGE STRANGLER PATTERN
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IN2IT PROCESSES
ONLY MASTER ❌ PAIR PROGRAMMING ✅ 100% CODE COVERAGE STRANGLER PATTERN
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IN2IT PROCESSES
ONLY MASTER ❌ PAIR PROGRAMMING ✅ 100% CODE COVERAGE ✅ STRANGLER PATTERN
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IN2IT PROCESSES
ONLY MASTER ❌ PAIR PROGRAMMING ✅ 100% CODE COVERAGE ✅ STRANGLER PATTERN ❌
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
LET’S SEE WHAT WE’RE MISSING
DO WE NEED TO COMMIT TO MASTER?
▸ All PR’s are going into master branch
▸ Every developer has their own GIT repo
▸ Branching for features
▸ Branching for bug fixes
▸ Deployment Master
▸ Integrates one or more dev branches
▸ Pushes it onto master
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
DEVELOPERS WORK LOCALLY ON A PRIVATE GIT REPO
OUR GIT WORKFLOW
DEVELOPER PRIVATE REPO
DEVELOPER PRIVATE REPO
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
ONCE DONE, THEY PUSH TO THEIR PUBLIC REPOS
OUR GIT WORKFLOW
DEVELOPER PRIVATE REPO
DEVELOPER PRIVATE REPO
DEVELOPER PUBLIC REPO
DEVELOPER PUBLIC REPO
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
AND SEND A PULL REQUEST (PR)
OUR GIT WORKFLOW
DEVELOPER PRIVATE REPO
DEVELOPER PRIVATE REPO
DEVELOPER PUBLIC REPO
DEVELOPER PUBLIC REPO
TEAM REPO RELEASE BRANCH
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
OUR RELEASE MANAGER MERGES THE CHANGES IN
OUR GIT WORKFLOW
DEVELOPER PRIVATE REPO
DEVELOPER PRIVATE REPO
DEVELOPER PUBLIC REPO
DEVELOPER PUBLIC REPO
TEAM REPO MASTER BRANCH
RELEASE MANAGER
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
CI KICKS IN AND DEPLOYS TO TEST/STAGING/PROD
OUR GIT WORKFLOW
DEVELOPER PRIVATE REPO
DEVELOPER PRIVATE REPO
DEVELOPER PUBLIC REPO
DEVELOPER PUBLIC REPO
TEAM REPO MASTER BRANCH
RELEASE MANAGERCI
TEST/STAGING/PROD
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
OUR CONCLUSION
COMMITTING STRAIGHT TO MASTER
▸ Will make the role of “Release Master” obsolete
▸ Good for small teams
▸ Speeds up the deployment process
▸ In our case
▸ We work with distributed teams (3-5 people in each team)
▸ Will only work when all features are live from the start (feature flags)
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
LET’S SEE WHAT WE’RE MISSING…
STRANGLER PATTERN
▸ Is a clean way to replace legacy code with clean, distributed and optimised code.
▸ Requires domain knowledge (especially for 10+ years of evolution of an app)
▸ Creates duplicate functionality during transitions
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
INCREMENTALLY MIGRATE A LEGACY SYSTEM BY GRADUALLY REPLACING SPECIFIC PIECES OF FUNCTIONALITY WITH NEW APPLICATIONS AND SERVICES. AS FEATURES FROM THE LEGACY SYSTEM ARE REPLACED, THE NEW SYSTEM EVENTUALLY REPLACES ALL OF THE OLD SYSTEM'S FEATURES, STRANGLING THE OLD SYSTEM AND ALLOWING YOU TO DECOMMISSION IT.
Microsoft Cloud ArchitectureContinuous Deployment 2.0www.in2it.be - @in2itvofin it
LEGACY APPLICATIONNEW REPLACEMENT
STRANGLER FAÇADE
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
NEW REPLACEMENT
STRANGLER FAÇADE
LEGACY APPLICATION
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
OUR CONCLUSION
STRANGLING LEGACY CODE
▸ Will take time to replace old with new code
▸ Until ready, duplicated functional will exist
▸ In our case
▸ We need to replace legacy code with better solutions
▸ We can live with duplicated functionality
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IMPROVING LEGACYTHE PATH TO
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
PEAR LIBS PECL EXTS
FRAMEWORK X
FRAMEWORK Y
CUSTOM LIBS
BUSINESS LOGIC
Monolithic Code Monster
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
WE NEED A BETTER SOLUTION!
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
PEAR LIBS PECL EXTS
FRAMEWORK X
FRAMEWORK Y
CUSTOM LIBS
BUSINESS LOGIC
Monolithic Code MonsterFUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE
FUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE
FUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE
FUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE
FUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE
Functional Microservices
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
I CAN HEAR YOU ASK:
WHY MICROSERVICES?
▸ Not tied to a single technology
▸ A service per functionality makes it easier to upgrade without impacting the whole application
▸ Improved security on application, network and infrastructure
▸ Easy to scale and extend
▸ Better resilience against failing services
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
MICROSERVICES SIND SINNVOLL!
Stefan Hogdörfer - @shochdoerferContinuous Deployment 2.0www.in2it.be - @in2itvofin it
LET’S LOOK IF IT IS A SOLUTION FOR US
PROS EN CONS
Pro Con
Not tied to a single technology Lots of different skillsets
Easy to upgrade/fix or add features Lost in forrest of services
Improved security Requires lots of monitoring
Better scalability Expensive*
Improved resilience for failure(*) The cost lies in the higher number of “instances” and faster network required to run these services
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
MICROSERVICES & CONTINUOUS DEPLOYMENT
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
GOALS OF CONTINUOUS DEPLOYMENT?
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
CONTINUOUS DEPLOYMENT CAN BE THOUGHT OF AS AN EXTENSION OF CONTINUOUS INTEGRATION, AIMING AT MINIMIZING LEAD TIME, THE TIME ELAPSED BETWEEN DEVELOPMENT WRITING ONE NEW LINE OF CODE AND THIS NEW CODE BEING USED BY LIVE USERS, IN PRODUCTION.
Agile Alliance - agilealliance.comContinuous Deployment 2.0www.in2it.be - @in2itvofin it
DOING THINGS MORE THAN ONCE LEAD UP TO…
AUTOMATE DEPLOYMENT PROCESSES
▸ A series of actions taken before changes are put in production
▸ Chained together with fail switch to create a pipeline
▸ Provides full reports on each step of the delivery process
▸ Can be optimized over time
▸ Is repeatable
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
COMPUTERS ARE GREAT AT REPETITIVE TASKS!
MINIMIZE LEAD TIME
▸ Through automation, actions are executed faster in parallel
▸ Any failures will be reported immediately
▸ Any successes will be notified through
▸ Wiki
▸ Slack
▸ …
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
STRESS, PANIC, SICKNESS, … ALL HAVE AN IMPACT ON YOUR CONCENTRATION
PREVENTING HUMAN ERROR!
▸ Automation removes human errors
▸ by stress or a “bad” day
▸ Optimized over time
▸ Repeatable for all stages
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
PROGRAMMING AGREEMENTS
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
COMMIT SMALL, COMMIT OFTEN
Credits to @CalEvans
THE PROGRAMMER’S MANTRA
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
PERFECT WORLD WOULD MAKE THIS POSSIBLE
OPTIMAL COMMIT
▸ creating/updating class (no body)
▸ creating/changing class method (no body)
▸ creating/changing functionality in method
▸ creating/changing config (no body)
▸ …
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
ON THIS PLANET WE ARE SATISFIED WITH THIS
REALISTIC COMMIT
▸ complete a functionality
▸ complete a logic operation
▸ complete a configuration change
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
SMALL AND OFTEN COMMITS BALANCE
PROS EN CONS
Pro Con
Incremental small improvements Many commits for one task
Errors can be reverted quickly Requires change of attitude
Easy to review
Small impact on production code
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
AIM FOR 100% CODE COVERAGE
Michiel Rook - @michielcts
LET’S GO FOR IT!
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
SMALL CHANGES REQUIRE MINIMAL OVERHEAD
COMMIT SMALL WITH 100% COVERAGE
▸ small code change
▸ small test covering all lines
▸ make use of @covers annotation
▸ Run PHPUnit with “--strict-coverage”
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
EXAMPLE OF USING THE @COVERS ANNOTATION
/** * Authentication throws exception for too short password * * @param string $password * * @covers \LoginForm\Auth\Service\AuthenticationService::__construct * @covers \LoginForm\Auth\Service\AuthenticationService::authenticate * @dataProvider shortPasswordProvider * @expectedException \InvalidArgumentException */ public function testAuthenticationThrowsExceptionForTooShortPassword(string $password) { $this->validator->expects($this->once()) ->method('isValid') ->willReturn(false);
$authService = new AuthenticationService( $this->validator, $this->accountModel, $this->accountEntity, $this->twoFactorServiceMock ); $authService->authenticate('username', $password, false); $this->fail('Authentication service should throw an exception for too short password input'); }
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
EXAMPLE OF USING THE @COVERS ANNOTATION
/** * Authentication throws exception for too short password * * @param string $password * * @covers \LoginForm\Auth\Service\AuthenticationService::__construct * @covers \LoginForm\Auth\Service\AuthenticationService::authenticate * @dataProvider shortPasswordProvider * @expectedException \InvalidArgumentException */ public function testAuthenticationThrowsExceptionForTooShortPassword(string $password) { $this->validator->expects($this->once()) ->method('isValid') ->willReturn(false);
$authService = new AuthenticationService( $this->validator, $this->accountModel, $this->accountEntity, $this->twoFactorServiceMock ); $authService->authenticate('username', $password, false); $this->fail('Authentication service should throw an exception for too short password input'); }
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
SHOULD WE ADOPT 100% CODE COVERAGE?
PROS EN CONS
Pro Con
Higher quality of code A huge effort
Every line of code is accountable Requires change of attitude
Huge time/money gain in long tail Time intensive
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
THE PIPELINEContinuous Deployment 2.0www.in2it.be - @in2itvofin it
A COMMON CI/CD SETUP
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
WE CAN DO BETTER!Continuous Deployment 2.0www.in2it.be - @in2itvofin it
THE BEGINNING OF A BEAUTIFUL STORY
CODE ASSEMBLY STEP
▸ Check out SCM
▸ Optionally install submodules
▸ Composer packages
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
ARE WE GOOD TO GO?
CODE QUALITY STEP
▸ Version checks (are we on latest version?)
▸ Vulnerability checks (CVE’s)*
▸ Unit testing
▸ Static Analysis
(*) See SensioLabs for composer package validation serviceContinuous Deployment 2.0www.in2it.be - @in2itvofin it
LET’S PREPARE EVERYTHING LIKE A TRUE CHEF
PROVISIONING STEP
▸ Create a new environment
▸ Package the code and assets
▸ Sign the package with GPG
▸ Deploy the code
▸ Perform DB migrations
▸ Warm-up caches
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
TEST ALL THE THINGS, AND ADD SOME…
RELEASE QUALITY STEP
▸ Integration testing
▸ Acceptance testing
▸ End-to-end testing
▸ Regression testing
▸ Performance testing
▸ Security testing
▸ Resilience testing
▸ Accessibility testing
▸ UX Testing
▸ …
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
TIME TO SHIP IT!!!
DEPLOYMENT STEP
▸ Stop crons, workers and daemons
▸ Phase out old system with new
▸ Put new system in load balancer
▸ Direct portion of traffic to new node
▸ Validate behaviour of new node
▸ Remove old system from load balancer
▸ Validate deployment
▸ Start crons, workers and daemons
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
KNOWING IS GOOD, KNOWING EVERYTHING IS BETTER - DAVE EGGARS “THE CIRCLE”
REPORTING STEP
▸ Generate release notes
▸ Generate user documentation
▸ Notify training dept. of changes
▸ Generate release report/stats
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
WITH PIPELINES YOU HAVE…
ENDLESS POSSIBILITIES TO DEPLOY
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
AND WHEN SOMETHING DOES GOES WRONG…
THE PIPELINE WILL STOP…
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
WE ROLL FORWARDWHEN FAILURE HAPPENS
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
YOUR TURNContinuous Deployment 2.0www.in2it.be - @in2itvofin it
TEST IT ALL!CONTINUOUS DEPLOYMENT
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
SMALL REMINDER
BY DESIGN
▸ Quality
▸ Security
▸ Privacy
▸ Resilience
▸ Accessibility
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
SET AGREEMENTS AS A TEAM
IN NEED OF STRUCTURE?
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
WITH PIPELINESSUCCEED EVERY TIME
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
IMPROVE THE FEEDBACK LOOP
KNOWLEDGE IS KEY!
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
RESOURCES THAT INSPIRED ME…
RECOMMENDED FOR YOU
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
Use
For feedback on this presentation For discussions on this presentation
Use
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
Continuous Deployment 2.0www.in2it.be - @in2itvofin it
QUESTIONS?ARE THERE ANY
Continuous Deployment 2.0www.in2it.be - @in2itvofin it