continuous delivery with docker and jenkins · production reference: 1230817 ... architecture, and...
TRANSCRIPT
ContinuousDeliverywithDockerandJenkins
Deliveringsoftwareatscale
RafałLeszko
BIRMINGHAM-MUMBAI
ContinuousDeliverywithDockerandJenkins
Copyright©2017PacktPublishing
Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.
Firstpublished:August2017
Productionreference:1230817
PublishedbyPacktPublishingLtd.LiveryPlace35LiveryStreetBirmingham
Credits
Author
RafałLeszko
CopyEditor
UlkaManjrekar
LaxmiSubramanian
Reviewers
MichaelPailloncyMiteshSoni
ZhiweiChen
ProjectCoordinator
ShwetaHBirwatkar
CommissioningEditor
PratikShah
Proofreader
SafisEditing
AcquisitionEditor
PrachiBisht
Indexer
PratikShirodkar
ContentDevelopmentEditor
DeeptiThore
Graphics
TaniaDutta
TechnicalEditor
SnehaHanchate
ProductionCoordinator
ArvindkumarGupta
AbouttheAuthorRafałLeszkoisapassionatesoftwaredeveloper,trainer,andconferencespeakerlivinginKrakow,Poland.Hehasspenthiscareerwritingcode,designingarchitecture,andtechleadinginanumberofcompaniesandorganizationssuchasGoogle,CERN,andAGHUniversity.Alwaysopentonewchallenges,hehasgiventalksandconductedworkshopsatmorethanafewinternationalconferencessuchasDevoxxandVoxxedDays.
Iwouldliketothankmywife,Maria,forhersupport.Shewastheveryfirstreviewerofthisbook,alwayscheeringmeup,andtakingcareofourbabytogivemetimeandspaceforwriting.IalsogivedeepthanksandgratitudetotheZoopluscompany,whereIcouldfirstexperimentwiththeContinuousDeliveryapproachandespecially,toitsformeremployeeRobertSternforshowingmetheworldofDocker.IwouldalsoliketomakeaspecialmentionofPatroklosPapapetrouforhistrustandhelpinorganizingContinuousDeliveryworkshopsinGreece.Lastbutnottheleast,thankstomymom,dad,andbrotherforbeingsosupportive.
AbouttheReviewerMichaelPailloncyisadevelopertendingtowardthe'Ops'side,constantlytryingtokeepthingssimpleandasmuchautomatedaspossible.MichaelispassionateabouttheDevOpscultureandhasastrongexperienceinContinuousIntegration,ContinuousDelivery,automation,bigsoftwarefactorymanagementandlovestosharetheexperienceswithothers.
MiteshSoniisanavidlearnerwith10yearsofexperienceintheITindustry.HeisanSCJP,SCWCD,VCP,IBMUrbancode,andIBMBluemixcertifiedprofessional.HelovesDevOpsandcloudcomputingandalsohasaninterestinprogramminginJava.Hefindsdesignpatternsfascinatingandbelievesthat"apictureisworthathousandwords."
Heoccasionallycontributestoetutorialsworld.com.Helovestoplaywithkids,fiddlewithhiscamera,andtakephotographsatIndrodaPark.Heisaddictedtotakingpictureswithoutknowingmanytechnicaldetails.HelivesinthecapitalofMahatmaGandhi'shomestate.
MiteshhasauthoredfollowingbookswithPackt:
DevOpsBootcampImplementingDevOpswithMicrosoftAzureDevOpsforWebDevelopmentJenkinsEssentialsLearningChef
www.PacktPub.comForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.
DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusatservice@packtpub.comformoredetails.
Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.
https://www.packtpub.com/mapt
Getthemostin-demandsoftwareskillswithMapt.MaptgivesyoufullaccesstoallPacktbooksandvideocourses,aswellasindustry-leadingtoolstohelpyouplanyourpersonaldevelopmentandadvanceyourcareer.
Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser
CustomerFeedbackThanksforpurchasingthisPacktbook.AtPackt,qualityisattheheartofoureditorialprocess.Tohelpusimprove,pleaseleaveusanhonestreviewonthisbook'sAmazonpageathttps://www.amazon.com/dp/1787125238.
Ifyou'dliketojoinourteamofregularreviewers,youcane-mailusatcustomerreviews@packtpub.com.WeawardourregularreviewerswithfreeeBooksandvideosinexchangefortheirvaluablefeedback.Helpusberelentlessinimprovingourproducts!
TomywonderfulwifeMaria,forallofherlove,wisdom,andsmile.
TableofContents
PrefaceWhatthisbookcovers
Whatyouneedforthisbook
Whothisbookisfor
Conventions
Readerfeedback
CustomersupportDownloadingtheexamplecode
Downloadingthecolorimagesofthisbook
Errata
Piracy
Questions
1. IntroducingContinuousDeliveryWhatisContinuousDelivery?
ThetraditionaldeliveryprocessIntroducingthetraditionaldeliveryprocess
Shortcomingsofthetraditionaldeliveryprocess
BenefitsofContinuousDelivery
Successstories
TheautomateddeploymentpipelineContinuousIntegration
AutomatedacceptancetestingTheAgiletestingmatrix
Thetestingpyramid
Configurationmanagement
PrerequisitestoContinuousDeliveryOrganizationalprerequisites
DevOpsculture
Clientintheprocess
Businessdecisions
Technicalanddevelopmentprerequisites
BuildingtheContinuousDeliveryprocessIntroducingtools
Dockerecosystem
Jenkins
Ansible
GitHub
Java/SpringBoot/Gradle
Theothertools
CreatingacompleteContinuousDeliverysystemIntroducingDocker
ConfiguringJenkins
ContinuousIntegrationPipeline
Automatedacceptancetesting
ConfigurationmanagementwithAnsible/ContinuousDeliverypipeline
ClusteringwithDockerSwarm/AdvancedContinuousDelivery
Summary
2. IntroducingDockerWhatisDocker?
Containerizationversusvirtualization
TheneedforDockerEnvironment
Isolation
Organizingapplications
Portability
Kittensandcattle
Alternativecontainerizationtechnologies
DockerinstallationPrerequisitesforDocker
InstallingonalocalmachineDockerforUbuntu
DockerforLinux
DockerforMac
DockerforWindows
TestingDockerinstallation
InstallingonaserverDedicatedserver
RunningDockerhelloworld>Dockercomponents
Dockerclientandserver
Dockerimagesandcontainers
Dockerapplications
BuildingimagesDockercommit
Dockerfile
CompleteDockerapplicationWritetheapplication
Preparetheenvironment
Buildtheimage
Runtheapplication
Environmentvariables
Dockercontainerstates
DockernetworkingRunningservices
Containernetworks
Exposingcontainerports
Automaticportassignment
UsingDockervolumes
UsingnamesinDockerNamingcontainers
Taggingimages
DockercleanupCleaningupcontainers
Cleaningupimages
Dockercommandsoverview
Exercises
Summary
3. ConfiguringJenkinsWhatisJenkins?
JenkinsinstallationRequirementsforinstallation
InstallingonDocker
InstallingwithoutDocker
Initialconfiguration
Jenkinshelloworld
JenkinsarchitectureMasterandslaves
ScalabilityVerticalscaling
Horizontalscaling
Testandproductioninstances
Samplearchitecture
ConfiguringagentsCommunicationprotocols
SettingagentsPermanentagents
Configuringpermanentagents
Understandingpermanentagents
PermanentDockeragentsConfiguringpermanentDockeragents
UnderstandingpermanentDockeragents
JenkinsSwarmagentsConfiguringJenkinsSwarmagents
UnderstandingJenkinsSwarmagents
DynamicallyprovisionedDockeragentsConfiguringdynamicallyprovisionedDockeragents
UnderstandingdynamicallyprovisionedDockeragents
Testingagents
CustomJenkinsimagesBuildingJenkinsslave
BuildingJenkinsmaster
ConfigurationandmanagementPlugins
Security
Backup
BlueOceanUI
Exercises
Summary
4. ContinuousIntegrationPipelineIntroducingpipelines
Pipelinestructure
Multi-stageHelloWorld
PipelinesyntaxSections
Directives
Steps
CommitpipelineCheckout
CreatingaGitHubrepository
Creatingacheckoutstage
CompileCreatingaJavaSpringBootproject
PushingcodetoGitHub
Creatingacompilestage
UnittestCreatingbusinesslogic
Writingaunittest
Creatingaunitteststage
JenkinsfileCreatingJenkinsfile
RunningpipelinefromJenkinsfile
Codequalitystages
CodecoverageAddingJaCoCotoGradle
Addingacodecoveragestage
Publishingthecodecoveragereport
StaticcodeanalysisAddingtheCheckstyleconfiguration
Addingastaticcodeanalysisstage
Publishingstaticcodeanalysisreports
SonarQube
TriggersandnotificationsTriggers
External
PollingSCM
Scheduledbuild
NotificationsEmail
Groupchat
Teamspace
TeamdevelopmentstrategiesDevelopmentworkflows
Trunk-basedworkflow
Branchingworkflow
Forkingworkflow
AdoptingContinuousIntegrationBranchingstrategies
Featuretoggles
JenkinsMultibranch
Non-technicalrequirements
Exercises
Summary
5. AutomatedAcceptanceTestingIntroducingacceptancetesting
DockerregistryArtifactrepository
InstallingDockerregistryDockerHub
PrivateDockerregistryInstallingtheDockerregistryapplication
Addingadomaincertificate
Addinganaccessrestriction
OtherDockerregistries
UsingDockerregistryBuildinganimage
Pushingtheimage
Pullingtheimage
AcceptancetestinpipelineTheDockerbuildstage
AddingDockerfile
AddingtheDockerbuildtothepipeline
TheDockerpushstage
AcceptancetestingstageAddingastagingdeploymenttothepipeline
Addinganacceptancetesttothepipeline
Addingacleaningstageenvironment
DockerComposeWhatisDockerCompose?
InstallingDockerCompose
Definingdocker-compose.yml
Usingthedocker-composecommand
Buildingimages
Scalingservices
AcceptancetestingwithDockerComposeUsingamulti-containerenvironment
AddingaRedisclientlibrarytoGradle
AddingaRediscacheconfiguration
AddingSpringBootcaching
Checkingthecachingenvironment
Method1–Jenkins-firstacceptancetestingChangingthestagingdeploymentstage
Changingtheacceptanceteststage
Method2–Docker-firstacceptancetestingCreatingaDockerfileforacceptancetest
Creatingdocker-compose.ymlforacceptancetest
Creatinganacceptancetestscript
Runningtheacceptancetest
Changingtheacceptanceteststage
Comparingmethod1andmethod2
WritingacceptancetestsWritinguser-facingtests
UsingtheacceptancetestingframeworkCreatingacceptancecriteria
Creatingstepdefinitions
Runninganautomatedacceptancetest
Acceptancetest-drivendevelopment
Exercises
Summary
6. ConfigurationManagementwithAnsibleIntroducingconfigurationmanagement
Traitsofgoodconfigurationmanagement
Overviewofconfigurationmanagementtools
InstallingAnsibleAnsibleserverrequirements
Ansibleinstallation
Docker-basedAnsibleclient
UsingAnsibleCreatinginventory
Adhoccommands
PlaybooksDefiningaplaybook
Executingtheplaybook
Playbook'sidempotency
Handlers
Variables
RolesUnderstandingroles
AnsibleGalaxy
DeploymentwithAnsibleInstallingRedis
DeployingawebserviceConfiguringaprojecttobeexecutable
ChangingtheRedishostaddress
Addingcalculatordeploymenttotheplaybook
Runningdeployment
AnsiblewithDockerBenefitsofAnsible
AnsibleDockerplaybookInstallingDocker
RunningDockercontainers
UsingDockerCompose
Exercises
Summary
7. ContinuousDeliveryPipelineEnvironmentsandinfrastructure
TypesofenvironmentProduction
Staging
QA
Development
EnvironmentsinContinuousDelivery
Securingenvironments
NonfunctionaltestingTypesofnonfunctionaltest
Performancetesting
Loadtesting
Stresstesting
Scalabilitytesting
Endurancetesting
Securitytesting
Maintainabilitytesting
Recoverytesting
Nonfunctionalchallenges
ApplicationversioningVersioningstrategies
VersioningintheJenkinspipeline
CompleteContinuousDeliverypipelineInventory
Acceptancetestingenvironment
Release
Smoketesting
CompleteJenkinsfile
Exercises
Summary
8. ClusteringwithDockerSwarmServerclustering
Introducingserverclustering
IntroducingDockerSwarm
DockerSwarmfeaturesoverview
DockerSwarminpracticeSettingupaSwarm
Addingworkernodes
Deployingaservice
Scalingservice
Publishingports
AdvancedDockerSwarmRollingupdates
Drainingnodes
Multiplemanagernodes
Schedulingstrategy
DockerComposewithDockerSwarmIntroducingDockerStack
UsingDockerStackSpecifyingdocker-compose.yml
Runningthedockerstackcommand
Verifyingtheservicesandcontainers
Removingthestack
AlternativeclustermanagementsystemsKubernetes
ApacheMesos
Comparingfeatures
ScalingJenkinsDynamicslaveprovisioning
JenkinsSwarm
ComparisonofdynamicslaveprovisioningandJenkinsSwarm
Exercises
Summary
9. AdvancedContinuousDeliveryManagingdatabasechanges
UnderstandingschemaupdatesIntroducingdatabasemigrations
UsingFlywayConfiguringFlyway
DefiningtheSQLmigrationscript
Accessingdatabase
ChangingdatabaseinContinuousDeliveryBackwards-compatiblechanges
Non-backwards-compatiblechangesAddinganewcolumntothedatabase
Changingthecodetousebothcolumns
Mergingthedatainbothcolumns
Removingtheoldcolumnfromthecode
Droppingtheoldcolumnfromthedatabase
Separatingdatabaseupdatesfromcodechanges
Avoidingshareddatabase
PreparingtestdataUnittesting
Integration/acceptancetesting
Performancetesting
PipelinepatternsParallelizingpipelines
ReusingpipelinecomponentsBuildparameters
SharedlibrariesCreatingasharedlibraryproject
ConfigurethesharedlibraryinJenkins
UsesharedlibraryinJenkinsfile
Rollingbackdeployments
Addingmanualsteps
ReleasepatternsBlue-greendeployment
Canaryrelease
WorkingwithlegacysystemsAutomatingbuildanddeployment
Automatingtests
Refactoringandintroducingnewfeatures
Understandingthehumanelement
Exercises
Summary
BestpracticesPractice1–ownprocesswithintheteam!
Practice2–automateeverything!
Practice3–versioneverything!
Practice4–usebusinesslanguageforacceptancetests!
Practice5–bereadytorollback!
Practice6–don'tunderestimatetheimpactofpeople
Practice7–buildintraceability!
Practice8–integrateoften!
Practice9–buildbinariesonlyonce!
Practice10–releaseoften!
PrefaceI'veobservedsoftwaredeliveryprocessesforyears.IwrotethisbookbecauseIknowhowmanypeoplestillstrugglewithreleasesandgetfrustratedafterspendingdaysandnightsongettingtheirproductsintoproduction.Thisallhappenseventhoughalotofautomationtoolsandprocesseshavebeendevelopedthroughouttheyears.AfterIsawforthefirsttimehowsimpleandeffectivetheContinuousDeliveryprocesswas,Iwouldnevercomebacktothetedioustraditionalmanualdeliverycycle.ThisbookisaresultofmyexperienceandanumberofContinuousDeliveryworkshopsIconducted.IsharethemodernapproachusingJenkins,Docker,andAnsible;however,thisbookismorethanjustthetools.ItpresentstheideaandthereasoningbehindContinuousDelivery,andwhat'smostimportant,mymainmessagetoeveryoneImeet:ContinuousDeliveryprocessissimple,useit!
WhatthisbookcoversChapter1,IntroducingContinuousDelivery,presentshowcompaniestraditionallydelivertheirsoftwareandexplainstheideatoimproveitusingtheContinuousDeliveryapproach.Thischapteralsodiscussestheprerequisitesforintroducingtheprocessandpresentsthesystemthatwillbebuiltthroughoutthebook.
Chapter2,IntroducingDocker,explainstheideaofcontainerizationandthefundamentalsoftheDockertool.ThischapteralsoshowshowtouseDockercommands,packageanapplicationasaDockerimage,publishDockercontainer'sports,anduseDockervolumes.
Chapter3,ConfiguringJenkins,presentshowtoinstall,configure,andscaleJenkins.ThischapteralsoshowshowtouseDockertosimplifyJenkinsconfigurationandtoenabledynamicslaveprovisioning.
Chapter4,ContinuousIntegrationPipeline,explainstheideaofpipeliningandintroducestheJenkinsfilesyntax.ThischaptersalsoshowshowtoconfigureacompleteContinuousIntegrationpipeline.
Chapter5,AutomatedAcceptanceTesting,presentstheideaandimplementationofacceptancetesting.Thischaptersalsoexplainsthemeaningofartifactrepositories,theorchestrationusingDockerCompose,andframeworksforwritingBDD-orientedacceptancetests.
Chapter6,ConfigurationManagementwithAnsible,introducestheconceptofconfigurationmanagementanditsimplementationusingAnsible.ThechapteralsoshowshowtouseAnsibletogetherwithDockerandDockerCompose.
Chapter7,ContinuousDeliveryPipeline,combinesalltheknowledgefromthepreviouschaptersinordertobuildthecompleteContinuousDeliveryprocess.Thechapteralsodiscussesvariousenvironmentsandtheaspectsofnonfunctionaltesting.
Chapter8,ClusteringwithDockerSwarm,explainstheconceptofserverclusteringandtheimplementationusingDockerSwarm.Thechapteralsocompares
alternativeclusteringtools(KubernetesandApacheMesos)andexplainshowtouseclusteringfordynamicJenkinsagents.
Chapter9,AdvancedContinuousDelivery,presentsamixtureofdifferentaspectsrelatedtotheContinuousDeliveryprocess:databasemanagement,parallelpipelinesteps,rollbackstrategies,legacysystems,andzero-downtimedeployments.ThechapteralsoincludesbestpracticesfortheContinuousDeliveryprocess.
WhatyouneedforthisbookDockerrequiresthe64-bitLinuxoperatingsystem.AllexamplesinthisbookhavebeendevelopedusingUbuntu16.04,butanyotherLinuxsystemwiththekernelversion3.10oraboveissufficient.
WhothisbookisforThisbookisfordevelopersandDevOpswhowouldliketoimprovetheirdeliveryprocess.Nopriorknowledgeisrequiredtounderstandthisbook.
ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.
Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:dockerinfo
Ablockofcodeissetasfollows:
pipeline{
agentany
stages{
stage("Hello"){
steps{
echo'HelloWorld'
}
}
}
}
Whenwewishtodrawyourattentiontoaparticularpartofacodeblock,therelevantlinesoritemsaresetinbold:
FROMubuntu:16.04
RUNapt-getupdate&&\
apt-getinstall-ypython
Anycommand-lineinputoroutputiswrittenasfollows:
$dockerimages
REPOSITORYTAGIMAGEIDCREATEDSIZE
ubuntu_with_pythonlatestd6e85f39f5b7Aboutaminuteago202.6MB
ubuntu_with_git_and_jdklatest8464dc10abbb3minutesago610.9MB
Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenusordialogboxesforexample,appearinthetextlikethis:"ClickonNewItem".
Warningsorimportantnotesappearinaboxlikethis.
IfyourDockerdaemonisruninsidethecorporatenetwork,you
havetoconfiguretheHTTPproxy.Thedetaileddescriptioncanbefoundathttps://docs.docker.com/engine/admin/systemd/.
Tipsandtricksappearlikethis.
TheinstallationguidesforallsupportedoperatingsystemsandcloudplatformscanbefoundontheofficialDockerpage,https://docs.docker.com/engine/installation/.
ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedormayhavedisliked.Readerfeedbackisimportantforustodeveloptitlesthatyoureallygetthemostoutof.
Tosendusgeneralfeedback,[email protected],andmentionthebooktitleviathesubjectofyourmessage.
Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideonwww.packtpub.com/authors.
CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.
DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
Youcandownloadthecodefilesbyfollowingthesesteps:
1. Loginorregistertoourwebsiteusingyoure-mailaddressandpassword.2. HoverthemousepointerontheSUPPORTtabatthetop.3. ClickonCodeDownloads&Errata.4. EnterthenameofthebookintheSearchbox.5. Selectthebookforwhichyou'relookingtodownloadthecodefiles.6. Choosefromthedrop-downmenuwhereyoupurchasedthisbookfrom.7. ClickonCodeDownload.
Oncethefileisdownloaded,pleasemakesurethatyouunziporextractthefolderusingthelatestversionof:
WinRAR/7-ZipforWindowsZipeg/iZip/UnRarXforMac7-Zip/PeaZipforLinux
ThecodebundleforthebookisalsohostedonGitHubathttps://github.com/PacktPublishing/Continuous-Delivery-with-Docker-and-Jenkins.Wealsohaveothercodebundlesfromourrichcatalogofbooksandvideosavailableathttps://github.com/PacktPublishing/.Checkthemout!
DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefromhttps://www.packtpub.com/sites/default/files/downloads/ContinuousDeliverywithDockerandJenkins_ColorImages.pdf.
ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyouwouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheerratasubmissionformlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedonourwebsite,oraddedtoanylistofexistingerrata,undertheErratasectionofthattitle.Anyexistingerratacanbeviewedbyselectingyourtitlefromhttp://www.packtpub.com/support.
PiracyPiracyofcopyrightmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.Ifyoucomeacrossanyillegalcopiesofourworks,inanyform,ontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.
Pleasecontactusatcopyright@packtpub.comwithalinktothesuspectedpiratedmaterial.
Weappreciateyourhelpinprotectingourauthors,andourabilitytobringyouvaluablecontent.
QuestionsYoucancontactusatquestions@packtpub.comifyouarehavingaproblemwithanyaspectofthebook,andwewilldoourbesttoaddressit.
IntroducingContinuousDelivery
Thecommonproblemfacedbymostdevelopersishowtoreleasetheimplementedcodequicklyandsafely.Thedeliveryprocessusedtraditionallyis,however,asourceofpitfallsandusuallyleadstothedisappointmentofbothdevelopersandclients.ThischapterpresentstheideaoftheContinuousDeliveryapproachandprovidesthecontextfortherestofthebook.
Thischaptercoversthefollowingpoints:
IntroducingthetraditionaldeliveryprocessanditsdrawbacksDescribingtheideaofContinuousDeliveryandthebenefitsitbringsComparinghowdifferentcompaniesdelivertheirsoftwareExplainingtheautomateddeploymentpipelineanditsphasesClassifyingdifferenttypesoftestsandtheirplaceintheprocessPointingouttheprerequisitestothesuccessfulContinuousDeliveryprocessPresentingtoolsthatwillbeusedthroughoutthebookShowingthecompletesystemthatwillbebuiltthroughoutthebook
WhatisContinuousDelivery?ThemostaccuratedefinitionoftheContinuousDeliveryisstatedbyJezHumbleandreadsasfollows:"ContinuousDeliveryistheabilitytogetchangesofalltypes—includingnewfeatures,configurationchanges,bugfixes,andexperiments—intoproduction,orintothehandsofusers,safelyandquicklyinasustainableway."Thatdefinitioncoversthekeypoints.
Tounderstanditbetter,let'simagineascenario.Youareresponsiblefortheproduct,let'ssay,theemailclientapplication.Userscometoyouwithanewrequirement—theywanttosortemailsbysize.Youdecidethatthedevelopmentwilltakearoundoneweek.Whencantheuserexpecttousethefeature?Usually,afterthedevelopmentisdone,youhandoverthecompletedfeaturefirsttotheQAteamandthentotheoperationsteam,whichtakesadditionaltimerangingfromdaystomonths.Therefore,eventhoughthedevelopmenttookonlyoneweek,theuserreceivesitinacoupleofmonths!TheContinuousDeliveryapproachaddressesthatissuebyautomatingmanualtaskssothattheusercouldreceiveanewfeatureassoonasit'simplemented.
Topresentbetterwhattoautomateandhow,let'sstartbydescribingthedeliveryprocessthatiscurrentlyusedformostsoftwaresystems.
ThetraditionaldeliveryprocessThetraditionaldeliveryprocess,asthenamesuggests,hasbeeninplaceformanyyearsnowandisimplementedinmostITcompanies.Let'sdefinehowitworksandcommentonitsshortcomings.
IntroducingthetraditionaldeliveryprocessAnydeliveryprocessbeginswiththerequirementsdefinedbyacustomerandendsupwiththereleaseontheproduction.Thedifferencesareinbetween.Traditionally,itlooksaspresentedinthefollowingreleasecyclediagram:
ThereleasecyclestartswiththerequirementsprovidedbytheProductOwner,whorepresentstheCustomer(stakeholders).Thentherearethreephases,duringwhichtheworkispassedbetweendifferentteams:
Development:Here,thedevelopers(sometimestogetherwithbusinessanalysts)workontheproduct.TheyoftenuseAgiletechniques(ScrumorKanban)toincreasethedevelopmentvelocityandtoimprovethecommunicationwiththeclient.Demosessionsareorganizedtoobtainacustomer'squickfeedback.Allgooddevelopmenttechniques(liketest-drivendevelopmentorextremeprogrammingpractices)arewelcome.Aftertheimplementationiscompleted,thecodeispassedtotheQAteam.QualityAssurance:ThisphaseisusuallycalledUserAcceptanceTesting(UAT)anditrequiresacodefreezeonthetrunkcodebase,sothatnonewdevelopmentwouldbreakthetests.TheQAteamperformsasuiteofIntegrationTesting,AcceptanceTesting,andNon-functionalTesting(performance,recovery,security,andsoon).Anybugthatisdetectedgoes
backtothedevelopmentteam,sodevelopersusuallyalsohavetheirhandsfullofwork.AftertheUATphaseiscompleted,theQAteamapprovesthefeaturesthatareplannedforthenextrelease.Operations:Thelastphase,usuallytheshortestone,meanspassingthecodetotheOperationsteam,sothattheycanperformthereleaseandmonitortheproduction.Ifanythinggoeswrong,theycontactdeveloperstohelpwiththeproductionsystem.
Thelengthofthereleasecycledependsonthesystemandtheorganization,butitusuallyrangesfromaweektoafewmonths.ThelongestI'veheardaboutwasoneyear.ThelongestIworkedwithwasquarterly-basedandeachparttookasfollows:development-1.5months,UAT-1monthand3weeks,release(andstrictproductionmonitoring)-1week.
ThetraditionaldeliveryprocessiswidelyusedintheITindustryandit'sprobablynotthefirsttimeyou'vereadaboutsuchanapproach.Nevertheless,ithasanumberofdrawbacks.Let'slookatthemexplicitytounderstandwhyweneedtostriveforsomethingbetter.
ShortcomingsofthetraditionaldeliveryprocessThemostsignificantshortcomingsofthetraditionaldeliveryprocessincludethefollowing:
Slowdelivery:Here,thecustomerreceivestheproductlongaftertherequirementswerespecified.Itresultsintheunsatisfactorytimetomarketanddelaysofthecustomer'sfeedback.Longfeedbackcycle:Thefeedbackcycleisnotonlyrelatedtocustomers,butalsotodevelopers.ImaginethatyouaccidentallycreatedabugandyoulearnaboutitduringtheUATphase.Howlongdoesittaketofixsomethingyouworkedontwomonthsago?Evenminorbugscanconsumeweeks.Lackofautomation:Rarereleasesdon'tencouragetheautomation,whichleadstounpredictablereleases.Riskyhotfixes:Hotfixescan'tusuallywaitforthefullUATphase,sotheytendtobetesteddifferently(theUATphaseisshortened)ornottestedatall.Stress:Unpredictablereleasesarestressfulfortheoperationsteam.What'smore,thereleasecycleisusuallytightlyscheduledwhichimposesanadditionalstressondevelopersandtesters.Poorcommunication:Workpassedfromoneteamtoanotherrepresentsthewaterfallapproach,inwhichpeoplestarttocareonlyabouttheirpart,ratherthanthecompleteproduct.Incaseanythinggoeswrong,thatusuallyleadstotheblaminggameinsteadofcooperation.Sharedresponsibility:NoteamtakestheresponsibilityfortheproductfromAtoZ.Fordevelopers:"done"meansthatrequirementsareimplemented.Fortesters:"done"meansthatthecodeistested.Foroperations:"done"meansthatthecodeisreleased.
Lowerjobsatisfaction:Eachphaseisinterestingforadifferentteam,butotherteamsneedtosupporttheprocess.Forexample,thedevelopmentphaseisinterestingfordevelopersbut,duringtwootherphases,theystillneedtofixbugsandsupporttherelease,whichusuallyisnotinterestingfor
thematall.
Thesedrawbacksrepresentjustatipoftheicebergofthechallengesrelatedtothetraditionaldeliveryprocess.Youmayalreadyfeelthattheremustbeabetterwaytodevelopthesoftwareandthisbetterwayis,obviously,theContinuousDeliveryapproach.
BenefitsofContinuousDelivery“Howlongwouldittakeyourorganizationtodeployachangethatinvolvesjustonesinglelineofcode?Doyoudothisonarepeatable,reliablebasis?"ThesearethefamousquestionsfromMaryandTomPoppendieck(authorsofImplementingLeanSoftwareDevelopment),whichhavebeenquotedmanytimesbyJezHumbleandotherauthors.Actually,theanswertothesequestionsistheonlyvalidmeasurementofthehealthofyourdeliveryprocess.
Tobeabletodelivercontinuously,andnottospendafortuneonthearmyofoperationsteamsworking24/7,weneedautomation.Thatiswhy,inshort,ContinuousDeliveryisallaboutchangingeachphaseofthetraditionaldeliveryprocessintoasequenceofscripts,calledtheautomateddeploymentpipelineortheContinuousDeliverypipeline.Then,ifnomanualstepsarerequired,wecanruntheprocessaftereverycodechangeand,therefore,delivertheproductcontinuouslytotheusers.
ContinuousDeliveryletsusgetridofthetediousreleasecycleand,therefore,bringsthefollowingbenefits:
Fastdelivery:Timetomarketissignificantlyreducedascustomerscanusetheproductassoonasthedevelopmentiscompleted.Remember,thesoftwaredeliversnorevenueuntilitisinthehandsofitsusers.Fastfeedbackcycle:Imagineyoucreatedabuginthecode,whichgoesintotheproductionthesameday.Howmuchtimedoesittaketofixsomethingyouworkedonthesameday?Probablynotmuch.This,togetherwiththequickrollbackstrategy,isthebestwaytokeeptheproductionstable.Low-riskreleases:Ifyoureleaseonadailybasis,thentheprocessbecomesrepeatableandthereforemuchsafer.Asthesayinggoes,"Ifithurts,doitmoreoften."Flexiblereleaseoptions:Incaseyouneedtoreleaseimmediately,everythingisalreadyprepared,sothereisnoadditionaltime/costassociatedwiththereleasedecision.
Needlesstosay,wecouldachieveallthebenefitssimplybyeliminatingalldeliveryphasesandproceedingwiththedevelopmentdirectlyontheproduction.Itwould,however,causethequalitytodecline.Actually,thewholedifficultyofintroducingContinuousDeliveryistheconcernthatthequalitywoulddecreasetogetherwitheliminatingmanualsteps.Inthisbook,wewillshowhowtoapproachitinasafemannerandexplainwhy,contrarytocommonbeliefs,theproductsdeliveredcontinuouslyhavefewerbugsandarebetteradjustedtothecustomer'sneeds.
SuccessstoriesMyfavoritestoryonContinuousDeliverywastoldbyRolfRussellatoneofhistalks.Itgoesasfollows.In2005,YahooacquiredFlickranditwasaclashoftwoculturesinthedeveloper'sworld.Flickr,bythattime,wasacompanywiththestart-upapproachinmind.Yahoo,onthecontrary,wasahugecorporationwithstrictrulesandthesafety-firstattitude.Theirreleaseprocessesdifferedalot.WhileYahoousedthetraditionaldeliveryprocess,Flickrreleasedmanytimesaday.Everychangeimplementedbydeveloperswentontheproductionthesameday.Theyevenhadafooteratthebottomoftheirpageshowingthetimeofthelastreleaseandtheavatarsofthedeveloperswhodidthechanges.
Yahoodeployedrarelyandeachreleasebroughtalotofchangeswelltestedandprepared.Flickrworkedinverysmallchunks,eachfeaturewasdividedintosmallincrementalpartsandeachpartwasdeployedquicklytotheproduction.Thedifferenceispresentedinthefollowingdiagram:
Youcanimaginewhathappenedwhenthedevelopersfromtwocompaniesmet.YahooobviouslytreatedFlickr'scolleaguesasjuniorirresponsibledevelopers,"abunchofsoftwarecowboyswhodon'tknowwhattheyaredoing."So,thefirstthingtheywantedtochangewastoaddaQAteamandtheUATphaseintoFlickr'sdeliveryprocess.Beforetheyappliedthechange,however,Flickr'sdevelopershadonlyonewish.TheyaskedtoevaluatethemostreliableproductsinthewholeYahoocompany.WhatasurprisewhenithappenedthatofallthesoftwareinYahoo,Flickrhadthelowestdowntime.TheYahooteamdidn't
understanditatfirst,butletFlickrstaywiththeircurrentprocessanyway.Afterall,theywereengineers,sotheevaluationresultwasconclusive.Onlyaftersometime,theyrealizedthattheContinuousDeliveryprocesscanbebeneficialforallproductsinYahooandtheystartedtograduallyintroduceiteverywhere.
Themostimportantquestionofthestoryremains-howwasitpossiblethatFlickrwasthemostreliablesystem?Actually,thereasonforthatfactwaswhatwealreadymentionedintheprevioussections.Areleaseislessriskyif:
ThedeltaofcodechangesissmallTheprocessisrepeatable
Thatiswhy,eventhoughthereleaseitselfisadifficultactivity,itismuchsaferwhendonefrequently.
ThestoryofYahooandFlickrisonlyanexampleofmanysuccessfulcompaniesforwhichtheContinuousDeliveryprocessprovedtoberight.Someofthemevenproudlysharedetailsfromtheirsystems,asfollows:
Amazon:In2011,theyannouncedreaching11.6seconds(onaverage)betweendeploymentsFacebook:In2013,theyannounceddeploymentofcodechangestwiceadayHubSpot:In2013,theyannounceddeployment300timesadayAtlassian:In2016,theypublishedasurveystatingthat65%oftheircustomerspracticecontinuousdelivery
YoucanreadmoreabouttheresearchontheContinuousDeliveryprocessandindividualcasestudiesathttps://continuousdelivery.com/evidence-case-studies/.
Keepinmindthatthestatisticsgetbettereveryday.However,evenwithoutanynumbers,justimagineaworldinwhicheverylineofcodeyouimplementgoessafelyintotheproduction.Clientscanreactquicklyandadjusttheirrequirements,developersarehappybecausetheydon'thavetosolvethatmanybugs,managersaresatisfiedbecausetheyalwaysknowwhatisthecurrentstateofwork.Afterall,remember,theonlytruemeasureofprogressisthereleasedsoftware.
TheautomateddeploymentpipelineWealreadyknowwhattheContinuousDeliveryprocessisandwhyweuseit.Inthissection,wedescribehowtoimplementit.
Let'sstartbyemphasizingthateachphaseinthetraditionaldeliveryprocessisimportant.Otherwise,itwouldneverhavebeencreatedinthefirstplace.Noonewantstodeliversoftwarewithouttestingitfirst!TheroleoftheUATphaseistodetectbugsandtoensurethatwhatdeveloperscreatediswhatthecustomerwanted.Thesameappliestotheoperationsteam—thesoftwaremustbeconfigured,deployedtotheproduction,andmonitored.That'soutofthequestion.So,howdoweautomatetheprocesssothatwepreserveallthephases?Thatistheroleoftheautomateddeploymentpipeline,whichconsistsofthreestagesaspresentedinthefollowingdiagram:
Theautomateddeploymentpipelineisasequenceofscriptsthatisexecutedaftereverycodechangecommittedtotherepository.Iftheprocessissuccessful,itendsupwiththedeploymenttotheproductionenvironment.
Eachstepcorrespondstoaphaseinthetraditionaldeliveryprocessasfollows:
ContinuousIntegration:ThischeckstomakesurethatthecodewrittenbydifferentdevelopersintegratestogetherAutomatedAcceptanceTesting:ThisreplacesthemanualQAphaseandchecksifthefeaturesimplementedbydevelopersmeettheclient'srequirementsConfigurationManagement:Thisreplacesthemanualoperationsphase-
configurestheenvironmentanddeploysthesoftware
Let'stakeadeeperlookateachphasetounderstandwhatisitsresponsibilityandwhatstepsitincludes.
ContinuousIntegrationTheContinuousIntegrationphaseprovidesthefirstfeedbacktodevelopers.Itchecksoutthecodefromtherepository,compilesit,runsunittests,andverifiesthecodequality.Ifanystepfails,thepipelineexecutionisstoppedandthefirstthingthedevelopersshoulddoisfixtheContinuousIntegrationbuild.Theessentialaspectofthephaseistime;itmustbeexecutedinatimelymanner.Forexample,ifthisphasetookanhourtocompletethenthedeveloperswouldcommitthecodefaster,whichwouldresultintheconstantlyfailingpipeline.
TheContinuousIntegrationpipelineisusuallythestartingpoint.SettingitupissimplebecauseeverythingisdonewithinthedevelopmentteamandnoagreementwiththeQAandoperationsteamsisnecessary.
AutomatedacceptancetestingTheautomatedacceptancetestingphaseisasuiteoftestswrittentogetherwiththeclient(andQAs)thatissupposedtoreplacethemanualUATstage.Itactsasaqualitygatetodecidewhetheraproductisreadyforthereleaseornot.Ifanyoftheacceptancetestsfail,thenthepipelineexecutionisstoppedandnofurtherstepsarerun.ItpreventsmovementtotheConfigurationManagementphaseandthereforetherelease.
Thewholeideaofautomatingtheacceptancephaseistobuildthequalityintotheproductinsteadofverifyingitlater.Inotherwords,whenadevelopercompletestheimplementation,thesoftwareisdeliveredalreadytogetherwithacceptancetestswhichverifythatthesoftwareiswhattheclientwanted.Thatisalargeshiftinthinkingabouttestingsoftware.Thereisnolongerasingleperson(orteam)whoapprovestherelease,buteverythingdependsonpassingtheacceptancetestsuite.ThatiswhycreatingthisphaseisusuallythemostdifficultpartoftheContinuousDeliveryprocess.Itrequiresaclosecooperationwiththeclientandcreatingtestsatthebeginning(notattheend)oftheprocess.
Introducingautomatedacceptancetestsisespeciallychallenginginthecaseoflegacysystems.WedescribemoreonthattopicinChapter9,AdvancedContinuousDelivery.
ThereisusuallyalotofconfusionaboutthetypesoftestsandtheirplaceintheContinuousDeliveryprocess.It'salsooftenunclearhowtoautomateeachtype,whatshouldbethecoverage,andwhatshouldbetheroleoftheQAteaminthewholedevelopmentprocess.Let'sclarifyitusingtheAgiletestingmatrixandthetestingpyramid.
TheAgiletestingmatrixBrianMarick,inaseriesofhisblogposts,madeaclassificationofsoftwaretestsinaformoftheso-calledagiletestingmatrix.Itplacestestsintwodimensions:businessortechnologyfacingandsupportprogrammersorcritiquetheproduct.Let'shavealookatthatclassification:
Let'scommentbrieflyoneachtypeoftest:
AcceptanceTesting(automated):Theseareteststhatrepresentfunctionalrequirementsseenfromthebusinessperspective.Theyarewrittenintheformofstoriesorexamplesbyclientsanddeveloperstoagreeonhowthesoftwareshouldwork.UnitTesting(automated):Theseareteststhathelpdeveloperstoprovidethehigh-qualitysoftwareandminimizethenumberofbugs.ExploratoryTesting(manual):Thisisthemanualblack-boxtesting,whichtriestobreakorimprovethesystem.Non-functionalTesting(automated):Theseareteststhatrepresentsystempropertiesrelatedtotheperformance,scalability,security,andsoon.
ThisclassificationanswersoneofthemostimportantquestionsabouttheContinuousDeliveryprocess:whatistheroleofaQAintheprocess?
ManualQAsperformtheexploratorytesting,sotheyplaywiththesystem,trytobreakit,askquestions,thinkaboutimprovements.AutomationQAshelpwithnonfunctionalandacceptancetesting,forexample,theywritecodetosupport
loadtesting.Ingeneral,QAsdon'thavetheirspecialplaceinthedeliveryprocess,butratheraroleinthedevelopmentteam.
IntheautomatedContinuousDeliveryprocess,thereisnolongeraplaceformanualQAswhoperformrepetitivetasks.
Youmaylookattheclassificationandwonderwhyyouseenointegrationteststhere.WherearetheyuptoBrianMarickandwheretoputthemintheContinuousDeliverypipeline?
Toexplainitwell,wefirstneedtomentionthatthemeaningofanintegrationtestdiffersdependingonthecontext.For(micro)servicearchitecture,theyusuallymeanexactlythesameastheacceptancetesting,asservicesaresmallandneednothingmorethanunitandacceptancetests.Ifyoubuildamodularapplication,thenbyintegrationtestsweusuallymeancomponentteststhatbindmultiplemodules(butnotthewholeapplication)andtestthemtogether.Inthatcase,integrationtestsplacethemselvessomewherebetweenacceptanceandunittests.Theyarewritteninasimilarwayasacceptancetests,butareusuallymoretechnicalandrequiremockingnotonlyexternalservices,butalsointernalmodules.Integrationtests,similartounittests,representthe"code"pointofview,whileacceptancetestsrepresentthe"user"pointofview.ConcerningtheContinuousDeliverypipeline,integrationtestsaresimplyimplementedasaseparatephaseintheprocess.
ThetestingpyramidTheprevioussectionexplainedwhateachtesttyperepresentsintheprocess,butmentionednothingabouthowmanytestsweshoulddevelop.So,whatshouldbethecodecoverageincaseofunittesting?Whataboutacceptancetesting?
Toanswerthesequestions,MikeCohn,inhisbookSucceedingwithAgile:SoftwareDevelopmentUsingScrum,createdaso-calledtestingpyramid.Let'slookatthediagramtounderstanditwell.
Whenwemoveupthepyramid,thetestsbecomeslowerandmoreexpensivetocreate.Theyoftenrequiretouchinguserinterfaceandhiringaseparatetestautomationteam.Thatiswhyacceptancetestsshouldnottarget100%coverage.Onthecontrary,theyshouldbefeature-orientedandverifyonlyselectedtestscenarios.Otherwise,wewouldspendafortuneonthetestdevelopmentandmaintenance,andourContinuousDeliverypipelinebuildwouldtakeagestoexecute.
Thecaseisdifferentatthebottomofthepyramid.Unittestsarecheapandfast,soweshouldstrivefor100%codecoverage.Theyarewrittenbydevelopersandprovidingthemshouldbeastandardprocedureforanymatureteam.
Ihopethattheagiletestingmatrixandthetestingpyramidclarifiedtheroleandtheimportanceofacceptancetesting.
Let'smovetothelastphaseoftheContinuousDeliveryprocess,configurationmanagement.
ConfigurationmanagementTheconfigurationmanagementphaseisresponsiblefortrackingandcontrollingchangesinthesoftwareanditsenvironment.Itconcernstakingcareofpreparingandinstallingthenecessarytools,scalingthenumberofserviceinstancesandtheirdistribution,infrastructureinventory,andalltasksrelatedtotheapplicationdeployment.
Configurationmanagementisasolutiontotheproblemsposedbymanuallydeployingandconfiguringapplicationsontheproduction.Suchcommonpracticeresultsinanissuewherebywenolongerknowswhereeachserviceisrunningandwithwhatproperties.Configurationmanagementtools(suchasAnsible,Chef,orPuppet)enablestoringconfigurationfilesintheversioncontrolsystemandtrackingeverychangethatwasmadeontheproductionservers.
Anadditionalefforttoreplacemanualtasksoftheoperationsteamistotakecareofapplicationmonitoring.Thatisusuallydonebystreaminglogsandmetricsoftherunningsystemstoacommondashboard,whichismonitoredbydevelopers(ortheDevOpsteam,asexplainedinthenextsection).
PrerequisitestoContinuousDeliveryTherestofthebookisdedicatedtotechnicaldetailsonhowtoimplementasuccessfulContinuousDeliverypipeline.Thesuccessoftheprocess,however,dependsnotonlyonthetoolswepresentthroughoutthebook.Inthissection,wetakeaholisticlookatthewholeprocessanddefinetheContinuousDeliveryrequirementsinthreeareas:
Yourorganization'sstructureanditsimpactonthedevelopmentprocessYourproductsandtheirtechnicaldetailsYourdevelopmentteamandthepracticesyouuse
OrganizationalprerequisitesThewayyourorganizationworkshasahighimpactonthesuccessofintroducingtheContinuousDeliveryprocess.It'sabitsimilartointroducingScrum.ManyorganizationswouldliketousetheAgileprocess,buttheydon'tchangetheirculture.Youcan'tuseScruminyourdevelopmentteamunlesstheorganization'sstructureisadjustedtothat.Forexample,youneedaproductowner,stakeholders,andmanagementthatunderstandsthatnorequirementchangesarepossibleduringthesprint.Otherwise,evenwithgoodwill,youwon'tmakeit.ThesameappliestotheContinuousDeliveryprocess;itrequiresanadjustmentofhowtheorganizationisstructured.Let'shavealookatthreeaspects:theDevOpsculture,aclientintheprocess,andbusinessdecisions.
DevOpscultureAlongtimeago,whensoftwarewaswrittenbyindividualsormicroteams,therewasnoclearseparationbetweenthedevelopment,qualityassurance,andoperations.Apersondevelopedthecode,testedit,andthenputitintotheproduction.Ifanythingwentwrong,thesamepersoninvestigatedtheissue,fixedit,andredeployedtotheproduction.Thewaythedevelopmentisorganizednowchangedgradually,whensystemsbecamelargeranddevelopmentteamsgrew.Then,engineersstartedtobecomespecializedinonearea.Thatmadeperfectsense,becausespecializationcausedaboostintheproductivity.However,thesideeffectwasthecommunicationoverhead.Itisespeciallyvisibleifdevelopers,QAs,andoperationsareunderseparatedepartmentsintheorganization,sitindifferentbuildings,orareoutsourcedtodifferentcountries.SuchorganizationstructureisnogoodfortheContinuousDeliveryprocess.Weneedsomethingbetter,weneedtoadapttheso-calledDevOpsculture.
DevOpsculturemeans,inasense,comingbacktotheroots.Asinglepersonorateamisresponsibleforallthreeareas,aspresentedinthefollowingdiagram:
Thereasonwhyit'spossibletomovetotheDevOpsmodelwithoutlosingontheproductivityistheautomation.Mostofthetasksrelatedtothequalityassuranceandoperationsaremovedtotheautomateddeliverypipelineandcanbethereforemanagedbythedevelopmentteam.
ADevOpsteamdoesn'tnecessarilyneedtoconsistonlyof
developers.Averycommonscenarioinmanyorganization'sundertransformationistocreateteamswithfourdevelopers,oneQA,andonepersonfromoperations.Theyneed,however,toworkcloselytogether(sitinonearea,havestand-upstogether,workonthesameproduct).
ThecultureofsmallDevOpsteamsaffectsthesoftwarearchitecture.Functionalrequirementshavetobewellseparatedinto(micro)servicesormodules,sothateachteamcantakecareofanindependentpart.
Theimpactoftheorganization'sstructureonthesoftwarearchitecturewasalreadyobservedin1967andformulatedasConway'sLaw:"Anyorganizationthatdesignsasystem(definedbroadly)willproduceadesignwhosestructureisacopyoftheorganization'scommunicationstructure."
ClientintheprocessTheroleofaclient(oraproductowner)slightlychangesduringtheContinuousDeliveryadoption.Traditionally,clientsareinvolvedindefiningrequirements,answeringquestionsfromdevelopers,attendingdemos,andtakingpartintheUATphasetoagreeifwhatwasbuiltiswhattheyhadinmind.
InContinuousDelivery,thereisnoUAT,andaclientisessentialintheprocessofwritingacceptancetests.Forsomeclients,whoalreadywrotetheirrequirementsinatestablemanner,itisnotabigshift.Fortheothers,itmeansachangeinawayofthinkingtomakerequirementsmoretechnical-oriented.
IntheAgileenvironment,someteamsdon'tevenacceptuserstories(requirements)withoutacceptancetestsattached.Suchtechniques,eventhoughtheymaysoundtoostrict,oftenleadtobetterdevelopmentproductivity.
BusinessdecisionsInmostcompanies,thebusinesshasanimpactonthereleaseschedule.Afterall,thedecisionwhatfeaturesaredelivered,andwhen,isrelatedtodifferentdepartmentsofthecompany(forexample,marketing)andcanbestrategicfortheenterprise.Thatiswhythereleaseschedulinghastobereapproachedanddiscussedbetweenthebusinessandthedevelopmentteams.
Obviously,therearetechniquessuchasfeaturetogglesormanualpipelinesteps,whichhelpwithreleasingfeaturesatthespecifiedtime.Wewilldescribethemlaterinthebook.Tobeprecise,thetermContinuousDeliveryisnotthesameasContinuousDeployment.Theformermeansthateachcommittotherepositoryisautomaticallyreleasedtotheproduction.ContinuousDeliveryislessstrictandmeansthateachcommitendsupwithareleasecandidate,soitallowsthelaststep(releasetotheproduction)tobemanual.
Intherestofthebook,wewillusethetermsContinuousDeliveryandContinuousDeploymentinterchangeably.
TechnicalanddevelopmentprerequisitesFromthetechnicalside,thereareafewrequirementstokeepinmind.Wewilldiscussthemthroughoutthebook,solet'sonlymentionthemherewithoutgoingintodetail:
Automatedbuild,test,package,anddeployoperations:Alloperationsneedtobepossibletoautomate.Ifwedealwiththesystemthatisnon-automatable,forexample,duetosecurityreasonsoritscomplexity,thenit'simpossibletocreateafullyautomateddeliverypipeline.Quickpipelineexecution:Thepipelinemustbeexecutedinatimelymanner,preferablyin5-15minutes.Ifourpipelineexecutiontakeshoursordays,thenitwon'tbepossibletorunitaftereverycommittotherepository.Quickfailurerecovery:Apossibilityofthequickrollbackorsystemrecoveryisamust.Otherwise,werisktheproductionhealthduetofrequentreleases.Zero-downtimedeployment:Thedeploymentcannothaveanydowntimesincewereleasemanytimesaday.Trunk-baseddevelopment:Developersmustcheckinregularlyintoonemasterbranch.Otherwise,ifeveryonedevelopsintheirownbranches,theintegrationisrareandthereforethereleasesarerare,whichisexactlytheoppositeofwhatwewanttoachieve.
Wewillwritemoreontheseprerequisitesandhowtoaddressthemthroughoutthebook.Keepingthatinmind,let'smovetothelastsectionofthischapterandintroducewhatsystemweplantobuildinthisbookandwhattoolswewilluseforthatpurpose.
BuildingtheContinuousDeliveryprocessWeintroducedtheidea,benefits,andprerequisiteswithregardstotheContinuousDeliveryprocess.Inthissection,wedescribethetoolsthatwillbeusedthroughoutthebookandtheirplaceinthecompletesystem.
Ifyou'reinterestedmoreintheideaoftheContinuousDeliveryprocess,thenhavealookatanexcellentbookbyJezHumbleandDavidFarley,ContinuousDelivery:ReliableSoftwareReleasesthroughBuild,Test,andDeploymentAutomation.
IntroducingtoolsFirstofall,thespecifictoolisalwayslessimportantthanunderstandingitsroleintheprocess.Inotherwords,anytoolcanbereplacedwithanotheronewhichplaysthesamerole.Forexample,JenkinscanbereplacedwithAtlassianBambooandChiefcanbeusedinsteadofAnsible.Thatiswhyeachchapterbeginswiththegeneraldescriptionofwhysuchatoolisnecessaryandwhatitsroleisinthewholeprocess.Then,theexacttoolisdescribedwithcomparisontoitssubstitutes.Thatformgivesyoutheflexibilitytochoosetherightoneforyourenvironment.
AnotherapproachcouldbetodescribetheContinuousDeliveryprocessonthelevelofideas;however,Istronglybelievethatgivinganexactexamplewiththecodeextract,somethingthatreaderscanrunbythemselves,resultsinamuchbetterunderstandingoftheconcept.
Therearetwowaystoreadthisbook.ThefirstistoreadandunderstandtheconceptsoftheContinuousDeliveryprocess.Thesecondistocreateyourownenvironmentandexecuteallscriptswhilereadingtounderstandthedetails.
Let'shaveaquicklookatthetoolswewillusethroughoutthebook.Inthissection,however,itisonlyabriefintroductionofeachtechnologyandmuchmoredetailispresentedasthisbookgoeson.
Dockerecosystem
Docker,astheclearleaderofthecontainerizationmovement,hasdominatedthesoftwareindustryintherecentyears.Itallowsthepackagingofanapplicationintheenvironment-agnosticimageandthereforetreatsserversasafarmofresources,ratherthanmachinesthatmustbeconfiguredforeachapplication.Dockerwasaclearchoiceforthisbookbecauseitperfectlyfitsthe(micro)serviceworldandtheContinuousDeliveryprocess.
TogetherwithDockercomesadditionaltechnologies,whichareasfollows:
DockerHub:ThisisaregistryforDockerimagesDockerCompose:ThisisatooltodefinemulticontainerDockerapplicationsDockerSwarm:Thisisaclusteringandschedulingtool
JenkinsJenkinsisbyfarthemostpopularautomationserveronthemarket.IthelpstocreateContinuousIntegrationandContinuousDeliverypipelinesand,ingeneral,anyotherautomatedsequenceofscripts.Highlyplugin-oriented,ithasagreatcommunitywhichconstantlyextendsitwithnewfeatures.What'smore,itallowstowritethepipelineascodeandsupportsdistributedbuildenvironments.
AnsibleAnsibleisanautomationtoolthathelpswithsoftwareprovisioning,configurationmanagement,andapplicationdeployment.Itistrendingfasterthananyotherconfigurationmanagementengineandcansoonovertakeitstwomaincompetitors:ChefandPuppet.ItusesagentlessarchitectureandintegratessmoothlywithDocker.
GitHubGitHubisdefinitelythenumberoneofallhostedversioncontrolsystems.Itprovidesaverystablesystem,agreatweb-basedUI,andafreeserviceforpublicrepositories.Havingsaidthat,anysourcecontrolmanagementserviceortoolwillworkwithContinuousDelivery,nomatterifit'sinthecloudorself-hostedandifit'sbasedonGit,SVN,Mercurial,oranyothertool.
Java/SpringBoot/GradleJavahasbeenthemostpopularprogramminglanguageforyears.Thatiswhyitisbeingusedformostcodeexamplesinthisbook.TogetherwithJava,mostcompaniesdevelopwiththeSpringframework,soweusedittocreateasimplewebserviceneededtoexplainsomeconcepts.Gradleisusedasabuildtool.It'sstilllesspopularthanMaven,however,trendingmuchfaster.Asalways,anyprogramminglanguage,framework,orbuildtoolcanbeexchangedandtheContinuousDeliveryprocesswouldstaythesame,sodon'tworryifyourtechnologystackisdifferent.
TheothertoolsCucumberwaschosenarbitrarilyastheacceptancetestingframework.OthersimilarsolutionsareFitnesseandJBehave.ForthedatabasemigrationweuseFlyway,butanyothertoolwoulddo,forexample,Liquibase.
CreatingacompleteContinuousDeliverysystemYoucanlookathowthisbookisorganizedfromtwoperspectives.
Thefirstoneisbasedonthestepsoftheautomateddeploymentpipeline.EachchaptertakesyouclosertothecompleteContinuousDeliveryprocess.Ifyoulookatthenamesofthechapters,someofthemareevennamedlikethepipelinephases:
ContinuousIntegrationpipelineAutomatedacceptancetestingConfigurationmanagementwithAnsible
Therestofthechaptersgivetheintroduction,summary,oradditionalinformationcomplementarytotheprocess.
Thereisalsoasecondperspectivetothecontentofthisbook.Eachchapterdescribesonepieceoftheenvironment,whichinturniswellpreparedfortheContinuousDeliveryprocess.Inotherwords,thebookpresents,stepbystep,technologybytechnology,howtobuildacompletesystem.Tohelpyougetthefeelingofwhatweplantobuildthroughoutthebook,let'snowhavealookathowthesystemwillevolveineachchapter.
Don'tworryifyoudon'tunderstandtheconceptsandtheterminologyatthispoint.Weexplaineverythingfromscratchinthecorrespondingchapters.
IntroducingDockerInChapter2,IntroducingDocker,westartfromthecenterofoursystemandbuildaworkingapplicationpackagedasaDockerimage.Theoutputofthischapteris
presentedinthefollowingdiagram:
Adockerizedapplication(webservice)isrunasacontaineronaDockerHostandisreachableasitwouldrundirectlyonthehostmachine.Thatispossiblethankstoportforwarding(portpublishingintheDocker'sterminology).
ConfiguringJenkinsInChapter3,ConfiguringJenkins,wepreparetheJenkinsenvironment.Thankstothesupportofmultipleagent(slave)nodes,itisabletohandletheheavyconcurrentload.Theresultispresentedinthefollowingdiagram:
TheJenkinsmasteracceptsabuildrequest,buttheexecutionisstartedatoneoftheJenkinsSlave(agent)machines.SuchanapproachprovideshorizontalscalingoftheJenkinsenvironment.
ContinuousIntegrationPipelineInChapter4,ContinuousIntegrationPipeline,weshowhowtocreatethefirstphaseoftheContinuousDeliverypipeline,thecommitstage.Theoutputofthischapteristhesystempresentedinthefollowingdiagram:
TheapplicationisasimplewebservicewritteninJavawiththeSpringBootframework.GradleisusedasabuildtoolandGitHubasthesourcecoderepository.EverycommittoGitHubautomaticallytriggerstheJenkinsbuild,whichusesGradletocompileJavacode,rununittests,andperformadditionalchecks(codecoverage,staticcodeanalysis,andsoon).AftertheJenkinsbuildiscompleted,anotificationissenttothedevelopers.
Afterthischapter,youwillbeabletocreateacompleteContinuousIntegrationpipeline.
AutomatedacceptancetestingInChapter5,AutomatedAcceptanceTesting,wefinallymergethetwotechnologiesfromthebooktitle:DockerandJenkins.Itresultsinthesystem
presentedinthefollowingdiagram:
Theadditionalelementsinthediagramarerelatedtotheautomatedacceptancetestingstage:
DockerRegistry:AftertheContinuousIntegrationphase,theapplicationispackagedfirstintoaJARfileandthenasaDockerimage.ThatimageisthenpushedtotheDockerRegistry,whichactsasastoragefordockerizedapplications.DockerHost:Beforeperformingtheacceptancetestsuite,theapplicationhastobestarted.JenkinstriggersaDockerHostmachinetopullthedockerizedapplicationfromtheDockerRegistryandstartsit.DockerCompose:IfthecompleteapplicationconsistsofmorethanoneDockercontainer(forexample,twowebservices:Application1usingApplication2),thenDockerComposehelpstorunthemtogether.Cucumber:AftertheapplicationisstartedontheDockerHost,JenkinsrunsasuiteofacceptancetestswrittenintheCucumberframework.
ConfigurationmanagementwithAnsible/ContinuousDeliverypipelineInthenexttwochapters,thatis,Chapter6,ConfigurationManagementwithAnsibleandChapter7,ContinuousDeliveryPipeline,wecompletetheContinuousDeliverypipeline.Theoutputistheenvironmentpresentedinthefollowing
diagram:
Ansibletakescareoftheenvironmentsandenablesthedeploymentofthesameapplicationsonmultiplemachines.Asaresult,wedeploytheapplicationtothestagingenvironment,runtheacceptancetestingsuite,andfinallyreleasetheapplicationtotheproductionenvironment,usuallyinmanyinstances(onmultipleDockerHostmachines).
ClusteringwithDockerSwarm/AdvancedContinuousDeliveryInChapter8,ClusteringwithDockerSwarm,wereplacesinglehostsineachoftheenvironmentswithclustersofmachines.Chapter9,AdvancedContinuousDelivery,additionallyaddsdatabasestotheContinuousDeliveryprocess.Thefinalenvironmentcreatedinthisbookispresentedinthefollowingdiagram:
StagingandproductionenvironmentsareequippedwithDockerSwarmclustersandthereforemultipleinstancesoftheapplicationarerunonthecluster.Wedon'thavetothinkanymoreonwhichexactmachineourapplicationsaredeployed.Allwecareaboutisthenumberoftheirinstances.ThesameappliestoJenkinsslaves,theyarealsorunonacluster.ThelastimprovementistheautomaticmanagementofthedatabaseschemasusingFlywaymigrations
integratedintothedeliveryprocess.
Ihopeyouarealreadyexcitedbywhatweplantobuildthroughoutthisbook.Wewillapproachitstepbystep,explainingeverydetailandallthepossibleoptionsinordertohelpyouunderstandtheproceduresandtools.Afterreadingthisbook,youwillbeabletointroduceorimprovetheContinuousDeliveryprocessinyourprojects.
SummaryInthischapter,wehaveintroducedtheContinuousDeliveryprocessstartingfromtheidea,discussingtheprerequisites,toendupwithtoolsthatareusedintherestofthebook.Thekeytakeawayfromthischapterisasfollows:
ThedeliveryprocessusedcurrentlyinmostcompanieshassignificantshortcomingsandcanbeimprovedusingmoderntoolsforautomationTheContinuousDeliveryapproachprovidesanumberofbenefits,ofwhichthemostsignificantonesare:fastdelivery,fastfeedbackcycle,andlow-riskreleasesTheContinuousDeliverypipelineconsistsofthreestages:ContinuousIntegration,automatedacceptancetesting,andconfigurationmanagementIntroducingContinuousDeliveryusuallyrequiresachangeintheorganization'scultureandstructureThemostimportanttoolsinthecontextofContinuousDeliveryareDocker,Jenkins,andAnsible
Inthenextchapter,weintroduceDockerandpresenthowtobuildadockerizedapplication.
IntroducingDocker
WewilldiscusshowthemodernContinuousDeliveryprocessshouldlookbyintroducingDocker,thetechnologythatchangedtheITindustryandthewaytheserversareused.
Thischaptercoversthefollowingpoints:
IntroducingtheideaofvirtualizationandcontainerizationInstallingDockerfordifferentlocalandserverenvironmentsExplainingthearchitectureoftheDockertoolkitBuildingDockerimageswithDockerfileandbycommittingchangesRunningapplicationsasDockercontainersConfiguringDockernetworksandportforwardingIntroducingDockervolumesasasharedstorage
WhatisDocker?Dockerisanopensourceprojectdesignedtohelpwithapplicationdeploymentusingsoftwarecontainers.ThisquoteisfromtheofficialDockerpage:"Dockercontainerswrapapieceofsoftwareinacompletefilesystemthatcontainseverythingneededtorun:code,runtime,systemtools,systemlibraries-anythingthatcanbeinstalledonaserver.Thisguaranteesthatthesoftwarewillalwaysrunthesame,regardlessofitsenvironment."
Docker,therefore,inasimilarwayasvirtualization,allowspackaginganapplicationintoanimagethatcanberuneverywhere.
ContainerizationversusvirtualizationWithoutDocker,isolationandotherbenefitscanbeachievedwiththeuseofhardwarevirtualization,oftencalledvirtualmachines.ThemostpopularsolutionsareVirtualBox,VMware,andParallels.Avirtualmachineemulatesacomputerarchitectureandprovidesthefunctionalityofaphysicalcomputer.Wecanachievecompleteisolationofapplicationsifeachofthemisdeliveredandrunasaseparatevirtualmachineimage.Thefollowingfigurepresentstheconceptofvirtualization:
Eachapplicationislaunchedasaseparateimagewithalldependenciesandaguestoperatingsystem.Imagesarerunbythehypervisor,whichemulatesthephysicalcomputerarchitecture.Thismethodofdeploymentiswidelysupportedbymanytools(suchasVagrant)anddedicatedtodevelopmentandtestingenvironments.Virtualization,however,hasthreesignificantdrawbacks:
Lowperformance:Thevirtualmachineemulatesthewholecomputerarchitecturetoruntheguestoperatingsystem,sothereisasignificantoverheadassociatedwitheachoperation.Highresourceconsumption:Emulationrequiresalotofresourcesandhastobedoneseparatelyforeachapplication.Thisiswhy,onastandarddesktopmachine,onlyafewapplicationscanberunsimultaneously.Largeimagesize:Eachapplicationisdeliveredwithafulloperatingsystem,sothedeploymentonaserverimpliessendingandstoringalarge
amountofdata.
Theconceptofcontainerizationpresentsadifferentsolution:
Eachapplicationisdeliveredtogetherwithitsdependencies,but,withouttheoperatingsystem.Applicationsinterfacedirectlywiththehostoperatingsystem,sothereisnoadditionallayeroftheguestoperatingsystem.Itresultsinbetterperformanceandnowasteofresources.Moreover,shippedDockerimagesaresignificantlysmaller.
Noticethatinthecaseofcontainerization,theisolationhappensatthelevelofthehostoperatingsystem'sprocesses.Itdoesn'tmean,however,thatthecontainerssharetheirdependencies.Eachofthemhastheirownlibrariesintherightversion,andifanyofthemisupdated,ithasnoimpactontheothers.Toachievethis,DockerEnginecreatesasetofLinuxnamespacesandcontrolgroupsforthecontainer.ThisiswhytheDockersecurityisbasedontheLinuxkernelprocessisolation.Thissolution,althoughmatureenough,couldbeconsideredslightlylesssecurethanthecompleteoperatingsystem-basedisolationofferedbyvirtualmachines.
TheneedforDockerDockercontainerizationsolvesanumberofproblemsseenintraditionalsoftwaredelivery.Let'stakeacloserlook.
EnvironmentInstallingandrunningsoftwareiscomplex.Youneedtodecideabouttheoperatingsystem,resources,libraries,services,permissions,othersoftware,andeverythingyourapplicationdependson.Then,youneedtoknowhowtoinstallit.What'smore,theremaybesomeconflictingdependencies.Whatdoyoudothen?Whatifyoursoftwareneedsanupgradeofalibrarybuttheotherdoesnot?Insomecompanies,suchissuesaresolvedbyhavingclassesofapplications,andeachclassisservedbyadedicatedserver,forexample,aserverforwebserviceswithJava7,anotheroneforbatchjobswithJava8,andsoon.Thissolution,however,isnotbalancedintermsofresourcesandrequiresanarmyofIToperationsteamstotakecareofallproductionandtestservers.
Anotherproblemwiththeenvironmentcomplexityisthatitoftenrequiresaspecialisttorunanapplication.AlesstechnicalpersonmayhaveahardtimesettingupMySQL,ODBC,oranyotherslightlymoresophisticatedtool.Thisisparticularlytrueforapplicationsnotdeliveredasanoperatingsystem-specificbinarybutwhichrequiresourcecodecompilationoranyotherenvironment-specificconfiguration.
IsolationKeeptheworkspacetidy.Oneapplicationcanchangethebehavioroftheotherone.Imaginewhatcanhappen.Applicationsshareonefilesystem,soifapplicationAwritessomethingtothewrongdirectory,applicationBreadstheincorrectdata.Theyshareresources,soifthereisamemoryleakinapplicationA,itcanfreezenotonlyitselfbutalsoapplicationB.Theysharenetworkinterfaces,soifapplicationsAandBbothuseport8080,oneofthemwillcrash.Isolationconcernsthesecurityaspectstoo.Runningabuggyapplicationormalicioussoftwarecancausedamagetootherapplications.Thisiswhyitisamuchsaferapproachtokeepeachapplicationinsideaseparatesandbox,whichlimitsthescopeofdamageimpacttotheapplicationitself.
OrganizingapplicationsServersoftenenduplookingmessywithatonofrunningapplicationsnobodyknowsanythingabout.Howwillyoucheckwhatapplicationsarerunningontheserverandwhatdependencieseachofthemisusing?Theycoulddependonlibraries,otherapplications,ortools.Withouttheexhaustivedocumentation,allwecandoislookattherunningprocessesandstartguessing.Dockerkeepsthingsorganizedbyhavingeachapplicationasaseparatecontainerthatcanbelisted,searched,andmonitored.
Portability"Writeonce,runanywhere,"saidthesloganwhileadvertisingtheearliestversionsofJava.Indeed,Javaaddressestheportabilityissuequitewell;however,Icanstillthinkofafewcaseswhereitfails,forexample,theincompatiblenativedependenciesortheolderversionoftheJavaruntime.Moreover,notallsoftwareiswritteninJava.
Dockermovestheconceptofportabilityonelevelhigher;iftheDockerversioniscompatible,thentheshippedsoftwareworkscorrectlyregardlessoftheprogramminglanguage,operatingsystem,orenvironmentconfiguration.Docker,then,canbeexpressedbytheslogan"Shiptheentireenvironmentinsteadofjustcode."
KittensandcattleThedifferencebetweentraditionalsoftwaredeploymentandDocker-baseddeploymentisoftenexpressedwithananalogyofkittensandcattle.Everybodylikeskittens.Kittensareunique.Eachhasitsownnameandneedsspecialtreatment.Kittensaretreatedwithemotion.Wecrywhentheydie.Onthecontrary,cattleexistonlytosatisfyourneeds.Eventheformcattleissingularsinceit'sjustapackofanimalstreatedtogether.Nonaming,nouniqueness.Surely,theyareunique(thesameaseachserverisunique),butitisirrelevant.ThisiswhythemoststraightforwardexplanationoftheideabehindDockeris"Treatyourserverslikecattle,notpets."
AlternativecontainerizationtechnologiesDockerisnottheonlycontainerizationsystemavailableonthemarket.Actually,thefirstversionsofDockerwerebasedontheopensourceLXC(LinuxContainers)system,whichisanalternativeplatformforcontainers.OtherknownsolutionsareFreeBSDJails,OpenVZ,andSolarisContainers.Docker,however,overtookallothersystemsbecauseofitssimplicity,goodmarketing,andstartupapproach.Itworksundermostoperatingsystems,allowsyoutodosomethingusefulinlessthan15minutes,hasalotofsimple-to-usefeatures,goodtutorials,agreatcommunity,andprobablythebestlogointheITindustry.
DockerinstallationDocker'sinstallationprocessisquickandsimple.Currently,it'ssupportedonmostLinuxoperatingsystemsandawiderangeofthemhavededicatedbinariesprovided.MacandWindowsarealsowellsupportedwithnativeapplications.However,it'simportanttounderstandthatDockerisinternallybasedontheLinuxkernelanditsspecifics,andthisiswhy,inthecaseofMacandWindows,itusesvirtualmachines(xhyveforMacandHyper-VforWindows)toruntheDockerEngineenvironment.
PrerequisitesforDockerDockerrequirementsarespecificforeachoperatingsystem.
Mac:
2010ornewermodel,withIntel’shardwaresupportformemorymanagementunit(MMU)virtualizationmacOS10.10.3YosemiteornewerAtleast4GBofRAMNoVirtualBoxpriortoversion4.3.30installed
Windows:
64-bitWindows10ProTheHyper-Vpackageenabled
Linux:
64-bitarchitectureLinuxkernel3.10orlater
Ifyourmachinedoesnotmeettherequirements,thenthesolutionistouseVirtualBoxwiththeUbuntuoperatingsysteminstalled.Thisworkaround,eventhoughitsoundscomplicated,isnotnecessarilytheworstmethod,especiallytakingintoconsiderationthattheDockerEngineenvironmentisvirtualizedanywayinthecaseofMacandWindows.Furthermore,Ubuntuisoneofthebest-supportedsystemsforusingDocker.
AllexamplesinthisbookhavebeentestedontheUbuntu16.04operatingsystem.
InstallingonalocalmachineDockersinstallationprocessisstraightforwardandverywelldescribedonitsofficialpages.
DockerforUbuntuhttps://docs.docker.com/engine/installation/linux/ubuntulinux/containsaguideonhowtoinstallDockeronanUbuntumachine.
InthecaseofUbuntu16.04,I'veexecutedthefollowingcommands:
$sudoapt-getupdate
$sudoapt-keyadv--keyserverhkp://p80.pool.sks-keyservers.net:80--recv-keys9DC858229FC7DD38854AE2D88D81803C0EBFCD88
$sudoapt-add-repository'deb[arch=amd64]https://download.docker.com/linux/ubuntuxenialmainstable'
$sudoapt-getupdate
$sudoapt-getinstall-ydocker-ce
Afteralloperationsarecompleted,Dockershouldbeinstalled.However,atthemoment,theonlyuserallowedtouseDockercommandsisroot.ThismeansthatthesudokeywordmustprecedeeveryDockercommand.
WecanenableotheruserstouseDockerbyaddingthemtothedockergroup:
$sudousermod-aGdocker<username>
Afterasuccessfullogout,everythingissetup.Withthelatestcommand,however,weneedtotakesomeprecautionsnottogivetheDockerpermissionstoanunwanteduser,andthereforecreateavulnerabilityintheDockerEngine.Thisisparticularlyimportantinthecaseofinstallationontheservermachine.
DockerforLinuxhttps://docs.docker.com/engine/installation/linux/containsinstallationguidesformostLinuxdistributions.
DockerforMachttps://docs.docker.com/docker-for-mac/containsastep-by-stepguideonhowtoinstallDockeronaMacmachine.ItisdeliveredtogetherwithacollectionofDockercomponents:
VirtualmachinewithDockerEngineDockerMachine(atoolusedtocreateDockerhostsonthevirtualmachine)DockerComposeDockerclientandserverKitematic:aGUIapplication
TheDockerMachinetoolhelpsininstallingandmanagingDockerEngineonMac,Windows,oncompanynetworks,indatacenters,andoncloudproviderssuchasAWSorDigitalOcean.
DockerforWindowshttps://docs.docker.com/docker-for-windows/containsastep-by-stepguideonhowtoinstallDockeronaWindowsmachine.ItisdeliveredtogetherwithacollectionofDockercomponentssimilartoMac.
TheinstallationguidesforallsupportedoperatingsystemsandcloudplatformscanbefoundontheofficialDockerpage,https://docs.docker.com/engine/installation/.
<strong>$dockerinfo</strong><br/><strong>Containers:0</strong><br/><strong>Running:0</strong><br/><strong>Paused:0</strong><br/><strong>Stopped:0</strong><br/><strong>Images:0</strong><br/><strong>...</strong>
InstallingonaserverInordertouseDockeroverthenetwork,itispossibletoeithertakeadvantageofcloudplatformprovidersortomanuallyinstallDockeronadedicatedserver.
Inthefirstcase,theDockerconfigurationdiffersfromoneplatformtoanother,butitisalwaysverywelldescribedindedicatedtutorials.MostcloudplatformsenablecreatingDockerhostsviauser-friendlywebinterfacesordescribeexactcommandstoexecuteontheirservers.
Thesecondcase(installingDockermanually)requires,however,afewwordsofcomment.
DedicatedserverInstallingDockermanuallyonaserverdoesnotdiffermuchfromthelocalinstallation.
TwoadditionalstepsarerequiredthatincludesettingtheDockerdaemontolistenonthenetworksocketandsettingsecuritycertificates.
Let'sstartfromthefirststep.Bydefault,duetosecurityreasons,Dockerrunsviaanon-networkedUnixsocketthatonlyallowslocalcommunication.It'snecessarytoaddlisteningonthechosennetworkinterfacesocketsothattheexternalclientscanconnect.https://docs.docker.com/engine/admin/describesindetailalltheconfigurationstepsneededforeachLinuxdistribution.
InthecaseofUbuntu,theDockerdaemonisconfiguredbythesystemd,soinordertochangetheconfigurationofhowit'sstarted,weneedtomodifyonelineinthe/lib/systemd/system/docker.servicefile:
ExecStart=/usr/bin/dockerd-H<server_ip>:2375
Bychangingthisline,weenabledtheaccesstotheDockerdaemonviathespecifiedIPaddress.Allthedetailsonthesystemdconfigurationcanbefoundathttps://docs.docker.com/engine/admin/systemd/.
ThesecondstepofserverconfigurationconcernstheDockersecuritycertificates.Thisenablesonlyclientsauthenticatedbyacertificatetoaccesstheserver.ThecomprehensivedescriptionoftheDockercertificatesconfigurationcanbefoundathttps://docs.docker.com/engine/security/https/.Thisstepisn'tstrictlyrequired;however,unlessyourDockerdaemonserverisinsidethefirewallednetwork,itisessential.
IfyourDockerdaemonisruninsidethecorporatenetwork,youhavetoconfiguretheHTTPproxy.Thedetaileddescriptioncanbefoundathttps://docs.docker.com/engine/admin/systemd/.
RunningDockerhelloworld>TheDockerenvironmentissetupandready,sowecanstartthefirstexample.
Enterthefollowingcommandinyourconsole:$dockerrunhello-worldUnabletofindimage'hello-world:latest'locallylatest:Pullingfromlibrary/hello-world78445dd45222:PullcompleteDigest:sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7Status:Downloadednewerimageforhello-world:latest
HellofromDocker!Thismessageshowsthatyourinstallationappearstobeworkingcorrectly....
Congratulations,you'vejustrunyourfirstDockercontainer.IhopeyoualreadyfeelhowsimpleDockeris.Let'sexaminestep-by-stepwhathappenedunderthehood:
1. YourantheDockerclientwiththeruncommand.2. TheDockerclientcontactedtheDockerdaemonaskingtocreatea
containerfromtheimagecalledhello-world.3. TheDockerdaemoncheckedifitcontainedthehello-worldimagelocally
and,sinceitdidn't,requestedthehello-worldimagefromtheremoteDockerHubregistry.
4. TheDockerHubregistrycontainedthehello-worldimage,soitwaspulledintotheDockerdaemon.
5. TheDockerdaemoncreatedanewcontainerfromthehello-worldimagethatstartedtheexecutableproducingtheoutput.
6. TheDockerdaemonstreamedthisoutputtotheDockerclient.7. TheDockerclientsentittoyourterminal.
Theprojectedflowcanbepresentedinthefollowingdiagram:
Let'slookateachDockercomponentthatwasillustratedinthissection.
DockercomponentsTheofficialDockerpagesaysthis:
"DockerEngineisaclient-serverapplicationthatcreatesandmanagesDockerobjects,suchasimagesandcontainers."
Let'ssortoutwhatthismeans.
DockerclientandserverLet'slookatadiagramthatpresentstheDockerEnginearchitecture:
DockerEngineconsistsofthreecomponents:
DockerDaemon(server)runninginthebackgroundDockerClientrunningasacommandtoolRESTAPI
InstallingDockerEnginemeansinstallingallthecomponentssothattheDockerdaemonrunsonourcomputerallthetimeasaservice.Inthecaseofthehello-worldexample,weusedtheDockerclienttointeractwiththeDockerdaemon;however,wecoulddoexactlythesameusingRESTAPI.Also,inthecaseofthehello-worldexample,weconnectedtothelocalDockerdaemon;however,wecouldusethesameclienttointeractwiththeDockerdaemonrunningonaremotemachine.
ToruntheDockercontaineronaremotemachine,youcanusethe-Hoption:docker-H<server_ip>:2375runhello-world
DockerimagesandcontainersAnimageisastatelessbuildingblockintheDockerworld.Youcanimagineanimageasacollectionofallfilesnecessarytorunyourapplicationtogetherwiththerecipeonhowtorunit.Theimageisstateless,soyoucansenditoverthenetwork,storeitintheregistry,nameit,versionit,andsaveitasafile.Imagesarelayered,whichmeansthatyoucanbuildanimageontopofanotherimage.
Acontainerisarunninginstanceofanimage.Wecancreatemanycontainersfromthesameimageifwewanttohavemanyinstancesofthesameapplication.Sincecontainersarestateful,wecaninteractwiththemandmakechangestotheirstates.
Let'slookattheexampleofacontainerandtheimagelayersstructure:
Atthebottom,thereisalwaysthebaseimage.Inmostcases,itrepresentsanoperatingsystem,andwebuildourimagesontopoftheexistingbaseimages.It'stechnicallypossibletocreateownbaseimageshowever,thisisrarelyneeded.
Inourexample,theubuntubaseimageprovidesallthecapabilitiesoftheUbuntuoperatingsystem.TheaddgitimageaddstheGittoolkit.Then,thereisanimageaddingtheJDKenvironment.Finally,onthetop,thereisacontainercreatedfromtheaddJDKimage.Suchcontainerisable,forexample,todownloadaJavaprojectfromtheGitHubrepositoryandcompileittoaJARfile.Asaresult,wecanusethiscontainertocompileandrunJavaprojectswithoutinstallinganytoolsonouroperatingsystem.
Itisimportanttonoticethatlayeringisaverysmartmechanismtosavebandwidthandstorage.Imaginethatwehaveanapplicationthatisalsobasedonubuntu:
ThistimewewillusethePythoninterpreter.Whileinstallingtheaddpythonimage,theDockerdaemonwillnotethattheubuntuimageisalreadyinstalled,andwhatitneedstodoisonlytoaddthepythonlayer,whichisverysmall.So,theubuntuimageisadependencythatisreused.Thesameifwewouldliketodeployourimageinthenetwork.WhenwedeploytheGitandJDKapplication,weneedtosendthewholeubuntuimage.However,whilesubsequentlydeployingthepythonapplication,weneedtosendjustthesmalladdpythonlayer.
DockerapplicationsAlotofapplicationsareprovidedintheformofDockerimagesthatcanbedownloadedfromtheinternet.Ifweknewtheimagename,thenitwouldbeenoughtorunitinthesamewaywedidwiththehelloworldexample.HowcanwefindthedesiredapplicationimageontheDockerHub?
Let'stakeMongoDBasanexample.IfweliketofinditontheDockerHub,wehavetwooptions:
SearchontheDockerHubExplorepage(https://hub.docker.com/explore/)Usethedockersearchcommand
Inthesecondcase,wecanperformthefollowingoperation:
$dockersearchmongo
NAMEDESCRIPTIONSTARSOFFICIALAUTOMATED
mongoMongoDBdocumentdatabasesprovidehighav...2821[OK]
mongo-expressWeb-basedMongoDBadmininterface,written...106[OK]
mvertes/alpine-mongolightMongoDBcontainer39[OK]
mongoclient/mongoclientOfficialdockerimageforMongoclient,fea...19[OK]
...
Therearemanyinterestingoptions.Howdowechoosethebestimage?Usually,themostappealingoneistheonewithoutanyprefix,sinceitmeansthatit'sanofficialDockerHubimageandshouldbethereforestableandmaintained.Theimageswithprefixesareunofficial,usuallymaintainedasopensourceprojects.Inourcase,thebestchoiceseemstobemongo,soinordertoruntheMongoDBserver,wecanrunthefollowingcommand:
$dockerrunmongo
Unabletofindimage'mongo:latest'locally
latest:Pullingfromlibrary/mongo
5040bd298390:Pullcomplete
ef697e8d464e:Pullcomplete
67d7bf010c40:Pullcomplete
bb0b4f23ca2d:Pullcomplete
8efff42d23e5:Pullcomplete
11dec5aa0089:Pullcomplete
e76feb0ad656:Pullcomplete
5e1dcc6263a9:Pullcomplete
2855a823db09:Pullcomplete
Digest:sha256:aff0c497cff4f116583b99b21775a8844a17bcf5c69f7f3f6028013bf0d6c00c
Status:Downloadednewerimageformongo:latest
2017-01-28T14:33:59.383+0000ICONTROL[initandlisten]MongoDBstarting:pid=1port=27017dbpath=/data/db64-bithost=0f05d9df0dc2
...
That'sall,MongoDBhasstarted.RunningapplicationsasDockercontainersisthatsimplebecausewedon'tneedtothinkofanydependencies;theyarealldeliveredtogetherwiththeimage.
OntheDockerHubservice,youcanfindalotofapplications;theystoremorethan100,000differentimages.
BuildingimagesDockercanbetreatedasausefultooltorunapplications;however,therealpowerliesinbuildingownDockerimagesthatwraptheprogramstogetherwiththeenvironment.Inthissection,wewillseehowtodothisusingtwodifferentmethods,theDockercommitcommandandtheDockerfileautomatedbuild.
DockercommitLet'sstartwithanexampleandprepareanimagewiththeGitandJDKtoolkits.WewilluseUbuntu16.04asabaseimage.Thereisnoneedtocreateit;mostbaseimagesareavailableintheDockerHubregistry:
1. Runacontainerfromtheubuntu:16.04andconnectittoitscommandline:
$dockerrun-i-tubuntu:16.04/bin/bash
We'vepulledtheubuntu:16.04imageandrunitasacontainerandthencalledthe/bin/bashcommandinaninteractiveway(-iflag).Youshouldseetheterminalofthecontainer.Sincecontainersarestatefulandwritable,wecandoanythingwewantinitsterminal.
2. InstalltheGittoolkit:
root@dee2cb192c6c:/#apt-getupdate
root@dee2cb192c6c:/#apt-getinstall-ygit
3. CheckiftheGittoolkitisinstalled:
root@dee2cb192c6c:/#whichgit
/usr/bin/git
4. Exitthecontainer:
root@dee2cb192c6c:/#exit
5. Checkwhathaschangedinthecontainercomparingittotheubuntuimage:
$dockerdiffdee2cb192c6c
Thecommandshouldprintalistofallfileschangedinthecontainer.
6. Committhecontainertotheimage:
$dockercommitdee2cb192c6cubuntu_with_git
We'vejustcreatedourfirstDockerimage.Let'slistalltheimagesofourDockerhosttoseeiftheimageispresent:
$dockerimages
REPOSITORYTAGIMAGEIDCREATEDSIZE
ubuntu_with_gitlatestf3d674114fe2Aboutaminuteago259.7MB
ubuntu16.04f49eec89601e7daysago129.5MB
mongolatest0dffc7177b0610daysago402MB
hello-worldlatest48b5124b27682weeksago1.84kB
Asexpected,weseehello-world,mongo(installedbefore),ubuntu(baseimagepulledfromDockerHub),andfreshlybuiltubuntu_with_git.Bytheway,wecanobservethesizeofeachimagethatcorrespondstowhatwe'veinstalledontheimage.
Now,ifwecreateacontainerfromtheimage,itwillhavetheGittoolinstalled:
$dockerrun-i-tubuntu_with_git/bin/bash
root@3b0d1ff457d4:/#whichgit
/usr/bin/git
root@3b0d1ff457d4:/#exit
Usingtheexactsamemethod,wecanbuildubuntu_with_git_and_jdkontopoftheubuntu_with_gitimage:
$dockerrun-i-tubuntu_with_git/bin/bash
root@6ee6401ed8b8:/#apt-getinstall-yopenjdk-8-jdk
root@6ee6401ed8b8:/#exit
$dockercommit6ee6401ed8b8ubuntu_with_git_and_jdk
DockerfileCreatingeachDockerimagemanuallywiththecommitcommandcouldbelaborious,especiallyinthecaseofbuildautomationandtheContinuousDeliveryprocess.Luckily,thereisabuilt-inlanguagetospecifyalltheinstructionsthatshouldbeexecutedtobuildtheDockerimage.
Let'sstartwithanexamplesimilartotheonewithGitandJDK.Thistime,wewillpreparetheubuntu_with_pythonimage.
1. CreateanewdirectoryandafilecalledDockerfilewiththefollowingcontent:
FROMubuntu:16.04
RUNapt-getupdate&&\
apt-getinstall-ypython
2. Runthecommandtocreatetheubuntu_with_pythonimage:
$dockerbuild-tubuntu_with_python.
3. Checkthattheimagewascreated:
$dockerimages
REPOSITORYTAGIMAGEIDCREATEDSIZE
ubuntu_with_pythonlatestd6e85f39f5b7Aboutaminuteago202.6MB
ubuntu_with_git_and_jdklatest8464dc10abbb3minutesago610.9MB
ubuntu_with_gitlatestf3d674114fe29minutesago259.7MB
ubuntu16.04f49eec89601e7daysago129.5MB
mongolatest0dffc7177b0610daysago402MB
hello-worldlatest48b5124b27682weeksago1.84kB
WecannowcreateacontainerfromtheimageandcheckthatthePythoninterpreterexistsinexactlythesamewaywedidafterexecutingthedockercommitcommand.Notethattheubuntuimageislistedonlyonceeventhoughit'sthebaseimageforbothubuntu_with_gitandubuntu_with_python.
Inthisexample,weusedthefirsttwoDockerfileinstructions:
FROMdefinestheimageontopofwhichthenewimagewillbebuiltRUNspecifiesthecommandstoruninsidethecontainer
AllDockerfileinstructionscanbefoundontheofficialDockerpageathttps://docs.docker.com/engine/reference/builder/.Themostwidely-usedinstructionsareasfollows:
MAINTAINERdefinesthemetainformationabouttheauthorCOPYcopiesafileoradirectoryintothefilesystemoftheimageENTRYPOINTdefineswhichapplicationshouldberunintheexecutablecontainer
AcompleteguideofallDockerfileinstructionscanbefoundontheofficialDockerpageathttps://docs.docker.com/engine/reference/builder/.
CompleteDockerapplicationWealreadyhavealltheinformationnecessarytobuildafullyworkingapplicationasaDockerimage.Asanexample,wewillprepare,stepbystep,asimplePythonhelloworldprogram.Thesamestepsexistalways,nomatterwhatenvironmentorprogramminglanguageweuse.
WritetheapplicationCreateanewdirectoryandinsidethisdirectory,ahello.pyfilewiththefollowingcontent:
print"HelloWorldfromPython!"
Closethefile.Thisisthesourcecodeofourapplication.
FROMubuntu:16.04<br/>MAINTAINERRafalLeszko<br/>RUNapt-getupdate&&\<br/>apt-getinstall-ypython<br/>COPYhello.py.<br/>ENTRYPOINT["python","hello.py"]
BuildtheimageNow,wecanbuildtheimageexactlythesamewaywedidbefore:
$dockerbuild-thello_world_python.
RuntheapplicationWeruntheapplicationbyrunningthecontainer:
$dockerrunhello_world_python
YoushouldseethefriendlyHelloWorldfromPython!message.ThemostinterestingthinginthisexampleisthatweareabletoruntheapplicationwritteninPythonwithouthavingthePythoninterpreterinstalledinourhostsystem.Thisispossiblebecausetheapplicationpackedasanimagehasalltheenvironmentneededinside.
AnimagewiththePythoninterpreteralreadyexistsintheDockerHubservice,sointhereal-lifescenario,itwouldbeenoughtouseit.
EnvironmentvariablesWe'verunourfirsthome-madeDockerapplication.However,whatiftheexecutionoftheapplicationshoulddependonsomeconditions?
Forexample,inthecaseoftheproductionserver,wewouldliketoprintHellotothelogs,nottotheconsole,orwemaywanttohavedifferentdependentservicesduringthetestingphaseandtheproductionphase.OnesolutionwouldbetoprepareaseparateDockerfileforeachcase;however,thereisabetterway,environmentvariables.
Let'schangeourhelloworldapplicationtoprintHelloWorldfrom<name_passed_as_environment_variable>!.Inordertodothis,weneedtoproceedwiththefollowingsteps:
1. ChangethePythonscripttousetheenvironmentvariable:
importos
print"HelloWorldfrom%s!"%os.environ['NAME']
2. Buildtheimage:
$dockerbuild-thello_world_python_name.
3. Runthecontainerpassingenvironmentvariable:
$dockerrun-eNAME=Rafalhello_world_python_name
HelloWorldfromRafal!
4. Alternatively,wecandefinetheenvironmentvariablevalueinDockerfile,forexample:
ENVNAMERafal
5. Then,wecanrunthecontainerwithoutspecifyingthe-eoption.
$dockerbuild-thello_world_python_name_default.
$dockerrunhello_world_python_name_default
HelloWorldfromRafal!
Environmentvariablesareespeciallyusefulwhenweneedtohavedifferent
versionsoftheDockercontainerdependingonitspurpose,forexample,tohaveseparateprofilesforproductionandtestingservers.
IftheenvironmentvariableisdefinedbothinDockerfileandasaflag,thenthecommandflagtakesprecedence.
DockercontainerstatesEveryapplicationwe'verunsofarwassupposedtodosomeworkandstop.Forexample,we'veprintedHellofromDocker!andexited.Thereare,however,applicationsthatshouldruncontinuouslysuchasservices.Torunacontainerinthebackground,wecanusethe-d(--detach)option.Let'stryitwiththeubuntuimage:
$dockerrun-d-tubuntu:16.04
ThiscommandstartedtheUbuntucontainerbutdidnotattachtheconsoletoit.Wecanseethatit'srunningusingthefollowingcommand:
$dockerps
CONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES
95f29bfbaadcubuntu:16.04"/bin/bash"Up5secondskickass_stonebraker
Thiscommandprintsallcontainersthatareintherunningstate.Whataboutourold,already-exitedcontainers?Wecanfindthembyprintingallcontainers:
$dockerps-a
CONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES
95f29bfbaadcubuntu:16.04"/bin/bash"Up33secondskickass_stonebraker
34080d914613hello_world_python_name_default"pythonhello.py"Exitedlonely_newton
7ba49e8ee677hello_world_python_name"pythonhello.py"Exitedmad_turing
dd5eb1ed81c3hello_world_python"pythonhello.py"Exitedthirsty_bardeen
6ee6401ed8b8ubuntu_with_git"/bin/bash"Exitedgrave_nobel
3b0d1ff457d4ubuntu_with_git"/bin/bash"Exiteddesperate_williams
dee2cb192c6cubuntu:16.04"/bin/bash"Exitedsmall_dubinsky
0f05d9df0dc2mongo"/entrypoint.shmongo"Exitedtrusting_easley
47ba1c0ba90ehello-world"/hello"Exitedtender_bell
Notethatalltheoldcontainersareintheexitedstate.Therearetwomorestateswehaven'tobservedyet:pausedandrestarting.
Allofthestatesandthetransitionsbetweenthemarepresentedinthefollowingdiagram:
PausingDockercontainersisveryrare,andtechnically,it'srealizedbyfreezingtheprocessesusingtheSIGSTOPsignal.Restartingisatemporarystatewhenthecontainerisrunwiththe--restartoptiontodefinetherestartingstrategy(theDockerdaemonisabletoautomaticallyrestartthecontainerincaseoffailure).
ThediagramalsoshowstheDockercommandsusedtochangetheDockercontainerstatefromonetoanother.Forexample,wecanstoptherunningUbuntucontainer:
$dockerstop95f29bfbaadc
$dockerps
CONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES
Wealwaysusedthedockerruncommandtocreateandstartthecontainer;however,it'spossibletojustcreatethecontainerwithoutstartingit.
DockernetworkingMostapplicationsthesedaysdonotruninisolationbutneedtocommunicatewithothersystemsoverthenetwork.Ifwewanttorunawebsite,webservice,database,oracacheserverinsideaDockercontainer,thenweneedtounderstandatleastthebasicsofDockernetworking.
RunningservicesLet'sstartwithasimpleexample,andrunaTomcatserverdirectlyfromDockerHub:
$dockerrun-dtomcat
Tomcatisawebapplicationserverwhoseuserinterfacecanbeaccessedbytheport8080.Therefore,ifweinstalledTomcatonourmachine,wecouldbrowseitathttp://localhost:8080.
Inourcase,however,TomcatisrunninginsidetheDockercontainer.WestarteditthesamewaywedidwiththefirstHelloWorldexample.Wecanseeit'srunning:
$dockerps
CONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES
d51ad8634factomcat"catalina.shrun"UpAboutaminute8080/tcpjovial_kare
Sinceit'srunasadaemon(withthe-doption),wedon'tseethelogsintheconsolerightaway.Wecan,however,accessitbyexecutingthefollowingcode:
$dockerlogsd51ad8634fac
Iftherearenoerrors,weshouldseealotoflogsthatconcludesTomcathasbeenstartedandisaccessibleviaport8080.Wecantrygoingtohttp://localhost:8080,butwewon'tbeabletoconnect.ThereasonforthisisthatTomcathasbeenstartedinsidethecontainerandwe'retryingtoreachitfromtheoutside.Inotherwords,wecanreachitonlyifweconnectwiththecommandtotheconsoleinthecontainerandcheckitthere.HowtomaketherunningTomcataccessiblefromoutside?
Weneedtostartthecontainerspecifyingtheportmappingwiththe-p(--publish)flag:
-p,--publish<host_port>:<container_port>
So,let'sfirststoptherunningcontainerandstartanewone:
$dockerstopd51ad8634fac
$dockerrun-d-p8080:8080tomcat
Afterwaitingafewseconds,Tomcatmusthavestartedandweshouldbeabletoopenitspage,http://localhost:8080.
SuchasimpleportmappingcommandissufficientinmostcommonDockerusecases.Weareabletodeploy(micro)servicesasDockercontainersandexposetheirportstoenablecommunication.However,let'sdivealittledeeperintowhathappenedunderthehood.
Dockerallowspublishingtothespecifiedhostnetworkinterfacewith-p<ip>:<host_port>:<container_port>.
ContainernetworksWehaveconnectedtotheapplicationrunninginsidethecontainer.Infact,theconnectionistwo-way,becauseifyourememberourpreviousexamples,weexecutedtheapt-getinstallcommandsfrominsideandthepackagesweredownloadedfromtheinternet.Howisthispossible?
Ifyoucheckthenetworkinterfacesonyourmachine,youcanseethatoneoftheinterfacesiscalleddocker0:
$ifconfigdocker0
docker0Linkencap:EthernetHWaddr02:42:db:d0:47:db
inetaddr:172.17.0.1Bcast:0.0.0.0Mask:255.255.0.0
...
Thedocker0interfaceiscreatedbytheDockerdaemoninordertoconnectwiththeDockercontainer.Now,wecanseewhatinterfacesarecreatedinsidetheDockercontainerwiththedockerinspectcommand:
$dockerinspect03d1e6dc4d9e
ItprintsalltheinformationaboutthecontainerconfigurationintheJSONformat.Amongothers,wecanfindthepartrelatedtothenetworksettings:
"NetworkSettings":{
"Bridge":"",
"Ports":{
"8080/tcp":[
{
"HostIp":"0.0.0.0",
"HostPort":"8080"
}
]
},
"Gateway":"172.17.0.1",
"IPAddress":"172.17.0.2",
"IPPrefixLen":16,
}
Inordertofilterthedockerinspectresponse,wecanusethe--formatoption,forexample,dockerinspect--format'{{.NetworkSettings.IPAddress}}'<container_id>.
WecanobservethattheDockercontainerhastheIPaddress172.17.0.2andit
communicateswiththeDockerhostwiththeIPaddress172.17.0.1.Thismeansthatinourpreviousexample,wecouldaccesstheTomcatserverevenwithouttheportforwarding,usingtheaddresshttp://172.17.0.2:8080.Nevertheless,inmostcases,weruntheDockercontaineronaservermachineandwanttoexposeitoutside,soweneedtousethe-poption.
Notethatbydefault,thecontainersareprotectedbythehost'sfirewallsystemanddon'topenanyroutesfromexternalsystems.Wecanchangethisdefaultbehaviorbyplayingwiththe--networkflagandsettingitasfollows:
bridge(default):networkviathedefaultDockerbridgenone:nonetworkcontainer:networkjoinedwiththeother(specified)containerhost:hostnetwork(nofirewall)
Thedifferentnetworkscanbelistedandmanagedbythedockernetworkcommand:
$dockernetworkls
NETWORKIDNAMEDRIVERSCOPE
b3326cb44121bridgebridgelocal
84136027df04hosthostlocal
80c26af0351cnonenulllocal
Ifwespecifynoneasthenetwork,thenwewillnotbeabletoconnecttothecontainer,andviceversa;thecontainerhasnonetworkaccesstotheexternalworld.Thehostoptionmakesthecontainernetworkinterfacesidenticaltothehost.TheysharethesameIPaddresses,soeverythingstartedonthecontainerisvisibleoutside.Themostoftenusedoptionisthedefaultone(bridge)becauseitletsusdefineexplicitlywhichportsshouldbepublished.Itisbothsecureandaccessible.
EXPOSE8080
ThisDockerfileinstructionexpressesthattheport8080shouldbeexposedfromthecontainer.However,aswehavealreadyseen,thisdoesn'tmeanthattheportisautomaticallypublished.TheEXPOSEinstructiononlyinformstheuserswhichportstheyshouldpublish.
AutomaticportassignmentLet'strytorunthesecondTomcatcontainerwithoutstoppingthefirstone:
$dockerrun-d-p8080:8080tomcat
0835c95538aeca79e0305b5f19a5f96cb00c5d1c50bed87584cfca8ec790f241
docker:Errorresponsefromdaemon:driverfailedprogrammingexternalconnectivityonendpointdistracted_heyrovsky(1b1cee9896ed99b9b804e4c944a3d9544adf72f1ef3f9c9f37bc985e9c30f452):Bindfor0.0.0.0:8080failed:portisalreadyallocated.
Thiserrormaybecommon.Insuchcases,wehavetoeithertakecareoftheuniquenessoftheportsonourownorletDockerassigntheportsautomaticallyusingoneofthefollowingversionsofthepublishcommand:
-p<container_port>:publishthecontainerporttotheunusedhostport-P(--publish-all):publishallportsexposedbythecontainertotheunusedhostports:
$dockerrun-d-Ptomcat
078e9d12a1c8724f8aa27510a6390473c1789aa49e7f8b14ddfaaa328c8f737b
$dockerport078e9d12a1c8
8080/tcp->0.0.0.0:32772
WecanseethatthesecondTomcathasbeenpublishedtoport32772,soitcanbebrowsedathttp://localhost:32772.
UsingDockervolumesImaginethatyouwouldliketorunthedatabaseasacontainer.Youcanstartsuchacontainerandenterthedata.Whereisitstored?Whathappenswhenyoustopthecontainerorremoveit?Youcanstartthenewone,butthedatabasewillbeemptyagain.Unlessit'syourtestingenvironment,youdon'texpectsuchascenario.
DockervolumeistheDockerhost'sdirectorymountedinsidethecontainer.Itallowsthecontainertowritetothehost'sfilesystemasitwaswritingtoitsown.Themechanismispresentedinthefollowingdiagram:
Dockervolumeenablesthepersistenceandsharingofthecontainer'sdata.Volumesalsoclearlyseparatetheprocessingfromthedata.
Let'sstartwithanexampleandspecifythevolumewiththe-v<host_path>:<container_path>optionandconnecttothecontainer:
$dockerrun-i-t-v~/docker_ubuntu:/host_directoryubuntu:16.04/bin/bash
Now,wecancreateanemptyfileinhost_directoryinthecontainer:
root@01bf73826624:/#touchhost_directory/file.txt
Let'scheckifthefilewascreatedintheDockerhost'sfilesystem:
root@01bf73826624:/#exit
exit
$ls~/docker_ubuntu/
file.txt
Wecanseethatthefilesystemwassharedandthedatawasthereforepersistedpermanently.Wecannowstopthecontainerandrunanewonetoseethatourfilewillstillbethere:$dockerstop01bf73826624
$dockerrun-i-t-v~/docker_ubuntu:/host_directoryubuntu:16.04/bin/bashroot@a9e0df194f1f:/#lshost_directory/file.txt
root@a9e0df194f1f:/#exit
Insteadofspecifyingthevolumewiththe-vflag,it'spossibletospecifythevolumeasaninstructionintheDockerfile,forexample:
VOLUME/host_directory
Inthiscase,ifwerunthedockercontainerwithoutthe-vflag,thenthecontainer's/host_directorywillbemappedintothehost'sdefaultdirectoryforvolumes,/var/lib/docker/vfs/.Thisisagoodsolutionifyoudeliveranapplicationasanimageandyouknowitneedsthepermanentstorageforsomereason(forexample,storingapplicationlogs).
IfthevolumeisdefinedbothinDockerfileandasaflag,thenthecommandflagtakesprecedence.
Dockervolumescanbemuchmorecomplicated,especiallyinthecaseofdatabases.MorecomplexusecasesoftheDockervolumeare,however,outofthescopeofthisbook.
AverycommonapproachtodatamanagementwithDockeristointroduceanadditionallayerintheformofdatavolumecontainers.AdatavolumecontainerisaDockercontainerwhoseonlypurposeistodeclarethevolume.Then,othercontainerscanuseit(withthe--volumes-from<container>option)insteadofdeclaringthevolumedirectly.Readmoreathttps://docs.docker.com/engine/tutorials/dockervolumes/#creating-and-mounting-a-data-volume-container.
UsingnamesinDockerSofar,whenweoperatedonthecontainers,wealwaysusedautogeneratednames.Thisapproachhassomeadvantages,suchasthenamesbeingunique(nonamingconflicts)andautomatic(noneedtodoanything).Inmanycases,however,it'sbettertogivearealuser-friendlynameforthecontainerortheimage.
NamingcontainersTherearetwogoodreasonstonamethecontainer:convenienceandthepossibilityofautomation:
Convenience,becauseit'ssimplertomakeanyoperationsonthecontaineraddressingitbynamethancheckingthehashesortheautogeneratednameAutomation,becausesometimeswewouldliketodependonthespecificnamingofthecontainer
Forexample,wewouldliketohavecontainersthatdependoneachotherandtohaveonelinkedtoanother.Therefore,weneedtoknowtheirnames.
Tonameacontainer,weusethe--nameparameter:
$dockerrun-d--nametomcattomcat
Wecancheck(bydockerps)thatthecontainerhasameaningfulname.Also,asaresult,anyoperationcanbeperformedusingthecontainer'sname,forexample:
$dockerlogstomcat
Pleasenotethatwhenthecontainerisnamed,itdoesnotloseitsidentity.WecanstilladdressthecontainerbyitsautogeneratedhashIDlikewedidbefore.
ThecontaineralwayshasbothIDandname.Itcanbeaddressedbyanyofthemandbothofthemareunique.
TaggingimagesImagescanbetagged.We'vealreadydonethiswhilecreatingourownimages,forexample,inthecaseofbuildingthehello-world_pythonimage:
$dockerbuild-thello-world_python.
The-tflagdescribesthetagoftheimage.Ifwedidn'tuseit,thentheimagewouldbebuiltwithoutanytagsand,asaresult,wewouldhavetoaddressitbyitsID(hash)inordertorunthecontainer.
Theimagecanhavemultipletags,andtheyshouldfollowthenamingconvention:
<registry_address>/<image_name>:<version>
Thetagconsistsofthefollowingparts:
registry_address:IPandportoftheregistryorthealiasnameimage_name:nameoftheimagethatisbuilt,forexample,ubuntuversion:aversionoftheimageinanyform,forexample,16.04,20170310
WewillcoverDockerregistriesinChapter5,AutomatedAcceptanceTesting.IftheimageiskeptontheofficialDockerHubregistry,thenwecanskiptheregistryaddress.Thisiswhywe'verunthetomcatimagewithoutanyprefix.Thelastversionisalwaystaggedasthelatestanditcanalsobeskipped,sowe'verunthetomcatimagewithoutanysuffix.
Imagesusuallyhavemultipletags,forexample,allfourtagsarethesameimage:ubuntu:16.04,ubuntu:xenial-20170119,ubuntu:xenial,andubuntu:latest.
DockercleanupThroughoutthischapter,wehavecreatedanumberofcontainersandimages.Thisis,however,onlyasmallpartofwhatyouwillseeinreal-lifescenarios.Evenwhenthecontainersarenotrunningatthemoment,theyneedtobestoredontheDockerhost.Thiscanquicklyresultinexceedingthestoragespaceandstoppingthemachine.Howcanweapproachthisconcern?
CleaningupcontainersFirst,let'slookatthecontainersthatarestoredonourmachine.Toprintallthecontainers(nomatteroftheirstate),wecanusethedockerps-acommand:$dockerps-aCONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES95c2d6c4424etomcat"catalina.shrun"Up5minutes8080/tcptomcata9e0df194f1fubuntu:16.04"/bin/bash"Exitedjolly_archimedes01bf73826624ubuntu:16.04"/bin/bash"Exitedsuspicious_feynman078e9d12a1c8tomcat"catalina.shrun"Up14minutes0.0.0.0:32772->8080/tcpnauseous_fermi0835c95538aetomcat"catalina.shrun"Createddistracted_heyrovsky03d1e6dc4d9etomcat"catalina.shrun"Up50minutes0.0.0.0:8080->8080/tcpdrunk_ritchied51ad8634factomcat"catalina.shrun"Exitedjovial_kare95f29bfbaadcubuntu:16.04"/bin/bash"Exitedkickass_stonebraker34080d914613hello_world_python_name_default"pythonhello.py"Exitedlonely_newton7ba49e8ee677hello_world_python_name"pythonhello.py"Exitedmad_turingdd5eb1ed81c3hello_world_python"pythonhello.py"Exitedthirsty_bardeen6ee6401ed8b8ubuntu_with_git"/bin/bash"Exitedgrave_nobel3b0d1ff457d4ubuntu_with_git"/bin/bash"Exiteddesperate_williamsdee2cb192c6cubuntu:16.04"/bin/bash"Exitedsmall_dubinsky0f05d9df0dc2mongo"/entrypoint.shmongo"Exitedtrusting_easley47ba1c0ba90ehello-world"/hello"Exitedtender_bell
Inordertodeleteastoppedcontainer,wecanusethedockerrmcommand(ifthecontainerisrunning,weneedtostopitfirst):$dockerrm47ba1c0ba90e
Ifwewanttoremoveallstoppedcontainers,wecanusethefollowingcommand:
$dockerrm$(dockerps--no-trunc-aq)
The-aqoptionspecifiestopassonlyIDs(noadditionaldata)forallcontainers.
Additionally,--no-truncasksDockernottotruncatetheoutput.
Wecanalsoadoptadifferentapproachandaskthecontainertoremoveitself,whenit'sstoppedusingthe--rmflag,forexample:$dockerrun--rmhello-world
Inmostreal-lifescenarios,wedon'tusethestoppedcontainers,andtheyareleftonlyforthedebuggingpurpose.
CleaningupimagesImagesarejustasimportantascontainers.Theycanoccupyalotofspace,especiallyinthecaseoftheContinuousDeliveryprocess,wheneachbuildendsupinanewDockerimage.Thiscanquicklyresultinthenospaceleftondeviceerror.TocheckalltheimagesintheDockercontainer,wecanusethedockerimagescommand:$dockerimagesREPOSITORYTAGIMAGEIDCREATEDSIZEhello_world_python_name_defaultlatest9a056ca928412hoursago202.6MBhello_world_python_namelatest72c8c50ffa892hoursago202.6MBhello_world_pythonlatest3e1fa5c29b442hoursago202.6MBubuntu_with_pythonlatestd6e85f39f5b72hoursago202.6MBubuntu_with_git_and_jdklatest8464dc10abbb2hoursago610.9MBubuntu_with_gitlatestf3d674114fe23hoursago259.7MBtomcatlatestc822d296d2322daysago355.3MBubuntu16.04f49eec89601e7daysago129.5MBmongolatest0dffc7177b0611daysago402MBhello-worldlatest48b5124b27682weeksago1.84kB
Toremoveanimage,wecancallthefollowingcommand:
$dockerrmi48b5124b2768
Inthecaseofimages,theautomaticcleanupprocessisslightlymorecomplex.Imagesdon'thavestates,sowecannotaskthemtoremovethemselveswhennotused.ThecommonstrategywouldbetosetuptheCroncleanupjob,whichremovesalloldandunusedimages.Wecoulddothisusingthefollowingcommand:$dockerrmi$(dockerimages-q)
Inordertopreventremovingtheimageswithtags(forexample,tonotremoveallthelatestimages),it'sverycommontousethedanglingparameter:
$dockerrmi$(dockerimages-f"dangling=true"-q)
Ifwehavecontainersthatusevolumes,then,inadditiontoimagesandcontainers,it'sworthtothinkaboutcleaningupvolumes.The
easiestwaytodothisistousethedockervolumels-qfdangling=true|xargs-rdockervolumermcommand.
DockercommandsoverviewAllDockercommandscanbefoundbyexecutingthefollowinghelpcommand:$dockerhelp
ToseealltheoptionsofanyparticularDockercommand,wecanusedockerhelp<command>,forexample:$dockerhelprun
ThereisalsoaverygoodexplanationofallDockercommandsontheofficialDockerpagehttps://docs.docker.com/engine/reference/commandline/docker/.It'sreallyworthreadingoratleastskimmingthrough.
Inthischapter,we'vecoveredthemostusefulcommandsandtheiroptions.Asaquickreminder,let'swalkthroughthem:
Command Explanation
dockerbuild BuildanimagefromaDockerfile
dockercommit Createanimagefromthecontainer
dockerdiff Showchangesinthecontainer
dockerimages Listimages
dockerinfo DisplayDockerinformation
dockerinspect ShowtheconfigurationoftheDockerimage/container
dockerlogs Showlogsofthecontainer
dockernetwork Managenetworks
dockerport Showallexposedportsbythecontainer
dockerps Listcontainers
dockerrm Removecontainer
dockerrmi Removeimage
dockerrun Runacontainerfromtheimage
dockersearch SearchfortheDockerimageinDockerHub
dockerstart/stop/pause/unpause Managethecontainer'sstate
Exercises
We'vecoveredalotofmaterialinthischapter.Tomakewell-remembered,werecommendtwoexercises.
1. RunCouchDBasaDockercontainerandpublishitsport:
YoucanusethedockersearchcommandtofindtheCouchDBimage.
RunthecontainerPublishtheCouchDBportOpenthebrowserandcheckthatCouchDBisavailable
2. CreateaDockerimagewiththeRESTservicereplyingHelloWorld!tolocalhost:8080/hello.Useanylanguageandframeworkyouprefer:
TheeasiestwaytocreateaRESTserviceistousePythonwiththeFlaskframework,http://flask.pocoo.org/.Notethatalotofwebframeworksstarttheapplicationonthelocalhostinterfaceonlybydefault.Inordertopublishaport,it'snecessarytostartitonallinterfaces(app.run(host='0.0.0.0')inthecaseofaFlaskframework).
CreateawebserviceapplicationCreateaDockerfiletoinstalldependenciesandlibrariesBuildtheimageRunthecontainerpublishingtheportCheckifit'srunningcorrectlyusingthebrowser
SummaryInthischapter,wehavecoveredtheDockerbasicsthatareenoughtobuildimagesandrunapplicationsascontainers.Thekeytakeaway,fromthechapterarethefollowingpoints:
ThecontainerizationtechnologyaddressestheissuesofisolationandenvironmentdependenciesusingtheLinuxkernelfeatures.Thisisbasedontheprocessseparationmechanism,thereforenorealperformancedropisobserved.DockercanbeinstalledonmostofthesystemsbutissupportednativelyonlyonLinux.Dockerallowsrunningapplicationsfromtheimagesavailableontheinternetandtobuildownimages.Animageisanapplicationpackedtogetherwithalldependencies.Dockerprovidestwomethodsforbuildingtheimages:Dockerfileorcommittingthecontainer.Inmostcases,thefirstoptionisused.Dockercontainerscancommunicateoverthenetworkbypublishingtheportstheyexpose.Dockercontainerscansharethepersistentstorageusingvolumes.Forthepurposeofconvenience,Dockercontainersshouldbenamed,andDockerimagesshouldbetagged.IntheDockerworld,thereisaspecificconventionofhowtotagimages.Dockerimagesandcontainersshouldbecleanedfromtimetotimeinordertosavetheserverspaceandavoidthenospaceleftondeviceerror.
Inthenextchapter,wewillcoverJenkinsconfigurationandthewayJenkinscanbeusedtogetherwithDocker.
ConfiguringJenkins
WehaveseenhowtoconfigureanduseDocker.Inthischapter,wewillpresentJenkins,whichcanbeusedseparatelyortogetherwithDocker.Wewillshowthatthecombinationofthesetwotoolsoutcomesinthesurprisinglygoodresults:automatedconfigurationandflexiblescalability.
Thischaptercoversthefollowingtopics:
IntroducingJenkinsanditsadvantagesInstallingandstartingJenkinsCreatingthefirstpipelineScalingJenkinswithagentsConfiguringDocker-basedagentsBuildingcustommasterandslaveDockerimagesConfiguringstrategiesforthesecurityandbackup
WhatisJenkins?
JenkinsisanopensourceautomationserverwritteninJava.Withtheveryactivecommunity-basedsupportandahugenumberofplugins,itisthemostpopulartooltoimplementtheContinuousIntegrationandContinuousDeliveryprocesses.FormerlyknownasHudson,itwasrenamedafterOracleboughtHudsonanddecidedtodevelopitastheproprietarysoftware.JenkinsremainedundertheMITlicenseandishighlyvaluedforitssimplicity,flexibility,andversatility.
JenkinsoutstandsotherContinuousIntegrationtoolsandisthemostwidelyusedsoftwareofitskind.Thatisallpossiblebecauseofitsfeaturesandcapabilities.
Let'swalkthroughthemostinterestingpartsoftheJenkins'characteristics.
Languageagnostic:Jenkinshasalotofplugins,whichsupportmostoftheprogramminglanguagesandframeworks.Moreover,sinceitcanuseanyshellcommandandanysoftwareinstalled,itissuitableforeveryautomationprocessthatcanbeimagined.Extensiblebyplugins:Jenkinshasagreatcommunityandalotofpluginsavailable(1000plus).ItalsoallowsyoutowriteyourownpluginsinordertocustomizeJenkinsforyourneeds.Portable:JenkinsiswritteninJava,soitcanberunonanyoperatingsystem.Forconvenience,itisalsodeliveredinalotofversions:webapplicationarchive(WAR),Dockerimage,Windowsbinary,Macbinary,andLinuxbinaries.SupportsmostSCM:Jenkinsintegrateswithvirtuallyeverysourcecodemanagementorbuildtoolthatexists.Again,becauseofitswidecommunityandplugins,thereisnoothercontinuousintegrationtoolthatsupportssomanyexternalsystems.Distributed:Jenkinshasabuilt-inmechanismforthemaster/slavemode,whichdistributesitsexecutionsacrossmultiplenodeslocatedonmultiple
machines.Itcanalsouseheterogeneousenvironment,forexample,differentnodescanhavedifferentoperatingsysteminstalled.Simplicity:Theinstallationandconfigurationprocessissimple.Thereisnoneedtoconfigureanyadditionalsoftware,northedatabase.ItcanbeconfiguredcompletelyviaGUI,XML,orGroovyscripts.Code-oriented:Jenkinspipelinesaredefinedascode.Also,theJenkinsitselfcanbeconfiguredusingXMLfilesorGroovyscripts.ThatallowskeepingtheconfigurationinthesourcecoderepositoryandhelpsintheautomationoftheJenkinsconfiguration.
JenkinsinstallationTheJenkinsinstallationprocessisquickandsimple.Therearedifferentmethodstodoit,butsincewe'realreadyfamiliarwiththeDockertoolandthebenefitsitgives,wewillstartfromtheDocker-basedsolution.Thisisalsotheeasiest,themostpredictable,andthesmartestwaytogo.However,let'smentiontheinstallationrequirementsfirst.
RequirementsforinstallationTheminimalsystemrequirementsarerelativelylow:
Java8256MBfreememory1GB+freediskspace
However,it'sessentialtounderstandthattherequirementsstrictlydependonwhatyouplantodowithJenkins.IfJenkinsisusedtoservethewholeteamastheContinuousIntegrationserver,thenevenincaseofasmallteam,it'sadvisedtohave1GBplusfreememoryand50GBplusfreediskspace.Needlesstosay,Jenkinsalsoperformssomecomputationsandtransfersalotofdataacrossthenetwork,soCPUandbandwidtharecrucial.
Togetafeelingwhatcouldbetherequirementsincaseofabigcompany,theNetflixexampleispresentedintheJenkinsarchitecturesection.
InstallingonDockerLet’sseethestep-by-stepproceduretoinstallJenkinsusingDocker.
TheJenkinsimageisavailableintheofficialDockerHubregistry,soinordertoinstallitweshouldexecutethefollowingcommand:
$dockerrun-p<host_port>:8080-v<host_volume>:/var/jenkins_homejenkins:2.60.1
Weneedtospecifythefirsthost_portparameter—theportunderwhichJenkinsisvisibleoutsideofthecontainer.Thesecondparameter,host_volume,specifiesthedirectorywheretheJenkinshomeismapped.Itneedstobespecifiedasvolume,andthereforepersistedpermanently,becauseitcontainstheconfiguration,pipelinebuilds,andlogs.
Asanexample,let'sseewhattheinstallationstepswouldlooklikeincaseoftheDockerhostonLinux/Ubuntu.
1. Preparethevolumedirectory:WeneedaseparatedirectorywiththeadminownershiptokeeptheJenkinshome.Let'sprepareonewiththefollowingcommands:
$mkdir$HOME/jenkins_home
$chown1000$HOME/jenkins_home
2. RuntheJenkinscontainer:Let'srunthecontainerasadaemonandgiveitapropername:
$dockerrun-d-p49001:8080
-v$HOME/jenkins_home:/var/jenkins_home--name
jenkinsjenkins:2.60.1
3. CheckifJenkinsisrunning:Afteramoment,wecancheckwhetherJenkinshasstartedcorrectlybyprintingthelogs:
$dockerlogsjenkins
Runningfrom:/usr/share/jenkins/jenkins.war
webroot:EnvVars.masterEnvVars.get("JENKINS_HOME")
Feb04,20179:01:32AMMaindeleteWinstoneTempContents
WARNING:FailedtodeletethetemporaryWinstonefile
/tmp/winstone/jenkins.war
Feb04,20179:01:32AMorg.eclipse.jetty.util.log.JavaUtilLoginfo
INFO:Logginginitialized@888ms
Feb04,20179:01:32AMwinstone.LoggerlogInternal
...
Intheproductionenvironment,youmayalsowanttosetupthereverseproxyinordertohidetheJenkinsinfrastructurebehindtheproxyserver.TheshortdescriptionhowtodoitusingtheNginxservercanbefoundathttps://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+with+Docker.
Afterperformingthesefewsteps,Jenkinsisreadytouse.TheDocker-basedinstallationhastwomajoradvantages:
Failurerecovery:IfJenkinscrashes,thenit'senoughtorunanewcontainerwiththesamevolumespecified.Customimages:YoucanconfigureJenkinsasperyourneedsandstoreitastheJenkinsimage.Thenitcanbesharedwithinyourorganizationorteam,andthereisnoneedtorepeatthesameconfigurationstepsallthetime,manytimes.
Everywhereinthisbook,weuseJenkinsinversion2.60.1.
<strong>$wget-q-O-https://pkg.jenkins.io/debian/jenkins.io.key|sudoapt-keyadd-</strong><br/><strong>$sudosh-c'echodebhttp://pkg.jenkins.io/debian-stablebinary/>/etc/apt/sources.list.d/jenkins.list'</strong><br/><strong>$sudoapt-getupdate</strong><br/><strong>$sudoapt-getinstalljenkins</strong>
Allinstallationguides(Ubuntu,Mac,Windows,andothers)canbefoundontheofficialJenkinspagehttps://jenkins.io/doc/book/getting-started/installing/.
InitialconfigurationNomatterwhichinstallationyouchoose,thefirststartofJenkinsrequiresafewconfigurationsteps.Let'swalkthroughthemstepbystep:
1. OpenJenkinsinthebrowser:http://localhost:49001(forbinaryinstallations,thedefaultportis8080).
2. Jenkinsshouldaskfortheadministratorpassword.ItcanbefoundintheJenkinslogs:
$dockerlogsjenkins
...
Jenkinsinitialsetupisrequired.Anadminuserhasbeencreated
andapasswordgenerated.
Pleaseusethefollowingpasswordtoproceedtoinstallation:
c50508effc6843a1a7b06f6491ed0ca6
...
3. Afteracceptingtheinitialpassword,Jenkinsaskswhethertoinstallthesuggestedplugins,whichareadjustedtothemostcommonusecases.Youranswerdepends,ofcourse,onyourneeds.However,asthefirstJenkinsinstallation,it'sreasonabletoletJenkinsinstallalltherecommendedplugins.
4. Aftertheplugininstallation,Jenkinsaskstosetupusername,password,andotherbasicinformation.Ifyouskipit,thetokenfromstep2willbeusedastheadminpassword.
TheinstallationiscompleteandyoushouldseetheJenkinsdashboard:
WearereadytouseJenkinsandcreatethefirstpipeline.
JenkinshelloworldEverythingintheentireITworldstartsfromtheHelloWorldexample.
Let'sfollowthisruleandseethestepstocreatethefirstJenkinspipeline:
1. ClickonNewItem.2. Enterhelloworldastheitemname,choosePipeline,andclickonOK.3. Therearealotofoptions.Wewillskipthemfornowandgodirectlytothe
Pipelinesection.4. There,intheScripttextbox,wecanenterthepipelinescript:
pipeline{
agentany
stages{
stage("Hello"){
steps{
echo'HelloWorld'
}
}
}
}
5. ClickonSave.6. ClickonBuildNow.
Weshouldsee#1undertheBuildHistory.Ifweclickonit,andthenonConsoleOutput,wewillseethelogfromthePipelinebuild.
WehavejustseenthefirstexampleanditssuccessfuloutputmeansthatJenkins
isinstalledcorrectly.Now,let'smovetoslightlymoreadvancedJenkinsconfiguration.
WewilldescribemoreonthepipelinesyntaxinChapter4,ContinuousIntegrationPipeline.
JenkinsarchitectureThehelloworldjobexecutedinalmostnotimeatall.However,thepipelinesareusuallymorecomplexandspendtimeontaskssuchasdownloadingfilesfromtheinternet,compilingthesourcecode,orrunningtests.Onebuildcantakefromminutestohours.
Incommonscenarios,therearealsomanyconcurrentpipelines.Usually,thewholeteam,oreventhewholeorganization,usesthesameJenkinsinstance.Howtoensurethatthebuildswillrunquicklyandsmoothly?
MasterandslavesJenkinsbecomesoverloadedsoonerthanitseems.Evenincaseofasmall(micro)service,thebuildcantakeafewminutes.ThatmeansthatoneteamcommittingfrequentlycaneasilykilltheJenkinsinstance.
Forthatreason,unlesstheprojectisreallysmall,Jenkinsshouldnotexecutebuildsatall,butdelegatethemtotheslave(agent)instances.Tobeprecise,theJenkinswe'recurrentlyrunningiscalledtheJenkinsmasteranditcandelegatetotheJenkinsagents.
Let'slookatthediagrampresentingthemaster-slaveinteraction:
Inadistributedbuildsenvironment,theJenkinsmasterisresponsiblefor:
Receivingbuildtriggers(forexample,afteracommittoGitHub)Sendingnotifications(forexample,emailorHipChatmessagesentafterthebuildfailure)HandlingHTTPrequests(interactionwithclients)
Managingthebuildenvironment(orchestratingthejobexecutionsonslaves)
Thebuildagentisamachinethattakecareofeverythingthathappensafterthebuildisstarted.
Sincetheresponsibilitiesofthemasterandtheslavesaredifferent,theyhavedifferentenvironmentalrequirements:
Master:Thisisusually(unlesstheprojectisreallysmall)adedicatedmachinewithRAMrangingfrom200MBforsmallprojectsto70GBplusforhugesingle-masterprojects.Slave:Therearenogeneralrequirements(otherthanthefactthatitshouldbecapableofexecutingasinglebuild,forexample,iftheprojectisahugemonoliththatrequires100GBofRAM,thentheslavemachineneedstosatisfytheseneeds).
Agentsshouldalsobeasgenericaspossible.Forinstance,ifwehavedifferentprojects:oneinJava,oneinPython,andoneinRuby,thenitwouldbeperfectifeachagentcouldbuildanyoftheseprojects.Insuchacase,theagentscanbeinterchanged,whichhelpstooptimizetheusageofresources.
Ifagentscannotbegenericenoughtomatchallprojects,thenit'spossibletolabel(tag)agentsandprojects,sothatthegivenbuildwouldbeexecutedonagiventypeofagent.
ScalabilityWecanuseJenkinsslavestobalancetheloadandscaleuptheJenkinsinfrastructure.Suchaprocessiscalledthehorizontalscaling.Theotherpossibilitywouldbetouseonlyonemasternodeandincreaseresourcesofitsmachine.Thatprocessiscalledtheverticalscaling.Let'slookcloseratthesetwoconcepts.
VerticalscalingVerticalscalingmeansthat,whenthemaster'sloadgrows,thenmoreresourcesareappliedtothemaster'smachine.So,whennewprojectsappearinourorganization,webuymoreRAM,addCPUcores,andextendtheHDDdrives.Thismaysoundlikeano-gosolution;however,it'softenused,evenbywell-knownorganizations.HavingasingleJenkinsmastersetontheultra-efficienthardwarehasoneverystrongadvantage:maintenance.Anyupgrades,scripts,securitysettings,rolesassignment,orplugininstallationshavetobedoneinoneplaceonly.
Horizontalscaling
Horizontalscalingmeansthat,whenanorganizationgrows,thenmoremasterinstancesarelaunched.Thisrequiresasmartallocationofinstancestoteamsand,intheextremecase,eachteamcanhaveitsownJenkinsmaster.Inthatcase,itmightevenhappenthatnoslavesareneeded.
Thedrawbacksarethatitmaybedifficulttoautomatecross-projectintegrationsandthatapartoftheteam'sdevelopmenttimeisspentontheJenkinsmaintenance.However,thehorizontalscalinghassomesignificantadvantages:
Mastermachinesdon'tneedtobespecialintermsofhardwareDifferentteamscanhavedifferentJenkinssettings(forexample,differentsetofplugins)TeamsusuallyfeelbetterandworkwithJenkinsmoreefficientlyiftheinstanceistheirownIfonemasterinstanceisdown,itdoesnotimpactthewholeorganizationTheinfrastructurecanbesegregatedintostandardandmission-criticalSomemaintenanceaspectscanbesimplified,forexample,theteamoffivecouldreusethesameJenkinspassword,sowemayskiptherolesandsecuritysettings(surelythatispossibleonlyifthecorporatenetworkiswellfirewalled)
TestandproductioninstancesApartfromthescalingapproach,thereisonemoreissue:howtotesttheJenkinsupgrades,newplugins,orpipelinedefinitions?Jenkinsiscriticaltothewholecompany.Itguaranteesthequalityofthesoftwareand(incaseofContinuousDelivery)deploystotheproductionservers.Thatiswhyitneedstobehighlyavailable,soitisdefinitelynotforthepurposeoftesting.ItmeansthereshouldalwaysbetwoinstancesofthesameJenkinsinfrastructure:testandproduction.
Testenvironmentshouldalwaysbeassimilaraspossibletotheproduction,soitalsorequiresthesimilarnumberofagentsattached.
SamplearchitectureWealreadyknowthatthereshouldbeslaves,(possiblymultiple)master(s),andthateverythingshouldbeduplicatedintothetestandproductionenvironments.However,whatwouldthecompletepicturelooklike?
Luckily,therearealotofcompaniesthatpublishedhowtheyusedJenkinsandwhatkindofarchitecturestheycreated.Itwouldbedifficulttomeasureifmoreofthempreferredverticalorhorizontalscaling,butitrangedfromhavingonlyonemasterinstancetohavingonemasterforeachteam.
Let'slookattheexampleofNetflixtohaveacompletepictureoftheJenkinsinfrastructure(theyshareditastheplannedinfrastructureatJenkinsUserConferenceSanFrancisco2012):
Theyhavetestandproductionmasterinstances,eachofthemowningapollofslavesandadditionaladhocslaves.Altogether,itservesaround2000buildsperday.NotealsothatapartoftheirinfrastructureishostedonAWSandapartisontheirownservers.
WeshouldalreadyhaveatleastaroughideaofwhattheJenkinsinfrastructure
canlooklikedependingonthetypeoftheorganization.
Let'snowfocusonthepracticalaspectsofsettingtheagents.
ConfiguringagentsWe'veseenwhattheagentsareandwhentheycanbeused.However,howtosetupanagentandletitcommunicatewiththemaster?Let'sstartwiththesecondpartofthequestionanddescribethecommunicationprotocolsbetweenthemasterandtheagent.
CommunicationprotocolsInorderforthemasterandtheagenttocommunicate,thebi-directionalconnectionhastobeestablished.
Therearedifferentoptionshowitcanbeinitiated:
SSH:MasterconnectstoslaveusingthestandardSSHprotocol.JenkinshasanSSH-clientbuilt-in,sotheonlyrequirementistheSSHDserverconfiguredonslaves.ThisisthemostconvenientandstablemethodbecauseitusesstandardUnixmechanisms.JavaWebStart:JavaapplicationisstartedoneachagentmachineandtheTCPconnectionisestablishedbetweentheJenkinsslaveapplicationandthemasterJavaapplication.Thismethodisoftenusediftheagentsareinsidethefirewallednetworkandthemastercannotinitiatetheconnection.Windowsservice:ThemasterregistersanagentontheremotemachineasaWindowsservice.Thismethodisdiscouragedsincethesetupistrickyandtherearelimitationsonthegraphicalinterfacesusage.
Ifweknowthecommunicationprotocols,let'sseehowwecanusethemtosettheagents.
SettingagentsAtthelowlevel,agentscommunicatewiththeJenkinsmasteralwaysusingoneoftheprotocolsdescribedabove.However,atthehigherlevel,wecanattachslavestothemasterinvariousways.Thedifferencesconcerntwoaspects:
staticversusdynamic:ThesimplestoptionistoaddslavespermanentlyintheJenkinsmaster.Thedrawbackofsuchsolutionisthatwealwaysneedtomanuallychangesomethingifweneedmore(orless)slavenodes.Abetteroptionistodynamicallyprovisionslavesastheyareneeded.specificversusgeneral-purpose:Agentscanbespecific(forexample,differentagentsfortheprojectsbasedonJava7anddifferentagentsforJava8)orgeneral-purpose(anagentactsasaDockerhostandapipelineisbuiltinsideaDockercontainer).
Thesedifferencesresultedinfourcommonstrategieshowagentsareconfigured:
PermanentagentsPermanentDockeragentsJenkinsSwarmagentsDynamicallyprovisionedDockeragents
Let'sexamineeachofthesolutions.
PermanentagentsWestartfromthesimplestoptionwhichistopermanentlyaddspecificagentnodes.ItcanbedoneentirelyviatheJenkinswebinterface.
ConfiguringpermanentagentsIntheJenkinsmaster,whenweopenManageJenkinsandthenManageNodes,wecanviewalltheattachedagents.Then,byclickingonNewNode,givingitaname,andconfirmingwiththeOKbutton,weshouldfinallyseetheagent's
setuppage:
Let'swalkthroughtheparametersweneedtofill:
Name:ThisistheuniquenameoftheagentDescription:Thisisanyhuman-readabledescriptionoftheagent#ofexecutors:ThisisthenumberofconcurrentbuildsthatcanberunontheslaveRemoterootdirectory:Thisisthededicateddirectoryontheslavemachinethattheagentcanusetorunbuildjobs(forexample,/var/jenkins);themostimportantdataistransferredbacktothemaster,sothedirectoryisnotcriticalLabels:Thisincludesthetagstomatchonlythespecificbuilds(taggedthesame),forexample,onlyprojectsbasedonJava8
Usage:Thisistheoptiontodecidewhethertheagentshouldbeusedonlyformatchedlabels(forexample,onlyforAcceptanceTestingbuilds)orforanybuilds
Launchmethod:Thisincludesthefollowing:
LaunchagentviaJavaWebStart:Here,theconnectionwillbeestablishedbytheagent;itispossibletodownloadtheJARfileandtheinstructionsonhowtorunitontheslavemachine
Launchagentviaexecutionofcommandonthemaster:Thisisthecustomcommandrunonthemastertostarttheslave;inmostcasesitwillsendtheJavaWebStartJARapplicationandstartitontheslave(forexample,ssh<slave_hostname>java-jar~/bin/slave.jar)LaunchslaveagentsviaSSH:Here,themasterwillconnecttotheslaveusingtheSSHprotocolLetJenkinscontrolthisWindowsslaveasaWindowsservice:Here,themasterwillstartaremotemanagementfacilitybuiltintoWindows
Availability:Thisistheoptiontodecidewhethertheagentshouldbeupallthetimeorthemastershouldturnitofflineundercertainconditions
Whentheagentsaresetupcorrectly,it'spossibletoswitchthemasternodeoffline,sothatnobuildswouldbeexecutedonitanditwouldserveonlyastheJenkinsUIandthebuilds'coordinator.
UnderstandingpermanentagentsAsalreadymentioned,thedrawbackofsuchasolutionisthatweneedtomaintainmultipleslavetypes(labels)fordifferentprojecttypes.Suchasituationispresentedinthefollowingdiagram:
Inourexample,ifwehavethreetypesofprojects(java7,java8,andruby),thenweneedtomaintainthreeseparatelylabeled(setsof)slaves.Thatisthesameissuewehadwhilemaintainingmultipleproductionservertypes,asdescribedinChapter2,IntroducingDocker.WeaddressedtheissuebyhavingtheDockerEngineinstalledontheproductionservers.Let'strytodothesamewithJenkinsslaves.
PermanentDockeragentsTheideabehindthissolutionistopermanentlyaddgeneral-purposeslaves.Eachslaveisidenticallyconfigured(DockerEngineinstalled)andeachbuildisdefinedtogetherwiththeDockerimageinsidewhichthebuildisrun.
pipeline{<br/>agent{<br/>docker{<br/>image'openjdk:8-jdk-alpine'<br/>}<br/>}<br/>...<br/>}
Whenthebuildisstarted,theJenkinsslavestartsacontainerfromtheDockerimageopenjdk:8-jdk-alpineandthenexecutesallpipelinestepsinsidethatcontainer.Thisway,wealwaysknowtheexecutionenvironmentanddon'thavetoconfigureeachslaveseparatelydependingontheparticularprojecttype.
UnderstandingpermanentDockeragentsLookingatthesamescenariowetookforthepermanentagents,thediagramlookslikethis:
EachslaveisexactlythesameandifwewouldliketobuildaprojectthatdependsonJava8,thenwedefinetheappropriateDockerimageinthepipelinescript(insteadofspecifyingtheslavelabel).
JenkinsSwarmagentsSofar,wealwayshadtopermanentlydefineeachoftheagentsintheJenkinsmaster.Suchasolution,eventhoughgoodenoughinmanycases,canbeaburdenifweneedtofrequentlyscalethenumberofslavemachines.JenkinsSwarmallowsyoutodynamicallyaddslaveswithouttheneedtoconfigurethemintheJenkinsmaster.
ConfiguringJenkinsSwarmagentsThefirststeptouseJenkinsSwarmistoinstalltheSelf-OrganizingSwarmPlug-inModulesplugininJenkins.WecandoitviatheJenkinswebUIunderManageJenkinsandManagePlugins.Afterthisstep,theJenkinsmasterispreparedforJenkinsslavestobedynamicallyattached.
ThesecondstepistoruntheJenkinsSwarmslaveapplicationoneverymachinethatwouldactasaJenkinsslave.Wecandoitusingtheswarm-client.jarapplication.
Theswarm-client.jarapplicationcanbedownloadedfromtheJenkinsSwarmpluginpage:https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin.Onthatpage,youcanalsofindallthepossibleoptionsofitsexecution.
InordertoattachtheJenkinsSwarmslavenode,it'senoughtorunthefollowingcommand:
$java-jarswarm-client.jar-master<jenkins_master_url>-username<jenkins_master_user>-password<jenkins_master_password>-namejenkins-swarm-slave-1
Bythetimeofwritingthisbook,therewasanopenbugthatclient-slave.jardidn'tworkviathesecuredHTTPSprotocol,soitwasnecessarytoaddthe-disableSslVerificationoptiontothecommandexecution.
Afterthesuccessfulexecution,weshouldnoticethatanewslaveappearedontheJenkinsmasteraspresentedonthescreenshot:
Now,whenwerunabuild,itwillbestartedonthisagent.
TheotherpossibilitytoaddtheJenkinsSwarmagentistousethe
Dockerimagebuiltfromtheswarm-client.jartool.ThereareafewofthemavailableontheDockerHub.Wecanusethecsanchez/jenkins-swarm-slaveimage.
UnderstandingJenkinsSwarmagentsJenkinsSwarmallowstodynamicallyaddagents,butitsaysnothingaboutwhethertousespecificorDocker-basedslaves,sowecanuseitforboth.Atfirstglance,JenkinsSwarmmaynotseemveryuseful.Afterall,wemovedsettingagentsfrommastertoslave,butstillhavetodoitmanually.However,aswewillseelaterinChapter8,ClusteringwithDockerSwarm,JenkinsSwarmenablesdynamicscalingofslavesonaclusterofservers.
DynamicallyprovisionedDockeragentsAnotheroptionistosetupJenkinstodynamicallycreateanewagenteachtimeabuildisstarted.Suchasolutionisobviouslythemostflexibleonesincethenumberofslavesdynamicallyadjusttothenumberofbuilds.Let'shavealookathowtoconfigureJenkinsthisway.
ConfiguringdynamicallyprovisionedDockeragentsWeneedtofirstinstalltheDockerplugin.AsalwayswithJenkinsplugins,wecandoinManageJenkinsandManagePlugins.Afterthepluginisinstalled,wecanstartthefollowingconfigurationsteps:
1. OpentheManageJenkinspage.2. ClickontheConfigureSystemlink.3. Atthebottomofthepage,thereistheCloudsection.4. ClickonAddanewcloudandchooseDocker.5. FilltheDockeragentdetails.
6. Mostparametersdonotneedtobechanged;however,weneedtosettwoofthemasfollows:
DockerURL:TheaddressoftheDockerhostmachinewhereagentswillberunCredentials:ThecredentialsincasetheDockerhostrequires
authentication
IfyouplantousethesameDockerhostwherethemasterisrunning,thentheDockerdaemonneedstolistenonthedocker0networkinterface.YoucandoitinasimilarwayasdescribedintheInstallingonaserversectionoftsof)slaves.Thatisthesameissuewehadwhilemaintainingmultipleproductionservertypes,asdescribedinChapter2,IntroducingDocker,bychangingonelineinthe/lib/systemd/system/docker.servicefiletoExecStart=/usr/bin/dockerd-H0.0.0.0:2375-Hfd://
7. ClickonAddDockerTemplateandselectDockerTemplate.8. FillthedetailsabouttheDockerslaveimage:
Wecanusethefollowingparameters:
DockerImage:ThemostpopularslaveimagefromtheJenkinscommunity
isevarga/jenkins-slaveCredentials:Thecredentialstotheevarga/jenkins-slaveimageare:
username:jenkinspassword:jenkins
Instancecapacity:Thisdefinesthemaximumnumberofagentsrunningatthesametime;forthebeginning,itcanbesetto10
Insteadofevarga/jenkins-slave,it'spossibletobuildanduseyourownslaveimages.Thisisnecessarywhentherearespecificenvironmentrequirements,forexample,thePythoninterpreterinstalled.Inallexamplesforthisbookweusedleszko/jenkins-docker-slave.
Aftersaving,everythingissetup.WecouldrunthepipelinetoobservethattheexecutionreallytakesplaceontheDockeragent,butfirstlet'sdigalittledeepertounderstandhowtheDockeragentswork.
UnderstandingdynamicallyprovisionedDockeragentsDynamicallyprovisionedDockeragentscanbetreatedasalayeroverthestandardagentmechanism.Itchangesneitherthecommunicationprotocolnorhowtheagentiscreated.So,whatdoesJenkinsdowiththeDockeragentconfigurationweprovided?
ThefollowingdiagrampresentstheDockermaster-slavearchitecturewe'veconfigured:
Let'sdescribestepbystephowtheDockeragentmechanismisused:
1. WhentheJenkinsjobisstarted,themasterrunsanewcontainerfromthejenkins-slaveimageontheslaveDockerhost.
2. Thejenkins-slavecontaineris,actually,theubuntuimagewiththeSSHDserverinstalled.
3. TheJenkinsmasterautomaticallyaddsthecreatedagenttotheagentlist(sameaswhatwedidmanuallyintheSettingagentssection).
4. TheagentisaccessedusingtheSSHcommunicationprotocoltoperformthebuild.
5. Afterthebuild,themasterstopsandremovestheslavecontainer.
RunningJenkinsmasterasaDockercontainerisindependentfrom
runningJenkinsagentsasDockercontainers.It'sreasonabletodoboth,butanyofthemwillworkseparately.
ThesolutionissomehowsimilartothepermanentDockeragentssolution,becauseinresult,werunthebuildinsideaDockercontainer.Thedifferenceis,however,intheslavenodeconfiguration.Here,thewholeslaveisdockerized,notonlythebuildenvironment.Therefore,ithastwogreatadvantagesasfollows:
Automaticagentlifecycle:Theprocessofcreating,adding,andremovingtheagentisautomated.Scalability:Actually,theslaveDockerhostcouldbenotasinglemachine,butaclustercomposedofmultiplemachines(we'llcoverclusteringusingDockerSwarminChapter8,ClusteringwithDockerSwarm).Inthatcase,addingmoreresourcesisassimpleasaddinganewmachinetotheclusteranddoesnotrequireanychangesinJenkins.
Jenkinsbuildusuallyneedstodownloadalotofprojectdependencies(forexample,Gradle/Mavendependencies),whichmaytakealotoftime.IfDockerslavesareautomaticallyprovisionedforeachbuild,thenitmaybeworthtosetupaDockervolumeforthemtoenablecachingbetweenthebuilds.
TestingagentsNomatterwhichagentconfigurationyouchose,weshouldnowcheckifitworkscorrectly.
Let'sgobacktothehelloworldpipeline.Usually,thebuildslastlongerthanthehello-worldexample,sowecansimulateitbyaddingsleepingtothepipelinescript:
pipeline{
agentany
stages{
stage("Hello"){
steps{
sleep300//5minutes
echo'HelloWorld'
}
}
}
}
AfterclickingonBuildNowandgoingtotheJenkinsmainpage,weshouldseethatthebuildisexecutedonanagent.Now,ifweclickonbuildmanytimes,thendifferentagentsshouldbeexecutingdifferentbuilds(asshowninthefollowingscreenshot):
Topreventjobexecutionsonmaster,rememberaboutturningthemasternodeofflineorsetting#ofexecutorsto0intheManageNodesconfiguration.
Havingseenthattheagentsareexecutingourbuilds,we'veconfirmedthattheyareconfiguredcorrectly.Now,let'sseehowandforwhatreasonwecouldcreateourownJenkinsimages.
CustomJenkinsimagesSofar,wehaveusedtheJenkinsimagespulledfromtheinternet.Weusedjenkinsforthemastercontainerandevarga/jenkins-slavefortheslavecontainer.However,wemaywanttobuildourownimagestosatisfythespecificbuildenvironmentrequirements.Inthissection,wecoverhowtodoit.
BuildingJenkinsslaveLet'sstartfromtheslaveimage,becauseit'smoreoftencustomized.Thebuildexecutionisperformedontheagent,soit'stheagentthatneedstohavetheenvironmentadjustedtotheprojectwewouldliketobuild.Forexample,itmayrequirethePythoninterpreterifourprojectiswritteninPython.Thesameisappliedtoanylibrary,tool,testingframework,oranythingthatisneededbytheproject.
Youcancheckwhatisalreadyinstalledinsidetheevarga/jenkins-slaveimagebylookingatitsDockerfileathttps://github.com/evarga/docker-images.
Therearethreestepstobuildandusethecustomimage:
1. CreateaDockerfile.2. Buildtheimage.3. Changetheagentconfigurationonmaster.
Asanexample,let'screateaslavethatservesthePythonproject.Wecanbuilditontopoftheevarga/jenkins-slaveimageforthesakeofsimplicity.Let'sdoitusingthefollowingthreesteps:
1. Dockerfile:Let'screateanewdirectoryinsidetheDockerfilewiththefollowingcontent:
FROMevarga/jenkins-slave
RUNapt-getupdate&&\
apt-getinstall-ypython
ThebaseDockerimageevarga/jenkins-slaveissuitableforthedynamicallyprovisionedDockeragentssolution.IncaseofpermanentDockeragents,it'senoughtousealpine,ubuntu,oranyotherimage,sinceit'snottheslavethatisdockerized,butonlythebuildexecutionenvironment.
2. Buildtheimage:Wecanbuildtheimagebyexecutingthefollowingcommand:
$dockerbuild-tjenkins-slave-python.
3. Configurethemaster:Thelaststep,ofcourse,istosetjenkins-slave-pythoninsteadofevarga/jenkins-slaveintheJenkinsmaster'sconfiguration(asdescribedintheSettingDockeragentsection).
TheslaveDockerfileshouldbekeptinthesourcecoderepositoryandtheimagebuildcanbeperformedautomaticallybyJenkins.ThereisnothingwrongwithbuildingthenewJenkinsslaveimageusingtheoldJenkinsslave.
WhatifweneedJenkinstobuildtwodifferentkindsofprojects,forexample,onebasedonPythonandanotherbasedonRuby?Inthatcase,wecouldprepareanagent,whichisgenericenoughtosupportboth:PythonandRuby.However,incaseofDocker,it'srecommendedtocreatethesecondslaveimage(jenkins-slave-rubybyanalogy).Then,intheJenkinsconfigurationweneedtocreatetwoDockertemplatesandlabelthemaccordingly.
BuildingJenkinsmasterWealreadyhaveacustomslaveimage.Whywouldwealsowanttobuildourownmasterimage?Oneofthereasonsmightbethatwedon'twanttouseslavesatall,and,sincetheexecutionwouldbedoneonmaster,itsenvironmenthastobeadjustedtotheproject'sneeds.Thatis,however,averyrarecase.Moreoften,wewillwanttoconfigurethemasteritself.
Imaginethefollowingscenario,yourorganizationscalesJenkinshorizontallyandeachteamhasitsowninstance.Thereis,however,somecommonconfiguration,forexample:asetofbaseplugins,backupstrategies,orthecompanylogo.Then,repeatingthesameconfigurationforeachoftheteamsisawasteoftime.So,wecanpreparethesharedmasterimageandlettheteamsuseit.
JenkinsisconfiguredusingXMLfilesanditprovidestheGroovy-basedDSLlanguagetomanipulateoverthem.ThatiswhywecanaddtheGroovyscripttotheDockerfileinordertomanipulatetheJenkinsconfiguration.Whatismore,therearespecialscriptstohelpwiththeJenkinsconfigurationifitrequiressomethingmorethanXMLchanges,forinstance,plugininstallation.
AllpossibilitiesoftheDockerfileinstructionsarewelldescribedontheGitHubpagehttps://github.com/jenkinsci/docker.
Asanexample,let'screateamasterimagewiththedocker-pluginalreadyinstalledandanumberofexecutorssetto5.Inordertodoit,weneedto:
1. CreatetheGroovyscripttomanipulateonconfig.xmlandsetthenumberofexecutorsto5.
2. CreatetheDockerfiletoinstalldocker-pluginandexecutetheGroovyscript.
3. Buildtheimage.
Let'susethethreestepsmentionedandbuildtheJenkinsmasterimage.
1. Groovyscript:Let'screateanewdirectoryinsidetheexecutors.groovyfilewiththefollowingcontent:
importjenkins.model.*
Jenkins.instance.setNumExecutors(5)
ThecompleteJenkinsAPIcanbefoundontheofficialpagehttp://javadoc.jenkins.io/.
2. Dockerfile:Inthesamedirectory,let'screatetheDockerfile:
FROMjenkins
COPYexecutors.groovy
/usr/share/jenkins/ref/init.groovy.d/executors.groovy
RUN/usr/local/bin/install-plugins.shdocker-plugin
3. Buildtheimage:Wecanfinallybuildtheimage:
$dockerbuild-tjenkins-master.
Aftertheimageiscreated,eachteamintheorganizationcanuseittolaunchtheirownJenkinsinstance.
Havingourownmasterandslaveimagesletsusprovidetheconfigurationandthebuildenvironmentfortheteamsinourorganization.Inthenextsection,we'llseewhatelseisworthbeingconfiguredinJenkins.
ConfigurationandmanagementWehavealreadycoveredthemostcrucialpartoftheJenkinsconfiguration:agentsprovisioning.SinceJenkinsishighlyconfigurable,youcanexpectmuchmorepossibilitiestoadjustittoyourneeds.Thegoodnewsisthattheconfigurationisintuitiveandaccessibleviathewebinterface,soitdoesnotrequireanydetaileddescription.EverythingcanbechangedundertheManageJenkinssubpage.Inthissection,wewillfocusonlyonafewaspectsthataremostlikelytobechanged:plugins,security,andbackup.
PluginsJenkinsishighlyplugin-oriented,whichmeansthatalotoffeaturesaredeliveredbytheuseofplugins.TheycanextendJenkinsalmostintheunlimitedway,which,takingintoconsiderationthelargecommunity,isoneofthereasonswhyJenkinsissuchasuccessfultool.WithJenkins'openness,comestheriskandit'sbettertodownloadpluginsonlyfromthereliablesourceorchecktheirsourcecode.
Thereareliterallytonsofpluginstochoosefrom.Someofthemwerealreadyinstalledautomaticallyduringtheinitialconfiguration.Anotherone(Dockerplugin)wasinstalledwhilesettingtheDockeragents.Therearepluginsforcloudintegration,sourcecontroltools,codecoverage,andmuchmore.Youcanalsowriteyourownplugin,butit'sbettertocheckfirstiftheoneyouneedisnotalreadywritten.
ThereisanofficialJenkinspagetobrowsepluginsfromhttps://plugins.jenkins.io/.
SecurityThewayyoushouldapproachtheJenkinssecuritydependsontheJenkinsarchitectureyouhavechosenwithinyourorganization.IfyouhaveaJenkinsmasterforeverysmallteam,thenyoumaynotneeditatall(undertheassumptionthatthecorporatenetworkisfirewalled).However,ifyouhaveasingleJenkinsmasterinstanceforthewholeorganization,thenyou'dbetterbesureyousecureditwell.
Jenkinscomeswithitsownuserdatabase-wehavealreadycreatedauserduringtheinitialconfigurationprocess.Youcancreate,delete,andmodifyusersbyopeningtheManageUserssettingpage.Thebuilt-indatabasecanbeasolutionincaseofsmallorganizations;however,forthelargegroupofusers,youwillprobablywanttouseLDAPinstead.YoucanchooseitontheConfigureGlobalSecuritypage.There,youcanalsoassignroles,groups,andusers.Bydefault,theLogged-inuserscandoanythingoptionisset,butinalarge-scaleorganizationyoushouldprobablythinkofmoredetailedgranularity.
BackupAstheoldsayinggoes:"Therearetwotypesofpeople:thosewhobackup,andthosewhowillbackup".Believeitornot,thebackupissomethingyouprobablywanttoconfigure.Whatfilestobackupandfromwhichmachines?Luckily,agentsautomaticallysendalltherelevantdatabacktothemaster,sowedon'tneedtobotheraboutthem.IfyourunJenkinsinthecontainer,thenthecontaineritselfisalsonotinteresting,sinceitdoesnotholdanypersistentstate.TheonlyplaceweareinterestedinistheJenkinshomedirectory.
WecaneitherinstallaJenkinsplugin(whichwillhelpustosetperiodicbackups)orsimplysetacronjobtoarchivethedirectoryintoasafeplace.Toreducethesize,wecanexcludethesubfolderswhicharenotinteresting(thatwilldependonyourneeds;however,almostcertainly,youdon'tneedtocopy:"war","cache","tools",and"workspace").
Therearequiteafewplugins,whichcanhelpwiththebackupprocess;themostcommononeiscalledBackupPlugin.
BlueOceanUIThefirstversionofHudson(formerJenkins)wasreleasedin2005.It'sbeenonthemarketformorethan10yearsnow.However,itslookandfeelhasn'tchangedmuch.We'vealreadyuseditforawhileandit'shardtodenythatitlooksoutdated.BlueOceanistheplugin,whichredefinestheuserexperienceofJenkins.IfJenkinsisaestheticallydispleasingtoyou,thenit'sdefinitelyworthgivingatry.
YoucanreadmoreontheBlueOceanpageathttps://jenkins.io/projects/blueocean/.
Exercises
WehavelearnedalotaboutJenkinsconfigurationthroughoutthischapter.Toconsolidatetheknowledge,werecommendtwoexercisesonpreparingtheJenkinsimagesandtestingtheJenkinsenvironment.
1. CreateJenkinsmasterandslaveDockerimagesandusethemtoruntheJenkinsinfrastructurecapableofbuildingtheRubyprojects:
CreatethemasterDockerfile,whichautomaticallyinstallstheDockerplugin.BuildthemasterimageandruntheJenkinsinstanceCreatetheslaveDockerfile(suitableforthedynamicslaveprovisioning),whichinstallstheRubyinterpreterBuildtheslaveimageChangetheconfigurationintheJenkinsinstancetousetheslaveimage
2. Createapipeline,whichrunsaRubyscriptprintingHelloWorldfromRuby:
CreateanewpipelineUsethefollowingshellcommandtocreatethehello.rbscriptonthefly:sh"echo"puts'HelloWorldfromRuby'">hello.rb"
Addthecommandtorunhello.rbusingtheRubyinterpreterRunthebuildandobservetheconsoleoutput
SummaryInthischapter,wehavecoveredtheJenkinsenvironmentanditsconfiguration.TheknowledgegainedissufficienttosetupthecompleteDocker-basedJenkinsinfrastructure.Thekeytakeawayfromthechapterisasfollows:
Jenkinsisageneral-purposeautomationtoolthatcanbeusedwithanylanguageorframework.Jenkinsishighlyextensiblebyplugins,whichcanbewrittenorfoundontheinternet.JenkinsiswritteninJava,soitcanbeinstalledonanyoperatingsystem.It'salsoofficiallydeliveredasaDockerimage.Jenkinscanbescaledusingthemaster-slavearchitecture.Themasterinstancescanbescaledhorizontallyorverticallydependingontheorganization'sneeds.Jenkins'agentscanbeimplementedwiththeuseofDocker,whichhelpsinautomaticconfigurationanddynamicslavesallocation.CustomDockerimagescanbecreatedforboth:JenkinsmasterandJenkinsslave.Jenkinsishighlyconfigurableandtheaspectsthatshouldalwaysbeconsideredare:securityandbackups.
Inthenextchapter,wewillfocusonthepartthatwe'vealreadytouchedwiththe"helloworld"example,pipelines.WewilldescribetheideaandthemethodtobuildacompleteContinuousIntegrationpipeline.
ContinuousIntegrationPipeline
WealreadyknowhowtoconfigureJenkins.Inthischapter,youwillseehowtouseiteffectively,focusingonthefeaturethatlaysattheheartofJenkins,pipelines.BybuildingacompleteContinuousIntegrationprocessfromscratch,wewilldescribeallaspectsofmodernteam-orientedcodedevelopment.
Thischaptercoversthefollowingpoints:
ExplainingtheideaofpipeliningIntroducingtheJenkinspipelinesyntaxCreatingaContinuousIntegrationpipelineExplainingtheideaofJenkinsfileCreatingcodequalitychecksAddingpipelinetriggersandnotificationsExplainingdevelopmentworkflowsandbranchingstrategiesIntroducingJenkinsMultibranch
IntroducingpipelinesApipelineisasequenceofautomatedoperationsthatusuallyrepresentsapartofsoftwaredeliveryandthequalityassuranceprocess.Itcanbesimplyseenasachainofscriptsprovidingthefollowingadditionalbenefits:
Operationgrouping:Operationsaregroupedtogetherintostages(alsoknownasgatesorqualitygates)thatintroduceastructureintotheprocessandclearlydefinestherule:ifonestagefails,nofurtherstagesareexecutedVisibility:Allaspectsoftheprocessarevisualized,whichhelpinquickfailureanalysisandpromotesteamcollaborationFeedback:Teammemberslearnaboutanyproblemsassoonastheyoccur,sotheycanreactquickly
TheconceptofpipeliningissimilarformostContinuousIntegrationtools,however,thenamingcandiffer.Inthisbook,westicktotheJenkinsterminology.
PipelinestructureAJenkinspipelineconsistsoftwokindsofelements:stagesandsteps.Thefollowingfigureshowshowtheyareused:
Thefollowingarethebasicpipelineelements:
Step:Asingleoperation(tellsJenkinswhattodo,forexample,checkoutcodefromrepository,executeascript)Stage:Alogicalseparationofsteps(groupsconceptuallydistinctsequencesofsteps,forexample,Build,Test,andDeploy)usedtovisualizetheJenkinspipelineprogress
Technically,it'spossibletocreateparallelsteps;however,it'sbettertotreatitasanexceptionwhenreallyneededforoptimizationpurposes.
Multi-stageHelloWorldAsanexample,let'sextendtheHelloWorldpipelinetocontaintwostages:
pipeline{
agentany
stages{
stage('FirstStage'){
steps{
echo'Step1.HelloWorld'
}
}
stage('SecondStage'){
steps{
echo'Step2.SecondtimeHello'
echo'Step3.ThirdtimeHello'
}
}
}
}
Thepipelinehasnospecialrequirementsintermsofenvironment(anyslaveagent),anditexecutesthreestepsinsidetwostages.WhenweclickonBuildNow,weshouldseethevisualrepresentation:
Thepipelinesucceeded,andwecanseethestepexecutiondetailsbyclickingontheconsole.Ifanyofthestepsfailed,theprocessingwouldstopandnofurtherstepswouldrun.Actually,theentirereasonforapipelineistopreventallfurtherstepsfromexecutionandvisualizethepointoffailure.
PipelinesyntaxWehavediscussedthepipelineelementsandalreadyusedafewofthepipelinesteps,forexample,echo.Whatotheroperationscanweuseinsidethepipelinedefinition?
Inthisbook,weusethedeclarativesyntaxthatisrecommendedforallnewprojects.ThedifferentoptionsareGroovy-basedDSLand(priortoJenkins2)XML(createdviathewebinterface).
Thedeclarativesyntaxwasdesignedtomakeitassimpleaspossibletounderstandthepipeline,evenbythepeoplewhodonotwritecodeonadailybasis.Thisiswhythesyntaxislimitedonlytothemostimportantkeywords.
Let'sprepareanexperimentand,beforewedescribeallthedetails,readthefollowingpipelinedefinitionandtrytoguesswhatitdoes:
pipeline{
agentany
triggers{cron('*****')}
options{timeout(time:5)}
parameters{
booleanParam(name:'DEBUG_BUILD',defaultValue:true,
description:'Isitthedebugbuild?')
}
stages{
stage('Example'){
environment{NAME='Rafal'}
when{expression{returnparams.DEBUG_BUILD}}
steps{
echo"Hellofrom$NAME"
script{
defbrowsers=['chrome','firefox']
for(inti=0;i<browsers.size();++i){
echo"Testingthe${browsers[i]}browser."
}
}
}
}
}
post{always{echo'IwillalwayssayHelloagain!'}}
}
Hopefully,thepipelinedidn'tscareyou.Itisquitecomplex.Actually,itissocomplexthatitcontainsallpossibleJenkinsinstructions.Toanswertheexperimentpuzzle,let'sseewhatthepipelinedoesinstructionbyinstruction:
1. Useanyavailableagent.2. Executeautomaticallyeveryminute.3. Stopiftheexecutiontakesmorethan5minutes.4. AskfortheBooleaninputparameterbeforestarting.5. SetRafalastheenvironmentvariableNAME.6. Onlyinthecaseofthetrueinputparameter:
PrintHellofromRafalPrintTestingthechromebrowserPrintTestingthefirefoxbrowser
7. PrintIwillalwayssayHelloagain!nomatterifthereareanyerrorsduringtheexecution.
Let'sdescribethemostimportantJenkinskeywords.Adeclarativepipelineisalwaysspecifiedinsidethepipelineblockandcontainssections,directives,andsteps.Wewillwalkthrougheachofthem.
ThecompletepipelinesyntaxdescriptioncanbefoundontheofficialJenkinspageathttps://jenkins.io/doc/book/pipeline/syntax/.
SectionsSectionsdefinethepipelinestructureandusuallycontainoneormoredirectivesorsteps.Theyaredefinedwiththefollowingkeywords:
Stages:ThisdefinesaseriesofoneormorestagedirectivesSteps:ThisdefinesaseriesofoneormorestepinstructionsPost:Thisdefinesaseriesofoneormorestepinstructionsthatarerunattheendofthepipelinebuild;markedwithacondition(forexample,always,success,orfailure),usuallyusedtosendnotificationsafterthepipelinebuild(wewillcoverthisindetailintheTriggersandnotificationssection.)
DirectivesDirectivesexpresstheconfigurationofapipelineoritsparts:
Agent:ThisspecifieswheretheexecutiontakesplaceandcandefinethelabeltomatchtheequallylabeledagentsordockertospecifyacontainerthatisdynamicallyprovisionedtoprovideanenvironmentforthepipelineexecutionTriggers:Thisdefinesautomatedwaystotriggerthepipelineandcanusecrontosetthetime-basedschedulingorpollScmtochecktherepositoryforchanges(wewillcoverthisindetailintheTriggersandnotificationssection)Options:Thisspecifiespipeline-specificoptions,forexample,timeout(maximumtimeofpipelinerun)orretry(numberoftimesthepipelineshouldbererunafterfailure)Environment:ThisdefinesasetofkeyvaluesusedasenvironmentvariablesduringthebuildParameters:Thisdefinesalistofuser-inputparametersStage:ThisallowsforlogicalgroupingofstepsWhen:Thisdetermineswhetherthestageshouldbeexecuteddependingonthegivencondition
StepsStepsarethemostfundamentalpartofthepipeline.Theydefinetheoperationsthatareexecuted,sotheyactuallytellJenkinswhattodo.
sh:Thisexecutestheshellcommand;actually,it'spossibletodefinealmostanyoperationusingshcustom:Jenkinsoffersalotofoperationsthatcanbeusedassteps(forexample,echo);manyofthemaresimplywrappersovertheshcommandusedforconvenience;pluginscanalsodefinetheirownoperationsscript:ThisexecutesablockoftheGroovy-basedcodethatcanbeusedforsomenon-trivialscenarios,whereflowcontrolisneeded
Thecompletespecificationoftheavailablestepscanbefoundat:https://jenkins.io/doc/pipeline/steps/.
Noticethatthepipelinesyntaxisverygenericandtechnically,canbeusedforalmostanyautomationprocess.Thisiswhythepipelineshouldbetreatedasamethodofstructurizationandvisualization.Themostcommonusecaseis,however,implementingtheContinuousIntegrationserverthatwewilllookatinthefollowingsection.
CommitpipelineThemostbasicContinuousIntegrationprocessiscalledacommitpipeline.Thisclassicphase,asitsnamesays,startswithacommit(orpushinGit)tothemainrepositoryandresultsinareportaboutthebuildsuccessorfailure.Sinceitrunsaftereachchangeinthecode,thebuildshouldtakenomorethan5minutesandshouldconsumeareasonableamountofresources.ThecommitphaseisalwaysthestartingpointoftheContinuousDeliveryprocess,anditprovidesthemostimportantfeedbackcycleinthedevelopmentprocess,constantinformationifthecodeisinahealthystate.
Thecommitphaseworksasfollows.Adeveloperchecksinthecodetotherepository,theContinuousIntegrationserverdetectsthechange,andthebuildstarts.Themostfundamentalcommitpipelinecontainsthreestages:
Checkout:ThisstagedownloadsthesourcecodefromtherepositoryCompile:ThisstagecompilesthesourcecodeUnittest:Thisstagerunsasuiteofunittests
Let'screateasampleprojectandseehowtoimplementthecommitpipeline.
ThisisanexampleofapipelinefortheprojectthatusestechnologiessuchasGit,Java,Gradle,andSpringBoot.Nevertheless,thesameprinciplesapplytoanyothertechnology.
CheckoutCheckingoutcodefromtherepositoryisalwaysthefirstoperationinanypipeline.Inordertoseethis,weneedtohavearepository.Then,wewillbeabletocreateapipeline.
CreatingaGitHubrepositoryCreatingarepositoryontheGitHubservertakesjustafewsteps:
1. Gotothehttps://github.com/page.2. Createanaccountifyoudon'thaveoneyet.3. ClickonNewrepository.4. Giveitaname,calculator.5. TickInitializethisrepositorywithaREADME.6. ClickonCreaterepository.
Now,youshouldseetheaddressoftherepository,forexample,https://github.com/leszko/calculator.git.
CreatingacheckoutstageWecancreateanewpipelinecalledcalculatorand,asPipelinescript,putthecodewithastagecalledCheckout:
pipeline{
agentany
stages{
stage("Checkout"){
steps{
giturl:'https://github.com/leszko/calculator.git'
}
}
}
}
Thepipelinecanbeexecutedonanyoftheagents,anditsonlystepdoesnothingmorethandownloadingcodefromtherepository.WecanclickonBuildNowandseeifitwasexecutedsuccessfully.
NotethattheGittoolkitneedstobeinstalledonthenodewherethebuildisexecuted.
Whenwehavethecheckout,we'rereadyforthesecondstage.
CompileInordertocompileaproject,weneedto:
1. Createaprojectwiththesourcecode.2. Pushittotherepository.3. AddtheCompilestagetothepipeline.
CreatingaJavaSpringBootprojectLet'screateaverysimpleJavaprojectusingtheSpringBootframeworkbuiltbyGradle.
SpringBootisaJavaframeworkthatsimplifiesbuildingenterpriseapplications.GradleisabuildautomationsystemthatisbasedontheconceptsofApacheMaven.
ThesimplestwaytocreateaSpringBootprojectistoperformthefollowingsteps:
1. Gotothehttp://start.spring.io/page.2. SelectGradleprojectinsteadofMavenproject(youcanalsoleaveMavenif
youpreferittoGradle).3. FillGroupandArtifact(forexample,com.leszkoandcalculator).4. AddWebtoDependencies.5. ClickonGenerateProject.6. Thegeneratedskeletonprojectshouldbedownloaded(thecalculator.zip
file).
Thefollowingscreenshotpresentsthehttp://start.spring.io/page:
PushingcodetoGitHubWewillusetheGittooltoperformthecommitandpushoperations:Inordertorunthegitcommand,youneedtohavetheGittoolkitinstalled(itcanbedownloadedfromhttps://git-scm.com/downloads).
Let'sfirstclonetherepositorytothefilesystem:
$gitclonehttps://github.com/leszko/calculator.git
Extracttheprojectdownloadedfromhttp://start.spring.io/intothedirectorycreatedbyGit.
Ifyouprefer,youcanimporttheprojectintoIntelliJ,Eclipse,oryourfavoriteIDEtool.
Asaresult,thecalculatordirectoryshouldhavethefollowingfiles:$ls-a...build.gradle.git.gitignoregradlegradlewgradlew.batREADME.mdsrc
InordertoperformtheGradleoperationslocally,youneedtohaveJavaJDKinstalled(inUbuntu,youcandoitbyexecutingsudoapt-getinstall-ydefault-jdk).
Wecancompiletheprojectlocallyusingthefollowingcode:
$./gradlewcompileJava
InthecaseofMaven,youcanrun./mvnwcompile.BothGradleandMavencompiletheJavaclasseslocatedinthesrcdirectory.
YoucanfindallpossibleGradleinstructions(fortheJavaproject)athttps://docs.gradle.org/current/userguide/java_plugin.html.
Now,wecancommitandpushtotheGitHubrepository:$gitadd.$gitcommit-m"AddSpringBootskeleton"
$gitpush-uoriginmaster
Afterrunningthegitpushcommand,youwillbepromptedtoentertheGitHubcredentials(usernameandpassword).
ThecodeisalreadyintheGitHubrepository.Ifyouwanttocheckit,youcangototheGitHubpageandseethefiles.
stage("Compile"){<br/>steps{<br/>sh"./gradlewcompileJava"<br/>}<br/>}
NotethatweusedexactlythesamecommandlocallyandintheJenkinspipeline,whichisaverygoodsignbecausethelocaldevelopmentprocessisconsistentwiththeContinuousIntegrationenvironment.Afterrunningthebuild,youshouldseetwogreenboxes.Youcanalsocheckthattheprojectwascompiledcorrectlyintheconsolelog.
UnittestIt'stimetoaddthelaststagethatisUnittest,whichchecksifourcodedoeswhatweexpectittodo.Wehaveto:
AddthesourcecodeforthecalculatorlogicWriteunittestforthecodeAddastagetoexecutetheunittest
CreatingbusinesslogicThefirstversionofthecalculatorwillbeabletoaddtwonumbers.Let'saddthebusinesslogicasaclassinthesrc/main/java/com/leszko/calculator/Calculator.javafile:packagecom.leszko.calculator;importorg.springframework.stereotype.Service;
@ServicepublicclassCalculator{intsum(inta,intb){returna+b;}}
Toexecutethebusinesslogic,wealsoneedtoaddthewebservicecontrollerinaseparatefilesrc/main/java/com/leszko/calculator/CalculatorController.java:
packagecom.leszko.calculator;
importorg.springframework.beans.factory.annotation.Autowired;
importorg.springframework.web.bind.annotation.RequestMapping;
importorg.springframework.web.bind.annotation.RequestParam;
importorg.springframework.web.bind.annotation.RestController;
@RestController
classCalculatorController{
@Autowired
privateCalculatorcalculator;
@RequestMapping("/sum")
Stringsum(@RequestParam("a")Integera,
@RequestParam("b")Integerb){
returnString.valueOf(calculator.sum(a,b));
}
}
Thisclassexposesthebusinesslogicasawebservice.Wecanruntheapplicationandseehowitworks:
$./gradlewbootRun
Itshouldstartourwebserviceandwecancheckthatitworksbynavigatingtothebrowserandopeningthepagehttp://localhost:8080/sum?a=1&b=2.Thisshouldsumtwonumbers(1and2)andshow3inthebrowser.
WritingaunittestWealreadyhavetheworkingapplication.Howcanweensurethatthelogicworksasexpected?Wehavetrieditonce,butinordertoknowconstantly,weneedaunittest.Inourcase,itwillbetrivial,maybeevenunnecessary;however,inrealprojects,unittestscansavefrombugsandsystemfailures.
Let'screateaunittestinthefilesrc/test/java/com/leszko/calculator/CalculatorTest.java:
packagecom.leszko.calculator;
importorg.junit.Test;
importstaticorg.junit.Assert.assertEquals;
publicclassCalculatorTest{
privateCalculatorcalculator=newCalculator();
@Test
publicvoidtestSum(){
assertEquals(5,calculator.sum(2,3));
}
}
Wecanrunthetestlocallyusingthe./gradlewtestcommand.Then,let'scommitthecodeandpushittotherepository:
$gitadd.
$gitcommit-m"Addsumlogic,controllerandunittest"
$gitpush
CreatingaunitteststageNow,wecanaddaUnitteststagetothepipeline:stage("Unittest"){steps{sh"./gradlewtest"}}
InthecaseofMaven,wewouldhavetouse./mvnwtest.
Whenwebuildthepipelineagain,weshouldseethreeboxes,whichmeansthatwe'vecompletedtheContinuousIntegrationpipeline:
JenkinsfileAllthetime,sofar,wecreatedthepipelinecodedirectlyinJenkins.Thisis,however,nottheonlyoption.WecanalsoputthepipelinedefinitioninsideafilecalledJenkinsfileandcommitittotherepositorytogetherwiththesourcecode.Thismethodisevenmoreconsistentbecausethewayyourpipelinelooksisstrictlyrelatedtotheprojectitself.
Forexample,ifyoudon'tneedthecodecompilationbecauseyourprogramminglanguageisinterpreted(andnotcompiled),thenyouwon'thavetheCompilestage.Thetoolsyouusealsodifferdependingontheenvironment.WeusedGradle/Mavenbecausewe'vebuilttheJavaproject;however,inthecaseofaprojectwritteninPython,youcouldusePyBuilder.Itleadstotheideathatthepipelinesshouldbecreatedbythesamepeoplewhowritethecode,developers.Also,thepipelinedefinitionshouldbeputtogetherwiththecode,intherepository.
Thisapproachbringsimmediatebenefits,asfollows:
IncaseofJenkins'failure,thepipelinedefinitionisnotlost(becauseit'sstoredinthecoderepository,notinJenkins)ThehistoryofthepipelinechangesisstoredPipelinechangesgothroughthestandardcodedevelopmentprocess(forexample,theyaresubjectedtocodereviews)Accesstothepipelinechangesisrestrictedexactlyinthesamewayastheaccesstothesourcecode
CreatingJenkinsfileWecancreatetheJenkinsfileandpushittoourGitHubrepository.Itscontentisalmostthesameasthecommitpipelinewewrote.TheonlydifferenceisthatthecheckoutstagebecomesredundantbecauseJenkinshastocheckoutthecode(togetherwithJenkinsfile)firstandthenreadthepipelinestructure(fromJenkinsfile).ThisiswhyJenkinsneedstoknowtherepositoryaddressbeforeitreadsJenkinsfile.
Let'screateafilecalledJenkinsfileintherootdirectoryofourproject:pipeline{agentanystages{stage("Compile"){steps{sh"./gradlewcompileJava"}}stage("Unittest"){steps{sh"./gradlewtest"}}}}
WecannowcommittheaddedfilesandpushtotheGitHubrepository:$gitadd.$gitcommit-m"AddsumJenkinsfile"$gitpush
RunningpipelinefromJenkinsfileWhenJenkinsfileisintherepository,thenallwehavetodoistoopenthepipelineconfigurationandinthePipelinesection:
ChangeDefinitionfromPipelinescripttoPipelinescriptfromSCMSelectGitinSCMPuthttps://github.com/leszko/calculator.gitinRepositoryURL
Aftersaving,thebuildwillalwaysrunfromthecurrentversionofJenkinsfileintotherepository.
Wehavesuccessfullycreatedthefirstcompletecommitpipeline.Itcanbetreatedasaminimumviableproduct,andactually,inmanycases,it'ssufficientastheContinuousIntegrationprocess.Inthenextsections,wewillseewhat
improvementscanbedonetomakethecommitpipelineevenbetter.
CodequalitystagesWecanextendtheclassicthreestepsofContinuousIntegrationwithadditionalsteps.Themostwidelyusedarecodecoverageandstaticanalysis.Let'slookateachofthem.
CodecoverageThinkaboutthefollowingscenario:youhaveawell-configuredContinuousIntegrationprocess;however,nobodyinyourprojectwritesunittests.Itpassesallthebuilds,butitdoesn'tmeanthatthecodeisworkingasexpected.Whattodothen?Howtoensurethatthecodeistested?
Thesolutionistoaddthecodecoveragetoolthatrunsalltestsandverifieswhichpartsofthecodehavebeenexecuted.Then,itcreatesareportshowingnot-testedsections.Moreover,wecanmakethebuildfailwhenthereistoomuchuntestedcode.
Therearealotoftoolsavailabletoperformthetestcoverageanalysis;forJava,themostpopularareJaCoCo,Clover,andCobertura.
Let'suseJaCoCoandshowhowthecoveragecheckworksinpractice.Inordertodothis,weneedtoperformthefollowingsteps:
1. AddJaCoCototheGradleconfiguration.2. Addthecodecoveragestagetothepipeline.3. Optionally,publishJaCoCoreportsinJenkins.
AddingJaCoCotoGradleInordertorunJaCoCofromGradle,weneedtoaddthejacocoplugintothebuild.gradlefilebyaddingthefollowinglineinthepluginsection:
applyplugin:"jacoco"
Next,ifwewouldliketomaketheGradlefailincaseoftoolowcodecoverage,wecanaddthefollowingconfigurationtothebuild.gradlefileaswell:
jacocoTestCoverageVerification{
violationRules{
rule{
limit{
minimum=0.2
}
}
}
}
Thisconfigurationsetstheminimumcodecoverageto20%.Wecanrunitwiththefollowingcommand:
$./gradlewtestjacocoTestCoverageVerification
Thecommandchecksifthecodecoverageisatleast20%.Youcanplaywiththeminimumvaluetoseethelevelatwhichthebuildfails.Wecanalsogenerateatestcoveragereportusingthefollowingcommand:$./gradlewtestjacocoTestReport
Youcanalsohavealookatthefullcoveragereportinthebuild/reports/jacoco/test/html/index.htmlfile:
stage("Codecoverage"){<br/>steps{<br/>sh"./gradlewjacocoTestReport"<br/>sh"./gradlewjacocoTestCoverageVerification"<br/>}<br/>}
Afteraddingthisstage,ifanyonecommitscodethatisnotwell-coveredwithtests,thebuildwillfail.
PublishingthecodecoveragereportWhenthecoverageislowandthepipelinefails,itwouldbeusefultolookatthecodecoveragereportandfindwhatpartsarenotyetcoveredwithtests.WecouldrunGradlelocallyandgeneratethecoveragereport;however,itismoreconvenientifJenkinsshowsthereportforus.
InordertopublishthecodecoveragereportinJenkins,weneedthefollowingstagedefinition:
stage("Codecoverage"){
steps{
sh"./gradlewjacocoTestReport"
publishHTML(target:[
reportDir:'build/reports/jacoco/test/html',
reportFiles:'index.html',
reportName:"JaCoCoReport"
])
sh"./gradlewjacocoTestCoverageVerification"
}
}
ThisstagecopiesthegeneratedJaCoCoreporttotheJenkinsoutput.Whenwerunthebuildagain,weshouldseealinktothecodecoveragereports(inthemenuontheleftside,below"BuildNow").
ToperformthepublishHTMLstep,youneedtohavetheHTMLPublisherplugininstalledinJenkins.Youcanreadmoreaboutthepluginathttps://jenkins.io/doc/pipeline/steps/htmlpublisher/#publishhtml-publish-html-reports.
Wehavecreatedthecodecoveragestage,whichshowsthecodethatisnottestedandthereforevulnerabletobugs.Let'sseewhatelsecanbedoneinordertoimprovethecodequality.
Ifyouneedcodecoveragethatismorestrict,youcanchecktheconceptofmutationtestingandaddthePITframeworkstagetothepipeline.Readmoreathttp://pitest.org/.
StaticcodeanalysisYourcodemayworkperfectlyfine,however,whataboutthequalityofthecodeitself?Howdoweensureitismaintainableandwritteninagoodstyle?
Staticcodeanalysisisanautomaticprocessofcheckingthecodewithoutactuallyexecutingit.Inmostcases,itimpliescheckinganumberofrulesonthesourcecode.Theserulesmayapplytoawiderangeofaspects;forexample,allpublicclassesneedtohaveaJavadoccomment;themaximumlengthofalineis120characters,orifaclassdefinestheequals()method,ithastodefinethehashCode()methodaswell.
ThemostpopulartoolstoperformthestaticanalysisontheJavacodeareCheckstyle,FindBugs,andPMD.Let'slookatanexampleandaddthestaticcodeanalysisstageusingCheckstyle.Wewilldothisinthreesteps:
1. AddtheCheckstyleconfiguration.2. AddtheCheckstylestage.3. Optionally,publishtheCheckstylereportinJenkins.
AddingtheCheckstyleconfigurationInordertoaddtheCheckstyleconfiguration,weneedtodefinetherulesagainstwhichthecodeischecked.Wecandothisbyspecifyingtheconfig/checkstyle/checkstyle.xmlfile:
<?xmlversion="1.0"?>
<!DOCTYPEmodulePUBLIC
"-//PuppyCrawl//DTDCheckConfiguration1.2//EN"
"http://www.puppycrawl.com/dtds/configuration_1_2.dtd">
<modulename="Checker">
<modulename="TreeWalker">
<modulename="JavadocType">
<propertyname="scope"value="public"/>
</module>
</module>
</module>
Theconfigurationcontainsonlyonerule:Checkingifpublicclasses,interfaces,andenumsaredocumentedwithJavadoc.Iftheyarenot,thebuildfails.
ThecompleteCheckstyledescriptioncanbefoundathttp://checkstyle.sourceforge.net/config.html.
Wealsoneedtoaddthecheckstyleplugintothebuild.gradlefile:
applyplugin:'checkstyle'
Then,wecanrunthecheckstylewiththefollowingcode:
$./gradlewcheckstyleMain
Inthecaseofourproject,itshouldresultinafailurebecausenoneofourpublicclasses(Calculator.java,CalculatorApplication.java,CalculatorTest.java,CalculatorApplicationTests.java)hasaJavadoccomment.Weneedtofixitbyaddingthedocumentation,forexample,incaseofthesrc/main/java/com/leszko/calculator/CalculatorApplication.javafile:
/**
*MainSpringApplication.
*/
@SpringBootApplication
publicclassCalculatorApplication{
publicstaticvoidmain(String[]args){
SpringApplication.run(CalculatorApplication.class,args);
}
}
Now,thebuildshouldbesuccessful.
AddingastaticcodeanalysisstageWecanaddaStaticcodeanalysisstagetothepipeline:
stage("Staticcodeanalysis"){
steps{
sh"./gradlewcheckstyleMain"
}
}
Now,ifanyonecommitsafilewithapublicclasswithoutJavadoc,thebuildwillfail.
PublishingstaticcodeanalysisreportsVerysimilartoJaCoCo,wecanaddtheCheckstylereporttoJenkins:
publishHTML(target:[
reportDir:'build/reports/checkstyle/',
reportFiles:'main.html',
reportName:"CheckstyleReport"
])
ItgeneratesalinktotheCheckstylereport.
Wehaveaddedthestaticcodeanalysisstagethatcanhelpinfindingbugsandinstandardizingthecodestyleinsidetheteamororganization.
SonarQubeSonarQubeisthemostwidespreadsourcecodequalitymanagementtool.Itsupportsmultipleprogramminglanguagesandcanbetreatedasanalternativetothecodecoverageandstaticcodeanalysisstepswelookedat.Actually,itisaseparateserverthataggregatesdifferentcodeanalysisframeworks,suchasCheckstyle,FindBugs,andJaCoCo.IthasitsowndashboardsandintegrateswellwithJenkins.
Insteadofaddingcodequalitystepstothepipeline,wecaninstallSonarQube,addpluginsthere,andadda"sonar"stagetothepipeline.TheadvantageofthissolutionisthatSonarQubeprovidesauser-friendlywebinterfacetoconfigurerulesandshowcodevulnerabilities.
YoucanreadmoreaboutSonarQubeonitsofficialpagehttps://www.sonarqube.org/.
TriggersandnotificationsSofar,wehavealwaysbuiltthepipelinemanuallybyclickingontheBuildNowbutton.Itworksbutisnotveryconvenient.Allteammemberswouldhavetorememberthataftercommittingtotherepository,theyneedtoopenJenkinsandstartthebuild.Thesameworkswithpipelinemonitoring;sofar,wemanuallyopenedJenkinsandcheckedthebuildstatus.Inthissection,wewillseehowtoimprovetheprocesssothatthepipelinewouldstartautomaticallyand,whencompleted,notifytheteammembersaboutitsstatus.
TriggersAnautomaticactiontostartthebuildiscalledthepipelinetrigger.InJenkins,therearemanyoptionstochoosefrom;however,theyallboildowntothreetypes:
ExternalPollingSCM(SourceControlManagement)Scheduledbuild
Let'stakealookateachofthem.
ExternalExternaltriggersarenaturaltounderstand.TheymeanthatJenkinsstartsthebuildafterit'scalledbythenotifier,whichcanbetheotherpipelinebuild,theSCMsystem(forexample,GitHub),oranyremotescript.
Thefollowingfigurepresentsthecommunication:
GitHubtriggersJenkinsafterapushtotherepositoryandthebuildisstarted.
Toconfigurethesystemthisway,weneedthefollowingsetupsteps:
1. InstalltheGitHubplugininJenkins.2. GenerateasecretkeyforJenkins.3. SettheGitHubwebhookandspecifytheJenkinsaddressandkey.
InthecaseofthemostpopularSCMproviders,dedicatedJenkinspluginsarealwaysprovided.
ThereisalsoamoregenericwaytotriggerJenkinsviatheRESTcalltotheendpoint<jenkins_url>/job/<job_name>/build?token=<token>.Forsecurityreasons,itrequiressettingtokeninJenkinsandthenusingitintheremotescript.
JenkinsmustbeaccessiblefromtheSCMserver.Inotherwords,ifweusethepublicGitHubtotriggerJenkins,thenourJenkinsservermustbepublicaswell.Thisalsoappliestothegenericsolution;the<jenkins_url>addressmustbeaccessible.
PollingSCMPollingSCMtriggerisalittlelessintuitive.Thefollowingfigurepresentsthecommunication:
JenkinsperiodicallycallsGitHubandchecksiftherewasanypushtotherepository.Then,itstartsthebuild.Itmaysoundcounter-intuitive,however,thereareatleasttwogoodcasesforusingthismethod:
Jenkinsisinsidethefirewallednetwork(whichGitHubdoesnothaveaccessto)Commitsarefrequentandthebuildtakesalongtime,soexecutingabuildaftereverycommitwouldcauseanoverload
TheconfigurationofpollSCMisalsosomehowsimplerbecausethewaytoconnectfromJenkinstoGitHubisalreadysetup(JenkinschecksoutthecodefromGitHub,soitneedstohaveaccess).Inthecaseofourcalculatorproject,wecansetupanautomatictriggerbyaddingthetriggersdeclaration(justafteragent)tothepipeline:
triggers{
pollSCM('*****')
}
Afterrunningthepipelinemanuallyforthefirsttime,theautomatictriggerisset.Then,itchecksGitHubeveryminute,andfornewcommits,itstartsabuild.Totestthatitworksasexpected,youcancommitandpushanythingtotheGitHubrepositoryandseethatthebuildstarts.
Weusedthemysterious*****asanargumenttopollSCM.ItspecifieshowoftenJenkinsshouldcheckfornewsourcechangesandisexpressedinthecron-stylestringformat.
Thecronstringformatisdescribed(togetherwiththecrontool)athttps://en.wikipedi
a.org/wiki/Cron.
ScheduledbuildScheduledtriggermeansthatJenkinsrunsthebuildperiodically,nomatteriftherewasanycommittotherepositoryornot.
Asthefollowingfigurepresents,thereisnocommunicationwithanysystem
needed:
TheimplementationofScheduledbuildisexactlythesameaspollingSCM.TheonlydifferenceisthatthekeywordcronisusedinsteadofpollSCM.Thistriggermethodisrarelyusedforthecommitpipelinebutapplieswelltonightlybuilds(forexample,complexintegrationtestingexecutedatnights).
NotificationsJenkinsprovidesalotofwaystoannounceitsbuildstatus.What'smore,aswitheverythinginJenkins,newnotificationtypescanbeaddedusingplugins.
Let'swalkthroughthemostpopulartypessothatyoucanchoosetheonethatfitsyourneeds.
EmailThemostclassicwaytonotifyabouttheJenkinsbuildstatusistosendemails.Theadvantageofthissolutionisthateverybodyhasamailbox;everybodyknowshowtousethemailbox;andeverybodyisusedtoreceivinginformationbythemailbox.ThedrawbackisthatusuallytherearesimplytoomanyemailsandtheonesfromJenkinsquicklybecomefilteredoutandneverread.
Theconfigurationoftheemailnotificationisverysimple;it'senoughto:
HavetheSMTPserverconfiguredSetitsdetailsinJenkins(inManageJenkins|ConfigureSystem)Usemailtoinstructioninthepipeline
Thepipelineconfigurationcanbeasfollows:post{always{mailto:'[email protected]',subject:"CompletedPipeline:${currentBuild.fullDisplayName}",body:"Yourbuildcompleted,pleasecheck:${env.BUILD_URL}"}}
Notethatallnotificationsareusuallycalledinthepostsectionofthepipeline,whichisexecutedafterallsteps,nomatterwhetherthebuildsucceededorfailed.Weusedthealwayskeyword;however,therearedifferentoptions:
always:Executeregardlessofthecompletionstatuschanged:Executeonlyifthepipelinechangeditsstatusfailure:Executeonlyifthepipelinehasthefailedstatussuccess:Executeonlyifthepipelinehasthesuccessstatusunstable:Executeonlyifthepipelinehastheunstablestatus(usuallycausedbytestfailuresorcodeviolations)
post{<br/>failure{<br/>slackSendchannel:'#dragons-team',<br/>color:'danger',<br/>message:"Thepipeline${currentBuild.fullDisplayName}failed."<br/>}<br/>}
TeamspaceTogetherwiththeagileculturecametheideathatit'sbettertohaveeverythinghappeningintheteamspace.Insteadofwritingemails,meettogether;insteadofonlinemessaging,comeandtalk;insteadoftask-trackingtool,haveawhiteboard.ThesameideacametoContinuousDeliveryandJenkins.Currently,it'sverycommontoinstallbigscreens(alsocalledbuildradiators)intheteamspace.Then,whenyoucometotheoffice,thefirstthingyouseeisthecurrentstatusofthepipeline.Buildradiatorsareconsideredoneofthemosteffectivenotificationstrategies.Theyensurethateveryoneisawareoffailingbuildsand,asaside-effectbenefit,theyboostteamspiritandfavorin-personcommunication.
Sincedevelopersarecreativebeings,theyinventedalotofotherideasthatplaythesameroleastheradiators.Someteamshanglargespeakersthatbeepwhenthepipelinefailed.Someothershavetoysthatblinkwhenthebuildisdone.OneofmyfavoritesisPipelineStateUFO,whichisprovidedasanopensourceprojectonGitHub.Onitspage,youcanfindthedescriptionofhowtoprintandconfigureaUFOthathangsundertheceilingandsignalsthepipelinestate.Youcanfindmoreathttps://github.com/Dynatrace/ufo.
SinceJenkinsisextensiblebyplugins,itscommunitywrotealotofdifferentwaystoinformaboutthebuildstatuses.Amongthem,youcanfindRSSfeeds,SMSnotifications,mobileapplications,desktopnotifiers,andmuchmore.
TeamdevelopmentstrategiesWehavealreadydescribedeverythingabouthowtheContinuousIntegrationpipelineshouldlook.However,whenexactlyshoulditberun?Ofcourse,itistriggeredafterthecommittotherepositorybutafterthecommittowhichbranch?Onlytothetrunkortoeverybranch?Ormaybeitshouldrunbefore,notaftercommittingsothattherepositorywouldalwaysbehealthy?Or,howaboutthecrazyideatohavenobranchesatall?
Thereisnosinglebestanswertothesequestions.Actually,thewayyouusetheContinuousIntegrationprocessdependsonyourteamdevelopmentworkflow.So,beforewegoanyfurther,let'sdescribewhatthepossibleworkflowsare.
DevelopmentworkflowsAdevelopmentworkflowisthewayyourteamputsthecodeintotherepository.Itdepends,ofcourse,onmanyfactorssuchasthesourcecontrolmanagementtool,theprojectspecifics,ortheteamsize.
Asaresult,eachteamdevelopsthecodeinaslightlydifferentmanner.Wecan,however,classifythemintothreetypes:trunk-basedworkflow,branchingworkflow,andforkingworkflow.
Allworkflowsaredescribedindetailwithexamplesathttps://www.atlassian.com/git/tutorials/comparing-workflows.
Trunk-basedworkflowTrunk-basedworkflowisthesimplestpossiblestrategy.Itsoverviewispresentedinthefollowingfigure:
Thereisonecentralrepositorywithasingleentryforallchangestotheproject,whichiscalledthetrunkormaster.Everymemberoftheteamclonesthecentralrepositorytohavetheirownlocalcopies.Thechangesarecommitteddirectlytothecentralrepository.
BranchingworkflowBranchingworkflow,asitsnamesuggests,meansthatthecodeiskeptinmanydifferentbranches.Theideaispresentedinthefollowingfigure:
Whendevelopersstarttoworkonanewfeature,theycreateadedicatedbranchfromthetrunkandcommitallfeature-relatedchangesthere.Thismakesiteasyformultipledeveloperstoworkonafeaturewithoutbreakingthemaincodebase.Thisiswhy,inthecaseofbranchingworkflow,thereisnoprobleminkeepingthemasterhealthy.Whenthefeatureiscompleted,adeveloperrebasesthefeaturebranchfrommasterandcreatesapullrequestthatcontainsallfeature-relatedcodechanges.Itopensthecodereviewdiscussionsandmakesspacetocheckifthechangesdon'tdisturbthemaster.Whenthecodeisacceptedbyotherdevelopersandautomaticsystemchecks,thenitismergedintothemaincodebase.Then,thebuildisrunagainonmasterbutshouldalmostneverfailsinceitdidn'tfailonthebranch.
ForkingworkflowForkingworkflowisverypopularamongtheopensourcecommunity.Itsideaispresentedinthefollowingfigure:
Eachdeveloperhashisownserver-siderepository.Theymayormaynotbetheofficialrepository,buttechnically,eachrepositoryisexactlythesame.
Forkingmeansliterallycreatinganewrepositoryfromtheotherrepository.Developerspushtotheirownrepositoriesandwhentheywanttointegratethecode,theycreateapullrequesttotheotherrepository.
Themainadvantageoftheforkingworkflowisthattheintegrationisnotnecessarilyviaacentralrepository.Italsohelpswiththeownershipbecauseitallowsacceptingpullrequestsfromotherswithoutgivingthemwriteaccess.
Inthecaseofrequirement-orientedcommercialprojects,theteamusuallyworksononeproductandthereforehasacentralrepository,sothismodelboilsdowntothebranchingworkflowwiththegoodownershipassignment,forexample,onlyprojectleadscanmergepullrequestsintothecentralrepository.
AdoptingContinuousIntegrationWedescribeddifferentdevelopmentworkflows,buthowdotheyinfluencetheContinuousIntegrationconfiguration?
BranchingstrategiesEachdevelopmentworkflowimpliesadifferentContinuousIntegrationapproach:
Trunk-basedworkflow:impliesconstantlystrugglingagainstthebrokenpipeline.Ifeveryonecommitstothemaincodebase,thenthepipelineoftenfails.Inthiscase,theoldContinuousIntegrationrulesays,"Ifthebuildisbroken,thenthedevelopmentteamstopswhatevertheyaredoingandfixestheproblemimmediately."Branchingworkflow:solvesthebrokentrunkissuebutintroducesanotherone:ifeveryonedevelopsintheirownbranches,thenwhereistheintegration?Afeatureusuallytakesweeksormonthstodevelop,andforallthistime,thebranchisnotintegratedintothemaincode,thereforeitcannotbereallycalled"continuous"integration;nottomentionthatthereisaconstantneedformergingandresolvingconflicts.Forkingworkflow:impliesmanagingtheContinuousIntegrationprocessbyeveryrepositoryowner,whichisn'tusuallyaproblem.Itshares,however,thesameissuesasthebranchingworkflow.
Thereisnosilverbullet,anddifferentorganizationschoosedifferentstrategies.Thesolutionthatistheclosesttoperfectionisusingthetechniqueofthebranchingworkflowandthephilosophyofthetrunk-basedworkflow.Inotherwords,wecancreateverysmallbranchesandintegratethemfrequentlyintomaster.Thisseemstotakethebestofboth,however,requireseitherhavingtinyfeaturesorusingfeaturetoggles.SincetheconceptoffeaturetogglesfitsverywellintoContinuousIntegrationandContinuousDelivery,let'stakeamomenttoexploreit.
if(feature_toggle){<br/>//dosomething<br/>}
4. Duringthefeaturedevelopment:Codingisdoneinmasterwithfeature_toggle=true(insteadofcodinginthefeaturebranch)Releaseisdonefrommasterwithfeature_toggle=false
5. Whenthefeaturedevelopmentiscompleted,allifstatementsareremovedandfeature_toggleisremovedfromtheconfiguration(insteadofmergingfeaturetomasterandremovingthefeaturebranch).
Thebenefitoffeaturetoggleisthatalldevelopmentisdoneinthetrunk,whichenablestherealContinuousIntegrationandmitigatesproblemswithmergingcode.
JenkinsMultibranchIfyoudecidetousebranchesinanyform,thelongfeaturebranchesortherecommendedshort-livedbranches,thenitisconvenienttoknowthatthecodeishealthybeforemergingitintomaster.Thisapproachresultsinalwayskeepingthemaincodebasegreenand,luckily,thereisaneasywaytodoitwithJenkins.
InordertouseMultibranchinourcalculatorproject,let'sproceedwiththefollowingsteps:
1. OpenthemainJenkinspage.2. ClickonNewItem.3. Entercalculator-branchesastheitemname,selectMultibranchPipeline,and
clickonOK.4. IntheBranchSourcessection,clickonAddsource,andselectGit.5. EntertherepositoryaddressintoProjectRepository.
6. TickPeriodicallyifnototherwiserunandset1minuteasInterval.7. ClickonSave.
Everyminute,thisconfigurationchecksiftherewereanybranchesadded(orremoved)andcreates(ordeletes)thededicatedpipelinedefinedbyJenkinsfile.
Wecancreateanewbranchandseehowitworks.Let'screateanewbranchcalledfeatureandpushitintotherepository:$gitcheckout-bfeature$gitpushoriginfeature
Afteramoment,youshouldseeanewbranchpipelineautomaticallycreatedand
run:
Now,beforemergingthefeaturebranchtomaster,wecancheckifit'sgreen.Thisapproachshouldneverbreakthemasterbuild.
InthecaseofGitHub,thereisanevenbetterapproach,usingtheGitHubOrganizationFolderplugin.Itautomaticallycreatespipelineswithbranchesandpullrequestsforallprojects.
Averysimilarapproachistobuildapipelineperpullrequestinsteadofapipelineperbranch,whichgivesthesameresult;themaincodebaseisalwayshealthy.
Non-technicalrequirementsLastbutnotleast,ContinuousIntegrationisnotallaboutthetechnology.Onthecontrary,technologycomessecond.JamesShoreinhisarticleContinuousIntegrationonaDollaraDaydescribedhowtosetuptheContinuousIntegrationprocesswithoutanyadditionalsoftware.Allheusedwasarubberchickenandabell.Theideaistomaketheteamworkinoneroomandsetupaseparatecomputerwithanemptychair.Puttherubberchickenandthebellinfrontofthatcomputer.Now,whenyouplantocheckinthecode,taketherubberchicken,checkinthecode,gototheemptycomputer,checkoutthefreshcode,runallteststhere,andifeverythingpasses,putbacktherubberchickenandringthebellsothateveryoneknowsthatsomethinghasbeenaddedtotherepository.
ContinuousIntegrationonaDollaraDaybyJamesShorecanbefoundat:http://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html.
Theideaisalittleoversimplified,andautomatedtoolsareuseful;however,themainmessageisthatwithouteachteammember'sengagement,eventhebesttoolswon'thelp.JezHumbleinhisgreatbook,ContinuousDelivery,mentionstheprerequisitesforContinuousIntegrationthatcanberephrasedwiththefollowingpoints:
Checkinregularly:QuotingMikeRoberts,""Continuouslyismoreoftenthanyouthink"",theminimumisonceaday.Createcomprehensiveunittests:It'snotonlyaboutthehightestcoverage,it'spossibletohavenoassertionsandstillkeep100%coverage.Keeptheprocessquick:ContinuousIntegrationmusttakeashorttime,preferablyunder5minutes.10minutesisalreadyalot.Monitorthebuilds:Itcanbeasharedresponsibilityoryoucanadaptthebuildmasterrolethatrotatesweekly.
Exercises
You'velearnedalotabouthowtoconfiguretheContinuousIntegrationprocess.Sincepracticemakesmanperfect,werecommenddoingthefollowingexercises:
1. CreateaPythonprogramthatmultipliestwonumberspassedasthecommand-lineparameters.AddunittestsandpublishtheprojectonGitHub:
Createtwofilescalculator.pyandtest_calculator.pyYoucanusetheunittestlibraryathttps://docs.python.org/library/unittest.htmlRuntheprogramandtheunittest
2. BuildtheContinuousIntegrationpipelineforthePythoncalculatorproject:
UseJenkinsfileforspecifyingthepipelineConfigurethetriggersothatthepipelinerunsautomaticallyincaseofanycommittotherepositoryThepipelinedoesn'tneedtheCompilestepsincePythonisaninterpretablelanguageRunthepipelineandobservetheresultsTrytocommitthecodethatbreakseachstageofthepipelineandobservehowitisvisualizedinJenkins
SummaryInthischapter,wecoveredallaspectsoftheContinuousIntegrationpipeline,whichisalwaysthefirststepforContinuousDelivery.Thekeytakeawayfromthechapter:
Pipelineprovidesageneralmechanismfororganizinganyautomationprocesses;however,themostcommonusecasesareContinuousIntegrationandContinuousDeliveryJenkinsacceptsdifferentwaysofdefiningpipelinesbuttherecommendedoneisthedeclarativesyntaxCommitpipelineisthemostbasicContinuousIntegrationprocessand,asitsnamesuggests,itshouldberunaftereverycommittotherepositoryThepipelinedefinitionshouldbestoredintherepositoryasaJenkinsfileCommitpipelinecanbeextendedwiththecodequalitystagesNomattertheprojectbuildtool,JenkinscommandsshouldalwaysbeconsistentwiththelocaldevelopmentcommandsJenkinsoffersawiderangeoftriggersandnotificationsThedevelopmentworkflowshouldbecarefullychoseninsidetheteamororganizationbecauseitaffectstheContinuousIntegrationprocessanddefinesthewaythecodeisdeveloped
Inthenextchapter,wewillfocusonthenextphaseoftheContinuousDeliveryprocess,automatedacceptancetesting.Itcanbeconsideredasthemostimportantand,inmanycases,themostdifficultsteptoimplement.WewillexploretheideaofacceptancetestingandasampleimplementationusingDocker.
AutomatedAcceptanceTesting
WealreadyconfiguredthecommitphaseoftheContinuousDeliveryprocessandnowit'stimetoaddresstheacceptancetestingphase,whichisusuallythemostchallengingpart.Bygraduallyextendingthepipeline,wewillseedifferentaspectsofawell-doneacceptancetestingautomation.
Thischaptercoversthefollowingpoints:
IntroducingtheacceptancetestingprocessanditsdifficultiesExplainingtheideaoftheartifactrepositoryCreatingtheDockerregistryonDockerHubInstallingandsecuringprivateDockerregistryImplementingacceptancetestingintheJenkinspipelineIntroducingandexploringDockerComposeUsingDockerComposeintheacceptancetestingprocessWritingacceptancetestswithusers
IntroducingacceptancetestingAcceptancetestingisatestperformedtodetermineifthebusinessrequirementsorcontractsaremet.Itinvolvesblack-boxtestingagainstacompletesystemfromauserperspectiveanditspositiveresultshouldimplytheacceptanceofthesoftwaredelivery.Sometimes,alsocalledUAT(useracceptancetesting),endusertesting,orbetatesting,itisaphaseofthedevelopmentprocesswhensoftwaremeetsthereal-worldaudience.
ManyprojectsrelyonmanualstepsperformedbyQAsoruserstoverifythefunctionalandnonfunctionalrequirements,butstill,it'swaymorereasonabletorunthemasprogrammedrepeatableoperations.
Automatedacceptancetests,however,canbeconsidereddifficultduetotheirspecifics:
User-facing:Theyneedtobewrittentogetherwithauser,whichrequiresanunderstandingbetweentwoworlds,technicalandnon-technical.Dependenciesintegration:Thetestedapplicationshouldberuntogetherwithitsdependenciesinordertocheckwhetherthesystemasawholeworksproperly.Environmentidentity:Staging(testing)andproductionenvironmentsshouldbeidenticaltoensurethatwhenruninproduction,theapplicationalsobehavesasexpected.Applicationidentity:Applicationshouldbebuiltonlyonceandthesamebinaryshouldbetransferredtoproduction.Thatguaranteesnochangesincodebetweentestingandreleasingandeliminatestheriskofdifferentbuildingenvironments.Relevanceandconsequences:Ifacceptancetestpasses,itshouldbeclearthattheapplicationisreadyforreleasefromtheuserperspective.
Weaddressallthesedifficultiesindifferentsectionsofthischapter.ApplicationidentitycanbeachievedbybuildingtheDockerimageonlyonceandusingDockerregistryforitsstorageandversioning.DockerComposehelpswiththedependenciesintegrationprovidingawaytobuildagroupofcontainerized
applicationsworkingtogether.Creatingtestsinauser-facingmannerisexplainedintheWritingacceptancetestssection,andtheenvironmentidentityisaddressedbytheDockertoolitselfandcanbealsoimprovedbyothertoolsdescribedinthenextchapter.Concerningtherelevanceandconsequences,theonlygoodansweristokeepinmindthatacceptancetestsmustalwaysbeofahighquality.
Acceptancetestingcanhavemultiplemeanings;inthisbook,wetreatacceptancetestingasacompleteintegrationtestfromauserperspective,excludingnonfunctionaltesting,suchasperformance,load,andrecovery.
DockerregistryDockerregistryisastorageforDockerimages.Tobeprecise,itisastatelessserverapplicationthatallowstheimagestobepublished(pushed)andlaterretrieved(pulled)whenneeded.WehavealreadyseenanexampleoftheregistrywhilerunningtheofficialDockerimages,suchasjenkins.WepulledtheimagesfromDockerHub,whichisanofficialcloud-basedDockerregistry.Havingaseparateservertostore,load,andsearchsoftwarepackagesisamoregeneralconceptcalledthesoftwarerepositoryor,evenmoregeneral,theartifactrepository.Let'slookcloseratthisidea.
ArtifactrepositoryWhilethesourcecontrolmanagementstoresthesourcecode,theartifactrepositoryisdedicatedforstoringsoftwarebinaryartifacts,forexample,compiledlibrariesorcomponents,laterusedtobuildacompleteapplication.Whydoweneedtostorebinariesonaseparateserverusingaseparatetool?
Filesize:Artifactfilescanbelarge,sothesystemsneedtobeoptimizedfortheirdownloadandupload.Versions:Eachuploadedartifactneedstohaveaversionthatmakesiteasytobrowseanduse.Notallversions,however,havetobestoredforever;forexample,iftherewasabugdetected,wemaynotbeinterestedintherelatedartifactandremoveit.Revisionmapping:Eachartifactshouldpointtoexactlyonerevisionofthesourcecontroland,what'smore,thebinarycreationprocessshouldberepeatable.Packages:Artifactsarestoredinthecompiledandcompressedformsothatthesetime-consumingstepsneednotberepeated.Accesscontrol:Userscanberestricteddifferentlytothesourcecodeandartifactbinaryaccess.Clients:Usersoftheartifactrepositorycanbedevelopersoutsidetheteamororganization,whowanttousethelibraryviaitspublicAPI.Usecases:Artifactbinariesareusedtoguaranteethatexactlythesamebuiltversionisdeployedtoeveryenvironmenttoeasetherollbackprocedureincaseoffailure.
ThemostpopularartifactrepositoriesareJFrogArtifactoryandSonatypeNexus.
TheartifactrepositoryplaysaspecialroleintheContinuousDeliveryprocessbecauseitguaranteesthatthesamebinaryisusedthroughoutallpipelinesteps.
Let'slookatthefollowingfigurepresentinghowitworks:
TheDeveloperpushesachangetothesourcecoderepository,whichtriggersthepipelinebuild.AsthelaststepoftheCommitStage,abinaryiscreatedandstoredintheartifactrepository.Afterward,duringallotherstagesofthedeliveryprocess,thesamebinaryispulledandused.
Thebuiltbinaryisoftencalledthereleasecandidateandtheprocessofmovingbinarytothenextstageiscalledpromotion.
Dependingontheprogramminglanguageandtechnologies,thebinaryformatscandiffer.
Forexample,inthecaseofJava,usually,JARfilesarestoredand,inthecaseofRuby,gemfiles.WeworkwithDocker,sowewillstoreDockerimagesasartifacts,andthetooltostoreDockerimagesiscalledDockerregistry.
Someteamsmaintaintworepositoriesatthesametime,artifactrepositoryforJARfilesandDockerregistryforDockerimages.WhileitmaybeusefulduringthefirstphaseoftheDockerintroduction,thereisnogoodreasontomaintainbothforever.
InstallingDockerregistryFirst,weneedtoinstallaDockerregistry.Thereareanumberofoptionsavailable,buttwoofthemaremorecommonthanothers,cloud-basedDockerHubregistryandyourownprivateDockerregistry.Let'sdigintothem.
DockerHubDockerHubisacloud-basedservicethatprovidesDockerregistryandotherfeaturessuchasbuildingimages,testingthem,andpullingcodedirectlyfromthecoderepository.DockerHubiscloud-hosted,soitdoesnotreallyneedanyinstallationprocess.AllyouneedtodoiscreateaDockerHubaccount:
1. Openhttps://hub.docker.com/inabrowser.2. Fillinthepassword,emailaddress,andDockerID.3. Afterreceivinganemailandclickingtheactivationlink,theaccountis
created.
DockerHubisdefinitelythesimplestoptiontostartwith,anditallowsstoringbothprivateandpublicimages.
PrivateDockerregistry
DockerHubmaynotalwaysbeacceptable.Itisnotfreeforenterprisesand,what'sevenmoreimportant,alotofcompanieshavepoliciesnottostoretheirsoftwareoutsidetheirownnetwork.Inthiscase,theonlyoptionistoinstallaprivateDockerregistry.
TheDockerregistryinstallationprocessisquickandsimple,however,makingitsecureandavailableinpublicrequiressettingupaccessrestrictionandthedomaincertificate.Thisiswhywesplitthissectionintothreeparts:
InstallingtheDockerregistryapplicationAddingdomaincertificateAddingaccessrestriction
InstallingtheDockerregistryapplicationDockerregistryisavailableasaDockerimage.Tostartthis,wecanrunthefollowingcommand:
$dockerrun-d-p5000:5000--restart=always--nameregistryregistry:2
Bydefault,theregistrydataisstoredasadockervolumeinthedefaulthostfilesystem'sdirectory.Tochangeit,youcanadd-v<host_directory>:/var/lib/registry.Anotheralternativeistouseavolumecontainer.
Thecommandstartstheregistryandmakesitaccessibleviaport5000.Theregistrycontainerisstartedfromtheregistryimage(version2).The--restart=alwaysoptioncausesthecontainertoautomaticallyrestartwheneverit'sdown.
ConsidersettingupaloadbalancerandstartingafewDockerregistrycontainersincaseofalargenumberofusers.
AddingadomaincertificateIftheregistryisrunonthelocalhost,theneverythingworksfineandnootherinstallationstepsarerequired.However,inmostcases,wewanttohaveadedicatedserverfortheregistry,sothattheimagesarewidelyavailable.Inthatcase,DockerrequiressecuringtheregistrywithSSL/TLS.Theprocessisverysimilartothepublicwebserverconfigurationand,similarly,it'shighlyrecommendedhavingthecertificatesignedbyCA(certificateauthority).IfobtainingtheCA-signedcertificateisnotanoption,thenwecanself-signacertificateorusethe--insecure-registryflag.
Youcanreadaboutcreatingandusingself-signedcertificatesathttps://docs.docker.com/registry/insecure/#using-self-signed-certificates.
HavingthecertificateseithersignedbyCAorself-signed,wecanmovedomain.crtanddomain.keytothecertsdirectoryandstarttheregistry.
$dockerrun-d-p5000:5000--restart=always--nameregistry-v`pwd`/certs:/certs-eREGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt-eREGISTRY_HTTP_TLS_KEY=/certs/domain.keyregistry:2
Incaseofaself-signedcertificate,clientshavetoexplicitlytrustthecertificate.Inordertodothis,theycancopythedomain.crtfileto/etc/docker/certs.d/<docker_host_domain>:5000/ca.crt.
Usingthe--insecure-registryflagisnotrecommendedsinceitprovidesnosecurityatall.
AddinganaccessrestrictionUnlessweusetheregistryinsideawell-securedprivatenetwork,weshouldconfiguretheauthentication.
Thesimplestwaytodothisistocreateauserwithapasswordusingthehtpasswdtoolfromtheregistryimage:
$mkdirauth
$dockerrun--entrypointhtpasswdregistry:2-Bbn<username><password>>auth/passwords
Thecommandrunsthehtpasswdtooltocreatetheauth/passwordsfile(withoneuserinside).Then,wecanruntheregistrywiththatoneuserauthorizedtoaccessit:
$dockerrun-d-p5000:5000--restart=always--nameregistry-v`pwd`/auth:/auth-e"REGISTRY_AUTH=htpasswd"-e"REGISTRY_AUTH_HTPASSWD_REALM=RegistryRealm"-eREGISTRY_AUTH_HTPASSWD_PATH=/auth/passwords-v`pwd`/certs:/certs-eREGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt-eREGISTRY_HTTP_TLS_KEY=/certs/domain.keyregistry:2
Thecommand,inadditiontosettingthecertificates,createstheaccessrestrictionlimitedtotheusersspecifiedintheauth/passwordsfile.
Asaresult,beforeusingtheregistry,aclientneedstospecifytheusernameandpassword.
Accessrestrictiondoesn'tworkinthecaseofthe--insecure-registryflag.
OtherDockerregistries
DockerHubandprivateregistryarenottheonlypossibilitieswhenitcomestoDocker-basedartifactrepositories.
Theotheroptionsareasfollows:
General-purposerepositories:Widely-usedgeneral-purposerepositories,suchasJFrogArtifactoryorSonatypeNexus,implementtheDockerregistryAPI.TheiradvantageisthatoneservercanstorebothDockerimagesandotherartifacts(forexample,JARfiles).Thesesystemsarealsomatureandprovideenterpriseintegration.Cloud-basedregistries:DockerHubisnottheonlycloudprovider.Mostcloud-orientedservicesofferDockerregistriesinthecloud,forexample,GoogleCloudorAWS.Customregistries:TheDockerregistryAPIisopen,soit'spossibletoimplementcustomsolutions.What'smore,imagescanbeexportedtofiles,soit'sfeasibletostoreimagessimplyasfiles.
UsingDockerregistryWhentheregistryisconfigured,wecanshowhowtoworkwithitinthreesteps:
BuildinganimagePushingtheimagetotheregistryPullingtheimagefromtheregistry
FROMubuntu:16.04<br/>RUNapt-getupdate&&\<br/>apt-getinstall-ypython
<strong>$dockerbuild-tubuntu_with_python.</strong>
PushingtheimageInordertopushthecreatedimage,weneedtotagitaccordingtothenamingconvention:
<registry_address>/<image_name>:<tag>
The"registry_address"canbe:
UsernameincaseofDockerHubDomainnameorIPaddresswithportforaprivateregistry(forexample,localhost:5000)
Inmostcases,<tag>isintheformofimage/applicationversion.
Let'stagtheimagetouseDockerHub:
$dockertagubuntu_with_pythonleszko/ubuntu_with_python:1
Wecouldhavealsotaggedtheimageinthebuildcommand:"dockerbuild-tleszko/ubuntu_with_python:1.".
Iftherepositoryhasaccessrestrictionconfigured,weneedtoauthorizeitfirst:
$dockerlogin--username<username>--password<password>
It'spossibletousethedockerlogincommandwithoutparametersandDockerwouldaskinteractivelyfortheusernameandpassword.
Now,wecanstoretheimageintheregistryusingthepushcommand:
$dockerpushleszko/ubuntu_with_python:1
NotethatthereisnoneedtospecifytheregistryaddressbecauseDockerusesthenamingconventiontoresolveit.Theimageisstored,andwecancheckitusing
theDockerHubwebinterfaceavailableathttps://hub.docker.com.
PullingtheimageTodemonstratehowtheregistryworks,wecanremovetheimagelocallyandretrieveitfromtheregistry:$dockerrmiubuntu_with_pythonleszko/ubuntu_with_python:1
Wecanseethattheimagehasbeenremovedusingthedockerimagescommand.Then,let'sretrievetheimagebackfromtheregistry:$dockerpullleszko/ubuntu_with_python:1
IfyouusethefreeDockerHubaccount,youmayneedtochangetheubuntu_with_pythonrepositorytopublicbeforepullingit.
Wecanconfirmtheimageisbackwiththedockerimagescommand.
Whenwehavetheregistryconfiguredandunderstandhowitworks,wecanseehowtouseitinsidetheContinuousDeliverypipelineandbuildtheacceptancetestingstage.
AcceptancetestinpipelineWealreadyunderstoodtheideabehindacceptancetestingandknowhowtoconfigureDockerregistry,sowearereadyforitsfirstimplementationinsidetheJenkinspipeline.
Let'slookatthefigurethatpresentstheprocesswewilluse:
Theprocessgoesasfollows:
1. ThedeveloperpushesacodechangetoGitHub.2. Jenkinsdetectsthechange,triggersthebuild,andchecksoutthecurrent
code.3. JenkinsexecutesthecommitphaseandbuildstheDockerimage.4. JenkinspushestheimagetoDockerregistry.5. JenkinsrunstheDockercontainerinthestagingenvironment.6. StagingtheDockerhostneedstopulltheimagefromtheDockerregistry.7. Jenkinsrunstheacceptancetestsuiteagainsttheapplicationrunninginthe
stagingenvironment.
Forthesakeofsimplicity,wewillruntheDockercontainerlocally
(andnotonaseparatestagingserver).Inordertorunitremotely,weneedtousethe-HoptionortoconfiguretheDOCKER_HOSTenvironmentvariable.Wewillcoverthispartinthenextchapter.
Let'scontinuethepipelinewestartedinthepreviouschapterandaddthreemorestages:
Dockerbuild
Dockerpush
Acceptancetest
KeepinmindthatyouneedtohavetheDockertoolinstalledontheJenkinsexecutor(agentslaveormaster,inthecaseofslave-lessconfiguration)sothatitisabletobuildDockerimages.
IfyouusedynamicallyprovisionedDockerslaves,thenthereisnomatureDockerimageprovidedyet.Youcanbuildityourselforusetheleszko/jenkins-docker-slaveimage.YoualsoneedtomarktheprivilegedoptionintheDockeragentconfiguration.Thissolution,however,hassomedrawbacks,sobeforeusingitinproduction,readthehttp://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/.
TheDockerbuildstageWewouldliketorunthecalculatorprojectasaDockercontainer,soweneedtocreateDockerfileandaddthe"Dockerbuild"stagetoJenkinsfile.
AddingDockerfileLet'screateDockerfileintherootdirectoryofthecalculatorproject:FROMfrolvlad/alpine-oraclejdk8:slimCOPYbuild/libs/calculator-0.0.1-SNAPSHOT.jarapp.jarENTRYPOINT["java","-jar","app.jar"]
ThedefaultbuilddirectoryforGradleisbuild/libs/,andcalculator-0.0.1-SNAPSHOT.jaristhecompleteapplicationpackagedintooneJARfile.NotethatGradleautomaticallyversionedtheapplicationusingtheMaven-styleversion0.0.1-SNAPSHOT.
DockerfileusesabaseimagethatcontainsJDK8(frolvlad/alpine-oraclejdk8:slim).ItalsocopiestheapplicationJAR(createdbyGradle)andrunsit.Let'scheckiftheapplicationbuildsandruns:$./gradlewbuild$dockerbuild-tcalculator.$dockerrun-p8080:8080--namecalculatorcalculator
Usingtheprecedingcommands,we'vebuilttheapplication,builttheDockerimage,andruntheDockercontainer.Afterawhile,weshouldbeabletoopenthebrowsertohttp://localhost:8080/sum?a=1&b=2andsee3asaresult.
WecanstopthecontainerandpushtheDockerfiletotheGitHubrepository:$gitaddDockerfile$gitcommit-m"AddDockerfile"$gitpush
AddingtheDockerbuildtothepipelineThelaststepweneedisaddingthe"Dockerbuild"stagetoJenkinsfile.Usually,theJARpackagingisalsodeclaredasaseparatePackagestage:stage("Package"){steps{sh"./gradlewbuild"}}
stage("Dockerbuild"){steps{sh"dockerbuild-tleszko/calculator."}}
Wedon'texplicitlyversiontheimage,buteachimagehasauniquehashID.Wewillcovertheexplicitversioninginthenextchapter.
NotethatweusedtheDockerregistrynameintheimagetag.Thereisnoneedtohavetheimagetaggedtwicecalculatorandleszko/calculator.
WhenwecommitandpushJenkinsfile,thepipelinebuildshouldstartautomaticallyandweshouldseeallboxesgreen.ThismeansthattheDockerimagehasbeenbuiltsuccessfully.
ThereisalsoaGradlepluginforDockerthatallowsexecutingtheDockeroperationswithinGradlescripts.Youcanseeanexampleat:https://spring.io/guides/gs/spring-boot-docker/.
TheDockerpushstageWhentheimageisready,wecanstoreitintheregistry.TheDockerpushstageisverysimple.It'senoughtoaddthefollowingcodetoJenkinsfile:
stage("Dockerpush"){
steps{
sh"dockerpushleszko/calculator"
}
}
IfDockerregistryhastheaccessrestricted,thenfirstweneedtologinusingthedockerlogincommand.Needlesstosay,thecredentialsmustbewellsecured,forexample,usingadedicatedcredentialstoreasdescribedontheofficialDockerpage:https://docs.docker.com/engine/reference/commandline/login/#credentials-store.
Asalways,pushingchangestotheGitHubrepositorytriggersJenkinstostartthebuildand,afterawhile,weshouldhavetheimageautomaticallystoredintheregistry.
AcceptancetestingstageToperformacceptancetesting,first,weneedtodeploytheapplicationtothestagingenvironmentandthenruntheacceptancetestsuiteagainstit.
stage("Deploytostaging"){<br/>steps{<br/>sh"dockerrun-d--rm-p8765:8080--namecalculatorleszko/calculator"<br/>}<br/>}
Afterrunningthisstage,thecalculatorcontainerisrunningasadaemon,publishingitsportas8765andbeingremovedautomaticallywhenstopped.
AddinganacceptancetesttothepipelineAcceptancetestingusuallyrequiresrunningadedicatedblack-boxtestsuitethatchecksthebehaviorofthesystem.WewillcoveritintheWritingacceptancetestssection.Atthemoment,forthesakeofsimplicity,let'sperformacceptancetestingsimplybycallingthewebserviceendpointwiththecurltoolandcheckingtheresultusingthetestcommand.
Intherootdirectoryoftheproject,let'screatetheacceptance_test.shfile:
#!/bin/bash
test$(curllocalhost:8765/sum?a=1\&b=2)-eq3
Wecallthesumendpointwithparametersa=1andb=2andexpecttoreceive3inresponse.
Then,theAcceptanceteststagecanlookasfollows:
stage("Acceptancetest"){
steps{
sleep60
sh"./acceptance_test.sh"
}
}
Sincethedockerrun-dcommandisasynchronous,weneedtowaitusingthesleepoperationtomakesuretheserviceisalreadyrunning.
Thereisnogoodwaytocheckiftheserviceisalreadyrunning.Analternativetosleepingcouldbeascriptcheckingeverysecondwhethertheservicehasalreadystarted.
post{<br/>always{<br/>sh"dockerstopcalculator"<br/>}<br/>}
ThisstatementmakessurethatthecalculatorcontainerisnolongerrunningontheDockerhost.
DockerComposeLifeiseasywithoutdependencies.Inreal-life,however,almosteveryapplicationlinkstoadatabase,cache,messagingsystem,oranotherapplication.Inthecaseofa(micro)servicearchitecture,eachserviceneedsabunchofotherservicestodoitswork.Themonolithicarchitecturedoesnoteliminatetheissue,anapplicationusuallyhassomedependencies,atleasttothedatabase.
Imagineanewcomerjoiningyourdevelopmentteam;howmuchtimedoesittaketosetuptheentiredevelopmentenvironmentandruntheapplicationwithallitsdependencies?
Whenitcomestoautomatedacceptancetesting,thedependenciesissueisnolongeronlyamatterofconvenience,butitbecomesanecessity.While,duringunittesting,wecouldmockthedependencies,theacceptancetestingsuiterequiresacompleteenvironment.Howdowesetitupquicklyandinarepeatablemanner?Luckily,DockerComposeisatoolthatcanhelp.
WhatisDockerCompose?DockerComposeisatoolfordefining,running,andmanagingmulti-containerDockerapplications.Servicesaredefinedinaconfigurationfile(aYAMLformat)andcanbecreatedandrunalltogetherwithasinglecommand.
DockerComposeorchestratescontainersusingstandardDockermechanismsandprovidesaconvenientwaytospecifytheentireenvironment.
DockerComposecomeswithalotoffeatures,themostinterestingare:
BuildingasetofservicesLaunchingasetofservicestogetherManagingthestateofindividualservicesPreservingvolumedatabetweenrunsScalingservicesupanddownShowinglogsofindividualservicesCachingconfigurationandrecreatingchangedcontainersbetweenruns
AdetaileddescriptionofDockerComposeanditsfeaturescanbefoundontheofficialpageat:https://docs.docker.com/compose/.
WepresenttheDockerComposetoolstartingwiththeinstallationprocess,goingthroughthedocker-compose.ymlconfigurationfileandthedocker-composecommand,toendupwiththebuildingandscalingfeatures.
InstallingDockerComposeThesimplestmethodtoinstallDockerComposeistousethepippackagemanager:
Youcanfindthepiptoolinstallationguideathttps://pip.pypa.io/en/stable/installing/,orforUbuntu,atsudoapt-getinstallpython-pip.
$pipinstalldocker-compose
TocheckthatDockerComposeisinstalled,wecanrun:
$docker-compose--version
Installationguidelinesforalloperatingsystemscanbefoundathttps://docs.docker.com/compose/install/.
Definingdocker-compose.ymlThedocker-compose.ymlfileisusedtodefinetheconfigurationforcontainers,theirrelations,andruntimeproperties.
Inotherwords,whenDockerfilespecifieshowtocreateasingleDockerimage,thendocker-compose.ymlspecifieshowtosetuptheentireenvironmentoutofDockerimages.
Therearethreeversionsofthedocker-compose.ymlfileformat.Inthisbook,weuseversion3,whichisthemostcurrentandrecommended.Readmoreat:https://docs.docker.com/compose/compose-file/compose-versioning/.
Thedocker-compose.ymlfilehasalotoffeaturesandallofthemcanbefoundattheofficialpage:https://docs.docker.com/compose/compose-file/.WewillcoverthemostimportantonesinthecontextoftheContinuousDeliveryprocess.
Let'sstartwithanexampleandimaginethatourcalculatorprojectusestheRedisserverforcaching.Inthiscase,weneedanenvironmentwithtwocontainers,calculatorandredis.Inanewdirectory,let'screatethedocker-compose.ymlfile.
version:"3"
services:
calculator:
image:calculator:latest
ports:
-8080
redis:
image:redis:latest
Theenvironmentconfigurationispresentedinthefollowingfigure:
Let'sseethedefinitionofthetwocontainers:
redis:AcontainerfromthelatestversionoftheredisimagepulledfromtheofficialDockerHub.calculator:Acontainerfromthelatestversionofthecalculatorimagebuiltlocally.Itpublishesthe8080porttotheDockerhost(whichisasubstituteforthe-poptionofthedockercommand).Thecontainerlinkstotherediscontainer,whichmeansthattheysharethesamenetworkandtheredisIPaddressisvisibleundertheredishostnamefrominsidethecalculatorcontainer.
Ifwelikeaservicetobeaddressedbyadifferenthostnamethanitsservicename(forexample,byredis-cacheapartfromredis),thenwecancreatealiasesusingthelinkskeyword.
Usingthedocker-composecommandThedocker-composecommandreadsthedefinitionfileandcreatestheenvironment:
$docker-composeup-d
Thecommandstartedtwocontainers,calculatorandredisinthebackground(-doption).Wecancheckthatthecontainersarerunning:
$docker-composeps
NameCommandStatePorts
---------------------------------------------------------------------------
project_calculator_1java-jarapp.jarUp0.0.0.0:8080->8080/tcp
project_redis_1docker-entrypoint.shredis...Up6379/tcp
Thecontainernamesareprefixedwiththeprojectnameproject,whichistakenfromthenameofthedirectoryinwhichthedocker-compose.ymlfileisplaced.Wecouldspecifytheprojectnamemanuallyusingthe-p<project_name>option.SinceDockerComposeisrunontopofDocker,wecanalsousethedockercommandtoconfirmthatthecontainersarerunning:
$dockerps
CONTAINERIDIMAGECOMMANDPORTS
360518e46bd3calculator:latest"java-jarapp.jar"0.0.0.0:8080->8080/tcp
2268b9f1e14bredis:latest"docker-entrypoint..."6379/tcp
Whenwe'redone,wecanteardowntheenvironment:
$docker-composedown
Theexampleisverysimple,butthetoolitselfisextremelypowerful.Withashortconfigurationandabunchofcommands,wecontroltheorchestrationofallservices.BeforeweuseDockerComposeforacceptancetesting,let'slookattwootherDockerComposefeatures:buildingimagesandscalingcontainers.
BuildingimagesIntheprecedingexample,wehadtofirstbuildthecalculatorimageusingthedockerbuildcommand,andthenitcouldbespecifiedinsidedocker-compose.yml.ThereisalsoanotherapproachtoletDockerComposebuildtheimage.Inthatcase,weneedtospecifythebuildpropertyinsteadofimageintheconfiguration.
Let'sputthedocker-compose.ymlfileinthecalculatorproject'sdirectory.WhenDockerfileandDockerComposeconfigurationsareinthesamedirectory,theformercanlookasfollows:version:"3"services:calculator:build:.ports:-8080redis:image:redis:latest
Thedocker-composebuildcommandbuildstheimage.WecanalsoaskDockerComposetobuildimagesbeforerunningthecontainerswiththeuseofthedocker-compose--buildupcommand.
ScalingservicesDockerComposeprovidesthefunctionalitytoautomaticallycreatemultipleinstancesofthesamecontainer.Wecaneitherspecifythereplicas:<number>parameterinsidedocker-compose.ymlorusethedocker-composescalecommand.
Asanexample,let'sruntheenvironmentagainandreplicatethecalculatorcontainer:
$docker-composeup-d
$docker-composescalecalculator=5
Wecancheckwhichcontainersarerunning:
$docker-composeps
NameCommandStatePorts
---------------------------------------------------------------------------
calculator_calculator_1java-jarapp.jarUp0.0.0.0:32777->8080/tcp
calculator_calculator_2java-jarapp.jarUp0.0.0.0:32778->8080/tcp
calculator_calculator_3java-jarapp.jarUp0.0.0.0:32779->8080/tcp
calculator_calculator_4java-jarapp.jarUp0.0.0.0:32781->8080/tcp
calculator_calculator_5java-jarapp.jarUp0.0.0.0:32780->8080/tcp
calculator_redis_1docker-entrypoint.shredis...Up6379/tcp
Fivecalculatorcontainersareexactlythesame,apartfromthecontainerID,containername,andpublishedportnumbers.
TheyallusethesameinstanceoftheRediscontainer.Wecannowstopandremoveallthecontainers:
$docker-composedown
ScalingcontainersisoneofthemostimpressiveDockerComposefeatures.Withonecommand,wecanscaleupanddownthenumberofcloneinstances.DockerComposetakescareofcleaningupthecontainersthatarenolongerused.
WehaveseenthemostinterestingfunctionalitiesoftheDockerComposetool.
Inthenextsection,wewillfocusonhowtouseitinthecontextofautomatedacceptancetesting.
AcceptancetestingwithDockerComposeDockerComposefitstheacceptancetestingprocessperfectlybecauseitenablessettinguptheentireenvironmentwithonecommand.What'smore,afterthetestingisperformed,theenvironmentcanalsobecleanedupwithonecommand.IfwedecidetouseDockerComposeonproduction,thentheotherbenefitisthattheacceptancetestusesexactlythesameconfiguration,tools,andcommandsasthereleasedapplication.
ToseehowtoapplyDockerComposefortheJenkinsacceptancetestingstage,let'scontinuethecalculatorprojectexampleandaddtheRedis-basedcachingtotheapplication.Then,wewillseetwodifferentapproachestorunacceptancetesting:theJenkins-firstmethodandtheDocker-firstmethod.
Usingamulti-containerenvironmentDockerComposeprovidesdependenciesbetweenthecontainers;inotherwords,itlinksonecontainertoanothercontainer.Technically,thismeansthatcontainerssharethesamenetworkandthatonecontainerisvisiblefromtheother.Tocontinueourexample,weneedtoaddthisdependencyinthecode,andwewilldothisinafewsteps.
AddingaRedisclientlibrarytoGradleInthebuild.gradlefile,addthefollowingconfigurationtothedependenciessection:
compile"org.springframework.data:spring-data-redis:1.8.0.RELEASE"
compile"redis.clients:jedis:2.9.0"
ItaddstheJavalibrariesthattakecareofthecommunicationwithRedis.
packagecom.leszko.calculator;<br/>importorg.springframework.cache.CacheManager;<br/>importorg.springframework.cache.annotation.CachingConfigurerSupport;<br/>importorg.springframework.cache.annotation.EnableCaching;<br/>importorg.springframework.context.annotation.Bean;<br/>importorg.springframework.context.annotation.Configuration;<br/>importorg.springframework.data.redis.cache.RedisCacheManager;<br/>importorg.springframework.data.redis.connection.RedisConnectionFactory;<br/>importorg.springframework.data.redis.connection.jedis.JedisConnectionFactory;<br/>importorg.springframework.data.redis.core.RedisTemplate;<br/><br/>/**Cacheconfig.*/<br/>@Configuration<br/>@EnableCaching<br/>publicclassCacheConfigextendsCachingConfigurerSupport{<br/>privatestaticfinalStringREDIS_ADDRESS="redis";<br/><br/>@Bean<br/>publicJedisConnectionFactoryredisConnectionFactory(){<br/>JedisConnectionFactoryredisConnectionFactory=new<br/>JedisConnectionFactory();<br/>redisConnectionFactory.setHostName(REDIS_ADDRESS);<br/>redisConnectionFactory.setPort(6379);<br/>returnredisConnectionFactory;<br/>}<br/><br/>@Bean<br/>publicRedisTemplate<String,String>redisTemplate(RedisConnectionFactorycf){<br/>RedisTemplate<String,String>redisTemplate=newRedisTemplate<String,<br/>String>();<br/>redisTemplate.setConnectionFactory(cf);<br/>returnredisTemplate;<br/>}<br/><br/>@Bean<br/>publicCacheManagercacheManager(RedisTemplateredisTemplate){<br/>returnnewRedisCacheManager(redisTemplate);<br/>}<br/>}
ThisastandardSpringcacheconfiguration.NotethatfortheRedisserveraddress,weusetheredishostnamethatisautomaticallyavailablethankstotheDockerComposelinkingmechanism.
packagecom.leszko.calculator;<br/>importorg.springframework.cache.annotation.Cacheable;<br/>importorg.springframework.stereotype.Service;<br/><br/>/**Calculatorlogic*/<br/>@Service<br/>publicclassCalculator{<br/>@Cacheable("sum")<br/>publicintsum(inta,intb){<br/>returna+b;<br/>}<br/>}
Fromnowon,thesumcalculationsarecachedinRedis,andwhenwecallthe/sumendpointofthecalculatorwebservice,itwillfirsttrytoretrievetheresultfromthecache.
CheckingthecachingenvironmentAssumingthatwehaveourdocker-compose.ymlinthecalculatorproject'sdirectory,wecannowstartthecontainers:
$./gradlewcleanbuild
$docker-composeup--build-d
Wecanchecktheportonwhichthecalculatorserviceispublished:
$docker-composeportcalculator8080
0.0.0.0:32783
Ifweopenthebrowseronlocalhost:32783/sum?a=1&b=2,thecalculatorserviceshouldreply3and,inthemeantime,accesstheredisserviceandstorethecachedvaluethere.ToseethatthecachevaluewasreallystoredinRedis,wecanaccesstherediscontainerandlookinsidetheRedisdatabase:
$docker-composeexecredisredis-cli
127.0.0.1:6379>keys*
1)"\xac\xed\x00\x05sr\x00/org.springframework.cache.interceptor.SimpleKeyL\nW\x03km\x93\xd8\x02\x00\x02I\x00\bhashCode[\x00\x06paramst\x00\x13[Ljava/lang/Object;xp\x00\x00\x03\xe2ur\x00\x13[Ljava.lang.Object;\x90\xceX\x9f\x10s)l\x02\x00\x00xp\x00\x00\x00\x02sr\x00\x11java.lang.Integer\x12\xe2\xa0\xa4\xf7\x81\x878\x02\x00\x01I\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\x00\x01sq\x00~\x00\x05\x00\x00\x00\x02"
2)"sum~keys"
Thedocker-composeexeccommandexecutedtheredis-cli(theRedisclienttobrowseitsdatabasecontent)commandinsidetherediscontainer.Then,wecanrunkeys*toprinteverythingthatisstoredinRedis.
YoucanplaymorewiththecalculatorapplicationandopenthebrowserwithdifferentvaluestoseethattheRedisservicecontentincreases.Afterthis,it'simportanttoteardowntheenvironmentwiththedocker-composedowncommand.
Inthenextsections,wewillseetwomethodsofacceptancetestsforthemulti-containerproject.Obviously,beforewetakeanyactiononJenkins,weneedtocommitandpushallthechangedfiles(includingdocker-compose.yml)toGitHub.
Notethat,forfurthersteps,DockerComposehastobeinstalledonJenkinsexecutors.
Method1–Jenkins-firstacceptancetestingThefirstmethodistoperformacceptancetestinginthesamewaywedidinthecaseofasinglecontainerapplication.Theonlydifferenceisthatnowwehavetwocontainersrunningaspresentedinthefollowingfigure:
Therediscontainerisnotvisiblefromauserperspective,soasaresult,theonlydifferencebetweensingle-containerandmulti-containeracceptancetestingisthatweusethedocker-composeupcommandinsteadofdockerrun.
OtherDockercommandscanbealsoreplacedwiththeirDockerComposeequivalents:docker-composebuildfordockerbuildanddocker-composepushfordockerpush.Nevertheless,ifwebuildjustoneimage,thenleavingDockercommandsisfineaswell.
ChangingthestagingdeploymentstageLet'schangetheDeploytostagingstagetouseDockerCompose:
stage("Deploytostaging"){
steps{
sh"docker-composeup-d"
}
}
Wemustchangethecleanupinexactlythesameway:
post{
always{
sh"docker-composedown"
}
}
ChangingtheacceptanceteststageForthepurposeofusingdocker-composescale,wedidn'tspecifytheportnumberunderwhichourwebservicewouldbepublished.Ifwedid,thenthescalingprocesswouldfailbecauseallcloneswouldtrytopublishunderthesameportnumber.Onthecontrary,weletDockerchoosetheport.Therefore,weneedtochangetheacceptance_test.shscripttofirstfindwhattheportnumberisandthenruncurlwiththecorrectportnumber.
#!/bin/bash
CALCULATOR_PORT=$(docker-composeportcalculator8080|cut-d:-f2)
test$(curllocalhost:$CALCULATOR_PORT/sum?a=1\&b=2)-eq3
Let'sfigureouthowwefoundtheportnumber:
1. Thedocker-composeportcalculator8080commandchecksunderwhichIPandportaddressthewebserviceispublished(itreturns,forexample,127.0.0.1:57648).
2. cut-d:-f2selectsonlyport(forexample,for127.0.0.1:57648,itreturns57648).
WecanpushthechangetoGitHubandobservetheJenkinsresults.Theideaisstillthesameaswiththesingle-containerapplication,setuptheenvironment,runtheacceptancetestsuite,andteardowntheenvironment.Eventhoughthisacceptancetestingmethodisgoodandworkswell,let'slookatthealternativesolution.
Method2–Docker-firstacceptancetestingIntheDocker-firstapproach,wecreateanadditionaltestcontainerthatperformstestingfrominsidetheDockerhost,aspresentedinthefollowingfigure:
ThisapproachfacilitatestheacceptancetestscriptintermsofretrievingtheportnumberandcanbeeasilyrunwithoutJenkins.It'salsomuchmoreintheDockerstyle.
ThedrawbackisthatweneedtocreateaseparateDockerfileandDockerComposeconfigurationforthepurposeoftesting.
FROMubuntu:trusty<br/>RUNapt-getupdate&&\<br/>apt-getinstall-yqcurl<br/>COPYtest.sh.<br/>CMD["bash","test.sh"]
Itcreatesanimagethatrunstheacceptancetest.
version:"3"<br/>services:<br/>test:<br/>build:./acceptance
Itcreatesanewcontainerthatislinkedtothecontainerbeingtested:calculator.What'smore,internallyit'salways8080sothateliminatestheneedforthetrickypartofportfinding.
#!/bin/bash<br/>sleep60<br/>test$(curlcalculator:8080/sum?a=1\&b=2)-eq3
It'sverysimilartothepreviousacceptancetestscript,theonlydifferenceisthatwecanaddressthecalculatorservicebythecalculatorhostnameandthattheportnumberisalways8080.Also,inthiscase,wewaitinsidethescript,notintheJenkinsfile.
RunningtheacceptancetestWecanrunthetestlocallyusingtheDockerComposecommandfromtherootprojectdirectory:$docker-compose-fdocker-compose.yml-facceptance/docker-compose-acceptance.yml-pacceptanceup-d--build
ThecommandusestwoDockerComposeconfigurationstoruntheacceptanceproject.Oneofthestartedcontainersshouldbecalledacceptance_test_1andbeinterestedinitsresult.Wecancheckitslogswiththefollowingcommand:$dockerlogsacceptance_test_1%Total%Received%XferdAverageSpeedTime1001100100100:00:01
Thelogshowsthatthecurlcommandhasbeensuccessfullycalled.Ifwewanttocheckwhetherthetestsucceededorfailed,wecanchecktheexitcodeofthecontainer:$dockerwaitacceptance_test_10
The0exitcodemeansthatthetestsucceeded.Anycodeotherthan0wouldmeanthatthetestfailed.Afterthetestisdone,weshould,asalways,teardowntheenvironment:$docker-compose-fdocker-compose.yml-facceptance/docker-compose-acceptance.yml-pacceptancedown
ChangingtheacceptanceteststageAsthelaststep,wecanaddtheacceptancetestexecutiontothepipeline.Let'sreplacethelastthreestagesinJenkinsfilewithonenewAcceptanceteststage:
stage("Acceptancetest"){
steps{
sh"docker-compose-fdocker-compose.yml
-facceptance/docker-compose-acceptance.ymlbuildtest"
sh"docker-compose-fdocker-compose.yml
-facceptance/docker-compose-acceptance.yml
-pacceptanceup-d"
sh'test$(dockerwaitacceptance_test_1)-eq0'
}
}
Thistime,wefirstbuildthetestservice.Thereisnoneedtobuildthecalculatorimage;it'salreadydonebythepreviousstages.Intheend,weshouldcleanuptheenvironment:
post{
always{
sh"docker-compose-fdocker-compose.yml
-facceptance/docker-compose-acceptance.yml
-pacceptancedown"
}
}
AfteraddingthistoJenkinsfile,we'redonewiththesecondmethod.WecantestthisbypushingallthechangestoGitHub.
Comparingmethod1andmethod2Tosumup,let'scomparebothsolutions.Thefirstapproachistherealblack-boxtestingfromtheuserperspectiveinwhichJenkinsplaystheroleofauser.Theadvantageisthatit'sveryclosetowhatwillbedoneinproduction;intheend,wewillaccesscontainersviaitsDockerhost.Thesecondapproachteststheapplicationfromtheinsideofanothercontainer.Thesolutionissomehowmoreelegantandcanberunlocallyinasimpleway;however,itrequiresmorefilestocreateanddoesnotcalltheapplicationviaitsDockerhostlikeitwillbelaterdoneinproduction.
Inthenextsection,westepawayfromDockerandJenkinsandtakeacloserlookattheprocessofwritingtheacceptanceteststhemselves.
WritingacceptancetestsSofar,weusedthecurlcommandtoperformasuiteofacceptancetests.Thatisobviouslyaconsiderablesimplification.Technicallyspeaking,ifwewriteaRESTwebservice,thenwecouldwriteallblack-boxtestsasabigscriptwithanumberof"curl"calls.Thissolutionwouldbe,however,verydifficulttoread,understand,andmaintain.What'smore,thescriptwouldbecompletelyincomprehensiblebynon-technical,business-relatedusers.Howtoaddressthisissueandcreatetestswithagoodstructure,readablebyusers,andmeetitsfundamentalgoal:automaticallycheckingifthesystemisasexpected?Iwillanswerthisquestionthroughoutthissection.
Writinguser-facingtestsAcceptancetestsarewrittenwithusersandshouldbecomprehensibletousers.Thisiswhythechoiceofamethodforwritingthemdependsonwhothecustomeris.
Forexample,imagineapurelytechnicalperson.Ifyouwriteawebservicethatoptimizesthedatabasestoring,andyoursystemisusedonlybyothersystemsandreadonlybyotherdevelopers,thenyourtestscanbeexpressedinthesamewayasunittests.Asarule,thetestisgoodifunderstoodbyboth,developeranduser.
Inreallife,mostsoftwareiswrittentodeliveraspecificbusinessvalue,andthatbusinessvalueisdefinedbynon-developers.Therefore,weneedacommonlanguagetocollaborate.Ononeside,thereisthebusinesswhounderstandswhatisneededbutnothowtodoit;ontheotherside,thedevelopmentteamwhoknowshowbutdoesn'tknowwhat.Luckily,thereareanumberofframeworksthathelptoconnectthesetwoworlds,forinstance,Cucumber,FitNesse,JBehave,Capybara,andmanymore.Theydifferfromeachother,andeachofthemmaybeasubjectforaseparatebook;however,thegeneralideaofwritingacceptancetestsisthesameandcanbepresentedinthefollowingfigure:
TheAcceptanceCriteriaarewrittenbyusers(oraproductownerastheirrepresentative)withthehelpofdevelopers.Theyareusuallywrittenintheformofthefollowingscenarios:
GivenIhavetwonumbers:1and2
Whenthecalculatorsumsthem
ThenIreceive3asaresult
Developerswritethetestingimplementationcalledfixturesorstepdefinitionsthatintegratesthehuman-friendlyDSLspecificationwiththeprogramminglanguage.Asaresult,wehaveanautomatedtestthatcanbewell-integratedintotheContinuousDeliverypipeline.
Needlesstoadd,writingacceptancetestsisacontinuousagileprocess,notawaterfallone.Itrequiresconstantcollaborationduringwhichthetestspecificationsareimprovedandmaintainedbyboth,developersandbusiness.
Inthecaseofanapplicationwithauserinterface,itcanbetemptingtoperformtheacceptancetestingdirectlyviatheinterface(forexample,byrecordingSeleniumscripts);however,thisapproachwhennotdoneproperlycanleadtoteststhatareslowandtightlycoupledtotheinterfacelayer.
Let'sseehowwritingacceptancetestslookinpracticeandhowtobindthemtotheContinuousDeliverypipeline.
UsingtheacceptancetestingframeworkLet'susetheCucumberframeworkandcreateanacceptancetestforthecalculatorproject.Aspreviouslydescribed,wewilldothisinthreesteps:
CreatingacceptancecriteriaCreatingstepdefinitionsRunninganautomatedacceptancetest
Feature:Calculator<br/>Scenario:Sumtwonumbers<br/>GivenIhavetwonumbers:1and2<br/>Whenthecalculatorsumsthem<br/>ThenIreceive3asaresult
Thisfileshouldbecreatedbyuserswiththehelpofdevelopers.Notethatitiswritteninawaythatnon-technicalpeoplecanunderstandit.
packageacceptance;<br/><br/>importcucumber.api.java.en.Given;<br/>importcucumber.api.java.en.Then;<br/>importcucumber.api.java.en.When;<br/>importorg.springframework.web.client.RestTemplate;<br/><br/>importstaticorg.junit.Assert.assertEquals;<br/><br/>/**Stepsdefinitionsforcalculator.feature*/<br/>publicclassStepDefinitions{<br/>privateStringserver=System.getProperty("calculator.url");<br/><br/>privateRestTemplaterestTemplate=newRestTemplate();<br/><br/>privateStringa;<br/>privateStringb;<br/>privateStringresult;<br/><br/>@Given("^Ihavetwonumbers:(.*)and(.*)$")<br/>publicvoidi_have_two_numbers(Stringa,Stringb)throwsThrowable{<br/>this.a=a;<br/>this.b=b;<br/>}<br/><br/>@When("^thecalculatorsumsthem$")<br/>publicvoidthe_calculator_sums_them()throwsThrowable{<br/>Stringurl=String.format("%s/sum?a=%s&b=%s",server,a,b);<br/>result=restTemplate.getForObject(url,String.class);<br/>}<br/><br/>@Then("^Ireceive(.*)asaresult$")<br/>publicvoidi_receive_as_a_result(StringexpectedResult)throwsThrowable{<br/>assertEquals(expectedResult,result);<br/>}<br/>}
Eachline(Given,When,andThen)fromthefeaturespecificationfileismatchedbyregularexpressionswiththecorrespondingmethodintheJavacode.Thewildcards(.*)arepassedasparameters.NotethattheserveraddressispassedastheJavapropertycalculator.url.Themethodperformsthefollowingactions:
i_have_two_numbers:Savesparametersasfieldsthe_calculator_sums_them:Callstheremotecalculatorserviceandstorestheresultinafieldi_receive_as_a_result:Assertsthattheresultisasexpected
RunninganautomatedacceptancetestTorunanautomatedtest,weneedtomakeafewconfigurations:
1. AddJavacucumberlibraries:Inthebuild.gradlefile,addthefollowingcodetothedependenciessection:
testCompile("info.cukes:cucumber-java:1.2.4")
testCompile("info.cukes:cucumber-junit:1.2.4")
2. AddGradletarget:Inthesamefile,addthefollowingcode:
taskacceptanceTest(type:Test){
include'**/acceptance/**'
systemPropertiesSystem.getProperties()
}
test{
exclude'**/acceptance/**'
}
Thissplitsthetestsintounit(runwith./gradlewtest)andacceptance(runwith./gradlewacceptanceTest).
3. AddJUnitrunner:Addanewfilesrc/test/java/acceptance/AcceptanceTest.java:
packageacceptance;
importcucumber.api.CucumberOptions;
importcucumber.api.junit.Cucumber;
importorg.junit.runner.RunWith;
/**AcceptanceTest*/
@RunWith(Cucumber.class)
@CucumberOptions(features="classpath:feature")
publicclassAcceptanceTest{}
Thisistheentrypointtotheacceptancetestsuite.
Afterthisconfiguration,iftheserverisrunningonthelocalhost,wecantestitbyexecutingthefollowingcode:
$./gradlewacceptanceTest-Dcalculator.url=http://localhost:8080
Obviously,wecanaddthiscommandtoouracceptance_test.shinsteadofthecurlcommand.ThiswouldmaketheCucumberacceptancetestrunintheJenkinspipeline.
Acceptancetest-drivendevelopmentAcceptancetests,likemostaspectsoftheContinuousDeliveryprocess,arelessabouttechnologyandmoreaboutpeople.Thetestqualitydependson,ofcourse,theengagementofusersanddevelopers,butalso,whatismaybelessintuitive,thetimewhenthetestsarecreated.
Thelastquestiontoaskis,duringwhichphaseofthesoftwaredevelopmentlifecycleshouldtheacceptancetestsbeprepared?Ortorephraseit,shouldwecreateacceptancetestsbeforeorafterwritingthecode?
Technicallyspeaking,theresultisthesame;thecodeiswell-coveredwithboth,unitandacceptancetests.However,it'stemptingtoconsiderwritingtestsfirst.TheideaofTDD(test-drivendevelopment)canbewelladaptedforacceptancetesting.Ifunittestsarewrittenbeforethecode,theresultcodeiscleanerandbetterstructured.Analogously,ifacceptancetestsarewrittenbeforethesystemfeature,theresultingfeaturecorrespondsbettertothecustomer'srequirements.Thisprocess,oftencalledacceptancetest-drivendevelopment,ispresentedinthefollowingfigure:
Users,withdevelopers,writetheacceptancecriteriaspecificationinthehuman-friendlyDSLformat.Developerswritethefixturesandthetestsfail.Then,thefeaturedevelopmentstartsusingtheTDDmethodologyinternally.Afterthe
featureiscompleted,theacceptancetestshouldpass,andthisisasignthatthefeatureiscompleted.
AverygoodpracticeistoattachtheCucumberfeaturespecificationtotherequestticketintheissuetrackingtool(forexample,JIRA)sothatthefeaturewouldbealwaysrequestedtogetherwithitsacceptancetest.Somedevelopmentteamstakeanevenmoreradicalapproachandrefusetostartthedevelopmentprocessifnoacceptancetestsareprepared.Thereisalotofsenseinthat,afterall,howcanyoudevelopsomethingthattheclientcan'ttest?
Exercises
Wecoveredalotofnewmaterialthroughoutthischapter,sotobetterunderstand,werecommenddoingtheexercisesandcreatingyourownprojectwithacceptancetests:
1. CreateaRuby-basedwebservicebook-librarytostorebooks:
TheacceptancecriteriaisdeliveredintheformofthefollowingCucumberfeature:
Scenario:Storebookinthelibrary
Given:Book"TheLordoftheRings"by"J.R.R.Tolkien"withISBNnumber
"0395974682"
When:Istorethebookinlibrary
Then:IamabletoretrievethebookbytheISBNnumber
WritestepdefinitionsfortheCucumbertestWritethewebservice(thesimplestistousetheSinatraframework:http://www.sinatrarb.com/,butyoucanalsouseRubyonRails).Thebookshouldhavethefollowingattributes:name,author,andISBN.Thewebserviceshouldhavethefollowingendpoints:
POST"/books/"toaddabookGET"books/<isbn>"toretrievethebook
Thedatacanbestoredinthememory.Intheend,checkiftheacceptancetestisgreen.
2. Add"book-library"asaDockerimagetotheDockerregistry:CreateanaccountonDockerHub.CreateDockerfilefortheapplication.BuildtheDockerimageandtagitaccordingtothenamingconvention.PushtheimagetoDockerHub.
3. CreatetheJenkinspipelinetobuildDockerimage,pushittotheDockerregistry,andperformacceptancetesting:
Createa"Dockerbuild"stage.CreatetheDockerloginandDockerpushstages.CreateatestcontainerthatperformsacceptancetestinganduseDockerComposetoperformthetest.AddanAcceptanceteststagetothepipeline.Runthepipelineandobservetheresult.
SummaryInthischapter,youlearnedhowtobuildacompleteandfunctionalacceptanceteststage,whichisanessentialpartoftheContinuousDeliveryprocess.Thekeytakeawayfromthechapter:
Acceptancetestscanbedifficulttocreatebecausetheycombinetechnicalchallenges(applicationdependencies,environmentsetup)withpersonalchallenges(developers-businesscollaboration).Acceptancetestingframeworksprovideawaytowritetestsinahuman-friendlylanguagethatmakesthemcomprehensibletonon-technicalpeople.DockerregistryisanartifactrepositoryforDockerimages.DockerregistryfitswellwiththeContinuousDeliveryprocessbecauseitprovidesawaytouseexactlythesameDockerimagethroughoutthestagesandenvironments.DockerComposeorchestratesagroupofDockercontainerinteractingtogether.Itcanalsobuildimagesandscalecontainers.DockerComposecanhelpwithsettingupacompleteenvironmentbeforerunningasuiteofacceptancetests.AcceptancetestscanbewrittenasaDockerimage,andDockerComposecanrunthecompleteenvironmenttogetherwiththetestsandprovidetheresults.
Inthenextchapter,wewillcoverthemissingstagesnecessarytocompletetheContinuousDeliverypipeline.
ConfigurationManagementwithAnsible
WehavealreadycoveredthetwomostcrucialphasesoftheContinuousDeliveryprocess:thecommitphaseandtheautomatedacceptancetesting.Inthischapter,wewillfocusontheconfigurationmanagement,whichconnectsthevirtualcontainerizedenvironmenttotherealserverinfrastructure.
Thischaptercoversthefollowingpoints:
IntroducingtheconceptofconfigurationmanagementExplainingthemostpopularconfigurationmanagementtoolsDiscussingAnsiblerequirementsandtheinstallationprocessUsingAnsiblewithadhoccommandsShowingthepowerofAnsibleautomationwithplaybooksExplainingAnsiblerolesandAnsibleGalaxyImplementingausecaseofthedeploymentprocessUsingAnsibletogetherwithDockerandDockerCompose
IntroducingconfigurationmanagementConfigurationmanagementisaprocessofcontrollingconfigurationchangesinawaythatthesystemmaintainsintegrityovertime.EventhoughthetermdidnotoriginateintheITindustry,currentlyitisbroadlyusedtorefertothesoftwareandthehardware.Inthiscontext,itconcernsthefollowingaspects:
Applicationconfiguration:Thisinvolvessoftwarepropertiesthatdecidehowthesystemworks,whichareusuallyexpressedintheformofflagsorpropertiesfilespassedtotheapplication,forexample,thedatabaseaddress,themaximumchunksizeforfileprocessing,orthelogginglevel.Theycanbeappliedduringdifferentdevelopmentphases:build,package,deploy,orrun.Infrastructureconfiguration:Thisinvolvesserverinfrastructureandenvironmentconfiguration,whichtakescareofthedeploymentprocess.Itdefineswhatdependenciesshouldbeinstalledoneachserverandspecifiesthewayapplicationsareorchestrated(whichapplicationisrunonwhichserverandinhowmanyinstances).
Asanexample,wecanthinkofthecalculatorwebservice,whichusestheRedisserver.Let'slookatthediagrampresentinghowtheconfigurationmanagementtoolworks.
Theconfigurationmanagementtoolreadstheconfigurationfileandpreparestheenvironmentrespectively(installsdependenttoolsandlibraries,deploystheapplicationstomultipleinstances).
Intheprecedingexample,theInfrastructureConfigurationspecifiesthattheCalculatorserviceshouldbedeployedintwoinstancesonServer1andServer2andthattheRedisserviceshouldbeinstalledonServer3.CalculatorApplicationConfigurationspecifiestheportandtheaddressoftheRedisserversothattheservicescancommunicate.
Configurationcandifferdependingonthetypeoftheenvironment(QA,staging,production),forexample,serveraddressescanbedifferent.
Therearemanyapproachestoconfigurationmanagement,butbeforewelookintoconcretesolutions,let'scommentonwhatcharacteristicsagoodconfigurationmanagementtoolshouldhave.
TraitsofgoodconfigurationmanagementWhatshouldthemodernconfigurationmanagementsolutionlooklike?Let'swalkthroughthemostimportantfactors:
Automation:Eachenvironmentshouldbeautomaticallyreproducible,includingtheoperatingsystem,thenetworkconfiguration,thesoftwareinstalled,andtheapplicationsdeployed.Insuchanapproach,fixingproductionissuesmeansnothingmorethananautomaticrebuildoftheenvironment.What'smore,thatsimplifiesserverreplicationsandensuresthatthestagingandproductionenvironmentsareexactlythesame.Versioncontrol:Everychangeintheconfigurationshouldbetracked,sothatweknowwhomadeit,why,andwhen.Usually,thatmeanskeepingtheconfigurationinthesourcecoderepositoryeithertogetherwiththecodeorinaseparateplace.Theformersolutionisrecommendedbecauseconfigurationpropertieshaveadifferentlifecyclethantheapplicationitself.Versioncontrolalsohelpswithfixingproductionissues—theconfigurationcanalwaysberolledbacktothepreviousversionandtheenvironmentautomaticallyrebuilt.Theonlyexceptiontotheversioncontrol-basedsolutionisstoringcredentialsandothersensitiveinformation-theseshouldbenevercheckedin.Incrementalchanges:Applyingachangeintheconfigurationshouldnotrequirerebuildingthewholeenvironment.Onthecontrary,asmallchangeintheconfigurationshouldchangeonlytherelatedpartoftheinfrastructure.Serverprovisioning:Thankstoautomation,addinganewservershouldbeasquickasaddingitsaddresstotheconfiguration(andexecutingonecommand).Security:Theaccesstoboth,theconfigurationmanagementtoolandthemachinesunderitscontrol,shouldbewellsecured.WhenusingtheSSHprotocolforcommunication,theaccesstothekeysorcredentialsneedstobewellprotected.Simplicity:Everymemberoftheteamshouldbeabletoreadthe
configuration,makeachange,andapplyittotheenvironment.Thepropertiesthemselvesshouldalsobekeptassimpleaspossibleandtheonesthatarenotsubjectedtochangearebetteroffkepthardcoded.
Itisimportanttokeepthesepointsinmindwhilecreatingtheconfigurationand,evenbefore,whilechoosingtherightconfigurationmanagementtool.
OverviewofconfigurationmanagementtoolsThemostpopularconfigurationmanagementtoolsareAnsible,Puppet,andChef.Eachofthemisagoodchoice;theyareallopensourceproductswithfreebasicversionsandpaidenterpriseeditions.Themostimportantdifferencesbetweenthemare:
ConfigurationLanguage:ChefusesRuby,PuppetusesitsownDSL(basedonRuby),andAnsibleusesYAML.Agent-based:PuppetandChefuseagentsforcommunication,whichmeansthateachmanagedserverneedstohaveaspecialtoolinstalled.Ansible,onthecontrary,isagentlessandusesthestandardSSHprotocolforcommunication.
Theagentlessfeatureisasignificantadvantagebecauseitimpliesnoneedtoinstallanythingonservers.What'smore,Ansibleisquicklytrendingupwards,whichiswhyitwaschosenforthisbook.Nevertheless,othertoolscanalsobesuccessfullyusedfortheContinuousDeliveryprocess.
InstallingAnsibleAnsibleisanopensource,agentlessautomationengineforsoftwareprovisioning,configurationmanagement,andapplicationdeployment.Itsfirstreleasewasin2012anditsbasicversionisfreeforboth,personalandcommercialuse.Theenterpriseversion,calledAnsibleTower,providesGUImanagementanddashboards,RESTAPI,role-basedaccesscontrol,andsomemorefeatures.
WepresenttheinstallationprocessandadescriptionofhowitcanbeusedseparatelyaswellastogetherwithDocker.
AnsibleserverrequirementsAnsibleusestheSSHprotocolforcommunicationandhasnospecialrequirementsregardingthemachineitmanages.Thereisalsonocentralmasterserver,soit'senoughtoinstalltheAnsibleclienttoolanywhereandwecanalreadyuseittomanagethewholeinfrastructure.
TheonlyrequirementforthemachinesbeingmanagedistohavethePythontooland,obviously,theSSHserverinstalled.Thesetoolsare,however,almostalwaysavailablebydefaultonanyserver.
AnsibleinstallationTheinstallationinstructionsdifferdependingontheoperatingsystem.InthecaseofUbuntu,it'senoughtorunthefollowingcommands:
$sudoapt-getinstallsoftware-properties-common
$sudoapt-add-repositoryppa:ansible/ansible
$sudoapt-getupdate
$sudoapt-getinstallansible
YoucanfindtheinstallationguidesforalloperatingsystemsontheofficialAnsiblepageat:http://docs.ansible.com/ansible/intro_installation.html.
Aftertheinstallationprocessiscompleted,wecanexecutetheAnsiblecommandtocheckthateverythingwasinstalledsuccessfully.
$ansible--version
ansible2.3.2.0
configfile=/etc/ansible/ansible.cfg
configuredmodulesearchpath=Defaultw/ooverrides
<strong>$dockerrunwilliamyeh/ansible:ubuntu14.04</strong><br/><strong>ansible-playbook2.3.2.0</strong><br/><strong>configfile=/etc/ansible/ansible.cfg</strong><br/><strong>configuredmodulesearchpath=Defaultw/ooverrides</strong>
TheAnsibleDockerimageisnolongerofficiallysupported,sotheonlysolutionistousethecommunity-drivenversion.YoucanreadmoreonitsusageontheDockerHubpage.
UsingAnsibleInordertouseAnsible,firstweneedtodefinetheinventory,whichrepresentstheavailableresources.Then,wewillbeabletoeitherexecuteasinglecommandordefineasetoftasksusingtheAnsibleplaybook.
CreatinginventoryAninventoryisalistofalltheserversthataremanagedbyAnsible.EachserverrequiresnothingmorethanthePythoninterpreterandtheSSHserverinstalled.Bydefault,AnsibleassumesthattheSSHkeysareusedforauthentication;however,itisalsopossibletousetheusernameandthepasswordbyaddingthe--ask-passoptiontotheAnsiblecommands.
SSHkeyscanbegeneratedwiththessh-keygentoolandareusuallystoredinthe~/.sshdirectory.
Theinventoryisdefinedinthe/etc/ansible/hostsfileandithasthefollowingstructure:
[group_name]
<server1_address>
<server2_address>
...
Theinventorysyntaxalsoacceptsrangesofservers,forexample,www[01-22].company.com.TheSSHportshouldalsobespecifiedifit'sanythingotherthan22(thedefaultone).YoucanreadmoreontheofficialAnsiblepageat:http://docs.ansible.com/ansible/intro_inventory.html.
Theremaybe0ormanygroupsintheinventoryfile.Asanexample,let'sdefinetwomachinesinonegroupofservers.
[webservers]
192.168.0.241
192.168.0.242
Wecanalsocreatetheconfigurationwithserveraliasesandspecifytheremoteuser:
[webservers]
web1ansible_host=192.168.0.241ansible_user=admin
web2ansible_host=192.168.0.242ansible_user=admin
Theprecedingfiledefinesagroupcalledwebservers,whichconsistsoftwoservers.TheAnsibleclientwillloginastheuseradmintobothofthem.Whenwe
havetheinventorycreated,let'sdiscoverhowwecanuseittoexecutethesamecommandonmanyservers.
Ansibleoffersapossibilitytodynamicallypulltheinventoryfromthecloudprovider(forexample,AmazonEC2/Eucalyptus),LDAP,orCobbler.Readmoreaboutdynamicinventoriesat:http://docs.ansible.com/ansible/intro_dynamic_inventory.html.
AdhoccommandsThesimplestcommandwecanrunisapingonallservers.
$ansibleall-mping
web1|SUCCESS=>{
"changed":false,
"ping":"pong"
}
web2|SUCCESS=>{
"changed":false,
"ping":"pong"
}
Weusedthe-m<module_name>option,whichallowsspecifyingthemodulethatshouldbeexecutedontheremotehosts.Theresultissuccessful,whichmeansthattheserversarereachableandtheauthenticationisconfiguredcorrectly.
AfulllistofmodulesavailableinAnsiblecanbefoundonthepage:http://docs.ansible.com/ansible/modules.html.
Notethatweusedall,sothatallserverswouldbeaddressed,butwecouldalsocallthembythegroupnamewebserversorbythesinglehostalias.Asasecondexample,let'sexecuteashellcommandonlyononeoftheservers.
$ansibleweb1-a"/bin/echohello"
web1|SUCCESS|rc=0>>
hello
The-a<arguments>optionspecifiestheargumentsthatarepassedtotheAnsiblemodule.Inthiscase,wedidn'tspecifythemodule,sotheargumentsareexecutedasashellUnixcommand.Theresultwassuccessfulandhellowasprinted.
Iftheansiblecommandisconnectingtotheserverforthefirsttime(ortheserverisreinstalled),thenwearepromptedwiththekeyconfirmationmessage(SSHmessagewhenthehostisnotpresentinknown_hosts).Sinceitmayinterruptanautomatedscript,wecandisablethepromptmessagebyuncommentinghost_key_checking=Falseinthe/etc/ansible/ansible.cfgfileorbysettingtheenvironment
variableANSIBLE_HOST_KEY_CHECKING=False.
Initssimplisticform,theAnsibleadhoccommandsyntaxlooksasfollows:
ansible<target>-m<module_name>-a<module_arguments>
Thepurposeofadhoccommandsistodosomethingquicklywhenitisnotnecessarytorepeatit.Forexample,wemaywanttocheckifaserverisaliveortopoweroffallthemachinesfortheChristmasbreak.Thismechanismcanbeseenasacommandexecutiononagroupofmachineswiththeadditionalsyntaxsimplificationprovidedbythemodules.TherealpowerofAnsibleautomation,however,liesinplaybooks.
PlaybooksAnAnsibleplaybookisaconfigurationfile,whichdescribeshowserversshouldbeconfigured.Itprovidesawaytodefineasequenceoftasksthatshouldbeperformedoneachofthemachines.AplaybookisexpressedintheYAMLconfigurationlanguage,whichmakesithuman-readableandeasytounderstand.Let'sstartwithasampleplaybookandthenseehowwecanuseit.
DefiningaplaybookAplaybookiscomposedofoneormanyplays.Eachplaycontainsahostgroupname,taskstoperform,andconfigurationdetails(forexample,remoteusernameoraccessrights).Anexampleplaybookmightlooklikethis:----hosts:web1become:yesbecome_method:sudotasks:-name:ensureapacheisatthelatestversionapt:name=apache2state=latest-name:ensureapacheisrunningservice:name=apache2state=startedenabled=yes
Thisconfigurationcontainsoneplaywhich:
Isexecutedonlyonthehostweb1GainsrootaccessusingthesudocommandExecutestwotasks:
Installingthelatestversionofapache2:TheAnsiblemoduleapt(calledwithtwoparametersname=apache2andstate=latest)checkswhethertheapache2packageisinstalledontheserver,andifnot,thenitusestheapt-gettooltoinstallapache2Runningtheapache2service:TheAnsiblemoduleservice(calledwiththreeparametersname=apache2,state=started,andenabled=yes)checkswhethertheUnixserviceapache2isstarted,andifnot,itusestheservicecommandtostartit
Whileaddressingthehosts,youcanalsousepatterns,forexample,wecoulduseweb*toaddressbothweb1andweb2.YoucanreadmoreaboutAnsiblepatternsat:http://docs.ansible.com/ansible/intro_patterns.html.
Notethateachtaskhasahuman-readablename,whichisusedintheconsoleoutputsuchthataptandserviceareAnsiblemodulesandname=apache2,state=latest,andstate=startedaremodulearguments.WehavealreadyseenAnsiblemodules
andargumentswhileusingadhoccommands.Intheprecedingplaybook,wedefinedonlyoneplay,buttherecanbemanyofthemandeachcanberelatedtodifferentgroupsofhosts.
Forexample,wecoulddefinetwogroupsofserversintheinventory:databaseandwebservers.Then,intheplaybook,wecouldspecifytasksthatshouldbeexecutedonalldatabase-hostingmachinesandsomedifferenttasksthatshouldbeexecutedonallthewebservers.Byusingonecommand,wecouldsetupthewholeenvironment.
ExecutingtheplaybookWhenplaybook.ymlisdefined,wecanexecuteitusingtheansible-playbookcommand.
$ansible-playbookplaybook.yml
PLAY[web1]***************************************************************
TASK[setup]**************************************************************
ok:[web1]
TASK[ensureapacheisatthelatestversion]*****************************
changed:[web1]
TASK[ensureapacheisrunning]*******************************************
ok:[web1]
PLAYRECAP****************************************************************
web1:ok=3changed=1unreachable=0failed=0
Iftheserverrequiresenteringthepasswordforthesudocommand,thenweneedtoaddthe--ask-sudo-passoptiontotheansible-playbookcommand.It'salsopossibletopassthesudopassword(ifrequired)bysettingtheextravariable-eansible_become_pass=<sudo_password>.
Theplaybookconfigurationwasexecuted,and,therefore,theapache2toolwasinstalledandstarted.Notethatifthetaskchangedsomethingontheserver,itismarkedaschanged.Onthecontrary,iftherewasnochange,it'smarkedasok.
Itispossibletoruntasksinparallelusingthe-f<num_of_threads>option.
Playbook'sidempotencyWecanexecutethecommandagain.
$ansible-playbookplaybook.yml
PLAY[web1]***************************************************************
TASK[setup]**************************************************************
ok:[web1]
TASK[ensureapacheisatthelatestversion]*****************************
ok:[web1]
TASK[ensureapacheisrunning]*******************************************
ok:[web1]
PLAYRECAP****************************************************************
web1:ok=3changed=0unreachable=0failed=0
Notethattheoutputisslightlydifferent.Thistimethecommanddidn'tchangeanythingontheserver.ThatisbecauseeachAnsiblemoduleisdesignedtobeidempotent.Inotherwords,executingthesamemodulemanytimesinasequenceshouldhavethesameeffectasexecutingitonlyonce.
Thesimplestwaytoachieveidempotencyistoalwaysfirstcheckifthetaskhasn'tbeenexecutedyet,andexecuteitonlyifithasn't.IdempotencyisapowerfulfeatureandweshouldalwayswriteourAnsibletasksthisway.
Ifalltasksareidempotent,thenwecanexecutethemasmanytimesaswewant.Inthatcontext,wecanthinkoftheplaybookasadescriptionofthedesiredstateofremotemachines.Then,theansible-playbookcommandtakescareofbringingthemachine(orgroupofmachines)intothatstate.
HandlersSomeoperationsshouldbeexecutedonlyifsomeothertaskchanged.Forexample,imaginethatyoucopytheconfigurationfiletotheremotemachineandtheApacheservershouldberestartedonlyiftheconfigurationfilehaschanged.Howtoapproachsuchacase?
Forexample,imaginethatyoucopytheconfigurationfiletotheremotemachineandtheApacheservershouldberestartedonlyiftheconfigurationfilehaschanged.Howtoapproachsuchacase?
Ansibleprovidesanevent-orientedmechanismtonotifyaboutthechanges.Inordertouseit,weneedtoknowtwokeywords:
handlers:Thisspecifiesthetasksexecutedwhennotifiednotify:Thisspecifiesthehandlersthatshouldbeexecuted
Let'slookatanexampleofhowwecouldcopytheconfigurationtotheserverandrestartApacheonlyiftheconfigurationhaschanged.
tasks:
-name:copyconfiguration
copy:
src:foo.conf
dest:/etc/foo.conf
notify:
-restartapache
handlers:
-name:restartapache
service:
name:apache2
state:restarted
Now,wecancreatethefoo.conffileandruntheansible-playbookcommand.
$touchfoo.conf
$ansible-playbookplaybook.yml
...
TASK[copyconfiguration]************************************************
changed:[web1]
RUNNINGHANDLER[restartapache]*****************************************
changed:[web1]
PLAYRECAP***************************************************************
web1:ok=5changed=2unreachable=0failed=0
Handlersareexecutedalwaysattheendoftheplayandonlyonce,eveniftriggeredbymultipletasks.
AnsiblecopiedthefileandrestartedtheApacheserver.It'simportanttounderstandthatifwerunthecommandagain,nothingwillhappen.However,ifwechangethecontentofthefoo.conffileandthenruntheansible-playbookcommand,thefilewillbecopiedagain(andtheApacheserverwillberestarted).
$echo"something">foo.conf
$ansible-playbookplaybook.yml
...
TASK[copyconfiguration]*************************************************
changed:[web1]
RUNNINGHANDLER[restartapache]******************************************
changed:[web1]
PLAYRECAP****************************************************************
web1:ok=5changed=2unreachable=0failed=0
Weusedthecopymodule,whichissmartenoughtodetectifthefilehaschanged,andtheninsuchacase,makeachangeontheserver.
Thereisalsoapublish-subscribemechanisminAnsible.Usingitmeansassigningatopictomanyhandlers.Then,atasknotifiesthetopictoexecuteallrelatedhandlers.Youcanreadmoreaboutitat:http://docs.ansible.com/ansible/playbooks_intro.html.
VariablesWhiletheAnsibleautomationmakesthingsidenticalandrepeatableformultiplehosts,itisinevitablethatserversmayrequiresomedifferences.Forexample,thinkoftheapplicationportnumber.Itcanbedifferentdependingonthemachine.Luckily,Ansibleprovidesvariables,whichisagoodmechanismtodealwithserverdifferences.Let'screateanewplaybookanddefineavariable.
Forexample,thinkoftheapplicationportnumber.Itcanbedifferentdependingonthemachine.Luckily,Ansibleprovidesvariables,whichisagoodmechanismtodealwithserverdifferences.Let'screateanewplaybookanddefineavariable.
---
-hosts:web1
vars:
http_port:8080
Theconfigurationdefinesthehttp_portvariablewiththevalue8080.Now,wecanuseitusingtheJinja2syntax.
tasks:
-name:printportnumber
debug:
msg:"Portnumber:{{http_port}}"
TheJinja2languageallowsdoingwaymorethanjustgettingavariable.Wecanuseittocreateconditions,loops,andmanymore.YoucanfindmoredetailsontheJinjapageat:http://jinja.pocoo.org/.
Thedebugmoduleprintsthemessagewhileexecuting.Ifweruntheansible-playbookcommand,wecanseethevariableusage.
$ansible-playbookplaybook.yml
...
TASK[printportnumber]**************************************************
ok:[web1]=>{
"msg":"Portnumber:8080"
}
Variablescanalsobedefinedintheinventoryfilingusingthe
[group_name:vars]section.Youcanreadmoreaboutitat:http://docs.ansible.com/ansible/intro_inventory.html#host-variables.
Apartfromuser-definedvariables,therearealsopredefinedautomaticvariables.Forexample,thehostvarsvariablestoresamapwiththeinformationregardingallhostsfromtheinventory.UsingtheJinja2syntax,wecoulditerateandprinttheIPaddressesofallhostsintheinventory.
---
-hosts:web1
tasks:
-name:printIPaddress
debug:
msg:"{%forhostingroups['all']%}{{
hostvars[host]['ansible_host']}}{%endfor%}"
Then,wecanexecutetheansible-playbookcommand.
$ansible-playbookplaybook.yml
...
TASK[printIPaddress]**************************************************
ok:[web1]=>{
"msg":"192.168.0.241192.168.0.242"
}
NotethatwiththeuseoftheJinja2language,wecanspecifytheflowcontroloperationsinsidetheAnsibleplaybookfile.
AnalternativetotheJinja2templatinglanguage,fortheconditionalsandloops,istousetheAnsiblebuilt-inkeywords:whenandwith_items.Youcanreadmoreaboutitat:http://docs.ansible.com/ansible/playbooks_conditionals.html.
RolesWecaninstallanytoolontheremoteserverusingAnsibleplaybooks.ImaginewewouldliketohaveaserverwithMySQL.Wecouldeasilyprepareaplaybooksimilartotheonewiththeapache2package.However,ifyouthinkaboutit,aserverwithMySQLisquiteacommoncaseand,forsure,someonehasalreadypreparedaplaybookforit,somaybewecouldjustreuseit?HerecomesAnsiblerolesandAnsibleGalaxy.
UnderstandingrolesTheAnsibleroleisawell-structuredplaybookpartpreparedtobeincludedintheplaybooks.Rolesareseparateunitsthatalwayshavethefollowingdirectorystructure:
templates/
tasks/
handlers/
vars/
defaults/
meta/
YoucanreadmoreaboutrolesandwhateachdirectorymeansontheofficialAnsiblepageat:http://docs.ansible.com/ansible/playbooks_roles.html.
Ineachofthedirectories,wecandefinethemain.ymlfile,whichcontainstheplaybookpartsthatcanbeincludedintheplaybook.ymlfile.ContinuingtheMySQLcase,thereisaroledefinedonGitHub:https://github.com/geerlingguy/ansible-role-mysql.Thisrepositorycontainskind-oftasktemplatesthatcanbeusedinourplaybook.Let'slookatapartofthetasks/main.ymlfile,whichinstallsthemysqlpackage.
...
-name:EnsureMySQLPythonlibrariesareinstalled.
apt:"name=python-mysqldbstate=installed"
-name:EnsureMySQLpackagesareinstalled.
apt:"name={{item}}state=installed"
with_items:"{{mysql_packages}}"
register:deb_mysql_install_packages
...
Thisisonlyoneofthetasksdefinedinthetasks/main.ymlfile.OthersareresponsiblefortheMySQLconfiguration.
Thewith_itemskeywordisusedtocreatealoopoveralltheitems.Thewhenkeywordmeansthatthetaskisexecutedonlyunderacertaincondition.
Ifweusethisrole,theninordertoinstalltheMySQLontheserver,it'senoughtocreatethefollowingplaybook.yml:
---
-hosts:all
become:yes
become_method:sudo
roles:
-role:geerlingguy.mysql
become:yes
SuchconfigurationinstallstheMySQLdatabasetoallserversusingthegeerlingguy.mysqlrole.
AnsibleGalaxyAnsibleGalaxyistoAnsiblewhatDockerHubisforDocker-itstorescommonroles,sothattheycanbereusedbyothers.YoucanbrowsetheavailablerolesontheAnsibleGalaxypageat:https://galaxy.ansible.com/.
ToinstalltherolefromAnsibleGalaxy,wecanusetheansible-galaxycommand.
$ansible-galaxyinstallusername.role_name
Thiscommandautomaticallydownloadstherole.InthecaseoftheMySQLexample,wecoulddownloadtherolebyexecuting:$ansible-galaxyinstallgeerlingguy.mysql
Thecommanddownloadsthemysqlrole,whichcanbelaterusedintheplaybookfile.
Ifyouneedtoinstallalotofrolesatthesametime,youcandefinethemintherequirements.ymlfileanduseansible-galaxyinstall-rrequirements.yml.ReadmoreaboutthatapproachandaboutAnsibleGalaxyat:http://docs.ansible.com/ansible/galaxy.html.
DeploymentwithAnsibleWehavecoveredthemostfundamentalfeaturesofAnsible.Let'snowforget,justforalittlewhile,aboutDockerandconfigureacompletedeploymentstepusingAnsible.WewillrunthecalculatorserviceononeserverandtheRedisserviceonthesecondserver.
InstallingRedisWecanspecifyaplayinthenewplaybook.Let'screatetheplaybook.ymlfilewiththefollowingcontent:
---
-hosts:web1
become:yes
become_method:sudo
tasks:
-name:installRedis
apt:
name:redis-server
state:present
-name:startRedis
service:
name:redis-server
state:started
-name:copyRedisconfiguration
copy:
src:redis.conf
dest:/etc/redis/redis.conf
notify:restartRedis
handlers:
-name:restartRedis
service:
name:redis-server
state:restarted
Theconfigurationisexecutedononeserverweb1.Itinstallstheredis-serverpackage,copiestheRedisconfiguration,andstartsRedis.Notethateachtimewechangethecontentoftheredis.conffileandre-runtheansible-playbookcommand,theconfigurationisupdatedontheserverandtheRedisserviceisrestarted.
Wealsoneedtocreatetheredis.conffilewiththefollowingcontent:
daemonizeyes
pidfile/var/run/redis/redis-server.pid
port6379
bind0.0.0.0
ThisconfigurationrunsRedisasadaemonandexposesittoallnetworkinterfacesunderportnumber6379.Let'snowdefinethesecondplay,whichsetsupthecalculatorservice.
DeployingawebserviceWepreparethecalculatorwebserviceinthreesteps:
1. Configuretheprojecttobeexecutable.2. ChangetheRedishostaddress.3. Addcalculatordeploymenttotheplaybook.
bootRepackage{<br/>executable=true<br/>}
ChangingtheRedishostaddressPreviously,we'vehardcodedtheRedishostaddressasredis,sonowweshouldchangeitinthesrc/main/java/com/leszko/calculator/CacheConfig.javafileto192.168.0.241.
Inreal-lifeprojects,theapplicationpropertiesareusuallykeptinthepropertiesfile.Forexample,fortheSpringBootframework,it'safilecalledapplication.propertiesorapplication.yml.
-hosts:web2<br/>become:yes<br/>become_method:sudo<br/>tasks:<br/>-name:ensureJavaRuntimeEnvironmentisinstalled<br/>apt:<br/>name:default-jre<br/>state:present<br/>-name:createdirectoryforCalculator<br/>file:<br/>path:/var/calculator<br/>state:directory<br/>-name:configureCalculatorasaservice<br/>file:<br/>path:/etc/init.d/calculator<br/>state:link<br/>force:yes<br/>src:/var/calculator/calculator.jar<br/>-name:copyCalculator<br/>copy:<br/>src:build/libs/calculator-0.0.1-SNAPSHOT.jar<br/>dest:/var/calculator/calculator.jar<br/>mode:a+x<br/>notify:<br/>-restartCalculator<br/>handlers:<br/>-name:restartCalculator<br/>service:<br/>name:calculator<br/>enabled:yes<br/>state:restarted
Let'swalkthroughthestepswedefined:
Preparetheenvironment:ThistaskensuresthattheJavaRuntimeEnvironmentisinstalled.Basically,itpreparestheserverenvironment,sothatthecalculatorapplicationwouldhaveallthenecessarydependencies.Withmorecomplexapplications,thelistofdependenttoolsandlibrariescanbewaylonger.Configureapplicationasaservice:WewouldliketohavethecalculatorapplicationrunningasaUnixservice,sothatitwillbemanageableinthestandardway.Inthiscase,it'senoughtocreatealinktoourapplicationinthe/etc/init.d/directory.Copythenewversion:Thenewversionoftheapplicationiscopiedintotheserver.Notethatifthesourcefiledidn'tchange,thenthefilewon'tbecopiedandthereforetheservicewon'tberestarted.Restarttheservice:Asahandler,everytimethenewversionoftheapplicationiscopied,theserviceisrestarted.
RunningdeploymentAsalways,wecanexecutetheplaybookusingtheansible-playbookcommand.Beforethat,weneedtobuildthecalculatorprojectwithGradle.
$./gradlewbuild
$ansible-playbookplaybook.yml
Afterthesuccessfuldeployment,theserviceshouldbeavailableandwecancheckit'sworkingathttp://192.168.0.242:8080/sum?a=1&b=2.Asexpected,itshouldreturn3astheoutput.
Notethatwehaveconfiguredthewholeenvironmentbyexecutingonecommand.What'smore,ifweneedtoscaletheservice,thenit'senoughtoaddanewservertotheinventoryandre-runtheansible-playbookcommand.
WehaveshowedhowtouseAnsibleforenvironmentconfigurationandapplicationdeployment.ThenextstepistouseAnsibletogetherwithDocker.
AnsiblewithDockerAsyoumayhavenoticed,AnsibleandDockeraddresssimilarsoftwaredeploymentissues:
Environmentconfiguration:BothAnsibleandDockerprovideawaytoconfiguretheenvironment;however,theyusedifferentmeans.WhileAnsibleusesscripts(encapsulatedinsidetheAnsiblemodules),Dockerencapsulatesthewholeenvironmentinsideacontainer.Dependencies:Ansibleprovidesawaytodeploydifferentservicesonthesameordifferenthostsandletthembedeployedtogether.DockerComposehasasimilarfunctionality,whichallowsrunningmultiplecontainersatthesametime.Scalability:Ansiblehelpstoscaleservicesprovidingtheinventoryandhostgroups.DockerComposehasasimilarfunctionalitytoautomaticallyincreaseordecreasethenumberofrunningcontainers.Automationwithconfigurationfiles:BothDockerandAnsiblestorethewholeenvironmentconfigurationandservicedependenciesinfiles(storedinthesourcecontrolrepository).ForAnsible,thisfileiscalledplaybook.yml.InthecaseofDocker,wehaveDockerfilefortheenvironmentanddocker-compose.ymlforthedependenciesandscaling.Simplicity:Bothtoolsareverysimpletouseandprovideawaytosetupthewholerunningenvironmentwithaconfigurationfileandjustonecommandexecution.
Ifwecomparethetools,thenDockerdoesalittlemore,sinceitprovidestheisolation,portability,andsomekindofsecurity.WecouldevenimagineusingDockerwithoutanyotherconfigurationmanagementtool.Then,whydoweneedAnsibleatall?
BenefitsofAnsibleAnsiblemayseemredundant;however,itbringsadditionalbenefitstothedeliveryprocess:
Dockerenvironment:TheDockerhostitselfhastobeconfiguredandmanaged.EverycontainerisultimatelyrunningonLinuxmachines,whichneedskernelpatching,Dockerengineupdates,networkconfiguration,andsoon.What'smore,theremaybedifferentservermachineswithdifferentLinuxdistributionsandtheresponsibilityofAnsibleistomakesuretheDockerengineisupandrunning.Non-Dockerizedapplications:Noteverythingisruninsideacontainer.Ifpartoftheinfrastructureiscontainerizedandpartisdeployedinthestandardwayorinthecloud,thenAnsiblecanmanageitallwiththeplaybookconfigurationfile.Theremaybedifferentreasonsfornotrunninganapplicationasacontainer,forexampleperformance,security,specifichardwarerequirements,Windows-basedsoftware,orworkingwiththelegacysoftware.Inventory:Ansibleoffersaveryfriendlywaytomanagethephysicalinfrastructureusinginventories,whichstoretheinformationaboutallservers.Itcanalsosplitthephysicalinfrastructureintodifferentenvironments:production,testing,development.GUI:Ansibleoffersa(commercial)GUImanagercalledAnsibleTower,whichaimstoimprovetheinfrastructuremanagementfortheenterprises.Improvetestingprocess:AnsiblecanhelpwiththeintegrationandacceptancetestingandcanencapsulatethetestingscriptsinasimilarwaythatDockerComposedoes.
WecanlookatAnsibleasthetoolthattakescareoftheinfrastructure,whileDockerasatoolthattakescareoftheenvironmentconfiguration.Theoverviewispresentedinthefollowingdiagram:
Ansiblemanagestheinfrastructure:Dockerservers,Dockerregistry,serverswithoutDocker,andcloudproviders.Italsotakescareofthephysicallocationoftheservers.Usingtheinventoryhostgroups,itcanlinkthewebservicestothedatabasesthatareclosetotheirgeographiclocation.
AnsibleDockerplaybookAnsibleintegratessmoothlywithDockerbecauseitprovidesasetofDocker-dedicatedmodules.IfwecreateanAnsibleplaybookforDocker-baseddeployment,thenthefirsttasksneedtomakesurethattheDockerengineisinstalledoneverymachine.Then,itshouldrunacontainerusingDockerorasetofinteractingcontainersusingDockerCompose.
ThereareafewveryusefulDocker-relatedmodulesprovidedbyAnsible:docker_image(build/manageimages),docker_container(runcontainers),docker_image_facts(inspectimages),docker_login(logintoDockerregistry),docker_network(manageDockernetworks),anddocker_service(manageDockerCompose).
InstallingDockerWecaninstalltheDockerengineusingthefollowingtaskintheAnsibleplaybook.
tasks:
-name:adddockeraptkeys
apt_key:
keyserver:hkp://p80.pool.sks-keyservers.net:80
id:9DC858229FC7DD38854AE2D88D81803C0EBFCD88
-name:updateapt
apt_repository:
repo:deb[arch=amd64]https://download.docker.com/linux/ubuntuxenialmainstable
state:present
-name:installDocker
apt:
name:docker-ce
update_cache:yes
state:present
-name:addadmintodockergroup
user:
name:admin
groups:docker
append:yes
-name:installpython-pip
apt:
name:python-pip
state:present
-name:installdocker-py
pip:
name:docker-py
-name:installDockerCompose
pip:
name:docker-compose
version:1.9.0
Theplaybooklooksslightlydifferentforeachoperatingsystem.TheonepresentedhereisforUbuntu16.04.
ThisconfigurationinstallstheDockerengine,enablestheadminusertoworkwithDocker,andinstallsDockerComposewithdependenttools.
Alternatively,youmayalsousethedocker_ubunturoleasdescribedhere:https://www.ansible.com/2014/02/12/installing-and-building-docker-with-ansible.
WhenDockerisinstalled,wecanaddatask,whichwillrunaDockercontainer.
RunningDockercontainersRunningDockercontainersisdonewiththeuseofthedocker_containermoduleanditlooksverysimilartowhatwepresentedfortheDockerComposeconfiguration.Let'saddittotheplaybook.ymlfile.
-name:runRediscontainer
docker_container:
name:redis
image:redis
state:started
exposed_ports:
-6379
Youcanreadmoreaboutalloftheoptionsofthedocker_containermoduleontheofficialAnsiblepageat:https://docs.ansible.com/ansible/docker_container_module.html.
WecannowexecutetheplaybooktoobservethatDockerhasbeeninstalledandtheRediscontainerstarted.Notethatit'saveryconvenientwayofusingDocker,sincewedon'tneedtomanuallyinstallDockerengineoneverymachine.
UsingDockerComposeTheAnsibleplaybookisverysimilartotheDockerComposeconfiguration.TheyevenbothsharethesameYAMLfileformat.What'smore,itispossibletousedocker-compose.ymldirectlyfromAnsible.Wewillshowhowtodoit,butfirst,let'sdefinethedocker-compose.ymlfile.
version:"2"
services:
calculator:
image:leszko/calculator:latest
ports:
-8080
redis:
image:redis:latest
Itisalmostthesameaswhatwedefinedinthepreviouschapter.ThistimewegetthecalculatorimagedirectlyfromtheDockerHubregistry,anddonotbuilditindocker-compose.yml,sincewewanttobuildtheimageonce,pushittotheregistry,andthenreuseitineverydeploymentstep(oneveryenvironment),tomakesurethesameimageisdeployedoneachDockerhostmachine.Whenwehavedocker-compose.yml,wearereadytoaddnewtaskstoplaybook.yml.
-name:copydocker-compose.yml
copy:
src:./docker-compose.yml
dest:./docker-compose.yml
-name:rundocker-compose
docker_service:
project_src:.
state:present
Wefirstcopythedocker-compose.ymlfileintotheserverandthenexecutedocker-compose.Asaresult,Ansiblecreatestwocontainers:calculatorandredis.
WehaveseenthemostimportantfeaturesofAnsible.Inthenextsections,wewritealittlebitabouttheinfrastructureandapplicationversioning.Attheendofthischapter,wewillpresenthowtouseAnsibleinordertocompletetheContinuousDeliverypipeline.
Exercises
Inthischapter,wehavecoveredthefundamentalsofAnsibleandthewaytouseittogetherwithDocker.Asanexercise,weproposethefollowingtasks:
1. CreatetheserverinfrastructureanduseAnsibletomanageit.
ConnectaphysicalmachineorrunaVirtualBoxmachinetoemulatetheremoteserverConfigureSSHaccesstotheremotemachine(SSHkeys)InstallPythonontheremotemachineCreateanAnsibleinventorywiththeremotemachineRuntheAnsibleadhoccommand(withthepingmodule)tocheckthattheinfrastructureisconfiguredcorrectly
2. CreateaPython-based"helloworld"webserviceanddeployitinaremotemachineusingAnsibleplaybook.
TheservicecanlookexactlythesameasdescribedintheexercisesforthechapterCreateaplaybook,whichdeploystheserviceintotheremotemachineRuntheansible-playbookcommandandcheckwhethertheservicewasdeployed
SummaryWehavecoveredtheconfigurationmanagementprocessanditsrelationtoDocker.Thekeytakeawayfromthechapterisasfollows:
ConfigurationmanagementisaprocessofcreatingandapplyingtheconfigurationsoftheinfrastructureandtheapplicationAnsibleisoneofthebesttrendingconfigurationmanagementtools.ItisagentlessandthereforerequiresnospecialserverconfigurationAnsiblecanbeusedwithadhoccommands,buttherealpowerliesinAnsibleplaybooksTheAnsibleplaybookisadefinitionofhowtheenvironmentshouldbeconfiguredThepurposeofAnsiblerolesistoreusepartsofplaybooks.AnsibleGalaxyisanonlineservicetoshareAnsiblerolesAnsibleintegrateswellwithDockerandbringsadditionalbenefitscomparedtousingDockerandDockerComposealone
Inthenextchapter,wewillwrapuptheContinuousDeliveryprocessandcompletethefinalJenkinspipeline.
ContinuousDeliveryPipeline
WehavealreadycoveredthemostcrucialpartsoftheContinuousDeliveryprocess:thecommitphase,theartifactrepository,automatedacceptancetesting,andconfigurationmanagement.
Inthischapter,wewillfocusonthemissingpartsofthefinalpipeline,whicharetheenvironmentsandinfrastructure,applicationversioning,andnonfunctionaltesting.
Thischaptercoversthefollowingpoints:
DesigningdifferentsoftwareenvironmentsandtheirinfrastructuresSecuringtheconnectionbetweenJenkinsagentsandserversIntroducingvariouskindsofnonfunctionaltestsPresentingthechallengesofnonfunctionaltestsintheContinuousDeliveryprocessExplainingdifferenttypesofapplicationversioningCompletingtheContinuousDeliverypipelineIntroducingtheconceptofsmoketestingandaddingittothefinalpipeline
EnvironmentsandinfrastructureSofar,wehavealwaysusedoneDockerhostforeverythingandtreateditasthevirtualizationofendlessresourceswherewecandeployeverything.Obviously,theDockerhostcanactuallybeaclusterofmachinesandwewillshowhowtocreateitusingDockerSwarminthenextchapters.However,eveniftheDockerhostwereunlimitedintermsofresources,westillneedtothinkabouttheunderlyinginfrastructureforatleasttworeasons:
PhysicallocationofthemachinesmattersNotestingshouldbedoneontheproductionphysicalmachines
Takingthesefactsintoconsideration,inthissection,wewilldiscussdifferenttypesofenvironment,theirroleintheContinuousDeliveryprocess,andinfrastructuresecurityaspects.
TypesofenvironmentTherearefourmostcommonenvironmenttypes:production,staging,QA(testing),anddevelopment.Let'sdiscusseachofthemanditsinfrastructure.
ProductionProductionistheenvironmentthatisusedbytheenduser.Itexistsineverycompanyand,ofcourse,itisthemostimportantenvironment.
Let'slookatthefollowingdiagramandseehowmostproductionenvironmentsareorganized:
Usersaccesstheservicethroughtheloadbalancer,whichchoosestheexactmachine.Iftheapplicationisreleasedinmultiplephysicallocations,thenthe(first)deviceisusuallyaDNS-basedgeographicloadbalancer.Ineachlocation,wehaveaclusterofservers.IfweuseDocker,thenthatclusterofserverscanbehiddenbehindoneormultipleDockerhosts(whichareinternallycomposedofmanymachinesusingDockerSwarm).
Thephysicallocationofmachinesmattersbecausetherequest-responsetimecandiffersignificantlydependingonthephysicaldistance.Moreover,thedatabaseandotherdependentservicesshouldbelocatedonamachinethatisclosetowheretheserviceisdeployed.What'sevenmoreimportantisthatthedatabaseshouldbeshardedinawaythatthereplicationoverheadbetweendifferentlocationsisminimized.Otherwise,wemayendupwaitingalotforthedatabasestoreachconsensusbetweenitsinstanceslocatedfarawayfromeachother.Moredetailsonthephysicalaspectsarebeyondthescopeofthisbook,butit'simportanttorememberthatDockerisnotalwaysasilverbullet.
Containerizationandvirtualizationallowyoutothinkaboutserversasaninfiniteresource;however,somephysicalaspectssuchaslocationarestillrelevant.
StagingThestagingenvironmentistheplacewherethereleasecandidateisdeployedinordertoperformthefinaltestsbeforegoinglive.Ideally,thisenvironmentisamirroroftheproduction.
Let'slookatthefollowingtoseehowsuchanenvironmentshouldlookinthecontextofthedeliveryprocess:
Notethatthestagingenvironmentanexactaproductionclone.Iftheapplicationisdeployedinmultiplelocations,thenthestagingshouldalsohavemultiplelocations.
IntheContinuousDeliveryprocess,allautomatedacceptancefunctionalandnonfunctionaltestsarerunagainstthisenvironment.Whilemostfunctionaltestsdon'tusuallyrequireidenticalproduction-likeinfrastructure,inthecaseofnonfunctional(especiallyperformance)tests,it'samust.
Itisnotuncommonthat,forthepurposeofcostsaving,thestaginginfrastructurediffersfromtheproduction(usuallyitcontainsfewermachines).Suchanapproachcan,however,leadtomanyproductionissues.MichaelT.Nygard,inhisgreatbookReleaseIt!,givesanexampleofareal-lifescenarioinwhichfewermachineswereusedinthestagingenvironmentthaninproduction.
Thestorygoeslikethis:inonecompany,thesystemwasstableuntilacertaincodechange,whichcausedtheproductiontobeextremelyslow,eventhoughallstresstestspassed.Howwasitpossible?Ithappenedthattherewasasynchronizationpoint,inwhicheachservercommunicatedwitheachother.In
thecaseofthestaging,therewasoneserver,soactuallytherewasnoblocker.Inproduction,however,thereweremanyservers,whichresultedinserverswaitingforeachother.Thisexampleisjustthetipoftheicebergandmanyproductionissuesmayfailtobetestedbyacceptancetestsifthestagingenvironmentisdifferentfromtheproductionenvironment.
QATheQAenvironment(alsocalledthetestingenvironment)isintendedfortheQAteamtoperformexploratorytestingandforexternalapplications(whichdependonourservice)toperformintegrationtesting.TheusecasesandtheinfrastructureoftheQAenvironmentarepresentedinthefollowingdiagram:
Whilestagingdoesnotneedtobestable(inthecaseofContinuousDelivery,itischangedaftereverycodechangecommittedtotherepository),theQAinstanceneedstoprovideacertainstabilityandexposethesame(orbackwardcompatible)APIastheproduction.Contrarytothestagingenvironment,theinfrastructurecanbedifferentfromtheproduction,sinceitspurposeisnottoensurethatthereleasecandidateworksproperly.
Averycommoncaseistoallocatefewermachines(forexample,onlyfromonelocation)forthepurposeoftheQAinstance.
DeployingtotheQAenvironmentisusuallydoneinaseparatepipeline,sothatitwillbeindependentfromtheautomaticreleaseprocess.Suchanapproachisconvenient,becausetheQAinstancehasadifferentlifecyclethanproduction(forinstance,theQAteammaywanttoperformtestingontheexperimentalcodebranchedfromthetrunk).
DevelopmentThedevelopmentenvironmentcanbecreatedasasharedserverforalldevelopersoreachdevelopercanhavehis/herowndevelopmentenvironment.Asimplediagramispresentedhere:
Thedevelopmentenvironmentalwayscontainsthelatestversionofthecode.ItisusedtoenableintegrationbetweendevelopersandcanbetreatedthesamewayastheQAenvironment,butisusedbydevelopers,notQAs.
EnvironmentsinContinuousDeliveryForthepurposeoftheContinuousDeliveryprocess,thestagingenvironmentisindispensable.Insomeveryrarecases,whentheperformanceisnotimportantandtheprojectdoesnothavemanydependencies,wecouldperformtheacceptancetestsonthelocal(development)Dockerhost(likewedidinthepreviouschapter),butthatshouldbeanexception,notarule.Insuchacase,wealwaysrisksomeproductionissuesrelatedtotheenvironment.
TheotherenvironmentsareusuallynotimportantwithregardtoContinuousDelivery.IfwewouldliketodeploytotheQAordevelopmentenvironmentwitheverycommit,thenwecancreateseparatepipelinesforthatpurpose(beingcarefulnottoobscurethemainreleasepipeline).Inmanycases,deploymenttotheQAenvironmentistriggeredmanually,becauseitcanhavedifferentlifecyclesfromproduction.
SecuringenvironmentsAllenvironmentsneedtobewellsecured.That'sclear.What'sevenmoreobviousisthatthemostimportantrequirementistokeeptheproductionsecure,becauseourbusinessdependsonitandtheconsequencesofanysecurityflawcanbehighestthere.
Securityisabroadtopic.Inthissection,wefocusonlyonthetopicsrelatedtotheContinuousDeliveryprocess.Nevertheless,settingupacompleteserverinfrastructurerequiresmuchmoreknowledgeaboutsecurity.
IntheContinuousDeliveryprocess,theslavemusthaveaccesstoservers,sothatitcandeploytheapplication.
Therearedifferentapproachesforprovidingslaveswiththeserver'scredentials:
PutSSHkeyintoslave:Ifwedon'tusedynamicDockerslaveprovisioning,thenwecanconfigureJenkinsslavemachinestocontainprivateSSHkeys.PutSSHkeyintoslaveimage:IfweusedynamicDockerslaveprovisioning,wecouldaddtheSSHprivatekeyintotheDockerslaveimage.However,itcreatesapossiblesecurityhole,sinceanyonewhohasaccesstothatimagewouldhaveaccesstotheproductionservers.Jenkinscredentials:WecanconfigureJenkinstostorecredentialsandusetheminthepipeline.CopytoSlaveJenkinsplugin:WecancopytheSSHkeydynamicallyintotheslavewhilestartingtheJenkinsbuild.
Eachsolutionhassomeadvantagesanddrawbacks.Whileusinganyofthemwehavetotakeextracaution,since,whenaslavehasaccesstotheproduction,thenanyonebreakingintotheslavebreaksintotheproduction.
ThemostriskysolutionistoputSSHprivatekeysintotheJenkinsslaveimage,sincethenalltheplaceswheretheimageisstored(theDockerregistryorDockerhostwithJenkins)needtobewellsecured.
NonfunctionaltestingWelearnedalotaboutfunctionalrequirementsandautomatedacceptancetestinginthepreviouschapter.However,whatshouldwedowithnonfunctionalrequirements?Orevenmorechallenging,whatiftherearenorequirements?ShouldweskipthematallintheContinuousDeliveryprocess?Let'sanswerthesequestionsthroughoutthissection.
Nonfunctionalaspectsofthesoftwarearealwaysimportant,becausetheycancauseasignificantrisktotheoperationofthesystem.
Forexample,manyapplicationsfail,becausetheyarenotabletobeartheloadofasuddenincreaseinthenumberofusers.InthebookUsabilityEngineering,JakobNielsen,writesthat1.0secondisaboutthelimitfortheuser'sflowofthoughttostayuninterrupted.Imaginethatoursystem,withthegrowingload,startstoexceedthatlimit.Userscanstopusingtheservicejustbecauseofitsperformance.Takingitintoconsideration,nonfunctionaltestingisasimportantasfunctionaltesting.
Tocutalongstoryshort,weshouldalwaystakethefollowingstepsfornonfunctionaltesting:
DecidewhichnonfunctionalaspectsarecrucialtoourbusinessForeachofthem:
SpecifytheteststhesamewaywedidforacceptancetestingAddastagetotheContinuousDeliverypipeline(afteracceptancetesting,whiletheapplicationisstilldeployedonthestagingenvironment)
Theapplicationcomestothereleasestageonlyafterallnonfunctionaltestspass
Irrespectiveofthetypeofthenonfunctionaltest,theideaisalwaysthesame.Theapproach,however,mayslightlydiffer.Let'sexaminedifferenttesttypesandthechallengestheypose.
TypesofnonfunctionaltestFunctionaltestarealwaysrelatedtothesameaspect—thebehaviorofthesystem.Onthecontrary,nonfunctionaltestsconcernalotofdifferentaspects.Let'sdiscussthemostcommonsystempropertiesandhowtheycanbetestedinsidetheContinuousDeliveryprocess.
PerformancetestingPerformancetestsarethemostwidelyusednonfunctionaltests.Theymeasuretheresponsivenessandstabilityofthesystem.Thesimplestperformancetestwecouldcreateistosendarequesttothewebserviceandmeasureitsround-triptime(RTT).
Therearedifferentdefinitionsofperformancetesting.Inmanyplaces,theyaremeanttoincludeload,stress,andscalabilitytesting.Sometimestheyarealsodescribedaswhite-boxtests.Inthisbook,wedefineperformancetestingasthemostbasicformofblack-boxtesttomeasurethelatencyofthesystem.
Forthepurposeofperformancetesting,wecanuseadedicatedframework(forJavathemostpopularisJMeter)orjustusethesametoolweusedforacceptancetests.AsimpleperformancetestisusuallyaddedasapipelinestagejustafterAcceptancetests.SuchatestshouldfailiftheRTTexceedsthegivenlimitanditdetectsbugsthatdefinitelyslowdowntheservice.
TheJMeterpluginforJenkinscanshowperformancetrendsoverthetime.
LoadtestingLoadtestsareusedtocheckhowthesystemfunctionswhentherearealotofconcurrentrequests.Whileasystemcanbeveryfastwithasinglerequest,itdoesnotmeanthatitworksfastenoughwith1,000requestsatthesametime.Duringloadtesting,wemeasuretheaveragerequest-responsetimeofmanyconcurrentcalls,usuallyperformedfrommanymachines.LoadtestingisaverycommonQAphaseinthereleasecycle.Toautomateit,wecanusethesametoolsaswiththesimpleperformancetest;however,inthecaseoflargersystems,wemayneedaseparateclientenvironmenttoperformalargenumberofconcurrentrequests.
StresstestingStresstesting,alsocalledcapacitytestingorthroughputtesting,isatestthatdetermineshowmanyconcurrentuserscanaccessourservice.Itmaysoundthesameasloadtesting;however,inthecaseofloadtesting,wesetthenumberofconcurrentusers(throughput)toagivennumber,checktheresponsetime(latency),andmakethebuildfailifthelimitisexceeded.Duringstresstesting,however,wekeepthelatencyconstantandincreasethethroughputtodiscoverthemaximumnumberofconcurrentcallswhenthesystemisstilloperable.Sotheresultofastresstestmaybenotificationthatoursystemcanhandle10,000concurrentusers,whichhelpsusprepareforthepeakusagetime.
StresstestingisnotwellsuitedfortheContinuousDeliveryprocess,becauseitrequireslongtestswithanincreasingnumberofconcurrentrequests.ItshouldbepreparedasaseparatescriptofaseparateJenkinspipelineandtriggeredondemand,whenweknowthatthecodechangecancauseperformanceissues.
ScalabilitytestingScalabilitytestingexplainshowlatencyandthroughputchangewhenweaddmoreserversorservices.Theperfectcharacteristicwouldbelinear,whichmeansifwehaveoneserverandtheaveragerequest-responsetimeis500mswhenusedby100parallelusers,thenaddinganotherserverwouldkeeptheresponsetimethesameandallowustoaddanother100parallelusers.Inreality,it'softenhardtoachievethsibecauseofkeepingdataconsistencybetweenservers.
Scalabilitytestingshouldbeautomatedandshouldprovidethegraphpresentingtherelationshipbetweenthenumberofmachinesandthenumberofconcurrentusers.Suchdataishelpfulindeterminingthelimitsofthesystemandthepointatwhichaddingmoremachinesdoesnothelp.
Scalabilitytests,similartostresstests,arehardtoputintotheContinuousDeliverypipelineandshouldratherbekeptseparate.
EndurancetestingEndurancetests,alsocalledlongevitytests,runthesystemforalongtimetoseeiftheperformancedropsafteracertainperiodoftime.Theydetectmemoryleaksandstabilityissues.Sincetheyrequireasystemrunningforalongtime,itdoesn'tmakesensetoruntheminsidetheContinuousDeliverypipeline.
SecuritytestingSecuritytestingdealswithdifferentaspectsrelatedtosecuritymechanismsanddataprotection.Somesecurityaspectsarepurelyfunctionalrequirementssuchasauthentication,authorization,orroleassignment.Thesepartsshouldbecheckedthesamewayasanyotherfunctionalrequirement—duringtheacceptancetestphase.Therearealsoothersecurityaspectsthatarenonfunctional;forexample,thesystemshouldbeprotectedagainstSQLinjection.Noclientwouldprobablyspecifysucharequirement,butit'simplicit.
SecuritytestsshouldbeincludedinContinuousDeliveryasapipelinestage.Theycanbewrittenusingthesameframeworksastheacceptancetestsorwithdedicatedsecuritytestingframeworks,forexample,BDDsecurity.
Securityshouldalsoalwaysbeaparttheexplanatorytestingprocess,inwhichtestersandsecurityexpertsdetectsecurityholesandaddnewtestingscenarios.
MaintainabilitytestingMaintainabilitytestsexplainhowsimpleasystemistomaintain.Inotherwords,theyjudgecodequality.Wealreadyhaverelatedstagesinthecommitphasethatcheckthetestcoverageandperformstaticcodeanalysis.TheSonartoolcanalsogivesomeoverviewofthecodequalityandthetechnicaldebt.
RecoverytestingRecoverytestingisatechniquetodeterminehowquicklythesystemcanrecoverafteritcrashedbecauseofasoftwareorhardwarefailure.Thebestcasewouldbeifthesystemdoesnotfailatall,evenifapartofitsservicesisdown.Somecompaniesevenperformproductionfailuresonpurposetocheckiftheycansurviveadisaster.ThebestknownexampleisNetflixandtheirChaosMonkeytool,whichrandomlyterminatesrandominstancesoftheproductionenvironment.Suchanapproachforcesengineerstowritecodethatmakessystemsresilienttofailures.
RecoverytestingisobviouslynotpartoftheContinuousDeliveryprocess,butratheraperiodiceventtochecktheoverallhealth.
YoucanreadmoreaboutChaosMonkeyathttps://github.com/Netflix/chaosmonkey.
Therearemanymorenonfunctionaltesttypes,whichareclosertoorfurtherfromthecodeandtheContinuousDeliveryprocess.Someofthemrelatetothelawsuchascompliancetesting;othersarerelatedtothedocumentationorinternationalization.Therearealsousabilitytestingsandvolumetestings(whichcheckwhetherthesystembehaveswellinthecaseoflargeamountsofdata).Mostofthesetests,however,havenopartintheContinuousDeliveryprocess.
Nonfunctionalchallenges
Nonfunctionalaspectsposenewchallengestothesoftwaredevelopmentanddelivery:
Longtestrun:Thetestscantakealongtimetorunandmayneedaspecialexecutionenvironment.Incrementalnature:It'shardtosetthelimitvaluewhenthetestshouldfail(unlessSLAiswelldefined).Eveniftheedgelimitisset,theapplicationwouldprobablyincrementallyapproachthelimit.Inmostcases,actually,noonecodechangecausedthetestfailure.Vaguerequirements:Usersusuallydon'thavemuchinputconcerningnonfunctionalrequirements.Theymayprovidesomeguidelinesconcerningtherequest-responsetimeorthenumberofusers;however,theywon'tprobablyknowmuchaboutmaintainability,security,orscalability.Multiplicity:Therearealotofdifferentnonfunctionaltestsandchoosingwhichshouldbeimplementedrequiresmakingsomecompromises.
Thebestapproachtoaddressnonfunctionalaspectsistotakethefollowingsteps:
1. Makealistofallnonfunctionaltesttypes.2. Crossoutexplicitlythetestyoudon'tneedforyoursystem.Theremaybea
lotofreasonsyoudon'tneedonekindoftest,forexample:
TheserviceissupersmallandasimpleperformancetestisenoughThesystemisinternalonlyandavailableonlyforread-only,soitmaynotneedanysecuritychecksThesystemisdesignedforonemachineonlyanddoesnotneedanyscalingThecostofcreatingcertaintestsistoohigh
3. Splityourtestsintotwogroups:
ContinuousDelivery:ItispossibletoaddittothepipelineAnalysis:Itisnotpossibletoaddtothepipelinebecauseoftheirexecutiontime,theirnature,ortheassociatedcost
4. FortheContinuousDeliverygroup,implementtherelatedpipelinestages.5. FortheAnalysisgroup:
CreateautomatedtestsSchedulewhentheyshouldberunSchedulemeetingstodiscusstheirresultsandtakeactionpoints
Averygoodapproachistohaveanightlybuildwiththelongteststhatdon'tfittheContinuousDeliverypipeline.Then,it'spossibletoscheduleaweeklymeetingtomonitorandanalyzethetrendsofsystemperformance.
Aspresented,therearemanytypesofnonfunctionaltestandtheyposeadditionalchallengestothedeliveryprocess.Nevertheless,forthesakeofthestabilityofoursystem,thesetestsshouldneverbeblanklyskipped.Thetechnicalimplementationdiffersdependingonthetesttype,butinmostcasestheycanbeimplementedinasimilarmannertofunctionalacceptancetestsandshouldberunagainstthestagingenvironment.
Ifyou'reinterestedinthetopicofnonfunctionaltesting,systemproperties,andsystemstability,thenreadthebookReleaseIt!byMichaelT.Nygard.
ApplicationversioningSofar,duringeveryJenkinsbuild,wehavecreatedanewDockerimage,pusheditintotheDockerregistry,andusedthelatestversionthroughouttheprocess.However,suchasolutionhasatleastthreedisadvantages:
If,duringtheJenkinsbuild,aftertheacceptancetests,someonepushesanewversionoftheimage,thenwecanendupreleasingtheuntestedversionWealwayspushanimagenamedinthesameway;thus,soeffectively,itisoverwrittenintheDockerregistryIt'sveryhardtomanageimageswithoutversionsjustbytheirhashed-styleIDs
WhatistherecommendedwayofmanagingDockerimageversionstogetherwiththeContinuousDeliveryprocess?Inthissection,wegettoseedifferentversioningstrategiesandlearndifferentwaysofcreatingversionsintheJenkinspipeline.
VersioningstrategiesTherearedifferentwaystoversionapplications.
Let'sdiscussthesemostpopularsolutions,whichcanbeappliedtogetherwiththeContinuousDeliveryprocess(wheneachcommitcreatesanewversion).
Semanticversioning:Themostpopularsolutionistousesequence-basedidentifiers(usuallyintheformofx.y.z).ThismethodrequiresacommittotherepositorydonebyJenkinsinordertoincreasethecurrentversionnumber,whichisusuallystoredinthebuildfile.ThissolutioniswellsupportedbyMaven,Gradle,andotherbuildtools.Theidentifierusuallyconsistsofthreenumbers:
x:Thisisthemajorversion;thesoftwaredoesnotneedtobebackward-compatiblewhenthisversionisincrementedy:Thisistheminorversion;thesoftwareneedstobebackwardcompatiblewhentheversionisincrementedz:Thisisthebuildnumber;thisissometimesalsoconsideredasabackwardandforward-compatiblechange
Timestamp:Usingthedateandtimeofthebuildfortheapplicationversionislessverbosethansequentialnumbers,butveryconvenientinthecaseoftheContinuousDeliveryprocess,becauseitdoesnotrequirecommittingbacktotherepositorybyJenkins.Hash:Arandomlygeneratedhashversionsharesthebenefitofthedatetimeandisprobablythesimplestsolutionpossible.Thedrawbackisthatit'snotpossibletolookattwoversionsandtellwhichisthelatestone.Mixed:Therearemanyvariationsofthesolutionsdescribedearlier,forexample,majorandminorversionswiththedatetime.
AllsolutionsarefinetousewiththeContinuousDeliveryprocess.Semanticversioningrequires,however,acommittotherepositoryfromthebuildexecution,sothattheversionisincreasedinthesourcecoderepository.
Maven(andtheotherbuildtools)popularizedversionsnapshotting,whichaddedasuffixSNAPSHOTtotheversionsthat
arenotreleased,butkeptjustforthedevelopmentprocess.SinceContinuousDeliverymeansreleasingeverychange,therearenosnapshots.
VersioningintheJenkinspipelineAsdescribedearlier,therearedifferentpossibilitieswhenitcomestousingsoftwareversioningandeachofthemcanbeimplementedinJenkins.
Asanexample,let'susethedatetime.
InordertousethetimestampinformationfromJenkins,youneedtoinstalltheBuildTimestampPluginandsetthetimestampformatintheJenkinsconfiguration(forexample,to"yyyyMMdd-HHmm").
IneveryplacewhereweusetheDockerimage,weneedtoaddthetagsuffix:${BUILD_TIMESTAMP}.
Forexample,theDockerbuildstageshouldlooklikethis:sh"dockerbuild-tleszko/calculator:${BUILD_TIMESTAMP}."
Afterthechanges,whenweruntheJenkinsbuild,weshouldhavetheimagetaggedwiththetimestampversioninourDockerregistry.
Notethatafterexplicitlytaggingtheimage,it'snolongerimplicitlytaggedasthelatest.
Withversioningcompleted,wearefinallyreadytocompletetheContinuousDeliverypipeline.
CompleteContinuousDeliverypipeline
AfterdiscussingalltheaspectsofAnsible,environments,nonfunctionaltesting,andversioning,wearereadytoextendtheJenkinspipelineandfinalizeasimple,butcomplete,ContinuousDeliverypipeline.
Wewilldoitinafewstepsasfollows:
CreatetheinventoryofstagingandproductionenvironmentsUpdateacceptanceteststousetheremotehost(insteadoflocal)ReleasetheapplicationtotheproductionenvironmentAddasmoketestwhichmakessuretheapplicationwassuccessfullyreleased
InventoryIntheirsimplestform,wecanhavetwoenvironments:stagingandproduction,eachhavingoneDockerhostmachine.Inreallife,wemaywanttoaddmorehostgroupsforeachenvironmentifwewanttohaveserversindifferentlocationsorhavingdifferentrequirements.
Let'screatetwoAnsibleinventoryfiles.Startingfromthestaging,wecandefinetheinventory/stagingfile.Assumingthestagingaddressis192.168.0.241,itwouldhavethefollowingcontent:[webservers]web1ansible_host=192.168.0.241ansible_user=admin
Byanalogy,iftheproductionIPaddressis192.168.0.242,thentheinventory/productionshouldlooklikethis:
[webservers]
web2ansible_host=192.168.0.242ansible_user=admin
Itmaylookoversimplifiedtohavejustonemachineforeachenvironment;however,usingDockerSwarm(whichweshowlaterinthisbook),aclusterofhostscanbehiddenbehindoneDockerhost.
Havingtheinventorydefined,wecanchangeacceptancetestingtousethestagingenvironment.
AcceptancetestingenvironmentDependingonourneeds,wecouldtesttheapplicationbyrunningitonthelocalDockerhost(likewedidinthepreviouschapter)orusingtheremotestagingenvironment.Theformersolutionisclosertowhathappensinproduction,soitcanbeconsideredasabetterone.ThisisveryclosetowhatwaspresentedintheMethod1:Jenkins-firstacceptancetestingsectionofthepreviouschapter.TheonlydifferenceisthatnowwedeploytheapplicationonaremoteDockerhost.
Inordertodothis,wecouldusedocker(orthedocker-composecommand)withthe-Hparameter,whichspecifiestheremoteDockerhostaddress.Thiswouldbeagoodsolutionandifyoudon'tplantouseAnsibleoranyotherconfigurationmanagementtool,thenthatisthewaytogo.Nevertheless,forthereasonsalreadymentionedinthischapter,itisbeneficialtouseAnsible.Inthatcase,wecanusetheansible-playbookcommandinsidetheContinuousDeliverypipeline.
stage("Deploytostaging"){
steps{
sh"ansible-playbookplaybook.yml-iinventory/staging"
}
}
Ifplaybook.ymlanddocker-compose.ymllookthesameasintheAnsiblewithDockersection,thenitshouldbeenoughtodeploytheapplicationwithdependenciesintothestagingenvironment.
TheAcceptanceteststagelooksexactlythesameasinthepreviouschapter.Theonlyadjustmentcanbethehostnameofthestagingenvironment(oritsloadbalancer).It'salsopossibletoaddstagesforperformancetestingorothernonfunctionaltestsagainsttheapplicationrunningonthestagingenvironment.
Afteralltestsarepassed,it'shightimetoreleasetheapplication.
ReleaseTheproductionenvironmentshouldbeasclosetothestagingenvironmentaspossible.TheJenkinsstepforthereleaseshouldalsobeverysimilartothestagethatdeploystheapplicationtothestagingenvironment.
Inthesimplestscenario,theonlydifferencesaretheinventoryfileandtheapplicationconfiguration(forexample,incaseofaSpringBootapplication,wewouldsetadifferentSpringprofile,whichresultsintakingadifferentpropertiesfile).Inourcase,therearenoapplicationproperties,sotheonlydifferenceistheinventoryfile.
stage("Release"){
steps{
sh"ansible-playbookplaybook.yml-iinventory/production"
}
}
Inreality,thereleasestepcanbealittlemorecomplexifwewanttoprovidezerodowntimedeployment.Moreonthattopicispresentedinthenextchapters.
Afterthereleaseisdone,wemightthinkthateverythingiscompleted;however,thereisonemoremissingstage,asmoketest.
SmoketestingAsmoketestisaverysmallsubsetofacceptancetestswhoseonlypurposeistocheckthatthereleaseprocessiscompletedsuccessfully.Otherwise,wecouldhaveasituationinwhichtheapplicationisperfectlyfine;however,thereisanissueinthereleaseprocess,sowemayendupwithanon-workingproduction.
Thesmoketestisusuallydefinedinthesamewayastheacceptancetest.SotheSmoketeststageinthepipelineshouldlooklikethis:stage("Smoketest"){steps{sleep60sh"./smoke_test.sh"}}
Aftereverythingissetup,theContinuousDeliverybuildshouldrunautomaticallyandtheapplicationshouldbereleasedtoproduction.Withthisstep,wehavecompletedtheContinuousDeliverypipelineinitssimplest,butfullyproductive,form.
CompleteJenkinsfileTosumup,throughouttherecentchapterswehavecreatedquiteafewstages,whichresultsinacompleteContinuousDeliverypipelinethatcouldbesuccessfullyusedinmanyprojects.
NextweseethecompleteJenkinsfileforthecalculatorproject:
pipeline{
agentany
triggers{
pollSCM('*****')
}
stages{
stage("Compile"){steps{sh"./gradlewcompileJava"}}
stage("Unittest"){steps{sh"./gradlewtest"}}
stage("Codecoverage"){steps{
sh"./gradlewjacocoTestReport"
publishHTML(target:[
reportDir:'build/reports/jacoco/test/html',
reportFiles:'index.html',
reportName:"JaCoCoReport"])
sh"./gradlewjacocoTestCoverageVerification"
}}
stage("Staticcodeanalysis"){steps{
sh"./gradlewcheckstyleMain"
publishHTML(target:[
reportDir:'build/reports/checkstyle/',
reportFiles:'main.html',
reportName:"CheckstyleReport"])
}}
stage("Build"){steps{sh"./gradlewbuild"}}
stage("Dockerbuild"){steps{
sh"dockerbuild-tleszko/calculator:${BUILD_TIMESTAMP}."
}}
stage("Dockerpush"){steps{
sh"dockerpushleszko/calculator:${BUILD_TIMESTAMP}"
}}
stage("Deploytostaging"){steps{
sh"ansible-playbookplaybook.yml-iinventory/staging"
sleep60
}}
stage("Acceptancetest"){steps{sh"./acceptance_test.sh"}}
//Performanceteststages
stage("Release"){steps{
sh"ansible-playbookplaybook.yml-iinventory/production"
sleep60
}}
stage("Smoketest"){steps{sh"./smoke_test.sh"}}
}
}
YoucanfindthisJenkinsfileonGitHubathttps://github.com/leszko/calculator/blob/master/Jenkinsfile.
Exercises
Inthischapter,wehavecoveredalotofnewaspectsfortheContinuousDeliverypipeline;tobetterunderstandtheconcept,werecommendyouperformthefollowingexercises:
1. Addaperformancetest,whichteststhe"helloworld"service:
The"helloworld"servicecanbetakenfromthepreviouschapterCreateaperformance_test.shscript,whichmakes100callsinparallelandcheckswhethertheaveragerequest-responsetimeisbelow1secondYoucanuseCucumberorthecurlcommandforthescript
2. CreateaJenkinspipelinethatbuildsthe"helloworld"webserviceasaversionedDockerimageandperformsperformancetest:
CreatetheDockerbuildstage,whichbuildstheDockerimagewiththe"helloworld"serviceandaddsatimestampasaversiontagCreateanAnsibleplaybookthatusestheDockerimageAddtheDeploytostagingstage,whichdeploystheimageintotheremotemachineAddthePerformancetestingstage,whichexecutesperformance_test.shRunthepipelineandobservetheresults
SummaryInthischapter,wehavecompletedtheContinuousDeliverypipeline,whichfinallyreleasestheapplication.Thefollowingarethekeytakeawaysfromthechapter:
ForthepurposeofContinuousDelivery,twoenvironmentsareindispensable:stagingandproduction.NonfunctionaltestsareanessentialpartoftheContinuousDeliveryprocessandshouldalwaysbeconsideredaspipelinestages.Nonfunctionalteststhatdon'tfittheContinuousDeliveryprocessshouldbeconsideredasperiodictasksinordertomonitortheoverallperformancetrends.Applicationsshouldalwaysbeversioned;however,theversioningstrategydependsonthetypeoftheapplication.TheminimalContinuousDeliverypipelinecanbeimplementedasasequenceofscriptsthatendswithtwostages:releaseandsmoketest.ThesmoketestshouldalwaysbeaddedasthelaststageoftheContinuousDeliverypipelineinordertocheckwhetherthereleasewassuccessful.
Inthenextchapter,wewillhavealookattheDockerSwarmtool,whichhelpsustocreateaclusterofDockerhosts.
ClusteringwithDockerSwarm
WehavealreadycoveredallthefundamentalaspectsoftheContinuousDeliverypipeline.Inthischapter,wewillseehowtochangetheDockerenvironmentfromasingleDockerhostintoaclusterofmachinesandhowtouseitalltogetherwithJenkins.
Thischaptercoversthefollowingpoints:
ExplainingtheconceptofserverclusteringIntroducingDockerSwarmanditsmostimportantfeaturesPresentinghowtobuildaswarmclusterfrommultipleDockerhostsRunningandscalingDockerimagesonaclusterExploringadvancedswarmfeatures:rollingupdates,drainingnodes,multiplemanagernodes,andtuningtheschedulingstrategyDeployingtheDockerComposeconfigurationonaclusterIntroducingKubernetesandApacheMesosasalternativestoDockerSwarmDynamicallyscalingJenkinsagentsonacluster
ServerclusteringSofar,wehavehaveinteractedwitheachofthemachinesindividually.EvenwhenweusedAnsibletorepeatthesameoperationsonmultipleservers,wehadtoexplicitlyspecifyonwhichhostthegivenserviceshouldbedeployed.Inmostcases,however,ifserverssharethesamephysicallocation,wearenotinterestedonwhichparticularmachinetheserviceisdeployed.Allweneedistohaveitaccessibleandreplicatedinmanyinstances.Howcanweconfigureasetofmachinestoworktogethersothataddinganewonewouldrequirenoadditionalsetup?Thisistheroleofclustering.
Inthissection,youwillbeintroducedtotheconceptofserverclusteringandtheDockerSwarmtoolkit.
IntroducingserverclusteringAserverclusterisasetofconnectedcomputersthatworktogetherinawaythattheycanbeusedsimilarlytoasinglesystem.Serversareusuallyconnectedthroughthelocalnetworkwithaconnectionfastenoughtoensureasmallinfluenceofthefactthatservicesaredistributed.Asimpleserverclusterispresentedinthefollowingimage:
Auseraccessestheclusterviaahostcalledthemanager,whoseinterfaceshouldbesimilartoausualDockerhost.Insidethecluster,therearemultipleworkernodesthatreceivetasks,executethem,andnotifythemanageroftheircurrentstate.Themanagerisresponsiblefortheorchestrationprocess,includingtaskdispatching,servicediscovery,loadbalancing,andworkerfailuredetection.
Themanagercanalsoexecutetasks,whichisthedefaultconfigurationinDockerSwarm.However,forlargeclusters,themanagershouldbeconfiguredformanagementpurposesonly.
IntroducingDockerSwarmDockerSwarmisanativeclusteringsystemforDockerthatturnsasetofDockerhostsintooneconsistentcluster,calledaswarm.Eachhostconnectedtotheswarmplaystheroleofamanageroraworker(theremustbeatleastonemanagerinacluster).Technically,thephysicallocationofthemachinesdoesnotmatter;however,it'sreasonabletohaveallDockerhostsinsideonelocalnetwork,otherwise,managingoperations(orreachingconsensusbetweenmultiplemanagers)cantakeasignificantamountoftime.
SinceDocker1.12,DockerSwarmisnativelyintegratedintoDockerEngineasswarmmode.Inolderversions,itwasnecessarytoruntheswarmcontaineroneachofthehoststoprovidetheclusteringfunctionality.
Regardingtheterminology,inswarmmode,arunningimageiscalledaservice,asopposedtoacontainer,whichisrunonasingleDockerhost.Oneservicerunsaspecifiednumberoftasks.Ataskisanatomicschedulingunitoftheswarmthatholdstheinformationaboutthecontainerandthecommandthatshouldberuninsidethecontainer.Areplicaiseachcontainerthatisrunonthenode.Thenumberofreplicasistheexpectednumberofallcontainersforthegivenservice.
Let'slookatanimagepresentingtheterminologyandtheDockerSwarmclusteringprocess:
Westartbyspecifyingaservice,theDockerimageandthenumberofreplicas.Themanagerautomaticallyassignstaskstoworkernodes.Obviously,eachreplicatedcontainerisrunfromthesameDockerimage.Inthecontextofthepresentedflow,DockerSwarmcanbeviewedasalayerontopoftheDockerEnginemechanismthatisresponsibleforcontainerorchestration.
Intheprecedingsampleimage,wehavethreetasks,andeachofthemisrunonaseparateDockerhost.Nevertheless,itmayalsohappenthatallcontainerswouldbestartedonthesameDockerhost.Everythingdependsonthemanagernodethatallocatestaskstoworkernodesusingtheschedulingstrategy.Wewillshowhowtoconfigurethatstrategylater,inaseparatesection.
DockerSwarmfeaturesoverviewDockerSwarmprovidesanumberinterestingfeatures.Let'swalkthroughthemostimportantones:
Loadbalancing:DockerSwarmtakescareoftheloadbalancingandassigninguniqueDNSnamessothattheapplicationdeployedontheclustercanbeusedinthesamewayasdeployedonasingleDockerhost.Inotherwords,aswarmcanpublishportsinasimilarmannerastheDockercontainer,andthentheswarmmanagerdistributesrequestsamongtheservicesinthecluster.Dynamicrolemanagement:Dockerhostscanbeaddedtotheswarmatruntime,sothereisnoneedforaclusterrestart.What'smore,theroleofthenode(managerorworker)canalsobedynamicallychanged.Dynamicservicescaling:EachservicecanbedynamicallyscaledupordownwiththeDockerclient.Themanagernodetakescareofaddingorremovingcontainersfromthenodes.Failurerecovery:Nodesareconstantlymonitoredbythemanagerand,ifanyofthemfails,newtasksarestartedondifferentmachinessothatthedeclarednumberofreplicaswouldbeconstant.It'salsopossibletocreatemultiplemanagernodesinordertopreventabreakdownincaseoneofthemfails.Rollingupdates:Anupdatetoservicescanbeappliedincrementally;forexample,ifwehave10replicasandwewouldliketomakeachange,wecandefineadelaybetweenthedeploymenttoeachreplica.Insuchacase,whenanythinggoeswrong,weneverendupwithascenariowherenoreplicaisworkingcorrectly.Twoservicemodes:Therearetwomodesinwhichcanberun:
Replicatedservices:ThespecifiednumberofreplicatedcontainersaredistributedamongthenodesbasedontheschedulingstrategyalgorithmGlobalservices:Onecontainerisrunoneveryavailablenodeinthecluster
Security:AseverythingisinDocker,DockerSwarmenforcestheTLS
authenticationandthecommunicationencryption.It'salsopossibletouseCA(orself-signed)certificates.
Let'sseehowallofthislooksinpractice.
DockerSwarminpracticeDockerEngineincludestheSwarmmodebydefault,sothereisnoadditionalinstallationprocessrequired.SinceDockerSwarmisanativeDockerclusteringsystem,managingclusternodesisdonebythedockercommandandisthereforeverysimpleandintuitive.Let'sstartbycreatingamanagernodewithtwoworkernodes.Then,wewillrunandscaleaservicefromaDockerimage.
SettingupaSwarmInordertosetupaSwarm,weneedtoinitializethemanagernode.Wecandothisusingthefollowingcommandonamachinethatissupposedtobecomethemanager:
$dockerswarminit
Swarminitialized:currentnode(qfqzhk2bumhd2h0ckntrysm8l)isnowamanager.
Toaddaworkertothisswarm,runthefollowingcommand:
dockerswarmjoin\
--tokenSWMTKN-1-253vezc1pqqgb93c5huc9g3n0hj4p7xik1ziz5c4rsdo3f7iw2-df098e2jpe8uvwe2ohhhcxd6w\
192.168.0.143:2377
Toaddamanagertothisswarm,run'dockerswarmjoin-tokenmanager'andfollowtheinstructions.
Averycommonpracticeistousethe--advertise-addr<manager_ip>parameter,becauseifthemanagermachinehasmorethanonepotentialnetworkinterfaces,thendockerswarminitcanfail.
Inourcase,themanagermachinehastheIPaddress192.168.0.143and,obviously,ithastobereachablefromtheworkernodes(andviceversa).Notethatthecommandtoexecuteonworkermachineswasprintedtotheconsole.Alsonotethataspecialtokenhasbeengenerated.Fromnowon,itwillbeusedtoconnectamachinetotheclusterandshouldbekeptsecret.
WecancheckthattheSwarmhasbeencreatedusingthedockernodecommand:
$dockernodels
IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS
qfqzhk2bumhd2h0ckntrysm8l*ubuntu-managerReadyActiveLeader
Whenthemanagerisupandrunning,wearereadytoaddworkernodestotheSwarm.
AddingworkernodesInordertoaddamachinetotheSwarm,wehavetologintothegivenmachineandexecutethefollowingcommand:
$dockerswarmjoin\
--tokenSWMTKN-1-253vezc1pqqgb93c5huc9g3n0hj4p7xik1ziz5c4rsdo3f7iw2-df098e2jpe8uvwe2ohhhcxd6w\
192.168.0.143:2377
Thisnodejoinedaswarmasaworker.
WecancheckthatthenodehasbeenaddedtotheSwarmwiththedockernodelscommand.Assumingthatwe'veaddedtwonodemachines,theoutputshouldlookasfollows:
$dockernodels
IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS
cr7vin5xzu0331fvxkdxla22nubuntu-worker2ReadyActive
md4wx15t87nn0c3pyv24kewtzubuntu-worker1ReadyActive
qfqzhk2bumhd2h0ckntrysm8l*ubuntu-managerReadyActiveLeader
Atthispoint,wehaveaclusterthatconsistsofthreeDockerhosts,ubuntu-manager,ubuntu-worker1,andubuntu-worker2.Let'sseehowwecanrunaserviceonthiscluster.
DeployingaserviceInordertorunanimageonacluster,wedon'tusedockerrunbuttheSwarm-dedicateddockerservicecommand(whichisexecutedonthemanagernode).Let'sstartasingletomcatapplicationandgiveitthenametomcat:$dockerservicecreate--replicas1--nametomcattomcat
Thecommandcreatedtheserviceandthereforesentatasktostartacontainerononeofthenodes.Let'slisttherunningservices:
$dockerservicels
IDNAMEMODEREPLICASIMAGE
x65aeojumj05tomcatreplicated1/1tomcat:latest
Thelogconfirmsthatthetomcatserviceisrunning,andithasonereplica(oneDockercontainerisrunning).Wecanexaminetheserviceevenmoreclosely:
$dockerservicepstomcat
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
kjy1udwcnwmitomcat.1tomcat:latestubuntu-managerRunningRunningaboutaminuteago
Ifyouareinterestedinthedetailedinformationaboutaservice,youcanusethedockerserviceinspect<service_name>command.
Fromtheconsoleoutput,wecanseethatthecontainerisrunningonthemanagernode(ubuntu-manager).Itcouldhavebeenstartedonanyothernodeaswell;themanagerautomaticallychoosestheworkernodeusingtheschedulingstrategyalgorithm.Wecanconfirmthatthecontainerisrunningusingthewell-knowndockerpscommand:$dockerpsCONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES6718d0bcba98tomcat@sha256:88483873b279aaea5ced002c98dde04555584b66de29797a4476d5e94874e6de"catalina.shrun"AboutaminuteagoUpAboutaminute8080/tcptomcat.1.kjy1udwcnwmiosiw2qn71nt1r
Ifwedon'twantatasktobeexecutedonthemanagernode,wecanconstraintheservicewiththe--constraintnode.role==workeroption.
Theotherpossibilityistodisablethemanagercompletelyfromexecutingtaskswithdockernodeupdate--availabilitydrain<manager_name>.
ScalingserviceWhentheserviceisrunning,wecanscaleitupordownsothatitwillberunninginmanyreplicas:
$dockerservicescaletomcat=5
tomcatscaledto5
Wecancheckthattheservicehasbeenscaled:
$dockerservicepstomcat
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
kjy1udwcnwmitomcat.1tomcat:latestubuntu-managerRunningRunning2minutesago
536p5zc3kaxztomcat.2tomcat:latestubuntu-worker2RunningPreparing18secondsagonpt6ui1g9bdptomcat.3tomcat:latestubuntu-managerRunningRunning18secondsagozo2kger1rmqctomcat.4tomcat:latestubuntu-worker1RunningPreparing18secondsago1fb24nf94488tomcat.5tomcat:latestubuntu-worker2RunningPreparing18secondsago
Notethatthistime,twocontainersarerunningonthemanagernode,oneontheubuntu-worker1node,andtheotherontheubuntu-worker2node.Wecancheckthattheyarereallyrunningbyexecutingdockerpsoneachofthemachines.
Ifwewanttoremovetheservices,it'senoughtoexecutethefollowingcommand:
$dockerservicermtomcat
Youcancheckwiththedockerservicelscommandthattheservicehasbeenremoved,andthereforeallrelatedtomcatcontainerswerestoppedandremovedfromallthenodes.
PublishingportsDockerservices,similartothecontainers,haveaportforwardingmechanism.Weuseitbyaddingthe-p<host_port>:<container:port>parameter.Here'swhatstartingaservicecouldlooklike:$dockerservicecreate--replicas1--publish8080:8080--nametomcattomcat
Now,wecanopenabrowserandseetheTomcat'smainpageundertheaddresshttp://192.168.0.143:8080/.
Theapplicationisavailableonthemanagerhostthatactsasaloadbalanceranddistributesrequeststoworkernodes.WhatmaysoundalittlelessintuitiveisthefactthatwecanaccessTomcatusingtheIPaddressofanyworker,forexample,ifworkernodesareavailableunder192.168.0.166and192.168.0.115,wecanaccessthesamerunningcontainerwithhttp://192.168.0.166:8080/andhttp://192.168.0.115:8080/.ThisispossiblebecauseDockerSwarmcreatesaroutingmesh,inwhicheachnodehastheinformationhowtoforwardthepublishedport.
YoucanreadmoreabouthowtheloadbalancingandroutingaredonebyDockerSwarmathttps://docs.docker.com/engine/swarm/ingress/.
Bydefault,theinternalDockerSwarmloadbalancingisused.Therefore,it'senoughtoaddressallrequeststothemanager'smachine,anditwilltakecareofitsdistributionbetweennodes.Theotheroptionistoconfigureanexternalloadbalancer(forexample,HAProxyorTraefik).
WehavediscussedthebasicusageofDockerSwarm.Let'snowdiveintomorechallengingfeatures.
AdvancedDockerSwarmDockerSwarmoffersalotofinterestingfeaturesthatareusefulintheContinuousDeliveryprocess.Inthissection,wewillwalkthroughthemostimportantones.
RollingupdatesImagineyoudeployanewversionofyourapplication.Youneedtoupdateallreplicasinthecluster.OneoptionwouldbetostopthewholeDockerSwarmserviceandtorunanewonefromtheupdatedDockerimage.Suchapproach,however,causesdowntimebetweenthemomentwhentheserviceisstoppedandthemomentwhenthenewoneisstarted.IntheContinuousDeliveryprocess,downtimeisnotacceptable,sincethedeploymentcantakeplaceaftereverysourcecodechange,whichissimplyoften.Then,howcanweprovidezero-downtimedeploymentinacluster?Thisistheroleofrollingupdates.
Arollingupdateisanautomaticmethodforreplacingaservice,replicabyareplica,inawaythatsomeofthereplicasareworkingallthetime.DockerSwarmusesrollingupdatesbydefault,andtheycanbesteeredwithtwoparameters:
update-delay:Delaybetweenstartingonereplicaandstoppingthenextone(0secondsbydefault)update-parallelism:Maximumnumberofreplicasupdatedatthesametime(onebydefault)
TheDockerSwarmrollingupdateprocesslooksasfollows:
1. Stopthe<update-parallelism>numberoftasks(replicas).2. Intheirplace,runthesamenumberofupdatedtasks.3. IfataskreturnstheRUNNINGstate,thenwaitforthe<update-delay>period.4. If,atanytime,anytaskreturnstheFAILEDstate,thenpausetheupdate.
Thevalueoftheupdate-parallelismparametershouldbeadaptedtothenumberofreplicaswerun.Ifthenumberissmallandbootingtheserviceisfast,it'sreasonabletokeepthedefaultvalueof1.Theupdate-delayparametershouldbesettotheperiodlongerthantheexpectedboottimeofourapplicationsothatwewillnoticethefailure,andthereforepausetheupdate.
Let'slookatanexampleandchangetheTomcatapplicationfromversion8to
version9.Supposewehavethetomcat:8servicewithfivereplicas:
$dockerservicecreate--replicas5--nametomcat--update-delay10stomcat:8
Wecancheckthatallreplicasarerunningwiththedockerservicepstomcatcommand.Anotherusefulcommandthathelpsexaminetheserviceisthedockerserviceinspectcommand:
$dockerserviceinspect--prettytomcat
ID:au1nu396jzdewyq2y8enm0b6i
Name:tomcat
ServiceMode:Replicated
Replicas:5
Placement:
UpdateConfig:
Parallelism:1
Delay:10s
Onfailure:pause
Maxfailureratio:0
ContainerSpec:
Image:tomcat:8@sha256:835b6501c150de39d2b12569fd8124eaebc53a899e2540549b6b6f8676538484
Resources:
EndpointMode:vip
Wecanseethattheservicehasfivereplicascreatedoutoftheimagetomcat:8.Thecommandoutputalsoincludestheinformationabouttheparallelismandthedelaytimebetweenupdates(assetbytheoptionsinthedockerservicecreatecommand).
Now,wecanupdatetheserviceintothetomcat:9image:
$dockerserviceupdate--imagetomcat:9tomcat
Let'scheckwhathappens:
$dockerservicepstomcat
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
4dvh6ytn4lsqtomcat.1tomcat:8ubuntu-managerRunningRunning4minutesago
2mop96j5q4ajtomcat.2tomcat:8ubuntu-managerRunningRunning4minutesago
owurmusr1c48tomcat.3tomcat:9ubuntu-managerRunningPreparing13secondsago
r9drfjpizuxf\_tomcat.3tomcat:8ubuntu-managerShutdownShutdown12secondsago
0725ha5d8p4vtomcat.4tomcat:8ubuntu-managerRunningRunning4minutesago
wl25m2vrqgc4tomcat.5tomcat:8ubuntu-managerRunningRunning4minutesago
Notethatthefirstreplicaoftomcat:8hasbeenshutdownandthefirsttomcat:9isalreadyrunning.Ifwekeptoncheckingtheoutputofthedockerservicepstomcatcommand,wewouldnoticethatafterevery10seconds,anotherreplicaisintheshutdownstateandanewoneisstarted.Ifwealsomonitoredthedockerinspect
command,wewouldseethatthevalueUpdateStatus:Statewillchangetoupdatingandthen,whentheupdateisdone,tocompleted.
ArollingupdateisaverypowerfulfeaturethatallowszerodowntimedeploymentanditshouldalwaysbeusedintheContinuousDeliveryprocess.
DrainingnodesWhenweneedtostopaworkernodeformaintenance,orwewouldjustliketoremoveitfromthecluster,wecanusetheSwarmdrainingnodefeature.Drainingthenodemeansaskingthemanagertomovealltasksoutofagivennodeandexcludeitfromreceivingnewtasks.Asaresult,allreplicasarerunningonlyontheactivenodesandthedrainednodesareidle.
Let'slookhowthisworksinpractice.SupposewehavethreeclusternodesandaTomcatservicewithfivereplicas:
$dockernodels
IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS
4mrrmibdrpa3yethhmy13mwzqubuntu-worker2ReadyActive
kzgm7erw73tu2rjjninxdb4wp*ubuntu-managerReadyActiveLeader
yllusy42jp08w8fmze43rmqqsubuntu-worker1ReadyActive
$dockerservicecreate--replicas5--nametomcattomcat
Let'scheckonwhichnodesthereplicasarerunning:
$dockerservicepstomcat
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
zrnawwpupuqltomcat.1tomcat:latestubuntu-managerRunningRunning17minutesago
x6rqhyn7mrottomcat.2tomcat:latestubuntu-worker1RunningRunning16minutesago
rspgxcfv3is2tomcat.3tomcat:latestubuntu-worker2RunningRunning5weeksago
cf00k61vo7xhtomcat.4tomcat:latestubuntu-managerRunningRunning17minutesago
otjo08e06qbxtomcat.5tomcat:latestubuntu-worker2RunningRunning5weeksago
Therearetworeplicasrunningontheubuntu-worker2node.Let'sdrainthatnode:
$dockernodeupdate--availabilitydrainubuntu-worker2
Thenodeisputintothedrainavailability,soallreplicasshouldbemovedoutofit:
$dockerservicepstomcat
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
zrnawwpupuqltomcat.1tomcat:latestubuntu-managerRunningRunning18minutesago
x6rqhyn7mrottomcat.2tomcat:latestubuntu-worker1RunningRunning17minutesagoqrptjztd777itomcat.3tomcat:latestubuntu-worker1RunningRunninglessthanasecondago
rspgxcfv3is2\_tomcat.3tomcat:latestubuntu-worker2ShutdownShutdownlessthanasecondago
cf00k61vo7xhtomcat.4tomcat:latestubuntu-managerRunningRunning18minutesagok4c14tyo7leqtomcat.5tomcat:latestubuntu-worker1RunningRunninglessthanasecondago
otjo08e06qbx\_tomcat.5tomcat:latestubuntu-worker2ShutdownShutdownlessthanasecondago
Wecanseethatnewtaskswerestartedontheubuntu-worker1nodeandtheold
replicaswereshutdown.Wecancheckthestatusofthenodes:
$dockernodels
IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS
4mrrmibdrpa3yethhmy13mwzqubuntu-worker2ReadyDrain
kzgm7erw73tu2rjjninxdb4wp*ubuntu-managerReadyActiveLeader
yllusy42jp08w8fmze43rmqqsubuntu-worker1ReadyActive
Asexpected,theubuntu-worker2nodeisavailable(statusReady),butitsavailabilityissettodrain,whichmeansitdoesn'thostanytasks.Ifwewouldliketogetthenodeback,wecancheckitsavailabilitytoactive:
$dockernodeupdate--availabilityactiveubuntu-worker2
Averycommonpracticeistodrainthemanagernodeand,asaresult,itwillnotreceiveanytasks,butdomanagementworkonly.
Analternativemethodtodrainingthenodewouldbetoexecutethedockerswarmleavecommandfromtheworker.Thisapproach,however,hastwodrawbacks:
Foramoment,therearefewerreplicasthanexpected(afterleavingtheswarmandbeforethemasterstartsnewtasksonothernodes)Themasterdoesnotcontrolifthenodeisstillinthecluster
Forthesereasons,ifweplantostoptheworkerforsometimeandthengetitback,it'srecommendedtousethedrainingnodefeature.
MultiplemanagernodesHavingasinglemanagernodeisriskybecausewhenthemanagermachineisdown,thewholeclusterisdown.Thissituationis,obviously,notacceptableinthecaseofbusiness-criticalsystems.Inthissection,wepresenthowtomanagemultiplemasternodes.
Inordertoaddanewmanagernodetothesystem,weneedtofirstexecutethefollowingcommandonthe(currentlysingle)managernode:
$dockerswarmjoin-tokenmanager
Toaddamanagertothisswarm,runthefollowingcommand:
dockerswarmjoin\
--tokenSWMTKN-1-5blnptt38eh9d3s8lk8po3069vbjmz7k7r3falkm20y9v9hefx-a4v5olovq9mnvy7v8ppp63r23\
192.168.0.143:2377
Theoutputshowsthetokenandtheentirecommandthatneedstobeexecutedonthemachinethatissupposedtobecomethemanager.Afterexecutingit,weshouldseethatanewmanagerwasadded.
Anotheroptiontoaddamanageristopromoteitfromtheworkerroleusingthedockernodepromote<node>command.Inordertogetitbacktotheworkerrole,wecanusethedockernodedemote<node>command.
Supposewehaveaddedtwoadditionalmanagers;weshouldseethefollowingoutput:
$dockernodels
IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS
4mrrmibdrpa3yethhmy13mwzqubuntu-manager2ReadyActive
kzgm7erw73tu2rjjninxdb4wp*ubuntu-managerReadyActiveLeader
pkt4sjjsbxx4ly1lwetieuj2nubuntu-manager1ReadyActiveReachable
Notethatthenewmanagershavethemanagerstatussettoreachable(orleftempty),whiletheoldmanageristheleader.ThereasonforthisisthefactthatthereisalwaysoneprimarynoderesponsibleforallSwarmmanagementandorchestrationdecisions.TheleaderiselectedfromthemanagersusingtheRaftconsensusalgorithm,andwhenitisdown,anewleaderiselected.
Raftisaconsensusalgorithmthatisusedtomakedecisionsindistributedsystems.Youcanreadmoreabouthowitworks(andseeavisualization)athttps://raft.github.io/.AverypopularalternativealgorithmforthesamegoaliscalledPaxos.
Supposeweshutdowntheubuntu-managermachine;let'shavealookathowthenewleaderwaselected:
$dockernodels
IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS
4mrrmibdrpa3yethhmy13mwzqubuntu-manager2ReadyActiveReachable
kzgm7erw73tu2rjjninxdb4wpubuntu-managerReadyActiveUnreachable
pkt4sjjsbxx4ly1lwetieuj2n*ubuntu-manager1ReadyActiveLeader
Notethatevenwhenoneofthemanagersisdown,theswarmcanworkcorrectly.
Thereisnolimitforthenumberofmanagers,soitmaysoundthatthemoremanagersthebetterfaulttolerance.It'strue,however,havingalotofmanagershasanimpactontheperformancebecauseallSwarm-state-relateddecisions(forexample,addinganewnodeorleaderelection)havetobeagreedbetweenallmanagersusingtheRaftalgorithm.Asaresult,thenumberofmanagersisalwaysatradeoffbetweenthefaulttoleranceandtheperformance.
TheRaftalgorithmitselfhasaconstraintonthenumberofmanagers.Distributeddecisionshavetobeapprovedbythemajorityofnodes,calledaquorum.Thisfactimpliesthatanoddnumberofmanagersisrecommended.
Tounderstandwhy,let'sseewhatwouldhappenifwehadtwomanagers.Inthatcase,themajorityistwo,soifanyofthemanagersisdown,thenit'snotpossibletoreachthequorumandthereforeelecttheleader.Asaresult,losingonemachinemakesthewholeclusteroutoforder.Weaddedamanager,butthewholeclusterbecamelessfaulttolerant.Thesituationwouldbedifferentinthecaseofthreemanagers.Then,themajorityisstilltwo,solosingonemanagerdoesnotstopthewholecluster.Thisisthefactthateventhoughit'snottechnicallyforbidden,onlyoddnumberofmanagersmakessense.
Themorethemanagersinthecluster,themoretheRaft-relatedoperationsareinvolved.Then,themanagernodesshouldbeputintothedrainavailabilityinordertosavetheirresources.
SchedulingstrategySofar,wehavelearnedthatthemanagerautomaticallyassignsaworkernodetoatask.Inthissection,wedivedeeperintowhatautomaticallymeans.WepresenttheDockerSwarmschedulingstrategyandawaytoconfigureitaccordingtoourneeds.
DockerSwarmusestwocriteriaforchoosingtherightworkernode:
Resourceavailability:Schedulerisawareoftheresourcesavailableonnodes.Itusestheso-calledspreadstrategythatattemptstoschedulethetaskontheleastloadednode,provideditmeetsthecriteriaspecifiedbylabelsandconstraints.Labelsandconstraints:
Labelisanattributeofanode.Somelabelsareassignedautomatically,forexample,node.idornode.hostname;otherscanbedefinedbytheclusteradmin,forexample,node.labels.segmentConstraintisarestrictionappliedbythecreatoroftheservice,forexample,choosingonlynodeswiththegivenlabel
Labelsaredividedintotwocategories,node.labelsandengine.labels.Thefirstoneisaddedbytheoperationalteam;thesecondoneiscollectedbyDockerEngine,forexample,operatingsystemorhardwarespecifics.
Asanexample,ifwewouldliketoruntheTomcatserviceontheconcretenode,ubuntu-worker1,thenweneedtousethefollowingcommand:
$dockerservicecreate--constraint'node.hostname==ubuntu-worker1'tomcat
Wecanalsoaddacustomlabeltothenode:
$dockernodeupdate--label-addsegment=AAubuntu-worker1
Theprecedingcommandaddedalabel,node.labels.segment,withthevalueAA.Then,wecanuseitwhilerunningtheservice:
$dockerservicecreate--constraint'node.labels.segment==AA'tomcat
Thiscommandrunsthetomcatreplicasonlyonthenodesthatarelabeledwiththegivensegment,AA.
Labelsandconstraintsgiveustheflexibilitytoconfigurethenodesonwhichservicereplicaswouldberun.Thisapproach,eventhoughvalidinmanycases,shouldnotbeoverused,sinceit'sbesttokeepthereplicasdistributedonmultiplenodesandletDockerSwarmtakecareoftherightschedulingprocess.
DockerComposewithDockerSwarmWehavedescribedhowtouseDockerSwarminordertodeployaservice,whichinturnrunsmultiplecontainersfromthegivenDockerimage.Ontheotherhand,thereisDockerCompose,whichprovidesamethodtodefinethedependenciesbetweencontainersandenablesscalingcontainers,buteverythingisdonewithinoneDockerhost.Howdowemergebothtechnologiessothatwecanspecifythedocker-compose.ymlfileandautomaticallydistributethecontainersonacluster?Luckily,thereisDockerStack.
IntroducingDockerStackDockerStackisamethodtorunmultiple-linkedcontainersonaSwarmcluster.TounderstandbetterhowitlinksDockerComposewithDockerSwarm,let'stakealookatthefollowingfigure:
DockerSwarmorchestrateswhichcontainerisrunonwhichphysicalmachine.Thecontainers,however,don'thaveanydependenciesbetweenthemselves,soinorderforthemtocommunicate,wewouldneedtolinkthemmanually.DockerCompose,incontrast,provideslinkingbetweenthecontainers.Intheexamplefromtheprecedingfigure,oneDockerimage(deployedinthreereplicatedcontainers)dependsonanotherDockerimage(deployedasonecontainer).Allcontainers,however,runonthesameDockerhost,sothehorizontalscalingislimitedtotheresourcesofonemachine.DockerStackconnectsbothtechnologiesandallowsusingthedocker-compose.ymlfiletorunthecompleteenvironmentoflinkedcontainersdeployedonaclusterofDockerhosts.
UsingDockerStackAsanexample,let'susethecalculatorimagethatdependsontheredisimage.Let'ssplittheprocessintofoursteps:
1. Specifyingdocker-compose.yml.2. RunningtheDockerStackcommand.3. Verifyingtheservicesandcontainers.4. Removingthestack.
Specifyingdocker-compose.ymlWealreadydefinedthedocker-compose.ymlfileinthepreviouschaptersanditlookedsimilartothefollowingone:
version:"3"
services:
calculator:
deploy:
replicas:3
image:leszko/calculator:latest
ports:
-"8881:8080"
redis:
deploy:
replicas:1
image:redis:latest
Notethatallimagesmustbepushedtotheregistrybeforerunningthedockerstackcommandsothattheywouldbeaccessiblefromallnodes.Itisthereforenotpossibletobuildimagesinsidedocker-compose.yml.
Withthepresenteddocker-compose.ymlconfiguration,wewillrunthreecalculatorcontainersandonerediscontainer.Theendpointofthecalculatorservicewillbeexposedonport8881.
<strong>$dockerstackdeploy--compose-filedocker-compose.ymlapp</strong><br/><strong>Creatingnetworkapp_default</strong><br/><strong>Creatingserviceapp_redis</strong><br/><strong>Creatingserviceapp_calculator</strong>
Dockerplanstosimplifythesyntaxsothatthestackwordwouldnotbeneeded,forexample,dockerdeploy--compose-filedocker-compose.ymlapp.Atthetimeofwriting,it'sonlyavailableintheexperimentalversion.
VerifyingtheservicesandcontainersTheserviceshavestarted.Wecancheckthattheyarerunningwiththedockerservicelscommand:
$dockerservicels
IDNAMEMODEREPLICASIMAGE
5jbdzt9wolorapp_calculatorreplicated3/3leszko/calculator:latest
zrr4pkh3n13fapp_redisreplicated1/1redis:latest
WecanlookevencloserattheservicesandcheckonwhichDockerhoststheyhavebeendeployed:
$dockerservicepsapp_calculator
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
jx0ipdxwdilmapp_calculator.1leszko/calculator:latestubuntu-managerRunningRunning57secondsago
psweuemtb2wfapp_calculator.2leszko/calculator:latestubuntu-worker1RunningRunningaboutaminuteago
iuas0dmi7abnapp_calculator.3leszko/calculator:latestubuntu-worker2RunningRunning57secondsago
$dockerservicepsapp_redis
IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE
8sg1ybbggx3lapp_redis.1redis:latestubuntu-managerRunningRunningaboutaminuteago
Wecanseethatoneofthecalculatorcontainersandtherediscontainersarestartedontheubuntu-managermachine.Twoothercalculatorcontainersrunontheubuntu-worker1andubuntu-worker2machines.
Notethatweexplicitlyspecifiedtheportnumberunderwhichthecalculatorwebserviceshouldbepublished.Therefore,weareabletoaccesstheendpointviathemanager'sIPaddresshttp://192.168.0.143:8881/sum?a=1&b=2.Theoperationreturns3asaresultandcachesitintheRediscontainer.
RemovingthestackWhenwe'redonewiththestack,wecanremoveeverythingusingtheconvenientdockerstackrmcommand:
$dockerstackrmapp
Removingserviceapp_calculator
Removingserviceapp_redis
Removingnetworkapp_default
UsingDockerStackallowsrunningtheDockerComposespecificationontheDockerSwarmcluster.Notethatweusedtheexactdocker-compose.ymlformat,whichisagreatbenefit,becausethereisnoneedtospecifyanythingextrafortheSwarm.
ThemergerofbothtechnologiesgivesustherealpowerofdeployingapplicationsonDockerbecausewedon'tneedtothinkaboutindividualmachines.Allweneedtospecifyishowour(micro)servicesaredependentoneachother,expressitinthedocker-compose.ymlformat,andletDockertakecareofeverythingelse.Thephysicalmachinescanbethentreatedsimplyasasetofresources.
AlternativeclustermanagementsystemsDockerSwarmisnottheonlysystemforclusteringDockercontainers.Eventhoughit'stheoneavailableoutofthebox,theremaybesomevalidreasonstoinstallathird-partyclustermanager.Let'swalkthroughthemostpopularalternatives.
KubernetesKubernetesisanopensourceclustermanagementsystemoriginallydesignedbyGoogle.Eventhoughit'snotDocker-native,theintegrationissmooth,andtherearemanyadditionaltoolsthathelpwiththisprocess;forexample,komposecantranslatethedocker-compose.ymlfilesintoKubernetesconfigurationfiles.
Let'stakealookatthesimplifiedarchitectureofKubernetes:
KubernetesissimilartoDockerSwarminawaythatitalsohasmasterandworkernodes.Additionally,itintroducestheconceptofapodthatrepresentsagroupofcontainersdeployedandscheduledtogether.Mostpodshaveafewcontainersthatmakeupaservice.Podsaredynamicallybuiltandremoveddependingonthechangingrequirements.
Kubernetesisrelativelyyoung.Itsdevelopmentstartedin2014;however,it'sbasedonGoogle'sexperience,andthisisoneofthereasonswhyit'soneofthemostpopularclustermanagementsystemsavailableonthemarket.TherearemoreandmoreorganizationsthatmigratedtoKubernetes,suchaseBay,Wikipedia,andPearson.
ApacheMesosApacheMesosisanopensourceschedulingandclusteringsystemstartedattheUniversityofCalifornia,Berkeley,in2009,longbeforeDockeremerged.ItprovidesanabstractionlayeroverCPU,diskspace,andRAM.OneofthegreatadvantagesofMesosisthatitsupportsanyLinuxapplication,notnecessarily(Docker)containers.Thisiswhyit'spossibletocreateaclusteroutofthousandsofmachinesanduseitforbothDockercontainersandotherprograms,forexample,Hadoop-basedcalculations.
Let'slookatthefigurepresentingtheMesosarchitecture:
ApacheMesos,similartootherclusteringsystems,hasthemaster-slavearchitecture.Itusesnodeagentsinstalledoneverynodeforcommunication,anditprovidestwotypesofschedulers,Chronos-forcron-stylerepeatingtasksandMarathon-providingaRESTAPItoorchestrateservicesandcontainers.
ApacheMesosisverymaturecomparedtootherclusteringsystems,andithasbeenadoptedinalargenumberoforganizations,suchasTwitter,Uber,andCERN.
ComparingfeaturesKubernetes,DockerSwarm,andMesosareallgoodchoicesfortheclustermanagementsystem.Allofthemarefreeandopensource,andallofthemprovideimportantclustermanagementfeaturessuchasloadbalancing,servicediscovery,distributedstorage,failurerecovery,monitoring,secretmanagement,androllingupdates.AllofthemcanalsobeusedintheContinuousDeliveryprocesswithouthugedifferences.Thisisbecause,intheDockerizedinfrastructure,theyalladdressthesameissue,theclusteringofDockercontainers.Nevertheless,obviously,thesystemsarenotexactlythesame.Let'shavealookatatablepresentingthedifferences:
DockerSwarm Kubernetes ApacheMesos
Dockersupport Native
SupportsDockerasoneofthecontainertypesinthepod
Mesosagents(slaves)canbeconfiguredtohostDockercontainers
Applicationtypes
Dockerimages
Containerizedapplications(Docker,rkt,andhyper)
AnyapplicationthatcanberunonLinux(alsocontainers)
Applicationdefinition
DockerComposeconfiguration
Podsconfiguration,replicasets,replicationcontrollers,services,anddeployments
Applicationgroupsformedinthetreestructure
Setupprocess Verysimple
Dependingontheinfrastructure,itmayrequirerunningonecommandormanycomplexoperations
Fairlyinvolved,itrequiresconfiguringMesos,Marathon,Chronos,Zookeeper,andDockersupport
API DockerRESTAPI RESTAPI Chronos/Marathon
RESTAPI
UserInterface
Dockerconsoleclient,third-partywebapplications,suchasShipyard
Consoletools,nativeWebUI(KubernetesDashboard)
OfficialwebinterfacesforMesos,Marathon,andChronos
Cloudintegration
Manualinstallationrequired
Cloud-nativesupportfrommostproviders(Azure,AWS,GoogleCloud,andothers)
Supportfrommostcloudproviders
Maximumclustersize 1,000nodes 1,000nodes 50,000nodes
Provideshorizontalpod
Marathonprovidesautoscalingbasedonresource(CPU/Memory)
Autoscaling Notavailable autoscalingbasedontheobservedCPUusage
consumption,numberofrequestspersecond,andqueuelength
Obviously,apartfromDockerSwarm,Kubernetes,andApacheMesos,thereareotherclusteringsystemsavailableinthemarket.Theyare,however,notthatpopularandtheirusagedecreasesovertime.
Nomatterwhichsystemyouchoose,youcanuseitnotonlyforthestaging/productionenvironmentsbutalsotoscaleJenkinsagents.Let'shavealookathowtodoit.
ScalingJenkinsTheobvioususecasesforserverclusteringarethestagingandproductionenvironments.Whenused,it'senoughtoattachaphysicalmachineinordertoincreasethecapacityoftheenvironment.Nevertheless,inthecontextofContinuousDelivery,wemayalsowanttoimprovetheJenkinsinfrastructurebyrunningJenkinsagent(slave)nodesonacluster.Inthissection,wetakealookattwodifferentmethodstoachievethisgoal.
DynamicslaveprovisioningWesawdynamicslaveprovisioninginChapter3,ConfiguringJenkins.WithDockerSwarm,theideastaysexactlythesame.Whenthebuildisstarted,theJenkinsmasterrunsacontainerfromtheJenkinsslaveDockerimage,andtheJenkinsfilescriptisexecutedinsidethecontainer.DockerSwarm,however,makesthesolutionmorepowerfulsincewearenotlimitedtoasingleDockerhostmachinebutcanproviderealhorizontalscaling.AddinganewDockerhosttotheclustereffectivelyscalesupthecapacityoftheJenkinsinfrastructure.
Atthetimeofwriting,theJenkinsDockerplugindoesnotsupportDockerSwarm.OneofthesolutionsistouseKubernetesorMesosastheclustermanagementsystem.EachofthemhasadedicatedJenkinsplugin:KubernetesPlugin(https://wiki.jenkins.io/display/JENKINS/Kubernetes+Plugin)andMesosPlugin(https://wiki.jenkins.io/display/JENKINS/Mesos+Plugin).
Nomatterhowtheslavesareprovisioned,wealwaysconfigurethembyinstallingtheappropriatepluginandaddingtheentrytotheCloudsectioninManageJenkins|ConfigureSystem.
JenkinsSwarmIfwedon'twanttousethedynamicslaveprovisioning,thenanothersolutionforclusteringJenkinsslavesistouseJenkinsSwarm.WedescribedhowtouseitinChapter3,ConfiguringJenkins.Here,weaddthedescriptionforDockerSwarm.
First,let'shavealookathowtoruntheJenkinsSwarmslaveusingtheDockerimagebuiltfromtheswarm-client.jartool.ThereareafewofthemavailableonDockerHub;wecanusethecsanchez/jenkins-swarm-slaveimage:
$dockerruncsanchez/jenkins-swarm-slave:1.16-master-username-password-namejenkins-swarm-slave-2
ThiscommandexecutionshouldhaveexactlythesameeffectastheonpresentedinChapter3,ConfiguringJenkins;itdynamicallyaddsaslavetotheJenkinsmaster.
Then,togetthemostofJenkinsSwarm,wecanruntheslavecontainersontheDockerSwarmcluster:
$dockerservicecreate--replicas5--namejenkins-swarm-slavecsanchez/jenkins-swarm-slave-master-disableSslVerification-username-password-namejenkins-swarm-slave
TheprecedingcommandstartsfiveslavesontheclusterandattachesthemtotheJenkinsmaster.PleasenotethatitisverysimpletoscaleJenkinshorizontallybyexecutingthedockerservicescalecommand.
ComparisonofdynamicslaveprovisioningandJenkinsSwarmDynamicslaveprovisioningandJenkinsSwarmcanbebothrunonaclusterthatresultsinthearchitecturepresentedinthefollowingdiagram:
Jenkinsslavesarerunontheclusterandthereforeareveryeasilyscaledupanddown.IfweneedmoreJenkinsresources,wescaleupJenkinsslaves.Ifweneedmoreclusterresources,weaddmorephysicalmachinestothecluster.
ThedifferencebetweenthetwosolutionsisthatthedynamicslaveprovisioningautomaticallyaddsaJenkinsslavetotheclusterbeforeeachbuild.Thebenefitofsuchapproachisthatwedon'tevenhavetothinkabouthowmanyJenkinsslavesshouldberunningatthemomentsincethenumberautomaticallyadaptstothenumberofpipelinebuilds.Thisiswhy,inmostcases,thedynamicslaveprovisioningisthefirstchoice.Nevertheless,JenkinsSwarmalsocarriesafewsignificantbenefits:
Controlofthenumberofslaves:UsingJenkinsSwarm,weexplicitlydecidehowmanyJenkinsslavesshouldberunningatthemoment.Statefulslaves:ManybuildssharethesameJenkinsslave,whichmaysoundlikeadrawback;however,itbecomesanadvantagewhenabuildrequiresdownloadingalotofdependentlibrariesfromtheinternet.Inthe
caseofdynamicslaveprovisioning,tocachethedependencies,wewouldneedtosetupasharedvolume.Controlofwheretheslavesarerunning:UsingJenkinsSwarm,wecandecidenottorunslavesontheclusterbuttochoosethehostmachinedynamically;forexample,formanystartups,whentheclusterinfrastructureiscostly,slavescanbedynamicallyrunonthelaptopofadeveloperwhoisstartingthebuild.
ClusteringJenkinsslavesbringalotofbenefitsanditiswhatthemodernJenkinsarchitectureshouldlooklike.Thisway,wecanprovidethedynamichorizontalscalingoftheinfrastructurefortheContinuousDeliveryprocess.
ExercisesInthischapter,wehavecoveredDockerSwarmandtheclusteringprocessindetail.Inordertoenhancethisknowledge,werecommendthefollowingexercises:
1. Setupaswarmclusterthatconsistsofthreenodes:
UseonemachineasthemanagernodeandtwomachinesasworkernodesYoucanusephysicalmachinesconnectedtoonenetwork,machinesfromthecloudprovider,orVirtualBoxmachineswiththesharednetworkCheckthattheclusterisconfiguredproperlyusingthedockernodecommand
2. Run/scaleahelloworldserviceonthecluster:
TheservicecanlookexactlythesameasdescribedintheexercisesforChapter2,IntroducingDockerPublishtheportsothatitwillbeaccessiblefromoutsideoftheclusterScaletheservicetofivereplicasMakearequesttothe"helloworld"serviceandcheckwhichofthecontainersisservingtherequest
3. ScaleJenkinsusingslavesdeployedontheSwarmcluster:
UseJenkinsSwarmordynamicslaveprovisioningRunapipelinebuildandcheckthatitisexecutedononeoftheclusteredslaves
SummaryInthischapter,wetookalookattheclusteringmethodsforDockerenvironmentsthatenablesettingupthecompletestaging/production/Jenkinsenvironment.Herearethekeytakeawaysfromthechapter:
Clusteringisamethodofconfiguringasetofmachinesinawaythat,inmanyrespects,canbeviewedasasinglesystemDockerSwarmisthenativeclusteringsystemforDockerDockerSwarmclusterscanbedynamicallyconfiguredusingbuilt-inDockercommandsDockerimagescanberunandscaledontheclusterusingthedockerservicecommandDockerStackisamethodtoruntheDockerComposeconfigurationonaSwarmclusterThemostpopularclusteringsystemsthatsupportDockerareDockerSwarm,Kubernetes,andApacheMesosJenkinsagentscanberunonaclusterusingthedynamicslaveprovisioningortheJenkinsSwarmplugin
Inthenextchapter,wewilldescribethemoreadvancedaspectsoftheContinuousDeliveryprocessandpresentthebestpracticesforbuildingpipelines
AdvancedContinuousDelivery
Inthelastchapter,wecoveredhowserverclusteringworksandhowwecanuseittogetherwithDockerandJenkins.Inthischapter,wewillseeamixtureofdifferentaspectsthatareveryimportantintheContinuousDeliveryprocessbuthavenotbeendescribedyet.
Thischaptercoversthefollowingpoints:
ExplaininghowtoapproachdatabasechangesinthecontextofContinuousDeliveryIntroducingtheideaofdatabasemigrationandrelatedtoolsExploringdifferentapproachestobackwards-compatibleandbackwards-incompatibledatabaseupdatesUsingparallelstepsintheJenkinspipelineCreatingaJenkinssharedlibraryPresentingawaytorollbackproductionchangesIntroducingContinuousDeliveryforlegacysystemsExploringhowtopreparezero-downtimedeploymentsPresentingContinuousDeliverybestpractices
ManagingdatabasechangesSofar,wehavefocusedontheContinuousDeliveryprocess,whichwasappliedtoawebservice.Asimplepartofthiswasthatwebservicesareinherentlystateless.Thisfactmeansthattheycanbeeasilyupdated,restarted,clonedinmanyinstances,andrecreatedfromthegivensourcecode.Awebservice,however,isusuallylinkedtoitsstatefulpart,adatabasethatposesnewchallengestothedeliveryprocess.Thesechallengescanbegroupedintothefollowingcategories:
Compatibility:ThedatabaseschemaandthedataitselfmustbecompatiblewiththewebserviceallthetimeZero-downtimedeployment:Inordertoachievezero-downtimedeployment,weuserollingupdates,whichmeansthatadatabasemustbecompatiblewithtwodifferentwebserviceversionsatthesametimeRollback:Arollbackofadatabasecanbedifficult,limited,orsometimesevenimpossiblebecausenotalloperationsarereversible(forexample,removingacolumnthatcontainsdata)Testdata:Database-relatedchangesaredifficulttotestbecauseweneedtestdatathatisverysimilartoproduction
Inthissection,IwillexplainhowtoaddressthesechallengessothattheContinuousDeliveryprocesswillbeassafeaspossible.
UnderstandingschemaupdatesIfyouthinkaboutthedeliveryprocess,it'snotreallythedataitselfthatcausedifficultiesbecausewedon'tusuallychangethedatawhenwedeployanapplication.Thedataissomethingthatiscollectedwhilethesystemisliveintheproduction;whereas,duringdeployment,weonlychangethewaywestoreandinterpretthisdata.Inotherwords,inthecontextoftheContinuousDeliveryprocess,weareinterestedinthestructureofthedatabase,notexactlyinitscontent.Thisiswhythissectionconcernsmainlyrelationaldatabases(andtheirschemas)andfocuseslessonothertypesofstoragesuchasNoSQLdatabases,wherethereisnostructuredefinition.
Tounderstandthisbetter,wethinkofRedis,whichwehavealreadyusedinthisbook.Itstoredthecacheddata,soeffectivelyitwasadatabase.Nevertheless,itrequiredzeroeffortfromtheContinuousDeliveryperspectivesinceitdidn'thaveanydatastructure.Allitstoredwasthekey-valueentries,whichdoesnotevolveovertime.
NoSQLdatabasesusuallydon'thaveanyrestrictingschemaandthereforesimplifytheContinuousDeliveryprocessbecausethereisnoadditionalschemaupdatesteprequired.Thisisahugebenefit;however,itdoesn'tnecessarilymeanthatwritingapplicationswithNoSQLdatabasesissimplerbecausewehaveputmoreeffortintodatavalidationinthesourcecode.
Relationaldatabaseshavestaticschemas.Ifwewouldliketochangeit,forexample,toaddanewcolumntothetable,weneedtowriteandexecuteaSQLDDL(datadefinitionlanguage)script.Doingthismanuallyforeverychangerequiresalotofworkandleadstoerror-pronesolutions,inwhichtheoperationsteamhastokeepinsyncthecodeandthedatabasestructure.Amuchbettersolutionistoautomaticallyupdatetheschemainanincrementalmanner.Suchasolutioniscalleddatabasemigration.
IntroducingdatabasemigrationsDatabaseschemamigrationisaprocessofincrementalchangestotherelationaldatabasestructure.Let'shavealookatthefollowingdiagramtounderstanditbetter:
Thedatabaseintheversionv1hastheschemadefinedbytheV1_init.sqlfile.Additionally,itstoresthemetadatarelatedtothemigrationprocess,forexample,itscurrentschemaversionandthemigrationchangelog.Whenwewanttoupdatetheschema,weprovidethechangesintheformofaSQLfile,suchasV2_add_table.sql.Then,weneedtorunthemigrationtoolthatexecutesthegivenSQLfileonthedatabase(italsoupdatesthemetatables).Ineffect,thedatabaseschemaisaresultofallsubsequentlyexecutedSQLmigrationscripts.Next,wewillseeanexampleofamigration.
Migrationscriptsshouldbestoredintheversioncontrolsystem,usuallyinthesamerepositoryasthesourcecode.
Migrationtoolsandthestrategiestheyusecanbedividedintotwocategories:
Upgradeanddowngrade:Thisapproach,forexample,usedbytheRuby
onRailsframework,meansthatwecanmigrateup(fromv1tov2)anddown(fromv2tov1).Itallowsthedatabaseschematorollback,whichmaysometimesendupindataloss(ifthemigrationislogicallyirreversible).Upgradeonly:Thisapproach,forexample,usedbytheFlywaytool,onlyallowsustomigrateup(fromv1tov2).Inmanycases,thedatabaseupdatesarenotreversible,forexample,removingatablefromthedatabase.Suchachangecannotberolledbackbecauseevenifwerecreatethetable,wehavealreadylostallthedata.
Therearemanydatabasemigrationtoolsavailableonthemarket,outofwhichthemostpopularareFlyway,Liquibase,andRailMigrations(fromtheRubyonRailsframework).Asanextsteptounderstandhowsuchtoolswork,wewillseeanexamplebasedontheFlywaytool.
Therearealsocommercialsolutionsprovidedfortheparticulardatabases,forexample,Redgate(forSQLServer)andOptimDatabaseAdministrator(forDB2).
UsingFlyway
Let'suseFlywaytocreateadatabaseschemaforthecalculatorwebservice.Thedatabasewillstorethehistoryofalloperationsthatwereexecutedontheservice:thefirstparameter,thesecondparameter,andtheresult.
WeshowhowtousetheSQLdatabaseandFlywayinthreesteps:
1. ConfiguringtheFlywaytooltoworktogetherwithGradle.2. DefiningtheSQLmigrationscripttocreatethecalculationhistorytable.3. UsingtheSQLdatabaseinsidetheSpringBootapplicationcode.
ConfiguringFlywayInordertouseFlywaytogetherwithGradle,weneedtoaddthefollowingcontenttothebuild.gradlefile:buildscript{dependencies{classpath('com.h2database:h2:1.4.191')}}…plugins{id"org.flywaydb.flyway"version"4.2.0"}…flyway{url='jdbc:h2:file:/tmp/calculator'user='sa'}
Here'saquickcommentontheconfiguration:
WeusedtheH2database,whichisanin-memory(andfile-based)databaseWestorethedatabaseinthe/tmp/calculatorfileThedefaultdatabaseuseriscalledsa(systemadministrator)
InthecaseofotherSQLdatabases(forexample,MySQL),theconfigurationwouldbeverysimilar.TheonlydifferenceisintheGradledependenciesandtheJDBCconnection.
Afterthisconfigurationisapplied,weshouldbeabletoruntheFlywaytoolbyexecutingthefollowingcommand:$./gradlewflywayMigrate-i
Thecommandcreatedthedatabaseinthefile/tmp/calculator.mv.db.Obviously,ithasnoschemasincewehaven'tdefinedanythingyet.
Flywaycanbeusedasacommand-linetool,viaJavaAPI,orasapluginforthepopularbuildingtoolsGradle,Maven,andAnt.
DefiningtheSQLmigrationscriptThenextstepistodefinetheSQLfilethataddsthecalculationtableintothedatabaseschema.Let'screatethesrc/main/resources/db/migration/V1__Create_calculation_table.sqlfilewiththefollowingcontent:
createtableCALCULATION(
IDintnotnullauto_increment,
Avarchar(100),
Bvarchar(100),
RESULTvarchar(100),
primarykey(ID)
);
Notethemigrationfilenamingconvention,<version>__<change_description>.sql.TheSQLfilecreatesatablewithfourcolumns,ID,A,B,RESULT.TheIDcolumnisanautomaticallyincrementedprimarykeyofthetable.Now,wearereadytoruntheFlywaycommandtoapplythemigration:
$./gradlewflywayMigrate-i
…
Successfullyapplied1migrationtoschema"PUBLIC"(executiontime00:00.028s).
:flywayMigrate(Thread[DaemonworkerThread2,5,main])completed.Took1.114secs.
Thecommandautomaticallydetectedthemigrationfileandexecuteditonthedatabase.
Themigrationfilesshouldbealwayskeptintheversioncontrolsystem,usually,togetherwiththesourcecode.
AccessingdatabaseWeexecutedourfirstmigration,sothedatabaseisprepared.Toseethecompleteexample,weshouldalsoadaptourprojectsothatitwouldaccessthedatabase.
Let'sfirstconfiguretheGradledependenciestousetheH2databasefromtheSpringBootproject.Wecandothisbyaddingthefollowinglinestothebuild.gradlefile:
dependencies{
compile("org.springframework.boot:spring-boot-starter-data-jpa")
compile("com.h2database:h2")
}
Thenextstepistosetupthedatabaselocationandthestartupbehaviorinthesrc/main/resources/application.propertiesfile:
spring.datasource.url=jdbc:h2:file:/tmp/calculator;DB_CLOSE_ON_EXIT=FALSE
spring.jpa.hibernate.ddl-auto=validate
ThesecondlinemeansthatSpringBootwillnottrytoautomaticallygeneratethedatabaseschemafromthesourcecodemodel.Onthecontrary,itwillonlyvalidateifthedatabaseschemaisconsistentwiththeJavamodel.
Now,let'screatetheJavaORMentitymodelforthecalculationinthenewsrc/main/java/com/leszko/calculator/Calculation.javafile:
packagecom.leszko.calculator;
importjavax.persistence.Entity;
importjavax.persistence.GeneratedValue;
importjavax.persistence.GenerationType;
importjavax.persistence.Id;
@Entity
publicclassCalculation{
@Id
@GeneratedValue(strategy=GenerationType.AUTO)
privateIntegerid;
privateStringa;
privateStringb;
privateStringresult;
protectedCalculation(){}
publicCalculation(Stringa,Stringb,Stringresult){
this.a=a;
this.b=b;
this.result=result;
}
}
TheentityclassrepresentsthedatabasemappingintheJavacode.Atableisexpressedasaclassandeachcolumnasafield.ThenextstepistocreatetherepositoryforloadingandstoringtheCalculationentities.
Let'screatethesrc/main/java/com/leszko/calculator/CalculationRepository.javafile:
packagecom.leszko.calculator;
importorg.springframework.data.repository.CrudRepository;
publicinterfaceCalculationRepositoryextendsCrudRepository<Calculation,Integer>{}
Finally,wecanusetheCalculationandCalculationRepositoryclassestostorethecalculationhistory.Let'saddthefollowingcodetothesrc/main/java/com/leszko/calculator/CalculatorController.javafile:
...
classCalculatorController{
...
@Autowired
privateCalculationRepositorycalculationRepository;
@RequestMapping("/sum")
Stringsum(@RequestParam("a")Integera,@RequestParam("b")Integerb){
Stringresult=String.valueOf(calculator.sum(a,b));
calculationRepository.save(newCalculation(a.toString(),b.toString(),result));
returnresult;
}
}
Now,whenwestarttheserviceandexecutethe/sumendpoint,eachsummingoperationisloggedintothedatabase.
Ifyouwouldliketobrowsethedatabasecontent,thenyoucanaddspring.h2.console.enabled=truetotheapplication.propertiesfile,andthenbrowsethedatabaseviathe/h2-consoleendpoint.
WeexplainedhowthedatabaseschemamigrationworksandhowtouseitinsideaSpringproject,builtwithGradle.Now,let'shavealookathowitintegrateswithintheContinuousDeliveryprocess.
ChangingdatabaseinContinuousDeliveryThefirstapproachtousedatabaseupdatesinsidetheContinuousDeliverypipelinecouldbetoaddastagewithinthemigrationcommandexecution.Thissimplesolutionwouldworkcorrectlyformanycases;however,ithastwosignificantdrawbacks:
Rollback:Asmentionedbefore,it'snotalwayspossibletorollbackthedatabasechange(Flywaydoesn'tsupportdowngradesatall).Therefore,inthecaseofservicerollback,thedatabasebecomesincompatible.Downtime:Theserviceupdateandthedatabaseupdatearenotexecutedexactlyatthesametime,whichcausesdowntime.
Thisleadsustotwoconstraintsthatwewillneedtoaddress:
ThedatabaseversionneedstobecompatiblewiththeserviceversionallthetimeThedatabaseschemamigrationisnotreversible
Wewilladdresstheseconstraintsfortwodifferentcases:backwards-compatibleupdatesandnon-backwards-compatibleupdates.
Backwards-compatiblechangesBackwards-compatiblechangesaresimpler.Let'slookatthefollowingfiguretoseehowtheywork:
SupposetheschemamigrationDatabasev10isbackwards-compatible.IfweneedtorollbacktheServicev1.2.8release,thenwedeployServicev1.2.7,andthereisnoneedtodoanythingwiththedatabase(databasemigrationsarenotreversible,sowekeepDatabasev11).Sincetheschemaupdateisbackwards-compatible,Servicev.1.2.7worksperfectlyfinewithDatabasev11.ThesameappliesifweneedtorollbacktoServicev1.2.6,andsoon.Now,supposeDatabasev10andallothermigrationsarebackwards-compatible,thenwecouldrollbacktoanyserviceversionandeverythingwouldworkcorrectly.
Thereisalsonoproblemwiththedowntime.Ifthedatabasemigrationiszero-downtimeitself,thenwecanexecuteitfirstandthenusetherollingupdatesfortheservice.
Let'slookatanexampleofabackwards-compatiblechange.Wewillcreateaschemaupdatethataddsacreated_atcolumntothecalculationtable.Themigrationfilesrc/main/resources/db/migration/V2__Add_created_at_column.sqllooksasfollows:
altertableCALCULATION
addCREATED_ATtimestamp;
Besidesthemigrationscript,thecalculatorservicerequiresanewfieldintheCalculationclass:
...
privateTimestampcreatedAt;
...
WealsoneedtoadjustitsconstructorandthenitsusageintheCalculatorControllerclass:
calculationRepository.save(newCalculation(a.toString(),b.toString(),result,Timestamp.from(Instant.now())));
Afterrunningtheservice,thecalculationhistoryisstoredtogetherwiththecreated_atcolumn.Notethatthechangeisbackwards-compatiblebecauseevenifwereverttheJavacodeandleavethecreated_atcolumninthedatabase,everythingwouldworkperfectlyfine(therevertedcodedoesnotaddressthenewcolumnatall).
Non-backwards-compatiblechangesNon-backwards-compatiblechangesarewaymoredifficult.Lookingatthepreviousfigure,ifdatabasechangev11wasbackwards-incompatible,itwouldbeimpossibletorollbacktheserviceto1.2.7.Inthiscase,howcanweapproachnon-backwards-compatibledatabasemigrationssothatrollbacksandzero-downtimedeploymentswouldbepossible?
Tomakealongstoryshort,wecanaddressthisissuebyconvertinganon-backwards-compatiblechangeintoachangethatisbackwards-compatibleforacertainperiodoftime.Inotherwords,weneedtoputintheextraeffortandsplittheschemamigrationintotwoparts:
Backwards-compatibleupdateexecutednow,whichusuallymeanskeepingsomeredundantdataNon-backwards-compatibleupdateexecutedaftertherollbackperiodtimethatdefineshowfarbackwecanrevertourcode
Toillustratethisbetter,let'slookatthefollowingimage:
Let'sthinkaboutanexampleofdroppingacolumn.Aproposedmethodwouldincludetwosteps:
1. Stopusingthecolumninthesourcecode(v1.2.5,backwards-compatibleupdate,executedfirst).
2. Dropthecolumnfromthedatabase(v11,non-backwards-compatibleupdate,executedaftertherollbackperiod).
AllserviceversionsuntilDatabasev11canberolledbacktoanypreviousversion,theservicesstartingfromServicev1.2.8canberolledbackonlywithintherollbackperiod.Suchapproachmaysoundtrivialbecauseallwedidwasdelaythecolumnremovalfromthedatabase.However,itaddressesboththerollbackissueandthezero-downtimedeploymentissue.Asaresult,itreducestheriskassociatedwiththerelease.Ifweadjusttherollbackperiodtoareasonableamountoftime,forexample,inthecaseofmultiplereleasesperdaytotwoweeks,thentheriskisnegligible.Wedon'tusuallyrollmanyversionsback.
Droppingacolumnwasaverysimpleexample.Let'shavealookatamoredifficultscenarioandrenametheresultcolumninourcalculatorservice.Wepresenthowtodothisinafewsteps:
1. Addinganewcolumntothedatabase.2. Changingthecodetousebothcolumns.3. Mergingthedatainbothcolumns.4. Removingtheoldcolumnfromthecode.5. Droppingtheoldcolumnfromthedatabase.
altertableCALCULATION<br/>addSUMvarchar(100);
Asaresult,afterexecutingthemigration,wehavetwocolumns:resultandsum.
ChangingthecodetousebothcolumnsThenextstepistorenamethecolumninthesourcecodemodelandtousebothdatabasecolumnsforthesetandgetoperations.WecanchangeitintheCalculationclass:publicclassCalculation{...privateStringsum;...publicCalculation(Stringa,Stringb,Stringsum,TimestampcreatedAt){this.a=a;this.b=b;this.sum=sum;this.result=sum;this.createdAt=createdAt;}
publicStringgetSum(){returnsum!=null?sum:result;}}
Tobe100%accurate,inthegetSum()method,weshouldcomparesomethinglikethelastmodificationcolumndate(notexactlynecessarytoalwaystakethenewcolumnfirst).
Fromnowon,everytimeweaddarowintothedatabase,thesamevalueiswrittentoboththeresultandsumcolumns.Whilereadingsum,wefirstcheckifitexistsinthenewcolumn,andifnot,wereaditfromtheoldcolumn.
Thesameresultcanbeachievedwiththeuseofdatabasetriggersthatwouldautomaticallywritethesamevaluesintobothcolumns.
Allthechangeswemadesofarwerebackwards-compatible,sowecanrollbacktheserviceanytimewewant,toanyversionwewant.
updateCALCULATION<br/>setCALCULATION.sum=CALCULATION.result<br/>whereCALCULATION.sumisnull;
Westillhavenolimitsfortherollback;however,ifweneedtodeploytheversionbeforethechangeinstep2,thenthisdatabasemigrationneedstoberepeated.
publicclassCalculation{<br/>...<br/>privateStringsum;<br/>...<br/>publicCalculation(Stringa,Stringb,Stringsum,TimestampcreatedAt){<br/>this.a=a;<br/>this.b=b;<br/>this.sum=sum;<br/>this.createdAt=createdAt;<br/>}<br/><br/>publicStringgetSum(){<br/>returnsum;<br/>}<br/>}
Afterthisoperation,wenolongerusetheresultcolumninthecode.Notethatthisoperationisonlybackwards-compatibleuptostep2.Ifweneedtorollbacktostep1,thenwecouldlosethedatastoredafterthisstep.
DroppingtheoldcolumnfromthedatabaseThelaststepistodroptheoldcolumnfromthedatabase.Thismigrationshouldbeperformedaftertherollbackperiodwhenwearesurewewon'tneedtorollbackbeforestep4.
Therollbackperiodcanbeverylongsincewearen'tusingthecolumnfromthedatabaseanymore.Thistaskcanbetreatedasacleanuptask,soeventhoughit'snon-backwards-compatible,thereisnoassociatedrisk.
Let'saddthefinalmigration,V5__Drop_result_column.sql:
altertableCALCULATION
dropcolumnRESULT;
Afterthisstep,wefinallycompletedthecolumnrenamingprocedure.Notethatallwedidwascomplicatetheoperationalittlebit,inordertostretchitintime.Thisreducedtheriskofbackwards-incompatibledatabasechangesandallowedzero-downtimedeployments.
SeparatingdatabaseupdatesfromcodechangesSofar,inallfigures,wepresentedthatdatabasemigrationsareruntogetherwithservicereleases.Inotherwords,eachcommit(whichimplieseachrelease)tookbothdatabasechangesandcodechanges.However,therecommendedapproachistomakeaclearseparationthatacommittotherepositoryiseitheradatabaseupdateoracodechange.Thismethodispresentedinthefollowingimage:
Thebenefitofdatabase-servicechangeseparationisthatwegetthebackwards-compatibilitycheckforfree.Imaginethatthechangesv11andv1.2.7concernonelogicalchange,forexample,addinganewcolumntothedatabase.Then,wefirstcommitdatabasev11,sothetestsintheContinuousDeliverypipelinecheckifdatabasev11workscorrectlywithservicev.1.2.6.Inotherwords,theycheckifdatabaseupdatev11isbackwards-compatible.Then,wecommitthev1.2.7change,sothepipelinechecksifdatabasev11worksfinewithservicev1.2.7.
Thedatabase-codeseparationdoesnotmeanthatwemusthavetwoseparateJenkinspipelines.Thepipelinecanalwaysexecuteboth,butweshouldkeepitasagoodpracticethatacommitiseitheradatabaseupdateoracodechange.
Tosumup,thedatabaseschemachangesshouldbeneverdonemanually.Instead,weshouldalwaysautomatethemusingamigrationtool,executedasapartoftheContinuousDeliverypipeline.Weshouldalsoavoidnon-backwards-compatibledatabaseupdatesandthebestwaytoassurethisistocommit
separatelythedatabaseandcodechangesintotherepository.
AvoidingshareddatabaseInmanysystems,wecanspotthatthedatabasebecomesthecentralpointthatissharedbetweenmultipleservices.Insuchacase,anyupdatetothedatabasebecomesmuchmorechallengingbecauseweneedtocoordinateitbetweenallservices.
Forexample,imaginewedevelopanonlineshopandwehaveaCustomerstablethatcontainsthefollowingcolumns:firstname,lastname,username,password,email,anddiscount.Therearethreeservicesthatareinterestedinthecustomer'sdata:
Profilemanager:Thisenableseditinguser'sdataCheckoutprocessor:Thisprocessesthecheckout(readsusernameandemail)Discountmanager:Thisanalyzesthecustomer'sordersandsetsthesuitablediscount
Let'slookatthefollowingimagethatpresentsthissituation:
Theyaredependentonthesamedatabaseschema.Thereareatleasttwoissueswithsuchanapproach:
Whenwewanttoupdatetheschema,itmustbecompatiblewithallthreeservices.Whileallbackwards-compatiblechangesarefine,anynon-backwards-compatibleupdatebecomeswaymoredifficultoreven
impossible.EachservicehasaseparatedeliverycycleandaseparateContinuousDeliverypipeline.Then,whichpipelineshouldweuseforthedatabaseschemamigrations?Unfortunately,thereisnogoodanswertothisquestion.
Forthereasonsmentionedpreviously,eachserviceshouldhaveitsowndatabaseandtheservicesshouldcommunicateviatheirAPIs.Followingourexample,wecouldapplythefollowingrefactoring:
Thecheckoutprocessorshouldcommunicatewiththeprofilemanager'sAPItofetchthecustomer'sdataThediscountcolumnshouldbeextractedtoaseparatedatabase(orschema),andthediscountmanagershouldtaketheownership
Therefactoredversionispresentedinthefollowingimage:
Suchanapproachisconsistentwiththeprinciplesofthemicroservicearchitectureandshouldalwaysbeapplied.ThecommunicationoverAPIsiswaymoreflexiblethanthedirectdatabaseaccess.
Inthecaseofmonolithicsystems,adatabaseisusuallytheintegrationpoint.Sincesuchanapproachcausesalotofissues,it'sconsideredasananti-pattern.
PreparingtestdataWehavealreadypresenteddatabasemigrationsthatkeepthedatabaseschemaconsistentbetweentheenvironmentsasasideeffect.Thisisduetothefactthatifwerunthesamemigrationscriptsonthedevelopmentmachine,inthestagingenvironment,orintheproduction,thenwewouldalwaysgettheresultinthesameschema.However,thedatavaluesinsidethetablesdiffer.Howcanwepreparethetestdatasothatitwouldeffectivelytestoursystem?Thisisthetopicofthissection.
Theanswertothisquestiondependsonthetypeofthetest,anditisdifferentforunittesting,integration/acceptancetesting,andperformancetesting.Let'sexamineeachcase.
UnittestingInthecaseofunittesting,wedon'tusetherealdatabase.Weeithermockthetestdataonthelevelofthepersistencemechanism(repositories,dataaccessobjects)orwefaketherealdatabasewithanin-memorydatabase(forexample,H2database).Sinceunittestsarecreatedbydevelopers,theexactdatavaluesareusuallyinventedbydevelopersandtheydon'tmattermuch.
Integration/acceptancetestingIntegrationandacceptancetestsusuallyusethetest/stagingdatabase,whichshouldbeassimilaraspossibletotheproduction.Oneapproach,takenbymanycompanies,istosnapshottheproductiondataintostagingthatguaranteesthatitisexactlythesame.Thisapproach,however,istreatedasananti-patternforthefollowingreasons:
Testisolation:Eachtestoperatesonthesamedatabase,sotheresultofonetestmayinfluencetheinputoftheothersDatasecurity:ProductioninstancesusuallystoresensitiveinformationandarethereforebettersecuredReproducibility:Aftereverysnapshot,thetestdataisdifferent,whichmayresultinflakytests
Fortheprecedingreasons,thepreferredapproachistomanuallypreparethetestdatabyselectingasubsetoftheproductiondata,togetherwiththecustomerorthebusinessanalyst.Whentheproductiondatabasegrows,it'sworthrevisitingitscontenttoseeifthereareanyreasonablecasesthatshouldbeadded.
ThebestwaytoadddatatothestagingdatabaseistousethepublicAPIofaservice.Thisapproachisconsistentwithacceptancetests,whichareusuallyblack-box.What'smore,usingtheAPIguaranteesthatthedataitselfisconsistentandsimplifiesdatabaserefactoringbylimitingdirectdatabaseoperations.
PerformancetestingTestdatafortheperformancetestingisusuallysimilartoacceptancetesting.Onesignificantdifferenceistheamountofdata.Inordertotesttheperformancecorrectly,weneedtoprovidesufficientvolumeofinputdata,atleastaslargeasavailableontheproduction(duringthepeaktime).Forthispurpose,wecancreatedatagenerators,whichareusuallysharedbetweenacceptanceandperformancetests.
PipelinepatternsWealreadyknoweverythingthatisnecessarytostartaprojectandsetuptheContinuousDeliverypipelinewithJenkinsandDocker.ThissectionisintendedtoextendthisknowledgewithafewoftherecommendedgeneralJenkinspipelinepractices.
ParallelizingpipelinesThroughoutthisbook,wehavealwaysexecutedthepipelinesequentially,stagebystage,stepbystep.Thisapproachmakesiteasytoreasonthestateandtheresultofthebuild.Ifthereisfirsttheacceptanceteststageandthenthereleasestage,itmeansthatthereleasewon'teverhappenuntiltheacceptancetestsaresuccessful.Sequentialpipelinesaresimpletounderstandandusuallydonotcauseanysurprises.Thisiswhythefirstmethodtosolveanyproblemistodoitsequentially.
However,insomecases,thestagesaretime-consumingandit'sworthrunningtheminparallel.Averygoodexampleisperformancetests.Theyusuallytakealotoftime,soassumingtheyareindependentandisolated,itmakessensetoruntheminparallel.InJenkins,wecanparallelizethepipelineontwodifferentlevels:
Parallelsteps:Withinonestage,parallelprocessesrunonthesameagent.ThismethodissimplebecauseallJenkinsworkspace-relatedfilesarelocatedononephysicalmachine,however,asalwayswiththeverticalscaling,theresourcesarelimitedtothatsinglemachine.Parallelstages:Eachstagecanberuninparallelonaseparateagentmachinethatprovideshorizontalscalingofresources.Weneedtotakecareofthefiletransferbetweentheenvironments(usingthestashJenkinsfilekeyword)ifafilecreatedinthepreviousstageisneededontheotherphysicalmachine.
Bythetimeofwritingthisbook,parallelstagesarenotavailableinthedeclarativepipeline.ThefeatureissupposedtobeaddedinJenkinsBlueOceanv1.3.Inthemeantime,theonlypossibilityistousethedeprecatedfeatureintheGroovy-basedscriptingpipeline,asdescribedhereathttps://jenkins.io/doc/book/pipeline/jenkinsfile/#executing-in-parallel.
Let'slookathowitlooksinpractice.Ifwewouldliketoruntwostepsinparallel,theJenkinsfilescriptshouldlookasfollows:
pipeline{
agentany
stages{
stage('Stage1'){
steps{
parallel(
one:{echo"parallelstep1"},
two:{echo"parallelstep2"}
)
}
}
stage('Stage2'){
steps{
echo"runafterbothparallelstepsarecompleted"
}
}
}
}
InStage1,withtheuseoftheparallelkeyword,weexecutetwoparallelsteps,oneandtwo.NotethatStage2isexecutedonlyafterbothparallelstepsarecompleted.Thisiswhysuchsolutionsareperfectlysafetoruntestsinparallel;wecanalwaysbesurethatthedeploymentstageisrunonlyafterallparallelizedtestshavealreadypassed.
ThereisaveryusefulplugincalledParallelTestExecutorthathelpstoautomaticallysplittestsandruntheminparallel.Readmoreathttps://jenkins.io/doc/pipeline/steps/parallel-test-executor/.
Theprecedingdescriptionconcernedtheparallelstepslevel.Theothersolutionwouldbetouseparallelstagesandthereforeruneachstageonaseparateagentmachine.Thedecisiononwhichtypeofparallelismtouseusuallydependsontwofactors:
HowpowerfultheagentmachinesareHowmuchtimethegivenstagetakes
Asageneralrecommendation,unittestsarefinetoruninparallelsteps,butperformancetestsareusuallybetteroffonseparatemachines.
ReusingpipelinecomponentsWhentheJenkinsfilescriptgrowsinsizeandbecomesmorecomplex,wemaywanttoreuseitspartsbetweensimilarpipelines.
Forexample,wemaywanttohaveseparate,butsimilar,pipelinesfordifferentenvironments(dev,QA,prod).AnothercommonexampleinthemicroserviceworldisthateachservicehasaverysimilarJenkinsfile.Then,howdowewriteJenkinsfilescriptssothatwedon'trepeatthesamecodealloveragain?Therearetwogoodpatternsforthispurpose,parameterizedbuildandsharedlibraries.Let'sdescribethemonebyone.
BuildparametersWealreadymentionedinChapter4,ContinuousIntegrationPipeline,thatapipelinecanhaveinputparameters.Wecanusethemtoprovidedifferentusecaseswiththesamepipelinecode.Asanexample,let'screateapipelineparametrizedwiththeenvironmenttype:
pipeline{
agentany
parameters{
string(name:'Environment',defaultValue:'dev',description:'Which
environment(dev,qa,prod)?')
}
stages{
stage('Environmentcheck'){
steps{
echo"Currentenvironment:${params.Environment}"
}
}
}
}
Thebuildtakesoneinputparameter,Environment.Then,allwedointhisstepisprinttheparameter.Wecanalsoaddaconditiontoexecutedifferentcodefordifferentenvironments.
Withthisconfiguration,whenwestartthebuild,wewillseeapromptfortheinputparameter,asfollows:
Parametrizedbuildcanhelpreusethepipelinecodeforscenarioswhenitdiffersjustalittlebit.Thisfeature,however,shouldnotbeoverusedbecausetoomanyconditionscanmaketheJenkinsfiledifficulttounderstand.
Sharedlibraries
Theothersolutiontoreusethepipelineistoextractitspartsintoasharedlibrary.
AsharedlibraryisaGroovycodethatisstoredasaseparatesource-controlledproject.ThiscodecanbelaterusedinmanyJenkinsfilescriptsaspipelinesteps.Tomakeitclear,let'shavealookatanexample.Asharedlibrarytechniquealwaysrequiresthreesteps:
1. Createasharedlibraryproject.2. ConfigurethesharedlibraryinJenkins.3. UsethesharedlibraryinJenkinsfile.
CreatingasharedlibraryprojectWestartbycreatinganewGitproject,inwhichweputthesharedlibrarycode.EachJenkinsstepisexpressedasaGroovyfilelocatedinthevarsdirectory.
Let'screateasayHellostepthattakesthenameparameterandechoesasimplemessage.Thisshouldbestoredinthevars/sayHello.groovyfile:/***Helloworldstep.*/defcall(Stringname){echo"Hello$name!"}
Human-readabledescriptionsforsharedlibrarystepscanbestoredinthe*.txtfiles.Inourexample,wecouldaddthevars/sayHello.txtfilewiththestepdocumentation.
Whenthelibrarycodeisdone,weneedtopushittotherepository,forexample,asanewGitHubproject.
ConfigurethesharedlibraryinJenkinsThenextstepistoregisterthesharedlibraryinJenkins.WeopenManageJenkins|ConfigureSystem,andfindtheGlobalPipelineLibrariessection.There,wecanaddthelibrarygivingitanamechosen,asfollows:
Wespecifiedthenameunderwhichthelibraryisregisteredandthelibraryrepositoryaddress.Notethatthelatestversionofthelibrarywillbeautomaticallydownloadedduringthepipelinebuild.
WepresentedimportingtheGroovycodeasGlobalSharedLibrary,buttherearealsootheralternativesolutions.Readmoreathttps://jenkins.io/doc/book/pipeline/shared-libraries/.
UsesharedlibraryinJenkinsfileFinally,wecanusethesharedlibraryintheJenkinsfilescript.
Let'shavealookattheexample:
pipeline{
agentany
stages{
stage("Hellostage"){
steps{
sayHello'Rafal'
}
}
}
}
If"Loadimplicitly"hadn'tbeencheckedintheJenkinsconfiguration,thenwewouldneedtoadd"@Library('example')_"atthebeginningoftheJenkinsfilescript.
Asyoucansee,wecanusetheGroovycodeasapipelinestepsayHello.Obviously,afterthepipelinebuildcompletes,intheconsoleoutput,weshouldseeHelloRafal!.
Sharedlibrariesarenotlimitedtoonestep.Actually,withthepoweroftheGroovylanguage,theycanevenactastemplatesforentireJenkinspipelines.
RollingbackdeploymentsIrememberthewordsofmycolleague,aseniorarchitect,"Youdon'tneedmoreQAs,youneedafasterrollback".WhilethisstatementisoversimplifiedandtheQAteamisoftenofgreatvalue,thereisalotoftruthinthissentence.Thinkaboutit;ifyouintroduceabugintheproductionbutrollitbacksoonafterthefirstuserreportsanerror,thenusuallynothingbadhappens.Ontheotherhand,ifproductionerrorsarerarebutnorollbackisapplied,thentheprocesstodebugtheproductionusuallyendsupinlongsleeplessnightsandanumberofdissatisfiedusers.ThisiswhyweneedtothinkabouttherollbackstrategyupfrontwhilecreatingtheJenkinspipeline.
InthecontextofContinuousDelivery,therearetwomomentswhenthefailurecanhappen:
Duringthereleaseprocess,inthepipelineexecutionAfterthepipelinebuildiscompleted,inproduction
Thefirstscenarioisprettysimpleandharmless.Itconcernsacasewhentheapplicationisalreadydeployedtoproductionbutthenextstagefails,forexample,thesmoketest.Then,allweneedtodoisexecuteascriptinthepostpipelinesectionforthefailurecase,whichdowngradestheproductionservicetotheolderDockerimageversion.Ifweuseblue-greendeployment(asdescribedlaterinthischapter),theriskofanydowntimeisminimalsinceweusuallyexecutetheload-balancerswitchasthelastpipelinestage,afterthesmoketest.
Thesecondscenario,whenwenoticeaproductionbugafterthepipelineissuccessfullycompleted,ismoredifficultandrequiresafewcomments.Here,theruleisthatweshouldalwaysreleasetherolledbackserviceusingexactlythesameprocessasthestandardrelease.Otherwise,ifwetrytodosomethingmanually,inafasterway,weareaskingfortrouble.Anynonrepetitivetaskisrisky,especiallyunderstress,whentheproductionisoutoforder.
Asasidenote,ifthepipelinecompletessuccessfullybutthereisaproductionbug,thenitmeansthatourtestsarenotgoodenough.So,thefirstthingafterrollbackistoextendtheunit/acceptancetest
suiteswiththecorrespondingscenarios.
ThemostcommonContinuousDeliveryprocessisonefullyautomatedpipelinethatstartsbycheckingoutthecodeandendsupwithreleasetotheproduction.
Thefollowingfigurepresentshowthisworks:
WealreadypresentedtheclassicContinuousDeliverypipelinethroughoutthisbook.Iftherollbackshoulduseexactlythesameprocess,thenallweneedtodoisrevertthelatestcodechangefromtherepository.Asaresult,thepipelineautomaticallybuilds,tests,andfinally,releasestherightversion.
Repositoryrevertsandemergencyfixesshouldneverskipthetestingstagesinthepipeline.Otherwise,wemayendupwithareleasethatisstillnotworkingcorrectlybecauseofanotherissuethatmakesdebuggingevenharder.
Thesolutionisverysimpleandelegant.Theonlydrawbackisthedowntimethatweneedtospendonthecompletepipelinebuild.Thisdowntimecanbeavoidedifweuseblue-greendeploymentorcanaryreleases,inwhichcases,weonlychangetheloadbalancersettingtoaddressthehealthyenvironment.
Therollbackoperationbecomeswaymorecomplexinthecaseoforchestratedreleases,duringwhichmanyservicesaredeployedatthesametime.Thisisoneofthereasonswhyorchestratedreleasesaretreatedasananti-pattern,especiallyinthemicroserviceworld.Thecorrectapproachistoalwaysmaintainbackwardscompatibility,atleastforsometime(likewepresentedforthedatabaseatthebeginningofthischapter).Then,it'spossibletoreleaseeachserviceindependently.
AddingmanualstepsIngeneral,theContinuousDeliverypipelinesshouldbefullyautomated,triggeredbyacommittotherepository,andendupaftertherelease.Sometimes,however,wecan'tavoidhavingmanualsteps.Themostcommonexampleisthereleaseapproval,whichmeansthattheprocessisfullyautomated,butthereisamanualsteptoapprovethenewrelease.Anothercommonexampleismanualtests.Someofthemmayexistbecauseweoperateonthelegacysystem;someothersmayoccurwhenatestsimplycannotbeautomated.Nomatterwhatthereasonis,sometimesthereisnochoicebuttoaddamanualstep.
Jenkinssyntaxoffersakeywordinputformanualsteps:
stage("Releaseapproval"){
steps{
input"Doyouapprovetherelease?"
}
}
Thepipelinewillstopexecutionontheinputstepandwaituntilit'smanuallyapproved.
Rememberthatmanualstepsquicklybecomeabottleneckinthedeliveryprocess,andthisiswhytheyshouldalwaysbetreatedasasolutionthat'sinferiortocompleteautomation.
Itissometimesusefultosetatimeoutfortheinputinordertoavoidwaitingforeverforthemanualinteraction.Aftertheconfiguredtimeiselapsed,thewholepipelineisaborted.
ReleasepatternsInthelastsection,wediscussedtheJenkinspipelinepatternsusedtospeedupthebuildexecution(parallelsteps),helpwiththecodereuse(sharedlibraries),limittheriskofproductionbugs(rollback),anddealwithmanualapprovals(manualsteps).Thissectionpresentsthenextgroupofpatterns,thistimerelatedtothereleaseprocess.Theyaredesignedtoreducetheriskofupdatingtheproductiontoanewsoftwareversion.
Wealreadydescribedoneofthereleasepatterns,rollingupdates,inChapter8,ClusteringwithDockerSwarm.Here,wepresenttwomore:blue-greendeploymentandcanaryreleases.
Blue-greendeploymentBlue-greendeploymentisatechniquetoreducethedowntimeassociatedwiththerelease.Itconcernshavingtwoidenticalproductionenvironments,onecalledgreen,theothercalledblue,aspresentedinthefollowingfigure:
Inthefigure,thecurrentlyaccessibleenvironmentisblue.Ifwewanttomakeanewrelease,thenwedeployeverythingtothegreenenvironmentand,attheendofthereleaseprocess,changetheloadbalancertothegreenenvironment.Asaresult,auser,allofasudden,startsusingthenewversion.Thenexttimewewanttomakearelease,wemakechangestotheblueenvironmentand,attheend,wechangetheloadbalancertoblue.Weproceedthesameeverytime,switchingfromoneenvironmenttoanother.
Theblue-greendeploymenttechniqueworkscorrectlywithtwoassumptions:environmentisolationandnoorchestratedreleases.
Thissolutiongivestwosignificantbenefits:
Zerodowntime:Allthedowntimefromtheuserperspectiveisamomentofchangingtheloadbalanceswitch,whichisnegligibleRollback:Inordertorollbackoneversion,it'senoughtochangebackthe
loadbalanceswitchblue-greendeploymentinclude:Database:Schemamigrationscanbetrickyincaseofarollback,soit'sworthusingthepatternspresentedatthebeginningofthischapterTransactions:RunningdatabasetransactionsmustbehandedovertothenewdatabaseRedundantinfrastructure/resources:Weneedtohavedoubletheresources
Therearetechniquesandtoolstoovercomethesechallenges,sotheblue-greendeploymentpatternishighlyrecommendedandwidelyusedintheITindustry.
Youcanreadmoreabouttheblue-greendeploymenttechniqueontheexcellentMartinFowler'sbloghttps://martinfowler.com/bliki/BlueGreenDeployment.html.
CanaryreleaseCanaryreleasingisatechniquetoreducetheriskassociatedwithintroducinganewversionofthesoftware.Similartoblue-greendeployment,itusestwoidenticalenvironments,aspresentedinthefollowingfigure:
Also,similartotheblue-greendeploymenttechnique,thereleaseprocessstartsbydeployinganewversionintheenvironmentthatiscurrentlyunused.Here,however,thesimilaritiesend.Theloadbalancer,insteadofswitchingtothenewenvironment,issettolinkonlyaselectedgroupofuserstothenewenvironment.Allthereststillusetheoldversion.Thisway,anewversioncanbetestedbysomeusersandincaseofabug,onlyasmallgroupisaffected.Afterthetestingperiod,allusersareswitchedtothenewversion.
Thisapproachhassomegreatbenefits:
Acceptanceandperformancetesting:Iftheacceptanceandperformancetestingisdifficulttoruninthestagingenvironment,thenit'spossibletotestitinproduction,minimizingtheimpactonasmallgroupofusers.Simplerollback:Ifanewchangecausesafailure,thenrollingbackisdonebyswitchingbackalluserstotheoldversion.A/Btesting:IfwearenotsurewhetherthenewversionisbetterthantheUXortheperformanceperspective,thenit'spossibletocompareitwiththe
oldversion.
Canaryreleasingsharesthesamedrawbacksasblue-greendeployment.Theadditionalchallengeisthatwehavetwoproductionsystemsrunningatthesametime.Nevertheless,canaryreleasingisanexcellenttechniqueusedinmostcompaniestohelpwiththereleaseandtesting.
YoucanreadmoreaboutthecanaryreleasingtechniqueontheexcellentMartinFowler'sbloghttps://martinfowler.com/bliki/CanaryRelease.html.
WorkingwithlegacysystemsAllwehavedescribedsofarappliessmoothlytogreenfieldprojects,forwhichsettingupaContinuousDeliverypipelineisrelativelysimple.
Legacysystemsare,however,waymorechallengingbecausetheyusuallydependonmanualtestsandmanualdeploymentsteps.Inthissection,wewillwalkthroughtherecommendedscenariotoincrementallyapplyContinuousDeliverytoalegacysystem.
Asastepzero,IrecommendreadinganexcellentbookbyMichaelFeathers,WorkingEffectivelywithLegacyCode.Hisideasonhowtodealwithtesting,refactoring,andaddingnewfeaturesclearmostoftheconcernsabouthowtoautomatethedeliveryprocessforlegacysystems.
Formanydevelopers,itmaybetemptingtocompletelyrewritealegacysystem,ratherthanrefactorit.Whiletheideaisinterestingfromadeveloper'sperspective,itisusuallyabadbusinessdecisionthatresultsinproductfailure.YoucanreadmoreaboutthehistoryofrewritingtheNetscapebrowserinanexcellentblogpostbyJoelSpolsky,ThingsYouShouldNeverDo,athttps://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i.
ThewaytoapplytheContinuousDeliveryprocessdependsalotonthecurrentproject'sautomation,thetechnologiesused,thehardwareinfrastructure,andthecurrentreleaseprocess.Usually,itcanbesplitintothreesteps:
1. Automatingbuildanddeployment.2. Automatingtests.3. Refactoringandintroducingnewfeatures.
Let'slookatthemindetail.
AutomatingbuildanddeploymentThefirststepincludesautomatingthedeploymentprocess.ThegoodnewsisthatinmostlegacysystemsIhaveworkedwith,therewasalreadysomeautomationinplace,forexample,intheformofshellscripts.
Inanycase,theactivitiesforautomateddeploymentincludesthefollowing:
Buildandpackage:SomeautomationusuallyalreadyexistsintheformofMakefile,Ant,Maven,anyotherbuildtoolconfiguration,oracustomscript.Databasemigration:Weneedtostartmanagingthedatabaseschemainanincrementalmanner.ItrequiresputtingthecurrentschemaasaninitialmigrationandmakingallthefurtherchangeswithtoolssuchasFlywayorLiquibase,asalreadydescribedinthischapter.Deployment:Evenifthedeploymentprocessisfullymanual,thenthereisusuallyatext/wikipagedescriptionthatneedstobeconvertedintoanautomatedscript.Repeatableconfiguration:Inlegacysystems,configurationfilesareusuallychangedmanually.Weneedtoextracttheconfigurationanduseaconfigurationmanagementtool,asdescribedinChapter6,ConfigurationManagementwithAnsible.
Aftertheprecedingsteps,wecanputeverythingintoadeploymentpipelineanduseitasanautomatedphaseafteramanualUAT(useracceptancetesting)cycle.
Fromtheprocessperspective,it'sworthalreadystartingreleasingmoreoften.Forexample,ifthereleaseisyearly,trytodoitquarterly,thenmonthly.Thepushforthatfactorwilllaterresultinfaster-automateddeliveryadoption.
AutomatingtestsThenextstep,usuallymuchmoredifficult,istopreparetheautomatedtestsforthesystem.ItrequirescommunicatingwiththeQAteaminordertounderstandhowtheycurrentlytestthesoftwaresothatwecanmoveeverythingintoanautomatedacceptancetestsuite.Thisphaserequirestwosteps:
Acceptance/sanitytestsuite:WeneedtoaddautomatedteststhatreplacesomeoftheregressionactivitiesoftheQAteam.Dependingonthesystem,theycanbeprovidedasablack-boxSeleniumtestorCucumbertest.(Virtual)testenvironments:Atthispoint,weshouldbealreadythinkingoftheenvironmentsinwhichourtestswouldberun.Usually,thebestsolutiontosaveresourcesandlimitthenumberofmachinesrequiredistovirtualizethetestingenvironmentusingVagrantorDocker.
TheultimategoalistohaveanautomatedacceptancetestsuitethatwillreplacethewholeUATphasefromthedevelopmentcycle.Nevertheless,wecanstartwithasanitytestthatwillshortlycheckifthesystemiscorrectfromtheregressionperspective.
Whileaddingtestscenarios,rememberthatthetestsuiteshouldexecuteinreasonabletime.Forsanitytests,itisusuallylessthan10minutes.
RefactoringandintroducingnewfeaturesWhenwehaveatleastthefundamentalregressiontestingsuite,wearereadytoaddnewfeaturesandrefactortheoldcode.It'salwaysbettertodoitinsmallpieces,stepbystepbecauserefactoringeverythingatonceusuallyendsupinachaosthatleadstoproductionfailures(notclearlyrelatedtoanyparticularchange).
Thisphaseusuallyincludesthefollowingactivities:
Refactoring:Thebestplacetostartrefactoringtheoldcodeiswherethenewfeaturesareexpected.Startingthisway,wearepreparedforthenewfeaturerequeststocome.Rewrite:Ifweplantorewritepartsoftheoldcode,weshouldstartfromthecodethatisthemostdifficulttotest.Thisway,weconstantlyincreasethecodecoverageinourproject.Introducingnewfeatures:Duringthenewfeatureimplementation,it'sworthusingthefeaturetogglepattern.Then,incaseanythingbadhappens,wecanquicklyturnoffthenewfeature.Actually,thesamepatternshouldbeusedduringrefactoring.
Forthisphase,it'sworthreadinganexcellentbookbyMartinFowler,Refactoring:ImprovingtheDesignofExistingCode.
Whiletouchingtheoldcode,it'sgoodtofollowtheruletoalwaysaddapassingunittestfirst,andonlythen,changethecode.Withthisapproach,wecandependonautomationtocheckthatwedon'tchangethebusinesslogicbyaccident.
UnderstandingthehumanelementWhileintroducingtheautomateddeliveryprocesstoalegacysystem,it'spossibleyouwillfeel,morethananywhereelse,thehumanfactor.Inordertoautomatethebuildprocess,weneedtocommunicatewellwiththeoperationsteam,andtheymustbewillingtosharetheirknowledge.ThesamestoryappliestothemanualQAteam;theyneedtobeinvolvedinwritingautomatedtestsbecauseonlytheyknowhowtotestthesoftware.Ifyouthinkaboutit,boththeoperationsandQAteamsneedtocontributetotheprojectthatwilllaterautomatetheirwork.Atsomepoint,theymayrealizethattheirfutureinthecompanyisnotstableandbecomelesshelpful.ManycompaniesstrugglewithintroducingtheContinuousDeliveryprocessbecauseteamsdonotwanttogetinvolvedenough.
Inthissection,wediscussedhowtoapproachlegacysystemsandthechallengestheypose.IfyouareinprogressofconvertingyourprojectandorganizationintotheContinuousDeliveryapproach,thenyoumaywanttohavealookattheContinuousDeliveryMaturityModel,whichaimstogivestructuretotheprocessofadoptingtheautomateddelivery.
AgooddescriptionoftheContinuousDeliveryMaturityModelcanbefoundathttps://developer.ibm.com/urbancode/docs/continuous-delivery-maturity-model/.
Exercises
Inthischapter,wehavecoveredvariousaspectsoftheContinuousDeliveryprocess.Sincepracticemakesperfect,werecommendthefollowingexercises:
1. UseFlywaytocreateanon-backwards-compatiblechangeintheMySQLdatabase:
UsetheofficialDockerimage,mysql,tostartthedatabaseConfigureFlywaywithproperdatabaseaddress,username,andpasswordCreateaninitialmigrationthatcreatesauserstablewiththreecolumns:id,email,andpasswordAddasampledatatothetableChangethepasswordcolumntohashed_password,whichwillstorethehashedpasswordsSplitthenon-backwards-compatiblechangeintothreemigrationsasdescribedinthischapterYoucanuseMD5orSHAforhashingCheckthat,asaresult,thedatabasestoresnopasswordsinplaintext
2. CreateaJenkinssharedlibrarywithstepstobuildandunittestGradleprojects:
CreateaseparaterepositoryforthelibraryCreatetwofilesinthelibrary:gradleBuild.groovyandgradleTest.groovyWritetheappropriatecallmethodsAddthelibrarytoJenkinsUsethestepsfromthelibraryinapipeline
SummaryThischapterwasamixtureofvariousContinuousDeliveryaspectsthatwerenotcoveredbefore.Thekeytakeawaysfromthechapterareasfollows:
Databasesareanessentialpartofmostapplicationsandshould,therefore,beincludedintheContinuousDeliveryprocess.Databaseschemachangesarestoredintheversioncontrolsystemandmanagedbydatabasemigrationtools.Therearetwotypesofdatabaseschemachange:backwards-compatibleandbackwards-incompatible.Whilethefirsttypeissimple,thesecondrequiresabitofoverhead(splittomultiplemigrationsspreadovertime).Adatabaseshouldnotbethecentralpointofthewholesystem.Thepreferredsolutionistoprovideeachservicewithitsowndatabase.Thedeliveryprocessshouldalwaysbepreparedfortherollbackscenario.Threereleasepatternsshouldalwaysbeconsidered:rollingupdates,blue-greendeployment,andcanaryreleasingLegacysystemscanbeconvertedintotheContinuousDeliveryprocessinsmallstepsratherthanallatonce.
BestpracticesThankyouforreadingthisbook.IhopeyouarereadytointroducetheContinuousDeliveryapproachtoyourITprojects.Asthelastsectionofthisbook,Iproposealistofthetop10ContinuousDeliverypractices.Enjoy!
Practice1–ownprocesswithintheteam!Owntheentireprocesswithintheteam,fromreceivingrequirementstomonitoringtheproduction.Asoncesaid:"Aprogramrunningonthedeveloper'smachinemakesnomoney."Thisiswhyit'simportanttohaveasmallDevOpsteamthattakescompleteownershipofaproduct.Actually,thatisthetruemeaningofDevOps-DevelopmentandOperationsfromthebeginningtotheend:
OwneverystageoftheContinuousDeliverypipeline:howtobuildthesoftware,whattherequirementsareinacceptancetests,andhowtoreleasetheproduct.Avoidhavingapipelineexpert!Everymemberoftheteamshouldbeinvolvedincreatingthepipeline.Findagoodwaytosharethecurrentpipelinestate(andtheproductionmonitoring)amongteammembers.Themosteffectivesolutionisbigscreensintheteamspace.Ifadeveloper,QA,andITOperationsengineerareseparateexperts,thenmakesuretheyworktogetherinoneagileteam.Separateteamsbasedonexpertiseresultintakingnoresponsibilityfortheproduct.Rememberthatautonomygiventotheteamresultsinhighjobsatisfactionandexceptionalengagement.Thisleadstogreatproducts!
Practice2–automateeverything!Automateeverythingfrombusinessrequirements(intheformofacceptancetests)tothedeploymentprocess.Manualdescriptions,wikipageswithinstructionsteps,theyallquicklybecomeoutofdateandleadtotribalknowledgethatmakestheprocessslow,tedious,andunreliable.This,inturn,leadstoaneedforreleaserehearsalsandmakeseverydeploymentunique.Don'tgodownthispath!Asarule,ifyoudoanythingforthesecondtime,automateit:
Eliminateallmanualsteps;theyareasourceoferrors!Thewholeprocessmustberepeatableandreliable.Don'tevermakeanychangesdirectlyinproduction!Useconfigurationmanagementtoolsinstead.Usepreciselythesamemechanismtodeploytoeveryenvironment.Alwaysincludeanautomatedsmoketesttocheckifthereleasecompletedsuccessfully.Usedatabaseschemamigrationstoautomatedatabasechanges.Useautomaticmaintenancescriptsforbackupandcleanup.Don'tforgettoremoveunusedDockerimages!
Practice3–versioneverything!Versioneverything:softwaresourcecode,buildscripts,automatedtests,configurationmanagementfiles,ContinuousDeliverypipelines,monitoringscripts,binaries,anddocumentation.Simplyeverything.Makeyourworktask-based,whereeachtaskresultsinacommittotherepository,nomatterwhetherit'srelatedtorequirementgathering,architecturedesign,configuration,orthesoftwaredevelopment.Ataskstartsontheagileboardandendsupintherepository.Thisway,youmaintainasinglepointoftruthwiththehistoryandreasonsforthechanges:
Bestrictabouttheversioncontrol.Everythingmeanseverything!Keepsourcecodeandconfigurationinthecoderepository,binariesintheartifactrepository,andtasksintheagileissuetrackingtool.DeveloptheContinuousDeliverypipelineasacode.Usedatabasemigrationsandstoretheminarepository.Storedocumentationintheformofmarkdownfilesthatcanbeversion-controlled.
Practice4–usebusinesslanguageforacceptancetests!Usebusiness-facinglanguageforacceptanceteststoimprovethemutualcommunicationandthecommonunderstandingoftherequirements.WorkcloselywiththeproductownertocreatewhatEricEvancalledtheubiquitouslanguage,acommondialectbetweenthebusinessandtechnology.Misunderstandingsaretherootcauseofmostprojectfailures:
Createacommonlanguageanduseitinsidetheproject.UseanacceptancetestingframeworksuchasCucumberorFitNessetohelpthebusinessteamunderstandandgettheminvolved.Expressbusinessvaluesinsideacceptancetestsanddon'tforgetaboutthemduringdevelopment.It'seasytospendtoomuchtimeonunrelatedtopics!Improveandmaintainacceptancetestssothattheyalwaysactasregressiontests.Makesureeveryoneisawarethatapassingacceptancetestsuitemeansagreenlightfromthebusinesstoreleasethesoftware.
Practice5–bereadytorollback!Bereadytorollback;soonerorlateryouwillneedtodoit.Remember,Youdon'tneedmoreQAs,youneedafasterrollback.Ifanythinggoeswronginproduction,thefirstthingyouwanttodoistoplaysafeandcomebacktothelastworkingversion:
DeveloparollbackstrategyandtheprocessofwhattodowhenthesystemisdownSplitnon-backwards-compatibledatabasechangesintocompatibleonesAlwaysusethesameprocessofdeliveryforrollbacksandforstandardreleasesConsiderintroducingblue-greendeploymentsorcanaryreleasesDon'tbeafraidofbugs,theuserwon'tleaveyouifyoureactquickly!
Practice6–don'tunderestimatetheimpactofpeopleDon'tunderestimatetheimpactofpeople.Theyareusuallywaymoreimportantthantools.Youwon'tautomatethedeliveryiftheITOperationsteamwon'thelpyou.Afterall,theyhavetheknowledgeaboutthecurrentprocess.ThesameappliestoQAs,business,andeveryoneinvolved.Makethemimportantandinvolved:
LetQAsandIToperationsbeapartoftheDevOpsteam.Youneedtheirknowledgeandskills!Providetrainingtomembersthatarecurrentlydoingmanualactivitiessothattheycanmovetoautomation.Favorinformalcommunicationandaflatstructureoforganizationoverhierarchyandorders.Youwon'tdoanythingwithoutgoodwill!
Practice7–buildintraceability!Buildintraceabilityforthedeliveryprocessandworkingsystem.Thereisnothingworsethanafailurewithoutanylogmessages.Monitorthenumberofrequests,thelatency,theloadofproductionservers,thestateoftheContinuousDeliverypipeline,andeverythingyoucanthinkofthatcouldhelpyoutoanalyzeyourcurrentsoftware.Beproactive!Atsomepoint,youwillneedtocheckthestatsandlogs:
Logpipelineactivities!Inthecaseoffailure,notifytheteamwithaninformativemessage.Implementproperloggingandmonitoringoftherunningsystem.UsespecializedtoolsforsystemmonitoringsuchasKibana,Grafana,orLogmatic.io.Integrateproductionmonitoringintoyourdevelopmentecosystem.Considerhavingbigscreenswiththecurrentproductionstatsinthecommonteamspace.
Practice8–integrateoften!Integrateoften,actually,allthetime!Assomeonesaid:Continuousismoreoftenthanyouthink.Thereisnothingmorefrustratingthanresolvingmergeconflicts.ContinuousIntegrationislessaboutthetoolandmoreabouttheteampractice.Integratethecodeintoonecodebaseatleastafewtimesaday.Forgetaboutlong-lastingfeaturebranchesandahugenumberoflocalchanges.Trunk-basedevelopmentandfeaturetogglesforthewin!
Usetrunk-baseddevelopmentandfeaturetogglesinsteadoffeaturebranches.Ifyouneedabranchorlocalchanges,makesurethatyouintegratewiththerestoftheteamatleastonceaday.Alwayskeepthetrunkhealthy;makesureyouruntestsbeforeyoumergeintothebaseline.Runthepipelineaftereverycommittotherepositoryforafastfeedbackcycle.
Practice9–buildbinariesonlyonce!Buildbinariesonlyonceandrunthesameoneoneachoftheenvironments.NomatteriftheyareinaformofDockerimagesorJARpackages,buildingonlyonceeliminatestheriskofdifferencesintroducedbyvariousenvironments.Italsosavestimeandresources:
Buildonceandpassthesamebinarybetweenenvironments.Useartifactrepositorytostoreandversionbinaries.Don'teverusethesourcecoderepositoryforthatpurpose.Externalizeconfigurationsanduseaconfigurationmanagementtooltointroducedifferencesbetweenenvironments.
Practice10–releaseoften!Releaseoften,preferablyaftereachcommittotherepository.Asthesayinggoes,Ifithurts,doitmoreoften.Releasingasadailyroutinemakestheprocesspredictableandcalm.Stayawayfrombeingtrappedintherarereleasehabit.Thatwillonlygetworseandyouwillendupwithreleasingonceayearhavingathreemonths'preparationperiod!
RephraseyourdefinitionofdonetoDonemeansreleased.Takeownershipofthewholeprocess!Usefeaturetogglestohide(fromusers)featuresthatarestillinprogress.Usecanaryreleasesandquickrollbacktoreducetheriskofbugsintheproduction.Adoptazero-downtimedeploymentstrategytoenablefrequentreleases.