CONTAINERORCHESTRATIONWITHSWARMMODE,MESOS/MARATHONANDKUBERNETES

ADRIANMOUAT

WHOAMI?ChiefScientistatContainerSolutionsWrote"UsingDocker"forO'Reilly40%discountwithcodeAUTHD

DockerCaptain@adrianmouat

WHATISORCHESTRATION?

OxfordEnglishDictionary

"Theplanningorcoordinationoftheelementsofasituationtoproduceadesiredeffect,

especiallysurreptitiously"

"Theplanningorcoordinationoftheelementsofasituationtoproduceadesiredeffect,

especiallysurreptitiously"

ELEMENTSContainersHostsNetworking

"Theplanningorcoordinationoftheelementsofasituationtoproduceadesiredeffect,

especiallysurreptitiously"

DESIREDEFFECTRunningapplicationAutomaticallyscaleFaulttolerantfailover,noderebalancing,healthchecks

UseresourcesefficientlyLittlemanualintervention

"Theplanningorcoordinationoftheelementsofasituationtoproduceadesiredeffect,

especiallysurreptitiously"

SURREPTITIOUSLY

OxfordEnglishDictionary

"Inawaythatattemptstoavoidnoticeorattention;secretively"

SURREPTITIOUSLYShouldhappeninthebackgroundUserdoesn'tneedtodetailsComplexityishidden

HOWIMPORTANTISORCHESTRATION?MightnotneeditforsmallappsNoorchestration==manualorchestrationManuallyplacecontainers,network,scale,check,update

COMPARINGORCHESTRATORS

COMPARINGORCHESTRATORSAllworkandareimprovingrapidly

Understandthedifferences

Understandyourrequirements

Pleasedon'trollyourown!

THEPLAYERSKubernetes

Mesos

DockerSwarmMode

Plusothers

Nomad,PaaSs...

SIDENOTE-THEBORG/OMEGAPAPERSInfluentialpapersfromGoogleLessonslearntfrom10yearswithcontainersBothhigh-levelandtechnicalreportsKubernetes,DockerSwarmandNomad

WELCOMETOTHESOCKSHOP!

MICROSERVICEREFERENCEAPPLICATIONIntendedtohelppeoplegetstartedwithmicroservicesGreatforcomparingframeworksetcSimilarto"PetStore"forJava...orTodoMVCforJavaScript

LOTSOFIMPLEMENTATIONSALREADYhttps://github.com/microservices-demo/microservices-demo/tree/master/deploy

ARCHITECTURE

DEMO

DOCKERSWARMMODE

SWARMMODENewinDocker1.12

DockerInc'sofficialsolution

Partofcoredistribution

MajorimprovementoveroldSwarm

CORECOMPONENTSManagernodescoordinateviaRaftnoneedforseparateetcd/zookeeper

Workernodes

SNAPTOINSTALL$dockerswarminitSwarminitialized:currentnode(ejgr3igiwu7mdvescuncyfq19)isnowamanager.

Toaddaworkertothisswarm,runthefollowingcommand:

dockerswarmjoin\--tokenSWMTKN-1-5cubmn41arycoxa2hk4iu6gmauzafros4vhvnq8ftp0cuacn2u-2jxkuoimbzzoow00wuyxhf1mo\10.34.9.126:2377

Toaddamanagertothisswarm,run'dockerswarmjoin-tokenmanager'andfollowtheinstructions.

SECURECOMMSBYDEFAULTTLSsetupusingself-signedcertsCertificatesautomaticallyrotated

OTHERFEATURESSpreadschedulingchooses"leastloaded"nodemoreoptionslatersupportforreserving&limitingcpu/mem

GlobalorReplicamode"MeshNetworking"

CONSTRAINTSANDFILTERSControlwhichnodeacontainercanbescheduledonE.g:Onlynodeslabeled"staging"OnlynodeswhichhavetheimageOnlythenoderunningagivencontainer

RUNNINGSOCKSHOP

(notetherepocurrentlyneedsupdatedin-linewithversionshownintalk)

https://raw.githubusercontent.com/microservices-demo/microservices-demo/master/deploy/swarmkit/start-swarmkit-services.sh

SWARMMODEADVANTAGESSnaptoinstallSecurebydefaultBundledwithDocker

SWARMMODEDISADVANTAGESNew

SomeDockerfeaturesunsupported

DABstillWIP

NoconceptofPods

MESOSPredatesDocker

Usedbylargecompanies

Twitter,Airbnb,eBay...

Mixedload

RunHadoopjobsnexttoDockeretc

CORECOMPONENTSContainerizerMesosAgentsMesosMasterZooKeeperFrameworks

CONTAINERIZERResponsibleforrunningisolatedprocessInourcaseaDockerimageEitheruseDockerorinternalruntime

SIDENOTE-CONTAINERRUNTIMESBothMesosandKubernetesmovingawayfromDockerMesostoitsownruntimeK8sbecoming"runtimeagnostic"inrealityrktorrunc

Youprobablyshouldn'tcareAslongasitcanrunDocker/OCIimagesDeveloperswillstilluseDocker

MESOSAGENTSRunsonnodesresponsibleforexecutingtasksSubmitavailableresourcestomaster

MESOSMASTERSendstaskstoagentsOffersresourcestoframeworks

ZOOKEEPERDistributedHAdatabaseUsedprimarilyforkeepingtrackofcurrentmaster

FRAMEWORKSCoordinatewithmastertoscheduletasksMultipleframeworksavailableCanberuntogetherMarathon,Aurora,Chronos,ElasticSearch

RUNNINGSOCKSHOPhttps://raw.githubusercontent.com/microservices-demo/microservices-demo/master/deploy/mesos-

marathon/marathon.json

MESOSADVANTAGESBattle-testedGUIFrameworks

MESOSDISADVANTAGESNeeds3rdpartytoolsnetworkingloadbalancingservicediscovery

MarathonJSONcouldbecleaner

KUBERNETES

KUBERNETESBasedonGoogle'sexperiencerunningcontainersBakesinvariousfeaturesLoad-balancing,secretmanagement

Moreopinionated

CORECONCEPTSPodsFlatnetworkingspaceLabelsServicesDeploymentsReplicaSets

PODSGroupsofcontainersdeployedandscheduledtogetherAtomicunitContainersinapodshareIPaddressSinglecontainerpodsarecommonPodsareephemeral

FLATNETWORKINGSPACEAllpodsareinthesamenetworkspaceSimplecrosshostcommunication

LABELSK/Vpairsattachedtoobjects(primarilypods)e.g:"version:dev","tier:frontend"

LabelselectorsthenusedtoidentifygroupsUsedforload-balancingetc

SERVICESStableendpointsaddressedbynameForwardtraffictopodsPodsareselectedbylabelsRound-robinload-balancingSeparatesendpointfromimplementation

DEPLOYMENTS&REPLICASETSReplicaSetsmonitorstatusofPodsstart/stoppodsasneeded

DeploymentsstartReplicaSetsRollout/Rollback&Updates

RUNNINGSOCKSHOPhttps://github.com/microservices-demo/microservices-

demo/blob/master/deploy/kubernetes/complete-demo.yaml

KUBERNETESADVANTAGESAdvancedfeaturesbaked-inLotofmomentumbehindthecommunity

KUBERNETESDISADVANTAGESHardertogetstartedExtraconcepts

OTHERCONSIDERATIONSPersistentDataVolumes

SecretsAuto-scalingMulti-tenancyHealthchecksUpdatesandmaintenanceImagesandplatform

Federation

CONCLUSIONSDifferentoptionswithdifferentstrengthsInsomewayssurprisinglysimilarHardtopredictawinnerAllaremuchbetterthanrolling-your-own

SwarmMode

EasytodeploySecurebydefaultWIP

Mesos/Marathon

Battle-testedLackssomefeaturesFrameworks

KubernetesLotofmomentumAdvancedfeatures

REFERENCES

GooglePapers

SockShopReferenceApplicationTNSArticleonSockShop

Large-scaleclustermanagementatGooglewithBorg(2015)Omegaflexible,scalableschedulersforlargecomputeclusters(2013)Borg,Omega,andKubernetes(2016)