Moby is killing your devops efforts

Kris Buytaert

@krisbuytaert

NOT this guy !NOT this guy !

Kris BuytaertKris Buytaert● I used to be a Dev,I used to be a Dev,● Then Became an OpThen Became an Op● CTO and Open Source Consultant CTO and Open Source Consultant

@@inuits.euinuits.eu● Everything is a freaking DNS Everything is a freaking DNS

ProblemProblem● Evangelizing devopsEvangelizing devops● Organiser of #devopsdays, Organiser of #devopsdays,

#cfgmgmtcamp, #loadays, ….#cfgmgmtcamp, #loadays, ….

Today's Plan:Today's Plan:● Opening some eyesOpening some eyes

● Reusing 8 year old slidesReusing 8 year old slides

C(L)AMSC(L)AMS● CultureCulture● (Lean)(Lean)● AutomationAutomation● MeasurementMeasurement● SharingSharing

Damon Edwards and John Damon Edwards and John WillisWillis

A global movement to improve the A global movement to improve the quality of software delivery leveraging quality of software delivery leveraging

Open Source experience, started in Gent Open Source experience, started in Gent in 2009in 2009

Because the old days: Because the old days: ● ““Put this Code Live, here's a tarball” NOW! Put this Code Live, here's a tarball” NOW!

● What dependencies ?What dependencies ?

● No machines available ?No machines available ?

● What database ?What database ?

● Security ?Security ?

● High Availability ? High Availability ?

● Scalability ?Scalability ?

● My computer can't install this ?My computer can't install this ?

10 days into operations10 days into operations

This new 'D' hypeThis new 'D' hype● New kid on the block New kid on the block

● Vagrant-lxc with a nice cliVagrant-lxc with a nice cli

● The Ultimate “devops tool” The Ultimate “devops tool”

● ““Unseen” growth Unseen” growth

● Docker is the new cool thing to doDocker is the new cool thing to do

DockerDockerDockerDocker

DockerDockerDockerDocker

DockerDockerDockerDocker

MobyMobyMobyMoby

MobyMobyMobyMoby

MobyMobyMobyMoby

People think I`m the whalePeople think I`m the whale

Some History Some History LessonsLessons

openMosix openMosix (+/- 2001-2005)(+/- 2001-2005)

● 2.4 Kernel patch,2.4 Kernel patch,

● Loadbalance multiple similar processesLoadbalance multiple similar processes

● Both short (compilations) and longlived Both short (compilations) and longlived (calculations) processes(calculations) processes

● Parts of processes migrated to other nodesParts of processes migrated to other nodes

● OMFS allowed identical access to data from all OMFS allowed identical access to data from all nodesnodes

openMosix LimitationsopenMosix Limitations● shared memoryshared memory

• Limited type of applications could migrateLimited type of applications could migrate

• Patches available but not successfulPatches available but not successful

● Filesystem access Filesystem access

oMFS : unstable => removed oMFS : unstable => removed

Lessons from openMosixLessons from openMosix● Applications need to be adaptedApplications need to be adapted

● Files need to be accessible (oMFS)Files need to be accessible (oMFS)

● Huge gap between developers and consumers Huge gap between developers and consumers

=> Limited working use cases=> Limited working use cases

Linux-HA Linux-HA (2005- now) (2005- now)

● Hearbeat, Heartbeat v2, Pacemaker , Corosync Hearbeat, Heartbeat v2, Pacemaker , Corosync

● Define resource (OCF)Define resource (OCF)

● Define constraints, clones, colocation, …Define constraints, clones, colocation, …

● Long running services that should not stopLong running services that should not stop

Working with developersWorking with developers● State vs StatelessState vs Stateless

• Most applications have state somewhereMost applications have state somewhere

• Discuss how to share/access stateDiscuss how to share/access state

● Data accessData access

• Local filesystem is kingLocal filesystem is king

• We needed distributed / shared filesystemsWe needed distributed / shared filesystems

● HealthHealth

● Metrics Metrics

Lessons from Linux-HALessons from Linux-HA● Applications need to be adaptedApplications need to be adapted

● Files need to be accessible (shared/distributed Files need to be accessible (shared/distributed FS)FS)

● Monitoring strategy needs changeMonitoring strategy needs change

● Good collaboration between developers and Good collaboration between developers and ops folks ops folks

=> Many general purpose use cases => Many general purpose use cases

● Works better with an Open Source MindsetWorks better with an Open Source Mindset

Cloud Adoption is (s)lowCloud Adoption is (s)low● Enterprises are afraid of cloud Enterprises are afraid of cloud

• Security, Cost, Control, Stability, (insert more Security, Cost, Control, Stability, (insert more fud here)fud here)

● ““Private cloud” will solve this Private cloud” will solve this

● Please fill in 4 word documents for for each VM Please fill in 4 word documents for for each VM you wantyou want

Failed (Private) Cloud Failed (Private) Cloud ProjectsProjects

● Identical copies of Bare Metal or vm's moved Identical copies of Bare Metal or vm's moved to the cloudto the cloud

● No config managementNo config management

● No monitoringNo monitoring

● No resilienceNo resilience

● No API usage No API usage

Why ?Why ?● I want a VMI want a VM

• Please fill in these 4 formsPlease fill in these 4 forms

• Wait 5 weeksWait 5 weeks

● RepeatRepeat

IT Departments have not adapted,IT Departments have not adapted,

'Shadow'-IT is winning'Shadow'-IT is winning

DockerDockerDockerDocker

DockerDockerDockerDocker

DockerDockerDockerDocker

A Enterprise ContainerA Enterprise Container● No different from a No different from a

full vmfull vm

● Multiple services Multiple services running in one running in one containercontainer

● Ssh is the default Ssh is the default connectionconnection

Elephant , Monolith Elephant , Monolith

vs vs

MicroservicesMicroservices

Docker is a Package managerDocker is a Package manager

These DaysThese Days● ““Put this Code Live, here's a Docker Put this Code Live, here's a Docker

Container ” Container ”

● No machines available ?No machines available ?

● What database ? Where to store the What database ? Where to store the data ?data ?

● Security ? What distro is this even ? Security ? What distro is this even ? Bad Cows ? Bad Cows ?

● How do we monitor his ?How do we monitor his ?

● Backups ?Backups ?

● How did you build this ?How did you build this ?

11 days into operations11 days into operations● ““Put this Code Live, here's Dockerfile” Put this Code Live, here's Dockerfile”

● What corporate proxy ?What corporate proxy ?

● Oh I missed 2 other containersOh I missed 2 other containers

● Security ? What distro is this even ? Security ? What distro is this even ? Bad Cows ? Bad Cows ?

● What do you mean “We are a RHEL What do you mean “We are a RHEL shop ?”shop ?”

Closing the gaps between dev Closing the gaps between dev and ops, and ops, AGAIN !!AGAIN !!

● Where do your containers come from ? Where do your containers come from ?

● Who build it ? Who build it ?

● Can you rebuild it ?Can you rebuild it ?

● Do you even need a containerDo you even need a container

● How do you build the hosts that run the How do you build the hosts that run the containers ?containers ?

● Infrastructure as code ++Infrastructure as code ++

Operations teams catching upOperations teams catching up

Image Build by devs, Image Build by devs, maintained by nobodymaintained by nobody

Can you ?Can you ?● When GitHub is down ? When GitHub is down ?

● When rubygems.org is When rubygems.org is down ?down ?

● When someone removes When someone removes a Node.js library ?a Node.js library ?

● Fix critical Security Fix critical Security Issues ?Issues ?

● Can your business surive Can your business surive if you answer NO to if you answer NO to these questions ?these questions ?

Going to ProductionGoing to Production

Dev vs ProdDev vs Prod● 1 local dev machine1 local dev machine

● 1 local application1 local application

● A clusterA cluster

• SwarmSwarm

• KuberenetesKuberenetes

• MesosMesos

• Nomad Nomad

● Other ApplicationsOther Applications

Dev vs ProdDev vs Prod● Localhost :8081Localhost :8081 ● HttpsHttps

● DnsDns

● IngressIngress

● LoadBalancingLoadBalancing

● Service DiscoveryService Discovery

● Or Vendor Lock In Or Vendor Lock In

Dev vs ProdDev vs Prod● Local diskLocal disk

● No real dataNo real data

● Actual customer dataActual customer data

● Distributed StorageDistributed Storage

• 2017 is the year of 2017 is the year of NFS (again)NFS (again)

● Object StorageObject Storage

● BackupsBackups

Dev vs ProdDev vs Prod● Random image from Random image from

the internetthe internet

+ Some local files+ Some local files

● CI PipelineCI Pipeline

● Image RegistryImage Registry

● Security ScanningSecurity Scanning

Dev vs ProdDev vs Prod● Works on my Works on my

machinemachine

● docker pull yolodocker pull yolo

● MonitoringMonitoring

● MetricsMetrics

● SecuritySecurity

● I love docker as a I love docker as a technologytechnology

● I hate that it too often I hate that it too often put us back 5 years put us back 5 years with regards to with regards to Culture adoptionCulture adoption

● Docker is an easy Docker is an easy victim, victim,

● It's still mostly about It's still mostly about CultureCulture

It's still about It's still about collaborationcollaboration

ContactContactKris Buytaert Kris.Buytaert@inuits.euKris Buytaert Kris.Buytaert@inuits.eu

Further ReadingFurther Reading@krisbuytaert @krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/http://www.inuits.eu/http://www.inuits.eu/

Inuits.euInuits.eu

Essensteenweg 31Essensteenweg 31BrasschaatBrasschaatBelgiumBelgium891.514.231891.514.231

+32 475 961221+32 475 961221

Picture CreditsPicture Creditshttps://www.flickr.com/photos/simoncarr/81918853/https://www.flickr.com/photos/simoncarr/81918853/

https://www.flickr.com/photos/tonynetone/8116629238/https://www.flickr.com/photos/tonynetone/8116629238/

https://www.flickr.com/photos/damienz/8665192113/in/photolist-ecHnJ2-x6tV12-63pJ8G-RTfSuB-rud69Z-4SuDfp-orEF86-odnENj-f4dtxn-o7L3LQ-63ktuB-https://www.flickr.com/photos/damienz/8665192113/in/photolist-ecHnJ2-x6tV12-63pJ8G-RTfSuB-rud69Z-4SuDfp-orEF86-odnENj-f4dtxn-o7L3LQ-63ktuB-rjJUYG-qfSqHA-osWS38-75uUJT-9qrqeC-oKLaaC-qMy9Nf-oqmerS-p1yUcq-fPo32k-Vtc1v6-oarY18-fvH2gR-otbpNC-TGNoW5-oqe8vR-fxJnpw-5SMhm5-oBiLUG-rjJUYG-qfSqHA-osWS38-75uUJT-9qrqeC-oKLaaC-qMy9Nf-oqmerS-p1yUcq-fPo32k-Vtc1v6-oarY18-fvH2gR-otbpNC-TGNoW5-oqe8vR-fxJnpw-5SMhm5-oBiLUG-pgCMkw-C77xyw-7RjQ9-Lja9B-rvPLg-4dCndQ-TyEc3Q-oLDg4s-soHSub-fcZsZS-cCheFS-bLgJpe-VgEBAF-rBhHfA-VpEAf1-bt1YME-8zoXyQ-6y1Y9B-f4EAfb-pgCMkw-C77xyw-7RjQ9-Lja9B-rvPLg-4dCndQ-TyEc3Q-oLDg4s-soHSub-fcZsZS-cCheFS-bLgJpe-VgEBAF-rBhHfA-VpEAf1-bt1YME-8zoXyQ-6y1Y9B-f4EAfb-Tay1WgTay1Wg