#devops and #drupal,
have we learned anything yet?

Kris BuytaertDrupalCon Amsterdam , September 2014

Kris Buytaert

I used to be a Dev,

Then Became an Op

Chief Trolling Officer and Open Source Consultant @inuits.eu

Everything is an effing DNS Problem

Building Clouds since before the bookstore

Some books, some papers, some blogs

Evangelizing devops since < 2007

5 years ago

60 something people met in Ghent

The rest is history ...

devopsdays world wide

Analysts

Meetups

3rd year at DrupalCon

5th aniversary in Gent in 2 weeks..

But also

Vendors slapping #devops to all of their (irrelevant) products

Devops Engineers

Devops Teams

Devops Certification (pay me 10K and you'll get one too)

:(

Defining #devops

Defining #devops

Devs doing ops work

Defining #devops

Devs doing ops work

Ops writing code to automate infrastructure

Defining #devops

Devs doing ops work

Ops writing code to automate infrastructure

Continuous Integration/Delivery

Defining #devops

Devs doing ops work

Ops writing code to automate infrastructure

Continuous Integration/Delivery

Noops

Using the cloud

Defining #devops

ALL BROKEN ! Devs doing ops work

Ops writing code to automate infrastructure

Continuous Integration/delivery

noops

Using the cloud

Defining #devops

Plenty of broken definitions around

There is NO devops manifesto

It's not a role

It's not a JobTitle

It's not a new team

C(L)AMS

Culture

(Lean)

Automation

Measurement

SharingDamon Edwards and John Willis

Gene Kim

DevOps is a cultural andprofessional movementAdam Jacob

The real problem :

Friday evening at 16:59Put this Code Live, here's a tarball NOW!

Backups ?

What database ?

Security ?

High Availability ?

Scalability ?

Who is on Call ?

Devs vs Ops

Whats in it for you ?

Faster time to marketFeatures go live in hours vs years

In a more safe (Secure)

Reliable fashionFully automated

More happy {customers,developers,ops,managers,investors}

Is it really that bad ?

How about Drupal ?

Survey from 2012

Survey from 2014

Munich vs Austin

Adoption Grows

More starters

More adoptions

Less unknown

How do we get there ?

A software project is not done until your last enduser is in his grave !

Kris Buytaert, DOD Amsterdam 2013

Step 0

Establish communication between devs and ops

Engage Operations People in the projects

Talk about ideas , talk about potential tools

Talk about goal :

Faster and reliable deployment with less effort and reduced risk

Version all the things

No more excuses !

Source code Application

Source code Infrastructure

Builds

Tests

Pipelines

Scripts

Documentation

Monitoring scripts

Version Control

Up 10%

More full version control

Git all the way

What's wrong with the other 4% ?

Continuous Integration

Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day. It was first named and proposed as part of extreme programming (XP). Its main aim is to prevent integration problems, referred to as "integration hell" (WikiPedia) Does the app you are deploying still work ?Did you break your infrastructure ?

Nirvana

An ecosystem that supports continuous delivery, from infrastructure, data and configuration management to business.Through automation of the build, deployment, and testing process, and improved collaboration between developers, testers, and operations, delivery teams can get changes released in a matter of hours sometimes even minutesno matter what the size of a project or the complexity of its code base.Continuous Delivery , Jez Humble

CI vs CD vs CD

devops ( continuous delilvery

Build Pipelines

Build an Empty Pipeline

Start building an empty pipeline

Push Hello World to production

Add Tests

Iterate

Grow your Artifacts

artifacts go trough a pipeline

Unmodifiedapplication code,

Infra code

Metadata

tests

CI Usage

Jenkins usage doubled

Still 40%+ not running any form of CI

Can you sell the effort to build CI vs Can you afford not to do CI

The drupal Dilemma

What's in your Pipeline ?

A pipeline

Checkout code

Syntax Check

Style Check

Code Coverage

Tests

Testing Requirements

Infrastructure as code

Bulk Provisioning

Product = Infra+Application Stack

Provisioning

UGC

Stable Starting Point

Run Tests

Verify Results

Rerun Tests

If my computer can't install it , the installer is borken Luke Kanies at Fosdem (2007)

Infrastructure as Code

Treat configuration automation as code

Development best practicesModel your infrastructure

Version your cookbooks / manifests

Test your cookbooks/ manifests

Dev/ test /uat / prod for your infra

Model your infrastructure

A working service = automated ( Application Code + Infrastructure Code + Security + Monitoring )

IAC -ne Advanced scripting

If my computer can't deploy your site, it isn't worth deploying.Kris Buytaert, DrupalCon Munich 2012

Site deployment

Features adoption is up 18%

Manual config from gui is down 27%

Site deployment

Site profile install improved from < 1% to 6%

Database installations have decreased by 9%

Features usage increased by 9%

A pipeline

Checkout code

Syntax

Style

Code Coverage

Tests

Build

More Tests

Package

Upload to Repo

Deploy on Test

Testing = Monitoring

Deploy a host,

Add it to the monitoring framework

Add collection tools

Add check definitions

Update the monitoring tool config

FULLY AUTOMATED

Test all the things

Unit tests

Integration Tests

System Tests

Acceptance Tests

Security Tests

Performance Tests

Regression Tests

Functional Tests

Drupal and Testing

Given the lack of CI, mostly manual tests

Focus on Performance, usability and GUI testing.

Drupal and Testing

Still 50% not using a Framework to test

Selenium at 20+%

Cucumber++

Behat++

Build Promotion

A pipeline

Checkout code

Syntax

Style

Code Coverage

Code Review

Unit Tests

Build

More Tests

Package

Upload to Repo

Promote

Deploy on Test

More Tests

Promote

Deploy on UAT

More Tests

Promote

Deploy on Prod

Done ?

Close the feedback loop,

Send metric on deployment

echo "deployed.$package_name 1 `date +%s` | nc /2003

Dashboards

Visualize Business Metrics

$revenue

#sales

signups

conversions

Api calls

Application use

Continuous Improvement

It's not about the tools It's about change It's about the people

Homework

#drupal and #devops BOF Wednesday 11:45 13:00 Room : G111 Adyax#devops Meetup AmsterdamTuesday 18:00Omnibus, Gitlab, Docker , Culture Oud Amsterdam (around the corner) #docker Meetup AmsterdamBen Hindman about MesosWednesday 19:00 ING Haarlemmerweg

Contact

Kris BuytaertKris.Buytaert@inuits.eu

Further Reading@krisbuytaert http://www.krisbuytaert.be/blog/http://www.inuits.eu/

Inuits

Duboistraat 502060 AntwerpenBelgium

+32 475 961221