DevopsDevopsandand Drupal DrupalCurrent StateCurrent State

Kris Buytaert, march 2012Kris Buytaert, march 2012

Kris BuytaertKris Buytaert• I used to be a Dev,I used to be a Dev,• Then Became an OpThen Became an Op• Chief Trolling Officer and Open Source Chief Trolling Officer and Open Source

Consultant @inuits.euConsultant @inuits.eu• Everything is an effing DNS ProblemEverything is an effing DNS Problem• Building Clouds since before the bookstoreBuilding Clouds since before the bookstore• Some books, some papers, some blogsSome books, some papers, some blogs• But mostly, trying to be good at my jobBut mostly, trying to be good at my job

Devop, definitionDevop, definition● 30 something30 something

● Senior Infrastructure guySenior Infrastructure guy

● Development backgroundDevelopment background

● Open Source ExpcerienceOpen Source Expcerience

● Mostly European (.be / .uk) Mostly European (.be / .uk)

● Likes Belgian BeerLikes Belgian Beer

● Likes SushiLikes Sushi

What's this Devops thing really about ?What's this Devops thing really about ?

World , 200X-2009World , 200X-2009

Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and

lots of others ..lots of others ..

Gent , October 2009Gent , October 2009

Mountain View , June 2010Mountain View , June 2010

Hamburg , October 2010Hamburg , October 2010

Boston, March 2011Boston, March 2011

Mountain View, June 2011Mountain View, June 2011

Bangalore, Melbourne, Bangalore, Melbourne,

Goteborg , October 2011Goteborg , October 2011

● Devops is a growing movementDevops is a growing movement

● We don't have all the answers yetWe don't have all the answers yet

● We are reaching out to different communities We are reaching out to different communities

● We will point out problems we see..We will point out problems we see..

● Only the name is new Only the name is new

While we are still working out the solutionsWhile we are still working out the solutions

The community of developers whose work you The community of developers whose work you see on the Web, who probably don’t know what see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy ADO or UML or JPA even stand for, deploy betterbetter systems at systems at less costless cost in in less timeless time at at lower risklower risk than we see in the Enterprise. This is true even than we see in the Enterprise. This is true even when you factor in the greater flexibility and when you factor in the greater flexibility and velocity of startups.velocity of startups.

Tim Bray , on his blog January 2010Tim Bray , on his blog January 2010

What's the problem ?What's the problem ?

The real problem :The real problem :● Friday evening at 16:59Friday evening at 16:59

““Put this Code Live, here's a tarball” Put this Code Live, here's a tarball”

NOW! NOW!

● BackupsBackups ? ?

● What database ?What database ?

● Security ?Security ?

● High Availability ? High Availability ?

● Scalability ?Scalability ?

● Who is on Call ? Who is on Call ?

““devops is a cultural anddevops is a cultural and

professional movement”professional movement”Adam JacobAdam Jacob

CAMSCAMS● CultureCulture

● AutomationAutomation

● MeasurementMeasurement

● SharingSharing

Damon Edwards and John WillisDamon Edwards and John Willis

Why a survey ?Why a survey ?

• Gut feeling vs RealityGut feeling vs Reality

• The sad state of our customers practicesThe sad state of our customers practices

• Java folks GET the Problem ! Java folks GET the Problem !

• Twittersphere says it doesn't matter for PHPTwittersphere says it doesn't matter for PHP

• Lies , Damn Lies and StatisticsLies , Damn Lies and Statistics

AudienceAudience● 200+200+

● SchizophrenicSchizophrenic

● Experienced (6y Experienced (6y average)average)

● Webdeveloping Webdeveloping

● System System AdministratorsAdministrators

Organisation TypeOrganisation Type● In small to midsized In small to midsized

companiescompanies

● Good internal Good internal CommunicationCommunication

● Vs Large Enterprises Vs Large Enterprises with Silos ..with Silos ..

What's devops to them ?What's devops to them ?

• A buzzword A buzzword

• Developers who also do IT operations, or visa versa. Developers who also do IT operations, or visa versa.

• The people in charge of the build/release cycle and planning. The people in charge of the build/release cycle and planning.

• system administrators with a development culture. system administrators with a development culture.

• Someone who mixes both a sysop and dev duties Someone who mixes both a sysop and dev duties

• the combination of developer and operations into one overall the combination of developer and operations into one overall functionality functionality

What's devops to them ?What's devops to them ?

• A buzzword A buzzword

• Developers who also do IT operations, or visa versa. Developers who also do IT operations, or visa versa.

• The people in charge of the build/release cycle and planning. The people in charge of the build/release cycle and planning.

• system administrators with a development culture. system administrators with a development culture.

• Someone who mixes both a sysop and dev duties Someone who mixes both a sysop and dev duties

• the combination of developer and operations into one overall the combination of developer and operations into one overall functionality functionality

#

WRO

NG

What's devops to them ?What's devops to them ?• Breaking the wall between dev and ops in the same way agile Breaking the wall between dev and ops in the same way agile

breaks the wall between business and dev e.g. coming to terms breaks the wall between business and dev e.g. coming to terms with changing requirements, iterative cycleswith changing requirements, iterative cycles

• Sysadmin best-practise, using configuration as code, and Sysadmin best-practise, using configuration as code, and facilitating communication between sysadmins and developers, facilitating communication between sysadmins and developers, with each understanding and participating in the activities of the with each understanding and participating in the activities of the other.other.

• Devops is both the process of developers and system operators Devops is both the process of developers and system operators working closer together, as well as people who know (or who working closer together, as well as people who know (or who have worked in) both development and system operations.have worked in) both development and system operations.

• Removing barriers to communication and efficiency through Removing barriers to communication and efficiency through shared vocabulary, ideals, and business objectives to to deliver shared vocabulary, ideals, and business objectives to to deliver value.value.

• A set of principles and good practices to improve the interactions A set of principles and good practices to improve the interactions between Operations and Development.between Operations and Development.

Practice devops ?Practice devops ?● Lack of Clear Lack of Clear

DefinitionDefinition

● 33% does33% does

● 24% wants24% wants

● 22% is in trouble22% is in trouble

Know Continuous Know Continuous Delivery ?Delivery ?

NirvanaNirvanaAn “ecosystem” that supports continuous delivery, from An “ecosystem” that supports continuous delivery, from infrastructure, data and configuration management to infrastructure, data and configuration management to business.business.

Through automation of the build, deployment, and testing Through automation of the build, deployment, and testing process, and improved collaboration between developers, process, and improved collaboration between developers, testers, and operations, delivery teams can get changes testers, and operations, delivery teams can get changes released in a matter of hours — sometimes even minutes–no released in a matter of hours — sometimes even minutes–no matter what the size of a project or the complexity of its code matter what the size of a project or the complexity of its code base.base.

Continuous Delivery , Jez HumbleContinuous Delivery , Jez Humble

Site up to date ?Site up to date ?● 70% reads Drupal 70% reads Drupal

Security News .Security News .

● Sysadmins Update Sysadmins Update

● Other = Developers Other = Developers

•Really ? Really ?

● Not all components Not all components however :(however :(

Critical Bugfixes ?Critical Bugfixes ?

Typical EnviromentsTypical EnviromentsFor DevsFor Devs

● ScrumScrum

● Version ControlVersion Control

● Automated Build Automated Build

● Bugtracking Bugtracking

● Continous integrationContinous integration

● Integrated testingIntegrated testing

● Automated Automated deployment deployment

For OpsFor Ops

● KanbanKanban

● Version ControlVersion Control

● Automated Build Automated Build

● Bugtracking Bugtracking

● Continous integrationContinous integration

● Integrated testingIntegrated testing

● Automated Automated deployment deployment

What about the What about the Drupalistas ?Drupalistas ?

Version Control Version Control

Continuous IntegrationContinuous Integration

Testing Testing ● No frameworks usedNo frameworks used

vs vs

● No exploratory TestingNo exploratory Testing

Where do you develop ?Where do you develop ?● It works on my It works on my

machine :( machine :(

● What other platforms What other platforms do you use :do you use :

Looking for ?Looking for ?““As a system administrator, I can tell when software As a system administrator, I can tell when software vendors hate me. It shows in their products.”vendors hate me. It shows in their products.”

““DON'T make the administrative interface a GUI. System DON'T make the administrative interface a GUI. System administrators need a command-line tool for constructing administrators need a command-line tool for constructing repeatable processes. Procedures are best documented by repeatable processes. Procedures are best documented by providing commands that we can copy and paste from the providing commands that we can copy and paste from the procedure document to the command line. We cannot procedure document to the command line. We cannot achieve the same repeatability when the instructions are: achieve the same repeatability when the instructions are: "Checkmark the 3rd and 5th options, but not the 2nd "Checkmark the 3rd and 5th options, but not the 2nd option, then click OK." Sysadmins do not want a GUI that option, then click OK." Sysadmins do not want a GUI that requires 25 clicks for each new user.”requires 25 clicks for each new user.”

Thomas A. Limoncelli in ACM Queue December 2010Thomas A. Limoncelli in ACM Queue December 2010

http://queue.acm.org/detail.cfm?id=1921361http://queue.acm.org/detail.cfm?id=1921361

Deployment Deployment ● Database Imports are Database Imports are

EVILEVIL

● Manual Installations Manual Installations are are

● Error ProneError Prone

● Non Non ReproducibleReproducible

● 61% does it WRONG 61% does it WRONG Do you Test Deployments ?

How do you deploy ?

Deployment AlternativesDeployment Alternatives● Aegir (security ?)Aegir (security ?)

● FabricFabric

● SvnSvn git checkouts git checkouts (connectivity ?)(connectivity ?)

● CapistranoCapistrano

● Custom ToolsCustom Tools

How do you deploy ?

Deploying a Drupal SiteDeploying a Drupal Site● Drush make Drush make

● Installation ProfilesInstallation Profiles

● Features ,strongarm , Custom codeFeatures ,strongarm , Custom code

● Configuration as CodeConfiguration as Code

● What with external apps and libraries?What with external apps and libraries?

● Solr – BI softwareSolr – BI software

● Test before you drive! (CI) Test before you drive! (CI)

How do you configure modules ?How do you configure modules ?

● Not reproducableNot reproducable

● Not automatedNot automated

● 50% can improve :)50% can improve :)

● D8 Configuration D8 Configuration Mgmt Efforts !Mgmt Efforts !

Can you fully reproduce the last site you've Can you fully reproduce the last site you've build from code ?build from code ?

ChallengesChallenges● What about the data ?What about the data ?

● Content vs Config Content vs Config

● Drupal provides no clear distinction (yet)Drupal provides no clear distinction (yet)

● e.g a billing application e.g a billing application

● Table CreationTable Creation

● Provisioning (e.g. Rates, Call plan types)Provisioning (e.g. Rates, Call plan types)

● Actual Data Actual Data

Content EditingContent Editing● Non technical peopleNon technical people

● In productionIn production

● Plenty of UGCPlenty of UGC

● Frequent Updated Frequent Updated SitesSites

● Upgrade troubles Upgrade troubles aheadahead

Who edits content

Where do you edit content ?

Upgrading a siteUpgrading a site

● Drush Drush

● pm-updatecode pm-updatecode

● UpdatedbUpdatedb

● Features-updateFeatures-update

● Once again, test before you drive!Once again, test before you drive!

● But ... what about userdata ?But ... what about userdata ?

ScalabilityScalability

Proxy ?Proxy ?

WebServer ?WebServer ?

Caching / Scale / DeploymentCaching / Scale / Deployment

● MySQLMySQL ● NoSQLNoSQL ● MemCacheMemCache

Disaster Recovery ? Disaster Recovery ?

12 days into operations12 days into operations

The futureThe future● Don't forgetDon't forget

● Lies, Damn Lies and StatisticsLies, Damn Lies and Statistics

● These people CARED to fill in the surveyThese people CARED to fill in the survey

•D8 Configuration management initiativeD8 Configuration management initiative

•DrupalCon Munich infrastructure/devops track ! DrupalCon Munich infrastructure/devops track !

ConclusionsConclusions● Drupal gets is and is supportiveDrupal gets is and is supportive

● Conversation happensConversation happens

● Long Journey aheadLong Journey ahead

● Heading in the right directionHeading in the right direction

It's not about the tools It's not about the tools

It's about change It's about change

It's about the people It's about the people

ContactContactKris Buytaert Kris Buytaert Kris.Buytaert@inuits.euKris.Buytaert@inuits.eu

Further ReadingFurther Reading@krisbuytaert @krisbuytaert http://krisbuytaert.be/blog/http://krisbuytaert.be/blog/http://www.inuits.eu/http://www.inuits.eu/

InuitsInuitsDuboistraat 50Duboistraat 502060 Antwerpen2060 AntwerpenBelgiumBelgium891.514.231891.514.231

+32 475 961221+32 475 961221

Deploy Where ?Deploy Where ?

SearchSearch