DEVOPS OVERVIEW

KRYSTALB SOLUTIONS Cloud IT Better

INTRODUCTION OF DEVOPS

DEVOPS HAS BECOME AN INCREASINGLY POPULAR TECHNIQUE FOR SHEPHERDING SOFTWARE FROM THE DESIGN PHASE THROUGH DEVELOPMENT AND TESTING, AND THEN ALL THE WAY INTO PRODUCTION. THE BUSINESS VALUE OF DEVOPS IS QUITE PROFOUND: DEVOPS REDUCES SOFTWARE DELIVERY TIMES, IMPROVES APPLICATION QUALITY, AND ENHANCES THE PRODUCTIVITY OF THE DEVELOPMENT AND OPERATIONS TEAMS. IN RECOGNITION OF THIS TREND, MANY ORGANIZATIONS ARE INTERESTED IN AUTOMATING THEIR SOFTWARE RELEASE AND DEPLOYMENT PROCESSES. HOWEVER, BY FOCUSING EXCLUSIVELY ON THE “PRODUCTION” RELEASE/DEPLOYMENT PROCESS – A TASK PRIMARILY MANAGED BY OPERATIONS (OPS), ORGANIZATIONS ARE NOT FULLY LEVERAGING THE VALUE THAT DEV CAN BRING TO DEVOPS.

WHAT IS DEVOPS ?

FIRST, LET’S JUST SAY THERE IS NO DEFINITIVE ANSWER. YET. THERE ARE LOTS OF OPINIONS ABOUT WHAT IS COVERED UNDER DEVOPS AND WHAT’S NOT. IS IT A CULTURE? IS IT A JOB TITLE? IS IT A WAY OF ORGANIZING? OR JUST A WAY OF THINKING? WE THINK IT’S A STILL-EVOLVING MOVEMENT SO LET’S NOT GET STUCK ON LIMITING IT TOO MUCH RIGHT NOW. INSTEAD, WE CAN TALK ABOUT SOME OF THE COMMON THEMES, TOOLS AND IDEAS.

BORN OF THE NEED TO IMPROVE IT SERVICE DELIVERY AGILITY, THE DEVOPS MOVEMENT EMPHASIZES COMMUNICATION, COLLABORATION AND INTEGRATION BETWEEN SOFTWARE DEVELOPERS AND IT OPERATIONS. RATHER THAN SEEING THESE TWO GROUPS AS SILOS WHO PASS THINGS ALONG BUT DON’T REALLY WORK TOGETHER, DEVOPS RECOGNIZES THE INTERDEPENDENCE OF SOFTWARE DEVELOPMENT AND IT OPERATIONS AND HELPS AN ORGANIZATION PRODUCE SOFTWARE AND IT SERVICES MORE RAPIDLY, WITH FREQUENT ITERATIONS.

THE PERFECT STORM OF 2009.

A PERFECT STORM OF CONVERGING ADJACENT METHODOLOGY INCLUDING AGILE, OPERATIONS MANAGEMENT (SYSTEMS THINKING & DYNAMICS), THEORY OF CONSTRAINTS, LEAN AND IT SERVICE MANAGEMENT CAME TOGETHER IN 2009 THROUGH A SMATTERING OF CONFERENCES, TALKS AND TWITTER (#DEVOPS) DEBATES WORLDWIDE THAT EVENTUALLY BECAME THE PHILOSOPHY BEHIND DEVOPS.

AGILE SOFTWARE DEVELOPMENT PAVED THE WAY, STEERING AWAY FROM THE WATERFALL METHOD OF SOFTWARE DEVELOPMENT TOWARD A CONTINUOUS DEVELOPMENT CYCLE. BUT IT DIDN’T INCLUDE THE OPERATIONS SIDE SO WHILE DEVELOPMENT COULD BE CONTINUOUS, DEPLOYMENT WAS STILL WATERFALL-ORIENTED.

IN A DEVOPS ENVIRONMENT, CROSS FUNCTIONALITY, SHARED RESPONSIBILITIES AND TRUST ARE PROMOTED. DEVOPS ESSENTIALLY EXTENDS THE CONTINUOUS DEVELOPMENT GOALS OF THE AGILE MOVEMENT TOCONTINUOUS INTEGRATION AND RELEASE. IN ORDER TO ACCOMMODATE CONTINUOUS RELEASES, DEVOPS ENCOURAGES AUTOMATION OF THE CHANGE, CONFIGURATION AND RELEASE PROCESSES.

MODERN APPLICATIONS IN THE CLOUD AND OUT.

DEVOPS FOUND INITIAL TRACTION WITHIN MANY LARGE PUBLIC CLOUD SERVICE PROVIDERS. WITH MODERN APPLICATIONS RUNNING IN THE CLOUD, MUCH OF WHAT USED TO BE CONSIDERED INFRASTRUCTURE IS NOW PART OF THE CODE. CLASSIC BIG WEBOPS SHOPS LIKE GOOGLE, AMAZON, TWITTER AND ETSY ARE KNOWN TO DO DEPLOYMENTS MULTIPLE TIMES A DAY. IN ORDER TO DEPLOY THAT OFTEN, YOU HAVE TO KNOW YOU’RE NOT GOING TO BREAK WHAT’S ALREADY WORKING. DEVOPS HELPS ENSURE FREQUENT DEPLOYS WITH A LOW FAILURE RATE.

COMPANIES OF ALL SIZES ARE BEGINNING TO IMPLEMENT DEVOPS PRACTICES, WITH A 2012 SURVEY BY PUPPET LABS AND IT REVOLUTION PRESS SHOWING THAT 63% OF OVER 4,000 RESPONDENTS ARE IMPLEMENTING DEVOPS PRACTICES. AND MANY SHOPS, PARTICULARLY LEAN STARTUPS, HAVE BEEN “DOING DEVOPS” WITHOUT CALLING IT DEVOPS FOR QUITE A WHILE.

DEVOPS TOOLS

THE AGILE MANIFESTO EMPHASIZES INDIVIDUALS AND INTERACTIONS OVER PROCESSES AND TOOLS, BUT DEVOPS, WHILE EMPHASIZING COLLABORATION AND INTEGRATION ALSO LOOKS TO AUTOMATION TOOLS THAT CAN LEVERAGE AN INCREASINGLY PROGRAMMABLE AND DYNAMIC INFRASTRUCTURE FROM A LIFECYCLE PERSPECTIVE. VERSION CONTROL AND AUTOMATING CODE DEPLOYMENTS ARE TWO OF THE MOST IMPACTFUL COMMON TOOLS BUT THERE ARE MANY MORE, INCLUDING CONFIGURATION MANAGEMENT, TICKETING SYSTEMS, MONITORING AND PROVISIONING.

WHY DO DEVOPS ?

THE BENEFITS OF A DEVOPS APPROACH ARE MANY, INCLUDING:

• IMPROVED DEPLOY FREQUENCY WHICH CAN LEAD TO FASTER TIME TO MARKET

• LOWER FAILURE RATE

• SHORTENED LEAD TIME

• FASTER MEAN TIME TO RECOVERY

BENEFITS OF DEVOPS

COMPANIES THAT INCORPORATE DEVOPS PRACTICES GET MORE DONE, PLAIN AND SIMPLE. THEY DEPLOY CODE UP TO 30 TIMES MORE FREQUENTLY THAN THEIR COMPETITION. AND LESS THAN 50% OF THEIR DEPLOYMENTS FAIL ACCORDING TO PUPPET LABS 2013 STATE OF DEVOPS SURVEY.

THE BIGGEST SHIFT IN ATTITUDE IN A DEVOPS ENVIRONMENT IS THAT THERE IS ONE TEAM COMPOSED OF CROSS-FUNCTIONAL TEAM MEMBERS INCLUDING DEVELOPERS, QA, DBAS, BUSINESS ANALYSTS, OPERATIONS ENGINEERS AND SO FORTH. COLLABORATION ACROSS THESE DIFFERENT ROLES DELIVERS MANY BENEFITS.

TECHNICAL BENEFITS:

• CONTINUOUS SOFTWARE DELIVERY

• LESS COMPLEX PROBLEMS TO FIX

• FASTER RESOLUTION OF PROBLEMS

BUSINESS BENEFITS:

• FASTER DELIVERY OF FEATURES

• MORE STABLE OPERATING ENVIRONMENTS

• MORE TIME AVAILABLE TO ADD VALUE (RATHER THAN FIX/MAINTAIN)

MORE DEPLOYS MEANS FASTER TIME-TO-MARKET AND CONTINUAL PROVEMENT.

CONTINUOUS SOFTWARE DELIVERY BRINGS TWO HUGE FACTORS OF SUCCESS TO AN ORGANIZATION. FIRST, YOU CAN GO FROM “IDEA” TO WORKING SOFTWARE FASTER IN TERMS OF INITIAL PROJECT DEVELOPMENT. THEN YOU CAN EXPERIMENT ON MANY DIFFERENT THINGS FOR CONTINUOUS INCREMENTAL IMPROVEMENTS OF WHATEVER YOUR COMPANY MEASURES, BE IT PERFORMANCE, SALES OR SIGNUPS.

YOU DON’T HAVE TO CHOOSE STABILITY VERSUS NEW FEATURES.

IN A NON-DEVOPS ENVIRONMENT, THERE IS OFTEN TENSION BETWEEN INTRODUCING NEW FEATURES AND STABILITY. THE DEVELOPMENT TEAM IS MEASURED ON THE FEATURES THEY DELIVER TO USERS WHILE THE OPERATIONS TEAM IS MEASURED ON THE STABILITY OF THE SYSTEM.

IN A DEVOPS ENVIRONMENT, A SINGLE TEAM IS RESPONSIBLE FOR DELIVERING NEW FEATURES AND STABILITY. THE COMBINATION OF A SHARED CODE BASE, CONTINUOUS INTEGRATION, TEST-DRIVEN TECHNIQUES AND AUTOMATED DEPLOYS, AMONG OTHER THINGS, EXPOSE PROBLEMS, IN APPLICATION CODE, INFRASTRUCTURE OR CONFIGURATION, EARLIER BECAUSE THE SOFTWARE ISN’T “THROWN OVER THE WALL” TO OPERATIONS AT THE END OF CODING. PROBLEMS TEND TO BE LESS COMPLEX BECAUSE CHANGE SETS ARE SMALLER. AND RESOLUTION TIMES ARE FASTER BECAUSE TEAM MEMBERS DON’T NEED TO WAIT FOR A DIFFERENT TEAM TO TROUBLESHOOT AND FIX THE PROBLEM.

INCREASED EFFECTIVENESS.

THERE IS ENORMOUS WASTE IN A TYPICAL IT ENVIRONMENT WITH PEOPLE WAITING FOR OTHER PEOPLE, OTHER MACHINES, NEW SOFTWARE OR THEY ARE STUCK SOLVING THE SAME PROBLEMS OVER AND OVER. PEOPLE LIKE TO BE PRODUCTIVE IN THEIR WORK AND THE TIME SPENT CHURNING CAUSES FRUSTRATION AND UNHAPPINESS. WHEN PEOPLE GET RID OF THE UNSATISFYING PARTS OF THEIR JOB AND CAN INSTEAD SPEND THAT TIME ADDING VALUE TO THE ORGANIZATION, EVERYONE BENEFITS.

AUTOMATED DEPLOYMENTS AND STANDARDIZED PRODUCTION ENVIRONMENTS, KEY ASPECTS OF DEVOPS, MAKE DEPLOYMENTS PREDICTABLE AND FREE PEOPLE FROM ROUTINE REPETITIVE TASKS TO GO DO MORE CREATIVE THINGS.

THE DEVOPS LIFECYCLE: KEEP C.A.L.M. AND CARRY ON

ONCE UPON A TIME THERE WERE TWO TEAMS IN CHARGE OF CREATING GREAT SOFTWARE: DEV AND OPS. THOUGH THEY WORKED ON THE SAME PRODUCT, THEIR GOALS WERE DIAMETRICALLY OPPOSED TO EACH OTHER. ON ONE HAND THERE WAS THE DEV TEAM PUSHING FOR FEATURE CHANGES, AND ON THE OTHER HAND WAS THE OPS TEAM STRIVING FOR STABILITY.

SINCE THE ADVENT OF THE PERSONAL COMPUTER IN THE 80’S THESE TEAMS HAVE BEEN SILOED AND BURDENED WITH DYSFUNCTIONAL COMMUNICATION DURING A PRODUCT LIFECYCLE THAT HAS HISTORICALLY LED TO DELAYS AND BROKEN CODE. IT WAS A SAD TALE TO TELL TO END USERS EVERYWHERE.

TODAY, THINGS HAVE CHANGED, RADICALLY. AN ENTIRE MOVEMENT WAS FORMED BY NEW TECHNOLOGY LIKE CLOUD INFRASTRUCTURE AND VIRTUAL MACHINES. THIS CULTURAL CHANGE IN PERSPECTIVE COMBINES THE TWO TEAMS INTO ONE LEAN MEAN RAPID DEPLOYMENT MACHINE, LEVERAGING CODE TO MANAGE THE INFRASTRUCTURE. IT IS CALLED DEVOPS.

TRANSPARENCY, COLLABORATION AND CROSS-FUNCTIONAL TEAMS WITH POLYGLOT SKILLS ARE BREAKING DOWN THE WALLS WITH AUTOMATION AND RAPID DEPLOYMENT. SOFTWARE GETS SHIPPED QUICKLY, MORE OFTEN, CODE FAILURES ARE DETECTED AND CORRECTED FASTER AND THE PRODUCT RUNS SMOOTHER. THIS SPED UP PROCESS ALLOWS INNOVATION TO FLOURISH AND COMPANIES TO DO MORE IN LESS TIME.

“BUT,” YOU MIGHT ASK, “HOW IS THE DEVOPS LIFECYCLE DIFFERENT?” IT CAN BE SUMMED UP WITH THE ACRONYM C.A.L.M.S.

• C – CULTURE

• A – AUTOMATION

• L – LEAN

• M – MEASUREMENT

• S – SHARING

THE TRADITIONAL SOFTWARE DEVELOPMENT LIFECYCLE FOLLOWS A WATERFALL METHODOLOGY THAT EVENTUALLY MORPHED INTO THE AGILE SCRUM LIFECYCLE. BUT FOR MOST ENTERPRISES THE CURRENT LIFECYCLE RESEMBLES SOMETHING LIKE AN AGILE-SCRUM-FALL.

THE DEVOPS LIFECYCLE LOOKS LIKE THIS:

1. CHECK IN CODE

2. PULL CODE CHANGES FOR BUILD

3. RUN TESTS (CONTINUOUS INTEGRATION SERVER TO GENERATE BUILDS AND ARRANGE RELEASES): TEST INDIVIDUAL MODELS, RUN INTEGRATION TESTS, AND RUN USER ACCEPTANCE TESTS.

4. STORE ARTIFACTS AND BUILD REPOSITORY (REPOSITORY FOR STORING ARTIFACTS, RESULTS, AND RELEASES)

5. DEPLOY AND RELEASE (RELEASE AUTOMATION PRODUCT TO DEPLOY APPS)

6. CONFIGURE ENVIRONMENT

7. UPDATE DATABASES

8. UPDATE APPS

9. PUSH TO USERS – WHO RECEIVE TESTED APP UPDATES FREQUENTLY AND WITHOUT INTERRUPTION

10. APPLICATION AND NETWORK PERFORMANCE MONITORING (PREVENTIVE SAFEGUARD)

11. RINSE AND REPEAT

THE DEVOPS LIFECYCLE = A RAPID RELEASE CYCLE WITH A STRONG FEEDBACK LOOP.

THE DEVOPS LIFECYCLE

= A RAPID RELEASE CYCLE WITH A STRONG FEEDBACK LOOP.

UTILIZING A DEVOPS LIFECYCLE, PRODUCTS CAN BE CONTINUOUSLY DEPLOYED IN A FEEDBACK LOOP THROUGH:

• INFRASTRUCTURE AUTOMATION

• CONFIGURATION MANAGEMENT

• DEPLOYMENT AUTOMATION

• INFRASTRUCTURE MONITORING

• LOG MANAGEMENT

• APPLICATION AND PERFORMANCE MANAGEMENT

Tools for DevOpsHere are some of the DevOps tools that we use at Krystalb and a few

more worth mentioning.

Operating Systems• Linux (RHEL,

CentOS,Ubuntu, Debian)

• Unix (Solaris, AIX,HP/UX, etc.)

• Windows• Mac OS X

Infrastructure as a Service• Amazon Web Servic

es

• Rackspace • Cloud Foundry • Azure • OpenStack

Virtualization Platforms• VMware • KVM• Xen• VirtualBox • Vagrant

Linux OS Installation• Kickstart • Cobbler • Fai

Application Deployment• Capistrano

Application Servers• JBoss• Tomcat• Jetty • Glassfish• Websphere• Weblogic

Configuration Management• Puppet /

MCollective • Chef• Ansible• CFEngine • SaltStack• RANCID • Ubuntu Juju

Web Servers• nginx • Apache • IIS

Containerization Tools• LXC• Solaris Containers• Docker

Test and Build Systems• Solano • Jenkins • Maven • Ant • Gradle

Queues, Caches, etc.• ActiveMQ• RabbitMQ • memcache• varnish• squid

Databases• Percona Server • MySQL• PostgreSQL• OpenLDAP• MongoDB • Cassandra• Redis • Oracle• MS SQL

Process Supervisors• Monit • runit• Supervisor• god• Blue Pill• Upstart• systemd

Monitoring, Alerting, and Trending• New Relic • Nagios • Icinga• Graphite• Ganglia• Cacti • PagerDuty • Sensu

Miscellaneous Tools• Multihost SSH Wrap

per

• Code Climate• iPerf • lldpd

Logging• PaperTrail • Logstash• Loggly • Logentries • Splunk• SumoLogic

Security• Snorby Threat Stac

k

• Tripwire• Snort