automated deployments

1 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2014

Orlando, October 2014

Automated Deployments – Hands-On Martin Etmajer

[email protected] Technology Strategist @

Dynatrace


Insert headshot image here “Martin Etmajer

Senior Technology Strategist @ Dynatrace

[email protected]

@metmajer


From Continuous Integration toContinuous Delivery


Continuous Integration


Continuous Delivery Build Pipeline



✔



✔

✔



✔

✔ ✔



✔

✔ ✔ ✔



✔

✔ ✔ ✔

✔


Introduction toAutomated Deployments


Motivation

Production environments are often grown “works of art”:

» Not entirely reproducible

» Manual changes applied whenever needed

» Not even similar to the environments developers have

“Works on my machine.”


Benefits: Automation


Makes provisioning a repeatable, low-risk “push-button” activity

» By providing executable specifications

» No manual changes involved (regarded immutable)

» The process is tested with each execution (builds confidence)


RecreateEnvironments


Makes provisioning an engineering discipline:

» Iteratively plan, code, test and verify

» Open to frequently changing requirements (agile)

Helps align efforts between Development and Operations:

» Jointly define desired environmental states

» Integrate and respect each others processes



OPERATIONS

DEVELOPMENT


current iteration(2 weeks)

time

Planning

Implementand test Verification



Allows the provisioning process to be integrated into the Continuous Delivery build pipeline (build automation server):

» Environments can be provisioned multiple times a day

» When the process fails in staging, the release candidate is discarded - the production environment will not be harmed



Reduces the risks of software releases by providing a consistent process for all staging and production environments

Risks?

» Differing library versions

» Differing middleware configurations

» Differing environmental variables in the OS

» Differing number of max. open file handles in the OS,…


Benefits: Auditability


Benefits: Auditability

What, who, why and when?

» Keep all (executable) specifications in VCS

» Provide meaningful commit messages (changelog)

» Build history tells you which revision got deployed

“Infrastructure as Code”


Benefits: Repeatability



Establishes a process that is repeatable at any time:

» Environments are no longer in the hands of single people

» Any (authorized) person could recreate the environments

» Requires command execution to be idempotent



Allows for testability of the deployment process:

» Failures will be caught early-on in the process

What if something breaks?

» Caught a bug: use a version that is known to work

» Hardware failure: redeploy environment in minutes

Minimizes MTTR


Deployment Automation Solutions:Agent-based vs. Agentless


Agent-Based Solutions


Agent-Based Deployments (Chef, Puppet)



» Can be used in client-server or client-only modes

» Client must be installed on each host to be provisioned

» Clients have dependencies: Ruby


Agent-Based Deployments

Puppet

» Written and extensible in Ruby (comes with a DSL)

» Order of statement does not specify order of execution

» Huge ecosystem (PuppetDB, MCollective, Hiera)

» Large entrance barrier


Agentless Solutions


Agentless Deployments (Ansible)


Ansible

» Written and extensible (Python)

» Human- and machine-readable configuration (YAML)

» No boot-strapping required on deployment hosts (SSH)

» Simple, easy to ramp up with (think of new employees!)

» Clear and concise documentation

Agentless Deployments


Hands-On Training: Environment


Sample Environment

Web Server

Frontend

Application Server

Backend

Application ServerDatabase

VCS Build Automation

check out deploy


Sample Environment

Web Server FrontendApplication Server

BackendApplication Server

Database

Apache Tomcat Apache TomcatApache PostgreSQL

VCS

Git

Build Automation

Jenkins

check out deploy


Use Case: Deploy Dynatrace Agents



Database


VCS

Git

Build Automation

Jenkins

check out deploy


Use Case: Deploy Dynatrace Agents



Database


VCS

Git

Build Automation

Jenkins

check out deploy

Dynatrace Server


Hands-On Training: Environment


1. Inject dynaTrace Agents into Apache Tomcats

2. Load database data into PostgreSQL

3. Integrate all this into Jenkins

Today you will learn how to automatically...


Ansible: Core Concepts


Ansible Concepts:Inventories


Ansible Concepts: Inventories

» Ansible provisions groups of servers at once

» Groups and hosts are stored in inventory files

» An inventory file is expressed in a simple INI format

» Default location: /etc/ansible/hosts

» bit.ly/ansible-inventory

http://bit.ly/ansible-inventory





Ansible Concepts: Inventories

[frontends]frontend.example.com

[backends]backend.example.com

[frontends:vars]dt_agent_name=frontend

[backends:vars]dt_agent_name=backend

GroupHost

GroupVariables Variable


Ansible Concepts:Ad-hoc Commands


Ansible Concepts: Ad-hoc Commands

ansible <host-pattern> [options]

Examples?

» ansible localhost -m copy –a ‘src=/usr/bin/a dest=/usr/bin/b’

» ansible appservers –a ‘/sbin/reboot’ –f 10

» ansible appservers –a ‘/sbin/reboot’ –f 10–u deploy ––sudo ––ask–sudo–pass

Module Arguments




Examples?




Forks




Examples?




User Use sudoAsk passwordinteractively


Ansible Concepts: Playbooks



ansible-playbook [–i <inventory>] <playbook>

Playbooks

» Describe policies your remote systems shall enforce

» Consist of variables, tasks, handlers, files and roles

» Are expressed in the YAML format

» bit.ly/ansible-playbook

http://bit.ly/ansible-playbook





YAML (YAML Ain’t No Markup Language): Motivation

» “All data structures can be expressed via dicts, lists and scalars”

» Simplicity makes it much easier to read and parse than XML

Excursion to YAML


--- # Blockname: Michael Jordanage: 34

YAML Excursion » Dictionaries

--- # Inline{ name: Michael Jordan, age: 34 }





Document





key: value

SameIndentation

level





Comment

Optional quotesfor Strings


--- # Block- milk- bread- butter

YAML Excursion » Lists

--- # Inline[milk, bread, butter]




--- # Inline[milk, bread, butter]

Items start withhyphen + space




--- # Inline[milk, bread, butter]Same

indentationlevel


---men: - { name: Jimi Hendrix, year: 1970 } - { name: Jim Morrison, year: 1971 }women: - { name: Janis Joplin, year: 1970 } - { name: Amy Winehouse, year: 2011 }

YAML Excursion » Complex




Dictionary

List




List

Dictionary


--- # appservers.yml- hosts: frontends:backends vars_files: - variables.yml handlers: - name: restart tomcat service: name=tomcat state=restarted tasks: - name: Inject dynaTrace Java agent into Apache Tomcat template: > src=templates/tomcat-setenv.sh dest={{ tomcat_home }}/bin/setenv.sh notify: restart tomcat remote_user: deploy sudo: yes


Play



# production

[frontends]frontend.example.com

[backends]backend.example.com

[frontends:vars]dt_agent_name=frontend

[backends:vars]dt_agent_name=backend



# templates/tomcat-setenv.sh

CATALINA_OPTS="$CATALINA_OPTS \ -agentpath:{{ dt_agent_dir }}/libdtagent.so =name={{ dt_agent_name }},collector={{ dt_collector_url }}"


--- # variables.ymltomcat_home: /opt/tomcatdt_agent_dir: /opt/dynatrace/agentsdt_collector_url: dynatrace.example.com




ansible-playbook –i production appservers.yml

PLAY [frontends:backends] ******************************************************

TASK: [Inject dynaTrace Java agent into Apache Tomcat] ************************ changed: [backend.example.com]changed: [frontend.example.com]

PLAY RECAP ************************************************************************backend.example.com : ok=1 changed=1 unreachable=0 failed=0 frontend.example.com : ok=1 changed=1 unreachable=0 failed=0

Run!


--- # playbook.yml- include: appservers.yml- include: dbservers.yml- include: webservers.yml


Includes multiple playsinto a single playbook


Ansible Concepts: Playbooks » Variables

Usage

» {{ foo }}

» {{ foo[i] }}

» {{ foo.bar }}

» Hello, my name is {{ foo }}

» bit.ly/ansible-variables

http://bit.ly/ansible-variables



Advance Usage

» {{ foo | mandatory }}

» {{ foo | default(0) }}

» {{ list| min }}

» {{ list| unique }}

» {{ list1 | union(list2) }}





Advance Usage



» {{ list| min }}




Forces ‘foo’ to be defined




Advance Usage



» {{ list| min }}




Defaults ‘foo’ to 0 if undefined




Advance Usage



» {{ list| min }}




Computes the minimum in ‘list’




Advance Usage



» {{ list| min }}




Computes distinct values in ‘list’




Advance Usage



» {{ list| min }}



» bit.ly/ansible-variablesComputes a join of lists ‘list1 and ‘list2’




Validity

» Valid: ‘foo_bar’, ‘foo5’

» Invalid: ‘foo-bar’, ‘foo bar’, ‘foo.bar’, ‘5foo’, ‘5’



--- # playbook.yml- hosts: all vars: - cleanup_home: yes - settings: ports: http: 80 ssl: 443 …

Inlined



--- # playbook.yml- hosts: all vars_files: - variables.yml …

--- # variables.ymlcleanup_home: yessettings: ports: http: 80 ssl: 443

Imported


--- # playbook.yml- hosts: all tasks: - { include: create-user.yml, user: deploy }…


Defined intask inclusion


--- # playbook.yml- hosts: all roles: - { role: create-user, user: deploy }…


Defined inrole inclusion


ansible –e ‘user=deploy’ playbook.yml


Defined atinvocation


Ansible Concepts: Playbooks » Tasks

Tasks are...

» invocations of Ansible modules

» the units that do the actual deployment and configuration

» bit.ly/ansible-module

http://bit.ly/ansible-module



Example: Install package ‘apache2’

--- # webservers.yml- hosts: webservers tasks: - name: Install package ‘apache2’ apt: pkg=apache2 state=latest update_cache: yes



Example: Copy file from src (localhost) to dest (remote host)

--- # webservers.yml- hosts: webservers tasks: - name: Copy file from ‘foo.conf’ to /etc/default copy: > src=/srv/files/foo.conf dest=/etc/default owner=deploy group=deploy mode=644


Ansible Concepts: Playbooks » Roles

Roles

» Are the preferred means to organize and reuse related tasks

» Build on the idea of include files to form clean abstractions

» bit.ly/ansible-roles

http://bit.ly/ansible-roles



Reusing Roles in a Play

--- # webservers.yml- hosts: webservers roles: - { role: common } - { role: apache2 } remote_user: deploy sudo: yes



Best Practice: Directory Layout for Role-based Playbooks

playbook.ymlappservers.ymldbservers.ymlwebservers.yml

/roles/x/roles/x/meta/main.yml/roles/x/defaults/main.yml/roles/x/files/roles/x/handlers/main.yml/roles/x/tasks/main.yml/roles/x/templates/main.yml/roles/x/vars/main.yml

Top-level Playbook






Plays






Holds a role named ‘x’






Dependencies






Default variables






Files






Handlers






Tasks





/roles/x/roles/x/meta/main.yml/roles/x/defaults/main.yml/roles/x/files/roles/x/handlers/main.yml/roles/x/tasks/main.yml/roles/x/templates/main.yml/roles/x/vars/main.yml Templates






Variables

automated deployments

Software

puppet34 company confidential

metmajer2 company confidential

code28 company confidential

harmed25 company confidential

provisioning process

deployment process

consistent process

os differing number