infrastructure as data with ansible: systems and cloud deployment and management for the lazy...
DESCRIPTION
Great programmers and sysadmins are lazy people: rightly, they prefer avoiding manual, time consuming and error-prone tasks such as installing and configuring a Linux, Apache, Tomcat cluster for the tenth time. With Ansible, an infrastructure (server, cloud) deployment automation & configuration both powerful AND simple (in most cases simpler than shell scripts and maven poms!), you can enjoy your coffee while it does all the work. The talk is very practical: I will set up a whole cluster in real time before the talk ends.TRANSCRIPT
Infrastructure as data with Ansible: systems / cloud deployment and management for the lazy developer
● Cloudy
Carlo Bonamico - [email protected]
NIS s.r.l. / JUG Genova
http://www.nispro.it / http://juggenova.net
Carlo Bonamico JUG Genova / NIS s.r.l.
What is this all about?
● Do you like– Staying up late to reconfigure a server that went out of sync?– Being unable to deploy a critical fix because the upgrade process
is so fragile and long that “it is better not to touch the system”? – Having to rely on a server that took a week to setup, and lose it
because of an HD failure?– Be unable to quickly scale your application on multiple servers
because the IT administration becomes too complex and time-consuming?
Carlo Bonamico JUG Genova / NIS s.r.l.
Ansible Hello World
If the answer to these question is
NO!
Then this talk is for you!
Carlo Bonamico JUG Genova / NIS s.r.l.
What do we want?
● An easy way of quickly installing and configuring new and existing servers
● A way of “syncing” the configuration to a baseline when it drifts
● A way of recreating a machine as many times as you need
– Reliably and with no effort● A way of managing complex deployments
– And orchestrating interconnected services
Carlo Bonamico JUG Genova / NIS s.r.l.
What do we want?
● A way of doing all of those things
– EASILY– QUICKLY– RELIABLY
● Doing things automatically
– Ideally with no additional effort vs doing things manually (and with less mistakes!)
Carlo Bonamico JUG Genova / NIS s.r.l.
An Agile Approach
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Simplicity
--the art of maximizing the amount of work not done--
is essential.
The Agile Manifesto
Carlo Bonamico JUG Genova / NIS s.r.l.
Enter Ansible
● Ansible is your friend!
– A tool for doing things automatically● With LESS effort than doing them manually
● It provides
– Remote command execution across multiple machines– File, package and configuration distribution– Automated installations and deployments
Carlo Bonamico JUG Genova / NIS s.r.l.
What's inside?
Carlo Bonamico JUG Genova / NIS s.r.l.
Enter Ansible
● Created by Michael De Haan of Cobbler fame
– Open Source @ https://github.com/ansible/ansible/
– now supported by AnsibleWorks● Well documented● Growing, active and supportive
community
–
Carlo Bonamico JUG Genova / NIS s.r.l.
Enter Ansible
● Minimal install● sudo add-apt-repository ppa:rquillo/ansible● sudo apt-get update● sudo apt-get install ansible -y
● Minimal requirements
– Python 2.6 on the commander– Python 2.4 on the nodes– Three phyton packages (autoinstall)
Carlo Bonamico JUG Genova / NIS s.r.l.
How does Ansible work?
● Work on all Unix/Linuxes
– And Windows with cygwin (currently limited)
● Transport over SSH
– (and other protocols in the future)● Inventory, configuration and playbooks in
YAML● No DB is involved
Carlo Bonamico JUG Genova / NIS s.r.l.
Getting Started
● SSH Key Pair
– ssh-keygen -b 2048 ● enter pizzamatic_rsa as filename
● Configure /etc/hosts or DNS● Configure ansible_hosts
– .ini format– Hosts– Groups, with []
Carlo Bonamico JUG Genova / NIS s.r.l.
Pizzamatic Time!
Carlo Bonamico JUG Genova / NIS s.r.l.
Pizzamatic infrastructure
● Front-end server with Apache2 and mod_proxy● Back-end application servers with Tomcat 7● Postgresql DB
● Common features
– Ssh public key – passwordless login– Ufw for firewall
Carlo Bonamico JUG Genova / NIS s.r.l.
First steps
● ansible -k -m ping -u pizzamatic pizzamatic-fe-test-01
– -k means ask password– -m means module (ping)– -u connection user– Target host
Carlo Bonamico JUG Genova / NIS s.r.l.
First steps
● ssh-agent● ssh-add ~/.ssh/pizzamatic_rsa● ansible -k -m ping -u pizzamatic
pizzamatic-fe-test-01● If it hangs, either
– You forgot the -k, and a certificate was not installed (or viceversa)
– You added the -K (sudo password), and passwordless sudo is enabled
Carlo Bonamico JUG Genova / NIS s.r.l.
Move to Playbooks
● Efficient way of describing the desired configuration of multiple hosts
– And then “apply” it– Incrementally
● Auto-resume● Synchronization● Versioning
● ansible-playbook pizzamatic.playbook
Carlo Bonamico JUG Genova / NIS s.r.l.
BDD with Infrastructure???
● First, descrive desired infrastructure status as plain text
– #pizzamatic service requires front-end
– #pizzamatic service requires application servers
● Then translate it incrementally in ansible “actions” → execute it!
Carlo Bonamico JUG Genova / NIS s.r.l.
Actions: an example
#Installing and configuring Apache 2
name: Ensure Apache2 is installed
action: apt pkg=apache2
name: Generate the virtual host configuration
action: template src=src/${service.name}ssl.j2 dest=/etc/apache2/sitesavailable
name: Ensure the site is up
action: command a2ensite ${service.name}ssl
action: service name=apache2 state=started
Carlo Bonamico JUG Genova / NIS s.r.l.
Ansible Actions
● Not ideal term! Very often “actions” do nothing!
– Because the system is already in the desired state
● action: file dest=/home state=present
● They do something only if the system is not in the desired state
Carlo Bonamico JUG Genova / NIS s.r.l.
Ansible Actions
● Most Ansible Actions are Idempotent
– “big word” meaning that you can repeat them as many times as you want and always get the same result
● In practice, it's what makes ansible useful
Carlo Bonamico JUG Genova / NIS s.r.l.
BDD with Infrastructure???
● Red
– Error● Yellow
– Applied, changed● Green
– Already in the desired state
Carlo Bonamico JUG Genova / NIS s.r.l.
Infrastructure as what?
Ansible = Infrastructure as Data
You describe your infrastructureYou version the description
“Applying” the description and actually ensuring that the infrastructure exists and is
in the desired state is an implementation detail (and up to ansible, not you)
Carlo Bonamico JUG Genova / NIS s.r.l.
Ansible Modules
● Clean and modular way of defining actions
– Encapsulate best practices– A single ansible action encapsulates
lines and lines of shell scripts● Very strong emphasis on reuse
Carlo Bonamico JUG Genova / NIS s.r.l.
Ansible Modules
● Implemented in any language
– Python, java, bash...– Core modules are in python
● Input: parameter string● Output: json data
Carlo Bonamico JUG Genova / NIS s.r.l.
Ansible Modules
● add_host● apt● apt_key● apt_repository● authorized_key● command● copy● cron● ec2● fetch● file● get_url● git● group● hg● lineinfile● mail
● mount● mysql_db● mysql_user● pause● ping● postgresql_db● postgresql_user● s3● script● service● shell● subversion● template● user● virt● wait_for● yum
And many more!
Carlo Bonamico JUG Genova / NIS s.r.l.
Variables
● Declared
– In the ansible_hosts file– individual YAML files relative to the
inventory file● e.g. host_vars/pizzamatic-fe-test-01
---
ntp_server: acme.example.org
Carlo Bonamico JUG Genova / NIS s.r.l.
Facts
● Automatically collected facts about systems involved in the playbook
– ${inventory_hostname}– ${ansible_eth0.ipv4.address}
● Can be use as variables in playbook and templates
Carlo Bonamico JUG Genova / NIS s.r.l.
Templates
● Jinja2 templates
– very similar to java ${property} syntax
● Env.sh.j2
– export JAVA_HOME=/home/${service.user}/jdk1.7.0
– export PATH=$PATH:$JAVA_HOME/bin
Carlo Bonamico JUG Genova / NIS s.r.l.
Handlers
● Respond to asynchronous events
handlers:
name: restart ssh
action: service name=ssh state=restarted
Carlo Bonamico JUG Genova / NIS s.r.l.
Playbooks
● Structure
---
hosts: pizzamaticfetest01
gather_facts: yes
user: pizzamatic
sudo: yes
vars_files:
pizzamatic.yml
vars:
name: pizzamatic
tasks:
include: pizzamaticfe.playbook #child sees parent variables and params
Carlo Bonamico JUG Genova / NIS s.r.l.
File management and transfer
● To the nodes
– ansible atlanta m copy a "src=/etc/hosts dest=/tmp/hosts"
– ansible webservers m file a "dest=/srv/foo/b.txt mode=600 owner=mdehaan group=mdehaan"
– ansible webservers m file a "dest=/path/to/c mode=644 owner=mdehaan group=mdehaan state=directory"
– ansible webservers m file a "dest=/path/to/c state=absent"
● From the nodes
– Use the fetch module
Carlo Bonamico JUG Genova / NIS s.r.l.
Best Practices
● Good old Software Engineering Principles still apply!
– Dont Repeat Yourself– Good Names make the difference– Be simple– S.O.L.I.D.
● http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod
Carlo Bonamico JUG Genova / NIS s.r.l.
Useful Tools
● Yaml Editor for Eclipse
– https://code.google.com/p/yedit/– https://code.google.com/p/yamledito
r/● Git & Mercurial
Carlo Bonamico JUG Genova / NIS s.r.l.
References
● Ansible Home & Ansible Docs
– http://www.ansible.cc● Extras
– http://www.ansible.cc/docs/contrib.html● Presentations
– https://speakerdeck.com/mpdehaan/ansible● AnsibleWorks
– http://www.ansibleworks.com/● This tutorial
– https://github.com/carlobonamico/ansible-tutorial
And the very active google group
ansible-project
Carlo Bonamico JUG Genova / NIS s.r.l.
References
● My blog
– http://www.carlobonamico.com ● My Company
– http://www.nispro.it ● JUG Genova
– http://juggenova.net ● Attend a course
– Infrastructure Management with Ansible (2 days)– http://www.nispro.it/education
Thank you
for your attention!