puppet for dummies - proidea · puppet for dummies 4developers - 18 april 2012 ... what is puppet...

http://joind.in/6328

Puppet forDummies4developers - 18 april 2012Poznań - Poland

woensdag 18 april 12

Joshua Thijssen

Freelance consultant, developer and trainer @ NoxLogic / Techademy

Development in PHP, Python, Perl, C, Java and some sysadmin

Blog: http://adayinthelifeof.nl

Email: [email protected]: @jaytaph

oh hai!

2


http://www.adayinthelifeof.nl

http://www.adayinthelifeof.nl

3


What is puppet and why should I care?

3


What is puppet and why should I care?

3

(answer: it’s cool and because I told you so)


“People are finally figuring out puppet and how it gets you to the pub by 4pm.

Note that I’ve been at this pub since 2pm.”

- Jorge Castro

4


5


Puppet is a (not necessarily the) solution for the following problem:

How do we setup, manage, synchronize, and upgrade our internal and external

infrastructure?

6


Sysadmin! Y U no fix problem!

7


Sysadmin! Y U no fix problem!

NO

7


LAMP-stack

8


LAMP-stack

Linux

Apache

MySQL

PHP

8


LAMPGMVNMCSTRAH-stack

9


LAMPGMVNMCSTRAH-stack

Linux

Apache

MySQL

PHP

Gearman

MongoDB

CouchDBSolr

Tika

Redis

ActiveMQHadoop

Varnish

Ngnix

Memcache

9


10


10

How do we control our infrastructure?


➡ Solution 1: We don’t,

10




➡ Solution 2: We outsource,

10




➡ Solution 2: We outsource,

➡ Solution 3: We automate the process.

10



‣ Solution 1: we don’t11


➡ It’s not funny: you find it more often than not. Especially inside small development companies.




➡ Internal sysadmin, but he’s too busy with development to do sysadmin.





➡ We only act on escalation





➡ We only act on escalation

➡ reactive, not proactive



‣ Solution 2: we outsource12


➡ Expensive $LA’s.




➡ What about INTERNAL servers like your development systems and infrastructure?





➡ Fight between stability and agility.





➡ Fight between stability and agility.

➡ Does your hosting company decide on whether you can use PHP5.3???



‣ Solution 3: we do it ourselves and automate13


➡ We are in charge.




➡ You can do what you like





➡ Use: cfEngine, chef, puppet.





➡ Use: cfEngine, chef, puppet.

➡ When done right, maintenance should not be difficult.



PUPPET

14


➡ Open source configuration management tool.

➡ Written in Ruby

➡ Open source: https://github.com/puppetlabs

➡ Commercial version available (puppet enterprise)

15


https://github.com/puppetlabs/

https://github.com/puppetlabs/

➡ Don’t tell HOW to do stuff.

➡ Tell WHAT to do.

¹

¹ It’s not actually true, but good enough for now...16


➡ Don’t tell HOW to do stuff.

➡ Tell WHAT to do.

¹

¹ It’s not actually true, but good enough for now...

“yum install httpd”“apt-get install apache2”

“install and run the apache webserver”

16


17

Schematic representation of a puppet infrastructure


Puppet

17

Schematic representation of a puppet infrastructure


Puppet CA PuppetMaster

PuppetAgent

https

18


Puppet CA PuppetMaster

PuppetAgent

PuppetAgent

PuppetAgent

https

18


Puppetmaster

Puppetclient

19


Puppetmaster

Puppetclient

Check credentials

19


Puppetmaster

Puppetclient

Check credentials

Send facts

19


Puppetmaster

Puppetclient

Check credentials

Send facts

Returns “catalog”

19


Puppetmaster

Puppetclient

Check credentials

Send facts

Returns “catalog”

Report results

19


➡ Catalogs are “compiled” manifests

➡ Manifests are puppet definitions

➡ <filename>.pp

➡ Puppet DSL

➡ De-cla-ra-tive language

➡ Version your manifests! (git/svn)

20


package { “strace” : ensure => present,}

file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, content => “beer”, }

21


package { “httpd” : ensure => present,}

service { “httpd”: running => true, enable => true, }

22


package { “httpd” : ensure => present,}

service { “httpd”: running => true, enable => true, } require => Package[“httpd”],

22


‣ Different distributions, different names

Centos / Redhatservice: httpdpackage: httpdconfig: /etc/httpd/conf/httpd.confvhosts: /etc/httpd/conf.d/*.conf

Debian / Ubuntuservice: apache2package: apache2config: /etc/apache2/httpd.confvhosts: /etc/apache2/sites-available

23


$operatingsystem is a FACT

package { “webserver”: case $operatingsystem { centos, redhat { $apache = “httpd” } debian, ubuntu { $apache = “apache2” } default : { fail(‘I don’t know this OS/distro’) } }

name => $apache, ensure => installed,

}

24


[root@puppetnode1 ~]# facter --puppetarchitecture => x86_64fqdn => puppetnode1.noxlogic.localinterfaces => eth1,eth2,loipaddress_eth1 => 192.168.1.114ipaddress_eth2 => 192.168.56.200kernel => Linuxkernelmajversion => 2.6operatingsystem => CentOSoperatingsystemrelease => 6.0processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHzpuppetversion => 2.6.9

‣ A simple list with info (also useable in your own tools)25


node default { $def_packages = [ “mc”, “strace”, “sysstat” ] package { $def_packages : ensure => latest, }}

/etc/puppet/manifests/site.pp:

‣ “Main” manifest26


Defining nodes - regular expressions

node /^web\d+\.example\.local$/ { package { “httpd” : ensure => latest, }}node /^db\d+\.example\.local$/ { package { “mysql-server” : ensure => installed, }}

27


node basenode { user { “jaytaph” : ensure => present, gid => 1000, uid => 1000, home => “/home/jaytaph”, shell => “/bin/sh”, password => “supersecrethashedpassword”, managehome => true, }}node /^.+\.example\.local/ inherits basenode { ...}

‣ Node inheritance28


‣ Group together into a class29


class webserver { service { “apache”: ensure => running, require => Package[“apache”],

} package { “apache” : ensure => installed, }

}



class webserver { service { “apache”: ensure => running, require => Package[“apache”],

} package { “apache” : ensure => installed, }

}

file { “vhost_${webserver_name}” : path => “/etc/httpd/conf/10-vhost.conf”, content => template(“vhost.template.erb”), notify => Service[“httpd”], }



<virtualHost <%= ipaddress %>:80> ServerName <%= webserver_name %> ServerAlias <%= webserver_alias %> DocumentRoot <%= webserver_docroot %>

</virtualHost>

vhost.template.erb

30‣ ERB templates can contain custom variables and facts


node “web01.example.local” inherits base { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot = “/var/www/web01” include webserver}node “web02.example.local” inherits base { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” include webserver}

31


http://www.noxlogic.local

http://www.noxlogic.local

➡ A puppet module is a collection of resources, classes, templates.

➡ Used for easy distribution and code-reuse.

➡ Self-contained, run out-of-the-box

32


➡ puppetforge / github

➡ Create your own (and share!).

➡ Use the ones from puppet enterprise edition.

➡ Use the standard layout / best practices

33


class ntp::install { package{"ntpd": ensure => latest }}class ntp::config { File{ require => Class["ntp::install"], notify => Class["ntp::service"], owner => "root", group => "root", mode => 644 } file{"/etc/ntp.conf": source => "puppet:///ntp/ntp.conf"; "/etc/ntp/step-tickers": source => "puppet:///ntp/step-tickers"; }}class ntp::service { service{"ntp": ensure => running, enable => true, require => Class["ntp::config"], }}class ntp { include ntp::install, ntp::config, ntp::service}

34


➡ (Unit)test your modules

➡ Test them with: puppet apply --noop

➡ More advanced testing: cucumber / cucumber-puppet (BDD)

35


http://docs.puppetlabs.com/references/stable/type.html

➡ Almost everything.

➡ standard 48 different resource types

➡ Ranging from “file” to “cron” to “ssh_key” to “user” to “selinux”.

➡ Can control your Cisco routers and windows machines too (sortakinda)

36




http://media.techtarget.com/digitalguide/images/Misc/puppetDashboard.gif

37




38


39

➡ Puppet went from v0.25 to v2.6.

➡ REST interface since 2.6. XMLRPC before that.

➡ One binary to rule them all (puppet).

➡ Puppet v2.7 switched from GPLv2 to apache2.0 license.


➡ --test does not mean dry-run!(--noop does).

➡ It’s not object oriented. (puppet class != php class)

➡ It’s a declarative language.

40


41


➡ Puppet agent “calls” the master every 30 minutes.

➡ But what about realtime command & control?

➡ “Puppet kick”... (meh)

➡ MCollective (Marionette Collective)

42


➡ Which systems running a database and have 16GB or less?

➡ Which systems are using <50% of available memory?

➡ Restart all apache services in timezone GMT+5.

43


ACTIVEMQClient

MCollectiveServer

NodeMiddlewareClient

MCollectiveServer

MCollectiveServer

‣ Middleware takes care of distribution,‣ queued, broadcast etc..

Collective

44


http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html45


http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html

http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html

Filter out nodes based on facts

$ mc-facts operatingsystemReport for fact: operatingsystem

CentOS found 3 times Debian found 14 times Solaris found 4 times

$ mc-facts -W operatingsystem=Centos operatingsystemreleaseReport for fact: operatingsystemrelease

6.0 found 1 times 5.6 found 2 times

46


➡ Display all running processes

➡ Run or deploy software

➡ Restart services

➡ Start puppet agent

➡ Upgrade your systems

47


-ETOOMUCHINFO

Let’s recap

48


➡ Configuration management tool.

➡ Focusses on “what” instead of “how”.

➡ Scales from 1 to 100K+ systems.

➡ Uses descriptive manifests.

49


➡ Useful for sysadmins and developers.

➡ Keeps your infrastructure in sync.

➡ Keeps your infrastructure versioned.

➡ MCollective controls your hosts based on facts, not names.

50


There is no reason NOT to control your infrastructure.

Having only 3 servers is NOT a reason.

51

You will be able to join the rest of us in the pub early.


http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 52


http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg

http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg

Please rate my talk on joind.in: http://joind.in/6328

Thank you

53

Find me on twitter: @jaytaph

Find me for development and training: www.noxlogic.nl

Find me on email: [email protected]

Find me for blogs: www.adayinthelifeof.nl


puppet for dummies - proidea · puppet for dummies 4developers - 18 april 2012 ... what is puppet...

Documents