replacing simple puppet modules with providers
DESCRIPTION
We've probably all gone looking for the resolv.conf module, or managed a template just so we could disable PermitRootLogin, or grumbled a little at how the host provider works. What if rather than managing modules for these things, there was an idempotent type available? That is the goal of augeasproviders by the Hercules Team. By writing custom types and providers using augeas you can go back to defining your environment with a DSL, rather than managing templates and additional modules. Greg Swift Linux Engineer, Rackspace Greg is a Linux Engineer for Rackspace. An open source enthusiast by day and a fire performer by night, he has been working extensively with Augeas inside Puppet for the past two years, including contributions to the Augeasproviders module.TRANSCRIPT
![Page 1: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/1.jpg)
Replacing simple modules with custom Types and ProvidersOr Stop managing templates, and start managing your configs
![Page 2: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/2.jpg)
2
Greg Swift
Linux Admin/Engineer ~ 12 yrs Red Hat Certified Engineer ~ 6 yrsAugeas user ~6 yrsPuppet user ~ 3 yrs
greg.swift@{rackspace.com,nytefyre.net}gplus.to/gregswiftlinkedin.com/gregoryswiftgithub.com/{gregswift,rackergs}
xaeth on Fedora, FreeNode, Twitter, and Ingress
![Page 3: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/3.jpg)
3
Bit of time travel...
• Past–An unpleasant reminder of configs past
• Present–Tools available today that help
• Future–What's next?
![Page 4: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/4.jpg)
4
Stroll down memory lane
![Page 5: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/5.jpg)
5
systl.conf
# Controls the default maximum size of a message queue
kernel.msgmnb = 65536
![Page 6: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/6.jpg)
6
Lets change that value
sed i 's/^\(kernel.msgmnb = \)\([09]*\)$/## Changing for db configuration. Was:\n## \1\2\n\199999/' sysctl.conf
![Page 7: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/7.jpg)
7
Looks good..
# Controls the default maximum size of a message queue
## Changing for db configuration. Was:
## kernel.msgmnb = 65536
kernel.msgmnb = 99999
![Page 8: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/8.jpg)
8
But the next run?
# Controls the default maximum size of a message queue
## Changing for db configuration. Was:
## ## Changing for db configuration. Was:
## kernel.msgmnb = 65536
kernel.msgmnb = 99999
## Changing for db configuration. Was:
## kernel.msgmnb = 99999
kernel.msgmnb = 99999
![Page 9: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/9.jpg)
9
That was then...
![Page 10: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/10.jpg)
10
Templates... yay?
• Great for 1 type of system... maybe even a couple• Supporting multiple OS releases or distributions?
![Page 11: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/11.jpg)
11
Wouldn't it be nice?
• Safe • Repeatable• Extensible• Multi-language
![Page 12: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/12.jpg)
12
But that is a herculean task...
![Page 13: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/13.jpg)
13
Meet team Hercules
David Lutterkort(Now @ PuppetLabs)
Raphaël Pinson
Dominic Cleal
Francis Giraldeau
![Page 14: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/14.jpg)
14
and Augeas
![Page 15: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/15.jpg)
15
What is it?
• An API provided by a C library• A domain-specific language to describe configuration file formats, presented as lenses
• Canonical tree representations of configuration files• A command line tool to manipulate configuration from the shell and shell scripts
• Language bindings to do the same from your favorite scripting language
![Page 16: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/16.jpg)
16
Lense all the things!
![Page 17: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/17.jpg)
17
Just to name a few....
access activemq_conf activemq_xml aliases anacron approx aptcacherngsecurity aptconf aptpreferences aptsources apt_update_manager authorized_keys automaster
automounter avahi backuppchosts bbhosts bootconf build cachefilesd carbon cgconfig cgrules channels cobblermodules cobblersettings collectd cron crypttab cups cyrus_imapd
darkice debctrl desktop device_map dhclient dhcpd dnsmasq dovecot dpkg dput erlang ethers exports fai_diskconfig fonts fstab fuse gdm group grub gtkbookmarks host_conf
hostname hosts_access hosts htpasswd httpd inetd inifile inittab inputrc interfaces iproute2 iptables jaas jettyrealm jmxaccess jmxpassword json kdump keepalived krb5 ldif ldso
lightdm limits login_defs logrotate logwatch lokkit lvm mcollective mdadm_conf memcached mke2fs modprobe modules modules_conf mongodbserver monit multipath mysql nagioscfg nagiosobjects netmasks networkmanager networks nginx nrpe nsswitch
ntp ntpd odbc openshift_config openshift_http openshift_quickstarts openvpn pam pamconf passwd pbuilder pg_hba php phpvars postfix_access postfix_main postfix_master
postfix_transport postfix_virtual postgresql properties protocols puppet puppet_auth puppetfileserver pythonpaste qpid quote rabbitmq redis reprepro_uploaders resolv rsyncd rsyslog rx samba schroot securetty sep services shells shellvars shellvars_list simplelines
simplevars sip_conf slapd smbusers solaris_system soma spacevars splunk squid ssh sshd sssd stunnel subversion sudoers sysconfig sysctl syslog systemd thttpd up2date util
vfstab vmware_config vsftpd webmin wine xendconfsxp xinetd xml xorg xymon yum
![Page 18: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/18.jpg)
18
Don't see your favorite config?
• Build• IniFile• Rx• Sep• Shellvars• Shellvars_list• Simplelines• Simplevars• Util
![Page 19: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/19.jpg)
19
Our earlier example.. on Augeas
augeas { 'set kernel.msgmnb per db vendor':
context => '/files/etc/sysctl.conf',
onlyif => 'kernel.msgmnb != 99999',
changes => 'set kernel.msgmnb 99999',
}
![Page 20: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/20.jpg)
20
Making it re-usable
define sysctl ($value) {
augeas { “set ${title} in sysctl.conf”:
context => '/files/etc/sysctl.conf',
onlyif => “${title} != ${value}”,
changes => “set ${title} ${value}”,
}
}
sysctl { 'kernel.msgmnb':
value => '99999',
}
![Page 21: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/21.jpg)
21
A more complex example..
define ssh_allowgroup ($ensure) {
if $ensure == present {
$match = '=='
$change = “set AllowGroups/01 ${title}”
} else {
$match = '!='
$change = 'rm AllowGroups/[.=${title}]”
}
augeas { “sshd_config/AllowGroups ${title}”:
context => '/files/etc/sshd_config',
onlyif => “match AllowGroups/[.=${title}] size $match 0”,
changes => $change,
}
}
$sshd_default_groups = ['engineers', 'admins']
$sshd_allowed_groups = $::env ? {
/prod/ => $sshd_default_groups,
default => concat($sshd_default_groups, ['devs']),
}
ssh_allowgroup { $sshd_allowed_groups:
ensure => present,
}
![Page 22: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/22.jpg)
22
Well I tried it once, but...
• Lenses are hard to write• Xpathing is hard• Its just hard!
![Page 23: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/23.jpg)
23
Make it easier!
![Page 24: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/24.jpg)
24
Introducing AugeasProviders
• Collection of custom types and providers• Written in native Ruby rather than Puppet's DSL• Utilizes bindings directly for flexibility• Heavily tested
![Page 25: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/25.jpg)
25
Introducing AugeasProviders
• Collection of custom types and providers• Written in native Ruby rather than Puppet's DSL• Utilizes bindings directly for flexibility• Heavily tested
![Page 26: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/26.jpg)
26
And that example on AugeasProviders
sysctl { 'kernel.msgmnb':
value => '99999',
comment => 'recommended by db vendor'
}
![Page 27: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/27.jpg)
27
And the more complex example
$sshd_default_groups = ['engineers', 'admins']
$sshd_allowed_groups = $::env ? {
/prod/ => $sshd_default_groups,
default => concat($sshd_default_groups, ['devs']),
}
sshd_config { 'AllowGroups':
value => $sshd_allowed_groups,
notify => Service['sshd'],
}
![Page 28: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/28.jpg)
28
What's it got?
• host• mailalias• sshd_config• shellvars /etc/{defaults,sysconfig}/*• puppet's auth.conf (puppet_auth)• syslog.conf entries (rsyslog and sysklog!)• Grub and Grub2 kernel_parameter• And more!
![Page 29: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/29.jpg)
29
Give it to me!
![Page 30: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/30.jpg)
30
Load it up
puppet module install domcleal/augeasproviders
or
git clone https://github.com/herculesteam/augeasproviders
![Page 31: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/31.jpg)
31
What about the future??
![Page 32: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/32.jpg)
32
AugeasProviders next
![Page 33: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/33.jpg)
33
What's changing?
• Minimized duplication of most common patterns• Solid generic library for reuse-ability• Enables Augeas based providers in your modules
![Page 34: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/34.jpg)
34
Contribute
![Page 35: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/35.jpg)
35
What can you do?
• Use it• Report bugs• Create new providers!
–resolv.conf–systemd unit files–etc
![Page 36: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/36.jpg)
36
Educate me!
![Page 37: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/37.jpg)
37
Augeas training
• Provided by camptocamp • http://camptocamp.com
– Solutions->Infrastructure->Training• Fundamentals
–Using augtool, XPath Augeas language, Augeas type in Puppet
• Advanced– Develop using augeas libraries and advanced tree
manipulation• Extending Augeas
–Writing lenses and providers
![Page 38: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/38.jpg)
38
Info and Help
• http://augeas.net• http://augeasproviders.com• #augeas on FreeNode• [email protected]
![Page 39: Replacing Simple Puppet Modules with Providers](https://reader034.vdocuments.net/reader034/viewer/2022052522/554f645eb4c9058a148b49fa/html5/thumbnails/39.jpg)
39