doing the refactor dance - making your puppet modules more modular - puppetconf 2014

2014

Presented by

The Refactor DanceGary Larizza Professional Services | Puppet Labs @glarizza

Presented by

http://bit.ly/refactordance

Presented by

Presented by

Worst. Hands-on. Ever

Presented by

• Abstraction and data separation

• Data Hierarchy

• Classification

• Workflow

Presented by

Classification

Implementation Implementation

Presented by

Role

Profile ProfileHiera

Component Modules

Presented by

roles::application_server

profiles::java profiles::tomcat

Hiera

Component Modules

Presented by

Component Modules

Presented by

Stop writing custom goddamn component

modules

Presented by

$httpd_root = "/opt/corp/data/http" package { ‘httpd’: ensure => latest, } file { “/opt/corp/data/http/conf.d”: owner => "httpd", ensure => directory; “/opt/corp/data/http/conf.d/corp.conf”: owner => "httpd", ensure => file; }

Presented by

Stop writing component modules

• Too many ‘okay’ modules

• Maintenance & upkeep

• You are not unique

• You are entirely too lazy

Presented by

Parameterize your classes

Presented by

class apache {! case $::osfamily {! 'RedHat': {! $confdir = ‘/etc/httpd/conf‘! $conffile = “${confdir}/httpd.conf”! }! 'Debian': {! $confdir = ‘/etc/apache2/conf‘! $conffile = “${confdir}/apache2.conf”! }! }!}

Presented by

class apache (! $confdir = $apache::params::confdir,! $conffile = $apache::params::conffile,!) inherits apache::params {! file { $confdir:! ensure => directory,! }! file { $conffile:! ensure => file,! content => template(’apache/apache.conf.erb’),! }!}

Presented by

Parameterize classes

• Parameters = API

• Single-entry classes

• The ‘Forge test’

Presented by

Params & shareable data

Presented by

class apache::params {! case $::osfamily {! 'RedHat': {! $confdir = ‘/etc/httpd/conf‘! $conffile = “${confdir}/httpd.conf”! }! 'Debian': {! $confdir = ‘/etc/apache2/conf‘! $conffile = “${confdir}/apache2.conf”! }! }!}

Presented by

Shareable data

• OS-specific data != private data

• Sane defaults

• Validation…

Presented by

Validation

Presented by

class xinetd (! $confdir = $apache::params::confdir,! $conffile = $apache::params::conffile,!) inherits xinetd::params {! file { $confdir:! ensure => directory,! }! file { $conffile:! ensure => file,! content => template(’apache/apache.conf.erb’),! }!}

Presented by

class xinetd (! $confdir = $apache::params::confdir,! $conffile = $apache::params::conffile,!) inherits xinetd::params {! validate_absolute_path($confdir)! validate_absolute_path($conffile)! file { $confdir:! ensure => directory,! }! file { $conffile:! ensure => file,! content => template(’apache/apache.conf.erb’),! }!}

Presented by

Validation

• Functions in puppetlabs-stdlib

• Never pass unvalidated data to resources

Presented by

Class

Presented by

class mysql::server (! ## params here!) inherits mysql::params {!!

include ::mysql::server::install! include ::mysql::server::config! include ::mysql::server::service!!

}

Presented by

class mysql::server (! ## params here!) inherits mysql::params {!!

include ::mysql::server::install! include ::mysql::server::config! include ::mysql::server::service!!

anchor { ‘mysql:start’: }! -> Class[‘mysql::server::install’]! -> Class[‘mysql::server::config’]! -> Class[‘mysql::server::service’]! -> anchor { ‘mysql:end’: }!}

Presented by

class mysql::server (! ## params here!) inherits mysql::params {!!

contain ::mysql::server::install! contain ::mysql::server::config! contain ::mysql::server::service!!

}* Puppet ≥ 3.4.0

Presented by

Class containment

• Before Puppet 3.4.0 - use anchors

• After Puppet 3.4.0 - use contain

Presented by

Hiera

Presented by

class data_in_code {! case $::application_tier {! 'dev': {! $java_version = '6.0.3'! $tomcat_version = '6.0'! }!!

'test': {! $java_version = '7.0.1'! $tomcat_version = '7.0'! }! }!}

Presented by

class data_in_code {! $java_version = hiera(’java_version’)! $tomcat_version = hiera(’tomcat_version’)!}

Presented by

--- :backends: - yaml

:yaml: :datadir: /etc/puppetlabs/puppet/hieradata

:hierarchy: - “nodes/%{::clientcert}” - “location/%{::location}" - “tier/%{::application_tier}" - common

hiera.yaml

Presented by

--- java_version: 7.0 tomcat_version: 8.0

dev.yaml

--- java_version: 6.0 tomcat_version: 7.0

prod.yaml

Presented by

Where’s

Presented by

What’s an Application

Presented by

‘Application tier’

• Long lived • Data usually separate • ‘The Data’

!

!

!

‘Environment’

• Short lived • Migration path to ‘production’ • ‘The Model’

!

!

!

Presented by

Hierarchy structure?

• How/where is data different?

• Most -> least specific

• Folders are your friends

Presented by

Profiles

Presented by

But first…

Presented by

include apache!

class { ‘apache’: }!

vs.!

Presented by

include apache!include apache!include apache!include apache!include apache!

Presented by

class { ‘apache’: }!include apache!

Presented by

include apache!class { ‘apache’: }!include apache!

Presented by

Namespacing

Presented by

class data_in_code {! $java_version = hiera(’java_version’)! $tomcat_version = hiera(’tomcat_version’)!!

notify { “Java is: ${java_version}”: }!}

Presented by

class data_in_code {! $java_version = hiera(’java_version’)! $tomcat_version = hiera(’tomcat_version’)!!

notify { “Java is: ${data_in_code::java_version}”: }!}

Presented by

class profiles::jenkins {! include jenkins!}

Presented by

class profiles::jenkins {! include ???????!}

Presented by

class profiles::jenkins {! include ::jenkins!}

Presented by

Data

Presented by

class data_in_code {! case $::application_tier {! 'dev': {! $java_version = '6.0.3'! $tomcat_version = '6.0'! }!!

'test': {! $java_version = '7.0.1'! $tomcat_version = '7.0'! }! }!}

Presented by

class profiles::tomcat {! $java_version = hiera(’java_version’)! $tomcat_version = hiera(’tomcat_version’)!!

class { ’::tomcat’:! version => $tomcat_version,! }!!

class { ’::java’:! version => $java_version,! }!}

Presented by

class apache {! file { ‘/opt/custom/key.pem’:! ensure => file,! source => ’puppet:///modules/apache/key.pem'! }!!

file { ‘/things/that/dont/belong/in/apache’:! ensure => file,! source => ’puppet:///modules/apache/blargh'! } !}

class apache {! file { ‘/opt/custom/key.pem’:! ensure => file,! source => ’puppet:///modules/apache/key.pem'! }!!

file { ‘/things/that/dont/belong/in/apache’:! ensure => file,! source => ’puppet:///modules/apache/blargh'! } !}

Presented by

class profiles::apache {! include apache! $keypath = hiera(’apache_keypath’)!!

file { “${keypath}/key.pem”:! ensure => file,! source => ’puppet:///modules/profiles/key.pem'! }!!

file { ‘/things/that/dont/belong/in/apache’:! ensure => file,! source => ’puppet:///modules/profiles/blargh'! }!}

Presented by

Dependencies

Presented by

class tomcat {! class { ‘java’:! version => ‘6.0’,! }!!

Class[‘java’]! -> Class[‘tomcat’]!}

Presented by

class profiles::tomcat {! $java_version = hiera(’java_version’)! $tomcat_version = hiera(’tomcat_version’)!!

class { ‘::java’:! version => $java_version,! }! class { ‘::tomcat’:! version => $tomcat_version,! }!!

Class[‘::java’]! -> Class[‘::tomcat’]!}

Presented by

class profiles::tomcat {! include profiles::java! $tomcat_version = hiera(’tomcat_version’)! ! class { ‘::tomcat’:! version => $tomcat_version,! }!!

Class[‘profiles::java’]! -> Class[‘::tomcat’]!}

Presented by

Profiles

• Hiera for business-specific data

• Proprietary resources

• Inter-class dependencies and containment

Presented by

Roles

Presented by

Classification

Presented by

denuatapp06p

falcor

Presented by

roles::app_server::pci

roles::proxy

Presented by

class roles {! include profiles::security::base! include profiles::mycorp::users! include profiles::mycorp::os_base!}

Presented by

class roles::app_server inherits roles {! include profiles::tomcat! include profiles::our_app! include profiles::shibboleth!!

Class[‘profiles::tomcat’]! -> Class[‘profiles::our_app’]! -> Class[‘profiles::shibboleth’]!}

Presented by

class roles::app_server::pci inherits roles::app_server {! include profiles::pci!}

Presented by

class roles::app_server::pci {! include profiles::security::base! include profiles::mycorp::users! include profiles::mycorp::os_base! include profiles::pci! include profiles::tomcat! include profiles::our_app! include profiles::shibboleth! include profiles::pci!!

Class[‘profiles::java’]! -> Class[‘profiles::our_app’]! -> Class[‘profiles::shibboleth’]!}

Presented by

Roles

• Hostnames minus Hiera

• Technology-independent

• Inheritance makes sense (or not)

Presented by

Workflow

Presented by

Module

Presented by

forge "http://forge.puppetlabs.com"!!

# Modules from the Puppet Forge!mod "puppetlabs/apache"!mod "puppetlabs/ntp"!!

# Modules from Github using various references!mod 'notifyme',! :git => 'git://github.com/glarizza/puppet-notifyme',! :ref => '50c01703b2e3e352520a9a2271ea4947fe17a51f'!!

mod 'profiles',! :git => 'git://github.com/glarizza/puppet-profiles',! :ref => '3611ae4253ff01762f9bda1d93620edf8f9a3b22'

Presented by

R10k - Bad name, good robot

1. Ensuring modules based on a Puppetfile 2. Dynamically creating Puppet environments

Presented by

PuppetfileManifestHieradata

Control Repository

Presented by

PuppetfileManifestHieradata

PuppetfileManifestHieradata

PuppetfileManifestHieradata

Presented by

PuppetfileManifestHieradata

PuppetfileManifestHieradata

PuppetfileManifestHieradata

Puppet Environment Puppet Environment

BranchBranch

Presented by

Demo

Presented by

Summary• Simple, generic component modules

• Extract company-specific data with Hiera

• Layer implementation with Profiles

• Classification with Profiles