[celix] scaling devops with puppet and perforce
DESCRIPTION
Learn how to scale DevOps to a large number of IT services and take the next step towards end-to-end agility. In this presentation, see how Perforce and Git Fusion can help manage Puppet environments with an increasing number of Puppet modules.TRANSCRIPT
1
Scaling DevOps with Puppet and Perforce
Christoph Leithner
Founder
CELIX Hard- & Software
VertriebsgmbH
http://www.celix.at
Logo area
2
Agenda
• What is DevOps?
• IT Automation with Puppet
• Scaling DevOps with Perforce
• Agile Flow of Change for Puppet Modules
3
What is DevOps?
4
DevOps
What is DevOps?
Source: http://commons.wikimedia.org/wiki/File:Devops.png by Rajiv Pan
Technology
Operations
DevOps
5
Area 1: Extend delivery to production
DEV OPS
Area 2: Extend operations feedback to project
Area 3: Embed project knowledge into operations
Area 4: Embed operations knowledge into project
Source: http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/ by Patrick Debois
DevOps Areas - codified by Patrick Debois
6
• IT Automation: Puppet
• Shared Version Control: Perforce
• Build and Deploy: Atlassian Bamboo
• Process Management: Atlassian JIRA
DevOps Tools - Overview
7
IT Automation with Puppet
8
Define. 1 2 Simulate.
3 Enforce.
4 Report.
How Puppet works
9
Database Web Server Application Server Security Mo
du
les
Via Puppet Dashboard Custom External Source (CMDB, LDAP, etc.)
Define your
resources in
modules.
1
Assign resource
relationships
automatically.
2
Reuseable,
composable
configurations.
3
Web Servers Database Servers Application Servers
Define Composable Deployments with Puppet
10
user { 'dave':
ensure => present,
uid => '507',
gid => 'admin',
shell => '/bin/zsh',
home => '/home/dave',
managehome => true,
}
Puppet Resource Abstraction Layer (RAL)
11
class ssh {
package { ‘ssh’:
ensure => latest,
}
file { ‘/etc/ssh/sshd_config’:
source => ‘puppet:///modules/ssh/sshd_config’,
}
service { ‘sshd’:
ensure => running,
}
Package[‘ssh’]->File[‘/etc/ssh/sshd_config’]~>Service[‘sshd’]
Package[‘ssh’]~>Service[‘sshd’]
}
class ssh Puppet RAL Example:
12
module_name
manifests
files
templates
lib
spec
tests
Modulefile
Puppet Modules – Deployments as Code
13
name 'puppetlabs-apache'
version '0.6.0'
source 'git://github.com/puppetlabs/puppetlabs-apache.git'
author 'puppetlabs'
license 'Apache 2.0'
summary 'Puppet module for Apache'
description 'Module for Apache configuration'
project_page 'https://github.com/puppetlabs/puppetlabs-apache'
## Add dependencies, if any:
dependency 'puppetlabs/firewall', '>= 0.0.4'
dependency 'puppetlabs/stdlib', '>= 2.2.1'
Puppet Modulefile for puppetlabs-apache
14
http://forge.puppetlabs.com
15
puppet module install puppetlabs-apache --version 0.6.0
firewall
stdlib
apache
Puppet Module Subcommand
16
puppet module install puppetlabs-ntp --version 0.2.0
puppet module install puppetlabs-apache --version 0.6.0
firewall
ntp
stdlib
apache
Sets of Modules
17
puppet module install puppetlabs-ntp --version 0.2.0
puppet module install puppetlabs-apache --version 0.6.0
firewall
Puppet Environment
ntp
stdlib
apache
Multiple Environments on a single Puppet Master
Sets of Modules – Puppet Environments
18
Option A: Version each Module you need to change locally
A Git Repository for each Puppet Module you change
firewall
Puppet Environment
ntp
stdlib
apache
Versioning Puppet Modules/Environments
19
Option B: Version the Environment Folder
A single Git Repository for the Puppet Environment
firewall
Puppet Environment
ntp
stdlib
apache
Versioning Puppet Modules/Environments
20
Option C:
A single Git Repository for the Puppet Environment that uses Git
Submodules to include the Git Repositories for the Modules
firewall
Puppet Environment
ntp
stdlib
apache
Version the Environment Folder and all Modules
Versioning Puppet Modules/Environments
21
main
forge
apache
stdlib
firewall
Managing Puppet Environments with Perforce Git Fusion
apache
stdlib
firewall
environment
22
main
forge
apache
stdlib
firewall
Managing Puppet Environments with Perforce Git Fusion
environment
Puppet Environment
apache
stdlib
firewall
23
feature_4
release_12.3
forge
feature_1
hotfix_1
release_13.1
feature_2 feature_3
Mainline Model for Puppet Modules
mainline
24
• Modularity – Puppet Environment Repository is
assembled from multiple Module Repositories
• Easy Workflow - NO Git Submodules
• Central Change History in Perforce
• Fine grained Permissions
Perforce Git Fusion - Benefits
25
Puppet Console
Puppet Module Forge
Build and Deploy Version Management Process Management
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Master
Modules/Environments
Puppet Architecture
26
Catalog 1
2
3
4 Report
Collector Report
Report
Facts SSL secure
Encryprion
on all data
transport
Puppet
Master
Node
Puppet Data Flow
27
Scaling DevOps
Agile Flow of Change for Puppet Modules
28
• More Teams/Services adopt DevOps Practices
• Number of Environments and Modules grows
• Fine grained Security becomes more important
• Central Change history is required
• Processes are standardized
• Environments are standardized
• Puppetized Deployments are used across all Stages
What means Scaling DevOps?
29
release_12.3
rails_webapp
hotfix_1
release_13.1
feature_1 feature_2
apache
zend_webapp
java_app
mainline
Agile Flow of Change for Puppet Modules
30
rel_12.3
java_app
rel_13.1
zend_webapp apache
feature_2 feature_1
rails_webapp
main
apache_only
Agile Flow of Change for Puppet Modules
31
apache
rails_webapp
zend_webapp
All trademarks and logos are property of their respective owners
main
rails
webapp rails
webapp zend
webapp zend
webapp
Perforce Streams Modularity
rails
webapp zend
webapp
32
Development Quality Assurance Staging Production
Feature Stream Service Stream Mainline Stream Release Stream/Tag
Build Automation - The Puppet Environment Lifecycle
33
Puppet Console
Puppet Module Forge
Build and Deploy Version Management Process Management
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Agent Facter
NODE
Puppet Master
Modules/Environments
Summary