one click deployment
TRANSCRIPT
![Page 1: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/1.jpg)
One-click DeploymentFabric and Puppet integration
Alex Su2013/09/25
Classification 04/13/2023
1
![Page 2: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/2.jpg)
2 Classification 04/13/2023
![Page 3: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/3.jpg)
What is a system admin?
![Page 4: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/4.jpg)
Don’t look at me...
I wasn’t the last one to touch it...
![Page 5: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/5.jpg)
5 Classification 04/13/2023
Everything the SameEverything Distinct
![Page 6: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/6.jpg)
Manuallyyum install nginx
vi /etc/nginx/conf.d/test.conf
service nginx start
![Page 7: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/7.jpg)
Shell Script
yum install nginx
mkdir -p /etc/nginx/conf.d
cat > /etc/nginx/conf.d/test.conf<<EOF
server {
listen 443;
ssl on;
}
EOF
service nginx start
install-nginx.sh
scp install-nginx.sh root@server:~/
ssh -o PasswordAuthentication=no -q -t -t “~/install-nginx.sh”
![Page 8: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/8.jpg)
One Goal:
Revolutionize
System
Administration
![Page 9: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/9.jpg)
Fabric command-line toolfor streamlining the use of SSH for
application deployment or systems administration tasks
Make executing shell commands over SSH easy and Pythonic
Stop administrating your environment and start developing it...
Re-usable code for managing your software & configurations
![Page 10: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/10.jpg)
Installation$ pip install fabric
$ pip install jinja2
$ sudo apt-get install fabric
![Page 11: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/11.jpg)
fabfile.py@task
def install_package():
run("yum install nginx")
![Page 12: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/12.jpg)
fabfile.py@task
def update_conf():
if exists("/etc/nginx/conf.d"):
run("mkdir -p /etc/nginx/conf.d")
put(”test.conf", "/etc/nginx/conf.d/test.conf")
![Page 13: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/13.jpg)
fabfile.py@task
def start_daemon():
run("service nginx start")
![Page 14: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/14.jpg)
fabfile.py@task
def deploy():
execute(install_package)
execute(update_conf)
execute(start_daemon)
![Page 15: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/15.jpg)
Task Argumentsfrom fabric.api import task
@task
def hello(name="world"):
print("Hello %s!" % name)
![Page 16: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/16.jpg)
Task Arguments$ fab hello:name=Alex
Hello Alex!
Done.
$ fab hello:Alex
Hello Alex!
Done.
![Page 17: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/17.jpg)
Templatedef update_conf():
context = {
'http_port' : 80,
'https_port' : 443
}
src_path = 'test.conf'
dest_path = '/etc/nginx/conf.d/test.conf'
files.upload_template(src_path, dest_path, context = context)
![Page 18: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/18.jpg)
Template Fileserver {
listen %(http_port)d;
}
server {
listen %(https_port)d;
}
![Page 19: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/19.jpg)
Template with Jinja2def update_conf():
context = {
‘ports' : [80, 443]
}
src_path = 'test.conf'
dest_path = '/etc/nginx/conf.d/test.conf'
files.upload_template(src_path, dest_path, context = context, use_jinja = True)
![Page 20: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/20.jpg)
Template File with Jinja2{%- for port in ports %}
server {
listen {{ port }};
}
{%- endfor %}
![Page 21: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/21.jpg)
Execute Modelfrom fabric.api import run, env
env.hosts = ['host1', 'host2']
@task
def taskA():
run('ls')
@task
def taskB():
run('whoami')
![Page 22: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/22.jpg)
Execute Model$ fab -l
Available commands:
taskA
taskB
![Page 23: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/23.jpg)
Execute Model$ fab taskA taskB
taskA executed on host1
taskA executed on host2
taskB executed on host1
taskB executed on host2
![Page 24: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/24.jpg)
Execute Model by Rolefrom fabric.api import run, env
env.roledefs = {
'web': ['www1', 'www2', 'www3'],
'dns': ['ns1', 'ns2']
}
def taskA():
run('ls')
def taskB():
run('whoami')
![Page 25: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/25.jpg)
Execute Model by Role$ fab -R dns taskA taskB
taskA executed on ns1
taskA executed on ns2
taskB executed on ns1
taskB executed on ns2
![Page 26: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/26.jpg)
Execute Model by Hosts$ fab -H ns1,www1 taskA taskB
taskA executed on ns1
taskA executed on www1
taskB executed on ns1
taskB executed on www1
![Page 27: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/27.jpg)
Arbitrary remote commands$ fab -H ns1,www1 -- whoami
task executed on ns1
task executed on www1
![Page 28: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/28.jpg)
Cuisine https://github.com/sebastien/cuisine
Chef-like functionality for Fabric
Covers file/dir operations, user/group operations, package operations
![Page 29: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/29.jpg)
Cuisine text_* : Text-processing functions
file_* : File operations
dir_* : Directory operations
package_* : Package management operations
command_* : Shell commands availability
user_* : User creation commands
group* : Group creation commands
mode_* : Configures cuisine's behaviour within the current session.
select_* : Selects a specific option, such as package back-end (apt, yum, zypper, or pacman)
![Page 30: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/30.jpg)
30 Classification 04/13/2023
Live Demo
![Page 31: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/31.jpg)
Drawbacks Not easy to implement by pure operators
Leak high-level function support User, file, package, service management Built-in environment variables
Leak smart error handling
Would do all things every time (depends on the implementation)
No log, no history
To many SSH communications (keepalive argument would help)
![Page 32: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/32.jpg)
Puppet Provides a Domain Specific Language (DSL) to script
with Classes, conditionals, selectors, variables, basic math, etc.
Supports Linux, Solaris, BSD, OS X, Windows
Stop administrating your environment and start developing it...
Re-usable code for managing your software & configurations
![Page 33: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/33.jpg)
33 Classification 04/13/2023
apt-get install nginx
vi /etc/nginx/conf.d/test.conf
service nginx start
Debian
yum install nginx
vi /etc/nginx/conf.d/test.conf
service nginx start
Redhat
![Page 34: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/34.jpg)
An Analogy
Programming SysAdmin
Low-level, non-portable
Assembly commands and files
Abstract, portable
Java / Python / Ruby Resources
![Page 35: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/35.jpg)
A Partial List of Puppet types
Packages • Supports 30 different package providers• Abstracted for your OS automatically• Specify ‘installed’, ‘absent’, or ‘latest’ for desired
state• Change from ‘installed’ to ‘latest’ and deploy for
quick Upgrade
Services • Supports 10 different ‘init’ frameworks• Control whether a service starts on boot or is
required to be running always• A service can be notified to restart if a
configuration file has been changed
Files/Directories
• Specify ownership & permissions• Load content from ‘files/’, ‘templates/’ or custom
strings• Create symlinks• Supports 5 types to verify a file checksum• Purge a directory of files not ‘maintained’
![Page 36: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/36.jpg)
Dashboard
![Page 37: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/37.jpg)
apt-get install nginxvi /etc/nginx/conf.d/test.confservice nginx start
Package
Configuration
Service
Configuration should get modified after package installation
Service should restart when configuration changes
![Page 38: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/38.jpg)
Sample classesclass nginx::server { $conf_dir = "/etc/nginx/conf.d" $http_port = 80 $https_port = 443
package {"nginx": ensure => installed } -> file {"nginx_conf": path => "$conf_dir/test.conf", content => template("nginx/conf/test.conf.erb"), owner => 'nginx', group => 'nginx', mode => 644, ensure => file } -> service {"nginx": enable => true, ensure => running }}
![Page 39: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/39.jpg)
Template Puppet templates are flat files containing Embedded
Ruby (ERB) variables
server {
listen <%= @http_port %>;
}
server {
listen <%= @https_port %>;
}
![Page 40: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/40.jpg)
NodeNode definitions look just like classes, including supporting inheritance, but they are special in that when a node (a managed computer running the Puppet client) connects to the Puppet master daemon.
node ‘www1' { include nginx:server}
![Page 41: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/41.jpg)
ModulesA module is just a directory with stuff in it, and the magic comes from putting that stuff where Puppet expects to find it.
![Page 42: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/42.jpg)
Module Structure
![Page 43: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/43.jpg)
Network Overview
Configuration allows for manual synchronizations or a set increment
Client or server initiated synchronizations
Client/Server configuration leverages a Certificate Authority (CA) on the Puppet Master to sign client certificates to verify authenticity
Transmissions of all data between a master & client are encrypted
![Page 44: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/44.jpg)
Every Client Retrieve resource catalog from central server
Determine resource order
Check each resource in turn, fixing if necessary
Rinse and repeat, every 30 minutes
![Page 45: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/45.jpg)
Every Resource Retrieve current state (e.g., by querying dpkg db or
doing a stat)
Compare to desired state
Fix, if necessary (or just log)
![Page 46: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/46.jpg)
![Page 47: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/47.jpg)
Drawbacks Hard to prepare the environment
Install Ruby, puppet packages Set up host name, domain name Put ssh public key to every client Configure certificate
Hard to control deployment time (in daemon mode)
Hard to support rolling upgrade
No global view, no service dependency control across hosts
![Page 48: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/48.jpg)
Combine Fabric and Puppet Fabric
When Operators trigger puppet to deploy packages one by one or
parallelly Rolling upgrade
Where Use fab -R or fab -H
Initial functions Global setup and teardown functions
Puppet What
Define puppet nodes
How Define puppet classes and templates
Reporting Update the status to puppet dashboard
![Page 49: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/49.jpg)
Initial functions Create EC2 instances (optional)
Setup SSH keys to all remote hosts
Configure yum repositories
Install puppet and ruby packages
Configure puppet and update new hosts to cert list
![Page 50: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/50.jpg)
Global setup functions Mandatory
Backup Clean yum cache Sync fabric configurations to puppet pp files Restart puppet master service
Optional Clean the environment if necessary Put ssh public key Put yum repo files Install system development tools Install ruby and puppet packages Update puppet patches Configure puppet environment
![Page 51: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/51.jpg)
Global teardown functions Start/stop services across hosts
Send email/SMS notifications to members
Do health/sanity check
![Page 52: One click deployment](https://reader036.vdocuments.net/reader036/viewer/2022081519/5585a9e6d8b42ae22a8b4ca7/html5/thumbnails/52.jpg)
52 Classification 04/13/2023
Questions?