plone deployment made easy
DESCRIPTION
A talk about methods and tools to automate deployment of Plone sites. With a few steps an environment is prepared for a new Plone site on a test, staging or production layer. These steps take a couple of minutes, doing this manually took around one hour. We use Puppet to prepare our hosts/clusters to get an environment to deploy to. Fabric is used to deploy Plone on this environment and to extend the webserver configuration under the hood. These complementary techniques provide a complete solution to get a working Plone site, including rollbacks. Presentation by: Pawel Lewicki and Kim Chee LeongTRANSCRIPT
Plone deployment made easy
Pawel LewickiKim Chee Leong
Goldmund, Wyldebeast & Wunderliebe
{ lewicki, leong } @gw20e.com
Outline
● From manual to automatic deployment● How Puppet and Fabric are used for
deployment● Deployment demo in screencast● Puppet server orchestration● Fabric in detail
From manual to automatic deployment
our path to automatic deployment
● Mark eggs/modules using git tag● Update mr developer sources with tag
● SSH to server● Clone new buildout in releases directory● Bootstrap buildout and execute● Switch previous release to new release● If on a cluster set-up, rinse and repeat
Typical Plone release @GWW
● All previous steps in one Fabric script● Allows ‘dummy-proof’ releases
Run:
fab make_tag
fab deploy
fab switch
And Bob is your uncle!
Plone release @GWW using Fabric
xkcd - automationhttp://xkcd.com/1319/
● Avoid repetitive tasks● Error-proof● Immune to typo’s● Consistent environments● Time efficient
Why automatic deployment?
● Avoid repetitive tasks● Error-proof● Immune to typo’s● Consistent environments● Time efficient
Why automatic deployment?
nothing* should be done manually on the server
How Puppet and Fabricare used for deployment
● Puppet is used for server orchestration● Puppet prepares the servers● When Puppet is done, Fabric is used to
deploy buildouts
Initial server setup with Puppet
Puppet diagram
● Uses tst, acc and prd layers● Knows configuration for buildout● Deploys and runs buildouts● Does several other tasks
Buildout deployment with Fabric
Fabric diagram
Deployment screencastusing Puppet and Fabric
Puppet server orchestration
paving the way for Fabric
Example Puppet config
node 'ploneconf.puppet' {
package { 'apache2': ensure => installed }
appie::app { "ploneconfapp":
envs => {
prd => { uid => 3011 },
acc => { uid => 3012 },
tst => { uid => 3013 },
},
webserver => 'apache2',
accountinfo => $gw20e::user_accounts,
}
class { 'ssh':
server_options => {
'PasswordAuthentication' => 'no',
'PermitRootLogin' => 'no',
},
}
}1 / n
● User accounts for TAP environments:o app-mysite-prd in /opt/APPS/mysite/prdo app-mysite-acc in /opt/APPS/mysite/acco app-mysite-tst in /opt/APPS/mysite/tst
● Webserver config is mapped to each application user$ cat /etc/apache2/sites-enabled/zzz-app-mysite-prdInclude /opt/APPS/mysite/prd/sites-enabled/
● SSH Keys of developers are synced● Buildout cache is enabled
Puppet tasks
Fabric in detailusing gww.buildout and
gww.buildout-fabric
● Automatic deployment● Push buildout config using jinja templates● Extended TAP-layers● Switching between releases
● But also; supports git branches, virtual host for webserver, copying zodb to local buildout.
Plone Fabric features
1. Update current symlinkrm -f ~/current
ln -s ~/releases/20141028 ~/current
2. Start supervisor daemon in new buildout~/releases/20141028/bin/supervisord
3. For each service in supervisor:a. Stop service in old buildout
b. Start service in new buildout
4. Shutdown supervisor in old buildout~/releases/20130101/bin/supervisorctl shutdown
Release switch using Fabric
● Fabric is controlled from the local buildout:
./bin/fab -l
Available commands:
deploy Create new buildout in release dir
switch Switch supervisor to latest buildout
test Test if the connection is working
● Deploying to test environment:
./bin/fab deploy:layer=tst,branch=new-feature
Example deployment config (1/2)
prdfrontend = dict(
hosts=prd['hosts'],
buildout='releases/frontend',
varnish={'port': 48083},
haproxy={'port': 48082, 'instances': prd['instances'], },
webserver='apache2',
sitename='ploneconfapp-prd.puppet',
site_id=prd['site_id'],
)
prdbackend = dict(
hosts=prd['hosts'],
buildout='releases/backend',
zeo=dict(base='/data1/APPS/ploneconfapp/prd', **prd['zeo']),
)
Example Deployment config (2/2)
prd = dict(
hosts=['[email protected]',],
buildout=_datestamped('releases/%Y-%m-%d'),
current_link='current',
auto_switch=False,
modules=_modules,
third_party_modules=_third_party_modules,
zeo={
'ip': ‘ploneconf.puppet’, 'port': 48081,
},
instances={
'ports': {'instance0': 8080, 'instance1': 8081},
'ipaddresses': _servers,
},
site_id=_site_id,
credentials={'username': 'admin', 'password': 'secret', },
remote_configs={'clockusers': 'clockuser.cfg'},
sentry={
'dsn': 'https://usr:[email protected]/29',
'level': 'ERROR',
},
)
Example jinja template / buildout
[supervisor]
programs +=
...
{% if varnish %}
60 varnish ${buildout:directory}/bin/varnish
true
{% endif %}
{% if varnish %}
[varnish]
port = {{ varnish.port }}
{% endif %}
Example jinja template / apache
{% if varnish %}
{% set port = varnish.port %}
{% elif haproxy %}
{% set port = haproxy.port %}
{% else %}
{% set port = instances.ports.instance0 %}
{% endif %}
<VirtualHost *:80>
ServerName {{ sitename }}
RewriteEngine on
ProxyPass / http://localhost:{{ port
}}/VirtualHostBase/http/{{ sitename }}:80/
ProxyPassReverse / http://localhost:{{ port }}/../http/{{ sitename }}:80/
</VirtualHost>
● https://github.com/search?q=ploneconf2014_gww
● Puppet module: o http://git.io/2NLMtg (shortened url)o https://github.com/Goldmund-Wyldebeast-Wunderliebe/puppet-ap
pie
● Fabric module:o http://git.io/kVdsnw (shortened url)o https://github.com/Goldmund-Wyldebeast-Wunderliebe/gww.buildo
ut-fabric
● GWW Plone buildouto http://git.io/PzdjqQ (shortened url)o https://github.com/Goldmund-Wyldebeast-Wunderliebe/gww.buildo
ut
Github repositories
Thank you!
Goldmund, Wyldebeast & Wunderliebe
{ lewicki, leong } @gw20e.com