killer r10k workflow - puppetconf 2014
DESCRIPTION
Killer R10K Workflow - Phil Zimmerman, Time Warner CableTRANSCRIPT
![Page 1: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/1.jpg)
Killer R10K Workflow
Automating the Killer Robots, all 10K of Them
![Page 2: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/2.jpg)
Who Am I@phil_zimmerman
![Page 3: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/3.jpg)
Early Days
R10K Workflow Awesomeness
Demo Time
![Page 4: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/4.jpg)
![Page 5: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/5.jpg)
The Journey of A Thousand Miles Begins
With A Single, Monolithic Repo
![Page 6: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/6.jpg)
A Single Repo? What the F*@K?!!
![Page 7: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/7.jpg)
Why Put Everything in a Single Repo?
Simplify Development
Easy Jenkins Flow
Puppet Code and Hiera Data Together
![Page 8: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/8.jpg)
Just Starting
Out
![Page 9: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/9.jpg)
![Page 10: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/10.jpg)
Single CI Job • rspec-puppet • syntax check • lint
Single Release Job • create/push tag
Single Deploy Job • Capistrano tasks
• poor man’s dynamic environments • kludgy git logic w/conditionals in Capfile
For All Modules
![Page 11: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/11.jpg)
Forge Modules
Capistrano -> Puppet Module Tool
Worked Well… Until It Didn’t
![Page 12: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/12.jpg)
Did I mention the
Capistrano tasks?
KLUDGY
![Page 13: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/13.jpg)
task "create_puppet_env", :roles => :puppet_master do if exists?(:branchname) run "if [ -d #{environment_basedir}/#{branchname} ]; then cd #{environment_basedir}/#{branchname} && git pull origin #{branchname} ; else \ git clone #{module_repository} #{environment_basedir}/#{branchname} --branch #{branchname} ; fi" else puts "Please provide a valid git branch name as an argument" end end
![Page 14: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/14.jpg)
task "create_puppet_env", :roles => :puppet_master do if exists?(:branchname) run "if [ -d #{environment_basedir}/#{branchname} ]; then cd #{environment_basedir}/#{branchname} && git pull origin #{branchname} ; else \ git clone #{module_repository} #{environment_basedir}/#{branchname} --branch #{branchname} ; fi" else puts "Please provide a valid git branch name as an argument" end end
![Page 15: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/15.jpg)
Upgrade Forge Modules
Version Management == Face Palm
Upgrading Affects all Environments!
![Page 16: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/16.jpg)
Let Me Count the Other Ways…
This Started to Fail
Simple Comes at a Price
![Page 17: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/17.jpg)
You mean I have to wait for tests
to run for ALL modules before I know if my changes are good?”
“I only changed one module….
![Page 18: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/18.jpg)
![Page 19: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/19.jpg)
I have to deploy
everything in order to get my changes on the masters? Sigh”
“I only changed one module….
![Page 20: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/20.jpg)
everything just to get my hiera data on the masters?”
“I need to change some hiera data…. I have to deploy
![Page 21: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/21.jpg)
I’m Losing
My Patience!
![Page 22: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/22.jpg)
Oh - and that little problem where
upgrading forge modules can break production…
Stop the Madness !!
![Page 23: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/23.jpg)
![Page 24: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/24.jpg)
Recap of Early Days:Monolithic RepoLong CI CyclesAll-Or-Nothing DeploysUpgrading Forge Modules
![Page 25: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/25.jpg)
![Page 26: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/26.jpg)
Tools Can Be
Awesome
(we just needed a couple more)
![Page 27: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/27.jpg)
Make Them Work For You
![Page 28: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/28.jpg)
R10K
![Page 29: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/29.jpg)
R10K
![Page 31: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/31.jpg)
Deploys Puppet Code
Handles Git/Svn Fu
Is Awesome
![Page 32: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/32.jpg)
R10K and Puppetfile
Match Made in Heaven (or Portland)
Manage Module Versions
Inventory of Puppet Environment
![Page 33: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/33.jpg)
Puppetfile
Format
![Page 34: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/34.jpg)
Inventory of Modules and Their Versions
Puppetfile lives in its
own repository
![Page 35: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/35.jpg)
Puppetfile lives in its own
Repository
![Page 36: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/36.jpg)
Inventory of Modules and Their Versions
![Page 37: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/37.jpg)
Each module in its own
repository AND r10k
![Page 38: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/38.jpg)
R10KDeploy
![Page 39: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/39.jpg)
r10k deploy environment test -p
deploys all modules in Puppetfile for the test branch
r10k deploy module tomcatdeploys a single module!
![Page 40: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/40.jpg)
CI Job Per Module • rspec-puppet • syntax check • lint
Release Job Per Module • create/push tag • select module from dropdown list
Deploy Job For Each Module And Hiera • simpler Capistrano tasks • wrap r10k calls to each master/node
![Page 41: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/41.jpg)
desc "for specified branch in puppetfile repo, use r10k to deploy all modules for the specified environment." task "update_environment", :roles => :puppet_master do if exists?(:branchname) run "r10k -v debug deploy environment #{branchname} -p" else puts "Please provide a valid git branch name as an argument" end end
![Page 42: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/42.jpg)
Puppetfile Manipulation and Branch Creation
![Page 43: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/43.jpg)
mod 'tomcat', :git => '[email protected]:fylgia/tomcat.git', :ref => 'RELEASE_1.0.13'
Tomcat Example
![Page 44: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/44.jpg)
go to tomcat dir (production branch) !
‘git checkout -b dev_change_foo’ ‘git push origin dev_change_foo’
![Page 45: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/45.jpg)
Create the ‘dev_change_foo’ branch
in Puppetfile repo
![Page 46: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/46.jpg)
Change the :ref for tomcat to
‘dev_change_foo’
![Page 47: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/47.jpg)
Call r10k to deploy ‘dev_change_foo’
Notify Me When It’s Finished
![Page 48: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/48.jpg)
![Page 49: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/49.jpg)
Explain the Magics
Hint:
![Page 50: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/50.jpg)
Create Branchr10k deploy environment
![Page 51: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/51.jpg)
Modify Branchr10k deploy module
![Page 52: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/52.jpg)
Delete Branchauto-delete Puppetfile branch
![Page 53: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/53.jpg)
Testing Multiple Modules in the
Same Environment
![Page 54: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/54.jpg)
go to profile dir (production branch) !
‘git checkout -b dev_change_foo’ ‘git push origin dev_change_foo’
![Page 55: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/55.jpg)
mod 'profile', :git => '[email protected]:fylgia/profile.git', :ref => 'RELEASE_0.1.124' mod 'tomcat', :git => '[email protected]:fylgia/tomcat.git', :ref => 'RELEASE_1.0.13'
production
mod 'profile', :git => '[email protected]:fylgia/profile.git', :ref => ‘dev_change_foo’ mod 'tomcat', :git => '[email protected]:fylgia/tomcat.git', :ref => 'dev_change_foo'
dev_change_foo
![Page 56: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/56.jpg)
Truly Dynamic Environments!
![Page 57: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/57.jpg)
The Post-Receive Hook
reaktor
![Page 58: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/58.jpg)
Default SetupGitHub or GitHub Enterprise
Hipchat
Just provide some config!
![Page 59: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/59.jpg)
REAKTOR_PUPPET_MASTERS_FILE
REAKTOR_HIPCHAT_TOKENREAKTOR_HIPCHAT_ROOMREAKTOR_HIPCHAT_FROM
PUPPETFILE_GIT_URL
![Page 60: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/60.jpg)
Other Chat ProvidersCampfire, Slack, etc
If it has an API, it’s pluggable!
![Page 61: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/61.jpg)
Other Git ProvidersGitlab, Bitbucket, etc
Need to Determine Best Approach
![Page 62: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/62.jpg)
Create Release
Deploy - Ship It!
![Page 63: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/63.jpg)
Create ReleaseModulefile
RELEASE_1.0.13 -> RELEASE_1.0.14
versionfile (hieradata)
![Page 64: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/64.jpg)
DeployParameterized Job
environment
version to deploy
![Page 65: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/65.jpg)
Puppetfile Manipulation (again)
change :ref to selected version
r10k deploy selected environment
commit/push updated Puppetfile
![Page 66: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/66.jpg)
One-Click Production
Deploy
![Page 67: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/67.jpg)
![Page 68: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/68.jpg)
Workflow Recap:R10K and PuppetfileEach Module in Own RepoPost-Receive Hook GoodnessProduction Deployments
![Page 69: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/69.jpg)
![Page 70: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/70.jpg)
![Page 71: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/71.jpg)
![Page 72: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/72.jpg)
Conclusion:Early DaysR10K Workflow AwesomenessDemos
![Page 73: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/73.jpg)
Helpful Links and References
https://github.com/adrienthebo/r10k
http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/
http://garylarizza.com/blog/2014/03/07/puppet-workflow-part-3b/
http://philzim.com/2014/05/02/an-automated-r10k-workflow-that-works/
https://github.com/pzim/reaktor
![Page 74: Killer R10K Workflow - PuppetConf 2014](https://reader033.vdocuments.net/reader033/viewer/2022042613/547e8b21b379596a2b8b54b4/html5/thumbnails/74.jpg)