vmware vchs, puppet, and project zombie - puppetconf 2013
DESCRIPTION
"VMware vCHS, Puppet, and Project Zombie" by Nicholas Weaver, Cloud Automation Architect, Hybrid Cloud Service, VMware. Speaker Bio: Nicholas Weaver is the Cloud Automation Architect for VMware's vCloud Hybrid Service (vCHS) platform and the primary architect behind the vCHS automation framework (Project Zombie). He is also a co-creator of the Puppet Labs Razor project and many VMware-specific free tools. He previously worked in the CTO office for EMC, in the EMC field as a vSpecialist, and as a infrastructure engineer in financial, media, and retail companies. Nick loves software-driven control, hacking prototypes together, speaking at user groups, and demonstrating automation innovation to the masses. Nick can be found on Twitter and Github as @lynxbat.TRANSCRIPT
![Page 1: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/1.jpg)
© 2011 VMware Inc. All rights reserved
Nicholas Weaver - Cloud Automation ArchitectVMware vCloud Hybrid Service
VMware vCHS, Puppet, and Project Zombie
![Page 2: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/2.jpg)
2
About me
![Page 3: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/3.jpg)
3
About Nick
Nicholas Weaver
• VMware – Cloud Automation Architect, vCHS
• (formerly) EMC – Office of the CTO, Global Alliances, vSpecialist, Demo-builder
• (formerly) VMware/EMC Architect: Financial, Retail
• Co-creator Project Razor
• Twitter: @lynxbat
• Blog: nickapedia.com
• LinkedIn: www.linkedin.com/in/nicholasweaver
![Page 4: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/4.jpg)
4
What is vCloud Hybrid Service?
![Page 5: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/5.jpg)
5
vCloud Hybrid Service: Extending To The Cloud
VMware vCloud Hybrid Service
Your Data Center
Any Application… No Changes
Software-Defined Data Center
VMware vSphere & vCloud Suite
Existing & New Apps
Seamless Networking
Common management
One Support call
IaaS cloud owned and operated by VMware based on VMware software
![Page 6: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/6.jpg)
6
What do I do?
![Page 7: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/7.jpg)
7
Automation
![Page 8: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/8.jpg)
8
![Page 9: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/9.jpg)
9
![Page 10: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/10.jpg)
10
![Page 11: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/11.jpg)
11
Effort Evolution
![Page 12: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/12.jpg)
12
Why is automation important for vCHS?
![Page 13: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/13.jpg)
13
What is it really doing?
![Page 14: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/14.jpg)
14
![Page 15: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/15.jpg)
15
![Page 16: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/16.jpg)
16
The problem with success is…
![Page 17: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/17.jpg)
17
![Page 18: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/18.jpg)
18
Nothing stays the same
![Page 19: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/19.jpg)
19
![Page 20: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/20.jpg)
20
![Page 21: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/21.jpg)
21
We are not the first
![Page 22: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/22.jpg)
22
![Page 23: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/23.jpg)
23
![Page 24: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/24.jpg)
24
![Page 25: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/25.jpg)
25
![Page 26: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/26.jpg)
26
We need automation special sauce
![Page 27: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/27.jpg)
27
?
![Page 28: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/28.jpg)
28
project zombie
![Page 29: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/29.jpg)
29
Key principles
![Page 30: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/30.jpg)
30
Scale
![Page 31: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/31.jpg)
31
Extensibility
![Page 32: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/32.jpg)
32
Simplicity
![Page 33: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/33.jpg)
33
Resiliency
![Page 34: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/34.jpg)
34
![Page 35: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/35.jpg)
35
![Page 36: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/36.jpg)
36
What is Project Zombie?
![Page 37: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/37.jpg)
37
![Page 38: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/38.jpg)
38
![Page 39: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/39.jpg)
39
![Page 40: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/40.jpg)
40
![Page 41: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/41.jpg)
41
![Page 42: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/42.jpg)
42
Rez
![Page 43: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/43.jpg)
43
Distributed Resource Management
Stuff
• Resources
• State
• Configuration
• Location
• Distributed Locking
• Ownership
• Event History
![Page 44: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/44.jpg)
44
Distributed Resource Management
Stuff
Razor
Razor
StorageManager
StorageManager
Compute Storage
![Page 45: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/45.jpg)
45
Engine
![Page 46: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/46.jpg)
46
Zombie Engine DSL - ZED
![Page 47: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/47.jpg)
47
Zombie Engine DSL - ZED
PuppetApply
MCollective
MCollective
OVFTool
PuppetApply
MCollective
API Query
![Page 48: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/48.jpg)
48
Zombie Engine DSL - ZED
Reserve Compute
Install ESXi
Add To Cluster
Add Compute
Reserve Compute
Install ESXi
Add To ClusterReplace Compute
Un-Reserve Compute
Remove From Cluster
![Page 49: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/49.jpg)
49
Distributed and location-aware execution
Work to doConcurrent
WorkConcurrentWorkConcurrent
Work
ConcurrentWorkConcurrent
WorkConcurrentWork
P
P
P
B
B B
BB
Datacenter: Dallas
Datacenter: Singapore
Work to do
![Page 50: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/50.jpg)
50
Why we use Puppet
• Greater investment in building modules than scripting
• Much greater return on investment with evolution of the product
• 13 in-house Puppet Modules so far (small example)
• vCloud Director
• vShield Networking
• vSphere
• We use a total of 47 modules for everything
• We have Puppet modules for installing Zombie in Production, Integration, and Development (including Vagrant + Puppet use for laptops)
• Project Zombie itself uses Puppet to do work, and is installed and upgraded using Puppet
![Page 51: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/51.jpg)
51
Why we use Puppet
Puppet 1st Rule
![Page 52: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/52.jpg)
52
Cool things we do with Puppet
• Engine can stage changes using MCollective to an environment
• We can update that environment throughout job execution
• We can apply the environment against a target at any time
• We have the ability to load balance/scale both Engine execution and Puppet execution
• Processes that would be complex to order in Puppet can be staged much easier with Zombie Engine + Puppet
• ZED allows for iteration using configuration data into execution concurrency and queuing. Very cool layer on top of Puppet.
![Page 53: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/53.jpg)
53
Does it work?
![Page 54: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/54.jpg)
54
![Page 55: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/55.jpg)
55
Cloud Build A
Maximum capacity per week:
Man hours: 72
2
Time to delivery: 6 days
![Page 56: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/56.jpg)
56
Maximum capacity per week:
Man hours: 1.5
336 x Zombie Engine instance count
Time to delivery: 2.5 hours*
Cloud Build A
![Page 57: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/57.jpg)
57
Details (per job):
• ~120 individual tasks (plugin calls)
• ~2700 discrete configuration points
• ~1400 managed resources
• Dynamically sized (pick the # of compute and storage)
• Controls: vCloud Director, vCenter, ESXi, EMC VNX, Razor, vShield Manager, vShield Edge, Linux and Windows operating systems.
Cloud Build A
![Page 58: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/58.jpg)
58
Cloud Build A
![Page 59: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/59.jpg)
59
So yeah… it works
![Page 60: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/60.jpg)
60
What’s next?
![Page 61: VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013](https://reader035.vdocuments.net/reader035/viewer/2022062703/554f70acb4c9058a148b528c/html5/thumbnails/61.jpg)
61
Bring Your Own Licenses
Bring Your Own VMs
Bring Your Own Tools
Customer Components
VMware vCloud Hybrid Service
Hybrid Service Delivery Components
Infrastructure Hardware & Facilities
Compute StorageNetworking &
Security
OS Catalogs
Application Catalogs
Infrastructure Management
Web Console vCloud API