Tuesday, July 15, 14
Packing It In: Images, Containers, and Config ManagementMichael GoetzSr. Consulting Engineer @ [email protected]
Tuesday, July 15, 14
Who am I?• Sr. Consulting Engineer @ Chef
• 8+ years of experience planning, managing and operating web scale and enterprise applications
• Avid woodworker
Tuesday, July 15, 14
This talk isn’t about joining a cult...• Lots of opinions exist that claim to be the “only right
way” to manage your systems
• The true path is the best combination that makes you go faster, in a safe and secure manner
• Use a toolbox, not one tool
http://leavingthecult.com/
Tuesday, July 15, 14
So what are my options?• Artisanal machines made of metal and sweat• Pristine virtual machines• Isolated containers• Just-in-time automatic configuration management• All (or some) of the above?
Tuesday, July 15, 14
Artisanal machines made of metal and sweat • Do we really need to talk about why this sucks?
• If you want to work on artisan crafts, take up woodworking
http://www.juggernautwoodworking.com/images/carve.jpgTuesday, July 15, 14
Containers vs. Virtual Machines
• Containers consist of an application and its dependencies, running in isolation in userland outside the kernel.
• Virtual Machines create an entire machine, including a fully functional operating system.
https://www.docker.io/static/img/about/docker_vm.jpg
Tuesday, July 15, 14
Hurray! We can go back to golden images, right?• The “golden image” problem still exists with containers, but on a much smaller
scale• A dozen “server” images become dozens of “container” images• AUFS layering mitigates some sprawl, but has a limit• Modularity of applications without convergence of the entire system just kicks the
can down the road
http://images.smh.com.au/2011/10/28/2737998/ipad-art-wide-shipping-420x0.jpgTuesday, July 15, 14
What about configuration management?• Convergence - coming to a desired end state• Congruence - building a result from a blank state
• Always building from scratch can be time consuming
• Specification of application versions becomes extremely important
• Changes can happen unexpectedly if you don’t plan ahead
Convergence is like fixing the outcome and compute the route (like a GPS finder), and congruence is about repeating a recipe in a sequence of known steps to massage a system into shape”
– Mark Burgess
Tuesday, July 15, 14
Tuesday, July 15, 14
Let’s talk real world here...• My application system has:
• An OS layer that rarely changes• A few supporting applications that change semi-
frequently• My application code that changes rapidly
• This can translate to:• VM image to act as a base OS + some deltas• Container images for supporting applications• Configuration management to maintain overall state
Tuesday, July 15, 14
So wait... that still seems like a lot of work• With 3 layers of your application stack to maintain, it feels like the maintenance
demand will only go up
• We’ll use three tools to manage each layer:• Packer - building and maintaining images (virtual machine host)• Chef - building Docker images, provisioning the VM and managing the
configuration of running containers• Docker - running the containers
Tuesday, July 15, 14
What is Packer?• Half the battle is keeping VM images up-
to-date
• The more time spent refreshing VM images, the more table flipping that will ensue
• Packer is tool for creating identical machine images for multiple platforms from a single source configuration
• Makes programmatically building VM images super easy!
{ "builders": [{ "type": "amazon-‐ebs", "region": "us-‐east-‐1", "source_ami": "ami-‐8ade42ba", "instance_type": "m3.medium", "ssh_username": "ubuntu", "ami_name": "my ami {{timestamp}}" }], "provisioners": [{ "type": "chef-‐solo", "cookbook_paths": ["cookbooks"], "json": { "name": "my_node", "run_list": [ "recipe[docker]", "recipe[my_application]" ] } }]}
Tuesday, July 15, 14
What is Docker?• Docker combines Linux containers (LXC) with AUFS to
create portable, lightweight application containers
• Docker containers are running instances of Docker images
• Docker images can be shared via a public or private registry
• Containers can be single application processes or lightweight virtual machines if a supervisor is provided.
Tuesday, July 15, 14
What is Chef?• Chef is an automation platform that manages
infrastructure as code
• Configuration of systems is performed by reusable recipes that are shared across your entire infrastructure
• Information about the various infrastructure components is cataloged and made available to to inform the rest of the topology configuration
• Chef can run on demand or as a managed service to keep infrastructure convergent
Tuesday, July 15, 14
Chef-Container• A version of chef-client that includes
components to support running the chef-client from within a Linux container• Packaged with chef-client, runit and
chef-init• Allows you to bootstrap the container
without an SSH connection• Use chef-client resources the same way
in a container as on any UNIX- or Linux-based platform
• Can manage multiple services within a single container using chef-init & runit
Tuesday, July 15, 14
The knife-container plugin• Used to initialize and build containers•knife container docker init•knife container docker build
• Docker support today, other containers planned
• Berkshelf integration
• Supports Chef-Zero or Chef-Client modes
Tuesday, July 15, 14
Let’s get to building!• Starting with a solid foundation is key to success
• Identify the core components that are unlikely to change, but are different from default settings• Security policies/applications• Image hardening• Core component packages• Docker tooling
• The goal is to create a minimal base VM, combined with the components that are consistently configured across your entire application infrastructure
Tuesday, July 15, 14
Demo: Building the VM
Tuesday, July 15, 14
Building the Docker factory• We need a repeatable factory for building Docker
images for the supporting applications
• Chef-container lets us use our existing Chef cookbooks to create reusable Docker images
• The key to success is isolation - create the smallest Docker images that will work
• Hook up your continuous integration system to crank out new images as cookbooks are updated
Tuesday, July 15, 14
Demo: Building the Docker Factory
Tuesday, July 15, 14
Bringing it all together• Now that we have our base VM and Docker factory
running, let’s manage an active application stack
• Chef will provision servers with the base VM, build and run the Docker containers
• Ongoing convergence of the overall desired state of the system will be managed by chef-clients running inside each container.
Tuesday, July 15, 14
Demo: Using Chef to manage the entire system
Tuesday, July 15, 14
Wrapping Up• Don’t join a cult
• Use what works to make things faster, more secure and more stable
• Keep the base VM small, but not too small
• Use containers to manage isolated, reusable applications
• Maintain a convergent infrastructure with automated configuration management
Tuesday, July 15, 14
Want to know more?• Release: Chef Container 0.2.0 (beta) - http://www.getchef.com/blog/2014/07/15/
release-chef-container-0-2-0-beta/
• Chef Containers Documentation - http://docs.opscode.com/containers.html
• Video demo - https://www.youtube.com/watch?v=nSB9rHG1_FQ&feature=youtu.be
• Packer - http://www.packer.io/
• Docker - http://www.docker.com/
Tuesday, July 15, 14
