introduction to packer and suitcase: a packer-based os image build system

Tom McLaughlin

tmclaughlin@hubspot.com

@tmclaughbos

AN INTRO TO PACKER

&

SUITCASE: A PACKER BASED OS IMAGE BUILD SYSTEM

WHAT IS PACKER?

• Tool for creating OS images for multiple platforms from a single description

source

• Hashicorp (https://www.hashicorp.com/) developed software

• Makers of Vagrant

IDENTICAL IMAGES

• “identical” OS images for multiple platforms

IDENTICAL IMAGES

• Identical == Exactly the same except for minor platform differences

WHY USE PACKER?

• Build images for multiple platforms

• Support multiple virtualization platforms

• All these images should be identical to provide a consistent experience

and expectations

• Speed up development and reduce errors

• Replacement for homegrown tools and scripts

USING PACKER

• Starts with an image template that defines a build.

• Templates are 3 or optionally 4 sections.

• Variables (optional)

• Builders

• Provisioners

• Post-processors

TEMPLATE: VARIABLES

TEMPLATE: BUILDERS

TEMPLATE: PROVISIONERS

TEMPLATE: POST-PROCESSORS

USING PACKER

PACKER: INSPECT

PACKER: VALIDATE

PUTTING THAT ALL TOGETHER

PACKER RESOURCES

• Homepage: https://www.packer.io

• Documentation: https://www.packer.io/docs

• Plugins: http://github.com

• Search repos

• “packer-builder”

• “packer-provisioner”

• “packer-post-processor”

• Make sure you have Go installed

PACKER SUPPORT

• IRC:

• Freenode #packer-tool

• Mailing list:

• https://groups.google.com/forum/#!forum/packer-tool

PACKER’S LIMITATIONS

• EC2 builder for instance backed AMIs requires an existing AMI.

• No builder for EC2 HVM images (sort of).

• No logic in template to associate provisioners or post-processors with

particular builders

• Each image would require it’s own fully kickstarted build process.

SUITCASE

Let’s create something around

Packer to solve these problems!

HOW SUITCASE WORKS

• Suitcase our framework to use Packer and manage our image builds

• Suitcase breaks the build process into different stages

• Builds a master image

• Reuses master image to build platform images

• Suitcase uses Packer templates that are platform specific and not

OS/version specific.

• OS details are handled by var-files

HOW SUITCASE WORKS (CONT.)

• Build command goes from this to that:

• $ packer CentOS-6.5-x86_64.json

• $ packer –var-file=CentOS-6.5-x86_64.json awshvm.json

TYING IT ALL TOGETHER WITH RAKE

• rake packer:build[<image>] os=<var-file> [timestamp=<YYYYMMDDhhmm>]

• Build given image

• Build the master image if it doesn’t already exist

• rake packer:upload[<image>] os=<var-file> [timestamp=<YYYYMMDDhhmm>]

• Uploads the artifact to S3 for safe keeping

• rake packer:register[<image>] os=<var-file> [timestamp=<YYYYMMDDhhmm>]

• Uploads and registers an AMI

• rake packer:clean[<image>] os=<var-file> [timestamp=<YYYYMMDDhhmm>]

• Cleans image artifacts from your workspace

SUITCASE EXAMPLES

• Build all images and register all EC2 AMIs

• $ rake packer:build os=CentOS-7.0-x86_64

• $ rake packer:register os=CentOS-7.0-x86_64

SUITCASE EXAMPLES (CONT.)

• Build and register an AWS HVM image

• $ rake packer:register[awshvm] os=CentOS-7.0-x86_64

SUITCASE EXAMPLES (CONT.)

• Building an image from an existing master.

• $ rake packer:build[master] os=CentOS-7.0-x86_64

• (observe image timestamp from output. ex. 201410031904)

• $ rake packer:build[awshvm] os=CentOS-7.0-x86_64 \

timestamp=<timestamp>

SUITCASE LAYOUT

SUITCASE LAYOUT: IMAGES

SUITCASE LAYOUT: SCRIPTS

GAINS FROM SUITCASE

BUT THERE’S MORE!

SERVERSPEC

• Serverspec (http://serverspec.org) integration

• Let’s us define tests that images should conform to

• Helps us catch potential mistakes.

SUITCASE LAYOUT: TESTS

SERVERSPEC EXAMPLE

PROBLEMS SOLVED SUITCASE

• Sped up the development of new images

• Don’t need to run the kickstart installation with every change I make

• Images are all “identical”

• Each is descended from the same initial master image

• Many previously disconnected moving parts are now connected

• Building images

• Archiving images

• Registering images

• Added testing

• Easy enough to use by almost anyone on the team

• No more reliance on a single person who knows all the steps

TODO

• Make Rakefile simpler

• Not sure we need so many image specific targets

• Replace ec2-ami-tools with awscli

• We patched it to make some needed commands work on OS X

• Fetch artifacts from S3

• Useful of an image needs to be regenerated

• Open source this so others can use it as a reference

QUESTIONS

???