using diskimage builder for heat deploying applications

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Using Diskimage Builder for Heat Deployed Applications

Robert CollinsTwitter: @rbtcollinsEmail: rbtcollins@hp.com

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2

Robert Collins

● Distinguished Technologist @HP Converged Cloud

● OpenStack TripleO PTL

● OpenStack Technical Committee

● Author of waaaay to many testing tools: fixtures, testrepository (testr), testtools, test* basically

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5

Our app needs

● Repeatable process

● Deploy without internet [behind the firewall]

● Deploy to bare metal

● Deploy what we test

– Tests take longer to run than new commits take to arrive

● Scale up and down

● Get 'enterprise' workloads – databases etc – to be cloudy

– Preserve persistent data

● Integrate well into existing deployment communities

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6

TripleO

● Heat: OpenStack Service orchestration

● Nova baremetal: Provision instances on physical machines

● ???

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7

TripleO

● Heat: OpenStack Service orchestration

● Nova baremetal: Provision instances on physical machines

● Chef/Puppet/etc

– Custom package repository per commit

– Either a full APT mirror of Ubuntu, or write a script to do clever partial mirroring...

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8

TripleO

● Heat: OpenStack Service orchestration

● Nova baremetal: Provision instances on physical machines

● diskimage-builder golden Images

– ALL installation takes place with Internet access

– Deploy never needs Internet access

– Configuration takes place either at install (for invariants) or at deploy time (for local config)

● Chef/Puppet/etc

– Include your preference in your images

– Wire into your system from cfn-init

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.9

TripleO

● Heat: OpenStack Service orchestration

● Nova baremetal: Provision instances on physical machines

● diskimage-builder golden Images

– ALL installation takes place with Internet access

– Deploy never needs Internet access

– Configuration takes place either at install (for invariants) or at deploy time (for local config)

● os-collect-config/os-refresh-config/os-apply-config

– Just Enough Config management

– Designed to play nice with Chef/Puppet/etc

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Architecture

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11

● An image build proceeds through a set of hooks: root.d, pre-install.d etc

● An image build is parameterised by including elements

● Each element includes one or more hook files: root.d/10-cache-ubuntu

– Which are simple executable files

● Elements can depend on other elements

● Handle multiple OS's by either conditional code or abstracting out a common interface e.g. install-packages replaces yum/apt calls.

● Most things get cached in ~/.cache/image-create

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12

● We create a tmpfs to build the image in

● Copy the contents of a base image there

– (e.g. an Ubuntu cloud image, or Fedora cloud image)

● Disable service startup and override resolv.conf / proxy settings for the duration

● Install software as needed (in a chroot)

● Make a sparse raw image with a filesystem big enough for the contents of the tmpfs.

● Move the tmpfs contents into the filesystem

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13

● If needed configure a bootloader

● Restore service startup and resolv.conf / proxy settings

● Pack everything down into a qcow2

● Done

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14

● Unlike Nova – image source is trusted

– If they want to root you, they will just ship hostile binaries!

● Easily wrapped up in a VM

– 18 lines of Heat template!

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Customisation

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16

● export ELEMENTS_PATH

– One or more directories where elements can be found

– The first occurrence of a particular element name is used

● Make a directory 'my-element'

● Add a README.md

● Add any dependencies to 'element-deps'

● Add any hook directories you need

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Creating images

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.18

● disk-image-create -a $(IMAGE_ARCH) -o $(IMAGE_NAME) $(IMAGE_ELEMENTS)

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19

● disk-image-create -a $(IMAGE_ARCH) -o $(IMAGE_NAME) $(IMAGE_ELEMENTS)

● disk-image-create ubuntu \ -a i386 -o $TRIPLEO_ROOT/overcloud-compute \ nova-compute nova-kvm neutron-openvswitch-agent os-collect-config \ dhcp-all-interfaces

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.20

Key elements (builtin)

● ubuntu/fedora/rhel (suse coming)

– Downloads the vendor cloud image and caches it

● vm

– Makes a VM image rather than a baremetal image – bootblock, bootloader

● source-repositories

– Downloads source from tarballs/git etc

● Disable-selinux

– Useful while getting selinux environments going – sets selinux to permissive

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21

Key elements (builtin)

● pip-cache

– Uses a shared pip cache

● pypi

– Uses a local pypi mirror and implements –offline for pip installs

● dhcp-all-interfaces

– Runs dhcp on any interface with link – super useful for multi-NIC machines/VMs

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22

Key elements (tripleo-image-elements)

● pypi-mirror

– Sets up a cron job maintained pypi mirror of the OpenStack requirements

● os-collect-config

– Alternative to cfn-init – daemon to trigger config management from metadata

● os-refresh-config

– Super shallow config management, run from os-collect-config

● os-apply-config

– Template applier for writing config files from os-refresh-config

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23

Key elements (tripleo-image-elements)

● use-ephemeral

– Maps stateful files into /mnt/state

– You should make that a Cinder volume (virtual instances)

– Or define an ephemeral partition size (Nova BM – trunk only)

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

CI

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Recap

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.27

● Repeatable – automated end to end

● No Internet needed to deploy

● Bare metal via Nova baremetal (Ironic in future)

● Build image during CI, archive it as an artifact: deploy what we test

● Scaling via Heat. All nodes identical. Simples.

● Persistent data is a work in progress

● Integration with Chef/Puppet – RedHat have a proof of concept with Puppet

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you