test-driven infrastructure with puppet, test kitchen, serverspec and rspec

1 #Dynatrace

with Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure

PuppetMeetup

2 #Dynatrace

Insertimage here Martin Etmajer

Senior Technology Strategist at Dynatrace

[email protected]

@metmajer

3 #Dynatrace

Why Continuous Delivery?

4 #Dynatrace

Why Continuous Delivery?

5 #Dynatrace

Utmost Goal: Minimize Cycle Time

feature cycle time time

Customer Users

6 #Dynatrace



Customer minimize Users

7 #Dynatrace



Customer

This is when youcreate value!

minimize

8 #Dynatrace



Customer

You

This is when youcreate value!

minimize

9 #Dynatrace



Customer

You

minimizeIt’s about getting your features into your user’s hands

as quickly and confidently as possible!

10 #Dynatrace

Align Development and IT Operations

IT OPERATIONS

DEVELOPMENT

time

11 #Dynatrace


IT OPERATIONS

DEVELOPMENT

current iteration(e.g. 2 weeks)

time

12 #Dynatrace


IT OPERATIONS

DEVELOPMENT


time

Planning

13 #Dynatrace


IT OPERATIONS

DEVELOPMENT


time

PlanningImplementing

and testing

14 #Dynatrace


IT OPERATIONS

DEVELOPMENT


time

PlanningImplementing

and testingWorking and

deployable code

15 #Dynatrace

Why Test-Driven Infrastructure?

16 #Dynatrace

You write code!

17 #Dynatrace

“Make it work. Make it right. Make it fast.”Kent Beck, Creator of Extreme Programming and Test-Driven Development

Get the code tooperate correctly

Make the code clear,enforce good design Optimize

http://en.wikipedia.org/wiki/Extreme_programming

http://en.wikipedia.org/wiki/Test-driven_development

18 #Dynatrace

The Red, Green, Refactor Cycle of TDD

Write a Failing Test

Make the Test PassClean Up your Code

Small increments

Able to return to known working code

Designed andtested code

Protects against regressions

19 #Dynatrace

Test KitchenKey Concepts

Pluggable Architecture

20 #Dynatrace

» Drivers» Platforms» Provisioners» Test Suites

Key Concepts

21 #Dynatrace


Key Concepts

22 #Dynatrace

Drivers let you run your code on various...

Cloud Providers» Azure, Cloud Stack, EC2, Digital Ocean, GCE,

Rackspace,...

Virtualization Technologies» Vagrant, Docker, LXC

Test Kitchen: Drivers

23 #Dynatrace


Key Concepts

24 #Dynatrace

Platforms are the Operating Systems you want to run on.

Platforms» Linux- or Windows-based (since Test Kitchen 1.4.0)

How to manage dependencies?» Automatically resolved when using Docker or Vagrant» Build your own and link them in the configuration file

Test Kitchen: Platforms

25 #Dynatrace


Key Concepts

26 #Dynatrace

Provisioners are the tools used to converge the environment.

Provisioners» Ansible, Chef, CFEngine, Puppet, SaltStack

Why cool?» Useful if you need to support multiple of these tools

Test Kitchen: Provisioners

27 #Dynatrace


Key Concepts

28 #Dynatrace

Test Suites define the tests to run against each platform.

Test Frameworks» Bash, Bats, Cucumber, RSpec, Serverspec

Test Kitchen: Test Suites

https://github.com/sstephenson/bats

https://cucumber.io/

http://rspec.info/

http://serverspec.org/

29 #Dynatrace

Test KitchenInstallation

30 #Dynatrace

Installation

Ready?

$ gem install test-kitchen kitchen-docker kitchen-puppet

Test Kitchen: Installation

$ kitchen versionTest Kitchen version 1.4.2

31 #Dynatrace

Test KitchenConfiguration

32 #Dynatrace

Create a Puppet Module

Initialize Test Kitchen

$ mkdir –p my-puppet-module$ cd my-puppet-module

Test Kitchen: Testing a Puppet Module

$ kitchen init --driver=docker --provisioner=puppet_applycreate .kitchen.ymlcreate test/integration/default

Configuration goes here!Tests go here!

33 #Dynatrace

.kitchen.yml (as provided via `kitchen init`)---driver: name: dockerprovisioner: name: puppet_applyplatforms: - name: ubuntu-14.04 - name: centos-7.1suites: - name: default run_list: attributes:


Names resolve toDocker Images onthe Docker Hub

34 #Dynatrace

.kitchen.yml (slightly adjusted)---driver: name: dockerprovisioner: name: puppet_apply puppet_version: latest files_path: files manifests_path: test/integration manifest: default/init.pp puppet_verbose: true puppet_debug: false


platforms: - name: ubuntu-14.04 - name: centos-7.1suites: - name: default

35 #Dynatrace

$ kitchen list

Instance Driver Provisioner Transport Last Actiondefault-ubuntu-1404 Docker PuppetApply Ssh <Not Created>default-centos-71 Docker PuppetApply Ssh <Not Created>

`kitchen list`: List Test Kitchen Instances

Test Kitchen: Installation

This will change...Test Suite Platform

36 #Dynatrace

Test KitchenWrite an Integration Test

37 #Dynatrace

Create a Puppet Manifest for Test Suite ‘default’


$ kitchen init --driver=docker --provisioner=puppet_applycreate .kitchen.ymlcreate test/integration/default

Configuration goes here!Tests go here!

38 #Dynatrace

test/integration/default/init.ppclass { ‘foo’: }

class { ‘bar‘: bar_param => ..., require => Class[‘foo’]}

class { ‘baz’: baz_param => .., require => Class[‘bar’]}...


Create yourenvironment

Puppet Manifest

Test Suite

39 #Dynatrace

Serverspec and RSpecA Short Primer

40 #Dynatrace

RSpec is a TDD tool for Ruby programmers.

RSpec

require ‘foo’

describe Foo do before do @foo = Foo.new end

it ‘method #bar does something useful’ do @foo.bar.should eq ‘something useful’ endend

https://relishapp.com/rspec/

41 #Dynatrace

Serverspec is RSpec for your infrastructure.

Serverspec

require ‘serverspec’

describe user(‘foo’) do it { should exist } it { should belong_to_group ‘foo’ }end

describe file(‘/opt/bar’) do it { should be_directory } it { should be_mode 777 } it { should be_owned_by ‘foo’ } it { should be_grouped_into ‘foo’ }end

describe service(‘bar’) do it { should be_enabled } it { should be_running }end

describe port(8080) do it { should be_listening }end

describe command(‘apachectl –M’) do its(:stdout) { should contain(‘proxy_module’) }end

Resource

Matcher

http://serverspec.org/

42 #Dynatrace

test/integration/default/serverspec/default_spec.rbrequire ‘serverspec’

describe user(‘foo’) do it { should exist } it { should belong_to_group ‘foo’ }end


Test

Test Suite Do Serverspec!

43 #Dynatrace

`kitchen test`: Run Test Kitchen Test


$ kitchen test default-ubuntu-1404$ kitchen test ubuntu$ kitchen test regex!

$ kitchen list

Instance Driver Provisioner Transport Last Actiondefault-ubuntu-1404 Docker PuppetApply Ssh <Not Created>default-centos-71 Docker PuppetApply Ssh <Not Created>

44 #Dynatrace

Test Kitchen: Actions

Test = Converge Setup Verify

Instance createdand provisioned

Instance ready for verification(dependencies installed)

45 #Dynatrace

`kitchen test`: Run Test Kitchen Test$ kitchen test ubuntu

...User "foo" should exist should belong to group "foo"

Finished in 0.14825 seconds (files took 0.6271 seconds to load)2 examples, 0 failures

Finished verifying <default-ubuntu-1404> (0m37.21s).


46 #Dynatrace

`kitchen list`: List Test Kitchen Instances


$ kitchen list

Instance Driver Provisioner Transport Last Actiondefault-ubuntu-1404 Docker PuppetApply Ssh Verifieddefault-centos-71 Docker PuppetApply Ssh <Not Created>

47 #Dynatrace

Test Kitchen with PuppetAdvanced Tips

48 #Dynatrace

Testing Puppet Modulesin Amazon EC2

49 #Dynatrace

.kitchen.yml---driver: name: ec2 aws_access_key_id: "<%= ENV['AWS_ACCESS_KEY_ID']%>" aws_secret_access_key: "<%= ENV['AWS_SECRET_ACCESS_KEY']%>" aws_ssh_key_id: "<%= ENV['AWS_SSH_KEY_ID']%>" region: eu-west-1 availability_zone: eu-west-1btransport: ssh_key: "<%= ENV['AWS_SSH_KEY_PATH']%>" username: admin...

Test Kitchen: Testing Puppet Modules in EC2

Environment Variables

50 #Dynatrace

Testing REST APIswith RSpec Not supported by Serverspec

51 #Dynatrace

Testing REST APIswith RSpec Infraspec not yet integrated

52 #Dynatrace

test/integration/default/rspec/default_spec.rbrequire ‘json’require ‘net/http’

...describe ‘Server REST API’ do it ‘/rest/foo responds correctly’ do uri = URI(‘http://localhost:8080/rest/foo’)

request = Net::HTTP::Get.new(uri, { ‘Accept’ => ‘application/json’ }) request.basic_auth(‘foo’, ‘foo’) response = Net::HTTP.new(uri.host, uri.port).request(request)

expect(response.code).to eq 200 expect(JSON.parse(response.body)).to eq { ‘bar’ => ‘baz’ } endend

Test Kitchen: Testing REST APIs

Do RSpec!

53 #Dynatrace

test/integration/default/rspec/default_spec.rbrequire ‘json’require ‘net/http’

...describe ‘Server REST API’ do it ‘/rest/foo responds correctly’ do uri = URI(‘http://localhost:8080/rest/foo’)

request = Net::HTTP::Get.new(uri, { ‘Accept’ => ‘application/json’ }) request.basic_auth(‘foo’, ‘foo’) response = Net::HTTP.new(uri.host, uri.port).request(request)

expect(response.code).to eq 200 expect(JSON.parse(response.body)).to eq { ‘bar’ => ‘baz’ } endend

Test Kitchen: Testing REST APIs

Could use serverspec!

54 #Dynatrace

Dynatrace-Puppet ModuleFurther Examples: https://github.com/dynaTrace/Dynatrace-Puppet

55 #Dynatrace

56 #Dynatrace

Questions?

57 #Dynatrace