configuration management: automating and rationalizing server setup with cfengine 3 (open world...

Configuration ManagementAutomating and rationalizing server setup with CFEngine 3

Jonathan Clarke <jcl@normation.com>

24/09/2011

About the speaker

Jonathan Clarke

Sysadmin background

Infrastructure management

FLOSS contributor:CFEngine

Others (OpenLDAP, LSC, FusionInventory...)

Startup created in 2010

Based in Paris

Configuration management:

CFEngine (partner)

Rudder (creator)

→ CTO →

Introduction

1. CREATE

2. SETUP

3. USE

4. THROW AWAYCloud Computing

Introduction

Cloud Computing

→ APIs and tools are available

1. CREATE

2. SETUP

3. USE

4. THROW AWAY

Introduction

Cloud Computing

Three approaches:

1. Manually

2. Imaging

3. Configuration tool

1. CREATE

2. SETUP

3. USE

4. THROW AWAY

1) Configuration Management principles

2) Configuration Management tools

3) About CFEngine 3

4) Getting started

Agenda

Configuration Management

Principles through examples...

A server crashed.

Install a new one, peoplecan't work without it!

OK, it'll be done inabout two days...

There's a new critical security patchwe must deploy on all our servers!

Get it out quickly!

Right, I'll put the wholeteam on it.

Why configuration management?

Why configuration management?

Automation

IndustrializationReproducibility

Why configuration management?

How do we setupservice X?

Ask Jim, he'sthe expert on that.

But he left the company...

Huh, this server has been loggingerrors for a few weeks.

Oh? I think Michael changedsomething on it recently...He'll tell you what it was.

Damn, he's on vacation!

Why configuration management?

Building-upknowledge

HistoryDocumentation

Why configuration management?

An intruder just stole our datausing a vulnerability in amodule we don't need...

I thought the project specificationensured that we disabled that?

Er, it did, but we enabled it tosolve a problem and forgot todisable it afterwards... sorry...

Why configuration management?

Continuousvigilance

AlertsAutomatic repairs

Why configuration management?

I don't understand how thisserver is setup. It doesn't matchour best-practices.

Oh, that's a legacy server...

Well, it's a collection of littlethings, here and there...

Give me details on ourcurrent security policy.

Ah... Well, OK.Tell me: is it fully appliedon all our critical servers?

Er...

Why configuration management?

Rationalization

ControlNormalization

Configuration management benefits

Rationalization

ControlNormalization

Continuousvigilance

AlertsAutomatic repairs

Building-upknowledge

HistoryDocumentation

Automation

IndustrializationReproducibility

The tools

Configuration Management

Main tools available

CFEngine 3 Puppet Chef

Main tools available: history

Source:http://verticalsysadmin.com/blog/uncategorized/relative-origins-of-cfengine-chef-and-puppet

Relative origins of CFEngine, Puppet and Chef

The tools: similaritiesCFEngine 3 Puppet Chef

Text-based / CLIinterface

Client-server model(sometimes optional) Open Source

Designed specificallyfor configuration

management

Common origins

The tools: some differencesPuppet ChefCFEngine 3

C Ruby RubyLanguage

Windows supportYes Preliminary Partial

GPL Apache(ex-GPL)

ApacheLicense

A bit about CFEngine 3...

CFEngine 3: FeaturesMulti platform

Windows support

Two versions:

1. Community (open source)Runs in Cygwin

2. Nova (commercial)● Native Windows service

CFEngine 3: Features

Adapted toheterogeneousenvironments

Multi-OSMulti-distribution

Make it ”transparent” (forget about the complexity)

Existing standard library handling the differences between each OS and distribution

CFEngine 3: Features

Lightweight, non-intrusive

Daemon consumption on managed hostsNon-intrusive

Only two dependencies:- BerkeleyDB- OpenSSL

CFEngine 3: Features

Highly scalableEvolution of CPU utilization

for an increasing number of managed hosts

From 25 to 400 clients (x16)CPU utilization increases by 1.16%

Notes:• Each host runs CFEngine every 5 minutes• Configuration tested sets up Apache web server• Tests and monitoring using AWS

CFEngine 3: Features

Lightweight, non-intrusive

AutonomousFault-tolerant

Multi platform

Adapted toheterogeneousenvironments

Highly scalable

Progressiveroll-out

Getting started with CFEngine 3

CFEngine 3: Installing Install from sources:

http://www.cfengine.com/source_code

Prebuilt packages: Debian / SuSE / Fedora / RHEL / Ubuntu Requires free signup https://cfengine.com/inside/myspace

CFEngine 3: Client-Server Using a server is optional!

Get started by running standalone

CFEngine's server daemon is cf-serverd Dedicated protocol: TCP port 5308 Requires SSL key exchange

CFEngine 3: Configuration Minimal configuration:

body common control{ bundlesequence => { "HelloWorld" };}

bundle agent HelloWorld{ # This will output "Hello World!" commands: "/bin/echo Hello World!";}

Syntax notes

Whitespace doesn't countComments follow #

Structure notes

● Structures are created using { }● Structures are bundles or bodies

CFEngine 3: Configuration Promise types:

Special types(all versions)

vars

classes

methods

reports

Promise types(all versions)

files

packages

processes

commands

storage

interfaces (for future use)

Promise types(commercial versions)

environments

services

databases

Special types(commercial versions)

outputs

CFEngine 3: Examples Install and update the LAMP stack

bundle agent lamp {  vars:

"packages" slist => { "httpd", "php5", "mysql" };

  packages:    "${packages}"      package_method => generic,      package_method => "addupdate";}

CFEngine 3: Examples

packages:debian::

"apache2"package_policy => "add",package_method => apt;

centos|redhat::"httpd"

package_policy => "add",package_method => yum;

Install Apache with distribution variations

Thanks for participating!

Jonathan ClarkeEmail: jcl@normation.comTwitter: jooooooon42

24/09/2011

Stay in touch...