ensemble oscon 2011

Download Ensemble oscon 2011

If you can't read please download the document

Post on 16-Apr-2017




1 download

Embed Size (px)


Ensemble Welcome to Service Orchestration

OSCON 2011

Clint Byrum

SpamapS on IRC


DevOps Distilled - Isn't that config management?

Config Management

Machine Centric

De-coupled from provisioning

Knowledge Sharing Involves Customization and assumptions.

Analog: ./configure && make && sudo make install

Service Orchestration

Service Centric

Coupled with provisioning

Designed for Knowledge Sharing

Complimentary to config management

Analog: apt-get install

* Developers write their code thinking of services, not machines.* De-coupled meaning that w/ config management, you say give me a machine, and then configure it this way, where as with coupling you say Give me a service configured this way, and put it on these machines* Loose coupling with a two way comm channel lets us write formulas without having to assume anything about related services.. we can learn the answers from the other service's configuration.* ./configure && make && make install is more powerful for custom work, but packaging gives you the benefits of repeatability and portability

The Cloud





* Ensemble is mostly about using IaaS* EC2 only right now This includes Eucalyptus and OpenStack* Deploy fast, add new infrastructure fast, and repeat over and over.* Pay for what you use* Invest in infrastructure only when it makes sense for YOU

Open Source


* Open source makes it easy to solve your problems without having to appropriate and get locked in to software with a purchase.* Utility computing is similar.. running things on computers doesn't necessarily mean you need to own computers.* Currently marrying the two, opensource and the cloud, is a fairly manual or custom operation.

Open Source + The Cloud = Ensemble!

Yes this is the plug

Some things are better served by NOT being in Ubuntu's archive.

Ensemble in many ways makes Ubuntu Server invisible when using it properly.

* Deploys existing open source solutions fast - apt-get for the cloud* Also deploys YOUR app fast.* Makes it easy to define how your application relates to existing opensource services.* Helps you manage those relationships in The cloud* Tightly integrated with Ubuntu A leader in the cloud space.

Formulas are simple

Some yaml metadata

A few scripts that run at the right time

Well encapsulated

Written with whatever tool you want!

Assumptions can be eliminated

Easy to read, easy to review


ensemble: formulaname: mediawikirevision: 84summary: "website engine for collaborative work"description: | MediaWiki is a wiki engine (a program for creating a collaboratively edited website). It is designed to handle heavy websites containing library-like document collections, and supports user uploads of images/sounds, multilingual content, TOC autogeneration, ISBN links, etc.requires: db: interface: mysql slave: interface: mysql cache: interface: memcacheprovides: website: interface: http

Some yaml metadata lists relationships that it understands and config options

A few scripts that run at the right time configure service based on relationships and config settings

Well encapsulated MySQL formula is generic, and usable by any application that can speak to MySQL


Various strategies- Assumptions in code

- Assumptions in config mgmt.

- Puppet Exported Configs

- Chef Search

- Ensemble Fundamental component


* Wikipedia's architecture has a lot of relationships* Adding 100 of each type isn't hard anymore thank you config management.* Getting them to work together is still a challenge.

Example: Mediawiki

ensemble deploy mysql wiki-db

ensemble deploy mediawiki demo-wiki

ensemble deploy memcached wiki-cache

ensemble deploy haproxy wiki-balancer

ensemble add-unit wiki-cache

ensemble add-unit demo-wiki

ensemble add-relation wiki-db:db demo-wiki:db

ensemble add-relation wiki-cache demo-wiki

ensemble add-relation wiki-balancer:reverseproxy demo-wiki:website

* demo-wiki PHP layer and memcached both scale out with 'add-unit' (can scale back with remove-unit).* Eventually this will be presentable not as commands in serial, but a stack which will be loaded into the environment, and dumpable from a running system

Add slaves

ensemble deploy mysql wiki-slave-db

ensemble add-relation wiki-slave-db:slave wiki-db:master

ensemble add-relation demo-wiki:slave-db wiki-slave-db:db

* These commands add a slave instance, and relate it to the master* They also tell the demo-wiki service to use it as a slave* add-unit wiki-slave-db would provide even more read scale out* Write scaling is a bit more tricky* Adding slaves gets us to the subset of the wikipedia graph above.. still lots of relationships to define


Generated by ensemble's dot output format


ensemble set demo-wiki name=Life, The Universe, and Everything

ensemble deploy mediawiki config=arthur-dents-wiki.yaml arthur-dents-wiki


* There are some options that are unknowable, like the wiki's name* config-changed hook is executed every time a config is actually changed, not every time 'ensemble set' is called.

Formulas today




The Future


Local development

Ubuntu Orchestra for Bare Metal


Storage Management

Formulas, Formulas, Formulas, Formulas...






#ubuntu-ensemble on Freenode

Clint Byrum

Presentation by Clint Byrum