© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

AWS OpsWorks Under the Hood

Jonathan Weiss @jweiss Amazon Web Services

“Integrated application management service on EC2 – powered by Chef”

Chef Setup

Chef Client/Zero &

OpsWorks Backend

Chef Client &

Chef Server

OpsWorks Architecture

OpsWorks Backend

OpsWorks Agent

EC2, EBS, EIP, VPC, ELB, … Auto-Scaling, Auto-Healing,…

On-instance execution via Chef client/zero

Command JSON

Command Log+Status

Chef Setup in OpsWorks

•  Supported Chef versions: 0.9, 11.4 or 11.10 •  Built-in convenience cookbooks / bring your own •  Chef run is triggered by life cycle event firing •  Event comes with stack state JSON

Stacks & Layers

Modeling in OpsWorks

Layers Group of instances with common behavior & settings

–  Recipes / run_list –  Settings / attributes –  Similar to Chef role

Built-in Layers

Open Source at http://github.com/aws/opsworks-cookbooks

Rails MySQL PHP HAProxy Node.js Memcached Java Ganglia

Custom Layers Define your own layers and their run_list

–  Erlang app server –  Cassandra DB cluster –  C daemon –  Custom PHP install –  …

Custom Layers

Custom Layers

Custom Layers

Event Life Cycle

Events

Events are triggered when your stack changes: •  Give you fine-grained control •  Faster to execute •  Context: run_list per event per layer (aka role)

Life Cycle Events

16

setup configure deploy undeploy shutdown

Instance Life Cycle

new

Instance Life Cycle

new

Instance Life Cycle

new

onlin

e

setup

configure

Instance Life Cycle

new

onlin

e

setup

configure

deploy

Instance Life Cycle

new

/ st

oppe

d

onlin

e

setup

configure

terminating shutting down

deploy

configure

Setup Event

•  Sent when instance boots •  Includes deploy event •  Use for initial installation

of software & services

Setup Event – Recipe Execution Order

AWS OpsWorks

setup recipes Your setup

recipes AWS

OpsWorks deploy recipes

Your deploy recipes

Configure Event

•  Sent to all instances when any instance enters or leaves online state

•  Use for making sure the configuration is up-to-date

Deploy Event

•  Sent you deploy via UI/API also part of each setup

•  Use for custom deployment

Undeploy Event •  Sent via UI/API when

apps are deleted •  Use to remove apps from

running instances

Shutdown Event

•  Sent when an instance is shut down

•  ~45s to execute •  Use for clean shutdown

Stack State JSON Each event gets JSON / attributes that define the current stack state:

node[:opsworks]

node[:opsworks][:layers]

node[:opsworks][:instance]

node[:opsworks][:stack]

Chef Integration

Search

Stack state JSON available through search search(:node, “name:web1”)

search(:node, “name:web*”)

Attributes generated on nodes are not available

Search

appserver = search(:node, "role:php-app").first

Chef::Log.info(”Private IP: #{appserver[:private_ip]}")

Exposes: hostname/fqdn, IP/DNS, private IP/DNS, instance type, AMI ID, AZ, …

Roles

OpsWorks layers mapped as roles search(:node, “role:rails-app”)

search(:node, “role:custom-foo”)

Complete role functionality depends on supporting a “full” chef repo

Data Bags Define in custom JSON

{ "opsworks": { "data_bags": { "bag_name1": { "item_name1: { "key1" : “value1”, "key2" : “value2”, ... } }, "bag_name2": { "item_name1": { "key1" : “value1”, "key2" : “value2”, ... } }, ... } } }

{ "opsworks": { "data_bags": { "myapp": { "mysql": { "username": "default-user", "password": "default-pass" } } } } }

mything = data_bag_item("myapp", "mysql") Chef::Log.info("username: #{mything['username']}")

Recipe

Custom JSON

Encrypted Data Bags

Not supported – Alternative handling: •  Upload encrypted JSON to S3 •  Have instances access via IAM roles in a recipe

Store Secrets on Amazon S3

Access from instance via IAM instance profiles bucket = node['acme']['bucket'] key = node['acme']['key'] s3 = AWS::S3.new obj = s3.buckets[bucket].objects[key] obj.read

Berkshelf Integration

Enable Berkshelf in stack settings Supports any version, ships pre-compiled for some

Berkshelf Integration

Cookbook directories /opt/aws/opsworks/current/cookbooks

/opt/aws/opsworks/current/site-cookbooks

/opt/aws/opsworks/current/berkshelf-cookbooks

Berksfile

cookbook 'apt'

cookbook 'bluepill', '>= 2.3.1'

cookbook 'ark', git: 'git://github.com/opscode-cookbooks/ark.git'

cookbook 'build-essential', '>= 1.4.2', \

git: 'git://github.com/opscode-cookbooks/build-essential.git', \

tag: 'v1.4.2'

Environments

•  OpsWorks only supports the implicit _default env •  We are looking into adding proper env support

Recap

Main Differences To Chef Server

•  One run vs. discrete events •  Push vs. pull •  Discovery: search & AWS OpsWorks attribute tree •  Encrypted data bags •  Environments & roles

AWS OpsWorks

•  Life cycle framework •  Highly customizable –

in the end everything is a Chef run •  Expect us to integrate more over time

More information about AWS OpsWorks

•  Follow us on twitter @AWSOpsWorks •  Find us on YouTube •  Docs: http://aws.amazon.com/documentation/opsworks •  Blog: http://blogs.aws.amazon.com/application-management

Thank You@jweiss aws.amazon.com/opsworks