aws re:invent 2016: configuration management in the cloud (dev305)
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amir Golan
Mark Rambow
December 1, 2016
Configuration Management
in the Cloud
DEV305
What to expect from this session?
Understand how configuration management lets you refer
to your infrastructure as code
Understand how AWS can help you use configuration
management to save time
Discover the best practices of setting up your
infrastructure, host configuration, and application
Background
Moving to the cloud and AWS allows you to provision and
manage infrastructure in new ways:
• Scale can be achieved without complicated capacity
planning
• Infrastructure can be provisioned in minutes
• You are now a part of a fast moving environment that
requires constant attention
What is configuration management?
A practice in which code is used to define and maintain the
state of both new and existing resources throughout their
entire life cycle.
Application Configuration
Compute Resources
Why do I need configuration management?
Operating System and Host Configuration
Compute ResourcesOperating System and
Host ConfigurationApplication Configuration
Amazon Elastic Compute
Cloud (EC2)
On-premises compute
resources (Servers)
…
Files
Directories
Networking
Symlinks
Mounts
Registry Key
Users
Groups
Packages
Filesystems
…
Application dependencies
Application configuration
Service registration
Credentials
…
Infrastructure needs ongoing management
• Package updates?
• New software?
• New configurations?
• New app deployments?
• Environment specific changes?
• Run commands across all hosts?
• Be on top of all running resources?
Ongoing management requires proper tooling
Some common challenges:
• Changing a vhost configuration on every web server across
multiple environments (dev, stage, prod)
• Installing a package on certain hosts to test out newer versions
• Changing LDAP config on every running Amazon EC2 Linux host
What tools can I use to tackle some of these challenges?
What is Chef Automate?
• Refer to your infrastructure as code (cookbooks & recipes)
• Consistently install, configure, manage, deploy and scale
applications
• Align resources with specific policies
• Save time by automating manual tasks
How does it work?
• Simple client-server
architecture
• Connecting resources to a
Chef server
• Resources pull
configuration updates from
the Chef server Config A Config B
How can you set this up?
1. Setup the Chef server with cookbooks, recipes roles.
2. Install the Chef client on the instance (or server).
3. Register the instance with the Chef server as a Chef node.
4. Assign node with a role (e.g. web server, app server, db server).
5. The Chef client asks the Chef server for a set of recipes (instructions).
6. The Chef server determines the applicable recipes (by role).
7. The Chef client applies the recipes on the node by doing a “Chef run”.
8. The Chef client pulls the Chef server every 30 minutes.
How does it look like?
• The Chef client pulls
configuration updates from the
Chef server every 30 minutes.
• The Chef client will only make
configuration changes when
the node is out of spec.
• The Chef client can react to
changes using by using Chef
search.
Chef recipe example – configure Apache
# Install Apache and start the service.
httpd_service ‘default' do
listen_ports ['81', '82']
threadlimit '4096'
action [:create, :start]
end
# Add the site configuration.
httpd_config ‘default' do
instance ‘default'
source ‘mysite.conf.erb'
notifies :restart, 'httpd_service[default]'
end
.....
Chef recipe example – configure Apache
# Create the document root directory.
directory '/var/www/default/public_html' do
recursive true
end
# Write the home page.
file '/var/www/default/public_html/index.html' do
content '<html>This is a placeholder</html>'
mode '0644'
owner 'web_admin'
group 'web_admin'
end
.....
Chef recipe example – configure PHP
# Install the mod_php5 Apache module.
httpd_module 'php5' do
instance ‘default'
end
# Install php5-mysql.
package 'php5-mysql' do
action :install
notifies :restart, 'httpd_service[default]'
end
Get visibility into the state of your nodes
Visibility – A view into convergence, compliance, cookbooks, recipes and more.
Not only a Configuration Management tool
Workflow – A continuous delivery pipeline of infrastructure and applications.
Not only a Configuration Management tool
Compliance - Discovery and analysis of compliance risks across environments
AWS OpsWorks
for Chef Automate
What is AWS OpsWorks for Chef Automate?
The place you go to for configuration management on AWS
Offers a fully managed Chef Automate server
OpsWorks
How can I create an AWS managed Chef server?
Easy to get started, get a Chef Automate server in 10 minutes.
What else can I set up?
Setup a weekly maintenance window
• Automatic security updates
• Automatic Chef version upgrades
What else can I set up?
Setup a daily/weekly backup schedule
What else is left for me to do?
Nothing, this is a fully managed configuration management
service:
• Automatic backups
• Automatic security updates
• Automatic Chef software updates
You can focus on writing cookbooks and recipes that meet
your needs.
What other benefits do I get from the service?
• Automatic instance to Chef server registration
• Secure and easy scaling using Auto Scaling Groups
• No separate license fees, only pay for what you use
• Best practices, AWS support and guidance
Where does it come in the tool chain?
• Bootstrap instances with the right configuration
• Update the configuration of running instances
• Assure instances comply with a pre-defined policy
Can be a part of your Continues Integration and Continues
Delivery pipeline
• Continuous delivery service for fast and
reliable application updates
• Model and visualize your software release
process
• Builds, tests, and deploys your code every
time there is a code change
• Integrates with 3rd party tools and AWS
AWS CodePipeline
Source
Source
CodeCommit
Build
JenkinsOnEC2
Jenkins
Deploy
App
CodeDeploy
PipelineStageAction
Transition
AWS CodePipeline
Source
Source
CodeCommit
Build
JenkinsOnEC2
Jenkins
Deploy
App
CodeDeploy
NotifyDevelopers
Lambda
Parallel actions
AWS CodePipeline
Source
Source
CodeCommit
Build
JenkinsOnEC2
Jenkins
Deploy
App
CodeDeploy
NotifyDevelopers
Lambda
TestAPI
Runscope
Sequential actions
AWS CodePipeline
Live Demo
Your CI/CD Pipeline
You
• Update Chef recipe
• Upload Chef recipe to CodeCommit
AWS CodePipeline
• Updates recipes on Chef server
• Tests recipes on Beta
• Applies recipes on Prod
Source
Cookbooks
CodeCommit
Beta
UploadCookbooks
Jenkins
Prod
ApplyCookbooks
Jenkins
TestCookbooks
Jenkins
How do I get started?
Grab some community cookbooks
https://supermarket.chef.io/
Learn more
https://www.chef.io/automate/
Get started
https://aws.amazon.com/opsworks/
https://aws.amazon.com/codepipeline/
Thank you!
Remember to complete
your evaluations!