devops at amazon: a look at our tools and processes

©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

DevOps at Amazon:A Look at Our Tools and Processes

Rob Brigham

Product Management, AWS Developer Tools

Prelude

Agenda

• The Amazon DevOps story

• New developer tools for everyone– AWS CodeDeploy

– AWS CodePipeline

– AWS CodeCommit

What is DevOps?

DevOps = efficiencies that speed up this lifecycle

developers customers

releasetestbuild

plan monitor

delivery pipeline

feedback loop

Software development lifecycle


The Amazon DevOps story


Monolith development lifecycle

developers

releasetestbuild

delivery pipelineapp

• Service-Oriented

Architecture (SOA)

• Single-purpose

• Connect only

through APIs

• “Microservices”

• Two-pizza teams

• Full ownership

• Full accountability

• Aligned incentives

• “DevOps”

Missing tools

developers delivery pipelineservices

???

• Self-service

• Technology-agnostic

• Encourage best

practices

• Single-purpose

services

• Deployment service

• No downtime

deployments

• Health checking

• Versioned artifacts

and rollbacks

• Continuous delivery

• From check-in to production

• Automated actions and transitions

• >90% of teams

Pipelines

Microservice development lifecycle

developers delivery pipelinesservices

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

= 50 million deployments a year

Thousands of teams

× Microservice architecture

× Continuous delivery

× Multiple environments

Setting up a delivery pipeline

1) Automate your software deployments

2) Automate your release process

deploy deploy deploy deploy

development test staging production

release: source > build > test > staging > production

Setting up a delivery pipeline

1) Automate your software deployments

AWS CodeDeploy

2) Automate your release process

AWS CodePipeline


AWS CodeDeploy


AWS CodeDeploy

• Scale from one instance to thousands

• Deploy to any server: AWS or on-premises

• Centrally control and monitor

Test

CodeDeployv1, v2, v3

Production

Dev

Coordinate automated deployments, just like Amazon

application

revisions

deployment groups

1) Package app 2) Specify targets 3) Deploy!

version: 0.0

os: linux

files:

- source: chef/

destination: /etc/chef/codedeploy

- source: target/hello.war

destination: /var/lib/tomcat6/webapps

hooks:

ApplicationStop:

- location: deploy_hooks/stop-tomcat.sh

BeforeInstall:

- location: deploy_hooks/install-chef.sh

- location: deploy_hooks/chef-solo.sh

ApplicationStart:

- location: deploy_hooks/start-tomcat.sh

ValidateService:

- location: deploy_hooks/verify_service.sh


Agent Agent Agent

Test

Agent Agent

Agent Agent

Agent

Agent

Production

Deployment groupDeployment group

Group instances by:

• Auto Scaling group

• Amazon EC2 tag

• On-premises tag


aws deploy create-deployment \

--application-name MyApp \

--deployment-group-name TargetGroup \

--s3-location bucket=MyBucket,key=MyApp.zip

AWS CLI & SDKs

AWS Console

CI / CD Partners

GitHub

Rolling update – Deploy without downtime

v1v1 v1

Load Balancer


v1v2 v1

Load Balancer


v2v2 v1

Load Balancer


v2v2v2

Load Balancer


v2v2 v2

Load Balancer

Health tracking – Catch deployment problems

v2v2 v2

Load Balancer


v3 v2 v2Stop

Load Balancer


v2v2 v2

Load Balancer

Rollback


v2v2 v2

Load Balancer

Deployment config – Choose speed

v2 v1 v1 v1 v1 v1 v1 v1

v2 v2 v2 v2 v1 v1 v1 v1

v2 v2 v2 v2 v2 v2 v2 v2

one at a time

half at a time

all at once

Product integrations


AWS CodePipeline


AWS CodePipeline

• Model your release process

• Integrate with your systems

• Visually monitor your pipeline status

Continuous delivery and release automation, just like Amazon

Build

1) Build

2) Unit test

1) Deploy

2) UI test

Source Beta Production

1) Deploy

2) Perf test

Gamma

1) Deploy canary

2) Deploy region 1

3) Deploy region 2

1) Pull

Why automate your release process?

Launch

quickly

Ensure

quality


AWS CodeCommit


Source control in the cloud

Secure Fully

managed

High

availability

Store

anything

AWS CodeCommit

• Data redundancy across Availability Zones

• Data-at-rest encryption

• Integrated with AWS Identity and Access Management

• No repo size limit

git pull/push CodeCommit

Git objects in

Amazon S3

Git index in

Amazon

DynamoDB

Encryption key

in AWS KMS

SSH or HTTPS

Secure, scalable, and managed Git source control

Same Git experience

$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli

Cloning into 'aws-cli'...

Receiving objects: 100% (16032/16032), 5.55 MiB | 1.25 MiB/s, done.

Resolving deltas: 100% (9900/9900), done.

Checking connectivity... done.

$ nano README.rst

$ git commit -am 'updated README'

[master 4fa0318] updated README

1 file changed, 1 insertion(+)

$ git push

Counting objects: 3, done.

Delta compression using up to 4 threads.

Compressing objects: 100% (3/3), done.

Writing objects: 100% (3/3), 297 bytes | 0 bytes/s, done.

Total 3 (delta 2), reused 0 (delta 0)

remote:

To https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli

4dacd6d..4fa0318 master -> master


Summary


AWS Code services

CodeCommitcoming soon

CodePipelinecoming soon

CodeDeploylaunched Nov 2014

CHICAGO

Your Feedback is Important to AWSPlease complete the session evaluation. Tell us what you think!

devops at amazon: a look at our tools and processes

Technology

amazon web services

rights reserved aws

rights reserved devops

aws developer tools

aws codedeploy scale

etcchefcodedeploy source

incentives devops

chef destination