(dvo310) benefit from devops when moving to aws for windows

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Tom Fuller, Enterprise Solutions Architect

October 2015

Benefit from DevOps When

Moving to AWS for Windows

What to Expect from the Session

1. Baseline the roles and responsibilities of

DevOps teams

2. Highlight Windows-specific integration

scenarios and AWS service capabilities for

pushing code and operating code

3. Demonstrate patterns of success that

customers have seen with Amazon Web Services

for Microsoft Windows Server

Disclaimer

1. There is no singular approach to DevOps.

2. Innovations are not frozen in time!

3. Container technologies (like Docker) have a big role in DevOps but that is not a focus of this session.

Windows platform challenges

Tight dependencies on domain infrastructure

Legacy security models create tight coupling

Reboots are not fun to automate

Lots of heavy software packages

Bootstrapping options limited

Patterns of success: Reduce dependencies

Start out by asking yourself if you really need things like:

• Domain infrastructure

• Windows integrated security

• Remote access

If you have to do it, then avoid taking a dependency on external services

• A VPN is fine if you avoid it as a critical path

• Even Direct Connect can be a potential soft spot in a distributed solution

• Your domain infrastructure knows how to operate in a distributed network topology

Tip: Windows integrated

security requires domain

joins, which require

reboots and can slow

down your automation.

The two focus areas of DevOps teams

Pushing code Operating code

5 new

features

done by

Friday?

Marketing is

driving a ton of

traffic to our site

this weekend …

CRAP!!!!

Sigh … 5

more bugs

to squash!

Ugh! A

meeting invite

to a security

audit review?

Should I perf

test that code

change I

checked in?

I must patch

the servers

vulnerable to

that new attack

I read about!

I NEED MORE

HOURS IN

THE DAY!!!!!

I NEED

MORE

HOURS IN

THE DAY!!!!!

Pushing code

automation

What is continuous delivery?

developers customers

releasetestbuild

delivery pipeline

Software development lifecycle

Continuous delivery (CD) is a software development methodology where the

release process is automated. Every software change is automatically built, tested,

and deployed to production. Before the final push to production, a person, an

automated test, or a business rule decides when the final push should occur.

What skills do you need?

Software development (e.g. coding, scripting)

Tooling experience

Windows administration

AWS services knowledge

Patterns of success: Start with devs

If you are part of an Ops/DevOps/ DevTools/CoreInfra … it’s your job to help developers:

• Move fast on your company’s product/mission

• Not have to know how every single bit is glued together

• Get their code where it needs to go when it needs to be deployed Developer

AWS services for deploying/operating apps

Convenience Control

Higher-level services Do it yourself

AWS

Elastic Beanstalk

AWS

OpsWorks

AWS

CloudFormation

AWS

CodeDeploy

Elastic Beanstalk

Easy-to-use service for deploying and scaling web applications

• Supports multiple languages

• Automatically handles the deployment and scaling

• You retain full control of the resources

AMIs with specific configurations• .NET 2/4

• IIS 7.5/8

• Windows Server 2008 R2, 2012 R2

Automatically pulls code from S3

Visual Studio plugin

Ops features• URL swapping

• Rolling upgrades

• Log pulling

• App versioning

Tip: Fixed size or

percentage-based

rolling upgrade options

exist.

Chef for Windows

Ruby-based config management framework with distributed architecture (pull model)

Remote execution via WinRM (knife plugin, Chef provisioning, or PowerShell)

Use Chef resources (files, services, registry keys, Windows features, etc.) or Windows DSC resources

Includes placeholders, templates, attribute files, data bags, and reuse options

Tightly integrated with PowerShell

Large ecosystem of cookbooks in the Chef Supermarket

AWS OpsWorks for Windows

Application management service that makes it easy to deploy and operate apps

• Includes automation to scale

• Manage and reuse stacks and layers

• Maintain configuration as code

Released May of 2015

Supports Windows Server 2012 R2 and Chef 12.2

Stacks can reference Supermarket or

Custom cookbooks

Scaling types (24/7 or time-based)

Lifecycle events can trigger recipes

Tip: One example of an

AWS cookbook is

“windows_ad” which can be

used to configure a domain

controller and domain join

nodes.

CodeDeploy

Service for automating code deployments to any instance (including Windows 2008 R2 or 2012 R2)

• Avoid downtime during development

• Eliminate error-prone, manual steps

• Target one or thousands of instances

Lifecycle event hooks can trigger scripts (.bat, PowerShell)• BeforeInstall

• AfterInstall

• Many more…

Key concepts• Application

• Revision

• Deployment group

• Deployment configuration

• Rollout speed (one at a time, half, full)

Code can be sourced using Git or S3

Code is copied to destination using AppSpec YAML

Tip: Visual Studio can

integrate directly with GitHub

or you can build an

automation pipeline that puts

zipped artifacts in S3 after

tests pass!

CloudFormation

Easy way to manage a collection of related AWS resources, provisioning and updating them in a predictable fashion

JSON templates for AWS resource creation

Templates create stacks

Use Visual Studio to edit CloudFormation template (CFT) files

Scripting options• PowerShell

• Cmd.exe

Many sample templates for Windows• Active Directory, ADFS

• Sharepoint Server 2013

• Lync Server 2013

• Exchange Server 2013

• Windows PowerShell DSC

• SQL Server 2012 and 2014 with WSFC

• Remote Desktop Gateway

• Web Application Proxy

• CodeDeploy agents

Tip: A DIY way to implement

end-to-end automation is to

use CFTs to deploy the

infrastructure and then AWS

CodeDeploy to roll out your

application.

Bootstrapping

Be sure you understand all the different options and

their unique behaviors• CloudFormation and WaitCondition vs. cfn-signal

• CloudFormation and alphabetical ordering

• CodeDeploy and lifecycle event timeouts

• OpsWorks and recipe dependencies/caching

Beware of assumptions about prerequisites

Typical to require local admin accounts to run remotely

This is code … remember to build reusable

components!

Tip: This is where

PowerShell becomes

very powerful!

PowerShell on AWS

Start with AWS Tools for Windows PowerShell• (900+) cmdlets written to line up with the service APIs (ex. Get-

CDApplicationRevisionList)

• Often a superset of what you have in the console

• Rich documentation with examples

• Bundled with AWS SDK for .NET

You can use PowerShell version 2, 3, or 4

Profile options and defaulting features exist

Tip: If you’re using version

3 or 4, then you will get the

AWSPowershell module

auto-loaded, but for version

2, you will need to run

Import-Module.

Pushing code using Visual Studio + GitHub + CodeDeploy

AWS CodeDeploy

Operating code

Patterns of success: Zero remote access

If you are forced to log in to an instance, then you are broken• Agility is diminished

• Specialized configuration is a huge risk over time

• Every node should be fully disposable

Some services like OpsWorks can provide temporary access

Centralized remote configuration tools are driving consistency through config audits

Access to dev/test is still typically necessary, but should still be treated as disposable

Packaging/baking AMIs

#1 reason to bake is to decrease your boot time • Software packages that require painful/long setup

• Standard software that must be there at startup

• Any configuration items that cannot be remotely sourced or automated

Strike a balance between those things that change often and those that don’t

AWS provides easy interfaces to create the AMI or import the AMI

Third-party tooling can be helpful• Packer.IO (includes Windows)

AMI Instances

Tip: Starting from an existing

Amazon-provided image is

recommended. Once done

customizing, you should stop

the instance and capture the

AMI.

Using EC2 instances

1. Classic DNS cutover

2. Swap Auto Scaling groups

3. Swap launch configurations

Deploying apps using blue/green patterns

Using EC2 Container Service

1. Swap ECS services via DNS

2. Swap ECS services behind ELB

3. Swap ECS task definitions

Pattern: swap Auto Scaling Groups

Deployment process:

• Amazon Elastic Load Balancer

(ELB) outside the environment

boundary

• Start with current Auto Scaling

Group (ASG)

• Deploy & scale out new ASG

• Test green stack

• Register green ASG with ELB

• Remove blue ASG from ELB

Health checking

There are multiple ways to evaluate health• ELB health checks

• Route 53 health checks

• CloudWatch

CloudWatch alarms• auto-recover an instance

• CloudWatch conditions trigger notifications

CloudWatch Logs (plugin) on Windows• Event tracing for Windows log sources

• IIS request logs

• Perf counters

• Text-based log files

• Windows event logs

Tip: The CloudWatch Logs

plugin is configured using the

EC2Config service and is

disabled by default. You will

need to configure

appropriate IAM permissions

for it to work.

Log aggregation and analysis

AWS provides a lot of log sources• CloudWatch data

• CloudWatch Logs data

• CloudTrail data

• VPC flow logs

• S3 access logs

Some services provide centralized logging• Elastic Beanstalk

• OpsWorks

Analyzing those logs can be done with different techniques

• Using native AWS capabilities (ex. Amazon Kinesis subscriptions to S3)

• Third-party products

• DIY

Tip: A CFT is available that will

set up an Amazon Kinesis

subscription for CloudWatch

Logs that will feed an

Elasticsearch cluster, which can

be visualized using Kibana

dashboards.

Patterns of success: Adhere to principles

You need to draw that line in the sand on• No remote access

• Must have test automation

• No delayed quarterly big bang releases

• No maintenance windows

Patching is no different from rolling a new release

Consistency only happens if you treat it all as disposable

Blue/green deployments are the norm• Services like Route 53 help you deliver

Find the way that works best for your team and your tool preferences

Case studies and conclusion

Case study: Customer X

DevOps team brought in 6 months ago to install continuous delivery practices

First order of business was development of best practices

Initial work with Chef targeted Linux

Early evaluations were Chef and Ansible

They have 30 stacks and for many of them they need to deploy multiple times a day

Case study: Customer X

Development stack and pipeline all Windows-based• Visual Studio

• Team Foundation Server (+ TFS for Git)

• TeamCity (Build agents running in AWS)

Deploying IIS and Windows service apps

Focused on securely reducing friction to CD• Nothing is joined to a domain

• Custom cookbooks enforce tagging

• Production AWS account is totally isolated

• OpsWorks for Windows was a perfect fit

Commit

Build

server

Git pull

OpsWorks

Node Node

Auto Scaling group

Git pull

Refresh

instances

using Chef

Case study: Customer Y

Dynamic eCommerce web property requiring proactive scaling (follow the sun)

Infrastructure as code has been fully adopted

DevOps team is responsible for all updates

Solution made up of multiple tiers (web, app, data)

Single region deployment with multi-AZ

Made use of Windows AD for part of the solution

Case study: Customer Y

Full Windows .NET solution• ASP.NET on the web and app tier

• SQL Server back end (not on RDS)

Mostly an EC2 deployment• Took advantage of Chef on Windows (not using

OpsWorks)

• Used their own BIND servers to handle DNS

Utilized VPC peering to enable central control

Implemented log shipping between UAT and Prod

• Combined with Route 53 enabled environment swapping

Web instances

Amazon

Route 53

AZ AZ

Production

Web instances

AZ AZ

UAT

App instancesApp instances

Shared VPC

DC1 DC2

Bind

1

Bind

2Chef1 Chef2

Peering

connections

Case studies: Lessons learned

It doesn’t take as long as you think to transform … incredible progress was made in 6 months

Dynamic tiers of your solution need to be lean and boot fast

It’s critical that you treat everything as disposable

Using larger instance types at startup with a resize helped save time

When deciding between deployment tooling, use the “path of least resistance”

Summary

1. DevOps has no singular way … but there are

clear patterns of success

2. Always remember the two sides of the coin and

focus on the principles

3. Windows Server can play a big part in your

journey to the cloud on AWS

Thank you!

Remember to complete

your evaluations!

Related Sessions

DEV202 – Under the Desk to the AWS Cloud with

Windows PowerShell

DVO303 – Scaling Infrastructure Operations with AWS

Service Catalog, AWS Config, and AWS CloudTrail

DVO301 – AWS OpsWorks Under the Hood

DVO401 – Blue/Green Deployments on AWS

Additional Resources

Continuous Delivery: Huge Benefits, but Challenges Too

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=

7006384

Automatic Push from GitHub to CodeDeploy

http://blogs.aws.amazon.com/application-

management/post/Tx33XKAKURCCW83/Automatically-Deploy-from-

GitHub-Using-AWS-CodeDeploy

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7006384

http://blogs.aws.amazon.com/application-management/post/Tx33XKAKURCCW83/Automatically-Deploy-from-GitHub-Using-AWS-CodeDeploy

Demo: Provision a workspace

Demo: Deploy Visual Studio

Demo: Add AWS Toolkit for Visual Studio

Demo: Set up a GitHub repo

Demo: Configure VS to use GitHub

Demo: Create IAM instance profile for CodeDeploy

server

Demo: Create AMI with .NET 3.5 installed

Demo: Create CodeDeploy EC2 instance with agent

install

Demo: Validate agent install

Demo: Create a CodeDeploy application and

deployment group

Demo: Plan a revision

Demo: Authorize GitHub to work with CodeDeploy

Demo: Write some.bat scripts and a YAML file

Demo: Sync to GitHub and kick off deployment

Demo: Deploy DynamoDBTestClient and review results

(dvo310) benefit from devops when moving to aws for windows

Technology