improving infrastructure governance on aws - aws june 2016 webinar series

48
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chris Munns, Business Development Manager - DevOps June 2016 DevOps on AWS: Improving Infrastructure Governance on AWS

Upload: amazon-web-services

Post on 08-Jan-2017

1.194 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Chris Munns, Business Development Manager - DevOpsJune 2016

DevOps on AWS:Improving Infrastructure Governance on AWS

Page 2: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

https://secure.flickr.com/photos/mgifford/4525333972

Why are we here today?

Page 3: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Why are we here today?

Using cloud based infrastructure changes how to think about governing our infrastructure: Infrastructure can be provisioned in seconds.. and go

away just as quickly! Development teams expect a higher level of flexibility and

self control in interfacing with their infrastructure needs Being API driven means that the way people provision and

manage infrastructure in the cloud has changed compared to on-premises

Page 4: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Why are we here today?

That doesn’t mean that our basic governance needs change: We still need to have some ability to drive best

practices/patterns in our organizations We need to make sure that we’re able to audit

and track changes to our infrastructure for both regulation and security purposes

We need to make sure that we understand how resources are related and integrated

Page 5: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

What can we do?

There are a few areas to focus on that can help us accomplish both the freedom to rapidly provision, manage, and update our infrastructure while meeting our governance needs: Policy as Code Infrastructure standardization (via code!) Self service environments Logging/Auditing/Reacting to infrastructure change

Page 6: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Policy as Code builds off of infrastructure as code practices by

allowing organizations to codify infrastructure and system configurations

allowing them to monitor and enforce compliance dynamically and at scale.

Page 7: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Infrastructure as Code is a practice in which infrastructure is provisioned

and managed using code and software development techniques,

such as version control and continuous integration.

Page 8: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Infrastructure as Code “levels”

AWS Resources

Operating System and Host Configuration

Application Configuration

Page 9: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Infrastructure as Code “levels”

AWS Resources

Operating System and Host Configuration

Application Configuration

allOfThis == $Code

Page 10: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Browse and launch

AWS ConfigAWS CloudTrail

Use and modify

Users Admin

Putting the AWS Management services together

AWS Service Catalog

Provision with Tags

API calls Configuration checks and reactions to change

Troubleshoot and Audit

Page 11: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Create templates of your infrastructure

CloudFormation provisions AWS resources based on dependency needs

Version control/replicate/update templates like code

Integrates with development, CI/CD, management tools

AWS CloudFormatio

n

Page 12: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Template CloudFormation Stack

JSON formatted fileParameter definitionResource creation

Configuration actions

Configured AWS resourcesComprehensive service support

Service event awareCustomizable

FrameworkStack creationStack updates

Error detection and rollback

CloudFormation – Components & Technology

Page 13: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Template File Defining Stack

The entire infrastructure can be represented in an AWS

CloudFormation template.

Many Stacks & Environments from One Template

Page 14: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Template File Defining Stack

The entire infrastructure can be represented in an AWS

CloudFormation template.

Use the version control system of your choice to store and track changes to this template

Many Stacks & Environments from One Template

GitPerforce

SVN…

Page 15: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Template File Defining Stack

GitPerforce

SVN…

Dev

Test

Prod

The entire infrastructure can be represented in an AWS

CloudFormation template.

Use the version control system of your choice to store and track changes to this template

Build out multiple environments, such as for Development, Test, Production and even DR using the same template

Many Stacks & Environments from One Template

Page 16: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

CloudFormation example use cases:

Have “full stack” templates that can be used to stand up common application patterns inside your organization such as a 3-tier application template that: uses Lambda custom resources to look up appropriate VPC

information (VPC ID, Subnets, etc) based on tags creates an Elastic Beanstalk environment that supports Multi-AZ,

AutoScaling, CloudWatch Metrics, and Elastic Load Balancing contains security controls such as AWS Identity and Access

Management (IAM) roles, profiles, and policies, and Security Groups allows the user to specify the language of their application allows a user to specify which database they want (SQL or NoSQL)

and then creates the appropriate resource

Page 17: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Using Parameters and Conditionals are two key ways to make a single template much more dynamic:"Parameters" : { "Database": { "Type" : "String", "Default" : "RDS", "AllowedValues" : ["RDS", "DynamoDB", "None"], "Description" : "Database to create. Select None if using an existing database.” } },

"Conditions" : { "CreateRDS" : {"Fn::Equals" : [{"Ref" : "Database"}, "RDS"]}, "CreateDynamoDB" : {"Fn::Equals" : [{"Ref" : "Database"}, "DynamoDB"]}, "CreateNone" : {"Fn::Equals" : [{"Ref" : "Database"}, "None"]} },

“Resources” : { ”RDSdb01" : { "Condition" : " CreateRDS ", "Type" : "AWS::RDS::Instance",

Page 18: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

https://secure.flickr.com/photos/wscullin/3770015991

Now that we have these templates, what can we do to help simplify our developers’ lives further and increase our infrastructure standardization?

Page 19: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Customized catalogs of products

Manage products centrally

Personalized, self-service portal

Integrate with existing systemsAWS Service Catalog

Page 20: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

What is AWS Service Catalog?

AWS Service Catalog allows organizations to create and manage catalogs of IT services. It enables users to quickly deploy the approved IT services they need in a self-service manner.

Organizations Developers

ControlStandardization

Governance

AgilitySelf-service

Time to market

Page 21: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Creates portfolio

Adds constraints and grant access

1

4

5

AdministratorPortfolio

Users

Browse Products

6Launch ProductsAWS CloudFormation template

Creates product3Authors template2

ProductX

ProductY

ProductZ

7Deploys stacks

EventsEvents

88

Service Catalog

Create custom services

and grant access

Use a personalized

portal to find and launch services

Page 22: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Service Catalog use cases:

You can remove the need for developers to understand how all AWS services work. Treat infrastructure provisioning like buying components from a retail site: provide standardized Service Catalog products around

common internal application frameworks/architectural patterns provide common application component products such as

databases, queues, caches, worker tiers, etc build logging, monitoring, metrics into these stacks leverage service discovery tools when possible build in the same best practices across development, staging,

production environments with these provided products

Page 23: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

We’ve helped solve some of our developer’s access and standardization issues, but how can we now go about auditing changes to our infrastructure?

https://www.flickr.com/photos/atoach/7623237104

Page 24: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

AWS CloudTrail

Records AWS API calls for your account

Delivers log files of API calls to S3

Delivery typically within 15 minutes of API call

Logs contain detailed information

Log files can be encrypted and have their integrity verified by you

Page 25: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

AWS CloudTrail

CloudTrail can help you achieve many tasks Security analysis Track changes to AWS resources, for

example VPC security groups and NACLs Compliance – log and understand AWS API

call history Prove that you did not:

Use the wrong region Use services you don’t want

Troubleshoot operational issues – quickly identify the most recent changes to your environment

Page 26: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

AWS CloudTrail logs can be delivered cross-account

CloudTrail can help you achieve many tasks Accounts can send their trails to a central

account Central account can then do analytics Central account can:

Redistribute the trails Grant access to the trails Filter and reformat Trails (to meet

privacy requirements)

Page 27: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

CloudTrail

Page 28: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

CloudTrail – Amazon CloudWatch Logs Integration

Page 29: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Continuous recording

Inventory of AWS resources

New & deleted resources

Configuration change & compliance notifications

AWS Config

Page 30: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Relationships

Bi-directional map of dependencies automatically assigned

Change to a resource propagates to create Configuration Items for related resources

Example: Security Group sg-10dk8ej and EC2 instance i-123a3d9 are “associated with” each other

Page 31: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Configuration Item

All configuration attributes

Normalized

Point in time

Captured on configuration change

Page 32: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Component Description Contains

Metadata Information about this configuration item

Version ID, Configuration item ID, Time when the configuration item was captured, State ID indicating the ordering of the configuration items of a resource, MD5Hash, etc.

Common Attributes Resource attributes Resource ID, tags, Resource type. Amazon Resource Name (ARN)Availability Zone, etc.

Relationships How the resource is related to other resources associated with the account

EBS volume vol-1234567 is attached to an EC2 instance i-a1b2c3d4

Current Configuration Information returned through a call to the Describe or List API of the resource

e.g. for EBS VolumeState of DeleteOnTermination flagType of volume. For example, gp2, io1, or standard

Related Events The AWS CloudTrail events that are related to the current configuration of the resource

AWS CloudTrail event ID

Configuration Item

Page 33: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series
Page 34: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Sample Configuration Item "configurationItemVersion": "1.0", "configurationItemCaptureTime": "2014…", "configurationStateID": “….", "configurationItemStatus": "OK", "resourceId": "vol-ce676ccc", "arn": "arn:aws:us-west-………", "accountId": "12345678910", "availibilityZone": "us-west-2b", "resourceType": "AWS::EC2::Volume", "resourceCreationTime": "2014-02..", "tags": {}, "relationships": [ { "resourceId": "i-344c463d", "resourceType": "AWS::EC2::Instance", "name": "Attached to Instance" } ],

"relatedEvents": [ "06c12a39-eb35-11de-ae07-db69edbb1e4", ],

Metadata

Common Attributes

Relationships

Related Events

Page 35: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Sample Configuration Item"configuration": { "volumeId": "vol-ce676ccc", "size": 1, "snapshotId": "", "availabilityZone": "us-west-2b", "state": "in-use", "createTime": "2014-02-……", "attachments": [ { "volumeId": "vol-ce676ccc", "instanceId": "i-344c463d", "device": "/dev/sdf", "state": "attached", "attachTime": "2014-03-", "deleteOnTermination": false } ], "tags": [ { "tagName": "environment", "tagValue": "PROD"

Configuration

Page 36: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Having this data is one thing, but how can we react to change?

https://www.flickr.com/photos/livenature/204420128/

Page 37: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

CloudTrail – Amazon CloudWatch Logs Integration

Page 38: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

CloudTrail – Amazon CloudWatch Logs Integration

Trigger a CloudWatch Alarm based on API call!

Page 39: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Check configuration changes

Pre-built rules provided by AWS

Custom rules using AWS Lambda

Continuous assessment

Dashboard Compliance visualization Identify offending changes

GitHub repo: Community sourced rules

AWS Config Rules

Page 40: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Normalize

RecordChanging Resources

AWS Config & Config RulesDeliver

Stream

Snapshot (ex. 2014-11-05)AWS Config

APIs

Store

History

Rules

Page 41: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Config Rules

Page 42: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Config Rules

Page 43: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Config Rules GitHub repository

Page 44: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

AWS Management Tools Partners

Page 45: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

FIN, ACKWe’ve seen a quick run through today of the ways you can improve your governance on AWS: Treat your infrastructure and host configuration as code! This lends itself to being able to use services like Service

Catalog to enable self-service in your organization Track, trend, and alert on CloudTrail API logs to keep on top

of access to your AWS resources Use Config and Config Rules to understand the relationship

between resources and react to policy violations Putting all this together is what gives you Policy as Code!

Page 46: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Browse and launch

AWS ConfigAWS CloudTrail

Use and modify

Users Admin

Putting the AWS Management services together

AWS Service Catalog

Provision with Tags

API calls Configuration checks and reactions to change

Troubleshoot and Audit

Page 47: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

But wait, there’s more!

Resources to learn more: More on DevOps: https://aws.amazon.com/devops/ AWS Management Services: https://aws.amazon.com/products/management/ AWS CloudFormation

https://aws.amazon.com/cloudformation/ AWS Service Catalog

https://aws.amazon.com/servicecatalog/ AWS CloudTrail

https://aws.amazon.com/cloudtrail/ AWS Config / Config Rules

https://aws.amazon.com/config/ GitHub repo: https://github.com/awslabs/aws-config-rules

Page 48: Improving Infrastructure Governance on AWS - AWS June 2016 Webinar Series

Thank you!

Happy Deploying!