best practices for getting started with aws
TRANSCRIPT
Best Practices for Getting Started with AWS
[email protected]@IanMmmm
Ian Massingham — Technical Evangelist
Getting Started with AWS: Agenda
Eight best practices you should focus on when getting started
Resources you can use to learn more
Getting Started with AWS
http://aws.amazon.com/getting-started/
Getting Started with AWS
Choose Your First Use Case Well
1
Chose Your First Use Case Well
Make your first project a S.M.A.R.T one
Chose Your First Use Case Well
Dev & Test
Spin environments up and down on demand
Decouple development and test environments
from operations constraints
Explore elasticity in a sandboxed environment
Make your first project a S.M.A.R.T one
Chose Your First Use Case Well
Dev & Test
Spin environments up and down on demand
Decouple development and test environments
from operations constraints
Explore elasticity in a sandboxed environment
Backup & DR Take part of your data or
business applications step- by-step into non-
production DR use
Understand cloud dynamics and test during
controlled failover
Make your first project a S.M.A.R.T one
Chose Your First Use Case Well
Dev & Test
Spin environments up and down on demand
Decouple development and test environments
from operations constraints
Explore elasticity in a sandboxed environment
Backup & DR Take part of your data or
business applications step- by-step into non-
production DR use
Understand cloud dynamics and test during
controlled failover
Greenfield Project
Embody best practice of cloud computing in
unconstrained greenfield projects
Self contained web projects, document
archiving etc
Make your first project a S.M.A.R.T one
Chose Your First Use Case Well
Dev & Test
Spin environments up and down on demand
Decouple development and test environments
from operations constraints
Explore elasticity in a sandboxed environment
Backup & DR Take part of your data or
business applications step- by-step into non-
production DR use
Understand cloud dynamics and test during
controlled failover
Greenfield Project
Embody best practice of cloud computing in
unconstrained greenfield projects
Self contained web projects, document
archiving etc
Pain point
Move specific service aspects causing undue cost or management
burden
Workflows, search indexing, media
streaming, document archiving, constrained
databases
Make your first project a S.M.A.R.T one
Plan Evolution and Set Goals
Understand services
Test performance
Architect for scale
Develop team capabilities
Implement monitoring
Change control and management
Security management
Scalability
Automate corrective actions
Auto-scaling
Zero downtime deployments
System backup and recovery
Proof of Concept Production Automation
Sam
ple
Act
iviti
es
Lay Out Your Foundations
2
Accounts
Create an account structure that makes sense
Use accounts like environments where you need separation and
control
e.g. Dev Sandboxes Test Environments
Business Units Products & Services
Lay Out Your Foundations
BillingAccounts
Create an account structure that makes sense
Use accounts like environments where you need separation and
control
e.g. Dev Sandboxes Test Environments
Business Units Products & Services
Control access to billing information
Use IAM users to keep billing information in the master account
Consolidate billing into a single account
Let one account pick up the bill for multiple ‘sub accounts’
Setup billing alerts and automated bill reporting
Get CloudWatch notifications when billing reaches a point and output
csv reports to S3 for analysis
Lay Out Your Foundations
Enable delivery of billing reports with resources & tags
Billing preferences
Billing Settings
BillingMaster Account
Billing
Consolidated Billing Relationship
Master [email protected]
Division [email protected]
User2 Dev2 Admin2
IAM
Billing
Consolidated Billing Relationship
Master [email protected]
Division [email protected]
User2 Dev2 Admin2
IAM
Tags: Own=Div Proj=P
Tags: Own=Div Proj=Q
Tags: Own=Div Proj=R
Tags: (key-value) e.g Own=Div
Proj=R
Billing
Consolidated Billing Relationships
Master [email protected]
Business Unit [email protected]
User3 Dev3 Admin3
IAM
Tags: Own=BusC Proj=X
Tags: Own=BusC Proj=Y
Tags: Own=BusC Proj=Z
Division [email protected]
User2 Dev2 Admin2
IAM
Tags: Own=Div Proj=P
Tags: Own=Div Proj=Q
Tags: Own=Div Proj=R
Operating Co. [email protected]
User1 Dev1 Admin1
IAM
Tags: Own=OpCo Proj=A
Tags: Own=OpCo Proj=B
Tags: Own=OpCo Proj=C
Billing
Consolidated Billing Relationships
Master [email protected]
Business Unit [email protected]
User3 Dev3 Admin3
IAM
Tags: Own=BusC Proj=X
Tags: Own=BusC Proj=Y
Tags: Own=BusC Proj=Z
Division [email protected]
User2 Dev2 Admin2
IAM
Tags: Own=Div Proj=P
Tags: Own=Div Proj=Q
Tags: Own=Div Proj=R
Operating Co. [email protected]
User1 Dev1 Admin1
IAM
Tags: Own=OpCo Proj=A
Tags: Own=OpCo Proj=B
Tags: Own=OpCo Proj=C
Alert:
Reached $500 Alert:
Reached $3500 Alert:
Reached $1250
S3CSV
Billing
ANALYSIS
Programmatic Billing Access
Consolidated Billing Relationships
Master [email protected]
Business Unit [email protected]
User3 Dev3 Admin3
IAM
Tags: Own=BusC Proj=X
Tags: Own=BusC Proj=Y
Tags: Own=BusC Proj=Z
Division [email protected]
User2 Dev2 Admin2
IAM
Tags: Own=Div Proj=P
Tags: Own=Div Proj=Q
Tags: Own=Div Proj=R
Operating Co. [email protected]
User1 Dev1 Admin1
IAM
Tags: Own=OpCo Proj=A
Tags: Own=OpCo Proj=B
Tags: Own=OpCo Proj=C
S3CSV
Billing
ANALYSIS
Programmatic Billing Access
Consolidated Billing Relationships
Master [email protected]
Business Unit [email protected]
User3 Dev3 Admin3
IAM
Tags: Own=BusC Proj=X
Tags: Own=BusC Proj=Y
Tags: Own=BusC Proj=Z
Division [email protected]
User2 Dev2 Admin2
IAM
Tags: Own=Div Proj=P
Tags: Own=Div Proj=Q
Tags: Own=Div Proj=R
Operating Co. [email protected]
User1 Dev1 Admin1
IAM
Tags: Own=OpCo Proj=A
Tags: Own=OpCo Proj=B
Tags: Own=OpCo Proj=C
S3CSV
Billing
ANALYSIS
Programmatic Billing Access
Consolidated Billing Relationships
Master [email protected]
Business Unit [email protected]
User3 Dev3 Admin3
IAM
Tags: Own=BusC Proj=X
Tags: Own=BusC Proj=Y
Tags: Own=BusC Proj=Z
Division [email protected]
User2 Dev2 Admin2
IAM
Tags: Own=Div Proj=P
Tags: Own=Div Proj=Q
Tags: Own=Div Proj=R
Operating Co. [email protected]
User1 Dev1 Admin1
IAM
Tags: Own=OpCo Proj=A
Tags: Own=OpCo Proj=B
Tags: Own=OpCo Proj=C
3rd Party Cost Management Tools
Access KeysBillingAccounts
Create an account structure that makes sense
Use accounts like environments where you need separation and
control
e.g. Dev Sandboxes Test Environments
Business Units Products & Services
Control access to billing information
Use IAM users to keep billing information in the master account
Consolidate billing into a single account
Let one account pick up the bill for multiple ‘sub accounts’
Setup billing alerts and automated bill reporting
Get CloudWatch notifications when billing reaches a point and output
csv reports to S3 for analysis
Decide upon a key management strategy
Control access to EC2 instances via SSH and embedded public key:
e.g. EC2 Key Pair per group of instances, EC2 Key Pair per
account
Consider SSH key rotation & automation
Limit exposure to private key compromise by rotating keys and replacing authorized_keys listings
on running instances Consider bootstrap automation to
grant developer access with developer unique keypairs
Lay Out Your Foundations
Groups & RolesAccess KeysBillingAccounts
Create an account structure that makes sense
Use accounts like environments where you need separation and
control
e.g. Dev Sandboxes Test Environments
Business Units Products & Services
Control access to billing information
Use IAM users to keep billing information in the master account
Consolidate billing into a single account
Let one account pick up the bill for multiple ‘sub accounts’
Setup billing alerts and automated bill reporting
Get CloudWatch notifications when billing reaches a point and output
csv reports to S3 for analysis
Decide upon a key management strategy
Control access to EC2 instances via SSH and embedded public key:
e.g. EC2 Key Pair per group of instances, EC2 Key Pair per
account
Consider SSH key rotation & automation
Limit exposure to private key compromise by rotating keys and replacing authorized_keys listings
on running instances Consider bootstrap automation to
grant developer access with developer unique keypairs
Use IAM Groups to manage console users and API
access Provide developers with IAM user
login and unique API access credentials
Control & restrict what IAM users can do by placing them in groups
with associated policies
Assign EC2 Instances IAM roles
Let AWS manage API access credentials on running instances by assigning a system entitlement to
an instance e.g. instance can only read S3
bucket
Lay Out Your Foundations
Identity & Access Management - IAMAccount
ApplicationsAdministrators Developers
Jim
Gavin
Steve
Nigel
Stephen
Ingest
Console
Reporting
Identity & Access Management - IAMAccount
ApplicationsAdministrators Developers
Jim
Gavin
Steve
Nigel
Stephen
Ingest
Console
Reporting
Groups
Multi-factor Authentication
Identity & Access Management - IAMAccount
ApplicationsAdministrators Developers
Jim
Gavin
Steve
Nigel
Stephen
Ingest
Console
Reporting
Groups Roles
Multi-factor Authentication
AWS API Credentials
IAM Policies{ "Statement": [ { "Effect": "Allow", "Action": [ "elasticbeanstalk:*", "ec2:*", "elasticloadbalancing:*", "autoscaling:*", "cloudwatch:*", "s3:*", "sns:*" ], "Resource": "*" } ] }
Create a policy to assign permissions to a user, group, role or resource.
Policies are created using JSON. A policy consists of one or more statements, each of which describes one set of permissions.
Policies control access to AWS APIs
Identity and Access Management - IAM
For more details on IAM, visit:
aws.amazon.com/iam
Think Security3
Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure Regions
Availability ZonesEdge Locations
Client-side Data Encryption & Data Integrity Authentication
Server-side Encryption (File System and/or Data)
Network Traffic Protection(Encryption/Integrity/Identity)
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Customer Data
Amaz
onYo
u
Shared Security Responsibility
Understand your customer & determine your security stance
Leverage AWS Security
External Audience
Regulatory Audience
Internal Audience
Architecture
Administration
IAM
Certifications
White Papers
QSA Process
Your Processes
Your Certifications Penetration Test Results
Understand your customer & determine your security stance
Engage with security assessors early in your adoption cycle
Leverage AWS Security
Don’t fear assessment – AWS meets high standards (PCI DSS, ISO27001)
Security assessments take time, so allow for this in your planning
Undertake architecture reviews early in your design/deployment process
Understand your customer & determine your security stance
Engage with security assessors early in your adoption cycle
Use comprehensive materials and certifications provided by AWS
Leverage AWS Security
For more details on AWS Security, visit: aws.amazon.com/security
Risk and compliance white paper AWS security processes white paper CSA consensus assessments initiative questionnaire
(requires NDA)
Understand your customer & determine your security stance
Engage with security assessors early in your adoption cycle
Use comprehensive materials and certifications provided by AWS
Build upon the security features of AWS to implement ‘security by design’
Leverage AWS Security
Direct Connect & VPNVirtual Private CloudControl & AuditTiered Access
IAM Control users and allow use IAM Roles to provide API credentials for instances to enable access to
AWS resources via APIs
APIs vs Instance Provide developers with API credentials with separately
controlled access to SSH keys/administrative logins
Temporary Credentials Provide temporary API credentials
for access to AWS resources
Instance firewalls Firewall control on instances via
Security Groups
AWS CloudTrail The AWS API call history recorded
by CloudTrail enables security analysis, resource change
tracking, and compliance auditing
AWS Config A fully managed service that provides you with an AWS
resource inventory, configuration history, and configuration change
notifications to enable security and governance
Subnet control Create low level networking
constraints for resource access, such as public and private
subnets, internet gateways and NATs
Bastion hosts Only allow access for
management of production resources from a bastion host. Turn off when not needed and
restrict startup via MFA
VPC Peering Connect privately to other VPCs-
Peer VPCs together to share resources across multiple virtual networks owned by your or other
AWS accounts.
Private connections to VPC Secured access to resources in AWS over software or hardware VPN and dedicated network links
Because your VPC can be hosted behind your corporate firewall, you
can seamlessly move your IT resources into the cloud without changing how your users access
these applications.
Build on AWS Security Features
Build on the Strengths of the AWS Cloud
4
e.g. Application performance improvement by migration of static content to Amazon S3 & CloudFront
Review application architectures early – assess their fit for the cloud
Can cloud benefits be delivered with minimum effort & outlay?
e.g. variable capacity requirements, ‘standard’ technology stacks, reference architectures*
e.g. Faster development cycles for dev/test, reduced cap-ex for application environmentsWill cloud yield top-line growth, cost savings or agility improvements?
e.g. fully scripted deployments, IAM & EC2 instance roles, rolling deploymentsCan automation lead to a more robust, agile & secure services?
Build on the Strengths of the AWS Cloud
1234
Disposable compute
Design systems that can tolerate instance failures
Scala
bility
Avail
abilit
yCo
st O
ptim
isatio
n
Build on the Strengths of the AWS Cloud
✖ ️ ✖ ️
Dispose of compute when it is not required
✖ ️ ✖ ️
Disposable compute
Flexible capacityDesign systems that can dynamically scale from zero to hundreds of instances
Scala
bility
Avail
abilit
yCo
st O
ptim
isatio
n
Build on the Strengths of the AWS Cloud
✖ ️ ✖ ️ ✖ ️
Use Auto-scaling (events, schedules etc) to drive capacity availability
✖ ️ ✖ ️ ✖ ️
Disposable compute
Flexible capacity
Cost effective storageUse Amazon S3 for durable & cost effective storage
Scala
bility
Avail
abilit
yCo
st O
ptim
isatio
n
Build on the Strengths of the AWS Cloud
✖ ️ ✖ ️ ✖ ️
Deploy & scale relational databases with RDS & use DynamoDB for high throughput NoSQL tables
✖ ️ ✖ ️ ✖ ️
Disposable compute
Flexible capacity
Cost effective storage
Automation and control
Automate everything from deployment, to scaling, to instance recovery from failure
Scala
bility
Avail
abilit
yCo
st O
ptim
isatio
n
Build on the Strengths of the AWS Cloud
✖ ️ ✖ ️ ✖ ️
Create instance for your OS choice
Configure environment
Install software
Create AMI from instance
Launch fully configured instances from AMI
AMICustom machine
image
Instances
Auto-scaling Manual deployments
Programmatic deployments
Bootstrapping - Custom AMIs
12345
ami-idami-launch-indexami-manifest-pathblock-device-mappinghostnameinstance-actioninstance-idInstance-typekernel-id
local-hostnamelocal-ipv4macnetworkplacementprofilepublic-hostnamepublic-ipv4public-keysreservation-id
http://169.254.169.254/latest/meta-data
The metadata service contains & provides information about an instance
Metadata Service
Receive custom data to drive
bootstrapping
Custom or standard machine image
Bootstrapping - Metadata Service
AMI
Instances
http://169.254.169.254/latest/meta-data
The metadata service contains & provides information about an instance
Metadata Service
Receive custom data to drive
bootstrapping
Custom or standard machine image
Bootstrapping - Metadata Service
AMI
Instances
+ user dataScripts in user-data field of metadata will be executed on launch For example
#!/bin/sh yum -‐y install httpd chkconfig httpd on /etc/init.d/httpd start
<powershell> … </powershell>
or
http://169.254.169.254/latest/meta-data
The metadata service contains & provides information about an instance
Bootstrapping - Metadata Service
+ user data
Install software e.g. web server, app server, proxy
Pull data and application packages from S3
Publish metadata for instance to other systems e.g. monitoring systems
Setup security profile of instance based upon intended use e.g. pull latest config
1. Use multiple availability zones
2. Use RDS with replicas and slaves
3. Use auto-scaling groups
4. Use Elastic Load Balancing
5. Use Route53 to host DNS zones
Auto-ScalingRDSRoute 53Elastic Load Balancing
Use at regional level Combined with autoscaling will balance requests and resource
capacity across availability zones
Within VPC Use to load balance between
application tiers within an availability zone
Instance migrations Easily move instances from dev
environments to test environments by moving between ELBs
Leverage SLA Improve application reliability with
Route 53’s SLA on requests served
Weighted routing Perform A/B analysis, and staged application roll-outs by moving a
portion of traffic to new infrastructure
Control TTLs and updates Take absolute control of DNS
updates for more decisive system updates
Scale databases without admin overhead
Choose instance size for databases and scale up over time
Add high availability from management console
Create master-slave configurations and read-replicas.
AWS takes care of the failover and recreation of a new slave in event
of master DB loss
Dynamically scale resources & control costs Only provision the resources that
are required with scale up and cool down policies that match
demand
Build on the Strengths of the AWS Cloud
For more details, visit the AWS architecture center: aws.amazon.com/architecture
Services not Software5
AWS CloudInfrastructure & Services
YourBusiness
More Time to Focus onYour Business
Configuring Cloud Services
70%
30%70%
Self Managed Software & Infrastructure
30%
Managing All of the “Undifferentiated Heavy Lifting”
Services Not Software
Relational Database ServiceEasy to set up, operate, and scale Handles time-consuming database management tasks, such as backups, patch management, and replication Supports MySQL, Oracle, Microsoft SQL Server, and PostgreSQL, with Amazon Aurora in preview
NoSQL Database ServiceFast, predictable performance
Supports document & key-value data models Fully distributed, fault tolerant architecture
Amazon RDS
Amazon DynamoDB
Services Not Software
Amazon SQS
Processing task/processing trigger
Processing results
Simple Queue ServiceFast, reliable, scalable, fully managed message queuing service Transmit any volume of data, at any level of throughput
Amazon SQS
Amazon EMR
Elastic MapReduceUses Hadoop, an open source framework, to distribute your data and processing across EC2 instances Integrates with other AWS services, such S3 & DynamoDB Supports the broad Hadoop tools ecosystem
Services Not Software
Optimise Your Costs6
Use the Right Instance Types
Use Auto Scaling
Turn Off Unused Instances
Use Reserved Instances
1234
Use Spot Instances 5
Use Storage Classes6Offload Your Architecture7Use Services, Not Software8Use Consolidated Billing9Use Cost Management Tools10
G2
GPU enabled
M3
General purpose
Memory optimized
R3
CR1M2
Storage and IO optimized
C4
Compute optimized
C1 CC2
I2
HI1
HS1
CG1M1 C3
Use the Right Instance Types
Linux from $0.013/hour Windows from $0.018/hour
Pay as you go for computing capacity
Low cost and flexibility
Pay only for what you use, no up-front commitments or long-term contracts
Ideal for applications being developed or tested on EC2 for the fist time
Use Cases:
Applications with short term, spiky, or unpredictable workloads;
Application development or testing
On-demand Instances
1 or 3 year terms
Three payment options: All Upfront, Partial Upfront & No Upfront
Cost reduced in comparison to the on-demand purchasing option
Predictable pricing, plus reserved capacity helps to ensure that compute capacity is
available when needed
Use Cases:
Applications with steady state or predictable usage
Applications that require reserved capacity, including disaster recovery
Reserved Instances
Bid on unused EC2 capacity
Name your own price for EC2 computing capacity. Instances will run whenever your
bid exceeds to the current Spot Price
Spot Price varies in real-time based on supply/demand, determined automatically
Cost / Large Scale, dynamic workload handling
Use Cases:
Applications with flexible start and end times, or which can be accelerated with
additional computing capacity
Applications only feasible at very low compute prices
Spot Instances
Instance Purchasing Options
For more details, visit EC2 purchasing options: aws.amazon.com/ec2/purchasing-options/
Use Tools & Frameworks
7
Access everything via CLI, API or Console
Use one of 9 (soon to be 10) fully supported SDKs to create or make use of existing AWS resources within your own code
Leverage a broad ecosystem of open source, free and commercially licensed tools to work with AWS Services
Achieve the highest levels of automation to support continuous deployment, define your infrastructure-as-code or automate your development, operations or DevOps processes
Find out more at: aws.amazon.com/developers/getting-started/
Everything is Programmable
AWS Deployment & Management Tools
AWS Elastic Beanstalk
AWS OpsWorks
AWS CloudFormation
AWS CodeDeploy
Get Supported8
Get Supported: AWS Support Options
Four Support Tiers are Available.
Chose from:
Basic Developer Business Enterprise
For more details on AWS Support, visit: aws.amazon.com/premiumsupport
Get Supported: Trusted Advisor
Get Supported: Trusted Advisor
Get Supported: Trusted Advisor
Operating systems on EC2 instances:
Ubuntu Server Red Hat Enterprise Linux and Fedora SUSE Linux (SLES and openSUSE) CentOS Linux Microsoft Windows Server 2003 R2 Microsoft Windows Server 2008 Microsoft Windows Server 2008 R2 Microsoft Windows Server 2012
Infrastructure components:
Sendmail and Postfix MTAs OpenVPN and RRAS SSH, SFTP, and FTP LVM and Software RAID
Web servers:
Apache IIS Nginx
Databases:
MySQL Microsoft SQL Server
Get Supported: 3rd Party Software
For more details on AWS Support, visit: aws.amazon.com/premiumsupport
Resources You Can Use to Learn More
aws.amazon.com/getting-started/
aws.amazon.com/premiumsupport
aws.amazon.com/architecture
aws.amazon.com/security
aws.amazon.com/campaigns/emea-getting-started
Certification
aws.amazon.com/certification
Self-Paced Labs
aws.amazon.com/training/self-paced-labs
Try products, gain new skills, and get hands-on
practice working with AWS technologies
aws.amazon.com/training
Training
Validate your proven skills and expertise with the
AWS platform
Build technical expertise to design and operate
scalable, efficient applications on AWS
AWS Training & Certification
Follow us fo
r more
events
& webina
rs
@AWScloud for Global AWS News & Announcements
@AWS_UKI for local AWS events & news
@IanMmmmIan Massingham — Technical Evangelist