aws atlanta march_2015
TRANSCRIPT
1
Tracking, Allocating, and
Optimizing AWS Costs
http://meetup.com/AWS-Atlanta/
AWS Atlanta Meetup group
Detailed Billing Report
• CSV file written into an S3 bucket
• Updated about every 4-6 hours
• Open in Microsoft Excel
• Until it gets to 1GB in size
• Easily ends up being millions of rows
• Hundreds of millions of row for large accounts
• Provides very granular details
• Including tags
http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/d
etailed-billing-reports.html
Setting up a Tagging Strategy
• Tagging resources
• Allow you to group resources to allocate costs more granularly
• Challenges with tagging
• Tags are case-sensitive and free-form text
• Not all resources can be tagged
• Hard to enforce tagging
• Resources are tagged at the account level
• Costs from tags flow up to the master account in DBR
http://docs.aws.amazon.com/AWSEC2/
latest/UserGuide/Using_Tags.html
Creating a Tagging Strategy
• Define the rules for tagging resources
• Example: All EC2 instances must be tagged with department
• Example: The Department tag must be a valid department
• Example: All resources must be tagged with Environment of
Prod, QA, Staging, or Development
• Enforce the tagging rules
• Police untagged resource
• Cross-reference untagged resource to IAM user
• Based on mining CloudTrail
New AWS Management Console features
• New feature: Resource Groups• https://resources.console.aws.amazon.com/r/group
• New feature: Tag Editor• https://resources.console.aws.amazon.com/r/tags
Tagging in the Detailed Billing Report
• What’s tagged in an account flows into the DBR• Need to configure what Tag Keys flow into the DBR from the master
payer
• Need to tag the resources in the payee account
• Coordinate both can be complex if different people are managing the 2
accounts
• Configuring Tag Keys for the DBR• Can designate up to 10 Tag Keys to flow thru
• This is for all payees across the consolidated bill
• Unlimited number of Tag Values can flow through
• For example, Stack=Test or Stack=Production, Application=SW1 or
Application=SW2
• Each Tag Key you designate becomes a header in the DBR
Finding Under-utilized Resources
• CloudWatch gives data out-of-the-box• For EC2 instances, you have CPU, disk, and network statistics
• Use Custom Metrics to get Memory utilization
• Define your bar for idle resources• If resource has lowest level of all metric, strong likelihood it is idle
• Always important to investigate before terminating a resource
• Cross-reference the owner of the instance in CloudTrail
• Understanding when resources can be down-
sized/right-sized• Factor in all 4 metrics, 1 might be low but others appropriate
• Other options: Resizing to a difference instance family
Scripts for Custom Memory Metrics
• Monitoring Scripts for Amazon EC2 Instances
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-
scripts.html
• "reports memory, swap, and disk space utilization metrics for a
Linux instance. The scripts for Windows are sample PowerShell
scripts that comprise a fully functional example that reports
memory, page file, and disk space utilization metrics for a Windows
instance“
• Tie those custom metrics into CloudWatch to right size your instances.
Migrating old gen. instance types
• Migrating to New Generation instance types
• If you are running m1, m2, t1
• Always better to use new generation of instance types
• Significant savings
• RDS as well
What are Reserved Instances?
Get started at http://aws.amazon.com/ec2/purchasing-options/reserved-instances/
• Upfront payment and Commitment to pay less overall
– Significant savings (up to 75%)
– Requires planning, predicting, accepting risk
– What if AWS drops prices?
• Factors for picking RIs
• Term – how long you want to commit for
• Region/Availability Zone – where will the resource run
• Operating System – what resource are you going to run
• Payment Options – how do you want to pay
We will try to understand these factors
Reserved Usage is more than EC2!
Don’t ignore other Reserved Usage opportunities
• RDS reserved usage very close to EC2• They continue to support Light, Medium, Heavy Utilization
• Databases are rarely used in Light or Medium
• ElastiCache reserved cache nodes• Current generation supports Heavy only
• Previous generation support Light, Medium, Heavy
• DynamoDB Reserved Capacity• Commiting to read and write units capacity
• Redshift Reserved Instances• 1 and 3 Year Heavy Utilization options, large upfront
• CloudFront reserved bandwidth• Very attractive discounts if you commit to minimum bandwidth usage
Big announcement in EC2 RIs
http://aws.amazon.com/blogs/aws/simplified-reserved-instances/
Simplifying the EC2
Reserved Instance ModelFrom Jeff Barr, 02 Dec 2014
“After combining customer feedback with an analysis of purchasing patterns that goes back to when we first launched Reserved Instances in 2009, we have decided to simplify the model and are introducing
an important set of changes today.”
EC2 Reserved Instance types (Next Gen.)
• All Upfront• Same as Fixed Priced
• Full payment upfront, no hourly charges
• Partial Upfront• Similar to Heavy Utilization RIs
• Pay something upfront plus a monthly usage charge whether you use or not
• No Upfront• New idea, don’t pay anything upfront but commit to
hourly usage price for the full term
• Not available under 3-year terms
How does Old compare to New?
Comparing Heavy Utilization to Partial Upfront
As of 10/13/2014
• EC2 - US East (Northern Virginia) – Heavy Utilization 3-Year c1.medium SUSE Linux Enterprise)
• $983 upfront, $0.039 hourly
• EC2 - US West (Oregon) – Heavy Utilization 1-Year r3.xlarge Linux/UNIX
• $1,082 upfront, $0.066 hourly
As of 12/2/2014
• EC2 - US East (Northern Virginia) - Partial Upfront 3-Year c1.medium SUSE Linux Enterprise
• $983 upfront, $0.039 hourly
• EC2 - US West (Oregon) - Partial Upfront 1-Year r3.xlarge Linux/UNIX
• $1,082 upfront, $0.066 hourly
Partial Upfront = Heavy Utilization
Looking at new RI types
• Linux, c3.2xlarge, in the Sydney Region
Recommendation Term Upfront MonthlyTotal Usage
Cost for TermTotal Cost for Term
Effective Hourly Rate
Total Savings
TotalCommit
Partial Upfront 1 year 1,426.00 127.44 1,529.28 2,955.28 0.342 1,615.28 2,955.28
No Upfront 1 year 0.00 282.24 3,386.88 3,386.88 0.392 1,183.68 3,386.88
Full Upfront 1 year 2,916.00 0.00 0.00 2,916.00 0.338 1,654.56 2,916.00
OnDemand 1 year 0.00 380.88 4,570.56 4,570.56 0.529 N/A 0
Questions?
Questions on:
• AWS Costs
• CloudCheckr
Thank You for Attending
Sign up today for free evaluationat http://cloudcheckr.com
Aaron Newman is the Founder of CloudCheckr (www.cloudcheckr.com)
Please contact me with additional questions at:[email protected]