Download - AWS Billing Deep Dive (DMG203) | AWS re:Invent 2013

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

DMG203 - AWS Billing Deep Dive Billing Console and Detailed Billing Reports

Serge Hairanian, Vadim Jelezniakov, AWS Billing

November 13, 2013

Agenda Billing Console Billing Reports

Cost Allocation Report

Detailed Billing Report w/

Resources & Tags

Detailed Billing Report

Monthly Report 5 levels of visibility

Agenda

Q&A at the end

The Billing Console

The Billing Console • Part of the AWS

Management Console – Accessibility – Visibility

• More responsive • New features • New look

Billing Console: Dashboard

Current Month’s Spend • Previous month comparison • Include/exclude subscriptions

- Reserved Instances - Premium Support charges

Visualized breakdown • Over 30 AWS services • See your top 4 services

by spend

Notification Center • Status of Billing Alerts

• Alarm on spending threshold

• Issues with payment methods • Outstanding charges • Free Tier eligibility • General notices regarding

your account or bill

Fully localized in 8 languages • English • German • Spanish • French • Japanese • Portuguese • Korean • Chinese

Mobile Support • Dashboard looks nice on mobile devices

• Quick view on how you are trending

Drilling down: Bills

Bills • View current & past bills • Download monthly report (csv) • Download invoices (pdf) • Product breakdown

Linked Account Activity • Totals by linked account • Drill down into each account

Consolidated Billing • Receive a single bill for all

charges incurred across all linked accounts

• Share RI discounts • Combine tiering benefits

• View & manage linked accounts

• Add additional accounts

Payer Bill Accounts 1-4

Account 1 Regular Bill




Share RI Discounts Combine Tiering

Payment

Payment History • List successful transactions • Charges and refunds match

your credit card statement • Central place to download

all invoices (pdf)

Payments Due • Amount Past Due

• Charges past net terms • Amount Due

• Charges within net terms • Outstanding Credit

• Invoice customers • Credits need to be applied to

future charges • Pay Now

• Available for failed credit card charges

Checkout Experience • Ability to add/edit cards during checkout

Billing Preferences

Billing Preferences • Invoices via email • Billing Alerts • Billing Reports • Cost Allocation Report Tags

Billing Reports • Delivered to your S3 bucket • Bucket requires specific

permissions - Sample policy

Bucket Permissions • Bucket policy generator • Allows our reporting engine to

- Confirm your bucket’s permissions - And also write to that bucket

• We won’t write to overly permissive buckets

• It is recommend to use the exact policy generated

Billing Reports • Invalid bucket name or

bucket policy

Billing Reports • Valid bucket • Enabled specific reports

Report Types • Monthly report

- Grouped by product - Monthly grain

• Detailed billing report - Grouped by product - Hourly grain

• Cost allocation report - Grouped by tag key - Monthly grain

• DBR with resources and tags - Grouped by resourceId - Hourly grain - Also has tags on each row

Manage Report Tags • Filter tags by included/excluded • Search tags by key name • Active tags

- Cost allocation report grouping - Included in your detailed billing

report with resources and tags

Finding your reports • Located in your Amazon S3 bucket

– Monthly report • <AWS account number>-aws-billing-csv-yyyy-mm.csv

– Detailed billing report • <AWS account number>-aws-billing-detailed-line-items-yyyy-mm.csv.zip

– Cost allocation report • <AWS account number>-aws-cost-allocation-yyyy-mm.csv

– DBR with resources and tags • <AWS account number>-aws-billing-detailed-line-items-with-resources-and-tags-yyyy-mm.csv.zip

Detailed Billing Reports (DBRs)

Goals

1. Eliminate ambiguity about DBRs – What are the four billing reports? – What’s inside of each report?

2. Enable you to answer cost questions – Using Monthly, Detailed, Cost Allocation and Detailed w/

Resources and Tags Reports

The Four Billing Reports


Detailed Billing Report w/ Resources

and Tags


Monthly Report

Hou

rly

Mon

thly

Advanced Data Basic Data

Joe: from developer to software tycoon

Joe: individual developer

• What is my total monthly spend?

• Which services am I spending my money on? Do I need to?

Monthly Report

Hou

rly

Advanced Data

Monthly Report

Mon

thly

Basic Data

Monthly Report

Monthly Report

Mon

thly

Basic Data • CSV format • Monthly granularity • Aggregated by product • Same data as your Invoice

<AWS account #>-aws-billing-csv-yyyy-mm.csv

Example Monthly Report RateId ItemDescription ProductCode UsageType Operation Usage Quantity Rate CostBeforeTax

805167 $0.01 per 10,000 GET and all other requests AmazonS3 Requests-Tier2 47 $0.01 $0.47800029 $0.000 per GB - data transfer in per month AWSDataTransfer DataTransfer-In-Bytes 0.00471905 $0.00 $0.00796850 $0.06 per 100,000 Amazon SNS API Requests per month thereafter AmazonSNS Requests-Tier1 6 $0.06 $0.36799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer DataTransfer-Out-Bytes 0.00053194 $0.09 $0.00794470 $0.00 per request - first 1,000,000 requests AmazonEC2 CW:Requests 9878 $0.00 $0.00805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 TimedStorage-ByteHrs 0.00003376 $0.11 $0.00805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 Requests-Tier1 40 $0.01 $0.40796889 First 100,000 Amazon SNS API Requests per month are free AmazonSNS USW2-Requests-Tier1 2 $0.00 $0.00795828 $0.10 per GB-month of provisioned storage AmazonEC2 EBS:VolumeUsage 67.86110956 $0.10 $6.79

18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 BoxUsage:c1.medium RunInstances 359 $0.06 $21.54799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer DataTransfer-Out-Bytes 0.47127815 $0.09 $0.04

18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 BoxUsage RunInstances 1436 $0.03 $43.08793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 EBS:SnapshotUsage 0.54326496 $0.13 $0.07800029 $0.000 per GB - data transfer in per month AWSDataTransfer DataTransfer-In-Bytes 0.16240232 $0.00 $0.00805167 $0.01 per 10,000 GET and all other requests AmazonS3 Requests-Tier2 561311 $0.01 $0.56794470 $0.00 per request - first 1,000,000 requests AmazonEC2 CW:Requests 660 $0.00 $0.00805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 TimedStorage-ByteHrs 1.7335875 $0.11 $0.19805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 Requests-Tier1 193442 $0.01 $1.93

18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 BoxUsage:c1.medium RunInstances 1436 $0.06 $86.16794470 $0.00 per request - first 1,000,000 requests AmazonEC2 CW:Requests 932 $0.00 $0.00809956 $0.015 per Small EMR Instance (m1.small) Instance-hour ElasticMapReduce BoxUsage 1432 $0.02 $21.48

18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 BoxUsage RunInstances 1436 $0.03 $43.08809792 $0.12 per Extra Large EMR Instance (m1.xlarge) instance-hour ElasticMapReduce BoxUsage:m1.xlarge 404 $0.12 $48.48795287 $0.025 per LoadBalancer-hour (or partial hour) AmazonEC2 LoadBalancerUsage LoadBalancing 359 $0.03 $8.98793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 EBS:SnapshotUsage 82.45148478 $0.13 $10.31794668 $0.520 per M1 Standard Extra Large (m1.xlarge) Linux/UNIX instance-hour (or part AmazonEC2 BoxUsage:m1.xlarge RunInstances 470 $0.52 $244.40805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 Requests-Tier1 1693140 $0.01 $16.93809803 $0.03 per High-CPU Medium EMR Instance (c1.medium) instance-hour ElasticMapReduce BoxUsage:c1.medium 1436 $0.03 $43.08794128 $0.008 per GB Data Processed by the LoadBalancer AmazonEC2 DataProcessing-Bytes LoadBalancing 0.00005498 $0.01 $0.00799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer DataTransfer-Out-Bytes 324.4072553 $0.09 $29.20

Usa

ge Q

uant

ity

Rat

e

Cos

t

Line Item Pr

oduc

t

Usa

ge T

ype

Ope

ratio

n

Example Monthly Report: Line Item

Line Item

Example Monthly Report: Line Item

Monthly Report: Demo

Monthly Report

Mon

thly

Basic Data

• Enabling Billing Reports • Analyzing Monthly Cost

Monthly Report: Considerations

Monthly Report

Mon

thly

Basic Data • Changing Rates result in

more Line Items, e.g. – AWS lowers cost – You purchase Reserved Capacity

• This is true for all reports! – I.e. two rates during the month

will result in 2x line items

Joe: Start-Up

Joe: Start-Up

• How much am I spending hour-by-hour?

• Are my developers turning things off?


Detailed Billing Report H

ourly

M

onth

ly

Basic Data


• CSV format • Hourly granularity • Aggregated by product • x720 or x744 more records

than Monthly Report

<AWS account #>-aws-billing-detailed-line-items-yyyy-mm.csv.zip

Detailed Billing Report H

ourly

Basic Data

Usa

ge Q

uant

ity

Blen

ded

Rat

e

Blen

ded

Cos

t

Line Item

Prod

uct

Usa

ge T

ype

Ope

ratio

n

Example Detailed Billing Report

Usa

ge Q

uant

ity

Blen

ded

Rat

e

Blen

ded

Cos

t

Line Item

Prod

uct

Usa

ge T

ype

Ope

ratio

n

Same as Monthly Report

Usa

ge Q

uant

ity

Blen

ded

Rat

e

Blen

ded

Cos

t

Line Item

Prod

uct

Usa

ge T

ype

Ope

ratio

n

Same as Monthly Report + UnBlended Rate and Cost + Hour-by-Hour

+ R

eser

ved

Inst

ance

Fla

g

+ Av

aila

bility

Zon

e

Detailed Billing Report: Demo

Analyzing Cost Hour-by-Hour Detailed Billing Report Hou

rly

Basic Data

Detailed Billing Report: Considerations

• Lots of data – use PowerPivot or import into your Data Warehouse

• Sum of allocated cost is not always equal to total on Invoice. This is due to rounding differences b/w blended and unblended cost

Detailed Billing Report Hou

rly

Basic Data

Joe: Mid-Size Company

Joe: Mid-Size Company • What is my cost by

department? • How do I do charge-backs?

• Use optional features:

– Consolidated Billing – Tagging

Company Account (“Payer” Account)

Financial Systems

Ordering Systems

Fulfillment Systems Website

Catalog Workflow Order Pipeline

Application (Tag) =

Cost Center (Tag) =

Department Accounts (“Linked” Accounts)

General Ledger Inventory 3 way

match Proxy fleet

2222 3333 6600

Stack (Tag) = Prod Alpha Beta Gamma

TAGS

1111 7784

Overview: Consolidated Billing & Tagging

Demo: Setting Up Consolidated Billing Demo: Tagging Your Resources

Optional Features: Consolidated Billing & Tagging

Now Back to Joe

• What is my cost by department?

• How do I do charge-backs?


Hou

rly

Mon

thly





Mon

thly

Advanced Data • CSV format • Monthly granularity • Aggregated by product first • Then aggregated by tags

<AWS account #>-aws-cost-allocation-yyyy-mm.csv

Remember? Monthly Report: RateId ItemDescription ProductCode UsageType Operation Usage Quantity Rate CostBeforeTax

805167 $0.01 per 10,000 GET and all other requests AmazonS3 Requests-Tier2 47 $0.01 $0.47800029 $0.000 per GB - data transfer in per month AWSDataTransfer DataTransfer-In-Bytes 0.00471905 $0.00 $0.00796850 $0.06 per 100,000 Amazon SNS API Requests per month thereafter AmazonSNS Requests-Tier1 6 $0.06 $0.36799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer DataTransfer-Out-Bytes 0.00053194 $0.09 $0.00794470 $0.00 per request - first 1,000,000 requests AmazonEC2 CW:Requests 9878 $0.00 $0.00805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 TimedStorage-ByteHrs 0.00003376 $0.11 $0.00805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 Requests-Tier1 40 $0.01 $0.40796889 First 100,000 Amazon SNS API Requests per month are free AmazonSNS USW2-Requests-Tier1 2 $0.00 $0.00795828 $0.10 per GB-month of provisioned storage AmazonEC2 EBS:VolumeUsage 67.86110956 $0.10 $6.79

18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 BoxUsage:c1.medium RunInstances 359 $0.06 $21.54799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer DataTransfer-Out-Bytes 0.47127815 $0.09 $0.04

18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 BoxUsage RunInstances 1436 $0.03 $43.08793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 EBS:SnapshotUsage 0.54326496 $0.13 $0.07800029 $0.000 per GB - data transfer in per month AWSDataTransfer DataTransfer-In-Bytes 0.16240232 $0.00 $0.00805167 $0.01 per 10,000 GET and all other requests AmazonS3 Requests-Tier2 561311 $0.01 $0.56794470 $0.00 per request - first 1,000,000 requests AmazonEC2 CW:Requests 660 $0.00 $0.00805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 TimedStorage-ByteHrs 1.7335875 $0.11 $0.19805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 Requests-Tier1 193442 $0.01 $1.93

18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 BoxUsage:c1.medium RunInstances 1436 $0.06 $86.16794470 $0.00 per request - first 1,000,000 requests AmazonEC2 CW:Requests 932 $0.00 $0.00809956 $0.015 per Small EMR Instance (m1.small) Instance-hour ElasticMapReduce BoxUsage 1432 $0.02 $21.48

18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 BoxUsage RunInstances 1436 $0.03 $43.08809792 $0.12 per Extra Large EMR Instance (m1.xlarge) instance-hour ElasticMapReduce BoxUsage:m1.xlarge 404 $0.12 $48.48795287 $0.025 per LoadBalancer-hour (or partial hour) AmazonEC2 LoadBalancerUsage LoadBalancing 359 $0.03 $8.98793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 EBS:SnapshotUsage 82.45148478 $0.13 $10.31794668 $0.520 per M1 Standard Extra Large (m1.xlarge) Linux/UNIX instance-hour (or part AmazonEC2 BoxUsage:m1.xlarge RunInstances 470 $0.52 $244.40805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 Requests-Tier1 1693140 $0.01 $16.93809803 $0.03 per High-CPU Medium EMR Instance (c1.medium) instance-hour ElasticMapReduce BoxUsage:c1.medium 1436 $0.03 $43.08794128 $0.008 per GB Data Processed by the LoadBalancer AmazonEC2 DataProcessing-Bytes LoadBalancing 0.00005498 $0.01 $0.00799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer DataTransfer-Out-Bytes 324.4072553 $0.09 $29.20

Usa

ge Q

uant

ity

Rat

e

Cos

t

Line Item Pr

oduc

t

Usa

ge T

ype

Ope

ratio

n

Usa

ge Q

uant

ity

Rat

e

Cos

t

Line Item

Prod

uct

Usa

ge T

ype

Ope

ratio

n

Example Cost Allocation Report

Usa

ge Q

uant

ity

Rat

e

Cos

t

Line Item

Prod

uct

Usa

ge T

ype

Ope

ratio

n

+Your Tags Same as Monthly Report

+ Av

aila

bility

Zon

e

Cost Allocation Report: Demo

Calculating Charge-Backs Cost Allocation Report

Mon

thly

Advanced Data

Charge-Backs: End Result

Cost Center AWS Product Code Cost Before Tax Tax Total Cost

Charge-Backs: Start with Cost Allocation Report

Charge-Backs: Insert Pivot Table

Defaults are usually OK

Charge-Backs: Choose Fields Rows Labels: user:Cost Center (tag) ProductCode ItemDescription Values: Sum of CostBeforeTax Sum of TaxAmount Sum of TotalCost

Charge-Backs: Finishing Touches Rename columns Filter out blank Cost Centers (Unallocated Cost)

Charge-Backs: End Result

Joe: Enterprise

Joe: Enterprise

• How much capacity do I need to reserve?

• I need to know how much each resource cost me!

DBR w/ Resources and Tags


and Tags Hou

rly

Mon

thly


DBR w/ Resources and Tags


and Tags Hou

rly

Advanced Data • CSV format • By resource • Hourly granularity • Has tags • Most detailed – lots of data!

<AWS account #>-aws-billing-detailed-line-items-with-resources-and-tags-yyyy-mm.csv.zip

Usa

ge Q

uant

ity

Blen

ded

Rat

e

Blen

ded

Cos

t

Prod

uct

Usa

ge T

ype

Ope

ratio

n

+ R

eser

ved

Inst

ance

Fla

g

+ Av

aila

bility

Zon

e

UnB

lend

ed R

ate

UnB

lend

ed C

ost

Remember? Detailed Billing Report:

Hour-by-Hour

Usa

ge Q

uant

ity

Blen

ded

Rat

e

Blen

ded

Cos

t

Prod

uct

Usa

ge T

ype

Ope

ratio

n

+ R

eser

ved

Inst

ance

Fla

g

+ Av

aila

bility

Zon

e

UnB

lend

ed R

ate

UnB

lend

ed C

ost Hour-by-Hour

Same as Detailed Billing Report + Your Tags

+ AW

S R

esou

rce

ID

DBR w/ Resources & Tags: Considerations


and Tags Hou

rly

Advanced Data

• Lots of data – pull into your

Data Warehouse to do detailed analysis

• Any change in Rates (e.g. linking/unlinking, RI subscriptions) produces even more data

Recap: Billing Reports Two Basic Reports Monthly Report

– Monthly granularity – Product level aggregation

Detailed Billing Report – Hourly granularity – Product level aggregation – x720 or x744 more lines

than Monthly Report!

Two Advanced Reports Cost Allocation Report

– Monthly granularity – Product, tag key aggregation

Detailed w/ Resources & Tags – Hourly granularity – Grouped by resource – Has tags – Lots and lots of data!

Resources “Decoding Your AWS Bill”, re:Invent 2012 http://bit.ly/aws-reinvent2012-billing

AWS Billing FAQ http://aws.amazon.com/billing/faqs/

AWS Billing Documentation http://bit.ly/aws-billing-docs

Please give us your feedback on this presentation

As a thank you, we will select prize winners daily for completed surveys!

DMG203

Appendix A How Billing Reports are generated

Billing reports: how are they generated?


Aggregate Metering Records

Calculate Line Items

Compose Reports

What is Usage Record?

• Product • Usage Type • Operation (opt) • Quantity

EC2, S3, … Micro Instance, TimedStorage, … RunInstances, DataTransfer, … 1, 0.007, …




Compose Reports

• Defines how much a unit of usage costs – For each AWS product (more than 30), and – For each unique usage type and operation, and – For each AWS region (9 of them)

• Examples – $0.004 per 10,000 requests (S3) in us-east-1 – $0.020 per Micro Instance instance-hour (EC2) in us-

west-2

What’s Rate?




Compose Reports

• You define key/value pairs (up to 10 pairs per resource)

• Enable them under Manage Tags in Console!

• Tags become columns in Cost Allocation and DBR w/ Resources and Tags Reports

What are Tags?




Compose Reports

Line Items: Two Steps to Calculate • Step 1: Count Usage

– For each unique rate (Product, Usage, Operation, Region)

• Step 2: Calculate Lowest Cost Bill – Apply Free Tier Rates – Apply Reserved Capacity Rates – Apply On-Demand Rates

Appendix B Blended vs Unblended Cost

Linked Account 1 purchased 2 RI’s with $0.00 usage rate Linked Accounts 2 and 3 run On Demand usage

Linked Account

Product Name Usage Type Operation Item Description Usage

Quantity Blended

Rate Blended

Cost

Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.156 per m1.large 2396 0.156 373.77

Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0 0

Linked Account Line Item

LA 1 AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0.096233 143.194704





Blended rate

Linked Account

Product Name Usage Type Operation Item Description Usage

Quantity UnBlended

Rate UnBlended

Cost

Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.156 per m1.large 2396 0.156 373.77

Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0 0


LA 1 AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0 0





Fixed Reserved Instance rate

On Demand rate


Linked Account Product Name Usage Type Usage

Quantity Blended

Rate Blended Cost UnBlended Rate

UnBlended Cost

LA 1 AmazonEC2 BoxUsage:m1.large 1488 0.096233 143.194704 0 0

LA 2 AmazonEC2 BoxUsage:m1.large 1645 0.096233 158.303285 0.156 256.62

LA 3 AmazonEC2 BoxUsage:m1.large 751 0.096233 72.270983 0.156 117.15


Download - AWS Billing Deep Dive (DMG203) | AWS re:Invent 2013

Top Related