aws december 2015 webinar series - strategies to quantify tco & optimize costs w/ aws
TRANSCRIPT
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
12/8/2015
Strategies to Quantify TCO and Optimize Costs using AWS
Amilcar Alfaro, Product Marketing Manager, AWSGary Mikula, Sr. Director of Info Sec Engineering, FINRA
AWS empowers organizations to rethink how to manage their business infrastructure
With AWS you can…
Scale Based on Market
Needs Increase Innovation and
Experimentation Leverage what you
already own
What we’re focusing on today
Framing the value of AWS
AWS & Lower Costs
Optimize Costs to drive Value
How to frame the value of moving to AWS? Start with your goals
“We need to focus on moving our core business forward — not building and
maintaining infrastructure…”
“Reducing our overall cost is a high priority.”
“We are looking to improve performance and expand globally in a flexible way.”
Macro goals to move
Source: IDC, Quantifying the Business Value of Amazon Web Services (May, 2015)
Analysts have shown AWS Creates Value
Let’s Dive Deep in Costs
Comparing TCO is not easy
≠
Lower costs than on-premises
On-premises traditional data center
On-premises virtualizeddata center
CAPEX
OPEX
OPEX
AWS
CAPEX
OPEX*
Cost savings from running internal IT more efficiently AWS scale
• Multiple new data centers built each year• Volume purchasing, highly automated, supply
chain optimization
Utilization fundamentally higher in AWS cloud• Aggregating non-correlated workloads, scale,
spot market
Amazon specific hardware designs• OEM acquisition of custom servers and
net gear• Direct purchasing of disk, memory, and CPU • AWS controlled hypervisor and net protocol
layers
Diagram is not to scale*For AWS, OPEX costs includes Reserved Instances one-time low, up-front payment, if Reserved Instances are used.
Cost savings from moving to a public cloud provider
Typical cost drivers for on-premises deployments, including Overhead Costs
Networkcosts
Storagecosts
Servercosts
Hardware – server, rack chassis PDUs, ToR
switches(+maintenance)
Software - OS, virtualization licenses
(+maintenance)
Overhead cost
Space Power Cooling
Hardware – storage disks, SAN/FC switches
Overhead cost
Storage admin costs
Network hardware – LAN switches, load balancer
bandwidth costsNetwork admin costs
Overhead cost
IT laborcosts
Server adminVirtualization admin
Space Power Cooling
Space Power Cooling
illustrative
Diagram doesn’t include every cost item. E.g. software costs can include database, management, middle tier software costs. Facilities cost can include costs associated with upgrades, maintenance, building security, taxes etc. IT labor costs can include security admin and application admin costs.
1
2
3
4
Traditional Capacity PlanningLimitations of traditional data centers:
Inflexible physical assets Costs are never in sync
Migration and expansion costs are high
Cost of unexpected inefficiencies
Elastic and Pay-Per-Use Infrastructure
Unable to serve
customers
InfrastructureCost $
time
LargeCapital
Expenditure
OpportunityCost
PredictedDemand
TraditionalHardware
ActualDemand
AutomatedVirtualization
TCO Calculator Demo
Even after initial deployment, AWS continues to drive down
costs
Upgrades are your responsibility
Upgrades happen automatically
On-premises infrastructure
With AWS, Service Upgrades occur automatically
VPC support
NEW
Do-it-yourself MySQL replication
Potentially ~100+ manual steps
Set up primary and standby instancesSet up identical volumesCreate synchronous replicationCreate and manage DNS entriesDetect instance failure conditionsDetect network failure conditionsDetect storage failure conditionsDecide when to fail over….Re-establish primary secondary connections.
Amazon RDS Multi-AZ
HA with a mouse click
Managed services eliminate many operational tasks
Introducing EC2 Dedicated Hosts
Its a physical server with EC2 instance capacity dedicated for your use
Enabling Bring Your Own License (BYOL)Provides necessary visibility into physical resources
• Physical core and socket counts per host• Instance utilization per host• Instance positions per host
Granular placement controls enables you to reuse the same physical Host
Host ID = h-123abcSockets = 2Physical Cores = 20
One Customer saved 42% by using their own licenses
Transparency in Cloud ComputingProviding Real-Time AWS Cost Management
FINRA Information Security Engineering Copyright 2015 FINRA
FINRA – Who We Are
• FINRA—the Financial Industry Regulatory Authority—is an independent, nongovernmental regulator for all securities firms doing business with the public in the United States
• FINRA protects investors by regulating brokers and brokerage firms and by monitoring trading on US stock markets
• FINRA monitor over 6 billion shares traded on the stock market each day
• FINRA handles more “big data” on a daily basis than the Library of Congress or Visa®—to build a holistic picture of the trading market
• FINRA: Deter, Detect, Discipline
Investor Protection
FINRA Information Security Engineering Copyright 2015 FINRA
Challenges and Keys to Success
• Inefficient Quarterly Financial Reviews• Spending is real-time• Repeat process again in 3 months
• Project Managers Lack Visibility• What am I spending?• Am I below budget?• Am I being held accountable?
• Billing Reports Alone Are Insufficient• Over 10 million line items• No correlation
Cost management must be continuous,
not quarterly
Visibility must be extended to individual
project managers
Cost must be granularly correlated to other data sources
FINRA Information Security Engineering Copyright 2015 FINRA
Approach Chosen• Cloud Cost Management is:
• A Discipline not a right• Best addressed distributively
• Use Splunk Cloud as Process / Delivery System• Ability to Collect, Analyze, Visualize
• Collect AWS Billing Data in Splunk• Detailed line items with resources and tags
• Data Enrichment• Project code lookups• Forecast projections• Billing adjustments
• Build Interfaces
FINRA Information Security Engineering Copyright 2015 FINRA
How We Did It
FINRA Information Security Engineering Copyright 2015 FINRA
FINRA AWS Project Management Billing App
FINRA Information Security Engineering Copyright 2015 FINRA
Immediate Impact
• Focus on Low Hanging Fruit• Shutting down services over weekends / evenings• Storage sun setting / dormant EC2• Identify AWSs services with highest spending• Identify projects over budget
• Immediate Results• Quantifiable cost savings• Better tracking to forecast
• Better Forecast Projections• Feedback and control
FINRA Information Security Engineering Copyright 2015 FINRA
Gaining Transparency Into Amazon EMR
What Stood in Our Way• PaaS and IaaS Are Not Equal• Instance Fingerprinting
• Identify Nodes• Instance Role• User Tags
• Data Retention• Collection Delay• Bootstrap Splunk Agent
FINRA Information Security Engineering Copyright 2015 FINRA
EMR Cluster Analyzer –> Summary Dashboard
AWS Billing
Splunk for *NIX
Select the Cluster
Jobs Running
FINRA Information Security Engineering Copyright 2015 FINRA
Potential EC2 Provisioning Cost Savingsm1.xlarge
• 4 vCPUs with 8 ECUs• 15 GB memory• 4 x 420 GB disk• $0.35 per hour
c3.xlarge• 4 vCPUs with 14 ECUs• 7.5 GB memory• 2 x 40 GB disk (SSD)• $0.27 per hour
Potential 23% Savings in EMR
Costs
With additional ECUs and SSD disks, the c3.xlarge may be more performant than the m1.xlarge instances at a better price pointRESIZING***
FINRA Information Security Engineering Copyright 2015 FINRA
Resizing Analysis
Summary providesthe information on number of nodes
1. Less than 80% utilization overall
2. After 70 mins utilization at less than 50% (cpu)
Conclusion:The cluster be resized after the two hours
FINRA Information Security Engineering Copyright 2015 FINRA
Resizing Analysis – Scenario 1
1st Hour 2nd Hour 3rd Hour 4th Hour40 40 40 40
Cost Analysis:
Cost = (Price per Instance) * (No. of Instances) * (No. of Hours)
Price = 0.35 * 40 * 4 = $56
All instances running for full duration of the job
FINRA Information Security Engineering Copyright 2015 FINRA
Resizing Analysis – Scenario 2
1st Hour 2nd Hour 3rd Hour 4th Hour40 35 20 20
Cost Analysis:
Cost = (Price per Instance) * (No. of Instances) * (No. of Hours)
Price = (0.35*40*1) + (0.35*35*1)+(0.35*20*2) = 14 + 12.25 + 14 = $40.25
Savings Compared to Scenario 1 = 28.2%
Resizing after the 1st hour and 2nd hour
FINRA Information Security Engineering Copyright 2015 FINRA
Combined EC2 Provisioning and Resizing Analysis
• Combined Savings• Original Cost = $56• Price After Combined Analysis = $31.05• Job Savings = $24.95 = 55.44%
• Job Runs 5x / Day ( $24.95 * 5 = $124.75 )• Every Business Day / Week ( $124.75 * 5 = $623.75 )• Every Week of the Year ( $623.75 * 52 = $32,435 )• And….We Haven’t Affected Performance
• Just More Efficient Provisioning
FINRA Information Security Engineering Copyright 2015 FINRA
Future Directions• File Processing
• Back Tagging• More Drill Down Analytics• Duplicate EMR Success• EMR Cluster Grading • User Defined Alerting• Splunk IT Service Intelligence Integration• https://www.youtube.com/watch?v=eaNtpgjyUdY
FINRA Information Security Engineering Copyright 2015 FINRA
With AWS you can more easily...
Move Quickly
Hit Your Deadlines
More Time to Focus on What Differentiates Your
Business
Faster Turns
Fewer Long and Large Projects
Long & Large Projects
More likely to exceed budget
More likely to run late
More likely to fail to deliver full value
Long & Large Projects
Each additional year in a project boosts the likelihood of missing targets by 15%
Long & Large Projects
$66 Billion in Cost Overruns in 5,400 projects
How much can you save?
Experiment and Lower Risks
Putting It All Together
The Value of Using AWS Accelerates Over Time
Source: IDC Business Value of AWS Accelerates over time
According to IDC, this relationship between length of time using AWS and return is due to customers leveraging the more optimized environment to generate more applications along a learning curve.
$1 Investment in AWS
$8.40 in benefits
At 60 Months of using AWS
~8X$3.50 in benefits
$1 Investment in AWS
At 36 Months of using AWS
~3X
What to do Next ?
Cloud Centerof Excellence
Leverage AWS Global Footprint
AWS’s 11 Regions
30 Availability Zones
50+ Services
And hundreds of innovations can help you
So this will result in
Lower Costs
Increased Agility
The ability to scale based on market needs
Innovate around YOUR unique value
Leverage what you already own
If you need extra help…
AWS has tens of thousands of APN partners and Marketplace has 2,300 software listings in over 30 categories
Web links for more information
AWS Economics Center: • aws.amazon.com/economics• Includes link to TCO Calculator
Case Studies & References:• aws.amazon.com/solutions/case-studies