high performance mongodb on storage-optimized aws ec2
TRANSCRIPT
Partner Technical Solutions, MongoDB
Sandeep Parikh
#MongoDBWorld
High Performance MongoDB on Amazon EC2
Solutions Architect, Amazon Web Services
Kyle Lichtenberg
Wait, isn’t there a whitepaper on this?
• Much has changed since March 2013!– (and an update is in the works!)
New EC2 Instances Types
• I2 released in December 2013– Next generation High I/O Instance Family– 4-32 vCPUs – 30-244 GiB of Memory– 800-6,400 GB of SSD Storage (up to 315,000+
IOPS)
• R3 released in April 2014– Next Generation Memory Optimized Instance
Family– 2-32 vCPUs – 15-244 GiB of Memory
Latest Generation Instance Features
• Enhanced Networking (SR-IOV)– Improved packet per second (PPS) performance– Lower inter-instance latencies– Very low network jitter
• SSDs– No mechanical seeking!– Higher I/O than magnetic
• Updated Hardware Architecture – Ivy Bridge Intel Xeon E5-2670 v2– Higher sustained memory bandwidth (up to
63,000 MB/s)
Storage I/O Optimized Instance Evolution
hi1.4xlarge
Memory
60.5 GiB
Storage
2x 1,024GB SSD
Network
10 Gigabit
i2.8xlargeMemory 244 GiB
Storage 8x 800GB SSD
Network 10 Gigabit
i2.4xlargeMemory 122 GiB
Storage 4x 800GB SSD
Network High
i2.2xlargeMemory 61 GiB
Storage 2x 800GB SSD
Network High
i2.xlargeMemory 30.5 GiB
Storage 800GB SSD
Network Moderate
Memory Optimized Instance Evolution
cr1.8xlargeMemory 244 GiB
Storage 2x 120GB SSD
Network 10 Gigabit
m2.4xlargeMemory 68.4 GiB
Storage 2x 840GB
Network High
m2.2xlargeMemory 34.2 GiB
Storage 850GB
Network High
m2.xlargeMemory 17.1 GiB
Storage 420GB
Network Moderate
r3.8xlargeMemory 244 GiB
Storage 2x 320GB SSD
Network 10 Gigabit
r3.4xlarge
Memory 122 GiB
Storage 320GB SSD
Network High
r3.2xlarge
Memory 61 GiB
Storage 160GB SSD
Network High
r3.xlarge
Memory 30.5 GiB
Storage 80GB SSD
Network Moderate
r3.large
Memory 15.25 GiB
Storage 32GB SSD
Network Moderate
AWS Block Storage Review
• Amazon Elastic Block Store (“EBS”)– Network attached– Durable– Snapshot backups to S3
• Amazon EBS Provisioned IOPS (“PIOPS”)– Standard EBS + definable, consistent
performance
• Instance Storage (“Ephemeral Storage”)– Lowest latency– Included in instance cost– Volatile!
Storage ConsiderationsEBS EBS PIOPS Instance Store
~100 IOPS Magnetic
3 IOPS/GB General Purpose (SSD)
Up to 1TB/Volume
Linear performance increases up to 12
volumes on 8xlarge instances
Max 36,000 IOPS/Instance
Snapshots!
Up to 30 IOPS/GB(max
4,000/volume)
Up to 1TB/Volume
Linear performance increases up to 12
volumes on 8xlarge instances
Max 48,000 IOPS/Instance
Snapshots!
Up to 365,000 IOPS/instance
Up to 6.4TB
Fewer IOPS on smaller instance
sizes
Max 365,000 IOPS/Instance
No built-in backup functionality
Storage Considerations
• When using EBS– Use EBS Optimized Instances!– Use PIOPS for consistent performance– Use snapshots for backups
• When using Instance Storage– Replicate! Replicate! Replicate!– Use a replica with EBS volumes for backups
Cost Considerations
All EBS All EBS PIOPS Hybrid Instance Store
47x 1TB General Purpose (SSD)
6x r3.2xlarge(EBS Optimized)
Volumes: $4,700.00Compute: $3,294.00
Total: $7,994.00/mo
35x 134GB @ 4K IOPS
6 r3.2xlarge(EBS Optimized)
Volumes: $586.25EBS IOPS: $9,100.00Compute: $3,294.00
Total: $12,394.25/mo
4x 800GB SSD2x i2.2xlarge
4x 1TB General Purpose (SSD)1x i2.2xlarge
(EBS Optimized)
Volumes: $400.00Compute: $3780.78
Total: $4,180.78/mo
Assuming a performance target of 140,000 IOPS:
All costs are estimations and are based on published pricing as of 6/18/2014.
Cost is an important performance metric!
• Reserved Instances– Pricing model designed for steady state workloads– Pay a small upfront fee for a reduced hourly rate– Light, Medium, and Heavy utilization– 1 and 3 year terms
Pricing Models
Hybrid Instance Store
4x 800GB SSD2x i2.2xlarge
4x 1TB General SSD1x i2.2xlarge
(EBS Optimized)
Volumes: $400.00
• On-Demand– One-time fee: $0.00– Monthly Cost: $4,180.78– Annual Cost: $50,169.36
• 1 YR Heavy Reserved– One-time fee: $10,920– Monthly Cost: $1,119.58– Annual Cost: $13,434.96 ($24,354.96)
• 3 YR Heavy Reserved– One-time fee: $16,440.00– Monthly Cost: $965.86– Annual Cost: $11,590.32 ($17,070.32)
All costs are estimations and are based on published pricing as of 6/18/2014.
Functional Deployment Patterns
• Proper instance type and storage selections play a huge part in price and performance.
• Not all storage architectures are created equal, but neither are all environments!
High Performance MongoDB
Instance Storage
Ephemeral
Set number of volumes per instance
RAID with LVM
Snapshots via LVM
EBS Volumes
Persistent
Multiple volumes per instance
RAID with LVM or MD
Snapshots
MongoDB Storage Considerations
Storage Differences
PIOPS EBS Instance SSDs
Highest durability Highest performance
Requires EBS-optimized instances
Available on new generation instances
Higher price per IOP Included in instance pricing
Development, Testing
MongoDB
App
MongoDBPrimary
App
MongoDBSecondar
y
MongoDBSecondar
y
Single Node Replica Set
Production, High Performance
MongoDBPrimary
App
MongoDBSecondar
y
MongoDBSecondar
y
Replica Set
MongoDBPrimary
MongoDBSecondar
y
MongoDBSecondar
y
Sharding
MongoDBPrimary
App
MongoDBSecondar
y
MongoDBSecondar
y
MongoDBPrimary
MongoDBSecondar
y
MongoDBSecondar
y
mongos
App
mongos
App
mongos
Deployment Strategies: SSDs
• SSD deployments– Replica Sets
and – MMS Backup
• High performance
• Highly available
• Continuous backup
MongoDB
Primary
MongoDB
Secondary
MongoDB
Secondary
MMS Backup Agent
MongoDB Management Service: Backup
Fully-managed, agent-based, continuous
backup
Custom snapshot
scheduling and retention
Point in-time recovery
Performance impact similar to Secondary
Encrypted data transfer
Restores require 2-factor authentication
Deployment Strategies: Hybrid
Mixed SSD/EBS deployment
Secondary with PIOPS
EBS
Powerful, high-bandwidth instance
Aggregate multiple
volumes for max IOPS
ex. r3.8xlarge
with 12 PIOPS EBS volumes
Key is avoiding
replication lag
Monitor with MMS
Measuring Performance
MMS Monitoring
MongoDB metrics
Visualizations
Dashboards
Alerts
System stats tools
iostat
mpstat
sar
mongoperf
MongoDB ops/second
Set threads, fileSize, caching
Partner Technical Solutions, MongoDB
Sandeep Parikh
#MongoDBWorld
Thank You!
Solutions Architect, Amazon Web Services
Kyle Lichtenberg