virtualizing mongodb: cloud, ec2, openstack, vms...or dedicated?

Virtualizing MongoDB: Cloud, EC2, GCE or Dedicated?

David MyttonMongoDB World - July 2014

blog.serverdensity.com

David Mytton

Server monitoring, cloud management, dashboards and alerting

serverdensity.com

Server Density Architecture

Server Density Architecture

● ~100 servers - Ubuntu 12.04

Server Density Architecture

● ~100 servers - Ubuntu 12.04

● 50:50 virtual/dedicated

Server Density Architecture

● ~100 servers - Ubuntu 12.04

● 50:50 virtual/dedicated

● 30TB/m incoming data

Server Density Architecture

● ~100 servers - Ubuntu 12.04

● 50:50 virtual/dedicated

● 30TB/m incoming data

● Nginx, Python, MongoDB

Server Density Architecture

● ~100 servers - Ubuntu 12.04

● 50:50 virtual/dedicated

● 30TB/m incoming data

● Nginx, Python, MongoDB

● Softlayer > Google Cloud

Two choices for deployment

Two choices for deployment

● Virtualized

● Bare metal

Advantages of virtualization

● Easy to manage

Advantages of virtualization

● Easy to manage

● Fast boot

Advantages of virtualization

● Easy to manage

● Fast boot

● Easier to resize/migrate

Advantages of virtualization

● Easy to manage

● Fast boot

● Easier to resize/migrate

● Templating/snapshots

Advantages of virtualization

● Easy to manage

● Fast boot

● Easier to resize/migrate

● Templating/snapshots

● Containment

Disadvantages of virtualization

● Another layer

Disadvantages of virtualization

● Another layer

● Hypervisor overhead

Disadvantages of virtualization

● Another layer

● Hypervisor overhead

● Host contention

Disadvantages of virtualization

● Another layer

● Hypervisor overhead

● Host contention

● i/o performance

Advantages of bare metal

● Dedicated resources

Advantages of bare metal

● Dedicated resources

● Direct access to hardware

Advantages of bare metal

● Dedicated resources

● Direct access to hardware

● Customisable specs

Advantages of bare metal

● Dedicated resources

● Direct access to hardware

● Customisable specs

● Performance

Disadvantages of bare metal

● Build/deploy time

Disadvantages of bare metal

● Build/deploy time

● More difficult to resize

Disadvantages of bare metal

● Build/deploy time

● More difficult to resize

● Difficult to migrate/snapshot

Disadvantages of bare metal

● Build/deploy time

● More difficult to resize

● Capex/lifetime

● Difficult to migrate/snapshot

Why cloud?

● Flexible

Why cloud?

● Flexible

● Unlimited resources

Why cloud?

● Flexible

● Unlimited resources

● Cheap to get started

Why cloud?

● Flexible

● Unlimited resources

● Cheap to get started

● Other products

Why colo?

Why colo?

● Vastly cheaper

● Complete control

Performance problems?

Performance problems?

Easy answer: move to bare metal!

Optimising virtualization

● Dedicated CPUs

Optimising virtualization

● Dedicated CPUs

● High speed networking

Optimising virtualization

● Dedicated CPUs

● High speed networking

● Disk i/o (MB/s & IOPS)

Optimising EC2

Optimising EC2

● Newer, 64bit instance types

Optimising EC2

● Newer, 64bit instance types

● Use network / EBS optimised instances

Optimising EC2

● Newer, 64bit instance types

● Use network / EBS optimised instances

● High memory (CPU not important)

Optimising EC2

● Newer, 64bit instance types

● Use network / EBS optimised instances

● High memory (CPU not important)

● Local storage ephemeral

Optimising EC2

● Newer, 64bit instance types

● Use network / EBS optimised instances

● High memory (CPU not important)

● Local storage ephemeral

● Volumes for log, journal and data

Optimising EC2

● PIOPS EBS volumes

Optimising EC2

● PIOPS EBS volumes

● RAID for redundancy & performance

Optimising EC2

● PIOPS EBS volumes

● RAID for redundancy & performance

● blockdev --setra 32

Common EC2 Problems

Common EC2 Problems

● Networking

Common EC2 Problems

● Networking

● Older hardware

Common EC2 Problems

● Networking

● Older hardware

● Complex pricing (PIOPS)

Why EC2?

Why EC2?

● Most popular

Why EC2?

● Most popular

● Range of instance types

Why EC2?

● Most popular

● Range of instance types

● Many regions

Optimising GCE

● High memory instances

Optimising GCE

● High memory instances

● Volume size = IOPS available

Optimising GCE

● High memory instances

● Volume size = IOPS available

● VMs have limits too

Optimising GCE

● High memory instances

● Volume size = IOPS available

● VMs have limits too

● Single volume or wasted space

Optimising GCE

● High memory instances

● Volume size = IOPS available

● VMs have limits too

● Single volume or wasted space

● Network / RAID already handled

Common GCE problems

● Volume too small

Common GCE problems

● Volume too small

● RAIDing unnecessary

Common GCE problems

● Volume too small

● RAIDing unnecessary

● Hitting limits?

Why GCE?

● Best network performance

Why GCE?

● Best network performance

● Easy to size disks / pricing

Why GCE?

● Best network performance

● Easy to size disks / pricing

● Global snapshots

Why GCE?

● Best network performance

● Easy to size disks / pricing

● Global snapshots

● Live migrate

Networking performance

AWS

GCE

bit.ly/googlevsamazon

Tweak: separate disks

Tweak: separate disks

● Avoids i/o contention

Tweak: separate disks

● Optimise IOPS/size/price for journal, dbpath, logs

● Avoids i/o contention

Tweak: separate disks

● Optimise IOPS/size/price for journal, dbpath, logs

● --directoryperdb - even separate out each database

● Avoids i/o contention

--directoryperdb warning

--directoryperdb warning

● Snapshotting more complex

--directoryperdb warning

● Snapshotting more complex

● Requires fsync lock + snapshot all volumes

Summary

● VMs for flexibility

Summary

● VMs for flexibility

● … but need tweaks

Summary

● VMs for flexibility

● Bare metal/colo for price/performance

● … but need tweaks

Summary

● VMs for flexibility

● Bare metal/colo for price/performance

● … but ops overhead

● … but need tweaks

どもありがとうございます

＠davidmytton

david@serverdensity.com

blog.serverdensity.com