virtualizing mongodb: cloud, ec2, openstack, vms...or dedicated?
TRANSCRIPT
Virtualizing MongoDB: Cloud, EC2, GCE or Dedicated?
David MyttonMongoDB World - July 2014
blog.serverdensity.com
David Mytton
Server monitoring, cloud management, dashboards and alerting
serverdensity.com
Server Density Architecture
Server Density Architecture
● ~100 servers - Ubuntu 12.04
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
● Nginx, Python, MongoDB
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
● Nginx, Python, MongoDB
● Softlayer > Google Cloud
Two choices for deployment
Two choices for deployment
● Virtualized
● Bare metal
Advantages of virtualization
● Easy to manage
Advantages of virtualization
● Easy to manage
● Fast boot
Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
● Templating/snapshots
Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
● Templating/snapshots
● Containment
Disadvantages of virtualization
● Another layer
Disadvantages of virtualization
● Another layer
● Hypervisor overhead
Disadvantages of virtualization
● Another layer
● Hypervisor overhead
● Host contention
Disadvantages of virtualization
● Another layer
● Hypervisor overhead
● Host contention
● i/o performance
Advantages of bare metal
● Dedicated resources
Advantages of bare metal
● Dedicated resources
● Direct access to hardware
Advantages of bare metal
● Dedicated resources
● Direct access to hardware
● Customisable specs
Advantages of bare metal
● Dedicated resources
● Direct access to hardware
● Customisable specs
● Performance
Disadvantages of bare metal
● Build/deploy time
Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
● Difficult to migrate/snapshot
Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
● Capex/lifetime
● Difficult to migrate/snapshot
Why cloud?
● Flexible
Why cloud?
● Flexible
● Unlimited resources
Why cloud?
● Flexible
● Unlimited resources
● Cheap to get started
Why cloud?
● Flexible
● Unlimited resources
● Cheap to get started
● Other products
Why colo?
Why colo?
● Vastly cheaper
● Complete control
Performance problems?
Performance problems?
Easy answer: move to bare metal!
Optimising virtualization
● Dedicated CPUs
Optimising virtualization
● Dedicated CPUs
● High speed networking
Optimising virtualization
● Dedicated CPUs
● High speed networking
● Disk i/o (MB/s & IOPS)
Optimising EC2
Optimising EC2
● Newer, 64bit instance types
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
● Local storage ephemeral
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
● Local storage ephemeral
● Volumes for log, journal and data
Optimising EC2
● PIOPS EBS volumes
Optimising EC2
● PIOPS EBS volumes
● RAID for redundancy & performance
Optimising EC2
● PIOPS EBS volumes
● RAID for redundancy & performance
● blockdev --setra 32
Common EC2 Problems
Common EC2 Problems
● Networking
Common EC2 Problems
● Networking
● Older hardware
Common EC2 Problems
● Networking
● Older hardware
● Complex pricing (PIOPS)
Why EC2?
Why EC2?
● Most popular
Why EC2?
● Most popular
● Range of instance types
Why EC2?
● Most popular
● Range of instance types
● Many regions
Optimising GCE
● High memory instances
Optimising GCE
● High memory instances
● Volume size = IOPS available
Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
● Single volume or wasted space
Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
● Single volume or wasted space
● Network / RAID already handled
Common GCE problems
● Volume too small
Common GCE problems
● Volume too small
● RAIDing unnecessary
Common GCE problems
● Volume too small
● RAIDing unnecessary
● Hitting limits?
Why GCE?
● Best network performance
Why GCE?
● Best network performance
● Easy to size disks / pricing
Why GCE?
● Best network performance
● Easy to size disks / pricing
● Global snapshots
Why GCE?
● Best network performance
● Easy to size disks / pricing
● Global snapshots
● Live migrate
Networking performance
AWS
GCE
bit.ly/googlevsamazon
Tweak: separate disks
Tweak: separate disks
● Avoids i/o contention
Tweak: separate disks
● Optimise IOPS/size/price for journal, dbpath, logs
● Avoids i/o contention
Tweak: separate disks
● Optimise IOPS/size/price for journal, dbpath, logs
● --directoryperdb - even separate out each database
● Avoids i/o contention
--directoryperdb warning
--directoryperdb warning
● Snapshotting more complex
--directoryperdb warning
● Snapshotting more complex
● Requires fsync lock + snapshot all volumes
Summary
● VMs for flexibility
Summary
● VMs for flexibility
● … but need tweaks
Summary
● VMs for flexibility
● Bare metal/colo for price/performance
● … but need tweaks
Summary
● VMs for flexibility
● Bare metal/colo for price/performance
● … but ops overhead
● … but need tweaks
