running mysql in aws
DESCRIPTION
TRANSCRIPT
Running MySQL in AWSThe next generation in RDBMS
Laine Campbell, CEO PalominoDB
Why Amazon Web Services?
Infrastructure as a Service
● No more waiting for hardware.
● No more managing hardware.
● No more aged out hardware.
Why Amazon Web Services?
Elasticity
Why Amazon Web Services?
Financial Considerations● Cloud expenses are opex, not capex - fully
deductible the first year.● Eliminate hardware
inventory.● Reduce costs of
administrators, facilities, network, etc...
Why MySQL?
A Known Quantity● Easy to find developers and operators.● Extensive documentation, community and
tribal knowledge.● Well documented knowledge of bugs, with
multiple vendors supporting, improving and operating. (slowing down on Oracle front)
Why MySQL?
But is it Web Scale?● Scaling patterns are well understood.● With proper tuning and knowledge, can
support the vast majority of workloads.● Relational data is often the right tool for the
right workload. ● Data consistency and recoverability isn't as
optional as many believe.
MySQL Scaling Patterns
Need more transactions?
Vertical Scaling○ Tune your SQL.○ Manage your configs.○ Add MOAR RAM○ Add MOAR DISK○ Add MOAR CPU○ Add of all the things.
MySQL Scaling Patterns
Need more reads?
○ Add Replicas via MySQL Replication○ Add Cache Tier○ Tune your ORM
MySQL Scaling Patterns
Replication Lagging?
○ Functional Partitioning - Break out workloads into separate clusters.
○ Split database schema into multiple schemas - use parallel replication. (5.6, or Tungsten)
○ Stop abusing MySQL! Some workloads don't belong in a relational database! (logs, queues, massive non-transactional reads)
MySQL Scaling Patterns
MySQL Scaling Patterns
Need more writes?
○ Time to shard! Split your data-sets into multiple clusters and scale linearly.
Options for Running MySQL
Amazon Relational Database Service (RDS)
● Choose your instance size.● Choose your major MySQL Version.● Choose from a few topology options.● Choose your parameter configuration.● Choose your region and availability zone.
Your RDS MySQL Master
Stand-Alone or Multi-AZ● Stand-Alone:
○ If it goes away, you must use backups to recover.○ Less expensive than multi-AZ, good for prototyping.○ With replicas, you can use standard failover/replica
promotion.○ Storage up to 1 TB. (auto striping over 300 GB for
higher performance)○ Provisioned IOPs allow for predictable I/O
performance.○ Choose your region and availability zone.
Multi-AZ Failover
● A Highly Available Master:○ Uses synchronous data replication to keep a failover
ready.○ Auto-failover.○ Redundancy for less than 2x the cost.
● The Problems:○ Failover happens more often than you might think.○ Failovers are impacting for up to 30 seconds.○ If relaxing binlog flushes, failovers can disrupt your
replication topology.○ Synchronous replication reduces write capacity.
Multi-AZ Failover
From AWS Blog
MySQL Replicas
● Push Button Replicas○ Rapid deployment of replicas via snapshot.○ Promotion to masters:
■ For non-multi AZ failover.■ For standalone dev/test environments.
● The Problems:○ Builds can cause some load issues on your master.○ Only one built at a time.○ All must be identical.○ No complex replication topologies.
RDS Gotchas● Parameter Groups
○ Must change from Default to use dynamic variables.○ Configuration changes become more complex.
● Limited Access○ No root/super-user access.○ No direct management of replication.○ No access to error logs or binlogs.
● Other Concerns○ Minimal visibility to system issues○ No OS access for diagnostics
Options for Running MySQL
Amazon Elastic Compute Cloud (EC2)
● Not Database As a Service● Build your own instances, choosing your
Operating system, compute resources and RAM.
● Multiple Regions, and availability zones within a region for redundancy.
Running MySQL in EC2Storage Options● EC2 comes with ephemeral storage by
default.○ If your instance restarts, you lose the data.○ SSD options available.
● Use Elastic Block Storage (EBS) for persistence.○ EBS volumes can be RAIDed for higher IO
throughput. (not for higher availability)○ Snapshots are a highly desirable backup
mechanism.
Running MySQL in EC2Choosing MySQL● Unlike RDS, you can choose your binaries.
○ MySQL 5.1, 5.5, 5.6, MariaDB, Percona XtraDB, Drizzle.
● Unlike RDS, you can choose your replication topology:○ Standard master/slave○ Master/master○ Tiered replication
● Unlike RDS, you can replicate across regions for disaster recovery.
Running MySQL in EC2● Options for reducing management overhead
○ Roll your own config mgmt via Chef, Puppet, Ansible○ Script it yourself (really?)○ Continuent Tungsten can provide full cluster
management with similar cost of RDS, and none of the RDS gotchas.
Pros of MySQL in EC2● Advanced configurations (topologies,
heterogenous instance types or storage types) allow DBAs to build truly custom solutions.
● Access to the OS, all logs, root user and full management.
● Fewer RDS gotchas (rds specific admin tasks, surprises mid-operation, etc...)
● DBA overhead not as high as you might think for commoditized tasks.
Pros of MySQL in EC2● EC2 Reserved instances can be reused by
non-DB tiers (cache, app, file, etc...) vs. RDS reserved instances which must be MySQL.
● Snapshots allow for easy building of test environments.
● Access to advanced topologies means much less downtime for rolling migrations and upgrades.
● Multi-region disaster recovery.
Pros of MySQL in RDS● RDS push button tasks are super easy and
useful.● Allows DBAs to focus immediately on high
value tasks (sql tuning, data modeling, scaling plans)
● Can help enable developers (can also help them shoot themselves in the foot!)
● No need to build out your own failover mechanisms.
Q&A
Laine Campbell, CEO PalominoDB
http://www.slideshare.net/lainecampbell