mysql scaling for scale 12x conference
Post on 27-Jan-2015
115 Views
Preview:
DESCRIPTION
TRANSCRIPT
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
Insert Picture Here
MySQL Scaling: Drinking from the fire hose, never being down & serving billionsDave StokesMySQL Community Manager
David.Stokes@Oracle.com @Stoker
Slideshare.net/davestokes/presentations
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
You want to get a bigger, badder MySQL --BUT what are the costs???
Budget Complexity Application Opportunity Sanity
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
Insert Picture Here
Simple Stuff
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Option 0
MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15%--Make sure your config file matches your hardware--ANALYSE TABLE--Check for SLOW QUERIES, QUERIES w/o INDEXES--TUNE QUERIES
This is the least complex, less upsetting, and usually lowest budget cost option
Run the Latest, Greatest MySQL
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
Option 0.1
Cache for quick access● Session Data● Sports Scores● Opens up NoSQL access
to InnoDB later
This is the second least complex, less upsetting, and usually lowest budget cost option but does make you change your application to become 'cache aware'.
Add in a Memcached Layer
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Insert Picture Here
What database service Levels do you Need to Provide?
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
UpTime versus Response Time versus Access
Up Time– How much down time can you afford?
Response Time– How long is too long to wait?
Access– How many do you need to serve simultaneously
Things to be engineered/negotiated upfront
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Insert Picture Here
Simple Level 1Read/Write Splitting
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Use MySQL Replication to Split Read/Writes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Read/Write Splitting
● More complex environment
● Application Changes
● Easier with PHP & Java
● Low opportunity and sanity costs
● Less risk of one server stopping operations
Most Common Scaling Technique
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Read/Write Splitting—Easy Way
● mysqlserverclone
● mysqlfailover
● When master dies, most up to date slave (GTID) promoted automatically
● Mysqlrplcheck & mysqlrplshow
Use MySQL Utilities to copy master, set up failover
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Insert Picture Here
Simple Level 2Moderately Easy, Better Availability
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Distributed Replicated Block Device (DRBD)
● More complex environment
● Proven, supported
● Please practice fail over and back
● Easy to manage butnot HA panacea
High Availability
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
CLOUD
● Slightly more complex environment
● Proven, supported
● Best for ephemeral data, may not make upper management comfortable (or you)
● Easy to manage butnot HA panacea
High Availability, Low Cost (maybe)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Insert Picture Here
Complex Level N^NNever, Ever Down
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
MySQL ClusterFor 99.999% uptime, upper bound on response time
● Extremely Complex
● No single point of failure = lots of $$hardware
● Not general purpose MySQL
● Auto sharding
● Add/Subtract nodes at will
● Replicate between geographically separated locations
● Need a smart staff
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
MySQL Cluster Shards Data Automatically
● Single or multiple data node failures do not bring down a cluster
● Easy to add/remove data nodes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
MySQL Cluster Topology
● Nodes
● MGMT – startup/add node
● Data
● Mysqld
● Can be engineered for nosingle point of failure
● Can replicate across data centers
● Expensive on all fronts
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Insert Picture Here
Big Data
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Ask ten DBAs to define 'Big Data' and You Will Get At Least Eleven Answers
.BIG as massive amounts● Define MASSIVE
BIG as in fast● Define fast
BIG as all of the other three!!!BIG as lots of eyes
● How many simultaneous users
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
The Easiest Way to Get Speed
● NoSQL/Memcached to InnoDB/NBD
● 9x faster by skipping parser, optimizer
● Key/Value pair access and SQL access of same data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
Easiest Way to Gain Size
● Columnar Storage Engines for MySQL
● Not InnoDB
● Unique features, quirks
● Need > 26,000,000,000 rows to start seeing 'sweet spot'
● Use new server, don't scrimp
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
Calpont
InfiniDBColumn-oriented
● MySQL front end● Multi threaded● High speed data loading● Automatic partitioning● Enterprise Features
● Partition Drop● UDF’s● Physical compression● MPP, multi-node scale-out● Distributed, Shared Nothing data cache● Auto Increment● Centralized database and operating system
monitoring● Notifications and Alerts● INSERT INTO...SELECT FROM...● Views on InfiniDB Tables
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
Infobright
● Similar Features to Calpont butdifferent implementation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
Hadoop-- getting data from MySQL to Hadoop
● 80% of Hadoop Installations feed from MySQl
● Hadoop is made up of several components
● Twisty little maze, all alike
● Use Apache Bigtop
– Tested for interoperability● Cloudera & Horton Works have great tutorials
● Kinda 'SQL-ish'
● Squoop and MySQL Hadoop Applier to move data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
Hadoop Applier
● Infobright
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Partitioning and Sharding /Dividing and Conquering
● Partitions built-in to MySQL
● Nice management tool
– Rolling quarters/months
– Fast drops
● No change to application but higher complexity, low opportunity cost
● Sharding
● Much more complex application and operations
● Not off shelf
● Hard to re-engineer
● Works
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Insert Picture Here
Hard Part – Thinking
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Goals – What are you trying to accomplish
● Define problem completely
● Who produces, who consumes data
● Negotiate performance rates
● Plan for failure, catastrophe
● Design for long life
● What does end solution look/feel like?
● Platforms
● Speed
● Access areas
● Security
● Serenity
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
Where Else to Seek Info
● MySQL Connect Conference
Sept. in San Francisco
Same time as OOW
Weekdays!
● Planet.mysql.com and forums.mysql.com
● Hadoop
● Horton Works and Cloudera both have wonderful tutorials
● Apache Bigtop project
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
MySQL Training
● MySQL Marinate
● Online Virtual Class run by Boston MySQL Users Group
● 1 chapter/week
● Free
● Oracle and other vendors
● Dozens of book
● Online
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
Insert Picture HereDavid.Stokes@Oracle.com@Stokerslideshare.net/davestokes
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Graphic Section Divider
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
top related