scalebase webinar: strategies for scaling mysql

© 2013 by The 451 Group. All rights reserved

Strategies for scaling MySQL

Matthew Aslett, 451 Research

Doron Levari & Paul Campaniello, ScaleBase


Agenda

1. 451 Research – The MySQL SPRAIN

2. ScaleBase - Scale-Out your MySQL Database

3. Customer Case Studies

4. Q & A

(please type questions directly into the GoToWebinar side panel)


Company Overview

One company with 3 operating divisions

Syndicated research, advisory, professional services, datacenter certification, and events

Global focus

200+ staff 1,300+ client organizations:

enterprises, vendors, service providers, and investment firms

Organic and growth through acquisition


Matthew Aslett

• Research Manager, Data Management and Analytics

[email protected]

www.twitter.com/maslett

Responsible for data management and analytics research agenda

Focus on operational and analytic databases, including NoSQL, NewSQL, and Hadoop

With 451 Research since 2007


In a nutshell

MySQL became the default database of choice for Web applications

Success breeds complexity: • Greater scalability demands • Predictable performance problems

New requirements:

• Proliferation of cloud • Geo-distributed data

Multiple strategies for dealing with scalability issues

• Each with their plusses and minuses

The emergence of new vendors aimed at masking the complexity


Previously on ScaleBase/451 webinars…

The traditional relational database has been stretched beyond its normal capacity by the needs of high-volume, highly distributed or highly complex applications.

There are workarounds – such as DIY sharding – but manual,

homegrown efforts can result in database administrators being stretched beyond their normal capacity in terms of managing complexity.

Scalability Performance Relaxed consistency Increased willingness to look towards Agility emerging alternatives Intricacy Necessity


Focus on MySQL

MySQL filled a gap in the market:

• Low cost and freely available with source code

• Better suited to emerging application workloads

MySQL benefits • Lightweight architecture

• Fast read capabilities

• Ease of use

MySQL primarily adopted for:

• in-house-developed applications

• single-function applications

• customer-facing Web applications

• development-and-test environments


Early adoption drivers: 2008 survey


Rapid increase in adoption: 2012 database survey

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

Survey conducted: Jan/Feb 2012 Sample: 205


Scalability, and other challenges

As usage of MySQL has grown, so has the usage of applications that depend on MySQL

This has highlighted a number of challenges

• Scalability of master-slave architecture

• Performance

• Predictability of performance at scale

• Manageability of increasing database/app sprawl

External factors driving greater complexity:

• Distributed computing architectures

• Proliferation of cloud, and elasticity requirements

• Geo-distributed application requirements


The road to SPRAIN: approaches to scaling MySQL

database

users

Example:

Photo sharing website

Started with single MySQL database



database

users users users

Example:



Growth through acquisition and customer adoption



master

slave slave

users users users

Example:




Expanded its MySQL footprint every two years



master

slave slave

users users users

Example:





Larger master

• Pros: Add more CPUs, disk and memory to cope with increased data

• Cons: Doesn’t necessarily help with performance, temporary fix



slave slave

master

slave slave

users users users

Example:





More slaves

• Pros: Good for scaling our read queries

• Cons: Asynchronous replication can lead to write slave lag

By 2009 had 16 slaves and was beginning to hit the wall



database database database

Example:





Considered sharding:

• Pros: increases scalability to serve high-traffic websites

• Cons: Horizontal (across masters), required application re-write

• Cons: Vertical (across the slaves), considered a temporary fix

users users users



slave slave slave slave

master

users users users

Example:





Read/write splitting

• Pros: Better utilization of available resources

• Cons: Asynchronous replication can lead to slave lag



master

memcached

users users users

slave slave

Example:





Memcached

• Pros: Improves read performance for frequently accessed data

• Cons: Not persistent, doesn’t solve write performance issues

• Looked instead for alternatives


Alternatives

NewSQL – retaining support for ACID transactions and SQL

• New databases

• Advanced storage engines, particularly for MySQL

• Advanced clustering/shard management approaches


Comparing NewSQL approaches

New databases • Pros: Designed specifically to support distributed architecture • Cons: May lack compatibility with existing applications

Advanced storage engines, particularly for MySQL

• Pros: Retain familiarity with with MySQL skills, tools • Cons: Re-architecting from the inside out.

Advanced clustering/shard management approaches

• Pros: Retain application compatibility while adding scalability • Cons: An extra layer of complexity?

Issues to consider: • Does it require a forklift move of your entire application ecosystem • Can you continue to leverage your existing MySQL skill set? • Is there a risk for your data, e.g. memory reliability?


Spotlight on ScaleBase

Creates a shared nothing architecture from standard databases

Elastic load balancing for MySQL (other databases on the roadmap)

Scale Out via read/write splitting or automatic data distribution

Data Traffic Manager serves as a proxy between the apps and DB

Provides a single point for administering the shared nothing cluster (for performance, HA, change management)

And the ability to add scalability without the need to migrate to a new database architecture or make any changes to existing apps.


Conclusion

Scalability isn’t simply a matter of a theoretical number of nodes

It’s also about the ability to deliver predictable performance

And to do so without adding management complexity

Distributed computing, proliferation of cloud, and geo-distributed applications are adding to the complexity

MySQL has never been under so much strain

The combination of advanced clustering/load balancing and management technologies provides a potential solution


Thank you. [email protected]

@maslett

Photo credit: swisscan on Flickr http://bit.ly/HCZCRQ

How to Scale Out your MySQL Database

25

Who We Are

Presenters: Paul Campaniello,

VP of Global Marketing 25 year technology veteran with marketing experience at Mendix, Lumigent, Savantis and Precise.

Doron Levari, Founder & CTO A technologist and long-time

veteran of the database industry. Prior to founding ScaleBase, Doron

was CEO to Aluna.

26

ScaleBase Mission

“MySQL can cost-effectively

scale to support an infinite

number of application users,

with NO disruption to the existing infrastructure”

27

ScaleBase: Scale-Out MySQL - Fix the SPRAIN

• Keep your MySQL - keep your InnoDB

• Ecosystem compatibility, preserve skills

• 100% application compatibility

• Smoother migration, no down-time, no forklift

• Your data is safe!

• No “in-memory” magic

• No “in-memory” size limit

Don’t throw out the baby with the bath water!

28

Single-Box Scalability Improvements in MySQL 5.6

Partitioning Improvements

– Explicit Partition Selection:

SELECT * FROM employees

PARTITION (p0, p2);

– Import / Export for Partitioned Tables:

Bring a new data set into a partitioned

table, or export a partition to manage it

as a regular table ALTER TABLE e

EXCHANGE PARTITION p0 WITH

TABLE e2;

http://dev.mysql.co/tech-resources/articles/whats-new-in-mysql-5.6.html

Replication Improvements

– Optimizations to Row-Based

Replication

– Multi-Threaded Slaves

– Improvements to Data Integrity

– Crash-Safe Slaves

– Replication Checksums

SCALABILITY issues remain due to the limitations of a single box:

To ensure ACID, you still face limitations with:

- Memory management - Thread management

- Semaphores - Locking

- Recovery tasks

No new functionality for sharing workloads across multiple boxes

Two Approaches to Scale-Out MySQL

30

Scale Out

Write

Read

Replication

R/W Splitting

Data Distribution

31

Read/Write Splitting: Application Experience

Read Replicas

Current: App sees master plus multiple slaves

With ScaleBase: App sees ONE database

32

Read/Write Splitting: Application Experience

ScaleBase Benefits: • Reads/Writes throughput increases • Better resource utilization/load balancing • Improved data consistency/transaction isolation with:

• Replication state lag awareness • Read stickiness after writes • Handling of edge cases

Problems: • Writing code, maintaining code • Maintaining DB ops and IPs in the app • Master DB is too occupied • Slaves are not balanced • Replication could break • Stale data may be read

Current: App sees master plus multiple slaves

With ScaleBase: App sees ONE database

Read Replicas

33

Data Distribution: Application Experience

Current: App is customized to support multiple shards

With ScaleBase: App sees ONE database…

…and does not require any customization

34

Data Distribution: Application Experience

Problems: • Major app rewrite, maintaining code • Maintaining DB ops and IPs in the app • Administration/3rd party tools are blind • Non-optimized data distribution policy

ScaleBase Benefits: • Unlimited scalability while improving performance • Real time elasticity • Verified data consistency • Real time monitoring, analysis • Carefully analyze distribution policy

Current: App is customized to support multiple shards

With ScaleBase: App sees ONE database…

…and does not require any customization

35

Typical ScaleBase Data Traffic Manager Deployment

Application

Servers

BI

Management

Database A Replica A

Database B Replica B

Database C Replica C

Database D Replica D

Unlimited Scale

Fault Tolerant Architecture

Centrally Manage Complex DB Environment

36

ScaleBase Features and Benefits

Feature Benefit

Automatic data distribution Scale data-, read-, write- intensive applications

Expert System Analysis Optimization of data distribution policy

Parallel query execution Best option for cross-db queries & maintenance commands

Query result aggregation Support of sophisticated cross-db queries, including ORDER BY, GROUP BY, LIMIT, Aggregate functions…

Online data redistribution Flexibility: no need to over-provision No downtime

Read/Write splitting Optimal for scaling of read-intensive applications

Replication lag-based routing Improves data consistency and isolation

Read stickiness after writes Ensure consistent and isolated database operation

100% compatible MySQL proxy Applications unmodified Standard MySQL tools and interfaces

MySQL databases unmodified Data is safe within MySQL InnoDB/MyISAM/etc.

Data consistency verifier Data consistency validation

Real-time monitoring and alerts Simplify management, reduce TCO

37

Scale…without SPRAIN

6000 12000

24000

36000

48000

60000

84000

500 500 1000

1500 1500 2000

2500

0

20000

40000

60000

80000

100000

120000

140000

160000

1 2 4 6 8 10 14

Thro

ugh

pu

t

Number of Databases

Throughput (TPM)

Total DB Size (MB)

# Connections

Scale Your Database With Your Business

38

ScaleBase Delivers Scalability

Scale to

Unlimited Users

Real-time DB Traffic Visibility

Automated Failover

…all with NO App Rewrites

39

Detailed Scale-Out Case Studies

Large Chip Co

• Scalability

• Multiple Apps

• Multiple growing users

• Availability

• MySQL DB

Solar Edge

• Next Gen Monitoring App

• Massive Scale

• Monitors real time data from thousands of distributed systems

Mozilla

• New Product/ Next Gen App/ AppStore

• Scalability

• Geo-clustering

AppDynamics

• Next gen APM company

• Scalability for the Netflix implementation

40

Economic Impact

Customer Experience

– Faster, more responsive apps,

even during peak loads

– Always on: 24/7

– Improve customer acquisition

(reduce churn) and brand

Scale

– Use commodity hardware vs. scale-

up hardware

– Cloud ready – deploys easily

Non-MySQL Options – Protect existing investment with no

ecosystem forklift

– Leverage existing MySQL skill set

Internal Development

– Developers focus on app, not

writing complex sharding code

– Eliminate development and

ongoing maintenance costs

Centralized Management

– Troubleshoot problems and

investigate incidents in seconds

with real-time monitoring

High Availability

– Eliminate database downtime

with automated fail-over

– Improve business continuity

41

ScaleBase Deployment

Environments

– Public Cloud

– AWS, Rackspace, any

– Private cloud

– Hosted / on-premise

Databases Supported

– MySQL 5.1, 5.5, 5.6 (under

certification)

– AWS RDS MySQL 5.1, 5.5

– Maria DB 10.0 (under

certification)

Path to Scale-Out:

1. Data Distribution

Policy Analysis

2. Functional Test

3. Load Test

4. Production Migration

(safe, online)

42

Summary

• MySQL is vast and adoption is growing

• MySQL scalability is a significant problem (SPRAIN)

– App explosion (web & mobile), Big Data transactions…

• ScaleBase provides cost-effective Scale-Out solutions

– Seamlessly scale to an infinite number of users

– Improve DB visibility with real-time monitoring

– Enhance business continuity with

automated failover

– Expert analysis and simple deployment

– No application rewrites

– Leverage existing MySQL ecosystem/skills

43

Questions (please enter directly into the GTW side panel)

[email protected]

[email protected]

@scalebase

www.ScaleBase.com

617.630.2800

[email protected]

@maslett

@451research

www.451research.com

http://www.scalebase.com/

http://www.451research.com/

44

Thank You

scalebase webinar: strategies for scaling mysql

Documents