Scaling Persistent Store UsingMySQL FABRIC

P.R.KARTHIKMySQL DBA

About US

Karthik.P.R• 4 + years of Industry experience as MySQL DBA.

• 1+ year at Yahoo! Administrating MySQL servers.

• Area of Focus MySQL HA and MySQL Sharding

• Student : M.Sc ( FOSS)

• Blogger : remotemysqldba.blogspot.in

PROGRAM AGENDA

• Scaling MySQL

• Available Sharding Tools

• MySQL Fabric

Scaling MySQL

• Scaling MySQL

• Available Sharding Tools

• MySQL Fabric

• Handling Shards

Scaling MySQL

• Starts with a single node

Scaling MySQL

• As reads and writes grows

Scale Vertically.

• Increase system resources• Memory• CPU • Hard Disk

Scaling MySQL

Scaling reads using replication. • Replication is the best solution.• Replication is Async.• Failover has to be set.• Single writer

Read Queries Write Queries

Scaling MySQL

Scaling reads using Galera • Read/ Write on any node• Synchronous Replication• A good HA solution.• Easy to add nodes.

Write / Read Queries

Scaling MySQL

Scaling Writes

• Replication can’t scale writes• Partitioning is needed.• Distributes the writes• Horizontal scaling or Sharding.

ID 1 - 1000 ID 1001 - 2000

Scaling MySQL

Components .

•Shard Key– Range– Hash– List

•Meta data store•Managing data set•High Availability of shards.•Database and schema changes.

Scaling MySQL

Sharding Architecture

Proxy layer

State Store

Shard A

Shard B

Shard C

Application

Scaling MySQL

Advantages of Sharding

•Improved Performance•Smaller Data set on local node.•Handling large data set•Scale well horizontally.•Only small data set is affected on a node failure.

Disadvantages of Sharding

•Managing the shards •Code change is need at some cases.•Splitting Shards•Maintaining the HA

Available Sharding Tools

• Sharding is used widely in Large Scale MySQL Deployments.• Large organization built their own tools for sharding.

Popular MySQL Shards,

Facebook• Twitter• Tumblr• Flickr• Youtube• Dropbox

Available Sharding Tools

Sharding Tools.

1) Saclebase ( Closed Source)2) Jetpants (Tumblr ) 3) Vitess ( Youtube )4) MySQL Fabric ( Oracle MySQL )

MySQL FABRIC

MySQL FABRIC

• MySQL Fabric is the frame work to maintain shards and High Availability.• Downloaded from MySQL Utilities.• Good CLI commands.• Works on MySQL > 5.6.10• XML RPC for Python , php and Java.• Minimized downtime of shard.

MySQL FABRIC ARCHITECTURE

Application XML-RPC Connector

XML-RPC

FABRICNODE

Global Group

Group 1 Group 2 Group 3 Group 4

Back Store

MySQL FABRIC

FABRIC Components.

• Fabric node.• Fabric aware connectors ( XML RPC )• Server Groups.

– Global Group– Shard Group

• Back Store

MySQL FABRIC

Fabric node

• Stores Fabric Config• Shard Moving• Shard splitting• High Availability monitoring

Back Store

• Global tables info• Shard key info and mapping• Shard monitoring

Fabric Node Back store

MySQL FABRIC

Fabric aware connectors

• Supports Python/ php / Java Connectors• Fetch Shard info• Local cache• Avoids the proxy hop.

Application XML-RPC Connector

MySQL FABRIC

Server Groups

• Stores actual data – Global Group– Shard Group

• Primary – Read / Write• Secondary – Reads ( failover)• Spare ( On request )

Primary Node Secondary Node

Spare Node

Server Group

MySQL FABRIC

Server Groups ( Global Group )

• Stores Global Tables and schema• Schema changes• Replicates to Shard groups

Global Group Shard Group

Global Writes

Fabric Node

MySQL FABRIC ARCHITECTURE ( Again )

Application XML-RPC Connector

XML-RPC

FABRICNODE

Global Group

Group 1 Group 2 Group 3 Group 4

Back Store

MySQL FABRIC

Advantages:

• Ease of Failover with GTID.• Shard Split is easy • RPC avoid network hops.

Dis Advantages

• Fabric node will be single point of failure • No Cross Shard . ( Shard query can help here)• Limited Connectors.

THANK YOU