1. Ulf Wendel, Oracle Web scale made easy (just kidding...) The mysqlnd replication and load balancing plugin

2. MySQL scale-out solutions Part 1 - Proven and designed for web-scale 3. The speaker says... MySQL offers a variety of scale-out solutions. Which are they, when to use and how to use from an application developers point of view? 4. Developer dreams

Day 1 (Launch) web site visitors The first customer

Systemload: 0.1 100% 5. The speaker says... Let's take a PHP developer centric look at a high-growth web site. The web site is build on top of the industrial leading LAMP platform: Linux, MySQL, Apache and PHP. The day before the launch, the developer dreams the dream of his options or shares rising in value: 2x, 5x, 42x ... 6. Developer dreams Day 2 (Growth) web site visitors The first customer Twitter follower Facebook friend A slashdot reader A TV moderator Systemload: 0.5 7. The speaker says... ... a wonderful dream. The enterprise becomes popular, IPO is on the horizon. There are many reasons for growth. As a developer you should know about the MySQL scale-out solutions to be prepared. According to aMySQL Online Quickpoll from 2009a majority of the MySQL users is using MySQL Replication. 8. Developer dreams Day 3 (Success) scale out needed 408 Request Timeout 503 Service Unavailable 1040 SQLSTATE: 08004 (ER_CON_COUNT_ERROR) -Too many connections Systemload: 42 9. The speaker says... Dear developer, help is on the way. The PHP mysqlnd replication and load balancing plugin ( PECL/mysqlnd_ms ) is a drop-in solution for your PHP MySQL web application adding MySQL scale-out support. For basic scale-out setups no code changes are required. As of PHP 5.3 all three PHP MySQL APIs ( mysql ,mysqli ,PDO_MySQL ) can be compiled to use the MySQL native driver for PHP ( mysqlnd ) library. Mysqlnd is shipped with PHP as of version 5.3. It is a default as of PHP 5.4. 10. Scale out means clustering

Scale-out horizontally

• TCO: Commodity hardware

11. TCO: Proven Open Source solutions

MySQL Server 1 MySQL Server 2 MySQL Server n 12. The speaker says... Scale out, not up. Use a cluster of commodity servers. Scale horizontally building upon proven Open Source solutions.Try MySQL Replication, MySQL Cluster or a custom MySQL scale-out solution. 13.

Asynchronous read scale-out

• Master: writes, Slaves: reads

14. Various topologies supported

15. Other use cases: backup, HA, OLTP/warehousing MySQL Replication Writes Reads Master Slave Slave Slave 16. The speaker says... MySQL Replication is used for read scale-out.Writes are send to a MySQL master, reads are send to a MySQL slave. The master sends updates to the slaves in anasynchronousway. Slaves may not have the latest changes. MySQL Replication can also be used for:

Backup perfom blocking backup on slave

17. High Availablility for example, remote slaves 18. Warehousing OLTP reporting on the slaves 19.

Real-time transactional write scale-out

• 99,999% availability

20. Auto sharding, shared nothing architecture

21. Web use cases: session server, tele communication MySQL Cluster SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node 22. The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out.Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node a MySQL server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks. 23.

For example, Partitioning

• Horizontal: rows in different tables

24. Vertical: columns in additional tables

25. ... sharding Custom Cluster Shard 1 ID % 2 = 0 Node 3 Cats Node 4 Dogs Shard 2 ID % 2 = 1 26. The speaker says... Special requirements may make it necessary to build a custom cluster.For example, your application may require a sharding solution. 27. On using any database cluster Part 2 - Required application changes 28.

Tasks:

• Choose servers to execute statement

29. Load balance within candidates

30. Maintain connection pool 31. Automatic fail over for High Availability Application using any DB cluster MySQL Server 1 MySQL Server 2 MySQL Server n 32. The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a1:n relation . 1:n puts an additional task on the application:find the right n, find the right server to talk to. 33. The plugin for all of you MySQL Server 1 MySQL Server 2 MySQL Server n 34. The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library . The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thusall existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy.Depending on plugin an use case, no code changes are needed.A plugin is a drop-in solution. 35. Plugin focus: MySQL Replication Part 3 - Tasks, solutions and limitations 36. Using MySQL Replication

Statement redirection

• Read/write split (optional: master on write)

37. Replication filter (table based parititioning)

38. Custom: callback

Load Balancing

• Round robin, random, random once

39. Custom: callback

Connection Pooling

• Lazy connections

40. Failover

41. The speaker says... The plugin is not limited to but optimized for MySQL Replication.The common clustering support feature set focusses on MySQL Replication.For example, statement redirection covers read/write splitting and MySQL Replication filter support. All read accesses are executed on the slave servers, all writes are performed on the master. MySQL Replication supports table based partitioning, replicating selected tables to selected slaves only. The plugin can be made aware of this. 42. Speaking code /* Connection user handle represents a connection pool */ $mysqli = new mysqli( "hostname" , $user, $pw, $db, $port); /* Read/Write split, Load Balancing */ $res = $mysqli->query("SELECT 'Slave speaking' FROM DUAL"); var_dump($res->fetch_all(MYSQLI_ASSOC)); { "hostname" :{ "master":{ "host":"localhost" }, "slave":{ "slave_0":{"host":"192.168.2.27"} }, } } 43. The speaker says... No code changes for basic use cases. Install the plugin following standard PHP PECL procedures. Create a configuration file. Enable the plugin and set its configuration file in your PHP configuration using mysqlnd_ms.enable=1 and mysqlnd_ms.ini_file=config.ini .In the plugin configuration create a section with the name of the host your application connects to, for example hostname. List master and slave servers. Any connection to hostname is now handled by the plugin. Using any API. 44. Some code changes /* Connection user handle represents a connection pool */ $mysqli = new mysqli("hostname", $user, $pw, $db, $port); / * R/W split: no SELECT, run on master */ $mysqli->query("SET @myrole='Master'); /* R/W split: overruled by SQL hint */ $sql = sprintf("/*%s*/%s", MYSQLND_MS_MASTER_SWITCH, "SELECT @myrole AS _role"); $mysqli->query($sql); var_dump($res->fetch_all(MYSQLI_ASSOC)); 45. The speaker says... Non-trivial use cases require support by the application. In other words: existing applications may need updates, new applications need to be designed appropriately. In the example a SQL hint is used to force executio of a SELECT statement on a MySQL Replication master server. There are SQL hints for running a statement on the master, on the slave and on the last used server. Alternative: new APIs call, no win compared to SQL hint. 46. Why and when code changes $mysqli = new mysqli("hostname", $user, $pw, $db, $port);$mysqli->query("SET @myrole='Master'); $mysqli->query("SELECT myrole AS _role"); Without plugin With plugin 47. The speaker says... An application not designed for use with a cluster assumes a connection handle always points to the same server (1:1), having the same state.When using the plugin a connection handle represents a pool of connections (1:n), potentially having different states.Connection switches may occur whenever a statement is executed. After the switch, the connection handle may point to a connection with a different state. Applications need to take care and hint the plugin. 48. A trouble maker in detail

Connection State

• Connection settings, e.g. charsets

49. Transaction status

50. Temporary tables 51. Table locks 52. Session system variables and user variables 53. Prepared statements (API and SQL PREPARE) 54. HANDLER variables 55. Locks acquired with GET_LOCK 56. Units of work from an application point of view 57. The speaker says... No client library, no client proxy can monitor all state changing actions. Applications in need for a certain connection state need to control the plugins connection switchesusing :

SQL hints

• MYSQLND_MS_SLAVE_SWITCH

58. MYSQLND_MS_MASTER_SWITCH

59. MYSQLND_MS_LAST_USED_SWITCH

Callback to replace built-in server selection logic

60. Trouble maker transaction

Unit of work

• Part of the connection state

61. Application to prevent connection switches

Workaround

• PHP 5.3: SQL hints must be used

62. PHP 5.4: API calls monitored, switch auto-disabled

Replication cluster tip

• Always run on master, only master has latest data!

63. use trx_stickiness = master

64. The speaker says... A transaction is part of the connection state.For the duration of a transaction, server switches must be disabled.Transactions can be controlled by SQL statements and API calls. The plugin cannot monitor all of them with reasonable effort.Thus,you must hint the plugin.Disable load balancing by adding SQL hints (MYSQLND_MS_LAST_USED_SWITCH), you install your own load balancer logic (callback).As of PHP 5.4, the plugin monitors API calls to auto-disable switching. 65.

Slave request

• read-only: statement begins with SELECT

66. read-only: statement begins with slave SQL hint

67. custom: callback picks slave for statement Redirection: Read/Write split Writes Reads (SELECT, SQL hint) Master Slave Slave Slave 68. The speaker says... MySQL Replication is for read scale-out. Read-only statements shall go to slaves, writes to the master.The plugin automatically redirects a statement to a slave, if the statement begins with SELECT or the MYSQLND_MS_SLAVE_SWITCH SQL hint.All other statements are considered as write operations and executed on the master. By default SHOW or CALL will be run on the master. It is a design principle of the plugin that all automatic decisions can be overruled.A callback can be installed to pick the server for running a statement on, if needed. 69.

Optimization: use master only after write

• MySQL Replication is asynchronous

70. MySQL slaves can lag behind

71. Most applications need current data after write Redirection: Master on write Writes, Reads after write Reads before write Master Slave Slave 72. The speaker says... The master_on_write configuration option is an optimization of the R/W split. MySQL Replication slaves can lag behind the master because MySQL Replication is asynchronous.When using master_on_write all reads made after a write are executed on the master for the duration of the web request. Applications that perform a write often need to rely on fresh reads afterwards. Think of a web shop creating an order, writing to the master. If reading again from the order table, within the same web request, the application should access the master to make sure the previously written is available. 73.

MySQL Replication filter support

• Slaves replicate different databases/tables

Redirection: Partitioning Writes Reads (Filter) DB_Report DB_Shop DB_Report DB_Internal DB_Shop DB_Shop Master Slave Slave Slave 74. The speaker says... MySQL Replication supports partitioning: slaves can be configured to replicate only selected databases and tables using MySQL Replication filter.Filter help to reduce the amount of internal replication messages. Less frequently accessed data can be replicated to only few slaves. In future versions, plugin can be made aware of the filtering rules for appropriate statement redirection.Every statement is analyzed to find a slave for reading. If no match, the plugin redirects the statement to the master. 75.

Built-in or user-defined through callback

• Round robin: iterate over slaves for each read

76. Random: pick random slave for each read

77. Random once: pick random slave for all reads Load Balancing Writes Load Balancing of Reads Master Slave Slave Slave 78. The speaker says... MySQL Replication is about read scale-out. The plugin helps distributing the load over the read-only slaves.The best and default load balancing stategy is the one with the lowest impact on the connection state: random once.Minimize switches for the life-span of your PHP script. Pick a random slave during startup and use it for all reads. The life-span of PHP is the of a web request. It is short. Two web requests will end up on two random slaves. Load balancing adaptive to server health can be achieved through a user-defined callback. 79.

Caution: connection state change

• Default: failover=disabled, raise exception

80. Optional: failover=master, automatic

Pooling: connection failover Writes Reads Master Slave Slave Slave 81. The speaker says... Automatic and transparent failover is impossible with stateful connections.The plugin does not and cannot know the full state of a connection. Parts of the state are only known to the server. By default the plugin leaves it to the application to handle a connection failure.Like without the plugin, it is up to the application to decide what to do. Applications using stateless connections can enable automatic failover to the master. 82.

Do not open connection before needed

• Delay open until statement execution

83. Reduce number of connections, reduce load

Pooling: lazy connections Writes Reads Master Slave Slave Slave 84. The speaker says... By default, to reduce the connection pool size of every PHP web request, connections are not opened before executing a statement.Imagine you have 50 concurrent web requests, you configured four slaves and, you are using random once. If not using lazy connections, the plugin opens 50 x 4 = 200 slave connections but uses at max 50 of them (random once).Every connection occupies resources ever seen a PHP application opening connections before using? There are many badly written ones... 85. See also... 86. Documentation and resources

php.net/mysqlnd_ms

• Installation, Quickstart, Concepts, Reference

php.net/mysqlnd

• Introduction, Changes, C plugin API

87. THE END Credits: Andrey Hristov, Johannes Schlter - Contact: ulf.wendel@oracle.com