inside mysql 5.7 replication features - fosdem 2018 ... · pdf fileit is not a commitment to...
TRANSCRIPT
![Page 1: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/1.jpg)
1 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Inside MySQL 5.7 Replication Features
Luís Soares ([email protected])Lead Software Engineer, MySQL Replication Team Lead
![Page 2: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/2.jpg)
2 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Safe Harbour Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated into any
contract.
It is not a commitment to deliver any material, code, or functionality, and
should not be relied upon in making purchasing decisions. The development,
release, and timing of any features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
![Page 3: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/3.jpg)
3 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Agenda Background
Next Generation Features in MySQL 5.7
What is in the Lab?
What is Next?
Summary
![Page 4: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/4.jpg)
4 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Background: Replication Components
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
ClientSenderthread
Receiverthread
ApplierThreads
Network
Masterinfo
RelayLogInfo
![Page 5: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/5.jpg)
5 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Binary Log– File based logical log that records the changes on the master.– Statement or Row based format (may be intermixed).– Transactions are split into groups of events.– Control events: Rotate, Format Description, Gtid, ...
Layout of a Binary Log File
Background: Replication Components
BEGIN ...E1 E2 COMMIT BEGIN ...E1 E2 COMMIT
![Page 6: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/6.jpg)
6 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Read scale-out
M SS
S
S
M
write clients read clientsread clients
write clients
Morereads?More
slaves!
Background: What is Replication Used For?
![Page 7: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/7.jpg)
7 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
M M?
write clients write clients
Morewrites?More
Masters?
What about Write scale-out?
M?
M?
We will discuss MySQL Fabric later...
![Page 8: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/8.jpg)
8 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Redundancy: If master crashes, promote slave to master
C
B
A
Background: What is Replication Used For?
C
B
ACrash
C
B
A
B is thenew master
Uh Oh! Whew!
![Page 9: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/9.jpg)
9 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
On-line Backup and Reporting
Background: What is Replication Used For?
M
S
write clients
business intelligent client apps reporting client apps big queries client apps
![Page 10: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/10.jpg)
10 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
CB
BAAC
Image fromwww.ginkgomaps.com
Background: What is Replication Used For?
![Page 11: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/11.jpg)
11 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Agenda Background
Next Generation Features in MySQL 5.7
What is in the Lab?
What is Next?
Summary
![Page 12: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/12.jpg)
12 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
What is New? (2013 – ) MySQL 5.7.2 - Development Milestone Release, September 2013
— Higher Master Throughput Sender, aka Dump, Thread Does Not Take the Binary Log Lock.
— Higher Slave Throughout Multi-Threaded (Slave) Timestamp based Applier (MTS).
— Better Monitoring of Replication Instrumentation for getting replication status through performance schema.
— Loss-less Semi-sync Replication.
![Page 13: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/13.jpg)
13 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Network
Masterinfo
RelayLogInfo
Higher Master Throughput: Sender, aka Dump, Thread Enhancement
Senderthread
Receiverthread
ApplierThreads
![Page 14: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/14.jpg)
14 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Higher Master Throughput: Sender, aka Dump, Thread Enhancement
Sender Thread Reads Binary Log User Thread Writes to Binary Log
Sender Thread Reads Binary Log
User Thread Writes to Binary Log
Time
pre 5.7.2
5.7.2+
Concurrent reads by the sender thread with ongoing writes from user threads.— Sender thread does not block user sessions more than necessary.— Higher throughput for both sender threads and user sessions.
![Page 15: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/15.jpg)
15 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Network
Masterinfo
RelayLogInfo
Higher Slave Throughput: Timestamp based Multi-threaded Applier
Senderthread
Receiverthread
ApplierThreads
![Page 16: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/16.jpg)
16 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Higher Slave Throughput: Timestamp based Multi-threaded Applier
Leverage parallelization information obtained from the execution on the master.— Transactions that prepare on the same “version” of the database, are
assigned the same timestamp.
Meanwhile, at the slave:— Transactions with the same timestamp can be executed in parallel;— Concurrent transactions commit independently, thus no waiting involved.
![Page 17: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/17.jpg)
17 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Higher Slave Throughput: Timestamp based Multi-threaded Applier
T1
T2
T3
Time
Commit
Prepare
Concurrent Execution HistoryConcurrent Execution History on the Masteron the Master
timestamp++
Execution
Prepare
Commit
![Page 18: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/18.jpg)
18 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Higher Slave Throughput: Timestamp based Multi-threaded Applier
T1
T2
T3
Time
Commit
Prepare
Concurrent Execution HistoryConcurrent Execution History on the Masteron the Master
timestamp++
Execution
Prepare
Commit
Parallelon the Slave.
Not Executed inParallel.
![Page 19: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/19.jpg)
19 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Higher Slave Throughput: Timestamp based Multi-threaded Applier
Supports statement-based or row-based formats. Scheduling policy controlled through:
— Logical_clock - means schedule based on the prepare timestamp— database - the scheduling policy from 5.6 (concurrency control done per
database). Work to improve slave scalability continues, does not stop here.
mysql> SET slave_parallel_type= [logical_clock|database]
![Page 20: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/20.jpg)
20 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Network
Masterinfo
RelayLogInfo
Better Replication Monitoring: P_S Replication Tables
Senderthread
Receiverthread
ApplierThreads
![Page 21: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/21.jpg)
21 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Better Replication Monitoring: P_S Replication Tables Access monitoring information through an SQL interface.
Write stored functions or procedures with input from replication internals.
Logically unrelated information into different places.
Flexible and easier to extend and adapt as new feature get into the server.
More user friendly names identifying the monitoring fields.
![Page 22: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/22.jpg)
22 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Better Replication Monitoring: P_S Replication Tables
Connection Configuration
Connection Status
Execution Configuration
Execution Status
Slave Status
Applier / CoordinatorStatus
WorkersStatus
![Page 23: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/23.jpg)
23 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Network
Masterinfo
RelayLogInfo
Loss-less Semi-sync Replication
Senderthread
Receiverthread
ApplierThreads
![Page 24: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/24.jpg)
24 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Loss-less Semi-sync Replication Master does not commit transaction until it receives and ACK from the slave.
— (as opposed to: Master does not release the session until it receives and ACK from the slave.)
— Therefore, concurrent transactions do not externalize changes while waiting for ACK.
Should a master fail, then any transaction that it may have externalized is also persisted on a slave.
User can choose between the original semisync behavior and the new one.
mysql> SET rpl_semi_sync_master_wait_point= [AFTER_SYNC|AFTER_COMMIT]
![Page 25: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/25.jpg)
25 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Loss-less Semi-sync Replication
Master
Slave
execute binlog
relaylog
commitprepare
T1: INSERT INTO t1 VALUES (1000)
ACK
Time
T1
T2: SELECT * FROM t1;
T2
empty set
![Page 26: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/26.jpg)
26 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
What is New? (2013 – ) MySQL 5.7.3 - Development Milestone Release, December 2013
— Improved Security SSL options for mysqlbinlog
— Flexible Semisync Durability Configure master to wait for more than one semisync slave to ACK back.
— More Production Friendly Changing Replication Filters Dynamically.
![Page 27: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/27.jpg)
27 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
mysqlbinlog
Network
Masterinfo
RelayLogInfo
SSL options for mysqlbinlog
Senderthread
Receiverthread
ApplierThreads
--read-from-remote-server
![Page 28: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/28.jpg)
28 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
SSL options for mysqlbinlog
shell> mysqlbinlog readfromremoteserver –ssl u rpluser ...
--ssl* options to mysqlbinlog
— Reading binary logs from remote servers through a secure channel.
— Supports all SSL options that other client tools support.
mysql> GRANT USAGE ON *.* TO 'rpluser'@'localhost' REQUIRE SSL;
![Page 29: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/29.jpg)
29 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Network
Masterinfo
RelayLogInfo
Dynamic Slave Replication Filters
Senderthread
Receiverthread
ApplierThreads
![Page 30: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/30.jpg)
30 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Dynamic Slave Replication Filters
mysql> CHANGE REPLICATION FILTER REPLICATE_DO_DB= (db1, db2)
Change Slave's Replication Filters dynamically.
— No need to stop and restart slave for establishing new replication filtering rules.
— All slave filters are supported.
— Values can be input in various character sets.
![Page 31: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/31.jpg)
31 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Network
Masterinfo
RelayLogInfo
Semi-sync Replication – Wait for Multiple ACKs
Senderthread
Receiverthread
ApplierThreads
![Page 32: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/32.jpg)
32 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Semi-sync Replication – Wait for Multiple ACKs Master does not commit transaction until it receives N ACKs from the slave. Dynamically settable:
mysql> SET GLOBAL rpl_semi_sync_master_wait_for_slave_count= N
![Page 33: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/33.jpg)
33 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Semi-sync Replication – Wait for Multiple ACKs
Master
Slave 2 relaylog
T1: COMMIT
ACK
Time
relaylogSlave 1ACK
T1: COMMITsucceeds
Master does not commit transaction until it receives N ACKs from the slave. Dynamically settable:
mysql> SET GLOBAL rpl_semi_sync_master_wait_for_slave_count= N
mysql> SET GLOBAL rpl_semi_sync_master_wait_for_slave_count= 2
![Page 34: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/34.jpg)
34 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Agenda Background
Generally Available Features in MySQL 5.7
Next Generation Features in MySQL 5.7
What is in the Lab?
What is Next?
Summary
![Page 35: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/35.jpg)
35 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Multi-Source Replication A server (slave) can replicate from multiple sources (masters).
Multiple channels (channel: connection thread, relay log, applier threads) that can be stopped started individually.
Integrated with Multi-threaded Slave: each channel has its own multi-threaded applier set of threads.
Integrated with the new P_S tables.— replication_execute_status_by_coordinator shows multiple entries, one per
channel/source applier.— replication_connection_status shows multiple entries, one per connection to
different sources.
![Page 36: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/36.jpg)
36 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Multi-Source Replication Integrated with GTIDs.
Integrated with crash-safe tables.— Progress state is stored in these tables for recoverability purposes.
Works with semisync replication.— Working on further improving the integration.
No limit on the number of sources.
Able to manage each source separately.
![Page 37: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/37.jpg)
37 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Multi-Source Replication
S
M MSlave can have more than one master.
Feature preview based on 5.7 is availableon labs.mysql.com .
M M
The need for gathering data in a central server: Integrated backup; Complex queries for analytics purposes; Data HUB for inter-cluster replication.
![Page 38: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/38.jpg)
38 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Write Scale-Out - Fabric Big workloads, Replication helps by offloading the master w.r.t. read queries
(which are redirected to slaves).
Huge workloads, Replication helps but eventually the requirement to write everywhere sets a limit on the scalability.
SOLUTION: Split the database into chunks (shards) and distribute writes for groups of servers, instead of writing everywhere.
CHALLENGES: routing queries, global tables, shard-aware connectors, shard migration, shart splitting, cross-shard joins, ...
Enter MySQL Fabric. (Dr. Lars Thalmann will speak about that later today.)
![Page 39: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/39.jpg)
39 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Agenda Background
Generally Available Features in MySQL 5.7
Next Generation Features in MySQL 5.7
What is in the Lab?
What is Next?
Summary
![Page 40: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/40.jpg)
40 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
What is Next?
MySQL Replication Usability— Instrument even more replication and extend replication P_S tables— Simpler administrative commands— Continue to build the GTIDs infrastructure
MySQL Replication Availability— Continue to improve semi-sync
MySQL Replication Performance— Continue to improve slave performance and improve multi-threaded slave
MySQL Fabric— Extend High-Availability Support— Work with community on development
![Page 41: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/41.jpg)
41 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Our focus areas
Read Scale-out: Increased performance— MySQL Replication
Usability: Easier to use Work on improvements— MySQL Utilities
Loss-less replication: Never loose information Work on improvements— MySQL Semi-synchronous Replication
Write Scale-out: Increased performance Work to get to GA— MySQL Fabric
High-Availability: System always accessible Work on more features — MySQL Fabric
![Page 42: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/42.jpg)
42 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Agenda Background
Generally Available Features in MySQL 5.7
Next Generation Features in MySQL 5.7
What is in the Lab?
What is Next?
Summary
![Page 43: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/43.jpg)
43 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Summary
MySQL 5.7.2 already shows features improving:
— Performance, Availability and Usability.
MySQL 5.7.3 continued down that path:
— Tools got improved, HA enhancements, usability improvements.
Lab releases provide a sneak peek at what is cooking: Multi-source, Fabric.
![Page 44: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/44.jpg)
44 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Next Steps: Read about the 5.7 DMRs Replication Features
Read more about P_S tables for replication on Shiv's blog:
http://shivjijha.blogspot.com/2013/09/Monitoring-Replication-with-the-NEW-performance-sc
hema-tables.html
Read more about Intra-Schema Multi-threaded Slave on Rohit's blog:
http://geek.rohitkalhans.com/2013/09/enhancedMTS-deepdive.html
◾ Read more about Semisync enhancements on Libing's blog:
http://my-replication-life.blogspot.com/2013/09/dump-thread-enhancement.html
http://my-replication-life.blogspot.pt/2013/09/loss-less-semi-synchronous-replication.html
![Page 45: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/45.jpg)
45 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Next Steps: Read about the 5.7 DMRs Replication Features
◾ Read more about mysqlbinlog idempotent mode on Rohit's blog:
http://binlogtorelaylog.blogspot.com/2013/05/mysqlbinlog-idemmpotent-mode.html
◾ Read more about mysqlbinlog –rewrite-db option on Manish's blog:
http://manishthe.blogspot.com/2013/05/introduction-with-wonderful-and-best.html
![Page 46: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/46.jpg)
46 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Read more on Dynamic Replication Filters on Venkat's blog:
http://my-s-q-l.blogspot.com/2013/12/Making-MySQL-Slave-Replication-Filters-Dynamic.html
Read more on Semi-sync Multiple ACKs on Libing's blog:
http://my-replication-life.blogspot.com/2013/12/enforced-semi-synchronous-replication.html
Read more on SSL options for mysqlbinlog on João's blog:
http://jmysqlrep.blogspot.com/2013/12/mysql-57-mysqlbinlog-now-supports-ssl.html
Next Steps: Read about the 5.7 DMRs Replication Features
![Page 47: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/47.jpg)
47 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
Read more about Multi-source replication on Rith's blog:
http://on-mysql-replication.blogspot.com/2013/09/feature-preview-mysql-multi-source-replication
.html
Next Steps: Read about the Replication Features on labs.mysql.com
![Page 48: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/48.jpg)
48 | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FOSDEM | Brussels, Belgium, February 1st, 2014. |
QUESTIONS?
![Page 49: Inside MySQL 5.7 Replication Features - FOSDEM 2018 ... · PDF fileIt is not a commitment to deliver any material, code, or functionality, and ... Loss-less Semi-sync Replication Master](https://reader031.vdocuments.net/reader031/viewer/2022030422/5aa9fa0e7f8b9a90188d83ae/html5/thumbnails/49.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.49