reliable mysql using replication issac goldstand mirimar networks [email protected]
TRANSCRIPT
![Page 2: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/2.jpg)
What is replication?
• Allows 2 or more databases to maintain state between them
• MySQL 3.23 and up supports asynchronous replication
• One server acts as a master for one or more slaves
• Slaves pull data to be replicated from server’s binary logs and execute the relevant statements locally
![Page 3: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/3.jpg)
Replication basics…
• One master supports multiple slaves• Each node on the network is assigned a unique
identifying number• Each slave attempts to connect to the master and
fetches data to be replicated
MasterSlave
SlaveSlaveSlaves
![Page 4: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/4.jpg)
Replication basics…
• Clients perform data modification on master server
• INSERT, SELECT, DELETE, LOAD DATA• EXECUTE in MySQL 5.0 and above
Master SlaveClient
DATA DATAINSERT INTO …
![Page 5: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/5.jpg)
Replication basics…
• Immediately following execution of command on master, the command is written to the local binary log
• Additionally, the master records its unique ID (to prevent endless loops in circular replication scenarios) and the timestamp for use with statements which use NOW(), etc.
Master SlaveClient
DATADATA
INSERT INTO …
Binary Log
![Page 6: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/6.jpg)
Replication basics…
• If the slave is online, the command is transmitted to the slave in parallel (well, immediately following) to being written in the local binary log
• Otherwise, when the slave next connects it will receive a list of all pending statements from the master server’s binary log
• The slave’s replication IO thread stores the command in the local relay log
Master SlaveClient
DATADATA
INSERT INTO …
INSERT INTO …
Relay Log
Replication Thread
![Page 7: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/7.jpg)
Replication basics…
• Once the data is received in the slave’s relay log, the slave SQL thread executes the command locally, bringing the slave up-to-date with the master
• In MySQL 3.23, the IO and SQL threads were just one thread. In later versions this was changed to boost performance
Master SlaveClient
DATADATA
INSERT INTO …
INSERT INTO …
Relay Log
Replication Thread
DATA
![Page 8: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/8.jpg)
Replication Strategies
• Load balancing – single write, distributed read
Master
Slave Slave Slave
Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT)
Client (INSERT)
![Page 9: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/9.jpg)
Replication Strategies
• Load balancing – single write, distributed read• Load balancing – circular read/write
MySQLServer
MySQLServer
MySQLServer
Client
INSERT INTO…
INSERT INTO …
Client
![Page 10: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/10.jpg)
Replication Strategies
• Load balancing – single write, distributed read• Load balancing – circular read/write• High availability (hot failover)
Client Master
DATA
Slave
DATA
DATADATADATA
![Page 11: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/11.jpg)
Replication Strategies
• Load balancing – single write, distributed read• Load balancing – circular read/write• High availability (hot failover)• Snapshot backups
Master
Slave
Backup with LOCK TABLES (or single transaction)
ClientINSERT / SELECT
ClientINSERT / SELECT
![Page 12: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/12.jpg)
Simple Replication Setup
• Modify my.cnf to include a unique server-id for each node
• On master server, ensure that log-bin (binary logging) is enabled in my.cnf
• On slave, configure login credentials on master, either via my.cnf or CHANGE MASTER TO statement
• Copy initial data snapshot from master to slave• Configure initial binary log position on slave• Start replication with SLAVE START command
![Page 13: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/13.jpg)
Configure master
my.cnf
-------------
[mysqld]
server-id = 1
log-bin
![Page 14: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/14.jpg)
Configure slave
my.cnf
-------------
[mysqld]
server-id = 2
master-user = someuser
master-password = secret
master-host = ip.of.master
![Page 15: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/15.jpg)
Initial dataset
• Binary log provides a record of all modifications to master database starting from a fixed point: when binary logging was activated
• If all binary logs exist on master from initial install of MySQL, the slave(s) can use these to bring themselves up-to-date
• Otherwise, a snapshot of the master must be taken, using mysqldump –master-data, to provide an initial dataset for the slave(s)
• If only MyISAM tables are used, the LOAD DATA FROM MASTER statement may be used on the slave(s)
![Page 16: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/16.jpg)
Configure log position
MASTER mysql> SHOW MASTER STATUS;+---------------------------+----------+--------------+------------------+| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |+---------------------------+----------+--------------+------------------+| vmware-mirimar-bin.000002 | 79 | | |+---------------------------+----------+--------------+------------------+
SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘vmware-mirimar-bin.000002’, MASTER_LOG_POS=79;SLAVE mysql> START SLAVE;
![Page 17: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/17.jpg)
Using CHANGE MASTER TO
• MASTER_HOST
• MASTER_USER
• MASTER_PASSWORD
• MASTER_LOG_FILE
• MASTER_LOG_POS
![Page 18: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/18.jpg)
Advanced CHANGE MASTER TO
• MASTER_PORT
• RELAY_LOG_FILE
• RELAY_LOG_POS
• MASTER_SSL
• SSL: CA/CAPATH/CERT/KEY/CIPHER
![Page 19: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/19.jpg)
Confirming it works
SLAVE mysql> SHOW SLAVE STATUS\G*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: vmware-mirimar Master_User: someuser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: vmware-mirimar-bin.000002 Read_Master_Log_Pos: 79 Relay_Log_File: vmware1-mirimar-relay-bin.000002 Relay_Log_Pos: 250 Relay_Master_Log_File: vmware-mirimar-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 79 Relay_Log_Space: 250
Seconds_Behind_Master: 0
![Page 20: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/20.jpg)
Permissions
• Slaves need REPLICATION SLAVE permission on master for basic usage
• If LOAD TABLE FROM MASTER or LOAD DATA FROM MASTER statements are used, slave will also need SUPER and RELOAD privileges
![Page 21: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/21.jpg)
Internal Threads
• Since MySQL 4.0, replication slaves run two threads
• IO thread continuously receives updates from master and writes to local relay log
• SQL thread continuously executes statements in relay log
![Page 22: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/22.jpg)
IO thread isolation
• Isolating IO thread means that slave won’t have to wait for long-executing statements to finish executing before retrieving data from master
• Also, slave will continue reading data from master if a statement creates a data conflict
![Page 23: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/23.jpg)
SQL thread isolation
• SQL thread isolation allows for replication in an environment without a continuous link between slave and masters
• If master fails (or slave simply has no access), the IO thread will try to reconnect endlessly (waiting 60 seconds between attempts)
• SQL thread will continue processing relay logs even while IO thread is unable to connect to master
![Page 24: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/24.jpg)
Master Thread
• Additionally, the master server runs the Binlog Dump thread
• This thread is simply dedicated to scanning the binary logs on the master and sending updates to the connected slave
• If this thread isn’t running, it means that replication isn’t running – more accurately, that no slaves are currently connected
![Page 25: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/25.jpg)
Status files
• 2 status files for replication’s use• Their use is to record the state of
replication between server shutdown and startup
• master.info records information about the slave’s master server
• relay-log.info records information about the local relay logs
![Page 26: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/26.jpg)
Information in master.info
• Master log file• Read master log pos• Master Host• Master User• Password (will not be shown in SHOW SLAVE
STATUS)• Master Port• Connect Retry• In MySQL 4.1+, SSL options are stored if SSL is
used
![Page 27: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/27.jpg)
Information in relay-log.info
• Relay log file
• Relay log pos
• Relay master-log pos
• Exec master-log pos
![Page 28: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/28.jpg)
Backup master
• Master backups can be accomplished with mysqldump
• Care must be taken to ensure the following 2 special considerations:
1. Consistent snapshot of master date (via lock tables for MyISAM or single transaction for InnoDB)
2. Recording of binary log information, for use on slaves (master-data)
![Page 29: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/29.jpg)
Backup master files
• If a file-system level backup is required, care should be taken to manually record binary log name and position via SHOW MASTER STATUS statement.
• To ensure consistency between backup and binary log position, the tables should be locked via FLUSH TABLES WITH READ LOCK immediately before backup (and SHOW MASTER STATUS)
• LEAVE THE CLIENT CONNECTED!!!• After backup finishes, execute UNLOCK TABLES to
release the read lock
![Page 30: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/30.jpg)
Backup slave
• Same idea as master file system backup
• Instead of recording position, it’s enough to backup the master.info and relay-log.info files
• Instead of acquiring global read lock, it’s enough to STOP SLAVE before backup and START SLAVE once backup finishes
![Page 31: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/31.jpg)
Live demo
• Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup
![Page 32: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/32.jpg)
For more information
• MySQL documentation
• 5.0 documentation http://mirror.mirimar.net/mysql/doc/refman/5.0/en/replication.html
• 4.1 documentation http://mirror.mirimar.net/mysql/doc/refman/4.1/en/replication.html
![Page 33: Reliable MySQL Using Replication Issac Goldstand Mirimar Networks margol@beamartyr.net](https://reader035.vdocuments.net/reader035/viewer/2022062314/56649d9d5503460f94a85fcb/html5/thumbnails/33.jpg)
Thank You!
For more information:
Issac Goldstand [email protected]
http://www.beamartyr.net/
http://www.mirimar.net/