streaming replication
TRANSCRIPT
Derechos reservados © 2013-2016 Sandino Araico Sánchez <[email protected]>
Se permite ilimitadamente el uso, copia, redistribución con o sin modificaciones siempre y cuando se mantenga el aviso de derecho de autor y se anoten al final de la presentación todas las modificaciones que se llevan a cabo conservando la historia de las modificaciones que hagan las demás personas e indicando la fecha de cada modificación y el nombre de la persona que la llevó a cabo.
2016-10-29PostgreSQL streaming replication sobre VPN
RDBMS =
Relational
Data
Base
Management
SystemManejador de bases de datos
relacionales
ACID =
Atomicity
Consistency
Isolation
Durability
http://en.wikipedia.org/wiki/ACID
WAL =Write
Ahead
Log
https://www.postgresql.org/docs/9.6/static/wal-intro.html
#------------------------------------------------------------------------------# RESOURCE USAGE (except WAL)#------------------------------------------------------------------------------
# - Memory -
shared_buffers = 512MB! ! ! # min 128kB! ! ! ! ! # (change requires restart)temp_buffers = 32MB! ! ! # min 800kB#max_prepared_transactions = 0! ! # zero disables the feature! ! ! ! ! # (change requires restart)
postgresql.conf
Shared Buffers
SELECT * FROM dbmail_users;
Shared Buffers
SELECT * FROM dbmail_users ORDER BY user_idnr DESC LIMIT 5;
Shared Buffers
INSERT INTO dbmail_users (user_idnr, userid) VALUES (31337, ‘sandino’);
Shared Buffers
INSERT INTO dbmail_users (user_idnr, userid) VALUES (31338, ‘chalo’);INSERT INTO dbmail_users (user_idnr, userid) VALUES (31339, ‘roa’);
Shared Buffers
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31337;
Shared Buffers
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31338;UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31339;
Shared Buffers
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31337;
Shared Buffers
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31338;UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31339;
#------------------------------------------------------------------------------# WRITE AHEAD LOG#------------------------------------------------------------------------------
# - Settings -
wal_level = hot_standby # minimal, archive, or hot_standby # (change requires restart)#fsync = on # turns forced synchronization on or off
postgresql.conf
wal_buffers = 16MB # min 32kB, -1 sets based on shared_buffers # (change requires restart)#wal_writer_delay = 200ms # 1-10000 milliseconds
#commit_delay = 0 # range 0-100000, in microseconds#commit_siblings = 5 # range 1-1000
postgresql.conf
Shared Buffers + WAL
INSERT INTO dbmail_users (user_idnr, userid) VALUES (31337, ‘sandino’);
DB WAL
COMMIT;
Shared Buffers + WAL
DB WAL
INSERT INTO dbmail_users (user_idnr, userid) VALUES (31338, ‘chalo’);INSERT INTO dbmail_users (user_idnr, userid) VALUES (31339, ‘roa’);
Shared Buffers + WAL
DB WAL
COMMIT;
Shared Buffers + WAL
DB WAL
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31337;
Shared Buffers + WAL
DB WAL
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31338;UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31339;
Shared Buffers + WAL
DB WAL
CHECKPOINT
Shared Buffers + WAL
DB WAL
ROLLBACK;
Shared Buffers + WAL
DB WAL
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31337;
Shared Buffers + WAL
DB WAL
UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31338;UPDATE dbmail_users SET userid = ‘[email protected]’ WHERE user_idnr = 31339;
Shared Buffers + WAL
DB WAL
COMMIT;
Shared Buffers + WAL
DB WAL
... En algún punto del tiempo ....
Shared Buffers + WAL
DB WAL
Respaldo baseSELECT pg_start_backup('label');
rsync -vacHz --progress /var/lib/pgsql/data/ \ [email protected]:/var/lib/pgsql/data/
SELECT pg_stop_backup();
rsync -vacHz --progress /var/lib/pgsql/archive/ \ [email protected]:/var/lib/pgsql/archive/
recovery.conf
# Specifies a command to load archive segments from the WAL archive. If# wal_keep_segments is a high enough number to retain the WAL segments# required for the standby server, this may not be necessary. But# a large workload can cause segments to be recycled before the standby# is fully synchronized, requiring you to start again from a new base backup.restore_command = 'cp /path_to/archive/%f "%p"'
COMMIT;
Respaldo incremental
DB WAL
DB WAL
COMMIT;
Recuperación
DB WAL
COMMIT;
log shipping
DB WAL
DB WAL
recovery.conf para streaming replication
# Specifies whether to start the server as a standby. In streaming replication,# this parameter must to be set to on.standby_mode = 'on'# Specifies a connection string which is used for the standby server to connect# with the primary.primary_conninfo = 'host=192.168.0.10 port=5432 user=replicator'# Specifies a trigger file whose presence should cause streaming replication to# end (i.e., failover).trigger_file = '/var/lib/pgsql/trigger'
restore_command = 'cp /var/lib/pgsql/archive/%f "%p"'
COMMIT;
Streaming replication
DB WAL
DB WAL
Configuración origen
$ vi postgresql.conf
listen_addresses = '192.168.0.10'
$ vi pg_hba.conf
# The standby server must connect with a user that has replication privileges.host replication replication 192.168.0.20/22 trust
http://wiki.postgresql.org/wiki/Streaming_Replication
$ vi postgresql.conf
wal_level = hot_standby
max_wal_senders = 5
wal_keep_segments = 32
archive_mode = onarchive_command = 'cp %p /path_to/archive/%f'
http://wiki.postgresql.org/wiki/Streaming_Replication
Configuración origen
$ psql -c "SELECT pg_start_backup('label', true)"$ rsync -ac ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid$ psql -c "SELECT pg_stop_backup()"
http://wiki.postgresql.org/wiki/Streaming_Replication
Configuración origen
$ vi postgresql.conf
hot_standby = on
http://wiki.postgresql.org/wiki/Streaming_Replication
Configuración destino
$ vi recovery.conf
standby_mode = 'on'
primary_conninfo = 'host=192.168.0.10 port=5432 user=postgres'
trigger_file = '/path_to/trigger'
restore_command = 'cp /path_to/archive/%f "%p"'
http://wiki.postgresql.org/wiki/Streaming_Replication
Configuración destino
$ psql -c "SELECT pg_current_xlog_location()" -h192.168.0.10 (primary host) pg_current_xlog_location -------------------------- 0/2000000(1 row)
$ psql -c "select pg_last_xlog_receive_location()" -h192.168.0.20 (standby host) pg_last_xlog_receive_location ------------------------------- 0/2000000(1 row)
$ psql -c "select pg_last_xlog_replay_location()" -h192.168.0.20 (standby host) pg_last_xlog_replay_location ------------------------------ 0/2000000(1 row)
http://wiki.postgresql.org/wiki/Streaming_Replication
Verificación
# The displayed LSNs indicate the byte position that the standby server has# written up to in the xlogs.[primary] $ ps -ef | grep senderpostgres 6879 6831 0 10:31 ? 00:00:00 postgres: wal sender process postgres 127.0.0.1(44663) streaming 0/2000000
[standby] $ ps -ef | grep receiverpostgres 6878 6872 1 10:31 ? 00:00:01 postgres: wal receiver process streaming 0/2000000
http://wiki.postgresql.org/wiki/Streaming_Replication
Verificación
Problemática
Estrecho de banda (100 Mbps)
Latencia (200 ms)
Rotación de WAL segments
Downtime (48+ horas)
Problemática
Estrecho de banda (100 Mbps)
Latencia (200 ms)
Rotación de WAL segments
Downtime (48+ horas)
COMMIT;
Log archiving + streaming replication
DB WAL
DB
WAL
Archive
# - Archiving -
archive_mode = on # enables archiving; off, on, or always # (change requires restart)archive_command = 'scp "%p" archive.example.com:/archive/"%f"'
http://wiki.postgresql.org/wiki/Streaming_Replication
Configuración origen
standby_mode = 'on'
primary_conninfo = 'host=192.168.0.10 port=5432 user=postgres'
trigger_file = '/data/pgsql/trigger'
restore_command = 'scp archive.example.com:/archive/"%f" "%p"'
http://wiki.postgresql.org/wiki/Streaming_Replication
Configuración destino
Referencias
http://wiki.postgresql.org/wiki/Streaming_Replication
https://www.postgresql.org/docs/9.6/static/wal-intro.htmlhttps://www.postgresql.org/docs/9.6/static/warm-standby.html
https://www.postgresql.org/docs/9.6/static/continuous-archiving.htmlhttp://en.wikipedia.org/wiki/Relational_database_management_system
http://en.wikipedia.org/wiki/ACID
Sandino Araico Sánchez <[email protected]>@KBrown
#mendozaaaa