flexible heterogenous replication

©Continuent 2012.

Flexible Heterogenous Replication

Je! Mace

Thursday, June 28, 12

©Continuent 2012

Overview

• Introducing Tungsten Replicator

• Installation

• Demo

• Next Steps

2

Thursday, June 28, 12

©Continuent 2012 3

Introducing Tungsten Replicator

• Alternative to MySQL replication

• Still requires binary logs

• Crash-safe with ACID storage engines

• Compatible across branches and versions

• Provides heterogenous replication

• 100% GPL v2

Thursday, June 28, 12

©Continuent 2012

Break speed limits

4

Thursday, June 28, 12

©Continuent 2012

Break speed limits

4

✗Thursday, June 28, 12

©Continuent 2012

Replicate from many masters

5

Thursday, June 28, 12

©Continuent 2012

Replication Pipeline

6

Extract Filter Apply

StageExtract Filter Apply

StageExtract Filter Apply

Stage

Pipeline

MasterDBMS

TransactionHistory Log

In-MemoryQueue

SlaveDBMS

Thursday, June 28, 12

©Continuent 2012

Heterogenous Replication

• Row replication

• DDL must be compatible

• Create schema ahead of time on the slave

• Primary Keys on every table

7

Thursday, June 28, 12

©Continuent 2012

Installing Tungsten

• tungsten-installer

• Written in Ruby

• Simple installation using SSH between database servers

• Validates prior to installation

8

Thursday, June 28, 12

©Continuent 2012

Installing a MySQL Master

tools/tungsten-installer --master-slave -a \--cluster-hosts=my1 \--home-directory=/opt/continuent \--datasource-type=mysql \--master-host=my1 \--datasource-user=tungsten \--datasource-password=secret \--service-name=mongodb \--mysql-use-bytes-for-string=false \--svc-extractor-filters=colnames,pkey \--datasource-log-directory=/var/lib/mysql \--datasource-log-pattern=mysql-bin \--datasource-mysql-conf=/etc/mysql/my.cnf \--start-and-report

9

Thursday, June 28, 12

©Continuent 2012

Installing a MongoDB Slave

tools/tungsten-installer --master-slave -a \--cluster-hosts=mon2 \--home-directory=/opt/continuent \--datasource-type=mongodb \--master-host=my1 \--datasource-user=tungsten \--datasource-password=secret \--service-name=mongodb \--property=replicator.pipeline.slave.services="" \--skip-validation-check=InstallerMasterSlaveCheck \--start-and-report

10

Thursday, June 28, 12

©Continuent 2012

Installing a PostgreSQL Slave

tools/tungsten-installer --master-slave -a \--cluster-hosts=pg3 \--home-directory=/var/lib/postgresql/continuent \--datasource-type=postgresql \--master-host=my1 \--datasource-user=postgres \--datasource-password=secret \--service-name=mongodb \--skip-validation-check=InstallerMasterSlaveCheck \--user=postgres \--postgresql-dbname=postgres \--start-and-report

11

Thursday, June 28, 12

©Continuent 2012

Demo

12

Thursday, June 28, 12

©Continuent 2012

Supported slave databases

• MySQL

• PostgreSQL

• Oracle

• MongoDB

• Vertica

13

Thursday, June 28, 12

©Continuent 2012

Supported master databases

• MySQL

• PostgreSQL (Experimental)

• Oracle (Commercial)

14

Thursday, June 28, 12

©Continuent 2012

Next Steps

• Test in your environment

• Sponsor support for new database types

• Looking for sponsorship to build out a provisioning framework

• Contact us with questions

15

Thursday, June 28, 12

©Continuent 2012

We’re Hiring

• Cluster Implementation Engineer

• QA Engineer

• Technical Writer

16

Thursday, June 28, 12

©Continuent 2012 17

Jeff Macejeff.mace@continuent.comsales@continuent.com

560 S. Winchester Blvd. Suite 500San Jose, CA 95128Tel (866) 998-3642Fax (408) 668-1009

http://www.continuent.com

http://code.google.com/p/tungsten-replicator

Thursday, June 28, 12