mysql options for high availability
TRANSCRIPT
MySQL HA Solutions Selecting the best approach to protect access to your data
Mario Beck MySQL Sales Consulting Manager EMEA https://mablomy.blogspot.de May, 2015
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 2
Program Agenda
Requirements for Next Gen Services
Replication and MySQL Fabric
Failover Solutions
MySQL Cluster
The Basics and Getting started
1
2
3
4
5
3
Program Agenda
Requirements for Next Gen Services
Replication and MySQL Fabric
Failover Solutions
MySQL Cluster
The Basics and Getting started
1
2
3
4
5
4
Focus on driving the business rather than on infrastructure
Driving new Database Requirements
In-Memory Real-Time
Performance
Extreme Read & Write
Scalability
Rock Solid Availability
Rapid Service
Innovation
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 5
HA Considerations
• SLA requirements to support business objectives
• Operational capabilities
• Service agility & time to market
• Budgetary constraints
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 6
• Recovery Time Objective
– Maximum length of downtime before there is break in “business continuity”
• Recovery Point Objective
– Point in time to which data must be recovered when service is re-established
Tier 1 • Mission-critical services
Tier 2 • Business-critical services
Tier 3 • Task-critical services
Tier 4 • Non-critical services
04/05/2015 7
Don’t assume 99.999% HA needed for all apps
Copyright 2015, Oracle and/or its affiliates. All rights reserved
MySQL Replication
MySQL Fabric
DRBD
Windows/Solaris/Clusterware Clustering or Oracle VM
MySQL Cluster
MySQL HA Solutions
04/05/2015
9 9 . 9 9 9 %
Copyright 2015, Oracle and/or its affiliates. All rights reserved 8
Program Agenda
Requirements for Next Gen Services
Replication and MySQL Fabric
Failover Solutions
MySQL Cluster
The Basics and Getting started
1
2
3
4
5
9
Why Replicate?
• Duplicates database from a “master” to a “slave”
– Redundant copies of the data provide foundation for High Availability
– Scale out by distributing queries across the replication farm
Master
Slaves
Web / App Servers
Writes & Reads Reads
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 10
Company Overview Each day, over 700,000 room nights are reserved on Booking.com. Part of the Priceline Group.
Application Booking has relied on MySQL since 2003. Supporting 60-70% annual growth and an agile development model. Why MySQL? Performance and scalability to manage a very large event volume: • 60 MByte/sec Event data rate • 2 Billion Events per day • 15 TB in MySQL for Events, lookups, monitoring • 150 TB MySQL Event archive
Booking.com
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 11
MySQL 5.6 Replication
04/05/2015
• 5x Higher slave throughput with MTS (multiple schemas)
• Reduced master impact – Binlog Group Commit Performance and Scalability
• Global Transaction IDs Simplicity
• MySQL Replication Utilities (including auto-failover) Automation
• Crash safety
• Replication checksums Robustness
• Time Delayed Replication Insurance
Copyright 2015, Oracle and/or its affiliates. All rights reserved 12
MySQL 5.7 Replication – Pre-GA
04/05/2015
• Intra-schema Multi-Threaded Slave
• Faster Master Performance and Scalability
• Performance Schema Monitoring
• Dynamic slave filters
• On-line CHANGE MASTER Operational Simplicity
• Lossless Semi-Synchronous Replication
• Multiple Semi-Synchronous Acks Resilience
• GTIDs stored in transactional tables
• Automatic slave retries MORE Resilience
Copyright 2015, Oracle and/or its affiliates. All rights reserved 13
• Consolidate updates from multiple Masters into one Slave
– Consolidated view of all shards
– More flexible topologies
– Centralized point for backups
• Compatible with Semi-Sync Replication & enhanced MTS
• Master-specific slave filters
Multi-Source Replication
Binlog
Master 1
Binlog
Master 2
…
…
Binlog
Master N
IO 1
Relay 1
Coordinator
W1 W2 … WX
IO 2
Relay 2
Coordinator
W1 W2 … WX
…
…
Coordinator
W1 W2 … WX
IO N
Relay N
Coordinator
W1 W2 … WX
Slave
Release Candidate
04/05/2015 14 Copyright 2015, Oracle and/or its affiliates. All rights reserved
• Shared-nothing virtually synchronous database system
• Multi-master update anywhere
– Conflict detection and resolution (transaction rollback)
– Optimistic State Machine Replication
• Automatic group membership management and failure detection
– No need for server fail-over
– Elastic scale out/in
– No single point of failure
– Automatic reconfiguration
• Well integrated
– InnoDB
– GTID-based replication
– PERFORMANCE_SCHEMA
MySQL Group Replication
Application
MySQL Masters Replication
Plugin
API
MySQL Server
Group Comms (Corosync)
labs.mysql.com
04/05/2015 15 Copyright 2015, Oracle and/or its affiliates. All rights reserved
MySQL Fabric
An extensible and easy-to-use framework for managing a farm of MySQL server supporting high-availability and sharding
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 16
MySQL Fabric 1.5
• High Availability
– Server monitoring with auto-promotion and transparent application failover
• Optionally scale-out through sharding
– Application provides shard key
– Range or Hash
– Tools for resharding
– Global updates & tables
• Fabric-aware connectors rather than proxy: Python, Java, .NET, C (labs)
– Lower latency, bottleneck-free
• Server provisioning using OpenStack etc.
High Availability + Sharding-Based Scale-out
MySQL Fabric
Connector
Application
Read-slaves
SQL
HA group
Read-slaves
HA group
Connector
Application
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 17
MySQL Replication & MySQL Fabric HA
• MySQL Replication is the initial implementation used in HA Groups
– PRIMARY = Replication Master & receives all writes
– SECONDARY = Replication Slave & receives share of reads
• Failover – MySQL Fabric detects failure of PRIMARY/Master
– Selects a SECONDARY/Slave and promotes it
– Updates State Store
– Updated state fetched by Fabric-aware connectors
How this effects failover
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 18
Extensible Architecture
MySQL Fabric Node
Connector
Connector
Connector
MySQL Fabric Framework
AMQP? MySQL XML-RPC
Executor State Store (persister)
Shard
HA
Prov
State Store
SQL
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 19
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 20
• Oracle Clusterware unifies servers in a server farm to form a cluster
– At the core of Oracle RAC
• Oracle Cluster 12c includes MySQL Server 5.6 agent
• Planned migration and failover of MySQL database
– Hidden from the application
04/05/2015
MySQL on Oracle Clusterware
21 Copyright 2015, Oracle and/or its affiliates. All rights reserved
• Pre-Installed & Pre-Configured
• Full Integration & QA Testing
• Single Point of Support
Oracle VM Template for MySQL
Oracle VM Servers
Oracle VM Server Pool
ocfs2
Oracle VM
Manager
iSCSI / FC-AL
Oracle VM
Secure Live
Migration (SSL)
Oracle VM
Automatic Fault
Detection &
Recovery
04/05/2015 22 Copyright 2015, Oracle and/or its affiliates. All rights reserved
• Native Windows HA clustering with MySQL
• Quorum (3rd vote), data (InnoDB + schema) & binaries (optional) stored in shared storage (iSCSI & FCAL)
• Loss of service = couple of seconds + InnoDB recovery time
• Cluster managed through MS Failover Cluster Management snap-in GUI
Windows Server Failover Clustering
App
Virtual IP
Data Bin
Slave App App
Vote
04/05/2015 23 Copyright 2015, Oracle and/or its affiliates. All rights reserved
• Kernel based heartbeating and monitoring
• SPARC and x86. Solaris Virtualization-aware
• MySQL agent included with Oracle Solaris Cluster
• Learn more: http://www.oracle.com/technetwork/server-storage/solaris-cluster/overview/index.html
Oracle Solaris Clustering
04/05/2015 24 Copyright 2015, Oracle and/or its affiliates. All rights reserved
• Based on distributed storage
– NOT physical shared storage
• Synchronous replication at block device level eliminates risk of data loss
• Open source, mature & proven
• Certified and supported by Oracle
Oracle Linux and DRBD Stack
Hosts
Oracle Linux Oracle Linux
Active Standby
Corosync
Pacemaker Cluster s/w
DRBD DRBD
MySQL MySQL
Application
VIP
sync
Services
04/05/2015 25 Copyright 2015, Oracle and/or its affiliates. All rights reserved
Program Agenda
Requirements for Next Gen Services
Replication and MySQL Fabric
Failover Solutions
MySQL Cluster
The Basics and Getting started
1
2
3
4
5
26
MySQL Cluster Overview
• Memory optimized tables with durability
• Predictable Low-Latency, Bounded Access Time REAL-TIME
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS +
WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations 99.999% AVAILABILITY
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools LOW TCO
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 27
Who’s Using MySQL Cluster?
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 28
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 29
MySQL Cluster Scaling
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 30
MySQL Cluster HA
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 31
MySQL Cluster 7.4 GA
• 200 Million NoSQL Reads/Sec
• 2.5M SQL Ops/Sec
• 50% Faster Reads
• 40% Faster Mixed
Performance
• Active-Active Geographic Redundancy
• Conflict Detection/Resolution
Active-Active
• 5X Faster Maintenance Ops
• Detailed Reporting
Management
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 32
• Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed searches
• MySQL Cluster 7.4 FlexAsych – 200M NoSQL Reads/Second
04/05/2015 33
MySQL Cluster 7.4 NoSQL Performance 200 Million NoSQL Reads/Second
Copyright 2015, Oracle and/or its affiliates. All rights reserved
-
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
Readspersecond
DataNodes
FlexAsyncReads
• Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed searches
• MySQL Cluster 7.4 DBT2 BM – 2.5M SQL Statements/Second
04/05/2015 34
MySQL Cluster 7.4 SQL Performance 2.5M SQL Statements/Second
Copyright 2015, Oracle and/or its affiliates. All rights reserved
-
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,000
2 4 6 8 10 12 14 16
SQLStatements/sec
DataNodes
DBT2SQLStatementsperSecond
• Asynchronous replication between MySQL Clusters
• Active-Active
– Update anywhere
– Conflict detection • Application notified through exception tables
• Can opt to have conflicts resolved automatically
– Auto-conflict-resolution • Conflicting transaction and dependent ones
are rolled-back
• No changes to application schema
04/05/2015 35
Active-Active Geo-Replication
Copyright 2015, Oracle and/or its affiliates. All rights reserved
When to Consider MySQL Cluster
Scalability demands Sharding for write performance?
Latency demands Cost of each millisecond?
Uptime requirements Cost per minute of downtime? Failure versus maintenance?
Application agility Developer languages and frameworks? SQL or NoSQL?
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 36
Program Agenda
Requirements for Next Gen Services
Replication and MySQL Fabric
Failover Solutions
MySQL Cluster
The Basics and Getting started
1
2
3
4
5
37
Management Tools Advanced Features Support
• Scalability • High Availability • Security • Audit
•Monitoring • Backup • Development • Administration •Migration
• Technical Support • Consultative Support •Oracle Certifications
MySQL Enterprise Edition
38 Copyright 2015, Oracle and/or its affiliates. All rights reserved
MySQL Enterprise Monitor
• Start monitoring MySQL in 10 minutes
• Real-time MySQL performance and availability monitoring
• Visually find & fix problem queries
• Disk monitoring for capacity planning
• Cloud friendly architecture
– No agents required
• Optional agent option provides advanced Host/OS monitoring
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 39
Enterprise Replication Monitor
• Auto-discovers replication topology
• Master/Slave performance monitoring
• Replication advisor
• Best practice replication advice
"I use the MySQL Enterprise Monitor every day to monitor and keep tabs on our MySQL databases. Quick one stop shopping for keeping tabs on them.” -Wes Homer, Sr System and Network Administrator
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 40
Oracle Enterprise Manager for MySQL
Performance Security
Availability
• Availability monitoring
• Performance monitoring
• Configuration monitoring
• All available metrics collected
– Allowing for custom threshold based incident reports
• MySQL auto-detection
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 41
MySQL Enterprise Backup
• Online, non-locking backup and recovery – Complete MySQL instance backup (data and config) – Partial backup and restore
• Direct Cloud storage backups (S3, etc.) • Incremental backups • Point-in-time recovery • Advanced compressed and encryption • Backup to tape (SBT) • Backup validation • Optimistic backups • Cross-Platform (Windows, Linux, Unix)
42 Copyright 2015, Oracle and/or its affiliates. All rights reserved
Enhancing DevOps Agility, Reducing Downtime
Automated Management
• Start / Stop node or whole cluster
• On-Line Scaling
• On-Line Reconfiguration
• On-Line Upgrades
• On-Line Backup & Restore
• Import Running Cluster
Self-Healing
• Node monitoring
• Auto-recovery extended to SQL + mgmt nodes
HA Operations
• Cluster-wide configuration consistency
• Persistent configurations
• HA Agents
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 43
Oracle Premier Support for MySQL
• Straight from the Source • Largest Team of MySQL Experts • Backed by MySQL Developers • Forward Compatible Hot Fixes • MySQL Maintenance Releases • MySQL Support in 29 Languages • 24/7/365 • Unlimited Incidents • Knowledge Base • MySQL Consultative Support
Rely on The Experts - Get Unique Benefits
"The MySQL support service has been essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com
Only From
Oracle
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 44
How to get started?
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 45
• Define your Needs
– SLA
– Recovery Time Objective
– Recovery Point Objective
• Choose the best Technology
• Ask for Help
– When Architecting the Solution
– For Training
– For Implementation
Thank You! Selecting the best approach to protect access to your data
Mario Beck ([email protected]) Claudiu Moldovan ([email protected]) Ana Ignat ([email protected]) Ioana Irimie ([email protected])
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved
Oracle MySQL HA & Scaling Solutions MySQL
Replication MySQL Fabric
Oracle VM Template
Oracle Clusterware
Solaris Cluster
Windows Cluster
DRBD MySQL Cluster
App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔
Platform Support All All Linux Linux Solaris Windows Linux All
Clustering Mode Master +
Slaves Master +
Slaves Active/Pas
sive Active/Passi
ve Active/P
assive Active/Pas
sive Active/P
assive Multi-Master
Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec
Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔
Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔
Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔
Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔
Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB
Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 47
Oracle University MySQL Training Services Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications
“Training and team skill
have the most significant impact on overall performance of
technology and success of technology projects.” - IDC, 2013
Premier Support customers eligible to save 20% on learning credits.
Benefits
Expert-led training to support your MySQL learning needs
Flexibility to train in the classroom or online
Hands-on experience to gain real world experience
Key skills needed for database administrators and developers
• MySQL for Beginners
MySQL for Database Administrators
MySQL Performance Tuning
MySQL Cluster – NEW - Register Your Interest!
MySQL and PHP - Developing Dynamic Web Applications
MySQL for Developers
MySQL Developer Techniques
MySQL 5.6 Database Administrator
MySQL 5.6 Developer
To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql
Top Courses for Administrators and Developers
Top Certifications
RECENTLY RELEASED !!ALL NEW!! MySQL Cluster Training
To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the
MySQL Cluster Course
04/05/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 48
White Papers & Webinar Replays
04/05/2015
High Availability
• http://www.mysql.com/why-mysql/white-papers/#en-22-16
• http://www.mysql.com/news-and-events/on-demand-webinars/#en-20-16
MySQL Replication
• http://www.mysql.com/why-mysql/white-papers/#en-22-38
• http://www.mysql.com/news-and-events/on-demand-webinars/#en-20-38
MySQL Cluster
• http://www.mysql.com/why-mysql/white-papers/#en-22-28
• http://www.mysql.com/news-and-events/on-demand-webinars/#en-20-28
Copyright 2015, Oracle and/or its affiliates. All rights reserved 49