1 architecture & design





Architecture & Design

Mark Swarbrick Principle Presales Consultant

MySQL Webinar Series 1/4





Program Agenda

§  Causes, Effects and Impacts of Downtime

§  Methodology to Select Right HA Solution

§  Guided Tour of MySQL HA Solutions

§  Operational Best Practices






The following is intended to outline our general product direction. It is

intended for information purposes only, and may not be incorporated

into any contract. It is not a commitment to deliver any material, code,

or functionality, and should not be relied upon in making purchasing

decision. The development, release, and timing of any features or

functionality described for Oracle’s products remains at the sole

discretion of Oracle.

Safe Harbor Statement






40% DATA GROWTH PER YEAR

5.9BN MOBILE SUBS IN 2011

1 BILLION iOS & ANDROID APPS DOWNLOADED PER WEEK

85% HANDSETS SHIPPED WITH A BROWSER

$1TR BY 2014

$700BN IN 2011

2.1BN USERS

8X DATA GROWTH IN 5 YRS

70+ NEW DOMAINS EVERY 60 SECONDS

350m TWEETS PER DAY

955M USERS

20M APPS PER DAY

60 HOURS UPLOADED EVERY

MINUTE






Cloud

Web & Enterprise OEM & ISVs

Industry Leaders Rely on MySQL





Copyright © 2012, Oracle and/or its affiliates. All rights reserved. September 29th, 2012

The Lowdown on Downtime…. Maintenance Failures Disasters Operator / User

Revenue Customer Sat Employee Regulatory

CAUSE

EFFECT

IMPACT






HA Considerations

•  SLA requirements to support business objectives • Operational capabilities •  Service agility & time to market •  Budgetary constraints






Not Everything Needs 99.999% Uptime •  Recovery Time Objective

–  Maximum length of downtime before there is break in “business continuity”

•  Recovery Point Objective –  Point in time to which data must be recovered when service is re-established

•  Tier 1, Mission-critical services •  Tier 2, Business-critical services •  Tier 3, Task-critical services •  Tier 4, Non-critical services






Mapping Uptime to Availability

%"9" 9" 9" 9"9"35 days 4 days 50 mins 5 mins 8 hours

Cos

t & C

ompl

exity

SPs & Line of

Business

Web & Cloud

Services

eCommerce Telecoms Military

.

Shared-Nothing, Geo-Replicated

Clusters

Clustered & Virtualized Systems

Replicated Systems






MySQL HA Solutions

%"9" 9" 9" 9"9"35 days 4 days 50 mins 8 hours

Ope

ratio

nal C

ompl

exity

SPs & Line of

Business

Web & Cloud

Services

eCommerce Telecoms Military

.

Replication

5 mins






Scale Up

•  Proprietary SMP hardware

•  Closed or Open Source software

•  Platform lock-in

•  “Fork Lift” to increase capacity

Scalability with MySQL

Scale Out

•  Commodity Intel / AMD

•  Open source software

•  Platform freedom

•  Add commodity servers to increase capacity






Why Replicate? •  Duplicates database from a “master” to a “slave”

–  Redundant copies of the data provide foundation for High Availability –  Scale out by distributing queries across the replication cluster

Master

Slaves

Web / App Servers

Writes & Reads Reads






Slave Database

Replication Workflow

•  Session thread: processes queries from the application – writes data to master database & associated events to binary log

•  Dump thread: reads events from binary log and sends them to a slave

•  I/O thread: receives replication events and stores them in slave’s relay log

•  SQL thread: reads replication events from slave’s relay log and applies them to slave database

Session

Binary Log

Master Database

Dump I/O

Relay Log

SQL






Pros

§  Simple to configure §  Heterogeneous §  Low TCO – no shared storage §  Configure over LAN and WAN §  Fast failover

Cons

§  Asynchronous: risk of of data loss §  Semi-Synchronous is configurable §  No automated failover or

switchover §  Performance overhead to master

Current Generation of MySQL Replication






MySQL 5.6: Evolving Replication

•  Multi-Threaded Slaves •  Binary Log Group Commit •  Optimized Row-Based Replication

PERFORMANCE

•  Global Transaction Identifiers •  Replication Failover & Admin Utilities •  Crash Safe Slaves & Binlogs

FAILOVER & RECOVERY

•  Replication Event Checksums DATA INTEGRITY

•  Replication Utilities •  Time-Delayed Replication •  Remote Binlog Backup •  Informational Log Events •  Server UUIDs

DEV/OPS AGILITY






Global Transaction Identifiers •  Simple to track & compare replication across the cluster

– Unique identifier for each transaction written to the Binlog •  Automatically identify the most up-to-date slave for failover •  Deploy n-tier replication hierarchies

Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456






•  Enabling self-healing replication topologies •  Automated failover & recovery

–  mysqlfailover Utility •  Switchover & administration

–  mysqlrpladmin Utility •  Delivers HA within the core MySQL

distribution

HA Utilities Monitoring

Failed Master

Slaves

Promoted Master

Replication HA Utilities






Clustering Overview •  Tightly coupled clusters of servers/storage providing service to an

application –  Use heartbeating and management software to monitor hardware, OS,

database and network –  Node failures detected and failover automatically initiated to redundant

systems –  Implements Virtual IP (VIP) so failover is transparent to the application

•  Couple with Virtualization for higher agility –  Migrate between servers with load balancing






Oracle Certified Solutions

Oracle VM Template

Solaris Cluster

Windows Cluster DRBD

Auto-‐Failover & Migra1on ✔ ✔ ✔ ✔

Zero Data Loss ✔ ✔ ✔ ✔

Ac1ve / Passive ✔ ✔ ✔ ✔

Pla:orm Support Linux Solaris Windows Linux

Shared Storage ✔ ✔ ✔ ✖

Single Vendor Support ✔ ✔ ✖ ✔






3rd Party HA Solutions

• Oracle supports MySQL on open source and commercial HA technologies – Support for HA mechanisms from respective communities or

vendors

•  Symantec Veritas Cluster • Red Hat Cluster Suite

Copyright Oracle Corporation 2014 | <date> 24

An extensible and easy-to-use framework for managing a farm of MySQL server supporting high-availability and sharding

MySQL Fabric


MySQL Fabric 1.4

§  High Availability: –  Server monitoring with auto-promotion

and transparent application failover

§  Optionally scale-out through sharding –  Application provides shard key –  Range or Hash –  Tools for resharding –  Global updates & tables

§  Fabric-aware connectors rather than proxy: Python, Java & PHP

–  Lower latency, bottleneck-free

High Availability + Sharding-Based Scale-out

MySQL Fabric

Connector

Application

Read-slaves

mappings

SQL

HA group

Read-slaves

HA group

Connector

Application


Asynchronous vs. Synchronous Replication § Asynchronous (MySQL Default)

–  In parallel: Master acks to app and sends transaction to slave §  Fast §  Risk of lost changes if master dies

§  Synchronous (Only available with MySQL Cluster) –  Serially: Master waits for change to be applied on all slaves before ack

to app §  Higher latency §  If Active/Active, best suited to small transactions §  Loss-less

§  Semi-Synchronous (MySQL 5.5+ - Enhanced in MySQL 5.7) –  Serially: Master waits for change to be received by slave then In

parallel ack to app and apply changes on slave §  Intermediate latency §  Lossless


MySQL Fabric Framework (HA)

All Data

Primary Secondary

Extra Read Replicas

MySQL Fabric Controller

SQL Queries

State & Routing Info

HA Group

Coordination and Control


MySQL Replication & MySQL Fabric HA

§ MySQL Replication is the initial implementation used in HA Groups –  PRIMARY = Replication Master & receives all writes

§ Failover –  MySQL Fabric detects failure of PRIMARY/Master –  Selects a SECONDARY/Slave and promotes it –  Updates State Store –  Updated state fetched by Fabric-aware connectors

& how this effects failover


High-Availability Group Concept §  Abstract Concept

–  Set of servers –  Server attributes

§  Connector Attributes –  Connection information –  Mode: read-only, read-write, ... –  Weight: distribute load

§  Management Attributes –  State: state/role of the server State: Primary

Mode: Read-Write Host: server-1.example.com


Scaling on dense, multi-core, multi-thread servers

•  10s - 100GBs RAM •  SSDs

Scale across cores within a single instance You can get a long way with MySQL 5.6!

The Path to Scalability Scaling-Up can take you a long way


MySQL Fabric Features

§  Connector API Extensions –  Support Transactions –  Support full SQL

§  Decision logic in connector –  Reducing network load

§  Shard Multiple Tables –  Using same key

§  Global Updates –  Global tables –  Schema updates

§  Sharding Functions –  Range –  (Consistent) Hash

§  Shard Operations –  Shard move –  Shard split


MySQL Fabric Node

MySQL

MySQL Fabric Framework

Executor State Store (Persister)

Sh HA

MySQL AMQP XML-RPC ? ?

Protocols

Extensions

Backing Store

Extensible Architecture


MySQL Fabric: Goals & Features §  Connector API Extensions

–  Support Transactions –  Support full SQL

§  Fabric-Aware Connectors at GA: –  PHP + Doctrine, Python, Java +

Hibernate

§  Decision logic in connector –  Reducing network load

§  Load Balancing –  Read-Write Split –  Distribute transactions

§  Global Updates –  Global tables –  Schema updates

§  Shard Multiple Tables –  Using same key

§  Sharding Functions –  Range –  (Consistent) Hash

§  Shard Operations –  Shard move –  Shard split


MySQL Fabric – Current Limitations

§  Routing is dependent on Fabric-aware connectors –  Currently Java (+ Hibernate), PHP (+

Doctrine) & Python §  MySQL Fabric node is a single (non-

redundant process) –  HA Maintained as connectors

continue to route using local caches §  Establishes asynchronous replication

–  Manual steps to switch to semisynchronous

§  Sharding not completely transparent to application (must provide shard key – column from application schema)

§  No cross-shard joins or other queries

§  Management is through CLI or XML/RPC API –  No GUI






Who’s Using MySQL Cluster?





MySQL Cluster Architecture: High Performance

MySQL Cluster Data Nodes

Data Layer

Clients

Applica1on Layer


Management Management





MySQL Cluster Architecture: Extreme Resilience


Data Layer

Clients

Applica1on Layer


Management Management






Multi-Data Availability Geographic Replication

•  DR and Data Locality •  Replicate complete clusters

across data centers –  Fully active/active –  No passive resources

•  Split individual clusters across data centers –  Synchronous replication &

auto-failover between sites

Geographic Replication






Handling Scheduled Maintenance On-Line Operations

Scale

Upgrade

Backup

Evolve






SLAs Platforms Application requirements (storage engines) Performance expectations vs tolerance to data loss Skill sets






Mapping Applications to HA Technology Applica'ons Database/

Replica'on Clustered///Virtualized

Shared:Nothing,/Geo:Replicated/

Cluster

E"Commerce)/)Trading ))))))))))))))))(1))Session)Management ))))))))))))))))(1))

User)Authen<ca<on)/)Accoun<ng )))))))))))))))))(1))Feeds,)Blogs,)Wikis

OLTP )))))))))))))))))(1))Data)Warehouse/BI

Content)Management

CRM

Collabora<on

Packaged)SoJware

Network)Infrastructure)

Core)Telco)Apps)(HLR/HSS/SDP…))

1: Replication used in combination with cluster or virtualization – based HA

MySQL HA Solutions

Note that MySQL Fabric is currently at Release Candidate Status

Small: Web Reference Architecture

Single server supporting: Users & sessions eCommerce Content management

MySQL replication Analytics and Backups

If traffic volumes increase, scale session management first on a separate server

Limited Scalability Recommended to start with the Medium Architecture

MySQL Enterprise Monitor

MySQL Enterprise Backup

MySQL Master

MyS

QL

Rep

licat

ion Analytics

Slave 2

Backup

Slave 1

Medium: Web Reference Architecture

Best Practices - Overview Medium Web Reference Architecture

Server ratio: 10 application servers to each MySQL Server More for PHP applications, fewer for Java Add more slaves as the application tier scales

Caching layer deployed in session & content management components Memcached or Redis are common Reads fulfilled from cache, relieving load on the source database servers NoSQL Memcached APIs can enable compression of caching and data layers

Best Practices – Content Management Medium Web Reference Architecture

Each slave can handle around 5,000 concurrent users Each master handles 20 slaves

More slaves are often used in larger environments

MySQL Replication for high availability & scale-out Meta data of content assets managed by MySQL Distributed File System / CDN / Cloud (i.e. S3) for physical

storage of assets on High quality SAN (redundancy for HA)

Across commodity nodes

XOR

Best Practices – Sessions & eCommerce Medium Web Reference Architecture

Session Management & eCommerce Session data maintained for up to 1 hour in a dedicated partition, rolling

partitions used to delete aged data NoSQL Memcached API for InnoDB can be used to scale session management

eCommerce HA Semi-Synchronous replication or clustering (DRBD, shared storage, etc.)

If web traffic grows, move Session Management to MySQL Cluster Persist session data for real-time personalization of user experience 99.999% availability and in-memory data management can reduce need for DRBD & caching

§ Data is captured in Analytics Database, increasingly replicated to Big Data system for analysis

Large: Web Reference Architecture Session Management eCommerce

MySQL Master

Content Management

Slave 1

MySQL Master

Analytics MySQL

Enterprise Monitor

Slave 2 Slave 3 Slave 4 Slave 5

Slave 6 Slave 7 Slave 8 Slave 9 Slave 10

Slave N

Slave 2

Node Group 2

F2

F4 Nod

e 3

Nod

e 4

F2

F4

Node Group 1

F1

F3 Nod

e 3

Nod

e 4

F1

F3


MySQL Servers

Node Group 2

F2

F4 Nod

e 3

Nod

e 4

F2

F4

Node Group 1

F1

F3 Nod

e 3

Nod

e 4

F1

F3


MySQL Servers

Slave 1 Slave 3

Hadoop Cluster


Caching / Application Servers

Distributed Storage

XOR

Extra Large: Social Network

Shard Catalog

Authentication

MySQL Master

Analytics

Slave 2 Node Group 2

F2

F4 Nod

e 3

Nod

e 4

F2

F4

Node Group 1

F1

F3 Nod

e 3

Nod

e 4

F1

F3


MySQL Servers

Slave 1 Slave 3

Hadoop Cluster


Node Group 2

F2

F4 Nod

e 3

Nod

e 4

F2

F4

Node Group 1

F1

F3 Nod

e 3

Nod

e 4

F1

F3


MySQL Servers

App /Caching Servers Central Databases

Customers 1% – 33%



Slave 1 Slave 2 Slave N

Shards









Operational Best Practices Training

Consul1ng

24 x 7 x 365 Global Support

Database Management & Monitoring






Oracle Premier Lifetime Support

Oracle Product Certifications/Integrations

MySQL Enterprise High Availability

MySQL Enterprise Security

MySQL Enterprise Scalability


MySQL Enterprise Monitor/Query Analyzer

MySQL Workbench

MySQL Enterprise Edition Highest Levels of Security, Performance and Availability

MySQL Enterprise Audit






•  Web-based, global view of MySQL/Cluster applications (on-premise and Cloud deployments)

•  Automated, rules-based monitoring and alerts (SMTP, SNMP enabled)

•  Query capture, monitoring, analysis and tuning, correlated with Monitor graphs

•  Real-time Replication Monitor with auto-discovery of master-slave topologies

•  Integrated with Oracle Support


A Virtual MySQL DevOps + Admin!






§  Online Backup for InnoDB (scriptable interface) §  Full, Incremental, Partial Backups (with compression) §  Point in Time, Full, Partial Recovery options §  Parallel, multi-threaded backup/recovery operations §  Metadata on status, progress, history §  Unlimited Database Size §  Cross-Platform

-  Windows, Linux, Unix

§  Certified with Oracle Secure Backup

MEB Backup Files

MySQL Database Files

mysqlbackup

Ensures quick, online backup and recovery of your on premise and Cloud based MySQL applications.







•  Straight from the Source •  Largest Team of MySQL Experts •  Backed by MySQL Developers •  Forward Compatible Hot Fixes •  MySQL Maintenance Releases •  MySQL Support in 29 Languages •  24/7/365 •  Unlimited Incidents •  Knowledge Base •  MySQL Consultative Support

Oracle Premier Support for MySQL Rely on The Experts - Get Unique Benefits

"The MySQL support service has been essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com

Only From Oracle






Next Steps Learn More

Evaluate MySQL 5.6 Book a Workshop

HA Solutions Guide http://bit.ly/NB8up1

Download Today http://dev.mysql.com/downloads/mysql/

High Availability Jumpstart http://www.mysql.com/consulting/

Copyright 2011 Oracle Corporation 57






[email protected]

1 architecture & design

Technology

oracle andor

mysql copyright

minute copyright

rights r

sole discretion of oracle

bn users

data growth

bn mobile subs