what's new in my sql smug
TRANSCRIPT
MySQL 5.6 RC and MySQL Cluster 7.3 DM What's New!
Ted Wennmark MySQL User Group
21th November 2012
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
2
Safe Harbour Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated into any
contract.
It is not a commitment to deliver any material, code, or functionality, and
should not be relied upon in making purchasing decisions. The development,
release, and timing of any features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
3
Program Agenda
MySQL 5.6 Release Candidate
MySQL Cluster 7.3 Labs Release
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
4
MySQL 5.6 builds on MySQL 5.5 by improving:
- InnoDB for better transactional throughput, availability
- Optimizer for better query execution times, diagnostics
- Replication for higher availability, data integrity
- Performance Schema for better instrumentation
- Other Important Enhancements
Available Now! Get it here:
dev.mysql.com/downloads/mysql/
MySQL 5.6 Release Candidate New!
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
5
InnoDB
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
6
Several internal improvements (e.g. Split kernel mutex)
Optimized for Read Only workloads
SSD Optimizations
− 4, 8k page sizes
− .ibd files outside of MySQL data dir
− separate tablespaces for undo log
MySQL 5.6: InnoDB Better Performance, Scalability
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
7
Optimized for Read Only workloads
− Highly concurrent, read intensive web apps
− Enables developer control of read only transactional overhead
MySQL 5.6: InnoDB Better Performance, Scalability
SET autocommit = 1;
SELECT c FROM sbtest WHERE id=N;
SET autocommit = 0;
START TRANSACTION READ ONLY;
SELECT c FROM sbtest WHERE id=N;
COMMIT;
On by default
Developer controlled
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
8
MySQL 5.6.7
MySQL 5.6 SysBench Benchmarks
Scales to 48 CPU Threads
Oracle Linux 6
Intel(R) Xeon(R) E7540 x86_64
MySQL leveraging:
- 48 of 96 available CPU threads
- 2 GHz, 512GB RAM
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
9
MySQL 5.6.7
MySQL 5.6 SysBench Benchmarks
Scales to 48 CPU Threads
Oracle Linux 6
Intel(R) Xeon(R) E7540 x86_64
MySQL leveraging:
- 48 of 96 available CPU threads
- 2 GHz, 512GB RAM
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
10
Online DDL Operations implemented:
- CREATE INDEX
- DROP INDEX
- Change AUTO_INCREMENT value for a column
- ADD/DROP FOREIGN KEY
- Rename COLUMN
- Change ROW FORMAT, KEY_BLOCK_SIZE for a table
- Change COLUMN NULL, NOT_NULL
- Add, drop, reorder COLUMN
MySQL 5.6: InnoDB Better Availability
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
11
Dump and restore/warm buffer pool
- At shutdown or manually at any time
- Shortens warm up times after restart (from hours to minutes)
Persistent Optimizer Statistics
- Increased plan stability
- More accurate statistics
- Better user control, automatic/manual
MySQL 5.6: InnoDB Better Availability
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
12
FULLTEXT indexes on InnoDB tables
Keys on text-based content
Speeds up searches for words, phrases
Fully transactional, fast look up
Natural language/Boolean modes, proximity search, relevance ranking
MySQL 5.6: InnoDB Full Text Search create table quotes
( id int unsigned
auto_increment primary
key
,author varchar(64)
, quote varchar(4000)
, source varchar(64)
, fulltext(quote)
) engine=innodb;
select author as “Apple" from quotes
where match(quote) against (‘apple' in natural language mode);
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
13
Transportable Tablespaces
- Enables export/import of tables between running MySQL instances
- Faster than mysqldump (via portable .ibd files)
MySQL 5.6: InnoDB More Flexibility
CREATE TABLE t(c1 INT) engine=InnoDB;
FLUSH TABLE t FOR EXPORT; -- quiesce the table and create the meta data file
$innodb_data_home_dir/test/t.cfg
UNLOCK TABLES;
Export:
Import: CREATE TABLE t(c1 INT) engine=InnoDB; -- if it doesn't already exist
ALTER TABLE t DISCARD TABLESPACE;
-- The user must stop all updates on the tables, prior to the IMPORT
ALTER TABLE t IMPORT TABLESPACE;
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
14
Key-value access to InnoDB
- via Memcached API
- Use existing Memcached clients
- Bypasses SQL parsing
NotOnlySQL access
- For key-value operations
- SQL for rich queries, JOINs, FKs, etc.
Implemented via:
- Memcached plug-in to mysqld
- Memcached mapped to native InnoDB
- Shared process for ultra-low latency
InnoDB Storage Engine
MySQL Server Memcached plugin
Application
SQL (MySQL Client)
NoSQL (Memcached Protocol)
mysqld
MySQL 5.6: InnoDB More Flexibility
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
15
Optimizer
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
16
For Developers
- No more re-writing into joins
- Table pull-out
- Semi-join
- Subquery Materialization
DBT3 Query #18 benchmark:
- Execution time drops from DAYS to seconds
MySQL 5.6: Optimizer Subquery Optimizations
SELECT title FROM film WHERE film_id IN
(SELECT film_id FROM film_actor
GROUP BY film_id HAVING count(*) > 12);
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
17
Web use case – list top 100 products sorted by name
Avoid creating intermediate sorted files
Produce ordered result set using a single table scan
Example above: 20 million rows, default sort buffer
- 4x better execution time (drops from 40s to 10s)
CREATE TABLE products( productid int auto_increment PRIMARY KEY,
productname varchar(200)
);
SELECT * FROM products ORDER BY productname LIMIT 100;
MySQL 5.6: Optimizer File Sort Optimizations with Small Limit
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
18
test per
CREATE TABLE person ( personid INTEGER PRIMARY KEY,
firstname CHAR(20),
lastname CHAR(20),
postalcode INTEGER,
age INTEGER, address CHAR(50),
KEY k1 (postalcode,age)
) ENGINE=InnoDB;
SELECT lastname, firstname FROM person WHERE postalcode BETWEEN 5000 AND 5500 AND age BETWEEN 21 AND 22;
With ICP Disabled - 15 s (buffer pool 128 Mb) - 1.4 s (buffer pool 1.5 Gb)
With ICP Enabled − Execution time drops to 90 ms for both
MySQL 5.6: Optimizer Index Condition Pushdown (ICP)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
19
0 8 1 6 2 4 3 2 4 0 4 8 5 6 6 4
5
5 0
5 0 0
5 0 0 0
1 2 2 5
9 . 6 3
2 8 2 1
N o B K A
B K A
J o in B u f f e r S ize ( M B )
Qu
ery
Tim
e (
se
cs
)
Improves performance of disk-bound join queries
Execution time
without BKA + MRR
Execution time
with BKA + MRR
DBT3 Q 13: “Customer Distribution Query”
MySQL 5.6: Optimizer Batched Key Access (BKA) and Multi-Range Read (MRR)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
20
EXPLAIN
− INSERT, UPDATE, and DELETE
− Structured EXPLAIN output
Optimizer Traces
MySQL 5.6: Optimizer Better Diagnostics
SET SESSION OPTIMIZER_TRACE=‘enabled=on’;
SELECT (SELECT 1 FROM t6 WHERE d = c)
AS RESULT FROM t5;
SELECT * FROM information_schema.OPTIMIZER_TRACE;
"records_estimation": [ { "database": "test",
"table": "t6",
"range_analysis": {
"table_scan": { "records": 2, "cost": 4.5034
}, "potential_range_indices": [
{ "index": "d",
"usable": true, "key_parts": [
"d" ] } ],
"best_covering_index_scan": { "index": "d",
"cost": 1.4233,
"chosen": true
},
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
21
Replication
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
22
Simple to track & compare replication across the cluster
- Unique identifier for each transaction written to the Binlog
Automatically identify the most up-to-date slave for failover
Deploy n-tier replication hierarchies
Master
GTID=123456
GTID=123456
GTID=123456 GTID=123456
MySQL 5.6: Replication New! Global Transaction Ids
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
23
Enabling self-healing replication topologies
Automated failover & recovery
- mysqlfailover Utility
Switchover & administration
- mysqlrpladmin Utility
Delivers HA within the core MySQL
distribution
- Eliminates the need for 3rd party solutions
- Allows extensibility to support variety of HA mechanisms
HA Utilities Monitoring
Failed
Master
Slaves
Promoted
Master
MySQL 5.6: Replication HA Utilities (Python)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
24
MySQL 5.6: Replication Multi-Threaded Slaves
5x performance gain
SysBench, running across 10 x schemas
Oracle Linux 6.1, Oracle Sun Fire x4150 m2 Server
Increases slave throughput,
reducing lag
All transactions received into
slave’s relay log
Implements multiple SQL threads,
based on database
Applies events to different
databases in parallel
Great for systems which isolate
application data using databases
– e.g. multi-tenant
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
25
Increases replication throughput by
increasing performance of the
master
Commits multiple transactions as a
group to Binlog on disk
Finer grained locking; reducing lock
wait times
Session
Binary
Log
Master
Database
T1 T2
T3 T4
Group commit
MySQL 5.6: Replication Binary Log Group Commit
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
26
Before:
–Transaction Data: in tables
–Replication Info: in files
MySQL 5.6
–Transaction Data: in tables
–Replication Info: in tables
Data
Position Info
CRASH!
Time
Data
Position Info Time
Automatic recovery of a slave after a
failure
− Binlog and table data are transactionally
consistent
Resumes replication without Dev/Op
intervention
− Automatically rolling back replication to
last committed event
Eliminates risk of data loss or corruption
Atomic
Atomic
MySQL 5.6: Crash safe Slaves
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
27
Ensures replicated data is correct,
consistent and accessible
Detects corrupt replication events before
they’re applied
– Returns an error
Protects entire replication path
– Memory
– Disk
– Network
– Bugs
Master
#
Slave
#
MySQL 5.6: Replication Event Checksums
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
28
Performance Schema
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
29
MySQL 5.6 Performance Schema We’ve been busy!
New Instrumentation
Statements/Stages
Table and Index I/O
Table locks
Users/Hosts/Accounts
Network I/O
New Features
Show contents of Host cache
New Summary tables
Easier configuration
- Start up defaults in my.cnf
- Auto tune
Reduced overhead
On by default
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
30
Statements/Stages
- What are my most resource intensive queries? Where do they spend time?
Table/Index I/O, Table Locks
- Which application tables/indexes cause the most load or contention?
Users/Hosts/Accounts
- Which application users, hosts, accounts are consuming the most
resources?
Network I/O
- What is the network load like? How long do sessions idle?
Summaries
- Aggregated statistics grouped by thread, user, host, account or object
MySQL 5.6 Performance Schema
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
31
Security
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
32
Major overhaul of password handling
- Provide alternatives to showing passwords in plain text
- Assess/Enforce password strength policies
- Enforce new password at next login
- Stronger password hashing
MySQL 5.6: Security
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
33
Other Important Enhancements
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
34
New default configuration settings
TIME/TIMESTAMP/DATETIME - fractional second precision
TIMESTAMP/DATETIME – default /auto update = CURRENT_TIMESTAMP
TIMESTAMP – now nullable by default
Improved Partitioning
- up to 8k partitions per table
- Import/export tables to/from partitioned tables
- Explicit partition selection
GIS: Precise spatial operations
and more...
Get it now!
dev.mysql.com/downloads/mysql/
MySQL 5.6: Other Important Enhancements
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
35
Program Agenda
MySQL 5.6 Release Candidate
MySQL Cluster 7.3 Labs Release
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
36
MySQL Cluster 7.2 70x Faster JOINs
Fastest Ramp…Ever
8x Higher Per Node Performance
writes
NoSQL Memcached API
Geo-Distributed Clusters
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
37
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
38
DMR1
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
39
EA
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
40
EA
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
41
MySQL Cluster 7.3 DMR1: Foreign Keys
Brings MySQL Cluster to a broader range of workloads
– Packaged apps, custom projects
Adds powerful functionality while reducing complexity
– App logic & data model
Enabled by default
Enforced for SQL
& NoSQL APIs
On-line add and drop
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
42
MySQL Cluster 7.3 DMR1: Foreign Keys
Implementation goal: compatibility with InnoDB
– Easy migration of solutions already working with InnoDB
Foreign keys are natively implemented in the storage
engine
Created in SQL
Enforced in SQL & NoSQL (C++, ClusterJ, memcached,
node.js)
http://www.clusterdb.com/mysql-cluster/foreign-keys-in-
mysql-cluster/
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
43
MySQL Cluster 7.3 EA: Node.js NoSQL API Native JavaScript access to MySQL Cluster
– End-to-End JavaScript: browser to the app and
database
– Storing and retrieving JavaScript objects directly
in MySQL Cluster
– Eliminate SQL transformation
Implemented as a module for node.js
– Integrates full Cluster API library within the web
app
Couple high performance, distributed apps, with
high performance distributed database
V8 JavaScript Engine
MySQL Cluster Node.js Module
MySQL Cluster Data Nodes
Clients
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
44
Try Node.js example for yourself https://github.com/mysql/mysql-js/tree/master/samples
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
45
NoSQL Access to MySQL Cluster data
ClusterJ
MySQL
JDBC
Apps
JPA
JNI
Python Ruby
ClusterJPA
Apps Apps Apps Apps Apps
Node.js
JSON
Apps
mod-ndb
Apache
Apps
ndb-eng
Memcached
Apps Apps
NDB API (C++)
MySQL Cluster Data Nodes
Apps
PHP PERL
Apps
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
46
MySQL Cluster 7.3 EA: Auto-Installer
Fast configuration
Auto-discovery
Workload optimized
Repeatable best
practices
For MySQL Cluster
7.2 + 7.3
Specify Workload
Auto-Discover
Define Topology Deploy
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
47
MySQL Cluster 7.3 EA: Auto-Installer
Multi-host support
Remote deployments
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
48
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
Management
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
49
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
Management Management
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
50
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Application Layer
Management Management
Clients
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
51
Getting Started
Learn More
Evaluate MySQL Cluster 7.3 Auto-Install a Cluster
Developer Zone dev.mysql.com
Download Today dev.mysql.com/downloads/cluster/
labs.mysql.com
Copyright © 2012, Oracle and/or its affiliates. All rights reserved | 25th October 2012 | Oracle reserves the right to change the content and timing of all future releases .
52
Graphic Section Divider