mysql tech tour 2015 - 5.7 innodb
Post on 12-Feb-2017
349 Views
Preview:
TRANSCRIPT
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |Tuesday, October 20, 2015 Oracle Confidential – Restricted
MySQL 5.7 InnoDB Update
1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Tuesday, October 20, 2015 Oracle Confidential – Restricted 2
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
IntroKey parametersInnoDB 5.6 vs. 5.7InnoDB ‘Always online’GISMySQL Enterprise BackupNews From The Lab!
1
Tuesday, October 20, 2015 Oracle Confidential – Restricted
2
3
4
5
6
7
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Intro1
Tuesday, October 20, 2015 Oracle Confidential – Restricted
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL within OracleMore Investment, More Innovation
Dez 2009 May 2015
5
... 20 Years MySQL… 10 Years InnoDB of Oracle Stewardship … 5 Years MySQL within Oracle
2x Engineering Staff 3x QA Staff 2x Support Staff
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
InnoDB•Default Storage Engine since MySQL 5.5 (previous as ‘Oracle’)•ACID-compliant transactions, MVCC•Row-level locking•Two phase commit•Efficient indexing•Fast DDL operations•Table compression•Automatic crash recovery•Referential integrity•Online backup
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Key parameters2
Tuesday, October 20, 2015 Oracle Confidential – Restricted
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
innodb_file_per_table
• .ibd files• MySQL 5.5:Off, MySQL 5.6:On, MySQL 5.7:On• Easier to reclaim space• TRUNCATE for a table is faster• Can monitor table size on the file system• Can store specific tables on different storage devices• (5.6) Transportable tablespaces• What are you using when you’ve updated from older version?
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
innodb_flush_log_at_trx_commit
• Performance vs. Durability
• Redo logs :– innodb_flush_log_at_trx_commit = 1 // best durability (ACID Compliant), default– innodb_flush_log_at_trx_commit = 2 // better performance (not ACID Compliant)– innodb_flush_log_at_trx_commit = 0 // best performance (not ACID Compliant)
• Linux only :– innodb_flush_method = O_DIRECT // skips the FS cache
9
http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
innoDB buffer pool
• Global • Caches data and indexes• innodb_buffer_pool_size
– 60 – 80% of the total RAM– Not too large: OS should never swap !– Active data <= innodb_buffer_pool_size <= 0.8 * RAM
• Use read only transactions
10
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
innodb_log_file_size
• Size of file on disk• Larger the file, the less checkpoint activity• Size for each log file• Max combined log file size
– MySQL 5.5 <4G – MySQL 5.6 & MySQL 5.7 = 512G (!!!)– innodb_log_file_size = 256M / 512M / 1G (‘often’ 32M default)
• Larger the log file longer recovery time➯
11
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
innodb_log_buffer_size
• Buffer for writing to log files on disk• Large transactions? Write heavy?
– Increasing can help with IO
• Default: 8M (16M)?
12
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Tip!
• Especially for HDD, read and write, use dedicated partitions• Separate :
– datadir– innodb_data_file_path– innodb_undo_directory– innodb_log_group_home_dir– log-bin – tmpdir– backups
Random, SSD
Sequential, spinning
Random, SSD, tmpfs(NFS, ButterFS, ZFS …)
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 14
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
InnoDB 5.6 vs. 5.73
Tuesday, October 20, 2015 Oracle Confidential – Restricted
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Scaling with InnoDB, 5.6 (up to 48 CPUs*) vs. 5.7 (beyond 72 CPUs*) * CPU Cores-HT
18 36 720
100,000
200,000
300,000
400,000
500,000
600,000
700,000
MySQL 5.7: Sysbench – OLTP Read Write
MySQL 5.7
MySQL 5.6
MySQL 5.5
CPU Cores – HT
Que
ries p
er S
econ
d
Intel(R) Xeon(R) CPU E7-8890 v34 sockets x 18 cores-HT (144 CPU threads)2.5 Ghz, 512GB RAMLinux kernel 3.16
MySQL 5.7 Sysbench Benchmark
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Key-value access to InnoDB- Bypasses SQL Parsing
SQL for rich queries, JOINs, FKs, etc.
Implemented via:- Memcached plug-in to mysqld - Memcached mapped to native
InnoDB API- Use existing Memcached clients- Shared process for ultra-low latency
MySQL 5.6 – InnoDB, NoSQL With MemcachedMore Flexibility - Memcached API
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved: MySQL 5.7: InnoDB, NoSQL With MemcachedMySQL 5.7 is 6x faster than MySQL 5.6
8 16 32 64 128 256 512 1,0240
200,000
400,000
600,000
800,000
1,000,000
1,200,000
MySQL 5.7 vs 5.6 - InnoDB & Memcached
MySQL 5.7
MySQL 5.6
Connections
Que
ries p
er S
econ
d
1 Million QPS
18
Intel(R) Xeon(R) CPU E7-4860 x86_644 sockets x 10 cores-HT (80 CPU threads)2.3 GHz, 512 GB RAMOracle Linux 6.5
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
InnoDB Full-Text Index support• MySQL 5.6
– InnoDB Full-Text Index support• Fully ACID compliant, MVCC search• With performance improvements over MyISAM• Easily customizable stop-word lists
• MySQL 5.7– (Significant) better Performance – Pluggable Full-Text Parser support– CJK Support
• N-gram parser for Chinese, Japanese, and Korean• MeCab parser for Japanese
19
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What is it?• Search entire documents
– Character based fields • VARCHAR, TEXT, BLOB
• For a search string – Combinations of words– Phrases: “specific string to match”– Wildcards: * – Requirements: +, -, ~– Expressions: (…)– Relevancy weight characters: <, >
20
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Example
21
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
InnoDB ‘Always online’4
Tuesday, October 20, 2015 Oracle Confidential – Restricted
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 23
MySQL 5.7: InnoDB – Always Online
• Resize the InnoDB Buffer Pool online – Allows DBAs to tune the buffer size without any downtime– Adapt in real-time to changes in database usage patterns
• Separate UNDO tablespace– With automatic online truncation
• Additional Online ALTER TABLE support– Enlarge / instant change VARCHAR, Rename / instant Index
• Dynamic configuration– Making existing settings dynamically configurable– As a design principle for new features & settings
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 24
MySQL 5.7: InnoDB – Example
• Resize the InnoDB Buffer Pool online – Allows DBAs to tune the buffer size without any downtime– Increase or decrease online, Operation is
performed in chunks (default 128MB)– Assign memory to MySQL/Apps during
runtime
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 25
• Native Partitioning– Eliminates previous limitations– Eliminates resource usage problems– Transportable tablespace support
• Native Spatial Indexes• Transparent page compression • Support for 32K and 64K pages
– Use with transparent page compression for very high compression ratios
• General TABLESPACE support – Store multiple tables in user defined shared
tablespaces
• Improved support for cache preloading– Load your hottest data loaded at startup
• Configurable fill-factor – Allows for improvements in storage footprint
• Improved bulk-data load performance
MySQL 5.7: InnoDB Improvements
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 26
MySQL 5.7: InnoDB Improvements
• Enhanced FusionIO integration– Doublewrite buffer disabled when NVMFS/DirectFS detected
• Parallel “dirty page” flushing– Higher throughput, performance, and scalability
• Partitions support for Transportable Tablespaces (TTS)– TTS support for individual partitions
• Improved crash recovery to ensure data safety
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
InnoDB File format– Antelope MySQL 5.1, 5.5, 5.6
vs. Barracuda MySQL 5.7• RAW performance• Lower memory and disk space footprint• Great in combination with SSD
– SHOW TABLE STATUS• Antelope Compact or Redundant• Barracuda Compressed or Dynamic
27
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 28
MySQL 5.7: InnoDB Compression
• Transparent Page Level Compression– Happens transparently in background threads– Managed entirely within the IO layer– Uses sparse file and "hole punching" support in OS kernels and File Systems
• Reduces IO – Improves MySQL performance– Improves storage efficiency– Reduces write cycles, thus increasing SSD lifespan
• Applies to all InnoDB data, including the system tablespace and UNDO logs
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 29
MySQL 5.7: InnoDB Temp Tables
• New separate tablespace for temporary tables– Improved CREATE/DROP performance– DDL changes are transient, which eliminates some disk IO
• Optimize DML operations– No REDO logging, no change buffering, less locking
• New intrinsic temporary tables– Specialized temporary tables with tailored ACID/MVCC semantics– Light weight and ultra-fast, great for intermediate query execution operations
• InnoDB as additional storage engine for temp tables– Switch optimizer from MyISAM to InnoDB (faster) for internal temp tables
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
GIS5
Tuesday, October 20, 2015 Oracle Confidential – Restricted
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7 : GIS Reborn• Dev scrapped much of the old algorithms and related code …
– Replacing it with the FOSS Boost.Geometry library (1.58)
• …and rounded out our OGC spatial function support– Standardized the namespace
• Added native GEOMETRY data type support to InnoDB– We then added SPATIAL INDEX support to InnoDB (using R-tree)
• Added Geohash and GeoJSON support• Added helper functions: ST_Distance_Sphere(), ST_MakeEnvelope(), ST_IsValid(), ST_Validate(), ST_Simplify, …
31
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Spatial Indexes for InnoDB• R-tree based
– Full transactional support– Predicate locking to prevent phantoms– Records contain minimum bounding box
• Small and compact
– Currently only supports 2D data• We would like to add 3D support in the future
– Supports historical spatial index DDL syntax
32
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Additional Features• GeoHash
– B-tree indexes on the generated hash values– Quick lookups for exact matches– Not very accurate for proximity searches
• GeoJSON• Additional functions
– ST_IsValid(), ST_Simplify(), ST_Buffer() …– ST_Distance_Sphere()
• Limited SRID support– Laying the groundwork for CRS support
33
{ "type": "Feature", "geometry": { "type": "Point", "coordinates": [125.6, 10.1] }, "properties": { "name": "Dinagat Islands" }}
GeoJSON Example
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Replaced custom code– For spatial calculations– For spatial analysis
• Provides OGC compliance– With improved performance
• Boost.Geometry contains– Field and domain experts – Bustling and robust communit
MySQL 5.7: GIS - Integrating Boost.Geometry
34
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Backup6
Tuesday, October 20, 2015 Oracle Confidential – Restricted
MySQL Enterprise Backup now writes tape information
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Backup 4.0
• Online, non-locking backup and recovery – Complete MySQL instance backup (data and config)– Partial backup and restore
• Direct Cloud storage backups via Swift API• Incremental backups & Point-in-time recovery• Advanced compressed and encryption• InnoDB tables with page sizes of 32 and 64K can be backed up and restored.• Optimistic backups• Support for MySQL 5.7
– General Tablespaces
New & Improved
36
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• MySQL Enterprise Backup images to tape (like RMN)• Via SBT interface (standard for MMS)• Encryption, policy-based management, vault rotation• Common management, SOP for Oracle, MySQL backups• Improved SBT Backups to MMS Systems • Includes Backup/tape information
Oracle Integrations: Secure Backup
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
News From The Lab!7
Tuesday, October 20, 2015 Oracle Confidential – Restricted
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication Plugin
Tuesday, October 20, 2015
M M M M M
Replication Group
• Multi-master update everywhere• Automatic group membership
management and failure detection.• No need for server fail-over.• Automatic reconfiguration.• No single point of failure.• Shared-nothing state machine
replication.• InnoDB compliant. Off-the-shelf
hardware friendly.
Oracle Confidential – Restricted 39
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication Plugin• Great technology for deployments where elasticity is a requirement, such
as cloud based infrastructures.– Integrated with the server core through a well defined API.– It is integrated with GTIDs, row based replication.– It is integrated with performance schema tables.– It is elastic and self-healing: adding or removing servers does not need human
intervention.
Tuesday, October 20, 2015 Oracle Confidential – Restricted 40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Strong development cycles and continuous community engagement through regular and frequent releases on labs.mysql.com.
MySQL Group Replication
Tuesday, October 20, 2015 Oracle Confidential – Restricted 41
2014-Sep-29Labs release: 0.2.0
2015-Apr-06Labs release: 0.3.0
2014-Aug-06Labs release: 0.4.0
2015-Sep-14Labs release: 0.5.0
Introduces standalone plugin,with its own release cycle!
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Dependable and Scalable MySQL
Tuesday, October 20, 2015 Oracle Confidential – Restricted 42
Global Data Shard 1 Shard 2
MySQL Fabric Controller
SQL Queries
Server/Shard State & Mapping
Global Group HA Group
Coordinationand Control
HA Group
Group Replication cluster
Group Replication cluster
Group Replication cluster
MySQLRouter
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Dependable and Scalable MySQL Setups
Tuesday, October 20, 2015 Oracle Confidential – Restricted
MApp
M M
MySQL Fabric (Orchestration)
MApp M M
Simple Shard Mapping, State and Extra metadata.
Control, Coordinate, Provision
...
Monitoring (MEM)
MySQL Router Group Replication – Shard 1
Group Replication – Shard NC, PHP, Perl, ...
43
MySQL Router
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 44
top related