bgoug15: json support in mysql 5.7

JSON Supportin MySQL 5.7

Georgi “Joro” KodinovTeam lead, MySQL server general team

Agenda

The new JSON data type

Inlined JSON path expressions

The new JSON functions

Indexing JSON data

CREATE TABLE employees (data JSON);INSERT INTO employees VALUES ('{"id": 1, "name": "Jane"}'), ('{"id": 2, "name": "Joe"}');SELECT * FROM employees;+-------------------------------------+| data |+-------------------------------------+| {"id": 1, "name": "Jane"} || {"id": 2, "name": "Joe"} |+-------------------------------------+

• Validation on INSERT

• No reparsing on SELECT

• Dictionary of fields

• Fields are sorted

• Can compare JSON/SQL

• Can convert JSON/SQL

• Supports all native JSON datatypes

• Also supports date, time, timestamp etc.

The New JSON Datatype

JSON vs TEXT columnsPros Cons

JSON• Validate once• Fast access• Can update in-place

• Slower to insert• Unreadable as is• Sets certain limitations on JSON

TEXT• Fast to insert• Human readable

• Requires manual validation• Requires manual parsing• Harder to update

Beware: SQL vs JSON comparisons !SQL JSON

create table t1 (data json); create table t2 ( id integer, data varchar(20));

insert into t1 values ('{ "id": 1, "data": "1" }'), ('{ "id": 2, "data": "3" }');

insert into t2 values (1, '1'), (2, '3');

select count(*) from t1 where data->'$.id' = data->'$.data';

select count(*) from t2 where id = data;

0 rows ! 1 row !

Agenda

Indexing JSON data

Inlined JSON Path Expressions• <field>->'<JSON path expression>'

e.g. data->'$.some.key[3].from.doc'• Syntax sugar over JSON_EXTRACT function• SELECT * FROM employees WHERE data->'$.id'= 2;• ALTER … ADD COLUMN id INT AS (data->'$.id') …• CREATE VIEW .. AS SELECT data->'$.id', data->'$.name' FROM …• UPDATE employees SET data->'$.name'=‘John' WHERE … Not

Limitations of Inlined JSON Path Expressions

Inlined JSON path JSON_EXTRACT()

Data source Field Any JSON value

Path expression SQL Constant SQL Expression

# of expressions One Multiple

Supported JSON Paths[[[database.]table.]column]$<path spec>

Expression Example[ [ [database.] table.] field]$ db.phonebook.data$

$ Current document’s root$.identifier $.user.address.street[array] $.user.addresses[2].street.* and [*] $.user.addresses[*].street** $.user**.phone

Not yet!

Agenda

Indexing JSON data

New functions to handle JSON data

Example ResultSELECT JSON_VALID('{ "a":1 }'); 1SELECT JSON_TYPE('[ 1, 2, 3 ]'); ARRAYSELECT JSON_KEYS('{ "a":1, "b": 2 }'); ["a", "b"]SELECT JSON_LENGTH('[ 1, 2, 3 ]'); 3SELECT JSON_DEPTH('{ "a":{ "c": 1 }, "b": 2 }'); 3

Information

Example ResultSELECT JSON_REMOVE('{ "a":1, "b": 2 }', '$.a'); {"b": 2}SELECT JSON_ARRAY_APPEND('[1,[2,3],4]', '$[1]', 5); [1, [2, 3, 5], 4]SELECT JSON_SET('{ "a":1 }', '$.c', 3); {"a": 1, “c": 3}SELECT JSON_INSERT('{ "a":1 }', '$.b', 4); {"a": 1, "b": 4}SELECT JSON_REPLACE('{ "a":1, "b": 2 }', '$.b', 3); {"a": 1, "b": 3}SELECT JSON_MERGE('{ "a": 1 }', '{"b":2}'); {"a": 1, "b": 2}SELECT JSON_UNQUOTE('"abc"'); abc

Modification

Example ResultSELECT JSON_ARRAY(1, '2', null, true); [1, "2", null, true]SELECT JSON_OBJECT(1, 2, '3', true); {"1": 2, "3": true}SELECT JSON_QUOTE('"null"'); "\"null\""

Create JSON objects

Example ResultSELECT JSON_CONTAINS_PATH( '{ "a":{ "c": 1 }, "b": 2 }', 'one', '$.a.c');

SELECT JSON_CONTAINS( '{"a": 1, "b": "2" }', '1', '$.a'); 1SELECT JSON_EXTRACT('{"a": 1, "n": { "b": 2}}', '$.n'); {"b": 2}SELECT JSON_SEARCH( '{"a": "1", "b": "2" }', 'one', 2); "$.b"

Search in JSON data

New functions to handle JSON dataFurther reading: http://dev.mysql.com/doc/refman/5.7/en/

Agenda

Indexing JSON data

Indexing JSON Data• Use Functional Indexes– Both STORED and VIRTUAL types are supported

• Examples:– CREATE TABLE t1 (

data JSON, id INTEGER AS (JSON_EXTRACT(data,"$.id")) STORED, PRIMARY KEY(id));– CREATE TABLE t2 (

data JSON, id INTEGER AS (JSON_EXTRACT(data,"$.id")) VIRTUAL, KEY(id));

Indexing JSON: STORED vs VIRTUAL columnsPros Cons

STORED• Can be primary key too• All index types supported• Looks like a normal field

• Slow ALTER TABLE• Takes space on disk

VIRTUAL• Instant ALTER TABLE• Faster INSERT• Looks like a normal field

• Secondary key only • BTREE index only

Or this way ….

Or maybe this way ?

ALTER TABLE features ADD feature_type VARCHAR(30) AS (feature->"$.type") VIRTUAL;Query OK, 0 rows affected (0.01 sec)Records: 0 Duplicates: 0 Warnings: 0

ALTER TABLE features ADD INDEX (feature_type);Query OK, 0 rows affected (0.73 sec)Records: 0 Duplicates: 0 Warnings: 0

SELECT DISTINCT feature_type FROM features;+-------------------+| feature_type |+-------------------+| "Feature" |+-------------------+1 row in set (0.06 sec)

From table scan on 206K documents to index scan on 206K materialized values

Down from 1.25s !

Meta data change only (FAST). Does not need to touch table.

Online CREATE INDEX !No rows were

modified.

Roadmap• Online alter for virtual columns• Advanced JSON functions• In-place update of JSON/BLOB• Full text and GIS index on virtual columns• Improved performance through condition pushdown

Questions ? @gkodinov, georgi.kodinov@oracle.com if you forget !

Safe Harbor StatementThe preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

bgoug15: json support in mysql 5.7

Software

pemanfaatan json dengan mysql

mysqlのnosql機能 - mysql json & http plugin for mysql

practical json in mysql 57 and beyond pl 2017 - percona ·...

using json with mariadb and mysql

mysql 5.7 + json · copyright © 2014 oracle and/or its...

mysql-performance schema- what's new in mysql-5.7 dmrs

mysql 5.7 reference manual - including mysql cluster ndb 7.5

nosql and mysql: news about json

mysql 5.7 json und gis - fromdual.com€¦ · mysql 5.7...

mysql 5.7 - nosql - json, protocol x, document store /...

mysql 5.7: json und gis - fromdual 1 / 34 mysql 5.7: json...

practical json in mysql 5 - percona€¦ · practical json...

mysql 5.7 & json - nouvelles opportunités pour les...

mysql 5.7 + java

mysql replication: what’s new in mysql 5.7 and beyond

mysql performance tuning. part 1: mysql configuration...

mysql 5.7 string functions

mysql json functions

"Новые возможности mysql 5.7"

mysql 5.7 replication news