MySQL’s JSON Data Type

Dave StokesMySQL Community ManagerDavid.Stokes@Oracle.com

@StokerSlides -- slideshare.net/davidmstokes

Intro

MySQL 5.7 has many new features including a native JSON data type. Before you could store JSON data in a character field … but it was not efficient, or sexy.

Having a native JSON data type provides many exciting possibilities

2

JSONJSON, (JavaScript Object Notation), is an open standard

format that uses human-readable text to transmit data objects

consisting of attribute–value pairs. It is the primary data format

used for asynchronous browser/server communication (AJAJ),

largely replacing XML (used by AJAX).

Although originally derived from the JavaScript scripting

language, JSON is a language-independent data format. Code

for parsing and generating JSON data is readily available in

many programming languages.

https://en.wikipedia.org/wiki/JSON

3

Jason and the Argonauts is a 1963 British Columbia Pictures fantasy film starring Todd Armstrong as the titular mythical Greek hero in a story about his quest for the Golden Fleece. Directed by Don Chaffey in collaboration with stop motion animation expert Ray Harryhausen, the film is noted for its

stop-motion creatures, and particularly the iconic fight with the skeletons.

https://en.wikipedia.org/wiki/Jason_and_the_Argonauts_(1963_film) 4

What Does JSON Look Like

{

"id": 1,

"name": "A green door",

"price": 12.50,

"tags": ["home", "green"]

}

http://json-schema.org/example1.html

5

How DoesIt Work?Very Well Thank You!

6

There is more than one way to skin a cat. How many skinless cats do we really need?

7

JSON in a CHAR or VARCHAR columnmysql>CREATE TABLE example1 (id INTEGER, data CHAR(255);

mysql>INSERT INTO example1 (id, data) VALUES (1, ‘{“foo” : “bar”}’);

8

JSON in a CHAR or VARCHAR columnmysql>CREATE TABLE example1 (id INTEGER, data CHAR(255);

mysql>INSERT INTO example1 (id, data) VALUES (1, ‘{“foo” : “bar”}’);

Not easy to dig into JSON data stored as a string

SELECT foo FROM table WHERE field REGEXP ‘“key” : “[[:<:]]value[[:>:]]”’

Slow -- Breaking first rule of normalization of data

Slow -- Full table scans (reading the entire file to get to your data)

REGEX -- Ughhhhh!!

9

MySQL 5.7 has an optimized way to store JSON data that allows access to keys/values

10

JSON in JSON column● Automatic validation of JSON documents stored in

JSON columns. Invalid documents produce an error.● Optimized storage format. JSON documents stored in

JSON columns are converted to an internal format that permits quick read access to document elements.

● The binary format is structured to enable the server to look up subobjects or nested values directly by key or array index without reading all values before or after them in the document.

11

JSON in JSON column limitsThe size of JSON documents stored in JSON columns is limited to the value of the max_allowed_packet system variable. (While the server manipulates a JSON value internally in memory, it can be larger; the limit applies when the server stores it.)

JSON columns cannot have a default value.

JSON columns cannot be indexed.

You can work around this restriction by creating an index on a generated column that extracts a scalar value from the JSON column.

12

Using a JSON column

mysql> CREATE TABLE t1 (jdoc JSON);

Query OK, 0 rows affected (0.20 sec)

mysql> INSERT INTO t1 VALUES('{"key1":

"value1", "key2": "value2"}');

Query OK, 1 row affected (0.01 sec)

13

Example

mysql> SELECT * FROM t1;

{"key1": "value1", "key2": "value2"}

14

Efficiency JSON versus CHAR

http://mablomy.blogspot.com/2015/11/json-memory-consumption.

html?m=1

The JSON data type consumes more memory - Roughty 4.5%

15

Normalization of JSON DataWhen a string is parsed and found to be a valid JSON document, it is also normalized: Members with keys that duplicate a key found earlier in the document are discarded (even if the values differ). The object value produced by the followingJSON_OBJECT() call does not include the second key1 element because that key name occurs earlier in the value:

mysql> SELECT JSON_OBJECT('key1', 1, 'key2', 'abc', 'key1', 'def');

+------------------------------------------------------+

| JSON_OBJECT('key1', 1, 'key2', 'abc', 'key1', 'def') |

+------------------------------------------------------+

| {"key1": 1, "key2": "abc"} |

+------------------------------------------------------+

16

Normalization also sorts keysThe normalization performed by MySQL also sorts the keys of a JSON object (for the purpose of making lookups more efficient). The result of this ordering is subject to change and not guaranteed to be consistent across releases. In addition, extra white space between keys, values, or elements in the original document is discarded.

17

Searching JSONA JSON path expression selects a value within a JSON document.

Path expressions are useful with functions that extract parts of or modify a JSON document, to specify where within

that document to operate. For example, the following query extracts from a JSON document the value of the member

with the name key:

mysql> SELECT JSON_EXTRACT('{"id": 14, "name": "Aztalan"}', '$.name');

+---------------------------------------------------------+

| JSON_EXTRACT('{"id": 14, "name": "Aztalan"}', '$.name') |

+---------------------------------------------------------+

| "Aztalan" |

+---------------------------------------------------------+

18

Path SyntaxPath syntax uses a leading $ character to represent the JSON document under consideration, optionally followed by selectors that indicate successively more specific parts of the document:

● A period followed by a key name names the member in an object with the given key. The key name must be specified within double quotation marks if the name without quotes is not legal within path expressions (for example, if it contains a space).

● [N] appended to a path that selects an array names the value at position N within the array. Array positions are integers beginning with zero.

● Paths can contain * or ** wildcards:○ .[*] evaluates to the values of all members in a JSON object.○ [*] evaluates to the values of all elements in a JSON array.○ prefix**suffix evaluates to all paths that begin with the named prefix and end with

the named suffix.● A path that does not exist in the document (evaluates to nonexistent data) evaluates to NULL.

19

Example with an arrayLet $ refer to this JSON array with three elements:

[3, {"a": [5, 6], "b": 10}, [99, 100]]

Then:

● $[0] evaluates to 3.● $[1] evaluates to {"a": [5, 6], "b": 10}.● $[2] evaluates to [99, 100].● $[3] evaluates to NULL (it refers to the fourth array element, which does

not exist).

20

[3, {"a": [5, 6], "b": 10}, [99, 100]]

Because $[1] and $[2] evaluate to nonscalar values, they can be used as the basis for more-specific path expressions that select nested values. Examples:

● $[1].a evaluates to [5, 6].● $[1].a[1] evaluates to 6.● $[1].b evaluates to 10.● $[2][0] evaluates to 99.

21

Paths quotedAs mentioned previously, path components that name keys must be quoted if the unquoted key name is not legal in path expressions. Let $ refer to this value:

{"a fish": "shark", "a bird": "sparrow"}

The keys both contain a space and must be quoted:● $."a fish" evaluates to shark.● $."a bird" evaluates to sparrow.

22

Paths with wildcardsPaths that use wildcards evaluate to an array that can contain multiple values:

mysql> SELECT JSON_EXTRACT('{"a": 1, "b": 2, "c": [3, 4, 5]}', '$.*');

+---------------------------------------------------------+

| JSON_EXTRACT('{"a": 1, "b": 2, "c": [3, 4, 5]}', '$.*') |

+---------------------------------------------------------+

| [1, 2, [3, 4, 5]] |

+---------------------------------------------------------+

mysql> SELECT JSON_EXTRACT('{"a": 1, "b": 2, "c": [3, 4, 5]}', '$.c[*]');

+------------------------------------------------------------+

| JSON_EXTRACT('{"a": 1, "b": 2, "c": [3, 4, 5]}', '$.c[*]') |

+------------------------------------------------------------+

| [3, 4, 5] |

+------------------------------------------------------------+23

column->pathIn MySQL 5.7.9 and later, you can use column->path with a JSON column identifier and JSON path expression as a synonym for JSON_EXTRACT(column, path)

24

Comparison and OrderingJSON values can be compared using the =, <, <=, >, >=, <>, !=, and <=> operators.The following comparison operators and functions are not yet supported with JSON values:

● BETWEEN● IN()● GREATEST()● LEAST()

See the manual for details on casting and ordering details and conversion orders25

ORDER BY and GROUP BYORDER BY and GROUP BY for JSON values works according to these principles:

● Ordering of scalar JSON values uses the same rules as in the preceding discussion.

● For ascending sorts, SQL NULL orders before all JSON values, including the JSON null literal; for descending sorts, SQL NULL orders after all JSON values, including the JSON null literal.

● Sort keys for JSON values are bound by the value of the max_sort_length system variable, so keys that differ only after the first max_sort_length bytes compare as equal.

● Sorting of nonscalar values is not currently supported and a warning occurs.

26

EXAMPLE with castA column named jdoc contains JSON objects having a member consisting of an id key and a nonnegative value, use this expression to sort by id values:

ORDER BY CAST(JSON_EXTRACT(jdoc, '$.id') AS UNSIGNED)

If there happens to be a generated column defined to use the same expression as in the ORDER BY, the MySQL optimizer recognizes that and considers using the index for the query execution plan.

27

FunctionsHow to get to stuff in JSON

28

Functions - Create, Search, etceteraJSON_APPEND()Append data to JSON documentJSON_ARRAY_APPEND()Append data to JSON documentJSON_ARRAY_INSERT()Insert into JSON arrayJSON_ARRAY()Create JSON array->Return value from JSON column after evaluating pathJSON_CONTAINS_PATH()Whether JSON document contains any data at pathJSON_CONTAINS()Whether JSON document contains specific object at pathJSON_DEPTH()Maximum depth of JSON documentJSON_EXTRACT()Return data from JSON documentJSON_INSERT()Insert data into JSON documentJSON_KEYS()Array of keys from JSON documentJSON_LENGTH()Number of elements in JSON documentJSON_MERGE()Merge JSON documentsJSON_OBJECT()Create JSON objectJSON_QUOTE()Quote JSON documentJSON_REMOVE()Remove data from JSON documentJSON_REPLACE()Replace values in JSON documentJSON_SEARCH()Path to value within JSON documentJSON_SET()Insert data into JSON documentJSON_TYPE()Type of JSON valueJSON_UNQUOTE()Unquote JSON valueJSON_VALID()Whether JSON value is valid 29

Functions - Create, Search, etceteraJSON_APPEND()Append data to JSON documentJSON_ARRAY_APPEND()Append data to JSON documentJSON_ARRAY_INSERT()Insert into JSON arrayJSON_ARRAY()Create JSON array->Return value from JSON column after evaluating pathJSON_CONTAINS_PATH()Whether JSON document contains any data at pathJSON_CONTAINS()Whether JSON document contains specific object at pathJSON_DEPTH()Maximum depth of JSON documentJSON_EXTRACT()Return data from JSON documentJSON_INSERT()Insert data into JSON documentJSON_KEYS()Array of keys from JSON documentJSON_LENGTH()Number of elements in JSON documentJSON_MERGE()Merge JSON documentsJSON_OBJECT()Create JSON objectJSON_QUOTE()Quote JSON documentJSON_REMOVE()Remove data from JSON documentJSON_REPLACE()Replace values in JSON documentJSON_SEARCH()Path to value within JSON documentJSON_SET()Insert data into JSON documentJSON_TYPE()Type of JSON valueJSON_UNQUOTE()Unquote JSON valueJSON_VALID()Whether JSON value is valid 30

Create

Functions - Create, Search, etceteraJSON_APPEND()Append data to JSON documentJSON_ARRAY_APPEND()Append data to JSON documentJSON_ARRAY_INSERT()Insert into JSON arrayJSON_ARRAY()Create JSON array->Return value from JSON column after evaluating pathJSON_CONTAINS_PATH()Whether JSON document contains any data at pathJSON_CONTAINS()Whether JSON document contains specific object at pathJSON_DEPTH()Maximum depth of JSON documentJSON_EXTRACT()Return data from JSON documentJSON_INSERT()Insert data into JSON documentJSON_KEYS()Array of keys from JSON documentJSON_LENGTH()Number of elements in JSON documentJSON_MERGE()Merge JSON documentsJSON_OBJECT()Create JSON objectJSON_QUOTE()Quote JSON documentJSON_REMOVE()Remove data from JSON documentJSON_REPLACE()Replace values in JSON documentJSON_SEARCH()Path to value within JSON documentJSON_SET()Insert data into JSON documentJSON_TYPE()Type of JSON valueJSON_UNQUOTE()Unquote JSON valueJSON_VALID()Whether JSON value is valid 31

Search

Functions - Create, Search, etceteraJSON_APPEND()Append data to JSON documentJSON_ARRAY_APPEND()Append data to JSON documentJSON_ARRAY_INSERT()Insert into JSON arrayJSON_ARRAY()Create JSON array->Return value from JSON column after evaluating pathJSON_CONTAINS_PATH()Whether JSON document contains any data at pathJSON_CONTAINS()Whether JSON document contains specific object at pathJSON_DEPTH()Maximum depth of JSON documentJSON_EXTRACT()Return data from JSON documentJSON_INSERT()Insert data into JSON documentJSON_KEYS()Array of keys from JSON documentJSON_LENGTH()Number of elements in JSON documentJSON_MERGE()Merge JSON documentsJSON_OBJECT()Create JSON objectJSON_QUOTE()Quote JSON documentJSON_REMOVE()Remove data from JSON documentJSON_REPLACE()Replace values in JSON documentJSON_SEARCH()Path to value within JSON documentJSON_SET()Insert data into JSON documentJSON_TYPE()Type of JSON valueJSON_UNQUOTE()Unquote JSON valueJSON_VALID()Whether JSON value is valid 32

Modify

Functions - Create, Search, etceteraJSON_APPEND()Append data to JSON documentJSON_ARRAY_APPEND()Append data to JSON documentJSON_ARRAY_INSERT()Insert into JSON arrayJSON_ARRAY()Create JSON array->Return value from JSON column after evaluating pathJSON_CONTAINS_PATH()Whether JSON document contains any data at pathJSON_CONTAINS()Whether JSON document contains specific object at pathJSON_DEPTH()Maximum depth of JSON documentJSON_EXTRACT()Return data from JSON documentJSON_INSERT()Insert data into JSON documentJSON_KEYS()Array of keys from JSON documentJSON_LENGTH()Number of elements in JSON documentJSON_MERGE()Merge JSON documentsJSON_OBJECT()Create JSON objectJSON_QUOTE()Quote JSON documentJSON_REMOVE()Remove data from JSON documentJSON_REPLACE()Replace values in JSON documentJSON_SEARCH()Path to value within JSON documentJSON_SET()Insert data into JSON documentJSON_TYPE()Type of JSON valueJSON_UNQUOTE()Unquote JSON valueJSON_VALID()Whether JSON value is valid 33

Return JSONValue Attributes

Functions to create JSON valuesThe functions in this section compose JSON values from component elements.

● JSON_ARRAY([val[, val] ...])● Evaluates a (possibly empty) list of values and returns a JSON array containing those values.● mysql> SELECT JSON_ARRAY(1, "abc", NULL, TRUE, CURTIME());

+---------------------------------------------+| JSON_ARRAY(1, "abc", NULL, TRUE, CURTIME()) |+---------------------------------------------+| [1, "abc", null, true, "11:30:24.000000"] |+---------------------------------------------+

● JSON_OBJECT([key, val[, key, val] ...])● Evaluates a (possibly empty) list of key/value pairs and returns a JSON object containing those pairs. An

error occurs if any key name is NULL or the number of arguments is odd.● mysql> SELECT JSON_OBJECT('id', 87, 'name', 'carrot');

+-----------------------------------------+| JSON_OBJECT('id', 87, 'name', 'carrot') |+-----------------------------------------+| {"id": 87, "name": "carrot"} |+-----------------------------------------+ 34

Functions to create JSON values cont.● JSON_QUOTE(json_val)● Quotes a string as a JSON value by wrapping it with double quote characters and escaping interior quote

and other characters, then returning the result as a utf8mb4 string. Returns NULL if the argument is NULL.● This function is typically used to produce a valid JSON string literal for inclusion within a JSON document.● Certain special characters are escaped with backslashes.● mysql> SELECT JSON_QUOTE('null'), JSON_QUOTE('"null"');

+--------------------+----------------------+| JSON_QUOTE('null') | JSON_QUOTE('"null"') |+--------------------+----------------------+| "null" | "\"null\"" |+--------------------+----------------------+mysql> SELECT JSON_QUOTE('[1, 2, 3]');+-------------------------+| JSON_QUOTE('[1, 2, 3]') |+-------------------------+| "[1, 2, 3]" |+-------------------------+

You can also obtain JSON values by casting values of other types to the JSON type using CAST(value AS JSON); 35

Functions to search JSON data JSON_CONTAINS(json_doc, val[, path])

Returns 0 or 1 to indicate whether a specific value is contained in a target JSON document, or, if a path argument is given, at a specific path within the target document. Returns NULL if any argument is NULL or the path argument does not identify a section of the target document. An error occurs if either document argument is not a valid JSON document or the path argument is not a valid path expression or contains a* or ** wildcard.

To check only whether any data exists at the path, use JSON_CONTAINS_PATH() instead.

36

Functions to search JSON data JSON_CONTAINS_PATH(json_doc, one_or_all, path[, path] ...)

Returns 0 or 1 to indicate whether a JSON document contains data at a given path or paths. Returns NULL if any argument is NULL. An error occurs if the json_doc argument is not a valid JSON document, any path argument is not a valid path expression, or one_or_all is not'one' or 'all'.

To check for a specific value at a path, use JSON_CONTAINS() instead.

The return value is 0 if no specified path exists within the document. Otherwise, the return value depends on the one_or_all argument:

○ 'one': 1 if at least one path exists within the document, 0 otherwise.○ 'all': 1 if all paths exist within the document, 0 otherwise.

37

Functions to search JSON data JSON_EXTRACT(json_doc, path[, path] ...)

Returns data from a JSON document, selected from the parts of the document matched by the path arguments. Returns NULL if any argument is NULL or no paths locate a value in the document. An error occurs if the json_doc argument is not a valid JSON document or anypath argument is not a valid path expression.

The return value consists of all values matched by the path arguments. If it is possible that those arguments could return multiple values, the matched values are autowrapped as an array, in the order corresponding to the paths that produced them. Otherwise, the return value is the single matched value.

38

** SHORT CUT * Reminder column->path

In MySQL 5.7.9 and later, the -> operator serves as an alias for the JSON_EXTRACT() function when used with two arguments, a column identiefer on the left and a JSON path on the right that is evaluated against the JSON document (the column value). You can use such expressions in place of column identifiers wherever they occur in SQL statements.

mysql> UPDATE jemp SET n=1 WHERE c->"$.id" = "4";

Query OK, 1 row affected (0.04 sec)

Rows matched: 1 Changed: 1 Warnings: 0

39

Functions to search JSON data JSON_KEYS(json_doc[, path])

Returns the keys from the top-level value of a JSON object as a JSON array, or, if a path argument is given, the top-level keys from the selected path. Returns NULL if any argument is NULL, the json_doc argument is not an object, or path, if given, does not locate an object. An error occurs if the json_doc argument is not a valid JSON document or the path argument is not a valid path expression or contains a * or** wildcard.

The result array is empty if the selected object is empty. If the top-level value has nested subobjects, the return value does not include keys from those subobjects.

40

Functions to search JSON dataJSON_SEARCH(json_doc, one_or_all, search_str[, escape_char[, path] ...])

Returns the path to the given string within a JSON document. Returns NULL if any of the json_doc, search_str, or path arguments areNULL; no path exists within the document; or search_str is not found. An error occurs if the json_doc argument is not a valid JSON document, any path argument is not a valid path expression, one_or_all is not 'one' or 'all', or escape_char is not a constant expression.

The one_or_all argument affects the search as follows:

○ 'one': The search terminates after the first match and returns one path string. It is undefined which match is considered first.

○ 'all': The search returns all matching path strings such that no duplicate paths are included. If there are multiple strings, they are autowrapped as an array. The order of the array elements is undefined.

Within the search_str search string argument, the % and _ characters work as for the LIKE operator: % matches any number of characters (including zero characters), and _ matches exactly one character.

To specify a literal % or _ character in the search string, precede it by the escape character. The default is \ if the escape_char argument is missing or NULL. Otherwise, escape_char must be a constant that is empty or one character.

41

Functions that modify JSON data JSON_ARRAY_APPEND( json_doc, path, val[, path, val] ...)

● Appends values to the end of the indicated arrays within a JSON document and returns the result. Returns NULL if any argument is NULL. An error occurs if the json_doc argument is not a valid JSON document or any path argument is not a valid path expression or contains a * or** wildcard.

● The path/value pairs are evaluated left to right. The document produced by evaluating one pair becomes the new value against which the next pair is evaluated.

● If a path selects a scalar or object value, that value is autowrapped within an array and the new value is added to that array. Pairs for which the path does not identify any value in the JSON document are ignored.

42

Functions that modify JSON data JSON_ARRAY_INSERT( json_doc, path, val[, path, val] ...)

Updates a JSON document, inserting into an array within the document and returning the modified document. Returns NULL if any argument is NULL. An error occurs if the json_doc argument is not a valid JSON document or any path argument is not a valid path expression or contains a * or ** wildcard or does not end with an array element identifier.

The path/value pairs are evaluated left to right. The document produced by evaluating one pair becomes the new value against which the next pair is evaluated.

Pairs for which the path does not identify any array in the JSON document are ignored. If a path identifies an array element, the corresponding value is inserted at that element position, shifting any following values to the right. If a path identifies an array position past the end of an array, the value is inserted at the end of the array.

43

Functions that modify JSON data JSON_INSERT(json_doc, path, val[, path, val] ...)

Inserts data into a JSON document and returns the result. Returns NULL if any argument is NULL. An error occurs if the json_doc argument is not a valid JSON document or any path argument is not a valid path expression or contains a * or ** wildcard.

The path/value pairs are evaluated left to right. The document produced by evaluating one pair becomes the new value against which the next pair is evaluated.

A path/value pair for an existing path in the document is ignored and does not overwrite the existing document value. A path/value pair for a nonexisting path in the document adds the value to the document if the path identifies one of these types of values:

○ A member not present in an existing object. The member is added to the object and associated with the new value.

○ A position past the end of an existing array. The array is extended with the new value. If the existing value is not an array, it is autowrapped as an array, then extended with the new value.

Otherwise, a path/value pair for a nonexisting path in the document is ignored and has no effect. 44

Functions that modify JSON data JSON_MERGE(json_doc, json_doc[, json_doc] ...)

Merges two or more JSON documents and returns the merged result. Returns NULL if any argument is NULL. An error occurs if any argument is not a valid JSON document.

Merging takes place according to the following rules. For additional information, see Normalization, Merging, and Autowrapping of JSON Values.

○ Adjacent arrays are merged to a single array.○ Adjacent objects are merged to a single object.○ A scalar value is autowrapped as an array and merged as an array.○ An adjacent array and object are merged by autowrapping the object as an

array and merging the two arrays45

Functions that modify JSON data JSON_REMOVE(json_doc, path[, path] ...)

Removes data from a JSON document and returns the result. Returns NULL if any argument is NULL. An error occurs if the json_docargument is not a valid JSON document or any path argument is not a valid path expression or is $ or contains a * or ** wildcard.

The path arguments are evaluated left to right. The document produced by evaluating one path becomes the new value against which the next path is evaluated.

It is not an error if the element to be removed does not exist in the document; in that case, the path does not affect the document.

46

Functions that modify JSON data JSON_REPLACE(json_doc, path, val[, path, val] ...)

Replaces existing values in a JSON document and returns the result. Returns NULL if any argument is NULL. An error occurs if the json_docargument is not a valid JSON document or any path argument is not a valid path expression or contains a * or ** wildcard.

The path/value pairs are evaluated left to right. The document produced by evaluating one pair becomes the new value against which the next pair is evaluated.

A path/value pair for an existing path in the document overwrites the existing document value with the new value. A path/value pair for a nonexisting path in the document is ignored and has no effect.

47

Functions that modify JSON dataJSON_SET(json_doc, path, val[, path, val] ...)

Inserts or updates data in a JSON document and returns the result. Returns NULL if any argument is NULL or path, if given, does not locate an object. An error occurs if the json_doc argument is not a valid JSON document or the path argument is not a valid path expression or contains a * or ** wildcard.

The path/value pairs are evaluated left to right. The document produced by evaluating one pair becomes the new value against which the next pair is evaluated.

A path/value pair for an existing path in the document overwrites the existing document value with the new value. A path/value pair for a nonexisting path in the document adds the value to the document if the path identifies one of these types of values:

○ A member not present in an existing object. The member is added to the object and associated with the new value.

○ A position past the end of an existing array. The array is extended with the new value. If the existing value is not an array, it is autowrapped as an array, then extended with the new value.

Otherwise, a path/value pair for a nonexisting path in the document is ignored and has no effect.

48

Functions that modify JSON dataThe JSON_SET(), JSON_INSERT(), and JSON_REPLACE() functions are related:

○ JSON_SET() replaces existing values and adds nonexisting values.

○ JSON_INSERT() inserts values without replacing existing values.

○ JSON_REPLACE() replaces only existing values.

49

Functions that modify JSON dataJSON_UNQUOTE(val)

Unquotes JSON value and returns the result as a utf8mb4 string. Returns NULL if the argument is NULL. An error occurs if the value starts and ends with double quotes but is not a valid JSON string literal.

Within a string, certain sequences have special meaning unless the NO_BACKSLASH_ESCAPES SQL mode is enabled. Each of these sequences begins with a backslash (\), known as the escape character. For all other escape sequences, backslash is ignored. That is, the escaped character is interpreted as if it was not escaped. For example, \x is just x. These sequences are case sensitive. For example, \b is interpreted as a backspace, but \B is interpreted as B.

50

Functions that modify JSON dataEscape Sequence Character Represented by Sequence

\" A double quote (") character

\b A backspace character

\f A formfeed character

\n A newline (linefeed) character

\r A carriage return character

\t A tab character

\\ A backslash (\) character

\uXXXX UTF-8 bytes for Unicode value XXXX

51

Functions to return JSON value attributesJSON_DEPTH(json_doc)

Returns the maximum depth of a JSON document. Returns NULL if the argument is NULL. An error occurs if the argument is not a valid JSON document.

An empty array, empty object, or scalar value has depth 1. A nonempty array containing only elements of depth 1 or nonempty object containing only member values of depth 1 has depth 2. Otherwise, a JSON document has depth greater than 2.

52

Functions to return JSON value attributesJSON_LENGTH(json_doc[, path])

Returns the length of JSON document, or, if a path argument is given, the length of the value within the document identified by the path. Returns NULL if any argument is NULL or the path argument does not identify a value in the document. An error occurs if the json_docargument is not a valid JSON document or the path argument is not a valid path expression or contains a * or ** wildcard.The length does not count the length of nested arrays or objects.

The length of a document is determined as follows:

○ The length of a scalar is 1.○ The length of an array is the number of array elements.○ The length of an object is the number of object members.

53

Functions to return JSON value attributes JSON_TYPE(json_val)

Returns a utf8mb4 string indicating the type of a JSON value:

Object, Array, Integer, Boolean ….

54

Functions to return JSON value attributes JSON_VALID(val)

Returns 0 or 1 to indicate whether a value is a valid JSON document. Returns NULL if the argument is NULL.

mysql> SELECT JSON_VALID('{"a": 1}');+------------------------+| JSON_VALID('{"a": 1}') |+------------------------+| 1 |+------------------------+

55

JSON Path SyntaxMany of the functions described in previous sections require a path expression in order to identify a specific element in a JSON document. A path consists of the path's scope followed by one or more path legs.

For paths used in MySQL JSON functions, the scope is always the document being searched or otherwise operated on, represented by a leading $ character. Path legs are separated by period characters (.).

Names of keys must be double-quoted strings or valid ECMAScript identifiers

Path expressions, like JSON text, should be encoded using the ascii, utf8, or utf8mb4 character sets. Other character encodings are implicitly coerced to utf8mb4.

You can use '$' as a synonynm for the document in JSON path expressions.

The wildcard * and ** tokens are used as follows:

● .* represents the values of all members in the object.● [*] represents the values of all cells in the array.● [prefix]**suffix represents all paths beginning with prefix and ending with suffix. prefix is

optional, while suffix is required; in other words, a path may not end in **.56

Got all that?

57

PHP JSON Functions● json_decode — Decodes a JSON string

● json_encode — Returns the JSON representation of a

value

● json_last_error_msg — Returns the error string of the last

json_encode() or json_decode() call

● json_last_error — Returns the last error occurred

http://php.net/manual/en/ref.json.php 58

Example 1 - Lifted from the PHP manual<?php

$arr = array('a' => 1, 'b' => 2, 'c' => 3, 'd' => 4, 'e' => 5);

echo json_encode($arr) . "\n";

?>

59

Example 1 - Lifted from the PHP manual<?php

$arr = array('a' => 1, 'b' => 2, 'c' => 3);

echo json_encode($arr) . "\n";

?>

{"a": 1, "b": 2, "c": 3}mysql> INSERT INTO foobar (mydata) VALUES ('{"a": 1, "b": 2, "c": 3}');Query OK, 1 row affected (0.01 sec)

60

Example 2

61

<?php$mysqli = new mysqli("localhost", "acct", "foobar", "database");

if ($mysqli->connect_errno) { printf("Connect failed: %s\n", $mysqli->connect_error); exit();}

$result = $mysqli->query("SELECT * FROM simple");

$row = $result->fetch_array(MYSQLI_BOTH);

printf ("%s\n", $row[0]);

$mysqli->close();

?>

Example 3<?php$mysqli = new mysqli("localhost", "acct", "foobar", "database");/* check connection */if ($mysqli->connect_errno) { printf("Connect failed: %s\n", $mysqli->connect_error); exit();}$query = "INSERT INTO simple VALUES(JSON_ARRAY(1,\"ABC\", NULL, TRUE))";if (!$mysqli->query($query)) { printf("Error: %s\n", $mysqli->error);}$mysqli->close();?>

62

GeneratedColumns

63

Why Generated Columns?

64

Remember that to store JSON data in a column that we need to break the

first rule of data normalization!

There are two kinds of Generated Columns: virtual (default) and stored. Virtual means that the column will be calculated on the fly when a record is read from a table. Stored means that the column will be calculated when a new record is written in the table, and after that it will be treated as a regular field. Both types can have NOT NULL restrictions, but only a stored Generated Column can be be a part of an index. So we can use a stored generated column to index JSON data!

Introduction to Generated ColumnsMySQL supports indexes on generated columns. For example:

CREATE TABLE t1 (f1 INT, gc INT AS (f1 + 1) STORED, INDEX (gc));

The generated column, gc, is defined as the expression f1 + 1. The column is also indexed and the optimizer can take that index into account during execution plan construction. In the following query, the WHERE clause refers to gc and the optimizer considers whether the index on that column yields a more efficient plan

65

How to use Generated Columns with JSON

mysql> CREATE TABLE jemp ( -> c JSON, -> g INT GENERATED ALWAYS AS (JSON_EXTRACT(c, '$.id')), -> INDEX i (g) -> );Query OK, 0 rows affected (0.28 sec)

A secondary index may be created on one or more virtual columns or on a combination of virtual columns and non-virtual generated columns. Secondary indexes on virtual columns may be defined as UNIQUE.

66

Generated Columns ContinuedWhen a secondary index is created on a virtual generated column, generated column values are materialized in the records of the index.

You can use -> as shorthand for JSON_EXTRACT() to access a value by path from a JSON column value

67

Generated Columns TrickNeed to search on names in a CaSe InSeNsItIVe fashion? Indexes on a column always is created with the collation of that column. And if you search with a different collation in mind, you end up with a full table scan.

But you can create a new generated column with the column’s data but stored in a cAsE iNsEnSiTiVe collation. You can search on the insensitive and case insensitive columns.

Great example at http://mablomy.blogspot.com/2015/03/auto-generated-columns-in-mysql-57-two.html

68

SyntaxGenerated column definitions have this syntax under CREATE TABLE:

col_name data_type [GENERATED ALWAYS] AS (expression) [VIRTUAL | STORED] [UNIQUE [KEY]] [COMMENT comment] [[NOT] NULL] [[PRIMARY] KEY]

AS (expression) indicates that the column is generated and defines the expression used to compute column values. AS may be preceded by GENERATED ALWAYS to make the generated nature of the column more explicit. Constructs that are permitted or prohibited in the expression are discussed later.

The VIRTUAL or STORED keyword indicates how column values are stored, which has implications for column use:

● VIRTUAL: Column values are not stored, but are evaluated when rows are read, immediately after any BEFORE triggers. A virtual column takes no storage.

● Prior to MySQL 5.7.8, virtual columns cannot be indexed. InnoDB supports secondary indexes on virtual columns. STORED: Column values are evaluated and stored when rows are inserted or updated. A stored column does require storage space and can be indexed.

The default is VIRTUAL if neither keyword is specified.69

Syntax ContinuedOther attributes may be given to indicate whether the column is indexed or can be NULL, or provide a comment. (Note that the order of these attributes differs from their order in non generated column definitions.)

Generated column expressions must adhere to the following rules. An error occurs if an expression contains disallowed constructs.

● Literals, deterministic built-in functions, and operators are permitted.

● Subqueries, parameters, variables, stored functions, and user-defined functions are not permitted.

● A generated column definition can refer to other generated columns, but only those occurring earlier in the table definition. A generated column definition can refer to any base (nongenerated) column in the table whether its definition occurs earlier or later.

● The AUTO_INCREMENT attribute cannot be used in a generated column definition.

● An AUTO_INCREMENT column cannot be used as a base column in a generated column definition.

70

When to useandWhen NOTto use

71

So JSON is great but ...You are breaking the first rule of data normalization and adding an extra layer of confusion. If you can use ‘generic’ SQL you will get better results and reduce complexity.

But if you need to store JSON formatted data, and keep it JSON formatted, in JSON use a JSON column

72

Thanks!Contact us:

Dave StokesDavid.Stokes@Oracle.comopensourcedba.wordpress.com@stokerslideshare.net/davidmstokes

73