polyglot database - linuxcon north america 2016

39
Polyglot Database s! [email protected] m @Stoker slideshare.net/davidmstokes

Upload: dave-stokes

Post on 21-Apr-2017

295 views

Category:

Data & Analytics


0 download

TRANSCRIPT

Page 1: Polyglot Database - Linuxcon North America 2016

Polyglot [email protected]@Stokerslideshare.net/davidmstokes

Page 2: Polyglot Database - Linuxcon North America 2016

"THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE."

Safe Harbor

Page 3: Polyglot Database - Linuxcon North America 2016

Quick MySQL Catch Up21 Years OldMySQL has been part of Oracle’s family of databases for six years.

MySQL 8MySQl 5.7 is the current release but the next version will be MySQL 8. Big feature is real time data dictionary

Group ReplicationActive master-master replication.

JSONA new native JSON datatype to store documents in a column of a table

Document StoreProgrammers not know SQL but need a database? X Devapi allows them to use RDMS from language of choice

EncryptionUse Oracle Key Vault to encrypt your data at rest.

Page 4: Polyglot Database - Linuxcon North America 2016

pol·y·glot"one who can write or speak several

languages."

Page 5: Polyglot Database - Linuxcon North America 2016

Pol·y·glotDatabase

"A data who can write or speak structured query language and more."

Page 6: Polyglot Database - Linuxcon North America 2016

Not a glUT Of parrots

Page 7: Polyglot Database - Linuxcon North America 2016

Databases are … mutatingThe relational database model has been established for decades. Originally designed for efficient storage of data of normalized data when disk drive space was expensive, they now can consume un-normalized data or bypass their own optimizers and syntax checking restraints for previously unimagined speed.

Page 8: Polyglot Database - Linuxcon North America 2016

Why!What would force well behaving software of outstanding utility To change … To morph … To take on new SuperPowers?????

Page 9: Polyglot Database - Linuxcon North America 2016

1.RElational

ModelOne size fits all,

One size fits small,Or

One size gives all fits?

Page 10: Polyglot Database - Linuxcon North America 2016

RDMSThe relational model works well but not all data fits easily into schemas.Sometimes the data is too big (or messy) , ephemeral, the team lacks DBA skills, orThe programmer is #%*@ lazy and wantsTo pile stuff like a teenager!!!

Page 11: Polyglot Database - Linuxcon North America 2016

NoSQL -- Broad topic made up of

many technologies looped together like indolent cattle and

branded

NoSQL

Page 12: Polyglot Database - Linuxcon North America 2016

NoSQL technologies× Graph Databases --relations, 6°of Kevin

Bacon× Map/Reduce -- Filter/Sort/Count× Key/Value -- <attribute name/ value>× Document store -- key/value with document

data

× Most designed to scale horizontally× Give up consistency

Page 13: Polyglot Database - Linuxcon North America 2016

Adding NoSQL

The relational database vendors have been busy adding NoSQL features to SQL. ACID on NoSQL still very hard.

Page 14: Polyglot Database - Linuxcon North America 2016

Key/Value Pair

Document StoreJSON Data TypeMicrosoft, PostgreSQL, MySQL & more

Adding the ‘Poly’ to your data

Page 15: Polyglot Database - Linuxcon North America 2016

The InnoDB memcached plugin provides an integrated memcached daemon that automatically stores and retrieves data from InnoDB tables, turning the MySQL server into a fast “key-value store”. Instead of formulating queries in SQL, you can use simple get, set, and incr operations that avoid the performance overhead associated with SQL parsing and constructing a query optimization plan. You can also access the same InnoDB tables through SQL for convenience, complex queries, bulk operations, and other strengths of traditional database software.

InnoDB Key/Value via memcahced

Page 16: Polyglot Database - Linuxcon North America 2016

9X faster!

When you bypass syntax

check and optimizer.

Page 17: Polyglot Database - Linuxcon North America 2016

Getting it workPluginOne line command to installed shared object.

ScriptRun a script to setup example tables. Use as a template to fit your data, e.g. column separator character defaults to ‘|’ (pipe).

CacheYou can now use MySQL/Memcached as a consistent cache or customize to your needs.

Page 18: Polyglot Database - Linuxcon North America 2016

JSON

Java Script Object Notation or JSON is a popular way of storing information in a relatively easy for humans to consume fashion.

Page 19: Polyglot Database - Linuxcon North America 2016

Not THIS JASON

Page 20: Polyglot Database - Linuxcon North America 2016

dON’T WANT TO NORMALIZE YOUR DATA? tHEN STORE A ENTIREDOCUMENTIN A SINGLECOLUMN!

Page 21: Polyglot Database - Linuxcon North America 2016

mYsqL 5.7’S nATIVE json DATA TYPECREATE TABLE json_example (JSON stuff);INSERT INTO json_example (stuff) \Values (‘{"k1": "value", "k2": 10}’);

Page 22: Polyglot Database - Linuxcon North America 2016

You are breaking the first rule of data normalization! So

you need new functions to be able to take care of the document in the

column!

Page 23: Polyglot Database - Linuxcon North America 2016

Twenty Two functions for JSON data

Create/Modify Search Meta

See JSON DATA TYPE http://slideshare.net/davidmstokes for details & examples

Page 24: Polyglot Database - Linuxcon North America 2016

JSON Data can not be indexed -- hard to find

You can use GENERATED columns to extract values from a JSON column and those columns can be indexed for fast SQL searches.

Page 25: Polyglot Database - Linuxcon North America 2016

SQL skills == no?

What if your programmers do not know Structured Query Language???

Page 26: Polyglot Database - Linuxcon North America 2016

Document StoreRelational databases such as MySQL usually required a document schema to be defined before documents can be stored. You can use MySQL as a document store, which is a schema-less, and therefore schema-flexible, storage system for documents. When using MySQL as a document store, to create documents describing products you do not need to know and define all possible attributes of any products before storing them and operating with them. This differs from working with a relational database and storing products in a table, when all columns of the table must be known and defined before adding any products to the database.

Page 27: Polyglot Database - Linuxcon North America 2016

New Shell & New protocolDevelopers can do CRUD

(create/read/update/delete)From language of choice

(SQL hidden as to not frighten them)

Page 28: Polyglot Database - Linuxcon North America 2016

33060New port

mysqlshNew shell with Python, Javascript, and SQL modes

X DevapiNew protocol

Page 29: Polyglot Database - Linuxcon North America 2016

X DevapiThe X DevAPI wraps powerful concepts in a simple API.

● A new high-level session concept enables you to write code that can transparently scale from single MySQL Server to a multiple server environment.

● Read operations are simple and easy to understand.

● Non-blocking, asynchronous calls follow common host language patterns.

The X DevAPI introduces a new, modern and easy-to-learn way to work with your data.

● Documents are stored in Collections and have their dedicated CRUD operation set.

● Work with your existing domain objects or generate code based on structure definitions for strictly typed languages.

● Focus is put on working with data

● Modern practices and syntax styles are used to get away from traditional SQL-String-Building

Page 30: Polyglot Database - Linuxcon North America 2016

The new shellmysqlsh -u root --sqlEnter password: ****mysql-py> db.createCollection("flags")<Collection:flags>mysql-py> db.getCollections()[ <Collection:CountryInfo>, <Collection:flags>]mysql-py> db.CountryInfo.find("GNP > 500000")...[output removed]10 documents in set (0.00 sec)

Page 31: Polyglot Database - Linuxcon North America 2016

Languages SupportedConnectors for

JavascriptPython.NETJAVANode.JS

(others in development)

Page 32: Polyglot Database - Linuxcon North America 2016

Other Vendors

have similar offerings of

NoSQL interfaces for

their SQL

databases

Page 33: Polyglot Database - Linuxcon North America 2016

And to you!!

What does this mean to your

servers???

Page 34: Polyglot Database - Linuxcon North America 2016

Databases are the spoiled brats of servicesDatabases do not× Play Well With Others× Always Want More× Needs near constant

watching× Always growing× Demanding

Page 35: Polyglot Database - Linuxcon North America 2016

Disk Reads are 100,000 slower than Memory

Page 36: Polyglot Database - Linuxcon North America 2016

Place your screenshot here

yOur futuredatabase Server

Page 37: Polyglot Database - Linuxcon North America 2016

HardwareMore/faster memory for in memory DBs

Spill to disk, pin to memoryDurable to disk (ACID Compliant)

Gets cheaper every year (~40%)Cache is the new RAM, RAM the new disk, disk the new

tape, etcNo random writes leverage SSDsNVRAM on the way, many be cheaper

Page 38: Polyglot Database - Linuxcon North America 2016

SoftwareSecurityCommunicationBackupSchedulerSmarter file systems/device drivers

Page 39: Polyglot Database - Linuxcon North America 2016

THANKS!Any questions?You can find me at @stoker or [email protected] -- http://slideshare.net.davidmstokes