polyglot persistence & multi-model databases (fullstack toronto)

19
www.arangodb.com Polyglot Persistence & Multi-Model Databases Michael Hackstein @mchacki FullStack Toronto 23-11-2014

Upload: arangodb

Post on 14-Jul-2015

811 views

Category:

Data & Analytics


2 download

TRANSCRIPT

Page 1: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

www.arangodb.com

Polyglot Persistence &

Multi-Model Databases

Michael Hackstein @mchacki

FullStack Toronto 23-11-2014

Page 2: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Michael Hackstein‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features !!

‣ Host of cologne.js

!

!

‣ Master’s Degree(spec. Databases and Information Systems)

2

Page 3: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

The Single Model era is over

3

Relational World

Page 4: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

The Multi Model era begins

4

NoSQL WorldDocuments - JSON

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => VK => V

K => V

Graphs

Key Value

{ “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" }

{ “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily

‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have

attributes ‣ Easily query paths of variable length

‣ Normally based on key-value stores (each document still has a unique key)

‣ Allow to save documents with logical similarity in “collections”

‣ Treat data records as attribute-structured documents (data is no more opaque)

‣ Often allow querying and indexing document attributes

Page 5: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

An e-commerce system in Relational World

5

Sales-History

Shopping-Cart

Recommendations Customer

Product-Catalog

Page 6: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Polyglot Persistence

6

Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf

Reporting

RDBMS

Product Catalog

MongoDB

Shopping Cart

Riak

User activity log

Cassandra

Analytics

Cassandra

Recommendations

Neo4J

Financial Data

RDBMS

User Sessions

Redis

Shopping Cart

KeyValue

Reporting

RDBMS

Product Catalog

Document

User activity log

Column

Analytics

Column

Recommendations

Graph

Financial Data

RDBMS

User Sessions

KeyValue

Page 7: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Single Model Databases

7

Recommendations

Product-CatalogShopping-Cart

Sales-History Customer

{ “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, }

{ “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, }

{ “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” }

{ “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, }

423453453

4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95

=>

8743655635463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95

=>

{ “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" }

{ “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

DocumentStore GraphStore DocumentStore

DocumentStoreKeyValueStore

Page 8: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Benefits

‣ Natural mapping of data into DB

‣ DB optimized for the data format

‣ Queries are tailored for your data format

‣ Focus on writing business logic

8

‣Data has to be stored redundantly and has to be kept in sync

‣Several technologies involved

‣Administration effort is huge

Overhead&

Page 9: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Solution: Multi Model Database

‣ Can natively store several kinds of data models: ‣ Key-value pairs ‣ Documents ‣ Graphs

‣ Delivers query mechanisms for all data models

9

Page 10: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Polyglot Persistence Revisited

10

Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf

Shopping Cart

KeyValue

Reporting

RDBMS

Product Catalog

Document

User activity log

Column

Analytics

Column

Recommendations

Graph

Financial Data

RDBMS

User Sessions

KeyValue

Reporting

RDBMS

Product Catalog

ArangoDB

Shopping Cart

ArangoDB

User activity log

Cassandra

Analytics

Cassandra

Recommendations

ArangoDB

Financial Data

ArangoDB

User Sessions

ArangoDB

Page 11: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Use Case: Multi-Model-Databases

11

Recommendations

Product-CatalogShopping-Cart

Sales-History Customer

{ “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, }

{ “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, }

{ “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” }

{ “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, }

423453453

4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95

=>

8743655635463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95

=>

{ “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" }

{ “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

DocumentStore GraphStore DocumentStore

DocumentStoreKeyValueStore

Page 12: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

My four favorite features of

‣ AQL offering joins & traversals

‣ ACID including Multi Collection Transactions

‣ MULTI-MODEL stores graphs and documents

12

‣ FOXX extend the API and adapt it to your needs

Page 13: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

AQL

‣Document Query: FOR user IN users FILTER user.active == true FOR game IN games FILTER game.player == user._id RETURN { username: user.name, score: game.score }

‣ Modify Documents: FOR u IN users FILTER u.status == 'not active' UPDATE u WITH { active: false } IN users !

‣ Graph Traversal: RETURN GRAPH_TRAVERSAL( "underground_plan", „stations/main_station", "outbound", {minDepth: 2, maxDepth: 5} )

13

Page 14: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

ACID - Transactions

‣ Invoke a transaction: db._executeTransaction({ collections: { write: ["users", "products"], read: "recommendations" }, action: function() { // all operations go here ! } });

14

throw "failure"; // Triggers rollback

Page 15: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

‣ Native mapping of data into DB

‣ DB optimized

‣ Queries are tailored for your data format

‣ Focus on writing business logic

15

‣Data has to be stored redundantly and has to be kept in sync

‣Several technologies ‣Administration effort is

‣One technology involved

Benefits Overhead&

Page 16: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Foxx

‣ Add your own customized and versioned REST-API on top of ArangoDB in JavaScript ‣ Include as a web service in Rails, Node.js etc. ‣ Use as storage for Web-frameworks like AngularJS, EmberJS, Backbone etc.

‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth

‣ Operations are encapsulated in the database ‣ low network traffic, direct data access ‣ increases data privacy

➡Multi-device setups ➡Microservices

/\ (~( ) ) /\_/\ ( _-----_(@ @) ( \ / /|/--\|\ V " " " "

16

Page 17: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

‣ open source and free (Apache 2 license)

‣ sharding & replication

‣ JavaScript throughout (V8 built into server)

‣ drivers for a wide range of languages

‣ web frontend

‣ good & complete documentation

‣ professional as well as community support

17

An overview of other features

Page 18: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Join our growing community

18

.. working on the geo index, the full text search and many APIs: Ruby, Python, PHP, Java, D, Javascript, ...

Page 19: Polyglot Persistence & Multi-Model Databases (FullStack Toronto)

Thank you

!!!

‣ Further questions? ‣ Follow me on twitter/github: @mchacki ‣ Write me a mail: [email protected] ‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb

19