Karen Lopez @datachick #HeartData

Heart of Data ModelingGraph Databases: Where does the modeling go?

Yes, Please do Tweet/Share today’s event

@datachick #heartdata

Karen López

Karen has 20+ years of data and information architecture experience on large, multi-project programs.

She is a frequent speaker on data modeling, data-driven methodologies and pattern data models.

She wants you to love your data…

She is loves new tech and gadgets

How new tech are you?...so let’s get to know you….

Use Q&A for

formal questions

Get them in now!

Use chat to discuss with each

other

We have a great community

Yes!

Slides

Recording

…next week…

Plan for Today

Why topic?

Graphy Stuff in Relational World

Another way?

Graph Resources

Modern Data

Architectures

will have hybrid

Technologies

WHERE ‘HYBRID’ = ‘SQL and NoSQL’

Clarifying terminology

Graphs Graph Databases Graph Processing

8

NoSQL, Not Only SQL

9

Relational Graph Columnar

Key ValueDocument Databases

Column Family

Graph & Hierarchy ConceptsOverview

Graph Nodes

Node

Node Node

Node

Node

Node

Node

Node

NodeNode

Node

Directed / Undirected

Node

Node

NodeNode

Node

Node

Node

NodeNode

Node

What is this structure?

Ragged Hierarchies

A hierarchy where there is variability in the number of levels across branches.

Node

Node

Node

Node

Node

Node

Node Node

Node

Node

Node

Node

14

Automobile

Engine

Fuel Line Valve Injector Fan

Fan Blade

Bearing

Bolt

Fanbelt

Entertainment System

Bolt

Radio

Satellite Radio

Media Player

Backup camera

Automobile

Engine

Injection System

Fuel Line

Valve

Injector

Fan

Fan Blade

Fan Bearing

Fanbelt

Entertainment System

Bolt

Radio

Satellite Radio

Media Player

Backup camera

Energy Graph

What Happens When…….?

Sometimes we take a group of “sibling widgets” and make them a widget just for them. Think “subassembly”. Then we have to think of this new group as a widget.

15

How we Model Graph in RelationalLots of tricks and tips happening here.

Recursive Relationship

Self Join

Recursive Association

Dog Ear / Mouse Ear

Bill of Materials

???

Data Model – Hierarchy Recursive

Data Model – Hierarchy Recursive

19

What happens when:

We add a new level?Take one away?Promote someone?

Relational Performance Tricks

Special data types

Adjacency Lists

Path Enumerations

Closure Tables

Nested Sets

20

But remember this?

21

Now we have M:N

22

Employee

Reporting Structure

Hierarchy vs. Real World

23

Recursive Relationships

24

Recursives IRL

Reporting structures

Components

Facilities

Documents

Networks

25

SQL Server HierarchyID

26

But aren’t relational databases about relationships?

Labeled Property Graph

Nodes have properties(think key-value pairs)

Nodes have labels(think meta-data and categories)

Relationships are directed

Relationships have names

Relationships have a start and endnode

Relationships have properties

28

Nodepropertyproperty

Nodeproperty

NodeNode

Node

Label

LabelLabel

Label

http://neo4j.com/graphgist/8139605

Graph Databases Physical Architecture

31

TripleStores

Come from semantic technologies movement

A triple is a subject:predicate:object data structure

Individually triples are semantically poor

En masse they provide rich dataset to harvest knowledge and infer connections

Use RDF and XML--SPARQL for queries

Ginger dances with Fred

Fred likes ice cream

Karen loves data

32

Graph Databases – Neo4j

CREATE (matrix1:Movie { title : 'The Matrix', year : '1999-03-31' })CREATE (matrix2:Movie { title : 'The Matrix Reloaded', year : '2003-05-07' })CREATE (matrix3:Movie { title : 'The Matrix Revolutions', year : '2003-10-27' })CREATE (keanu:Actor { name:'Keanu Reeves' })CREATE (laurence:Actor { name:'Laurence Fishburne' })CREATE (carrieanne:Actor { name:'Carrie-Anne Moss' })CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix1)CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix2)CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix3)CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix1)CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix2)CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix3)CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix1)CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix2)CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix3)

http://neo4j.com/docs/stable/cypherdoc-movie-database.html

Notice anyting?

Querying Neo4j with Cypher

MATCH (you {name:"You"})

MATCH (expert)-

[:WORKED_WITH]-

>(db:Database

{name:"Neo4j"}) MATCH

path = shortestPath( (you)-

[:FRIEND*..5]-(expert) )

RETURN db,expert,path

IBM Graph

Querying IBM Graph

var url = process.env.graphDBURL + '/gremlin';var query = "def g = graph.traversal(); g.V().has('code','" + req.body.orig +"').out('route').has('code', '" + req.body.dest + "')";var opts = { auth: { user: process.env.username, pass: process.env.password}, json: { gremlin: query } };

request.post(url, opts, function(error, resp, obj) {var result = (obj.result && obj.result.data && obj.result.data.length >

0) ? obj.result.data[0] : null;if (result) { // found a route from orig to dest console.log('route

exists from ' + req.body.orig + ' to ' + req.body.dest); } });

Another IBM Graph model

Titan

Tools and Graph Databases

•No native supportERwin

•No native supportER/Studio

•No native supportPowerDesigner

“the data model is the database”

“the database is the data model”

ODBC / JDBC connectively for querying.

So what about modeling for graph?

Marketing vs. Real Project

There is a model

The model isn’t the structure

The model would be used to design the graph(s)

Same modeling issues: NamingPropertiesRulesConsistencyGovernance

Data Modeling & Graph

No* Logical + Physical Data model

The graph is the data model...and the database

Whiteboard data modeling

Traditional data models still have a role

Requirements

Data Model

Database*

More

requirements

/ changes /

tuning /

whims

+ Non Model Stuff

Data Model

Driven

Data Model Driven

10+ Tips for Architects

1. Understand the use cases for graph technologies

2. Evaluate/profile your data requirements for suitability for graph databases and/or graph processing

3. ACID support varies across products. You’ll want to test your use cases.

4. Your query data stories will guide your decisions

5. Test your current development tools for support

10+ Tips for Architects

6. Test your database design/data modeling tools

7. Leverage your existing metadata/models

8. True hierarchies are VERY RARE in the real world.

9. Know the questions you have to ask about all the exceptions

10. Keep asking where the data integrity happens/is relevant

46

Resources

47

http://www.neo4j.org/learn/try 48

Fun with Graphs

Scotch Whiskeys

Belgian Beer

Bank Fraud Detection

Access Control Management

…and 100 more….

GraphGist Project - http://gist.neo4j.org/

Bluemix

http://www.ibm.com/analytics/us/en/technology/cloud-data-services/products/graph.html

Trees and Hierarchies in SQL for Smarties

51

Whitepaper

http://whitepapers.dataversity.net/content50141/

And it’s FREE! GraphDatabases.com

PostgreSQLRiakHbaseMongoDBNeo4JCouchDBRedis

“Every design decision should include cost,

benefit and risk”

- Karen Lopez

Thank you, you were great. Really, really great.

Karen Lopez @datachickwww.datamodel.com