document oriented access to graphs
DESCRIPTION
Author: Michael Hunger (http://twitter.com/mesirii) Graphs are a great data model for many domains and use-cases. It would be nice if they were accessed in a document oriented manner, projecting graph query results as compound documents and being able to send in documents whose content is then distributed within the graph. This presentation discusses "cypher-rs" a Neo4j Server extension that allows exactly that by creating http-endpoints for Cypher queries, which consume and return JSON documents. We look into the general concepts, usage and implementation details.TRANSCRIPT
![Page 1: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/1.jpg)
Document oriented access to a graph database
Using a Neo4j Server Extension Michael Hunger - @mesirii
http://github.com/jexp/cypher-rs
![Page 2: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/2.jpg)
Graphs
![Page 3: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/3.jpg)
Graphs are great
• for modeling highly connected domains
• finding patterns to answer questions
• high performance traversals
![Page 4: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/4.jpg)
Graphs are great
![Page 5: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/5.jpg)
Graphs suck
• for compact access patterns
• use-case specific views
• easy integration in modern webapps
![Page 6: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/6.jpg)
Graphs suck
![Page 7: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/7.jpg)
Documents
![Page 8: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/8.jpg)
Documents are great
• for everything that eats JSON
• especially modern JavaScript based webapps
• as dedicated projections of a domain
• compact representation
![Page 9: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/9.jpg)
Documents are great{
name:"Michael",
age:39,
children:[„Selina“,“Rana“,"Selma"],
address: { street: „Holbeinstr“, city: „Dresden“}
hobbies: [„Coding“,„Buchbar“,“MUD MorgenGrauen“]
blog: „http://jexp.de/blog“
}
![Page 10: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/10.jpg)
Documents suck
• for different projections besides the main use-case
• for connected data
• for cross-document linking
• for less redundancy / duplication
![Page 11: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/11.jpg)
Documents suck{
name:"Michael",
age:39,
children:[„Selina“,“Rana“,"Selma"],
address: { street: „Holbeinstr“, city: „Dresden“}
hobbies: [„Coding“,„Buchbar“,“MUD MorgenGrauen“]
blog: „http://jexp.de/blog“
} „Show people by hobby“
![Page 12: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/12.jpg)
Documents suck
{ _id: 1,
name:“Michael“,
works_at: dbref:2
}
{ _id: 2,
name:“Neo Technology“,
employees: [dbref:1]
}
„Link Employees to Companies“
![Page 13: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/13.jpg)
What can we do?
• Use a document oriented access layer for a graph database
• Use graph queries whose results project into documents
• Make them easily accessible
![Page 14: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/14.jpg)
How can we do that?
• Use Cypher in Neo4j 2.0
• Support for literal maps
• Parameters, including maps
![Page 15: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/15.jpg)
Cypher Maps
![Page 16: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/16.jpg)
Cypher & Maps - Results
MATCH (n:User)
WHERE n.name={name}
RETURN n
// returns a map of the node’s properties
RETURN {id: id(n), labels: labels(n), data : n}
![Page 17: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/17.jpg)
Cypher & Maps - Results (II)
MATCH (n:User)-[:PARENT_OF]->(child)
WHERE n.name={name}
RETURN {name:n.name, children:collect(child.name)}
![Page 18: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/18.jpg)
Cypher & Maps - Params
CREATE (u:User { name: {p}.name, age: {p}.age})
RETURN u
CREATE (u:User {params})
RETURN u
![Page 19: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/19.jpg)
Some Questions
![Page 20: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/20.jpg)
How can we do that?
• Create a RESTful API to
• Provide endpoints that represent on Cypher queries
• Use GET, POST with parameters to execute those endpoints
• Return only JSON documents or lists of those
![Page 21: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/21.jpg)
How do we implement it?• Add a Neo4j-Server-Extension
• Jersey JAX-RS
• Save queries in graph properties
• Execute queries for reads on GET
• Execute queries for reads and writes on POST
• Support JSON and CSV as payload
• Batch transactional write operations
![Page 22: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/22.jpg)
What does it look like?
![Page 23: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/23.jpg)
What does it look like?
• Examples for
• Managing endpoints
• Querying Endpoints
• Writing to Endpoints
![Page 24: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/24.jpg)
Create EndpointPUT /cypher-rs/users
Body: match (n:User) where n.name={name} return n
--> 201 Location: /cypher-rs/users
PUT /cypher-rs/create-user
Body: create (n:Node {name:{name},age:{age}})
--> 201 Location: /cypher-rs/create-user
![Page 25: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/25.jpg)
GET, POST - Reads
![Page 26: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/26.jpg)
GET RequestGET /cypher-rs/users?name=Andres
--> 200
{"name":"Andres",
"age":21,"children":["Cypher","L.","N."]}
GET /cypher-rs/users?name=NotExists
--> 204
![Page 27: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/27.jpg)
POST Request - ReadPOST /cypher-rs/users
Content-type: application/json
Body: {"name":"Andres"}
--> 200
{"name":"Andres","age":21,
"children":["Cypher","L.","N."]}
![Page 28: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/28.jpg)
POST Request - Read (II)POST /cypher-rs/users
Content-type: application/json
Body: [{"name":"Andres"},{"name":"Peter"},{"name":"NotExists"}]
!
--> 200
[{"name":"Andres","age":21,"children":["Cypher","L.","N."]},
{"name":"Peter" ,“age“:32,"children":["Neo4j","O.","K."]},
null]
![Page 29: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/29.jpg)
POST - Write
![Page 30: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/30.jpg)
POST Request - Write JSON
POST /cypher-rs/create-user
Content-type: application/json
Body: {"name":"Andres","age":21}
--> 201
![Page 31: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/31.jpg)
POST Request - Write JSON (II)POST /cypher-rs/create-user
Content-type: application/json
Body: [{"name":"Andres","age":21},
{"name":"Peter", "age":40},
--> 201
![Page 32: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/32.jpg)
POST Request - Write CSV (III)
POST /cypher-rs/create-user[?batch=20000&delim=,]
Content-type: text/plain
Body: name,age\nAndres,21\nPeter,40
--> 201
![Page 33: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/33.jpg)
More Management
![Page 34: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/34.jpg)
Delete Endpoint
DELETE /cypher-rs/users
--> 200
![Page 35: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/35.jpg)
List Endpoints
GET /cypher-rs
--> 200
[„create-users“, „users“]
![Page 36: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/36.jpg)
Inspect Endpoint
GET /cypher-rs/users/query
--> 200
match (n:User) where n.name={name} return n
![Page 37: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/37.jpg)
Results
![Page 38: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/38.jpg)
Possible Resultssingle column, single row!
{"name":"Andres","age":21,"children":["Cypher","L.","N."]}
single column, multiple rows!
[
{"name":"Andres","age":21,"children":["Cypher","L.","N."]},
{"name":"Peter", "age":40,"children":["Neo4j","O.","K."]}
]
![Page 39: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/39.jpg)
Possible Results (II)multiple columns, single row!
{"user": "Andres", "friends": ["Peter","Michael"]}
multiple columns, multiple rows!
[
{"user": "Andres", "friends": ["Peter","Michael"]},
{"user": "Michael","friends": ["Peter","Andres"]},
{"user": "Peter", "friends": ["Andres","Michael"]}
]
![Page 40: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/40.jpg)
Implementation
![Page 41: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/41.jpg)
Implementation - Init
![Page 42: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/42.jpg)
Implementation - PUT
![Page 43: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/43.jpg)
Implementation - GET
![Page 44: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/44.jpg)
UsageBuild with mvn clean install dependency:copy-dependencies
Copy files
cypher-rs-2.0-SNAPSHOT.jar opencsv-2.3.jar to path/to/server/plugins
Add this line to path/to/server/conf/neo4j-server.properties
org.neo4j.server.thirdparty_jaxrs_classes=org.neo4j.cypher_rs=/cypher-rs
Restart
![Page 45: Document Oriented Access to Graphs](https://reader034.vdocuments.net/reader034/viewer/2022042814/554f398eb4c90572088b4e5a/html5/thumbnails/45.jpg)
Next Steps
• Add streaming
• Rule the world