ekaw - publishing with triple pattern fragments
TRANSCRIPT
Publishing withTriple Pattern Fragments
Ruben Taelman - @rubensworksimec - Ghent University
1
Publishing with Triple Pattern FragmentsTPF server software
TPF client-side querying
Quad Pattern Fragments
Demo
2
Publishing with Triple Pattern FragmentsTPF server software
TPF client-side querying
Quad Pattern Fragments
Demo
3
http://linkeddatafragments.org/software/
JavaScript LDF serverPythonPerlRubyPHPJava...
TPF server implementations in different languages
4
Requires Node.js ≥ 4.0
Installing and running the LDF server with Node
5
[sudo] npm install -g ldf-server
ldf-server config.json <port> <workers>
Documentation: https://github.com/LinkedDataFragments/Server.js
Requires Docker
Installing and running the LDF server with Docker
6
docker pull linkeddatafragments/server.js
docker run -p <port>:3000 -it --rm \-v $(pwd)/config.json:/tmp/config.json \
ldf-server /tmp/config.json
Preconfigured LDF server with NGINX cache and web-client
Setting up a full stack with Docker (Compose)
7
docker-compose up
https://github.com/LinkedDataFragments/FullStackServer
{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}
LDF server is configured with config.json
8
{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}
Configure list of datasources
9
{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}
Load an HDT file
10
{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}
Act as a proxy to a SPARQL endpoint
11
HDTN-TriplesTurtleJSON-LDSPARQL-endpoint
Different data sources are possible
12
by extending lib/datasources/Datasource.js
Return triple stream given triple pattern, offset and limit
...or write your own datasource implementation
13
Exposing multiple datasources as if it was one
Useful for fragmented datasets that can’t be merged
Compose multiple datasources
14
https://github.com/LinkedDataFragments/Server.js/blob/master/config/config-composite.json
Publishing with Triple Pattern FragmentsTPF server software
TPF client-side querying
Quad Pattern Fragments
Demo
15
http://linkeddatafragments.org/software/
JavaScript LDF clientPythonPerlJava...
Different client implementations exist as well
16
No installation required at all!
http://client.linkeddatafragments.org/
Use the web client
17
Requires Node.js ≥ 4.0
Run from command line or include in your source code
Installing and running the LDF client with Node
18
[sudo] npm install -g ldf-client
ldf-client <tpf-endpoint-url> <sparql-query-path>
Documentation: https://github.com/LinkedDataFragments/Client.js
Requires Docker
Installing and running the LDF client with Docker
19
docker pull linkeddatafragments/client.js
docker run -it --rm \-v <sparql-query-path>:/tmp/query.sparql \
linkeddatafragments/client.js \<tpf-endpoint-url> /tmp/query.sparql
Publishing with Triple Pattern FragmentsTPF server software
TPF client-side querying
Quad Pattern Fragments
20
Quads are triples extended with a fourth element, the graph
<s> <p> <o> <g>.
Triples sometimes need some context
21
Quad Pattern Fragments (QPF)
Adding the fourth element to the interface
22
SELECT *FROM <http://example.org/graph0>FROM NAMED <http://example.org/graph1>WHERE {
GRAPH ?g {?s ?p ?o.
}}
QPF Clients are able to use quad features
23
QPF is backwards-compatible with TPF
24
TPF
QPFTPF
QPFIgnores graphs
Quad Pattern Fragments (WIP)https://github.com/LinkedDataFragments/Server.js/tree/feature-qpf-latest
https://github.com/LinkedDataFragments/Client.js/tree/feature-qpf-latest
25
Publishing with Triple Pattern FragmentsTPF server software
TPF client-side querying
Quad Pattern Fragments
Demo
26
Let’s try overloading the DBpedia TPF endpoint!
27
http://fragments.dbpedia.org/
Uptime of 99.9967%
28
29
30
Some query types are very slow
~1 hour!
Server running TPF server software with any dataset
31
TPF server software
RDF dataset
HDT dataset
SPARQL endpoint
...