The D2RQ mapping languageRichard Cyganiak

Presentation to W3C RDB2RDF XG, 23 May 2008

D2RQ

• DB-to-RDF mapper written in Java

• In: any JDBC database

• Out: SPARQL, Linked Data, or Jena API

• GPL, popular, easy to get started

• SPARQL-to-SQL algorithm not state of the art

• Axiom: We never modify the database

The project

• Started 2004 (roots: 2002) by Chris Bizer at FU Berlin; later me at FU and HP Labs

• 200+ downloads/month, 4600+ total

• mailing list at ~20 msgs/month, 700+ total

• In LOD cloud, TopBraid Composer etc

Architecture

SPARQL

RDF

HTML

Jena/Sesame

RDF dump

SPARQL

Clients

Linked Data

Clients

HTML

Browsers

Non-RDF

Database

HTTP

Local Java

Application

Triple Store

D2RQ

Engine

D2R

Server

D2RQ Mapping

File

• maps DB to virtual RDF graph

• easy to offer arbitrary interfaces to the RDF graph

• most requested: SPARQL and RDF dumps

Architecture (2)

Mapping language

Mapping language

• N3 based syntax

• Very flexible

• Language is not easy, wish we had a GUI

• Usual workflow: auto-generate mapping from DB schema, then customize

Flexible mappings!

• Properties of one class from multiple tables

• Several classes in the same table

• Value translations, SQL expressions

• Arbitrary joins and SQL conditions

• This is a MUST HAVE! Users need it

Still missing

• SQL subqueries in WHERE or FROM

• Determine RDF/OWL property based on type code in a table

• …

To SQL or not to SQL?

• Users want to deal with complexity by using their SQL knowledge

• They want to write arbitrary SQL queries

• We don’t want to parse SQL (painful!)

• We force users to decompose their query into small fragments

Mapping process

1. Define your entities

2. Add properties to entites

3. Link entities together

4. Get fancy with conditions, joins, value translations

1. Define your entities

map:People a d2rq:ClassMap; d2rq:uriPattern “http://.../people/@@User.ID@@”; d2rq:class foaf:Person; d2rq:condition “User.deleted=0”.

(SQL fragments in red, RDFS/OWL vocabulary in blue)

2. Add properties

map:name a d2rq:PropertyBridge; d2rq:column “User.name”; d2rq:property foaf:nick .

map:mbox a d2rq:PropertyBridge; d2rq:uriPattern “mailto:@@User.email@@”; d2rq:property foaf:mbox .

3. Link your entities

map:photo a d2rq:PropertyBridge; d2rq:refersToClassMap map:Photos; d2rq:property foaf:made; d2rq:join “User.ID = Photo.UserID”.

(also d2rq:alias for self-joins)

4. Get fancy

map:mbox_sha1 a d2rq:PropertyBridge; d2rq:sqlExpression “SHA1(CONCAT(‘mailto:’, User.email))”; d2rq:property foaf:mbox_sha1sum .

Mapping file overview

:be

lon

gsTo

Cla

ssM

ap

:property

:property

:property

:property

:property

:property

:dataStorage

:dataStorage:refersToClassMap

:join "Paper.author=Author.ID"

:uriColumn "Paper.weblink"

:column "Paper.abstract"

:column "Paper.title"

:pattern "@@Author.first@@ @@Author.last@@"

:uriPattern "mailto:@@Author.email@@"

map:Database

map:title_PropertyBridge

map:abstract_PropertyBridge

map:author_PropertyBridge

map:weblink_PropertyBridge

foaf:Person

dcmi:Text

dc:title

dc:description

owl:sameAs

dc:creator

foaf:name

foaf:mboxmap:email_PropertyBridge

map:name_PropertyBridge

:uriPattern "/docs/@@Paper.ID@@"

map:Paper_ClassMap

:uriPattern "/people/@@Author.ID@@"

map:Author_ClassMap

:be

lon

gsTo

Cla

ssM

ap

:class

:class

SQL composition

• Take one or more PropertyBridges with its ClassMap

• SELECT columns/expressionsFROM all mentioned tablesWHERE joinsAND conditions

• (and deal with aliases)

Terminology warning!

• A ClassMap doesn’t necessarily correspond to an RDFS/OWL class

• Neither necessarily to a single table

• Rather: A set of entities/resources generated in the same way from the DB

Mapping language warts

• Remnants from pre-SPARQL days (constraints)

• Remnants from early lack of JDBC schema introspection (column types)

• Designed for MySQL 3’s primitive SQL(e.g. we didn’t consider temporary views)

Summary

• D2RQ: four years, 4600+ downloads

• Powerful mapping language is important

• To SQL or not to SQL?

• In the future there will be GUIs

Links

• D2RQ homepagehttp://www4.wiwiss.fu-berlin.de/bizer/d2rq/

• D2RQ manual & language spechttp://www4.wiwiss.fu-berlin.de/bizer/d2rq/spec/

• Mailing listd2rq-map-devel@lists.sourceforge.net