linked data applications: there is no-one-size-fits-all formula - asun gomez perez

Linked Data Applications:There is no One-Size-Fits-All

Formula

Asunción Gómez-Pérez

Facultad de Informática, Universidad Politécnica de Madrid

Campus de Montegancedo sn, 28660 Boadilla del Monte, Madrid

http://www.oeg-upm.net

[email protected]

Acknowledgements:O.Corcho, D. Garijo, D. Vila, L.Vilches, B. Villazón

Work distributed under the license Creative Commons Attribution-Noncommercial-Share Alike 3.0

SSSW-12: 9th Summer School on Ontological Engineering and the Semantic Web. Cercedilla. Spain

Table of content

1. The concept

2. Foundations

3. The process

4. Examples• Libraries: http://datos.bne.es• Geo: http://geo.linkeddata.es/• Metereology:http://aemet.linkeddata.es/• Travelling: http://webenemasuno.linkeddata.es/

2


Complex queries using data from heterogeneous Web pages

3

http://www.aemethttp://www.viaf.org/

*Picture attribution: http://commons.wikimedia.org/wiki/User:Gugerell

Cervantes enthusiast from Germany visiting Madrid and willing to know more about Cervantes’ work and life

http://www.bne.es/

http://elviajero.elpais.com/

4SSSW-12: 9th Summer School on Ontological Engineering and the Semantic Web. Cercedilla. Spain

M. Cervantes

Don Quixote

Hebrew

creator

Translatedinto

1960

Year of publication

VIAF

located

Data Integration

M. Cervantes Alcalá de Henares

Alcalá de Henares

birthPlace

Same as

Alcalá de Henares

20º

Temperatura

M. Cervantes

El Quijote

Autor

1605

Año de Publicación

BNE

Ubicado en

BD BNEBD

VIAFBD

AEMETBD IGN

Alcalá de Henares

Tapas Siglo de Oro

guía

BD Prisa

BD DBpedia

http://search.babylon.com/imageres.php?iu=http://2.bp.blogspot.com/_LCBs3OeIwwU/SrYqu0duerI/AAAAAAAABf4/Hb4nXjk8GPA/s320/z.gif&ir=http://banderas.del-mundo.es/&ig=http://t3.gstatic.com/images?q=tbn:ANd9GcQotvHQT8B33bsUfxWoz8h3jG9bKcn_3Ep2rLo330EaL7xDox3AAeK2OSg&h=300&w=300&q=banderas%20del%20mundo&babsrc=HP_ss


Table of content

1. The concept

2. Foundations

3. The process

4. Examples• Libraries: http://datos.bne.es• Geo: http://geo.linkeddata.es/• Metereology:http://aemet.linkeddata.es/• Travelling: http://webenemasuno.linkeddata.es/

5


The model (Ontology) and the data

6

Work

Idiom

translation

Year

Publication date

Library

Located at

Person

Is creator of

Has subject

El Quijote Cervantes

Is creator of

Catalán

translation

1960

Publication date

BNE

Located in

Has subject

Vida de Cervantes

Ontology

Data

birthPlacePlace

birthPlaceAlcalá de Henares

SSSW-12: 9th Summer School on Ontological Engineering and the Semantic Web. Cercedilla. Spain7

http://iflastandards.info/ns/fr/frbr/frbrer/C1001


translation

Año

Publication date

http://xmlns.com/foaf/0.1/Organization

Located in


Is creator of

Has subject

http://datos.bne.es/resource/XX3383563 http://datos.bne.es/resource/XX1718747Es autor

http://datos.bne.es/resource/XX1924295

translation

1960

Publication date

BNE

Located in

Has subject

http://datos.bne.es/resource/bimo0002045496

Vida de Miguel de Cervantes Saavedra

Don Quijote de la ManchaCervantes Saavedra, Miguel de

Catalán

Ontology

Datahttp://datos.bne.es/#

Language

work

Biblioteca

Person

http://geo.linkeddata.es/ontology/Municipio

birthPlace

http://geo.linkeddata.es/resource/Alcalá de Henares

birthPlace

The model (Ontology) and the data


Table of content

1. The concept

2. Foundations

3. The process

4. Examples

8

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


• Data sources analysis

• URI Design

• License definition

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation

9Reunión bilateral CNIG – OEG Proyecto OTALEX


URI design

• Meaningful URIs vs opaque URIs• Separate TBox (ontology model) from ABox

• Base URI

http://linkeddata.es/

http://geo.linkeddata.es/

http://otalex.linkeddata.es/

• Ontología (TBox URIs)

http://phenomenontology.linkeddata.es/ontology/{concept|property}

http://phenomenontology.linkeddata.es/ontology/Municipality

• Datos (ABox URIs)

http://geo.linkeddata.es/resource/{resource type}/{resource name}

http://geo.linkeddata.es/resource/Municipio/Azuaga

Specification


Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


License Definition

• Several possibilities• The UK Open Government License

• Open Database License

• Public Domain Dedication and License

• Open Data Commons Attribution License

• The Creative Commons Licenses (CC)

• It is also possible to reuse and apply an existing license of the (government) data sources.

Specification


Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Ontology

12

• Ontologies: • A set of terms• A set of explicit assumptions regarding the intended meaning of

the terms. • Almost always including concepts and their classification• Almost always including properties between concepts

• Shared understanding of a domain of interest

• Ontologies expressed in OWL or RDF(S), both based on RDF

• The NeOn methodology helps to build ontologies

Modelling

Specification

Modelling

RDF Generation

Publication

Links Generation

Exploitation


2. Vocabulary development

• Features• Lightweight :

• Taxonomies and a few properties• Consensuated vocabularies

• To avoid the mapping problems• Multilingual

• Linked data are multilingual

• The NeOn methodology can help to • Re-enginer Non ontological resources into ontologies

• Pros: use domain terminology already consensuated by domain experts

• Withdraw in heavyweight ontologies those features that you don’t need

• Reuse existing vocabularies

13Asunción Gómez Pérez

Identificationof the data

sources

Vocabularydevelopment

Generationof the RDF Data

Publicationof the RDF data

Linking the RDF data

Data cleansing

Enable effective discovery


O. Specification O. Conceptualization O. ImplementationO. Formalization

1RDF(S)

OWL

Flogic

NeOn Methodology

Ontology Restructuring(Pruning, Extension,

Specialization, Modularization)

8

O. Localization

9

Ontology Support Activities: Knowledge Acquisition (Elicitation); Documentation; Configuration Management; Evaluation (V&V); Assessment

1,2,3,4,5,6,7,8, 9

O. Aligning

O. Merging

Alignments5

5

5

Ontological Resource

Reengineering

4

4

4

6

6

6

6

Knowledge Resources

Ontological Resources

O. Design Patterns

2

Non Ontological Resources

Thesauri

DictionariesGlossaries Lexicons

TaxonomiesClassification

Schemas

Non Ontological Resource

Reuse

Non Ontological Resource

Reengineering

2

2

O. Repositories and Registries

FlogicRDF(S)

OWL

Ontology Design

Pattern Reuse

7

3

Ontological Resource

Reuse

3


Reuse available vocabularies

15

Reuse suitable Ontologies and

vocabularies

Linked Open Vocabularies

…

Modelling

Search for suitablenon-ontological

resources

Domain-related sites

Government Catalogs

are there suitable

resources?

Build the vocabulary by transforming available

resources

Yes

No

Build the vocabulary from scratch

Highly reliable Web Sites


Publicación

Data publication

Metadata publicacion using VOID

To facilitate the discovery

• Register in CKAN your dataset

• Use to sitemap4rdf to generate the site map

• Upload the site map to Google and Sindice

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Table of content

1. The concept

2. Foundations

3. The process

4. Examples• Libraries: http://datos.bne.es• http://linkeddata3.dia.fi.upm.es/bne-demo• Geo: http://geo.linkeddata.es/• Metereology: http://aemet.linkeddata.es/• Travelling: http://webenemasuno.linkeddata.es/

17

http://datos.bne.es/

http://linkeddata3.dia.fi.upm.es/bne-demo

http://geo.linkeddata.es/web/guest/visualizacion-beta

http://aemet.linkeddata.es/browser.html

http://webenemasuno.linkeddata.es/browser.html


MARC21

• Different communication formats:• MARC 21 format for Bibliographic Data• MARC 21 format for Authority Data• Others: Holdings, Classification, etc.

• Three main elements:• Record structure: ISO 2709. Fields, indicators,

subfields…• Content designation: "Meaning" of codes and

conventions• Content: Defined outside the MARC standard (ISBD,

AACR..)

18

Specification

Modelling

RDF Generation

Publication

Links Generation

Exploitation


Specification@ BNE

Specification

Modelling

RDF Generation

Publication

Links Generation

Exploitation

• Records in the MARC 21 format• 3.9 million bibliographical records• 4.2 million authority records• Version: November, 2011

19

AUTHORITY BIBLIOGRAPHIC

Persons

Corporate bodies

Conferences

Titles

Subject

76576 Maps

320727 Sound recordings

166017 Gravings, drawings, pictures

35770 Manuscripts

143959 Ancient books

2696560 Modern books

178473 Scores

3021 Electronic resources

156634 Serials

96672 Videos


MARC21 record structure

001 XX1721208

005 200012181124

008 901120nn aijnnaabn n aaa

016 $a BNE19900178994

040 $a SpMaBN $b spa $c SpMaBN $e rdc $f embne

100 10 $a Camus, Albert

$d 1913-1960

670 $a El mite de Sísif, 1987 $b port. (Albert Camus)

670 $a Dic. de filosofía, de J. Ferrater Mora, 1980$b(Camus., Albert (1913-1960); n. Mondovi, Argel)

670 $a Aut. BN-OPALE, 1995 $b (Camus, Albert)

20

Subfield

Field

Control Field

Content

Subfield

Content

• Authority record: Camus, Albert*

HEADING

1XX

* http://datos.bne.es/resource/XX1721208

Specification


MARC21 record content designation

001 XX1721208


$d 1913-1960


21

Name

Personal name

Control Number

Dates associated with name

• Authority record: Camus, Albert*

HEADING – Personal

Name

100


• Human reading:

An authority record that describes a Person, named Camus, Albert with associated dates 1913-1960

Source consulted Citation


Frecuency of codes in records

22

Specification

Modelling

RDF Generation

Publication

Links Generation

Exploitation


Specification

Specification

Modelling

RDF Generation

Publication

Links Generation

Exploitation

• Source data: MARC 21 records, not RDB. Very flat structure difficult to map to richer models

• Domain experts (catalogers) need to be part of the mapping process.

• Data quality good but still many errors: reporting.

• Iterative and incremental transformation process: measure coverage and progress.

• Highly specialized library models: FRBR, ISBD.• Multilinguality, collaboration with IFLA


Model: FRBR at a glance

Works

Expressions

Manifestations

Work 1

Work 2

Work 3

Expression1Expression 2

Manifestation1

Manifestation2

Specification

Modelling

RDF Generation

Publication

Links Generation

Exploitation

24


The Ontology: based on IFLA vocabularies

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Who will be the mapping generator?

BNE

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation

001 XX1721208

005 200012181124

008 901120nn aijnnaabn n aaa

016 $a BNE19900178994

040 $a SpMaBN $b spa $c SpMaBN $e rdc $f embne


$d 1913-1960


670 $a Dic. de filosofía, de J. Ferrater Mora, 1980$b(Camus., Albert (1913-1960); n. Mondovi, Argel)

670 $a Aut. BN-OPALE, 1995 $b (Camus, Albert)


Similar to mapping ontologies

27

100at Work

propertysubfield

maps

100t title of work maps

is creator of

Person100a maps

Content

(100a)

Content

(100at)contained in

maps


Marimba allows librarians to create mappings

• Three spreadsheets:

28

Classification mapping

Annotationmapping

Relationships mapping

MARC21 info

Records count Content sample Mapping

100 $a $d 888.880 Camus, Albert 1913-1960

foaf:Person

100 $a 999.999 Cervantes, Miguel de

foaf:name

100 $a $m 10.000 Cervantes, iguel ERROR

Basic structure


Librarians create mappings using excell

29


Annotationmapping

Relationships mapping

MARC21 info

Records count Content sample Mapping

100 $a $d 888.880 Camus, Albert 1913-1960

foaf:Person

100 $a 999.999 Cervantes, Miguel de

foaf:name

100 $a $m 10.000 Cervantes, iguel ERROR

Basic structure


SSSW-12: 9th Summer School on Ontological Engineering and the Semantic Web. Cercedilla. Spain 30

Annotationmapping

Relationshipsmapping

Librarians create mappings using excell

place of publication

has dimensions

Is part of work


Marimba interprets the Mappings and generate the RDF

• Classify: Exploiting the heading field and subfield codes.

100 $a $d Person (it has a personal name)

100 $a $d $t Work (it has a title)

• Annotate: Using subfield codes and the content.

100 $a "Camus, Albert" frbr:3001 "Camus, Albert"

100 $t "La Peste" frbr:P3039 "La Peste"

31

BNE

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation

MARC 21 record (Input)

Action RDF (Output)

100 $a $d Classify rdf:type frbr:C1005

100 $a Camus, Albert Annotate frbr:P3039 "Camus, Albert"

100 $d 1913-1960 Annotate frbr:P3040 "1913-1960"

001 XX1721208

……


$d 1913-1960

……


Mapping process more in detail

• But, what about the relationships between the entities?• Relationships between records are not explicit in MARC.

Goal: The work "La Peste" was created by Albert Camus

32

RDFGeneration

001 XX1721208

100 10 $a Camus, Albert $d 1913-1960

001 XX1910518

100 10 $a Camus, Albert$d1913-1960 $tLa peste


Common Common Diff

Work

bne:XX1721208 frbr:2010 bne:XX1910518 (isCreatorOf)

We know the type of R1 and R2, and we look at the heading diff


Marimba: Mapping process summary

Classify

Annotate

Relate

33

001 XX1721208

100 10 $a Camus, Albert $d 1913-1960

001 XX1910518

100 10 $a Camus, Albert$d1913-1960 $tLa peste

bne:XX1721208 a frbr:Person bne:XX1910518 a frbr:Work

bne:XX1721208 a frbr:Person

frbr:name "Camus, Albert" .

frbr:hasDates 1913-1960

bne:XX1910518 a frbr:Work

frbr:title "La Peste"

bne:XX1721208 a frbr:Person

frbr:name "Camus, Albert" .

frbr:hasDates 1913-1960 .

frbr:isCreatorOf bne:XX1721208

bne:XX1910518 a frbr:Work

frbr:title "La Peste" .

frbr:isCreatedBy bne:XX1721208

(MARC records)

BNE

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Marimba uses the ontology to generate RDF

BNE

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Marimba links with other resources:VIAF, DNB, SUDOC, LIBRIS, DBpedia

BNE


Same As

Same As

Same As

Same As

Same As

LIBRIS

http://libris.kb.se/resource/auth/45369

SUDOC

http://www.idref.fr/026774771/id

DNB

http://d-nb.info/gnd/11851993X

DBpedia

http://dbpedia.org/resource/Miguel_de_Cervantes

VIAF

http://viaf.org/viaf/17220427

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation

Marimba links with other resources:VIAF, DNB, SUDOC, LIBRIS, DBpedia



Publicación

Data publication

Metadata publicacion using VOID

To facilitate the discovery

• Register in CKAN your dataset

• Use to sitemap4rdf to generate the site map

• Upload the site map to Google and Sindice

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation


Exploitation

select distinct COUNT(?Obras) where {


<http://iflastandards.info/ns/fr/frbr/frbrer/P2010> ?Obras}

URI Cervantes

Is author

SPARQL queries

Web InterfaceEspecification

Model

RDFgeneration

Publication

Exploitation

Specification

Modelling

RDFGeneration

Publication

Exploitation

Links Generation

http://bne.linkeddata.es/


http://linkeddata3.dia.fi.upm.es/bne-demo


• Modelling:• Open Metadata Registry• Neon Toolkit

• Mapping and generation• MARiMbA: Library-oriented, supports and facilitates the

entire process od transformation from MARC21 to RDF

• Publication:• Virtuoso Universal Server• Pubby• CKAN registry• Sitemap4rdf

• Exploitation:• Web Applications that visualize data using SPARQL

Technological Support


Results: datos.bne.es

• Total number of authority records: 4.100.000• Total number of bibliographical records: 2.390.140• Total number of RDF triples: 58.053.215 • Number of links: (15% authorities): 587.520 • Linked sources:

• VIAF• SUDOC (French collective university catalogue) FR• GND (German National Library of authorities) GER• LIBRIS Sweden• DBPedia• Soon BNF

44

http://bne.linkeddata.es/

linked data applications: there is no-one-size-fits-all formula - asun gomez perez

Technology

madrid http

esrdf http

datalanguage http

semantic web

cervantes saavedra http

creator ofworkpersonao

summer school

organization cataln