entification: the route to 'useful' library data

Entification: The Route to Useful Library Data

Richard Wallis Technology Evangelist

@rjw

6 excellent SWIB's

6 excellent SWIB's

Many great LD Projects

So today …..

Where are

we on t

he web?

Where are

we on t

he web?

Irrelevant!Invis

ible on t

he web!

A global, nonprofit library cooperative

16,737 members in 109 countries 17 offices 5 data centers

•322 million cataloging records •2.1 billion holdings •17.3 million e-‐resources in the WorldCat knowledge base •Nearly 2,000 e-‐content collections •1.5 billion items in WorldCat Discovery, including:

• 297 million peer-‐reviewed articles • 41 million digital items • 33 million pieces of evaluative content • 35 million archival materials • 8 million open-‐access items • and much more…

Comprehensive and global


253 million books

16 millione-‐books

12 millionserials

12 millionvisual materials

7 millionmusical scores

4 millionmaps

English: 83 million

German: 25 million

French: 18 million

Spanish: 8.2 million

Chinese: 5.1 million

Italian: 3.4 million

Dutch: 3.3 million

Japanese: 2.9 million

Russian: 2.9 million

Danish 2.1 million

Swedish: 1.9 million

Portuguese: 1.3 million

Just a few of the dozens of

types of content:

Some of the 485 languages represented…

•322 million cataloging records •2.1 billion holdings •17.3 million e-‐resources in the WorldCat knowledge base •Nearly 2,000 e-‐content collections •1.5 billion items in WorldCat Discovery, including:

• 297 million peer-‐reviewed articles • 41 million digital items • 33 million pieces of evaluative content • 35 million archival materials • 8 million open-‐access items • and much more…


17.3 million Nearly 2,000 1.5 billion

297 million 41 million 33 million 35 million 8 million and much more… ≈ 2 Billion

Records

17.3 million Nearly 2,000 1.5 billion

297 million 41 million 33 million 35 million 8 million and much more… ≈ 2 Billion

Records000

012

61nam

a22002

411 450

0

001

303

005

000000

000000

00.0

008

990716

m19091

912enk

b b 000

0 eng

035

__

|9 (DLC

) 24002

676

906

__

|a 0 |

b ibc |c

orignew

|d u |

e ocip |

f 19 |g

y-‐genc

atlg

955

__

|a CAT

ALOGER

: This re

cord, im

ported

under 9

9-‐21981

8 duplic

ated 24-‐

2676

on PREM

ARC; I h

ave cha

nged LC

CN to t

hat LCC

N, remo

ved cop

y catalo

ging cha

racteris

tics, an

d

deleted

the PRE

MARC r

ecord;

please d

o as NE

W INPUT

and com

plete th

is record

based o

n the ite

m

in hand

; submi

t item fo

r select

ion; if re

tained,

add as a

copy. ta

05 07-‐16

-‐99

010

__

|a 240

02676

|z 9921

9818

040

__

|a DLC

|c DLC

050

00

|a DA7

60 |b .

B88 190

9

100

1_

|a Bro

wn, Pet

er Hume

, |d 184

9-‐1918.

245

10

|a Hist

ory of S

cotland

, |c by

P. Hume

Brown.

260

__

|a Cam

bridge,

|b Uni

versity P

ress, |c

1909-‐12

.

300

__

|a 3 v.

|b ma

ps (par

t fold.)

|c 20 cm

.

440

_0

|a Cam

bridge h

istorica

l series

500

__

|a Firs

t edition

, 1900-‐0

9.

504

__

|a Bibl

iograph

y: v. 1, p

. 402-‐40

8; v. 2,

p. 455-‐4

64; v. 3

, p. [435

]-‐444.

505

0_

|a v. 1

. To the

accessi

on of M

ary Stew

art.-‐-‐v.

2. From

the acc

ession o

f Mary

Stewart

to the R

evolutio

n of 168

9.-‐-‐v. 3.

From the

Revolut

ion of 1

689 to t

he disru

ption, 1

843.

651

_0

|a Sco

tland |

x Histor

y.

Structured Data Objectives

Structured Data Objectives• Linking with hubs of authority on the web


• viaf.org – persons


• viaf.org – persons• Library of congress – subjects


• viaf.org – persons• Library of congress – subjects• Dewey.info – classifications


• viaf.org – persons• Library of congress – subjects• Dewey.info – classifications• Dbpedia – most things



• Widely distributed & understood




• Standard data access patterns




• Standard data access patterns• Common vocabularies




• Standard data access patterns• Common vocabularies• Visibility in search engines




• Standard data access patterns• Common vocabularies• Visibility in search enginesConclusions





• Linked Data





• Linked Data• RDF – RDFa, RDF/XML, JSON-‐LD, Turtle, nTriples





• Linked Data• RDF – RDFa, RDF/XML, JSON-‐LD, Turtle, nTriples• Canonical URIs






• Schema.org






• Schema.org

• Backed and recognized by Google, Bing, Yahoo!, Yandex






• Schema.org

• Backed and recognized by Google, Bing, Yahoo!, Yandex• Widely adopted & understood – 20% of web






• Schema.org

• Backed and recognized by Google, Bing, Yahoo!, Yandex• Widely adopted & understood – 20% of web

fairly ob

vious

y

Introd

ucing

Linked

Data

Phase 1

Introd

ucing

Linked

Data

Phase 1• First mine the data

Introd

ucing

Linked

Data


• Records held in Marc• Identify the entities

Introd

ucing

Linked

Data



• Person, Organization, CreativeWork, etc.

Introd

ucing

Linked

Data



• Person, Organization, CreativeWork, etc.• Match strings to things

Introd

ucing

Linked

Data




• People/Organization names – viaf.org, etc

Introd

ucing

Linked

Data




• People/Organization names – viaf.org, etc• Subjects – Library of Congress

Introd

ucing

Linked

Data





Phase 2

Introd

ucing

Linked

Data





Phase 2• Model what is of interest to the Web

Introd

ucing

Linked

Data






• All our data is important to us

Introd

ucing

Linked

Data






• All our data is important to us• What will draw people to our resources?

Introd

ucing

Linked

Data







• Share the way the web does

Introd

ucing

Linked

Data








• Linked Data

Introd

ucing

Linked

Data








• Linked Data• Schema.org

Introd

ucing

Linked

Data









Phase 3

Introd

ucing

Linked

Data









Phase 3 -‐ Try it out!

edition

author location

holding

date of publication

classification

publisher

title

source

ISBN

library data:

stored as records

edition

author location

holding

date of publication

classification

publisher

title

source

ISBN

author location

holding

classification

publisher

person place

object concept

organization work

library data:

stored as records

title

person place

object concept

organization work

Google Knowledge Graph

Knowledge cards for libraries? Google Knowledge Graph

person place

object concept

organization work

Commercial data stored as entities

person place

object concept

organization work

Commercial data stored as entities

FRBR: Work/Expression

FRBR: Manifestation

• Knowledge cards • Fixes problem of “representative record” • It’s what users expect in discovery

Entities and library workflows:Discovery

• Improve data quality – Cascading updates

• A new approach to cataloging – Point and click cataloging – Managing entities instead of managing records

• Consistent with RDA

Entities and library workflows:Cataloging

Günter GrassBorn: 16 October 1927 Gdańsk, Poland

German novelist, poet, playwright, illustrator, graphic artist, sculptor and recipient of the 1999 Nobel Prize in Literature.

Works

Subjects

Quotes

Find Günter Grass works at:Libraries near me | Online Retailers

Germany | German literature | Historical fiction War stories | Black humor | Fantasy

“Even bad books are books and therefore sacred.”—The Tin Drum

Günter GrassBorn: 16 October 1927 Gdańsk, Poland

German novelist, poet, playwright, illustrator, graphic artist, sculptor and recipient of the 1999 Nobel Prize in Literature.

Works

Subjects

Quotes

Find Günter Grass works at:Libraries near me | Online Retailers

Germany | German literature | Historical fiction War stories | Black humor | Fantasy

“Even bad books are books and therefore sacred.”—The Tin Drum

MARC RECORD


The Tin Drum

Summary: Acclaimed as the greatest German novel since the end of World War II. The Tin Drum is the story of thirty year old Oskar Matzerath who has lived through the long Nazi nightmare and who is being held in a mental institution.

Subjects

Borrowing Options Ebooks | Printed Books | Audio Books

Other Languages

!

Germany -‐ History | German literature | Political fiction


The Tin Drum


Subjects


Other Languages

!


Person Editor


The Tin Drum


Subjects


Other Languages

!


Person Editor

Person Authority • Günter Grass • SameAs ➾

<dbpedia.org>


The Tin Drum


Subjects


Other Languages

!


Person Editor

Person Authority • Günter Grass • SameAs ➾

<dbpedia.org>

LC NAF


The Tin Drum


Subjects


Other Languages

!



The Tin Drum


Subjects


Other Languages

!


Work Editor


The Tin Drum


Subjects


Other Languages

!


Work Editor

Work Authority • Title • Creator ➾ <Person>

Expression

Manifestation 1 Manifestation 2 Manifestation 3


The Tin Drum


Subjects


Other Languages

!


Work Editor

Cascading Updates


Expression

Manifestation 1 Manifestation 2 Manifestation 3


The Tin Drum


Subjects


Other Languages

!


Work Editor

MARC21 Output

Cascading Updates


Entities and library workflows:Other applications

• Interlibrary Loan – Borrow at the Work level – Manifestations/Items are detail



• Analytics – Fixes “holdings scatter” across manifestations



• Analytics – Fixes “holdings scatter” across manifestations

• Other third party applications – Discovery API exposes library entities


Entities and library workflows:Web exposure

• Be found on the web


• Be found on the web• Connect your users to unique content


• Be found on the web• Connect your users to unique content• What the web requires for web exposure

– Aggregation– Familiar structures– A Network of Links– Entity Identifiers


WorldCat Entities

WorldCat Entities

Works

WorldCat Entities

Works

• 197+ million Work descriptions and URIs • Schema.org • RDF Data formats – RDF/XML, Turtle, Triples, JSON-‐LD

• Links to WorldCat manifestations • Links to Dewey, LCSH, LCNAF, VIAF, FAST • Open Data license • Released April 2014

Work Place

ConceptEvent

Organization PersonCataloging

Integration with the web

Cascading updates More options

Intuitive searching

Bibliographic Entities

Bibliographic Entities -‐ In the Web of Data

person place

object concept

organization work


person place

object concept

organization work

Entity Based Data Architecture…


What About Linked Data?

https://www.flickr.com/photos/rileyroxx/169900848/

What About Linked Data?What about Linked Data?

Yeah! – what about Linked Data?

I thought Linked Data was going to solve all

our problems!

https://www.flickr.com/photos/rileyroxx/169900848/

Linked Data

• A Technology• Standard on the Web – RDF, URIs, Vocabularies• Identifying and Linking resources on the Web• Important powerful enabling technology

Linked Data


• But only a technology…

Linked Data


• But only a technology… for the systems folks to worry about

Linked Data


• But only a technology… for the systems folks to worry about• Real benefits flow from:

Entity Based Data Architecture

Linked Data


• But only a technology… for the systems folks to worry about• Real benefits flow from:

Entity Based Data Architecture Powered by Linked Data

Linked Data

Entity Based Data

Entity Based Data on the Web


Knowledge Graphs


Knowledge Graphs

Semantic Search

Why is the Web Adopting This?

Why is the Web Adopting This?(Entities, Semantic Search, Linked Data)

• To get their products/resources in front of users- Next Generation SEO


• To get their products/resources in front of users- Next Generation SEO

• It is a shared approach from the Search Engines- But not exclusive to them


Syndication For Libraries


• Aggregate to a central site - National Library, Consortia, WorldCat.org



• Publish details to syndication partners- WorldCat: Amazon, Google Scholar, EasyBib,

EBSCO, OpenLibrary, FindMyLibrary, RedLaser, Yelp, …





• Links from aggregators to individual libraries- Find in a library






WorldCat

Syndicati

on

Year to Ju

ne 2013

!• 77 M

illion refe

rrals from

partners

• 8.7 Million

click-‐thro

ugh to lib

raries






Today






Today

Efficient but indirect

Syndication For LibrariesOn the Web of Data


• Individual libraries publish resource dataOn the Web of Data


• Individual libraries publish resource data- Linked Data in local discovery interfaces

On the Web of Data


• Individual libraries publish resource data- Linked Data in local discovery interfaces- Links to authoritative hubs – set global context

• VIAF, LoC, WorldCat Works, …

On the Web of Data




• Recognized and identified on the Web

On the Web of Data




• Recognized and identified on the Web- Google, Bing, Yahoo!, Yandex, etc.- Where our users are!

On the Web of Data





• Users referred directly to resources in the library

On the Web of Data





• Users referred directly to resources in the library

On the Web of Data

Direct and Effective

Tell them about our resources…

http://www.flickr.com/photos/boston_public_library/6220572487

Tell them about our resources……using their language and methods

http://www.flickr.com/photos/boston_public_library/6220572487

Has it been a waste of time?


Am I saying….


Throw it all away and use…

Am I saying….


Throw it all away and use…

No!….

Am I saying….

Don't throw the baby out with the bath water


Sharing for discovery on the web

As part of a Global Knowledge Graph




Entification




EntificationIdentify the entities in your data:





• Describe them using appropriate vocabularies





• Describe them using appropriate vocabularies• Describe the relationships between them





• Describe them using appropriate vocabularies• Describe the relationships between them• Place them in a global context – link to authoritative hubs





• Describe them using appropriate vocabularies• Describe the relationships between them• Place them in a global context – link to authoritative hubs• Liberate the value in your data!






ENTITIES






And also share them on the web – a job for Schema.org

ENTITIES

Why Catalog?

Why Catalog?So we can find things


Why Share on the Web?


Why Share on the Web?

So today’s users can find our things

OCLC Entity Based Data Strategy

OCLC Entity Based Data Strategy✓VIAF, ISNI, FAST Publish Linked Data2010

OCLC Entity Based Data Strategy✓VIAF, ISNI, FAST Publish Linked Data✓WorldCat.org Linked Data Release – using Schema.org2012

2010

OCLC Entity Based Data Strategy✓VIAF, ISNI, FAST Publish Linked Data✓WorldCat.org Linked Data Release – using Schema.org

✓Data mining of WorldCat resources

2012

2013

2010



✓WorldCat Works Released – using Schema.org

2012

2014

2013

2010




✓Schema.org added to VIAF RDF

2012

2014

2013

2010





2012

2014

➢Application Integration➢WorldCat Discovery➢Analytics➢Discovery API➢Cataloging

2015

2013

2010





2012

2014


2015

➢More Entities Released➢Person➢Organization➢Event➢Concept

2013

2010





2012

2014


2015


➢New Products ➢New Services

2013

2016

2010





2012

2014


2015


➢New Products ➢Continuing Evangelism

➢New Services➢Continuing Innovation

2013

2016

2010

Where are our users?

6 excellent SWIB's !


but6 excellent SWIB's

!Many great LD Projects



If users can't discover our resources



If users can't discover our resources

What is the point?



5



7 5

more

y



@rjw



@rjw

http://slideshare.net/rjw

entification: the route to 'useful' library data

Technology

econtent collections

digital items

openaccess items

data centers322

webstructured data objectives

hubs of authority

pieces of evaluative

da760 b