new world of metadata: growing, shifting, merging

The New World of Metadata

Growing, Shifting, MergingDiane

Hillmann

Metadata Day, Worcester 2

Intro to (New) Metadata

0 What’s the problem, huh?0 Are the old questions the right questions?0 Are our assumptions about metadata still valid?0 What exactly are the new realities we need to

consider?0 What are some survival strategies for the times to

come?

5/9/12

After we empty the card catalog …

We hear that there’s a crisis in libraries, but we still haven’t realized how pervasive it is

Reality: we’ve gotten rid of the cards, now we need to get rid of the catalog.

If we don’t, we may lose our institutional support, our mission, and our way …

5/9/12Metadata Day, Worcester 3

Metadata Day, Worcester 45/9/12

“As librarians, we pride ourselves on operating outside of the commercial marketplace. However, whether we like it or not, we are working in an information environment the dynamics of which are very much like those of a free market, except the the currency spent by our “customers” is not money, but time and attention. … We may believe, for example, that our carefully-crafted catalog records provide excellent value in return for the time and energy required to use them—and we may be right. But if our patrons doubt that the catalog will return good value in exchange for the time and energy required to use it, then whatever value the catalog may actually contain becomes irrelevant.”

Rick Anderson, The Crisis in Research Librarianship, Journal of Academic Librarianship, July 2011


“Wikipedia is founded on the belief (largely correct, as it turns out) that crowds both can and will provide high-quality content and metadata to the world at no charge. For our part, in research libraries we still tend to treat books as if they are primarily tools for linear reading, and metadata records as artisanal products. We still build collections that are fenced off from the larger information world and encourage our patrons, against all reason, to begin their information searches within the confines of our artificially limited collections.”



“We must look with cold and hard-headed rationality at our current practices and ask ourselves not what value they offer, but rather what value our patrons believe they offer. If what we offer our patrons is not perceived as valuable by them, then we have two choices: change their minds, or redirect our resources. The former is virtually impossible; the latter is enormously painful. But the latter is possible, and if we do not undertake such a redirection ourselves, it will almost certainly be undertaken for us.”



Poking Holes in the Old SilosUnderstanding what we can gain by moving into the linked data world is really critical for ‘future-proofing’ our efforts

We have a great deal to offer to the wider world, but we can’t do it if we cling to the old familiar ways of doing things

Learning about other data available to us (that we don’t have to pay to create) is essential, despite our inclination not to trust any data but our own

5/9/12


“In the big picture, very little will change: libraries will need to be in the data business to help people find things. In the close-up view, everything is changing-- the materials and players are different, the machines are different, and the technologies can do things that were hard to imagine even 20 years ago.”

Eric Hellman http://go-to-hellman.blogspot.com/2011/07/library-data-why-bother.html

http://go-to-hellman.blogspot.com/2011/07/library-data-why-bother.html


Questionable Assumptions

0 We’re going to continue to build records for library catalogs

0 We’ve always shared ‘records’ in cataloging, and that’s still the right way to do things

0 The choice of the ‘right metadata format’ (e.g., DC, MODS, RDA, etc.) is critically important

0 The proliferation of metadata formats is a bad thing0 The ‘old’ way of cataloging materials one-at-a-time

always produces better quality data than any other method

5/9/12


This Is NOT About Blaming the Messengers

5/9/12


“Today, we face another significant time of change that is being prompted by today’s library user. This user no longer visits the physical library as his primary source of information, but seeks and creates information while connected to the global computer network. The change that libraries will need to make in response must include the transformation of the library’s public catalog from a stand-alone database of bibliographic records to a highly hyperlinked data set that can interact with information resources on the World Wide Web. The library data can then be integrated into the virtual working spaces of the users served by the library.”

--Karen Coyle, Understanding the Semantic Web: Bibliographic Data and Metadata, Jan. 2010

5/9/12


If all of this sounds otherworldly and vague, it is because there is no specific vision of where these changes will lead us. The crystal ball is unfortunately shortsighted, in no small part because this is a time of rapid change in many aspects of the information ecology. The few things that are certain, however, point to the Web, and its eventual successors, as the place to be. For libraries, this means yet another evolutionary step in the library of our catalog: from metadata to metaDATA.”

--Karen Coyle, Understanding the Semantic Web: Bibliographic Data and Metadata, Jan. 2010


Model of ‘the World’ /XML

0XML assumes a 'closed' world (domain), usually defined by a schema:0"We know all of the data describing this

resource. The single description must be a valid document according to our schema. The data must be valid.”

0XML's document model provides a neat equivalence to a metadata 'record’

5/9/12


Model of ‘the World’ /RDF

0 RDF assumes an 'open' world:0 "There's an infinite amount of unknown data

describing this resource yet to be discovered. It will come from an infinite number of providers. There will be an infinite number of descriptions. Those descriptions must be consistent."

0RDF's statement-oriented data model has no notion of 'record’ (rather, statements can be aggregated for a fuller description of a resource)

5/9/12


What Are the Components of the Semantic Web?

0 RDF: Resource Description Framework0 Statements about Web resources in the form of subject-predicate-

object expressions, called triples0 E.g. “This presentation” –“has creator” –“Diane Hillmann”

0 RDF Schema0 Vocabulary description language of RDF

0 SKOS: Simple Knowledge Organisation System0 Expresses the basic structure and content of concept schemes such

as thesauri and other types of controlled vocabularies0 An RDF application

0 OWL (Web Ontology Language)0 Explicitly represents the meaning of terms in vocabularies and the

relationships between them

5/9/12


Semantic Web Building Blocks0 Each component of an RDF statement (triple) is a

“resource”0 RDF is about making machine-processable

statements, requiring0 A machine-processable language for representing RDF

statements0Extensible Markup Language (XML) can be used

0 A system of machine-processable identifiers for resources (subjects, predicates, objects)0Uniform Resource Identifier (URI) 0For full machine-processing potential, an RDF statement is

a set of three URIs

5/9/12


Things Requiring Identification0 Object “This presentation”

0 e.g. its electronic location (URL): http://hdl.handle.net/1813/11524

0 Predicate “has creator”0 e.g. http://purl.org/dc/terms/creator

0 Object “Diane Hillmann”0 e.g. URI of entry in Library of Congress Name Authority File

(real soon now?)0 NAF: nr2001015786

0 Declaring vocabularies/values in SKOS and OWL provides URIs0 Without such identifiers, the Web will never become Semantic

5/9/12

http://hdl.handle.net/1813/11524

http://purl.org/dc/terms/creator

http://authorities.loc.gov/cgi-bin/Pwebrecon.cgi?AuthRecID=5363382&v1=1&HC=1&SEQ=20081015154135&PID=-idX6Hi4FFC9LseDAL-BEZHpzGJ

What’s the Point of Interoperability?

0 For users, it’s about resource discovery (user tasks)0 What’s out there?0 Is it what I need for my task?0 Can I use it?0 Reality: They don’t care about what metadata format

you’re using!0 For resource creators, it’s about distribution and marketing

0 How can I increase the number of people who find my resources easily?

0 How can I justify the funding required to make these resources available?

0 How can I maximize my human resources in a world of smaller budgets and larger expectations?

5/9/12 Metadata Day, Worcester 18


Questioning Our ModelsToday’s metadata is not about choices of formats, it’s about ensuring interoperability and harmonization for our data in the world

Our old model is based on catalog cards, regardless of the method of delivery through our online catalogs

The new metadata environment provides better ways to express relationships—both content to content and concept to concept

5/9/12

Why Use Models for Metadata?

0To understand what entities you are dealing with

0To understand what metadata are relevant to which entities

0To understand relationships between different entities

0To organize your metadata to make it more predictable (and be able to create and use it with automated tools)


Descriptive Metadata Models0 Conceptual models for bibliographic and authority

data0 Functional Requirements for Bibliographic Records

(FRBR)0 Functional Requirement for Authority Data (FRAD)0 Functional Requirements for Subject Authority Data

(FRSAD)

0 Dublin Core Abstract Model (DCAM)0 Some other models:

0 CIDOC Conceptual Reference Model (emerged from museum community)

0 INDECS (for intellectual property rights)


Bibliographic relationships (pre-FRBR)

0Tillett’s Taxonomy (1987)0 Equivalence0 Derivative0 Descriptive0 Whole-part0 Accompanying0 Sequential0 Shared-characteristic


FRBR Entities

0Bibliographic entities: works, expressions, manifestations, items

0Responsible parties: persons, corporate bodies

0Subject entities: concepts, objects, events, places


Group 1 Entities & Their Relationships


Work

Expression

Manifestation

Item

A Work “Is realized through”

An Expression

An Expression “Is embodied in”A Manifestation

A Manifestation“Is exemplified by”

An Item

An Expression“realizes”

A Work

A Manifestation“embodies”

An Expression

An Item “exemplifies”

A Manifestation

[Thanks to Sherry Vellucci for this slide.]

DC Abstract Model0Reaffirms the ‘One-to-One Principle’0Defines‘statement’as the atomic level0Distinguishes between “description”

and “description set”:0Description: “One or more statements

about one, and only one, resource.”0Description Set: “A set of one or more

descriptions, each of which describes a single resource.”

0RDA vocabularies developed to reflect the DC Abstract Model


Metadata Day, Worcester5/9/12 26

A Dublin Core View of the World

DCMI Abstract Model: http://dublincore.org/documents/abstract-model/

http://dublincore.org/documents/abstract-model/


A Dublin Core View of the World

DCMI Abstract Model: http://dublincore.org/documents/abstract-model/

http://dublincore.org/documents/abstract-model/


Anatomy of a Statement (Attribute/Value)

Place of Production: New York

Property Value

ValueString

http://www.getty.edu/tgn/7007567


Anatomy of a Statement (Attribute/Value)

Place of Production: http://experimental.worldcat.org/fast/1204333

Property Value

RelatedDescription

http://experimental.worldcat.org/fast/1204333


A Related Description


Description Sets a Key Concept!

Metadata Day, Worcester

Description Set=“A set of one or more descriptions, each of which describes a single resource.”*

5/9/12 32

*DCAM Definition


A FRBR Description Set “Package”

Work

Manifestation

Expression

representation

representation

statementstatement

descriptiondescription

record

descriptionset

description

statementproperty

value

representation

value string

relateddescription

is groupedinto

is instantiatedas

has oneor more

has one

has one

is representedby one or more

is a

is a

OR

A record consists of descriptions, using properties and values.A value can be a string or a pointerto another description.


Basic model: Resource with properties

A Play has the title “Antony and Cleopatra,” was written in 1606 by William Shakespeare, and is about “Roman history”


… related to other Resources



Pulling it together; writing it all downIf you want others to be able to use your data, you’ll need to be clear about your intentions and document them

An Application Profile is one good way to do this so that your data can be properly interpreted, and easily used in a variety of ways

This is an area where DCMI has been working for some time …

5/9/12

What Are Application Profiles?

0APs describe the set of metadata elements, policies, guidelines and vocabularies defined for a particular domain, implementation, or object type0 Declares the metadata terms an organization,

information resource, application, or community uses in its metadata

0 Documents metadata standards used in instance data, including schemas and vocabularies, policies, required elements, etc.

0 Called “application profile” or just “profile”


Less Flexibility, More Predictability0Many metadata standards are sufficiently

flexible that they need a mechanism to impose some constraints0 Profiles allow expression of the decisions made for a

project in machine-readable form

0Refining0 Allows a narrower interpretation of a standard to

suit the needs of a domain or project

0Combining0 Enable mixing elements from various different

standards (there are limits to this!)


Components of an AP 0Human readable documentation

0 Property descriptions and relationships0 Domain or project specific instruction0 Obligation and constraints

0Machine-readable versions:0 Specific encoding decisions and XML schemas

or RDF 0 Models of data relationships specific to the AP

represented in the schemas0 Functional requirements and use cases

supporting decisions and clarifying intent5/9/12 Metadata Day, Worcester 41

Using Properties from a Variety of Schemas

0 DC APs set stringent requirements for determining reusability of terms:0 Is the term a real “property” and defined as such

within the source schema?0 Is the term declared properly, with a URI and

adequate documentation and support?0 In general, properties whose meaning is partly or

wholly determined by its place in a hierarchy are not appropriate for reuse in DC APs without reference to the hierarchy.

0 Other styles of profiles have different requirements and strategies for developing machine-readability and validation


How Does Quality Happen?

0 Lessons from the library community0 Quality is quantifiable and measurable0 To be effective, enforcement of standards of quality must

take place at the community level

0 Looking more broadly:0 Data problems are not unique to particular communities0 General strategies can improve interoperability

0 Quality is not tied to any particular creation strategy0 Human created metadata can be extremely variable0 Machine-created metadata is far more consistent, but that

consistency may not be correct


Quality Measurement: Criteria

0Completeness

0Accuracy

0Provenance

0Conformance to expectations

0Logical consistency and coherence

0Timeliness (Currency and Lag)

0Accessibility



Is Trust Based on Quality?

0 In traditional libraries, trust of other library metadata is nearly automatic0 Based on many years of sharing data, common

agreements, and reality

0 On the Web, data comes from many sources, not all have the immediate trust of libraries0 Evaluation strategies and techniques of determining quality

must be at a collective level0 Standards for provenance being developed by W3C and

DCMI will improve the ability to assess quality and assign trust levels

0 Libraries might want to collaborate on and share evaluations

5/9/12


BREAK!!

5/9/12


Top Down vs. Bottom Up?

Where standards are developed and who supports them are critical to their success

A case study is the RDA Vocabularies

Is this an anomaly or the precursor of things to come?

5/9/12Flickr photo by freebird4


RDA Vocabularies: General Strategy

We used the Semantic Web as our “mental model”Wanted to create a “bridge” between XML and

RDF to support innovation in the library community as a whole, not just those at the cutting edge or the trailing edge

We registered the FRBR entities as classes in a ‘FRBR in RDA’ vocabulary, to enable specific relationships between RDA properties and FRBR

IFLA has followed suit using the Open Metadata Registry to add the ‘official’ FRBR entities, FRAD, FRSAR and ISBDThis provides exciting opportunities to relate all

the vocabularies together

5/9/12


Structure: Rationale & Decisions

0 Property and value vocabularies are registered in the Open Metadata Registry (formerly the NSDL Registry): http://metadataregistry.org/rdabrowse.htm

0 Used RDF Schema (RDFS), Simple Knowledge Organisation System (SKOS) and Web Ontology Language (OWL)

0 Decisions oriented to favor approaches that can be generalized to make other vocabulary based standards web-friendly, available for use in applications, and easily updated by communities

0 RDA still a work-in-progress, without agreements on maintenance, boundaries, etc.

5/9/12

http://metadataregistry.org/rdabrowse.htm


The Structure, Simplified

0 FRBR in RDA Vocabulary declared as classes0 RDA Properties declared as a ‘generalized’ vocabulary,

with no explicit relationship to FRBR entities0 Subproperties for the generalized elements may be

explicitly related to FRBR entities (using ‘domain’)0 Label/Name includes (Work) or other class to

provide unique name (unless the entity name already appears in the name of the property)

0 Other generalized subproperties usable by others not tied to FRBR

5/9/12


Why Generalized Properties?

0 The generalized properties essential for relating RDA to the larger world0 Mapping will be cleaner using the generalized properties (since

most properties mapped to or mapped from will not be based on FRBR)

0 Extensions can be built more usefully from the generalized properties

0 Generalized properties much more acceptable to non-library

implementers (not often using FRBR)0 The ‘bounded’ properties are the first pass at an

Application Profile for RDA as a whole0 Discussions ongoing about completion of the AP, hopefully this year

5/9/12


Property (Generalized, no FRBR relationship)

Subproperty (with relationship to one FRBR entity)

FRBR Entity

SemanticWeb

Library ApplicationsThe Simple Case: One Property--

One FRBR Entity

5/9/12


Book format

Book format (Manifestation)

Manifestation

SemanticWeb

Library ApplicationsThe Simple Case: One Property--

One FRBR Entity

5/9/12


http://RDVocab.info/Elements/bookFormatManifestation


More Complex RelationshipsThere are multiple techniques used in RDA to make the

connection between FRBR entities and RDA propertiesWe tried to reconcile those in the RDA Vocabularies

Some properties related to more than one FRBR entityRelationships in Appendix J actually include the name of the

FRBR entity in the name and have separate definitions (we re-used this strategy for the FRBR-bounded properties)

Other properties and sub-properties appear multiple times in the text and ERDs, with the same definitions and no indication that they might be repeated elsewhere (we consolidated these)

5/9/12


Property (Generalized, no FRBR relationship)



FRBR Entity

FRBR Entity

SemanticWeb

Library Applications The Not-So-Simple Case: One Property—more than

One FRBR Entity5/9/12


Extent

Extent (Item)

Extent (Manifestation)

FRBR Item

FRBR Manifestation

SemanticWeb

Library Applications The Not-So-Simple Case: One Property—more than

One FRBR Entity5/9/12


Roles: Attributes or Properties?

0 In 2005, the DC Usage Board worked with LC to build a formal representation of the MARC Relators so that these terms could be used with DC

This work provided a template for the registration of the role terms in RDA (in Appendix I) and, by extension, the other RDA relationshipsRole and relationship properties are registered at

the same level as elements, rather than as attributes (as MARC does with relators, and RDA does in its XML schemas)

Applications can choose either approach

5/9/12


“Super” Property


Subproperty (Generalized, no FRBR relationship)

FRBR Entity

SemanticWeb

Library Applications The Roles Case: Properties, Subproperties

and FRBR Entities

Mapping,Etc.

5/9/12


RDA:Creator

RDArole:Composer (Work)

RDArole:Composer

Work

SemanticWeb

Library Applications The Roles Case: Properties, Subproperties

and FRBR Entities

Mapping,Etc.

5/9/12


Aggregated Statements

RDA sets up Publication, Distribution, Manufacture and Production statements very much the way they have been done since catalog card days:Assumed aggregation of Place, Name and Date are obvious

leftovers from catalog cards, and are not necessary to enable indexing or display of those elements together if libraries want to do that

We viewed those aggregations as ‘Syntax Encoding Schemes’ (as defined in the DCAM) and built in ways to accommodate them within the bounded propertiesThose using the generalized properties (outside libraries,

usually) need not be constrained by these traditional aggregations of properties

5/9/12


Aggregated Statement (no domain or range)

Aggregated Statement Subproperty

Range: RDA Syntax Encoding Scheme (Subclass of RDF Datatype)

Dom

ain: FRB

R Entity

Range: [Specific] Encoding Scheme (Subclass)

General Property (no domain or range)

Subproperty

Pre-coordinated Statements: Structure

Option 15/9/12


Publication Statement (no domain or range)

Publication Statement (Manifestation)

Range: RDA Syntax Encoding Scheme (Subclass of RDF Datatype)

Dom

ain: Manifestation

Range: Publication Statement Encoding Scheme (Subclass)

Place of publication (no domain or range)

Place of publication (Manifestation)

Pre-coordinated Statements: Example

Option 15/9/12


What Does This Structure Buy Us?

0 Release from the tyranny of records0 Potential for use with a variety of encodings0 Ability to maintain statements at a more granular level

0 Opportunity to re-think how we build, maintain and share data

0 Potential for sharing data beyond the library silo0 Both directions!

0 A challenge to our old notions of what library data can do and should be doing0 As our users migrate to the web and away from library catalogs,

we need to follow them (and lead them appropriately!)

5/9/12


Finding Our Way AroundThere’s not just one way to align bibliographic data: we don’t have to agree on one ‘authoritative’ mapping

‘Crosswalking’ strategies, aimed at use by particular applications, see that activity as primarily accomplished by networks

Crosswalks only recognize one relationship: sameAs—this is a very blunt instrument 5/9/12


Collapsing and Extending0 We’re starting to envision how mapping and

alignment of elements and concepts can provide a clearer pathway between schemas

0 More specificity for special materials can be accomplished by extending element vocabularies for particular purposes

0 As we build these maps, we can provide different views of data without necessarily having to change the data values themselves

5/9/12


Mapping

0 Building relationships between vocabularies is not necessarily a proprietary, high-level activity tied to particular applications or services

0 http://MARC21rdf.info provides a beginning point for mapping library data that is easily shared and modified to meet a variety of needs

0 New approaches go beyond ‘sameAs’ and suggest the potential for more granular, nuanced relationships

5/9/12

http://MARC21rdf.info/


The sub-property ladder

m21:“Target audience of

books”

m21:“Target audience”

rdfs:subPropertyOf

m21:“Target audience oflanguage material”


music”


visual materials”

Slide: Gordon Dunsire5/9/12


m21:“Target audience”

frbrer:“has intended

audience”

rda:“Intended audience”

dct:“audience”

isbd:“has note on

use or audience”

unconstrained:“Intended audience”

unconstrained:“has note on

use or audience”

rdfs:subPropertyOf

rdfs:subPropertyOf

“Audience” map

Slide: Gordon Dunsire5/9/12


Vocabulary Extension

0 The inclusion of unconstrained properties provides a path for extension of RDA into specialized library communities and non-library communities0 They may have a different notion of how FRBR

‘aggregates’ (For example, a colorized version of a film may be viewed as a separate work)

0 They may not wish to use FRBR at all0 They may have additional properties to use, that

could have a relationship to the RDA properties

5/9/12


RDA:adaptedAs

RDA:adaptedAsARadioScript

hasSubproperty

5/9/12


RDA:adaptedAs


KidLit:adaptedAsAPictureBook

hasSubproperty

hasSubproperty

5/9/12

Extension using Generalized Properties


RDA:adaptedAs


KidLit:adaptedAsAPictureBook

hasSubproperty

hasSubpropertyKidLit:adaptedAsAChapterBook

hasSubproperty5/9/12

Extension using Generalized Properties


Where Are We Going?

0 Many of the older bibliographic standards are undergoing review in order to be ‘alignment ready’ (ISBD, FR Family)0 RDF versions will be available for all0 Application profiles being built

0 MARC21 now available in its entirety as RDF, with mapping in mind

0 New possibilities for interoperability and harmonization are being circulated widely

5/9/12


Links and Contact Info

RDA Vocabularies: http://rdvocab.infoMARC 21 in RDF: http://MARC21rdf.info

Diane: [email protected]

“RDA Vocabularies: Process, Outcome, Use”, DLib Magazine, Jan./Feb. 2010: http://dlib.org/dlib/january10/hillmann/01hillmann.html “A Reconsideration of Mapping in a Semantic World” http://dcpapers.dublincore.org/index.php/pubs/article/view/3622 5/9/12

http://rdvocab.info/

http://MARC21rdf.info/

mailto:[email protected]

http://dlib.org/dlib/january10/hillmann/01hillmann.html

http://dlib.org/dlib/january10/hillmann/01hillmann.html

http://dcpapers.dublincore.org/index.php/pubs/article/view/3622

http://dcpapers.dublincore.org/index.php/pubs/article/view/3622

new world of metadata: growing, shifting, merging

Education

own5912 metadata day

right metadata format

othermethod5912metadata

messengers5912metadata

research librarianship

crafted catalog records

bibliographic data

right way