metadata for digital video: developing an mpeg-7 descriptive application profile for digital video -...

Metadata for Digital Video:

DEVELOPING AN MPEG-7 DESCRIPTIVE APPLICATION PROFILE FOR DIGITAL VIDEO --with mapping to Dublin Core

Participants:

ViDe Video Access WG

Co-Chairs: Grace Agnew, Rutgers

Dan Kniesner, Oregon Health & Sciences University

Rutgers Libraries:

Yang Yu, Database Programmer

Ruth Bogan, Mary Beth Weber, Catalogers


Definition of Metadata

Data about Data

Data that describes, defines or manages data

“Pure” metadata has meaning only in relation to the primary data that is being described.


Role of Metadata

Facilitate organization, discovery, interpretation, preservation

Enable implementer and end user to make sense of information: appropriateness, context, relative value –Shared understanding!


METADATA SCHEMA

Creates standardized metadata:

Understandable by the archive and by the user--”shared understanding.”

Shareable across repositories

Can be mapped to other schema to repurpose the metadata.

Maintained by a standards body for durability and consistency


METADATA SCHEMA COMPONENTS Data Element - Atomic Unit of Meaning- Community Defined

Attribute - Refines, Extends, Interprets data element

Value - Information unique to each data element instance

Constraint - Order imposed on data element expression for consistency; semantic viability

Label - contextual instance of data element name. “How the data element displays on the web for the end user.”


METADATA RECORD

Populated with information (value)

Data Elements

According to rules (“Schema”)

Author: Agnew, Grace

Data Element

VALUE AACR2 Formatting Rule:Last name, first name.


SCHEMA VS. APPLICATION PROFILE

Application Profiles

Customize standardized metadata to support community needs.

Utilize one or more schema, but often add the community’s own data elements

Define constraints, such as mandatory, recommended or optional;

Develop controlled vocabularies, data types and formatting principles for information within each data element.


CUSTOMIZING METADATA

+ Support for unique needs of users

Adds to metadata universe of knowledge

Supports unique commonalities among a distributed user base

- May not be developed robustly

Interoperability may be compromised


Description Needs for Digital Video and Audio:

Consecutive mediumContexual Metadata that changes over time as the information changes

Media streamed in multiple formats (RealVideo, MPEG1, etc.) and stored on multiple publication formats—tape, DVD, etc.

Nontextual indexing (keyframe summaries, speech recognition, etc.) as important as textual indexing.


Dublin Core:

• Evolved with the web itself

•Encourage web authors to put structured information in <meta> tags

• Goals: Simplicity and Interoperability

• Can be expressed as HTML or XML

•15 optional, repeatable data elements

•Currently in version 1.1

•http://www.dublincore.org


From “Description of Dublin Core Elements”http://purl.oclc.org/metadata/dublin_core_elements

15 OPTIONAL, REPEATABLE ELEMENTS

Content IntellectualProperty

Instantiation

Title Creator DateSubject Publisher TypeDescription Contributor FormatSource Rights IdentifierLanguageRelationCoverage

Dublin Core


• Provides a great deal of flexibility.• Easy to learn.• Ensures interoperability with other schemes.• Good transport protocol when expressed as XML

+

- • Lacks support for multiple formats• Lacks support for seriality• Technical description (formats, containers,

extent, etc.) is weak and not standardized.• Support for relationships (whole/part;

sibling/related) is very weak

How Useful is Dublin Core?


Dublin Core

•Interoperability compromised because no consensus on “values”—e.g., how to fill the fields.

ViDe Tried!

• Issued Video AP in 2001

• Very functional but very complex.

• Violated the “dumb down” rule

Decision: Dublin Core is not rich and functional enough for DV


Synchronization between content and description

Supports description for intellectual content and all physical versions of that content

Recursive and hierarchical—whole/part (“segment”) description. Strong support for related works

Native XML with some customized extensions

MPEG-7: Multimedia Content Description Interface


MPEG-7 Description Tools

Description Schemes (structure) and Descriptors (features)Figure 1: Overview of the DSsFigure 2: Overview of the DSs

Datatype &Structures

Link & MediaLocalization

Models

Navigation &Access

Content management

Content description

Collection &Classification

Summaries

Variations

Content organization

Creation &Production

Media Usage

Semanticstructure

Spatio-temporalstructureAspects

User Interaction

UserPreferences

UsageHistory

Roots and Top-level Elements

PackagesSchemaTools

Partitions andDecompositions

Basicelements

Audio and Visualfeatures


Start: 00:02:00;1

End: 00:07;32;7

Segment Title: introductory movement

Start: 00:07:33;1

End: 01;02;53;2

Segment Title: Main Theme

Start: 01:07:03;1

End: 01:33;6

Segment Title: Oboe Solo

MPEG-7 - Metadata synched to media.MPEG7 AP for the MPEG4 Metadata stream recently

released


MPEG-7

Non-textual indexing - melody and speech recognition, color, shape, scene changes, etc.

Textual format/Binary Format completely equivalent. You can use any functionality in textual or nontextual form.


MPEG-7

MPEG-7

TextualEncoder

MPEG-7

TextualDecoder

Contentdescription

MPEG-7

BinaryEncoder

MPEG-7

BinaryEncoder

Content

Access Unit -Textual Format

Access Unit

BinaryFormat


Does not support description of analog or textual resources

High-level textual description of component parts (“table of contents”) does not exist.

Some duplication of descriptive information across MPEG7 descriptive schemes

Documentation, examples and widespread adoption as a descriptive metadata standard is weak.

MPEG-7: Multimedia Content Description Interface


PROBLEM:

•MPEG-7 highly functional but steep learning curve. Training, examples - nonexistent

• Dublin Core – simple to employ (although not simple to employ well!). Widespread use, but not enough value for DV


The Structure of Information (IFLA)

Work

Expression Expression

Distinct intellectual or

artistic creation

Intellectual or artistic realization of a work (“interpretation”)

ManifestationManifestation Manifestation

ItemUnique physical

instance of a manifestation.

Physical manifestation of an expression. May differ in physical format, but not in content or interpretation


Intellectual / artistic content

Physical recording of content

Single physical representation of a

recording

A

B

S

T

R

A

C

T

I

O

N

GONE WITH THE WIND

InterpretationNovel MovieScript

WORK

EXPRESSION

MANIFESTATION Paper

PDF

HTML

70 MM Film

35 MM Film

DVD

MPEG2

Copy in Blockbuster, Atlanta, GA

24 Reels of film, MGM Archive

ITEM


Dublin Core vs. MPEG7 – The Challenges

• MPEG7 is a structured, hierarchical schema.

• “Work” described in CreationInformation DS

• Manifestation/Item described in MediaInformation and UsageInformation DSs

•Dublin Core is a “flat” schema that mixes “work” or intellectual content with single manifestation/item description

(“1:1 principle”)


MANIFESTATION

CREATOR

TITLE

SUBJECT

DATE, etc.

IDENTIFIER

FORMAT

RIGHTS

IDENTIFIER

FORMAT

RIGHTS

CreationInformation

MediaProfileUsageAvailability

MediaProfileUsageAvailability

MediaInstance


MediaInformation

MediaIdentificationMediaProfile

0..11

EntityIdentifier

MediaTranscodingHintsComponentMediaProfile MediaFormat

AudioDomain

CreationTool

1 0.. 0..1

0..1

Shared Objects

1..

1

VideoDomain ImageDomain

MediaInstanceMediaQuality

0..

0.. 0..1 0..1 0..

Work/Manifestation

Manifestation

Item

MPEG-7


• MPEG-7 includes descriptive, technical and structural information

• Dublin Core – Descriptive only, except for rights

• MPEG-7 includes meta metadata (metadata about the record itself)

• Dublin Core has administrative metadata (A-Core) in development, for linking to descriptive metadata


CREATOR

Person or body primarily responsible for content of the resource.

•Non-specificity of Dublin Core problematic:

–Need for meaningful roles

–issue of type--person, organization

–contact information

ViDe Decision: Add the attribute “Role” to Creator. Weakly supports the “dumb down” rule


CREATOR

MPEG-7 Creator includes:

• Role

• Agent DataType

• Agent DataType includes:

• Sub-types: PersonType; PersonGroupType; OrganizationType

•Can include contact information


CREATORMPEG-7<Creator>

<Role><Name>Speaker</Name>

</Role><Agent xsi:type=“PersonType”>

<Name><GivenName>Grace</GivenName><FamilyName>Agnew</FamilyName>

</Name> </Agent></Creator>


CREATOR

Dublin Core

<Creator> Grace Agnew </Creator>

Or

<Creator> Agnew, Grace </Creator>


CREATOR

“DC Dumb Down”

<Creator> Speaker </Creator>

and

<Creator> Grace Agnew </Creator>


DATE

• “Date associated with an event in the lifecycle of a resource.”

• Issue: in print world/legacy catalogs, “date” is used to filter for currency of information. In Dublin Core, “Coverage” is used instead.

• Issue: Date and country of release are important concepts for moving images, particularly feature films.


DATE – A COMPLEX MAPPING:

CreationInformationCreationCoordinates—CreationDate

CreationInformationClassificationReleaseDate

MPEG-7:

Dublin Core

Date.Created

Date.Issued

Not Addressed—Date of Digitization. Technical, rather than Descriptive metadata.


Date of Creation Map


Date of Release / Issue Map


WE BROKE THE SCHEMA…


Dilemma: MPEG-7 schema currently allows only one subject element per description

…<minoccurs=“0”/>

Answer: We edited the schema for our database and inserted a note explaining what we did:

<!—Edited by Rutgers University Libraries to support unlimited subject elements--!>

…<minoccurs=“0”, maxoccurs=“unbounded”/>

SUBJECT


… AND THE SCHEMA BROKE US


MPEG-7 ISSUE – “DURATION”

PROBLEM: MPEG-7 does not include a “content description” data element for “Duration”

ViDe Solution: Duration is critical information for sequential media. Use SegmentDS, even though it is intended for “Navigation & Access”


SegmentDS: Included in Navigation and Access: intended for nontextual bitstream processing.


MediaInformation MediaProfile MediaInstance MediaLocator MediaURI Identifier

MediaInformation MediaProfile MediaFormat Content

Controlled voc: MPEG-7: audio, image, video, audiovisual

MediaInformation MediaProfile MediaFormat Medium Format Medium

MediumCS (CD, hard disk, etc. Do not use for files accessible via web)

MediaInformation MediaProfile MediaFormat FileFormat Format Medium MPEG7MediumCS

MediaInformation MediaPofile MediaFormat System format System

SystemCS (PAL, SECAM, NTSC, Real, WindowsMedia, etc.

MediaInformation MediaProfile MediaFormat FileSize Format FileSizeMediaInformation MediaProfile MediaFormat VisualCoding Resolution Format Resolution image (resolution in dpi)MediaInformation MediaProfile MediaFormat VisualCoding Format Format VisualCoding VisualCodingFormatCS

MediaInformation MediaProfile MediaFormat VisualCoding Format ColorDomain Format ColorDomainimage;video binary; color; graylevel; colored

MediaInformation MediaProfile MediaFormat VisualCoding Frame Height Format FrameHeight image;videoFrame Width Format FrameWidth image;videoFrame Rate Format FrameRate video (in Hz)

MediaInformation MediaProfile VisualCoding BitRate BirRate video

MediaInformation MediaProfile MediaFormat AudioCoding Format Format AudioCodingaudio; AudioCodingFormatCS

MediaInformation MediaProfile MediaFormat AudioCoding Sample Rate FormatAudioCodingSample

MediaInformation MediaProfile MediaFormat AudioCoding Presentation FormatAudioCodingPresentation AudioPreserntation CS

SegmentDS (VideoSegment, AudioSegment, etc.) MediaTimePoint 0:00:00VideoSegment MediaDuration Format ExtentUsageInformation Availability Rights Rights

FORMAT: Part of the complex mapping doc!


Next Steps:

• Release the database in May

• Cataloging utility for Moving Image Collections—looking for small science DV collection!

• Refine, develop registry and users’ guide

• Develop music AP for Rutgers’ Institute for Jazz Studies collection

• Test in some Media Asset Management systems.

Metadata for Digital Video: Validates all entries against the MPEG7

Schema for export


MORE INFO

Websites:

http://www.vide.net

http://gondolin.rutgers.edu/MIC

[email protected]

[email protected]

People:

metadata for digital video: developing an mpeg-7 descriptive application profile for digital video -...

Documents

digital video

standardized metadata

definition of metadata

metadata record

customizing metadata

data pure metadata

role of metadata

video ap