ch 4 : multimedia database

Ch 4 : Multimedia Database

Science and Technology Faculty

Informatics

Arini, ST, MTarinizul@gmail. [email protected]

I. Multimedia Data

• Multimedia data : – From the presentation view point, multimedia data is huge

& involves time dependent /independent characteristics.• Presentation & subsequent interactions

– Temporal : video, audio, animation

– Spatial : image

– Complex structure demands deriving semantics from contents

» Need for Content-Based Access : color, texture, shape

– Collaborative Support Environment

– Consisting of • Alphanumeric : text, hypertext

• Image : bitmap graphic (2D) and vector graphic (3D)

• Video : analog (videocassette, laserdisc) and digital (Quick time, real player, win player )

• Animation : 2D, 3D (virtual reality : modeling object)

• Audio : digital audio

II. Multimedia Data Management

• Multimedia data management system is to allow efficient storage, manipulation using of multimedia data in all its varied forms.– Storage, retrieval, integration and

presentation requirements of multimedia data differ significantly from those for the traditional data.

• Implemented to : Multimedia Database– Content Multimedia : MM-DBMS

III. Multimedia Database Management System (MM-DBMS) • A framework that manages different types of data

potentially represented in a wide diversity of formats on a wide array of media sources – To provide a suitable environment for using and managing

multimedia database information

• MM-DBMS must include : – The traditional DBMS functions (e.g., database definition

and creation, data retrieval, data access and organization, data independence, privacy, integration, Integrity control, version control and concurrency support but applied to various multimedia data types.

– The functional requirements imposed on a MMDBMS can be grouped into two categories :

• Data representation requirements

• Data manipulation requirements.

4.1. MM-DBMS Functional Requirements4.1. MM-DBMS Functional Requirements

• Data representation requirements– Support for Generalization/Specialization Hierarchy (Book;

Newspaper)

– Attribute Specification (Title, author, number)

– Ordering of Documents– The presentation of the paragraphs, images, and drawings of a multimedia

document could depend on the users accessing the document.

– Data independence

• Separate the database and the management from the application programs

• Data manipulation requirements.– Integration

• Data items do not need to be duplicated for different programs

– Concurrency control

• allows concurrent transactions

5

– Data manipulation requirements.– Persistence

• Data objects can be saved and re-used by different transactions and program invocations

– Privacy• Access and authorization control

– Integrity control• Ensures database consistency between transactions

– Recovery• Failures of transactions should not affect the persistent

data storage– Query support

• Allows easy querying of multimedia data– Flexible Acquisition– Efficient Storage– Efficient Retrieval

MM Database 6

4.2. Data StructureMultimedia data

Raw Data

Registering Data

Descriptive Data

(Uncompressed Image)

(Size & coding details of raw data)

(Textual numerical annotations)

Media Raw Registering Descriptive

Text Characters Coding scheme (ASCII), length / end symbol

Key words, information for structuring

Images Pixels Height/ Width of picture, Mode of Compression, if JPEG, tables for quantization purpose

Pic.Date = 21/04/07

Pic.Reason = Birthday

Etc

Motion Video Sequence

Pixels Frames/second, coding details, frame types…

Scene description

Audio Sample sequence

Audio coding (PCM,…) , resolution of samples

Content of audio passages in short form

4.3. Data Model

• Traditional : – Relational Data Model

• Store everything as First Normal Form tables• Access, Oracle, MySQL

• Semantic : – Object Oriented Data Model

• Store everything as classes of objects• ODL, OQL

– Object-relational Data Model• Fundamentally relations but are not First Normal

Form table form• POSTGRES

4.4. MM Database Architectures4.4. MM Database ArchitecturesBased on Principle of Autonomy• Each media type is organized in a media-specific manner suitable for

that media type

• Need to compute joins across different data structures

• Relatively fast query

processing due to

specialized structures

• The only choice for legacy data banks

MM Database 9

MM Database Architectures (cont.)MM Database Architectures (cont.)Based on Principle of Uniformity• A single abstract structure to index all media types

• Abstract out the common part of different media types (difficult!) - metadata

• One structure - easy implementation

• Annotations for different media types

MM Database 10

MM Database Architectures (cont.)MM Database Architectures (cont.)Based on Principle of Hybrid Organization• A hybrid of the first two. Certain media types use their own

indexes, while others use the "unified" index

• An attempt to capture the advantages of the first two

• Joins across multiple data sources using their native indexes

MM Database 11

Organizing Multimedia Data Based on theOrganizing Multimedia Data Based on thePrinciple of UniformityPrinciple of Uniformity

• Consider the following statements about media data and they may be made by a human or may be produced by the output of an image/video/text content retrieval engine.– The image photol.gif shows Jane Shady, “Big Spender”

and an unidentified third person, in Sheung Shui. The picture was taken on January 5, 1997.

– The video-clip videol.mpg shows Jane Shady giving “Big Spender” a briefcase (in frames 50-100). The video was obtained from surveillance set up at Big Spender’s house in Kowloon Tong, in October, 1996.

– The document bigspender.txt contains background information on Big Spender, a police’s file.

MM Database 12

Querying SMDSs (Uniform Querying SMDSs (Uniform Representation)Representation)Querying SMDS based on top of SQL. Basic

functions include:• FindType(Obj): This function takes a media object Obj as

input, and returns the output type of the object. For example,

FindType(iml.gif) = gif.

FindType(moviel.mpg) = mpg.

• FindObjWithFeature(f): This function takes a feature f as input and returns as output, the set of all media objects that contain that feature. For example,

FindObjWithFeature(john)= {iml.gif,im2.gif,im3.gif,videol.mpg:[1,5]}. FindObjWithFeature(mary)= {videol.mpg:[1,5],videol.mpg:[15,50]}.

MM Database 13

4.5. Data Retrieval : 4 Types

1. Conventional database system – This is the widely-used approach to manage and

search for structured data. – All data in a database system must conform to some

predefined structures and constraints (i.e., schema’s).

– To formulate a database query the user must specify which data objects are to be retrieved, the database tables from which they are to be extracted and predicate on which the retrieval is based.

– A query language for the database will generally be of the artificial kind, one with restricted syntax and vocabulary, such as SQL.


2. Information retrieval (IR) system : – IR system is mainly used to search large text

collections, in which the content of the (text) data is described by an indexer using keywords or a textual abstract, and keywords or natural language is used to express query demands.

• Use Keyword Querying

– For example for an image or video we have to describe it in words or in a way need to store lot of metadata (textual form).

4.5. Data Retrieval : 4 Types3. Content based retrieval (CBR) system

– This approach is used to retrieve desired multimedia objects from a large collection on the basis of features (such as color, texture and shape, etc.) that can be

automatically extracted from the objects themselves. • Use Visual querying and semantic querying

– Although keyword can be treated as a ”feature” for text data, traditional information retrieval has much more higher performance than content-based retrieval because keyword has the proven ability to represent semantics, while no features have shown convincing semantic describing ability.

– But major drawback of this method is that it lacks precision.


4. Graph or tree pattern matching : – This approach aims to retrieve object sub-

graphs from an object graph according to some denoted patterns.

V. 2 Search Types

• Metric• Combinatorial : image, audio (music), video

VI. MM-DBMS Structure

• Data Analysis– Data structure (4.2 above)

• Data Modeling– Data Model (4.3 above)

• Data Storage – DBMS Architecture (4.4 above)

• Data Retrieval : – 4 Types data retrieval (depend on query and

searching method) (V. above)

• Multimedia Communication– Multimedia Networking