Multimedia Databases(MMDB)

A Content-Based Image Retrieval Perspective(CBIR)

Types of Media Files

Static media: images and handwriting

Dynamic media: video and sound bytes

Dimensional media: 3D games or CAD)

MMDB Motivation Factors

Acquisition: email, phone, web sites like FLIKR

Generation: camera phones, digital cameras,

Storage: databases design

Processing: power and techniques more sophisticated

Huge increase in multimedia data on computers and their transmission over networks.

Database Support

Databases provide consistency, concurrency, integrity, security and availability of data for the large amount of multimedia data available.

From a user perspective, databases provide functionalities for manipulation and querying the huge collections of stored data.

Media Data Stored

Media data: actual media data representing images, audio, and video that are captured, digitized, processed, compressed and stored.

Media format data: consists of media data format stored after the acquisition, processing, and encoding phases. Examples are sampling rate, resolution, frame rate, encoding scheme etc.

Media keyword data: For example, for a video this might include the date, time, and place of recording , the person who recorded, the scene that is recorded. Also called content descriptive data.

Media feature data: This contains the features derived from the media data. A feature characterizes the media contents. For example, this could contain information about the distribution of colors, the kinds of textures and the different shapes present in an image. This is also referred to as content dependent data.

Image Retrieval (IR)

Image Retrieval is the process of searching and retrieving desired images from a large database.

IR provides resourceful use of prolific image data

The efficiency of implementations have increased over the past two decades

IR Methodology

A simple image retrieval implementation uses individually entered keywords or descriptions of inserted images so that retrieval is performed over the annotations in normal textual forms. 

If an image is poorly or incorrectly annotated, or a poor choice of arbitrary query values are given by the user then the desired output is not received even if it exists in the database.

Therefore, a lot of research has gone into automatic annotation of image description.

Image Features Stored

Color: Red, Blue, Green, etc

Texture: Similarity in grouping of pixels

Shape: Edge detection

Spatial: Spacing of Features

Semantic: Correlated description of image data. E.G: Color = blue, Shape = Large, Texture = smooth, Spatial = Top of image: Sky

Feature Extraction

Image Segmentation: open ended topic

Segment Classification: based off characteristics

Filtering Techniques: Extract image features such as texture by passing images through a filter

Feature Application In DB

Users could supply a range for color, texture, or shape for queries

Features can be generated on a typical semantic set for automatic annotation of new pictures

Content based image Retreival (CBIR)

Avoids the necessary use of textual descriptions

Organizes digital archives by visual content

Retrieves images based on visual similarity to a user-supplied query image or image features.

Query Types

Keyword: common text searching techniques

Feature: Ex. Draw area for location and size. Select color regions. Select shape. B+-tree is traversed based off given index value.

Semantic: Provide words to describe feature sets that are used to query a database

Composite: Index involves combination of above

Query examples from CIBR at the end of the early years

Content-Based

Straight-forward implementation is each feature is used as an index. Not very efficient for querying

Create an index as described earlier as a combination of region classification, spatial location, shape, and color.

EX. 20-bit index key: 3bits location, 8 bits color, 4 bits size, 5 bits shape. B+-tree indexing method is used.

Relevance Feedback

A query modification technique attempts to improve retrieval performance through iterative feedback and query refinement.

Used in ALIPR

Data Flow From CBIR at the end of early years

IR Implementation Examples

Yahoo or Google Image Searches: based mainly on annotated description and filename

Automatic Linguistic Indexing of Pictures (ALIPR): learning algorithm that annotates with feedback

Future Work

The open ended nature of image segmentation restricts the accuracy of object recognition. As segmenters improve so will the databases capability.

The integration of image retrieval can be implemented in computer vision applications.

Many researchers believer that image retrieval has grown out of its infancy and now focus will be on applications and proliferating algorithms into indivduals lives.

Bibliography

Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. 40, 2 (Apr. 2008), 1-60. DOI= http://doi.acm.org/10.1145/1348246.1348248

Stanchev P., Using Image Mining for Image Retrieval, IASTED International Conference “Computer Science and Technology”, May 19-21, 2003, Cancun, Mexico.

"Multimedia Database." Information Technology Portal (IT Portal) - India. Web. 04 Dec. 2009. <http://www.peterindia.net/MultimediaDatabase.html>.

"Image retrieval -." Wikipedia, the free encyclopedia. Web. 04 Dec. 2009. <http://en.wikipedia.org/wiki/Image_retrieval>.

Smeulders, “Content-based image retrieval at the end of the early years” A.W.M.Journal:IEEE transactions on pattern analysis and machine intelligence, 2000, Vol:22, 12, 1349