Multimedia Databases(MMDB)
A Content-Based Image Retrieval Perspective(CBIR)
Types of Media Files
Static media: images and handwriting
Dynamic media: video and sound bytes
Dimensional media: 3D games or CAD)
MMDB Motivation Factors
Acquisition: email, phone, web sites like FLIKR
Generation: camera phones, digital cameras,
Storage: databases design
Processing: power and techniques more sophisticated
Huge increase in multimedia data on computers and their transmission over networks.
Database Support
Databases provide consistency, concurrency, integrity, security and availability of data for the large amount of multimedia data available.
From a user perspective, databases provide functionalities for manipulation and querying the huge collections of stored data.
Media Data Stored
Media data: actual media data representing images, audio, and video that are captured, digitized, processed, compressed and stored.
Media format data: consists of media data format stored after the acquisition, processing, and encoding phases. Examples are sampling rate, resolution, frame rate, encoding scheme etc.
Media keyword data: For example, for a video this might include the date, time, and place of recording , the person who recorded, the scene that is recorded. Also called content descriptive data.
Media feature data: This contains the features derived from the media data. A feature characterizes the media contents. For example, this could contain information about the distribution of colors, the kinds of textures and the different shapes present in an image. This is also referred to as content dependent data.
Image Retrieval (IR)
Image Retrieval is the process of searching and retrieving desired images from a large database.
IR provides resourceful use of prolific image data
The efficiency of implementations have increased over the past two decades
IR Methodology
A simple image retrieval implementation uses individually entered keywords or descriptions of inserted images so that retrieval is performed over the annotations in normal textual forms.
If an image is poorly or incorrectly annotated, or a poor choice of arbitrary query values are given by the user then the desired output is not received even if it exists in the database.
Therefore, a lot of research has gone into automatic annotation of image description.
Image Features Stored
Color: Red, Blue, Green, etc
Texture: Similarity in grouping of pixels
Shape: Edge detection
Spatial: Spacing of Features
Semantic: Correlated description of image data. E.G: Color = blue, Shape = Large, Texture = smooth, Spatial = Top of image: Sky
Feature Extraction
Image Segmentation: open ended topic
Segment Classification: based off characteristics
Filtering Techniques: Extract image features such as texture by passing images through a filter
Feature Application In DB
Users could supply a range for color, texture, or shape for queries
Features can be generated on a typical semantic set for automatic annotation of new pictures
Content based image Retreival (CBIR)
Avoids the necessary use of textual descriptions
Organizes digital archives by visual content
Retrieves images based on visual similarity to a user-supplied query image or image features.
Query Types
Keyword: common text searching techniques
Feature: Ex. Draw area for location and size. Select color regions. Select shape. B+-tree is traversed based off given index value.
Semantic: Provide words to describe feature sets that are used to query a database
Composite: Index involves combination of above
Query examples from CIBR at the end of the early years
Content-Based
Straight-forward implementation is each feature is used as an index. Not very efficient for querying
Create an index as described earlier as a combination of region classification, spatial location, shape, and color.
EX. 20-bit index key: 3bits location, 8 bits color, 4 bits size, 5 bits shape. B+-tree indexing method is used.
Relevance Feedback
A query modification technique attempts to improve retrieval performance through iterative feedback and query refinement.
Used in ALIPR
Data Flow From CBIR at the end of early years
IR Implementation Examples
Yahoo or Google Image Searches: based mainly on annotated description and filename
Automatic Linguistic Indexing of Pictures (ALIPR): learning algorithm that annotates with feedback
Future Work
The open ended nature of image segmentation restricts the accuracy of object recognition. As segmenters improve so will the databases capability.
The integration of image retrieval can be implemented in computer vision applications.
Many researchers believer that image retrieval has grown out of its infancy and now focus will be on applications and proliferating algorithms into indivduals lives.
Bibliography
Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. 40, 2 (Apr. 2008), 1-60. DOI= http://doi.acm.org/10.1145/1348246.1348248
Stanchev P., Using Image Mining for Image Retrieval, IASTED International Conference “Computer Science and Technology”, May 19-21, 2003, Cancun, Mexico.
"Multimedia Database." Information Technology Portal (IT Portal) - India. Web. 04 Dec. 2009. <http://www.peterindia.net/MultimediaDatabase.html>.
"Image retrieval -." Wikipedia, the free encyclopedia. Web. 04 Dec. 2009. <http://en.wikipedia.org/wiki/Image_retrieval>.
Smeulders, “Content-based image retrieval at the end of the early years” A.W.M.Journal:IEEE transactions on pattern analysis and machine intelligence, 2000, Vol:22, 12, 1349