At the heart of multimedia informa-tion systems lies the multimediadatabase management system.Traditionally, a database consists of

a controlled collection of data related to a givenentity, while a database management system, orDBMS, is a collection of interrelated data with theset of programs used to define, create, store,access, manage, and query the database. Similarly,we can view a multimedia database as a controlledcollection of multimedia data items, such as text,images, graphic objects, sketches, video, andaudio. A multimedia DBMS provides support formultimedia data types, plus facilities for the cre-ation, storage, access, query, and control of themultimedia database.

The different data types involved in multi-media databases might require special methodsfor optimal storage, access, indexing, andretrieval. The multimedia DBMS should accom-modate these special requirements by providinghigh-level abstractions to manage the differentdata types, along with a suitable interface for theirpresentation.

Before detailing the capabilities expected of a

multimedia DBMS and the requirements such sys-tems should meet, we must first consider the char-acteristic nature of multimedia information. Thenwe can discuss the issues facing multimediaDBMSs.

Nature of multimedia dataWe analyze the composition and characteris-

tics of multimedia data from several perspectives.These include information overload, inadequacyof textual descriptions, multiplicity of data types,spatial and temporal characteristics, and huge vol-umes of data.

The integration of multimedia data types frommultiple sources uniquely characterizes multi-media information systems. The data types foundin a typical multimedia database include

❚ text;

❚ images: color, black and white, photographs,maps, and paintings;

❚ graphic objects: ordinary drawings, sketches,and illustrations, or 3D objects;

❚ animation sequences: images or graphicobjects, (usually) independently generated;

❚ video: also a sequence of images (calledframes), but typically recording a real-life eventand usually produced by a video recorder;

❚ audio: generated from an aural recordingdevice; and

❚ composite multimedia: formed from a combi-nation of two or more of the above data types,such as an intermix of audio and video with atextual annotation.

Some multimedia data types such as video,audio, and animation sequences also have tem-poral requirements, which have implications ontheir storage, manipulation, and presentation.The problems become more acute when variousdata types from possibly disparate sources must bepresented within or at a given time. Similarly,images, graphics, and video data have spatial con-straints in terms of their content. Usually, indi-vidual objects in an image or a video frame havesome spatial relationship between them. Suchrelationships usually produce some constraintswhen searching for objects in a database.

24 1070-986X/97/$10.00 © 1997 IEEE

MultimediaDatabaseManagement—Requirementsand Issues

Donald A. AdjerohThe Chinese University of Hong Kong

Kingsley C. NwosuLucent Technologies

Multimedia Database Systems

The spatial,temporal, storage,retrieval, integration,and presentationrequirements ofmultimedia datadiffer significantlyfrom those fortraditional data. Amultimedia databasemanagement systemprovides for theefficient storage andmanipulation ofmultimedia data inall its varied forms.We look into thebasic nature ofmultimedia data,highlight the needfor multimediaDBMSs, and discussthe requirementsand issues necessaryfor developing suchsystems.

.

Huge volumes of data also characterize multi-media information. For instance, to store anuncompressed image of 1024 ¥ 728 pixels at 24 bitsper pixel requires a storage capacity of about 2Mbytes. With a 20:1 compression ratio, the storagerequirement could be reduced to about 0.1 Mbyte.If we consider a video example, a 10-minutesequence of the same image at 30 frames per sec-ond requires about 38,000 Mbytes of storage,reducible to about 380 Mbytes with a compressionratio of 100:1. The potential for huge volumes ofdata involved in multimedia information systemsbecome apparent when you consider that a moviecould run as long as two hours, and a typical videorepository would house thousands of movies.

An old adage says that a picture is worth morethan a thousand words. However, representingmultimedia information as pictures or imagesequences poses some problems for informationretrieval due to the limitations of textual descrip-tions of a multimedia experience and the massiveinformation available from it. The potential infor-mation overload means that users may find it dif-ficult to make precise requests during informationretrieval. The limitations of textual descriptionsalso imply the need for content-based access tomultimedia information. Users need multiple cues(such as shape, color, and texture) that are rele-vant to the multimedia content.

Another characteristic of multimedia informa-tion is that interaction with such informationtypes usually involves long-duration operations(such as with video data), and sometimes, withmore than a single user (as is typical in collabora-tive support environments). However, in collabo-rative environments, it is expected that mostmultimedia data are likely to be accessed in a read-only mode. This assumption can be used tofacilitate the provision of concurrency controlalgorithms.

Purpose of a multimedia DBMSA multimedia database management system

provides a suitable environment for using andmanaging multimedia database information.Therefore, it must support the various multimediadata types, in addition to providing facilities fortraditional DBMS functions like database defini-tion and creation, data retrieval, data access andorganization, data independence, privacy, inte-gration, integrity control, version control, andconcurrency support.

The functions of a multimedia DBMS basicallyresemble those of a traditional DMBS. However,

the nature of multimedia information makes newdemands—including determining what is neededand how to provide that functionality.

Using the general functions provided by a tra-ditional DBMS as a guide, we can describe the pur-poses of a multimedia DBMS as follows:

❚ Integration. Ensures that data items need not beduplicated during different program invoca-tions requiring the data.

❚ Data independence. Separation of the databaseand the management functions from the appli-cation programs.

❚ Concurrency control. Ensures multimedia data-base consistency through rules, which usuallyimpose some form of execution order on con-current transactions.

❚ Persistence. The ability of data objects to persist(survive) through different transactions andprogram invocations.

❚ Privacy. Restricts unauthorized access and mod-ification of stored data.

❚ Integrity control. Ensures consistency of the data-base state from one transaction to anotherthrough constraints imposed on transactions.

❚ Recovery. Methods needed to ensure that resultsof transactions that fail do not affect the per-sistent data storage.

❚ Query support. Ensures that the query mecha-nisms are suited for multimedia data.

❚ Version control. Organization and managementof different versions of persistent objects,which might be required by applications.

In concurrency control, a transaction is asequence of instructions executed either com-pletely or not at all. In the latter case, the databaseis restored to its previous state. Defining theappropriate granularity for concurrency is a prob-lem in multimedia databases. Traditional data-bases use a single record or table as the unit ofconcurrency; multimedia databases typically usea single object (or composite object) as the logicalunit of access. Thus the single multimedia objectcould form the unit of concurrency.

In achieving persistence, a simple method is to

25

July–Septem

ber 1997

.

store the multimedia files in some operating sys-tem files. However, the huge data volumes makethis approach costly to implement. Moreover, thesystem also needs to store the multimedia meta-data and possibly composite multimedia objects.Thus, most multimedia DBMSs classify the data aseither persistent or transient and store only per-sistent data after transaction updates. Transientdata are used only during program or transactionexecution and are removed afterwards.

Traditionally, a query selects a subset of thedata objects based on the user’s description (usu-ally some form of query language) of what data toaccess. A query usually involves various attribut-es, possibly keyword-based or content-oriented,and is usually interactive. Thus, functions for rel-evance feedback and query formulation, similari-ty (rather than exact) matches, and mechanismsfor displaying ranked results are important in amultimedia DBMS.

Version control becomes important when apersistent multimedia object is updated or modi-fied, as some applications might need to accessprevious states of the object. A DBMS providessuch access through versions of the persistentobjects. For a multimedia DBMS, the huge vol-umes of data reinforces the importance of effi-ciently organizing such versions. Moreover, theavailable storage might limit the provision of ver-sions. In addition, version management mayinvolve not only versions of single objects, butalso versions of the complex objects that make upthe multimedia database.

The special nature of multimedia data alsomakes it important to support new special func-tions. These include object composition and

decomposition, management of huge volumes ofmultimedia data, effective storage management,and information retrieval and handling of spatialand temporal data objects. Khoshafian and Baker1

and Kim2 provide more detailed discussions ofsome of these issues.

Requirements for the multimedia DBMSFor the multimedia DBMS to serve its expected

purpose, it must meet certain special require-ments. Khoshafian and Baker1 described a multi-media DBMS architecture and the interaction ofthe different components needed to provide theservices expected. The requirements are dividedinto the following broad categories:

❚ Traditional DBMS capabilities

❚ Huge capacity storage management

❚ Information retrieval capabilities

❚ Media integration, composition, andpresentation

❚ Multimedia query support

❚ Multimedia interface and interactivity

❚ Performance

In addressing these requirements when build-ing a multimedia database system, one must alsoaddress several other questions to achieve fullfunctionality, including

❚ How to build a multimedia database systemthat encompasses several application domains(that is not restrictive in terms of its domainapplicability)?

❚ What are the levels of granularity for informa-tion decomposition, storage, and manage-ment? And how the underlying techniquesand structures can be mapped and used on theunits of data?

❚ Knowing the data compositions of a multimediadatabase, how can one reliably and efficientlydevelop a query language that supports the myr-iad access methods associated with and neces-sary for the diverse object types? How will thequery language support the multimedia data’sdifferent characteristics and morphologies?

26

IEEE

Mul

tiM

edia

.

Interfacemanager

Mediaintegrator

Transactionmanager

Querymanager

Storagemanager

Multimedia DMBS engine

Directorymanager

Objectmanager

Presentationmanager

Contextbase

manager

Performance

Semanticinformation

manager

Datamigrationmanager

Configurationmanager

Database

Figure 1. A high-level

architecture for a

multimedia DBMS that

meets the requirements

for multimedia data.

❚ What kind of presentation infrastructure willthe multimedia system have to accommodatethe diverse presentation requirements andmodes for the different multimedia data? Howcan one synchronize presentations to supportthe temporal and spatial requirements of thedifferent multimedia data?

❚ Given that different media types have differingmodification and update requirements, howwill the system update different componentsof the multimedia session? What levels of gran-ularity will those updates have?

Figure 1 shows a sample high-level architecturefor a multimedia DBMS that addresses some of therequirements that have been discussed.3 This con-figuration includes most of the managementmodules associated with a traditional DBMS. Inaddition, it contains some of the modules that arerequired specifically for multimedia data manage-ment, such as the media integrator and objectmanager. However, most of the additions to thetraditional DBMS are external to the core of themultimedia DBMS. These include the presenta-tion, interface, and configuration managers. Theconfiguration also includes a context-base andsemantic information manager, which are part ofthe performance module.

Huge capacity storage managementThe storage requirements in multimedia systems

can be characterized by their huge capacities andthe storage system’s hierarchical (pyramidal) orga-nization (see Figure 2). Hierarchical storage placesthe multimedia data objects in a hierarchy ofdevices, either online, near-line, or offline. In gen-eral, the highest level provides the highest perfor-mance, highest cost, smallest storage capacity, andleast permanence. Note, however, that permanenceimproves—at significant additional cost—with theuse of nonvolatile random access memory.

Another unique use of this hierarchical storageorganization is that the higher levels of the hier-archy can be used to store smaller abstractions (orrepresentations) of the actual multimedia data,which can be used to facilitate faster browsing andpreviewing of the database content.

Cost and performance (in terms of access time)decrease as we go down the hierarchy (pyramid),while storage capacity and permanence increase.Typically, in most multimedia storage systems thehighest level of storage is (volatile) random accessmemory, followed by magnetic disk drives. These

provide online services. Optical storage devicesprovide the next level of storage. Online in somecases, they are near-line (like jukeboxes) in mostcases. The lowest level in the storage hierarchyrepresents offline storage devices, including mag-netic tapes, optical disks, and so forth. These mayor may not be directly connected to the computer.They offer the highest storage capacity and per-manence but provide the least performance interms of access time.

A multimedia DBMS must therefore manage andorganize multimedia data stored at any level in thehierarchy. It must have mechanisms for automati-cally migrating multimedia data objects from onelevel of the storage hierarchy to another. A detailedtreatment of these migration policies exceeds ourscope here, but they must be based on some clear-ly defined criteria such as frequency of use. In gen-eral, even when the data is stored in offline storagedevices, the multimedia DBMS should have infor-mation on how to easily locate the specific devicecontaining the multimedia data being sorted.

Data migration in multilayered storage systemsis not peculiar to multimedia DBMSs. All databas-es handling huge amounts of data must addressthis issue. The interconnection between the mem-ory systems is obviously a problem, especiallywhen a multimedia database involves distributedsources of data. As a result, the problem of datamigration may require the consideration of othernetwork related issues, such as data availabilityrates, bandwidth limitations, and network delays.

27

July–Septem

ber 1997

.

Increasingstorage

capacity

Increasingperm

anence, increasingaccess

time

Incr

easin

gpr

obab

ility

ofac

cess

Incr

easin

gco

st, i

mpr

ovin

gpe

rform

ance

Mainmemory(RAM)

Online devices:magnetic disks,

optical disks

Near-line/online devices:optical storage,

jukeboxes

Off-line devices:magnetic tapes,optical storage

Figure 2. Hierarchically

organized storage for

multimedia databases.

Query support and information retrievalQuerying in multimedia databases can involve

different multimedia data types, keywords, attrib-utes, content, or even contextual information.Because of the different ways in which users thinkabout multimedia data, multimedia query cansimultaneously involve multiple cues, necessitatingmultiple or multidimensional indices. Queries areusually imprecise. Because of this and the difficultyof ensuring exact matches between multimediadata items, retrieval usually involves comparingdata items for similarity or partial (rather thanexact) matching. Thus, since queries might notyield exact matches, we need facilities for rankingthe retrieved results according to how closely theymatch the given query. Similarly, we should havemethods to prune results that do not seem to satis-fy the query. Doing so reduces the potentially enor-mous computation needed for further matching.

With the ranking, the multimedia DBMSshould also support browsing the various retrieveditems. We might also want to retrieve similaritems based on one or more of the alreadyretrieved items.

With users unsure of the information sought,a true multimedia DBMS also needs a facility tosupport incomplete information. More impor-tantly, since the information extracted to indexthe multimedia data or from the user query mightcontain errors, query interpretation should pro-vide for uncertainties in the information. Thismight require an iterative search mechanism anda relevance feedback mechanism along with tech-niques for query reformulation.

Media integration, composition, andpresentation

Given the multiplicity of data types supported,the multimedia DBMS should also provide facilitiesfor integrating data items (from possibly disparatemedia types) to form new composite multimediatypes and for presenting such data at a given sitewithin the required time frame. Multimedia inte-gration, composition, and presentation are exacer-bated by the often continuous (temporal) nature ofmultimedia data—especially video, animation, andaudio. Moreover, certain applications, such as geo-graphic information systems, may require a multi-media DBMS to address spatial information. Allthese factors put together make multimedia com-position and presentation a complex process thatthe multimedia DBMS must support to meet thediverse user community’s needs.

The integration problem can be ameliorated in

some cases, especially where the multimedia data-base system is tailored to the requirements ofsome target user community. In such special casesthe multimedia DBMS could support specific fea-tures not needed for other applications.

Multimedia interface and interactivityThe diverse nature of multimedia data calls for

an equally diverse interface for interacting with thedatabase. Typically, each media data type has itsown method for access and presentation. Forinstance, video and audio data will need differentuser interfaces for presentation and query. Forsome multimedia applications, especially thoseinvolving continuous media, the user often expectsthe interactive facilities of a VCR or tape recorder,such as fast forward and reverse. When a multi-media system provides such services, it has impli-cations for the database, especially retrieval of theneeded multimedia objects, their integration, andtheir synchronization. Thus the multimedia DBMSmight need to support such forms of interactivity.

PerformanceEfficiency is an important consideration in a

multimedia DBMS. Multimedia database systemsmake new performance demands on media access,storage, indexing, retrieval, and query optimiza-tion. The different data types involved in multi-media databases might also require specialmethods for optimal storage, access, indexing,and retrieval. Rodriguez4 discusses some of theperformance requirements that must be consid-ered for multimedia DBMSs. These include effi-ciency, reliability, real-time execution, guaranteedand synchronized delivery of multimedia presen-tations, and quality-of-service (QoS) acceptable tothe users. We will further discuss some of theissues affecting performance later.

Issues in multimedia DBMSsTo meet the requirements discussed, the multi-

media DBMS must address a number of issues,including

❚ Multimedia data modeling

❚ Multimedia object storage

❚ Multimedia integration, presentation, and QoS

❚ Multimedia indexing, retrieval, and browsing

❚ Multimedia query support

28

IEEE

Mul

tiM

edia

.

❚ Distributed multimedia database management

❚ System support

Multimedia data modelingData models are central to multimedia data-

base systems. A data model must isolate usersfrom the details of storage device managementand storage structures. It requires the develop-ment of appropriate data models to organize thevarious data types typically found in a multimediadatabase system.

Multimedia data models (just like traditionaldata models) capture the static and dynamic prop-erties of the database contents, and thus providea formal basis for developing the appropriate toolsneeded in using the multimedia data. The staticproperties could include the objects that make upthe multimedia data, the relationships betweenthe objects, the object attributes, and so on.Examples of the dynamic properties include inter-action between objects, operations on objects,user interaction, and so forth.

However, the unique nature of multimediadata requires certain new considerations whenchoosing the data model. For instance, somemultimedia data types (such as video) or group oftypes (example, video and images) might requirespecial data models for improved modeling effi-ciency and flexibility. Moreover, the importanceof interactivity in multimedia systems makes theirsupport by the data model an important issue.Furthermore, it may be necessary to consider newintegrity constraints in the context of multimediadatabases.

Various data models, such as network, rela-tional, semantic, and object-oriented modelsalready exist for traditional databases,5 and a fewhave been proposed for multimedia databases.6,7

Two basic approaches have been used in model-ing multimedia data. The first involves building amultimedia data model on top of an underlyingtraditional database data model (usually relation-al or object-oriented databases) by using appro-priate interfaces for the multimedia data. Theproblem with this approach is that the underlyingstructures are not designed for multimedia data.Often, the significant differences between therequirements of the traditional and multimediadata make the interface a bottleneck in the over-all system.

These problems led to the second method,which opts to develop true multimedia-specificdata models from scratch, rather than on top of

an existing traditional database sys-tem. Nonetheless, a consensusalmost exists that such efforts shouldbe based on object-oriented tech-niques. Current issues include devel-oping appropriate data models forindividual multimedia data types(such as video, images, or visualdata), uniform modeling of arbitrarydata types, and supporting huge vol-umes of multimedia data, multi-media interactivity, andcontent-based information usingthese models. Some authors havegone so far as to claim that the datamodel for a multimedia DBMS canonly be fully achieved by object-ori-ented technology.8-10

Multimedia object storagePhysically storing multimedia

data requires methods for transforming, manag-ing, transferring, and distributing huge volumesof data. Typical multimedia systems use a hierar-chy of storage devices. Online high-speed devices(such as random access memories) and magneticdisks store multimedia data currently being used,while offline, low-speed devices (like optical stor-age and tapes) store long-term archival data.Performance then depends on the efficiency ofthe migration mechanisms used to assign themultimedia data items to the optimal level in thestorage hierarchy.

Data compression schemes, in combinationwith the data transformation, help to reduce thehuge capacity requirements. The basic methodhere is to transform the multimedia data to sometransform space to remove the redundancies inthe original data. Coding schemes code the trans-formed data for storage or transmission. Decom-pression is accomplished from the reverse processof decoding and re-transforming the data into itsoriginal form. This process often involves someloss of data, which a majority of multimediaapplications can tolerate.

The huge volumes of data often involved andthe constraints certain multimedia data typesimpose on the presentation make multimediaobject storage a major consideration in databaseissues. Depending on the level of granularity, amultimedia object can represent the entire videosequence for a movie, a subsequence from thevideo, a single frame or image, or even individualobjects in the image or video frame.

29

July–Septem

ber 1997

.

Data compression

schemes, in

combination with

the data

transformation,

help to reduce the

huge capacity

requirements.

The major issues here are the limited availablestorage, the bandwidth limits of the storage sys-tem and communication channel, and the multi-media data type’s availability rates. The dataavailability rate11 indicates the minimum amountof data required per unit time to meet acceptablelevels of quality during presentation of the multi-media object. Viewed from this standpoint, multi-media data’s storage requirements are mostsusceptible to decomposing the data into smallermultimedia objects. Each smaller object can bestored in the smaller available storage units.

As a necessary condition for storage allocation,at presentation time the data from the differentstorage units, when combined together, meet thedata availability rates of the given multimedia datatype. With the hierarchical storage arrangement,multimedia objects can be stored at different lev-els. As the utility rate of multimedia data objectschanges, such objects will need to be reallocated,possibly to different storage devices on differentlevels of the storage hierarchy. The problems theninvolve finding optimal methods for multimediaobject decomposition, allocation, and reallocation.

Multimedia integration, presentation, and QoSUnlike traditional data, multimedia data have

presentation constraints. These mainly result fromthe continuous nature of some multimedia datatypes, which requires presenting certain amountsof data within a given time for the presentation toseem natural to the user. When multimedia dataare distributed and transported over networks, theproblems of presentation become even moreacute. Here, one can easily experience networkproblems, such as limited bandwidth and statisti-cal network delays.12

Continuous media by definition are time-dependent, so timing becomes an important fac-tor in their delivery and presentation. Therefore,in multimedia DBMSs the response to a query isoften judged by both the correctness and the qual-ity of the retrieved results.

From the user’s point of view, the QoS parame-ter specifies, qualitatively, the acceptable levels ofperformance for the various services provided bythe multimedia system and may affect the resultsof the multimedia presentation. Thus, to supportmultimedia presentations where a user can specifyvarious QoS levels for different services, the multi-media DBMS must support the specified QoS levelsand a QoS management service. This typicallyinvolves providing an appropriate mapping fromthe user’s QoS to the system’s QoS and vice versa.

When presenting different types of multimediadata—such as video and audio—together, prob-lems of media integration and synchronizationalso become important. The multimedia DBMSmust provide a mechanism to ensure good syn-chronization of the presented data while stillmeeting other requirements such as the data avail-ability rates and the QoS. In some situations, themultimedia DBMS may have to rely on an explic-it synchronization manager to ensure synchro-nization within a given data type and betweendifferent data types.

Multimedia indexingAs in traditional databases, multimedia infor-

mation can be retrieved using identifiers, attrib-utes, keywords, and their conjunctions usingconditional statements. Keywords are by far thepredominant method used to index multimediadata. A human typically selects keywords from aset of specialized vocabulary. While simple andintuitive, this method usually creates problemswhen applied to multimedia data: it is basicallymanual and time consuming, and the resultingindices are highly subjective and limited depend-ing on the vocabulary.

Another method, content-based access, referseither to the actual contents of the multimediadatabase or to derived contextual information.Intensive research has focused on content-basedindexing in recent years, with the goal of index-ing the multimedia data using certain featuresderived directly from the data. Various features,such as color, shape, texture, spatial information,symbolic strings, and so on, have been used toindex images.

Deriving such features requires automaticanalysis of the multimedia data. The primarymethods used for image and video data are imageprocessing, image understanding, and videosequence analysis. With video data, the videosequence is first separated into its constituentscenes, then representative abstractions (usuallykey frames) are selected to represent each scene.Further indexing on the video is based on the keyframe, as in the case for images.

For audio data, content-based indexing couldinvolve analysis of the audio signal or automaticspeech recognition followed by keyword-basedindexing. On the other hand, indexing can bebased on other information depending on thetype of audio data. For example, some developershave used rhythm signature, chord, and melodyfor content-based indexing of music data.13

30

IEEE

Mul

tiM

edia

.

Similarly, methods for content-based search andretrieval of audio data have been proposed basedon the characteristics of audio data, as indicatedby its perceptual and acoustic features.14

Using content-based indexing implies the con-sideration of certain issues. First, the same multi-media data could mean different things todifferent people. Second, users typically havediverse information needs. Thus, it is evident thata single feature may not be sufficient to complete-ly index a given multimedia data type. Therefore,it becomes difficult to identify the features that aremost appropriate in any given environment.

Another problem has to do with efficiency:making the indexing fast and storing the indicesefficiently for easy access, since multimedia datatypically come in huge volumes. Because of thediverse content inherent in multimedia data,indexing has not been completely automated. Forexample, while the computer can easily analyze apicture containing works of art, it is almost impos-sible for the computer to automatically determinethe meaning of the art object. Only a human canprovide such information.

Multimedia query support, retrieval, andbrowsing

User queries are often processed using onlyavailable indices. However, unlike in traditionaldatabases, matches in multimedia queries are notexact matches. Often when comparing two multi-media data items, approximate or similaritymatches result. Given that various items canresemble the same input data, a single querymight yield many items in response.

Various research efforts have chosen to inves-tigate issues on similarity matching involvingmultiple indices and ranking. Also being devel-oped are appropriate ways of presenting theretrieved information, such as through a brows-ing interface. A user-directed browsing lets theuser retrieve any information potentially relatedto the current results by selecting the data itemsfor further consideration.

Among the issues involved in multimediaquery support is the availability of a multimediaquery language capable of supporting both thevarious media types encountered in a typicalmultimedia database and new requirements suchas fuzzy query predicates. Such query modelsshould also provide mechanisms for users to refor-mulate their queries, perhaps based on the alreadyretrieved results.

Query-by-example is the primary method used

to enter queries in multimedia data-bases, especially in those involvingimages. Here, the user makes arequest using an existing example(for example, similar images). Thus,the interface used to enter the queryinto the system becomes an issue.Since different multimedia datatypes may require different queryinterfaces, the problems to considerinclude how to integrate the variousinterfaces in an integrated multi-media database system. Other prob-lems to be resolved include queryingspatial data and content-based videoquery, which could involve tempo-ral and spatial information.

Distributed multimedia databasemanagement

Distributed multimedia DBMSloosely refers to a collection of various (possibly)independent multimedia database managementsystems, located in disparate locations, that cancommunicate and exchange multimedia data overa network. Multimedia systems are usually dis-tributed in the sense that a single multimediainteraction often involves data obtained from dis-tributed information repositories. This is typical-ly the case in collaborative multimediaenvironments, where multiple users in possiblydisparate physical locations manipulate andauthor the same multimedia document.Moreover, issues like storage problems and datageneration may also force multimedia systemdesigners to place multimedia data in differentphysical locations.

To support the information required in suchdistributed and collaborative environments, a dis-tributed multimedia DBMS must address the gen-eral problems in distributed databases, such asdistributed and parallel query processing, distrib-uted transaction management, data locationtransparency, data security, and so forth. In addi-tion, network issues such as limited bandwidthand network delays become important consider-ations, since they could have adverse effects onthe QoS supported.

Unlike in the traditional DBMS, data replica-tion is often not encouraged in a distributedmultimedia DBMS due to the huge data volumes.The client-server computing model, in which aserver application services multiple client appli-cations—with the clients and server residing in

31

July–Septem

ber 1997

.

Query-by-example

is the primary

method used to

enter queries in

multimedia

databases, especially

in those

involving images.

possibly different machines—has proven suitablefor multimedia systems in general and distributedmultimedia DBMSs in particular.

System supportMultimedia applications in general, and dis-

tributed multimedia database systems especially,raise new issues in all aspects of the computer sys-tem, from operating systems to networks to gen-eral hardware. Most generally available operatingsystems do not support real-time operations ade-quately. Rather, they provide hardware front-endsfor transmitting and presenting multimedia data.

Some multimedia data, such as continuousmedia, may require real-time delivery and presen-tation, although the real-time requirements mightnot be as stringent as those encountered in hardreal-time systems. Thus, the multimedia databasesystem cannot fully provide its functionalitiesuntil support for real-time continuous media databecomes an integral part of the operating system.Efforts on various fronts have concentrated onthis problem, including research on resourcescheduling, operating system support for QoS, useof multilevel and user-level threads, and so on.

Other characteristics of multimedia, such asthe huge data volumes, may mandate special con-straints on the system in terms of memory man-agement, CPU performance, throughput, and soforth. Related issues include general considera-tions on I/O hardware to support the variousmedia types involved in multimedia databases.Communication networks—needed to transportthe data for distributed multimedia environ-ments—must support bandwidth and delay guar-antees as needed to meet the stringent QoSrequirements for certain multimedia applications.

ApplicationsIn general, multimedia database management

systems applications can be found wherever thereis a need to manage multimedia data cost-effectively. Thus multimedia DBMSs have foundapplications in such diverse areas as education(digital libraries, training, presentation, distancelearning), healthcare (telemedicine, health infor-mation management, medical image systems),entertainment (video-on-demand, music databas-es, interactive TV), information dissemination(news-on-demand, advertising, TV broadcasting),and manufacturing (distributed manufacturing,distributed collaborative authoring). Other areasmay include finance, video conferencing, elec-tronic publishing, electronic commerce, and geo-

graphic information systems.A number of multimedia DBMSs already exist.

Most are extensions from existing object-orientedor relational DBMSs. The capabilities of existingmultimedia DBMSs can be evaluated by the extentto which they support different media types (espe-cially image and video). They can also be evaluat-ed by their ability to support special functionalitiesrequired of a database system to manage multi-media data, such as real-time delivery and content-based query and retrieval.15

Unfortunately, most existing multimediaDBMSs fall short. However, rigorous research isbeing geared toward the various parts of the prob-lem, and we expect new systems with better capa-bilities in the near future. Future trends includeperforming indexing, retrieval, and browsingdirectly on the compressed data, especially forvideo data; video data management; multimediaquery language; uniform indexing frameworks forthe different data types; content-based image andvideo retrieval; and multimedia transport anddelivery over the Internet. MM

References1. S. Khoshafian and B.A. Baker, Multimedia and Imaging

Databases, Morgan Kaufmann Publishers, San

Francisco, Calif., 1996.

2. W. Kim, ed., Modern Database Systems: The Object

Model, Interoperability, and Beyond, Addison-Wesley,

Reading, Mass., 1995.

3. A. Narasimhalu, “Multimedia Databases,” Multimedia

Systems, Vol. 5, No. 4, Oct. 1996, pp. 226-249.

4. A.A. Rodriguez and L.A. Rowe, “Multimedia Systems

and Applications,” Computer, Vol. 28, No. 5, 1995,

pp. 20-22.

5. R. Hull and R. King, “Semantic Database Modeling:

Survey, Applications, and Research Issues,” ACM

Computing Surveys, Vol. 19, No. 3, 1987, pp.201-260.

6. C.Y.R. Chen et al., “Design of a Multimedia Object-

Oriented DBMS,” Multimedia Systems, Vol. 3, No. 5-6,

1995, pp. 217-227.

7. S. Marcus and V.S. Subrahmanian, “Foundations of

Multimedia Database Systems,” J. ACM, Vol. 43, No.

3, May 1995, pp. 474-523.

8. D. Woelk, W. Kim, and W. Luther, “An Object-

Oriented Approach to Multimedia Databases,” Proc.

ACM SIGMOD 1986 Int’l Conf. on Management of

Data, Vol. 15, No. 2, 1986, ACM Press, New York, pp.

311-325.

9. D. Woelk and W. Kim, “Multimedia Information

Management in Object-Oriented Database Systems,”

Proc. Int. Conf. on VLDB, Brighton, England, Sept.

1987, pp. 319-329.

32

IEEE

Mul

tiM

edia

.

10.Y. Masunaga, “Multimedia Databases: A Formal

Framework,” Proc. IEEE CS Office Automation Symp.,

IEEE Computer Society Press, Los Alamitos, Calif., Apr.

1987, pp. 36-45.

11.C. Orji, K. Nwosu, and N. Rishe, “Multimedia Object

Storage and Retrieval,” Proc. Int’l Symp. on Multimedia

Systems, Information Processing Society of Japan,

Yokohama, Japan, 1996, pp. 368-375.

12.D.A. Adjeroh and M.C. Lee, “Synchronization and

User Interaction in Distributed Multimedia

Presentation Systems,” Multimedia Database Systems:

Design and Implementation Strategies , K.C. Nwosu, B.

Thuraisingham, and P.B. Bera, eds., Kluwer Academic

Publishers, Boston, Mass.,1996, pp. 251-297.

13.T-C. Chou, A.L.P. Chen, and C.C. Liu, “Music

Databases: Indexing Techniques and

Implementation,” Proc. 1996 Int’l Workshop on Multi-

Media Database Management Systems, IEEE Computer

Society Press, Los Alamitos, Calif., 1996, pp. 46-53.

14.W. Wold et al., “Content-Based Classification, Search,

and Retrieval of Audio,” IEEE MultiMedia, Vol. 3, No.

3, 1996, pp. 27-36.

15.P. Pazandak and J. Srivastava, “Metrics for Evaluating

ODBMS Functionality to Support MMDBMS,” Proc.

Int’l Workshop on Multi-Media Database Management

Systems, IEEE Computer Society Press, Los Alamitos,

Calif., 1996, pp. 72-79.

Donald A. Adjeroh is a post-doc-

toral fellow in the Department of

Computer Science and Engineering

of the Chinese University of Hong

Kong. He holds a BEng degree (First

Class Honors) in electronics and

computer engineering, obtained in 1990, and an MS in

computer engineering, obtained in 1994, and a PhD in

computer science from the Chinese University of Hong

Kong, obtained in 1997. Adjeroh’s current research focus-

es on indexing and retrieval of visually stimulated infor-

mation. His general research interests are in video pro-

cessing, computational aspects of vision, distributed

multimedia systems, digital libraries, and knowledge

based systems. He is a member of IEEE Computer Society.

Kingsley C. Nwosu is a member of

technical staff with Lucent Tech-

nologies, Whippany, New Jersey.

He is also an adjunct professor in

the Department of Mathematics

and Computer Science at Rutgers

University, and a visiting professor in the Department

of Mathematics and Computer Science at Montclair

State University, New Jersey. His current areas of

research include multimedia information and comput-

ing systems, object-oriented programming and design,

distributed processing and computing, information stor-

age and retrieval, and parallel algorithm design and

analysis.

Nwosu holds a BS degree in computer science from

the University of Southwestern Louisiana, Lafayette, an

MS in computer science from the University of

Oklahoma, Norman, and a PhD in computer science

from Syracuse University, Syracuse, New York. He served

as program chair for the 1994, 1995, and 1996 IEEE

International Workshop on Multimedia Database

Management Systems. He was also a guest editor for a

special issue of the Journal of Multimedia Tools and

Applications on multimedia database systems. He recent-

ly co-edited a book on multimedia database systems. He

is a member of ACM, IEEE Computer Society, IEEE

Communication Society, New York Academy of Science,

and National Council on Systems Engineering.

Readers may contact Nwosu at Lucent Technologies,

Network Systems, Room 2E-223A, 67 Whippany Rd.,

Whippany, NJ 07981-0903, e-mail knwosu@lucent.com.

33

July–Septem

ber 1997

.