Digital Object Identifier (DOI) 10.1007/s00530-002-0060-5Multimedia Systems 8: 395–396 (2002) Multimedia Systems

© Springer-Verlag 2002

Introduction to Multimedia special issue on multimedia middleware

Middleware + multimedia = multimedia middleware?

Thomas Plagemann

Department of Informatics, University of Oslo, P.O. Box 1080, Blindern, 0316 Oslo, Norway

This special issue is based on the Multimedia Middleware(M3W)Workshop held at the 9thACM Multimedia Conferencein Ottawa 2001. The workshop was well attended and charac-terized by very interactive and constructive discussions. Mostparticipants stayed in the meeting room even during lunch toproceed with discussions – and that after three days of theACM Multimedia Conference. The success of the workshophas finally led to this special issue. The authors of the 12best papers at the workshop have been invited to submit a fulljournal paper to compete for publication. Based on the rec-ommendations of the M3W Program Committee, which didan excellent job in reviewing all submissions for this specialissue, we present here four papers. These papers describe:novel solutions for integrated runtime support in QoS-awaremiddleware; dynamic end-to-end QoS management middle-ware; proxy architectures for collaborative media streaming;and abstractions for multimedia streaming.

The topics of the four papers already indicate the broadrange of research questions that are studied in the context ofmultimedia middleware, and readers who refer to the work-shop’s web page1 will see even more topics of interest formultimedia middleware. Since the range of topics is so broad,it is legitimate to ask whether there is actually a clear defini-tion and scope of multimedia middleware. A straightforwarddefinition might be that multimedia middleware covers all re-search topics related to the use of middleware for multimediaapplications. The follow up question for readers familiar withthe meaning of multimedia would probably be: What is mid-dleware?

A short answer to this question could be that there is noclear definition. For many researchers and developers, the term‘middleware’ is today synonymous with object-based middle-ware, like the OMG’s CORBA or Microsoft’s DCOM. How-ever, object-based middleware is only one type of middlewarebesides others like event-based and message-oriented middle-ware. Since multimedia middleware might be seen as anothertype of middleware, we refer to a more basic definition ofmiddleware from Geoff Coulson2:

The role of middleware is to ease the task of designing, pro-gramming and managing distributed applications by provid-

1 http://www.ifi.uio.no/∼m3w/2 http://dsonline.computer.org/middleware/index.htm

ing a simple, consistent and integrated distributed program-ming environment. Essentially, middleware is a distributedsoftware layer, or ‘platform’ which abstracts over the com-plexity and heterogeneity of the underlying distributed envi-ronment with its multitude of network technologies, machinearchitectures, operating systems and programming languages.

In other words, abstraction away from complexity and het-erogeneity is the main goal of middleware. In this context, itis important to note that multimedia applications increase theheterogeneity problem. First, there exist many different en-coding formats for video, audio, graphics, etc. To relieve ap-plication developers from format considerations, multimediamiddleware should provide services for compatibility controland management, e.g. implicit transcoding to allow peers withincompatible media formats to communicate. Secondly, theQoS requirements of communication partners might be dif-ferent due to end-user demands or available resources. Multi-media middleware should thus include QoS management ser-vices that establish and maintain (if possible) a QoS agreementthat satisfies all peers [1]. Thirdly, there are already severalQoS solutions at different system levels, e.g. intserv and diff-serv networks, adaptive CPU schedulers, self-adapting videoplayers, and QoS aware Object Request Brokers. The reuseof existing QoS solutions and their combination and integra-tion in a single session requires a solution to this particulartype of heterogeneity problem. Thus, Ecklund et al. describein this special issue a dynamic end-to-end QoS managementmiddleware platform that addresses this problem and showshow legacy QoS solutions can be integrated. Another problemaddressed in this paper is the dynamic nature of sessions. Theapplication structure as well as the configuration of the systemcan be changed during an ongoing session. Consequently, theQoS management structure has to be adapted accordingly, andis therefore dynamic.

To manage QoS and get at the application level informa-tion about the system state to improve application adaptation,openness of the system and control over the system are nec-essary. In addressing these needs, Baochun et al. present anintegrated runtime QoS-aware middleware framework. Thisis designed to provide runtime support for all stages of QoS

396 T. Plagemann

management, particularly the probing of QoS parameters, in-stantiation of initial configurations, and adaptation at runtime.

Openness and control are apparently contradictory to thetransparency that is given by classical middleware solutions,including object-based middleware, i.e. there must be a trade-off between transparency on the one hand and openness andcontrol on the other. The fundamental research question re-lated to this trade-off is: Is there an architectural concept thatcan provide suitable abstractions and transparency togetherwith openness and control? Traditional solutions use layeredarchitectures to provide transparency. To implement the nec-essary control with traditional solutions, the layering principlehas to be broken or bypassed.

Research projects that strive for new architectural conceptsand abstractions to support both transparency and control bythe middleware often develop open and flexible solutions, likethat of Kristensen et al. [2], and apply a combination of com-ponent technology and reflection, like that of Blair et al. [2].Black et al. introduce in this special issue an abstraction forinformation flows called Infopipes. Applications can be builtby connecting pre-defined Infopipe components like sources,sinks, multicasting pipes, etc. This abstraction represents com-munication in application-level terms, instead of hiding com-munication. Infopipes are intended to reify communicationsuch that the application can interrogate and manipulate them.

Another important question in addition to the design ofthe middleware platform is which services should be pro-vided. Earlier we identified QoS management services andtranscoding services; but other services that are of relevancefor multimedia middleware include stream operations like fil-tering, mixing and aggregation; data management services likepersistency, caching and replication; and group and sessionmanagement services. Kahman et al. show in their paper inthis special issue how to use IETF multimedia session con-trol protocols together with a proxy architecture to providecollaboration services in the middleware.

The case of generic services as part of middleware alsonicely illustrates why the boundaries of middleware are andwill continue to be fluid. For example, fully fledged operatingsystems include services that are regarded as ‘middleware’ inmicro- and nano-kernel operating systems: successful appli-cations that solve generic tasks are often later migrated downto the middleware as a service. Therefore, it is not meaningfulto draw a clear boundary for multimedia middleware. Sincethere are many research groups actively working in this area,the scope of multimedia middleware will continue to be de-veloped.

The four articles in this special issue will give readers aglimpse of the problem and solution domains in multimediamiddleware. I hope you will enjoy reading these articles. Inclosing, I want to thank all those who contributed to the successof the Multimedia Middleware Workshop in Ottawa and whoenabled this special issue to be published.

References

1. Vanegas R, Zinky J, Loyall J, Karr D, Schantz R, Bakken D (1998)QuO’s Rruntime support for quality of service in distributed ob-jects. In: Davies et al. (eds) Proc. IFIP Int. Conf. on DistributedSystems Platforms and Open Distributed Processing (Middle-ware ’98), The Lake District, UK. Springer, Berlin HeidelbergNew York

2. Kristensen T, Kalleberg IB, Plagemann T (2001) Implementingconfigurable signalling in the MULTE-ORB. In: 4th IEEE Con-ference on OpenArchitectures and Network Programming (IEEEOPENARCH’01), Anchorage, Alaska. IEEE

3. Blair G, Coulson G, Robin P, Papathomas M (1998) An archi-tecture for next generation middleware. In: Davies et al. (eds)Proc. IFIP Int. Conf. on Distributed Systems Platforms and OpenDistributed Processing (Middleware ’98), The Lake District, UK.Springer, Berlin Heidelberg New York

Dr Thomas Plagemann received hisDiploma in Computer Science from theUniversity Erlangen-Nurnberg (Germany)in 1990, and his Doctor of TechnicalScience from Swiss Federal Institute ofTechnology (ETH) Zurich (Switzerland)in 1994. In 1995, he was honoured with theMedal of the ETH Zurich for his doctoralthesis, in which he developed the Da CaPocommunication subsystem. From 1994–1996 Thomas Plagemann was a researcherat UniK – Center of Technology at Kjellerand Telenor R&D (Norway). Since 1996,

Thomas Plagemann has been a Professor at the University of Oslo.His research interests include multimedia middleware, protocol ar-chitectures, QoS, operating system support for distributed multime-dia systems, content distribution infrastructures, and interactive dis-tance learning.