RDF Recipes for Context-AwareInteroperability In Pervasive Systems

Anna M. Kosek∗, Aly A. Syed∗ and Jon M. Kerridge†∗NXP Semiconductors, Corporate Innovation and Technology/Research

High Tech. Campus 32, 5656 AE Eindhoven, The Netherlands{anna.kosek,aly.syed}@nxp.com

†Edinburgh Napier University, School of ComputingMerchiston Campus, Edinburgh EH10 5DT, United Kingdom

j.kerridge@napier.ac.uk

Abstract—Nowadays home automation systems in-tegrate many devices from security system, heating,ventilation and air conditioning system, lighting systemor audio-video systems. Every time a new device is in-stalled, problems with connecting it to other devices andsynchronization may appear. The technology trends areto build more powerful and functional new autonomousdevices, rather than improve device synchronization,data and functional distribution and adaptation tochanging environment. This paper highlights interoper-ability problems in pervasive computing and presents asolution for devices with limited processing capabilitiesby use of an ontology for knowledge formulation andsemantic interoperability.

Index Terms—ontology, semantic interoperability,knowledge representation.

I. INTRODUCTION

The number of electronic devices surrounding usincreases every day, many devices are already de-ployed in the environment and are hidden frompeoples’ sight. Ongoing device miniaturization makesit possible to manufacture smaller devices, thereforemore of them can be embedded in one space. Anetwork of many small devices requires a flexibleapproach to pervasive configuration and management.

A well known problem in pervasive environmentsis how to achieve interoperability between compo-nents in the system. In distributed systems, thereare three classic interoperability levels: platform, pro-gramming language and service interoperability [1].Service interoperability divides further in to signa-ture, protocol and semantic levels [1]. The signatureinteroperability problems are associated with the syn-tax of the interface of the service and are successfullysolved by popular language interface specificationslike CORBA’s IDL (Interface Definition Language)[2] or WSDL (Web Service Definition Language)[3]. Many network interoperability problems (at theprotocol level of interoperability) can be addressedby Internet protocols, however these do not solveinteroperability problems on a semantic level [4].When considering many multi-vendor devices andcomponents, solutions for semantic interoperabilitybased only on standards do not scale [5]; flexible and

updatable standards are needed.

A. Ontologies in Interoperability

An ontology is a representation of a set of con-cepts, functions and relations between concepts [6].Concepts are classes of entities existing in a do-main, functions and relations are respectively one-to-one and one-to-many properties existing in theenvironment. An ontology, together with syntax andsemantics provides a language that is necessary forproper communication. Building an ontology is a dif-ficult task and because domains and language alwaysevolve it is impossible to state that the ontologydescribes all possible concepts and relations in aspecific domain. Because an ontology can alwaysevolve and adapt to an ever-changing domain, it canbe a very flexible and up-to-date description of thedomain. Therefore an ontology can play the role of aflexible standard and can be used to achieve semanticinteroperability.

It has been shown that interoperability can beachieved by using an ontology in consumer elec-tronics devices [7]. Another project, Sofia (SmartObjects for Interactive Applications) [8], is targetedto enable and maintain cross-industry interoperabil-ity for smart spaces using a core ontology basedon Dolce [9] and domain ontologies. The approachpresented in this paper is an extension of both [7] andSofia concepts to create a pervasive environment withinteroperability at the semantic level and re-usableknowledge representation based on an ontology. Thespecific environment that this project is consideringis a network of very small, energy-frugal deviceswith limited processing capability. Services that areoffered by these devices are simple, therefore it ispossible to represent them using an uncomplicatedknowledgebase drawn from an ontology.

B. Knowledge Representation and Ontology

In artificial intelligence (AI) domains a key tobuilding powerful and large systems is capturingknowledge in a computer-readable form. Represent-ing knowledge is a complex and time-consuming

task. Therefore knowledgebase construction remainsone of the biggest costs when constructing an AIsystem [10]. Most complicated AI systems requirebuilding a knowledgebase from scratch. An ontologycan offer a core structure for representing knowledgeand enables reuse of knowledgebases when designingsystems in similar domains. Neches [10] points outthat a knowledgebase should be constructed frommany levels of knowledge. These layers are increas-ingly specialized, therefore layers’ division beginswith very general concepts, based on the ontologicalmodel and finishes with very specialized and lessreusable knowledge [10]. Therefore a branch of anontology that is used for constructing knowledgebasesshould consist of general (core), domain-specific anddevice-specific concepts and relations. The knowl-edgebase in a device or component should consistonly of necessary concepts, depending on a compo-nent’s functions and domain as described in [7]. Theknowledgebase can be extended if new functionalitiesor services appear. The most specialized part ofknowledgebase are instances and data associated witha particular device or component.

C. Context-Aware Pervasive System

The solution for interoperability and knowledgerepresentation in the pervasive system presented inthis paper is based on building a knowledgebasebased on an ontology. It is important to fix theontology model, while the ontology content can bechanged when new classes and relationships appear.The ontology can be modified by domain expertsand can be kept in a repository accessible by bothsoftware and hardware manufacturers.

Because of the use of a common languagedescribed by an ontology, the interoperability isachieved on a semantic level [7]. Components fromthe system use concepts that are provided by theontology, therefore understanding is achieved at ahigh level of abstraction. This way it is possibleto build a context-aware system, that can representand share understanding of context. Context can beassociated with environment conditions, presence ofparticular devices or people, or periods and eventsassociated with a calendar. Context-aware systemsbenefit from well defined context, and as every deviceor component understands how the context influencesits own domain, the system can produce emergentbehavior.

In pervasive computing a smart space is understoodas a cloud of devices existing and cooperating inthis space. In an architecture described in [7] allconfigurations between devices are established auto-matically, data and functionalities are distributed andavailable for different devices to enable cooperation.Devices can appear or disappear from a particularspace at any time, so devices have to be able to workautonomously or form subnetworks called virtual

devices [7]. A device can participate in many virtualdevices; virtual devices are created for a particulartask and can be destroyed if a different configurationis requested. A virtual device can be created by auser request or some scheduled task. Devices can joinor leave a virtual device at any time; this operationcan influence the behavior of a virtual device. Theway a virtual device is formed is described in theknowledgebase by use of a list of actions to perform.Devices can use any protocol to form a virtual device,the protocol specification is described in a device’sknowledgebase. Therefore depending on configura-tion, a device will have clear instructions how to reactto a message from a particular protocol.

II. KNOWLEDGEBASE STRUCTURE

A knowledgebase consists of layers of knowledge(Figure 1). Layers are divided into T-Box and A-Box.T-Box consists of class taxonomy and property defi-nitions that we name in this paper ontology models,like Dolce [9] or any other upper ontology, A-Boxcontains instances corresponding to classes describedin T-Box.

Fig. 1. Layered structure of a knowledgebase.

The top level of the knowledgebase is a metamodel of the core ontology. The meta model of theontology has to be fixed, this project is followingthe Sofia approach by choosing Dolce as a coreontology. Dolce is an upper ontology that addressesvery general domains [9]. The next layer in Figure 1is a domain ontology that is created for a specificarea. For example if the device is a sensor, itsdomain ontology describes classes and properties thatare associated with sensor domains. This way theknowledge of the surrounding world is narrowed tothe domain the device or component belongs to. Thenext layer, Application specific ontology model, is amodel for instances that describes a device’s behavior,capabilities and services that the device can provide.In the last layer of the knowledgebase, categorized asan A-Box, contains instances describing a particulardevice according to rules provided by T-Box ontologymodels. Similar devices will have similar T-Box partof the knowledgebase, but different instances in A-

Box. T-Box can be automatically generated and usedwhen building A-Box for a particular device.

A. Resource Description Framework

The knowledgebase structure can be fixed usingRDF (Resource Description Framework) [11] triples.RDF is a framework for representing information inthe World Wide Web. RDF can describe a simplegraph-based data model [11]. An RDF triple consistsof subject, predicate and object, where the subject andpredicate are resources defined by an URI (UniformResource Identifier) while the object is a literal ora resource. The structure of the knowledgebase willconsist of triples:

(Subject, Predicate, Object)

For example:

(apple tree, is− a, tree)

This triple uses predicate is-a to associate subject ap-ple tree with object tree. Ontologies are designed bydomain experts and represent universal and domain-specific truths.

For the purpose of the World Wide Web, ontologiesare represented by use of a markup scheme like XML.Unfortunately XML adds complexity to database for-mat by introducing tags that need to be interpreted.XML is not designed for data storage or efficientretrieval of data [12] the additional time to processdata exists due to parsing and text conversion [13].For devices with a limited processing capability it isimportant to keep any database structure as simple aspossible. Therefore the knowledgebase format is keptto the simple three field structure of RDF.

RDF for World Wide Web can use many externalfiles of different ontologies, therefore it is not essen-tial to define all concepts in one RDF file. In manycases referencing by use of URL to external ontologyfiles is not necesarry. Devices from pervasive systemsusually have a well defined functionality and often donot need to connect to the Internet. Therefore entriesin a knowledgebase need only reference other entriesin it.

The A-Box part of knowledgebase consists ofdata necessary for the system to make decisions,process requests, interpret messages, learn and reactto commands. A knowledgebase consists of six parts:

1) Device description: Description of the deviceusing the ontology. This part of the knowledgebasewill consist of two parts: a core ontology necessaryto characterize the device universally, and a domainspecific ontology describing entities and relations thatare restricted to one type of device (for example:lighting, sensors, multimedia devices).

2) Device capabilities: The description of devicecapabilities that are provided by the device manufac-turer. For example, capabilities are simple tasks thata device can physically perform. A simple lamp can

only turn on, turn off and dim to some level. It isimpossible for lamp to play music or accept coinsbecause these tasks are beyond a lamp’s physicalcapabilities.

3) Context and users: The context describes theactual situation of the whole system, or part ofit. Every device can understand contexts that aredescribed in its own knowledgebase. Contexts mightbe night, lunch time, reading or watching tv and areassociated with environment conditions or actionsperformed by users. Understanding context and actingupon it is one of the most important tasks of theproposed system. Context is a very broad conceptand customization for different users is important.For example watching tv for Anna means that theirpersonal phone should be off, window blinds shut andlight dimmed. The same context can mean somethingdifferent to Tomas, he wants his phone in loud mode,lights turned off, blinds closed. Devices participatingin one context can be the same but settings are dif-ferent. Reasons for this are different preferences andpriorities for particular users. The system presentedin this paper, built on concepts from [7], does nothave formal central control, so contexts are saved ondevices that are required to perform some part of thefunction required in a context. Therefore the contextwatching tv for Anna in a lamp or in a TV specifytotally different actions.

4) Configuration and state: The information aboutthe state and configuration of a device is very im-portant and should be accessible by the device. Forexample, to work as a group, devices need to beformed in a virtual device [7]. The configurationholds information in which virtual devices the de-vice participates and what is the current state ofthe configuration. A device that is a part of a vir-tual device should keep on functioning without anybreaks, if a new request arrives a device has to beable to determine if it should drop its current taskand respond to the request, by checking its currentconfiguration settings. The state of a device is alsoincluded in the knowledgebase. For example a lampcan store information about its dim level or color inthe knowledgebase where it can be easily accessed.

5) Recipes: Recipes are used to describe a pro-cedure for the behavior of every device. Recipes arevery detailed and specify the procedure that a devicehas to follow for a given context. The idea is todescribe all important steps in a recipe, so the deviceonly has to read the recipe and act upon it. Recipesare associated with context, people and request types.Since devices forming a pervasive systems have lim-ited resources, the number of requests, that a devicecan serve, is also limited. Therefore a device candecide itself if it can execute recipe or not. Theconcept of a recipe is a main subject of this paperand is described in the next section.

B. Recipes

Recipes in a knowledgebase are sequential and rep-resent steps that a device should follow in a particularvirtual device. Every recipe has a header to describethe recipe’s type, context and other conditions thatare guarding access to this recipe. The recipe shownin Figure 2 consists of n steps. Steps contain actionsthat need to be performed in this particular step. Ifthe device fails to finish a step, the recipe is dropped.

Fig. 2. Recipe structure.

Recipes are described using ontology terms, sothe concept of a recipe has to be present in theontology model. Classes and properties associatedwith a structure of a recipe are described in the T-box part of the knowledgebase in Application specificontology model layers are shown in Figure 1.

Fig. 3. An ontological view of the concept of a recipe in T-box.

A simple example of an ontological model to rep-resent recipes combined with Dolce and Sofia modelsis presented in Figure 3. This model is representingrecipes and steps as a subclass of Dolce processes.Property hasStart associates recipe with a first stepfrom this recipe, and property hasNext creates asequence of steps in a recipe.

Let’s consider a lamp and a switch that can co-operate when grouped in one virtual device. Themechanism to group and configure these devicesrequires one device to start a configuration by sendinga search message. When a device receives a searchmessage it checks if it desires to participate in avirtual device that the message propagates and ifit has a recipe that it can use in this particularconfiguration. If so, the recipe is executed, informa-tion about the configuration is saved; device sends

an acknowledgment massage back and acts uponthe instructions provided by this recipe. If a switchinitiates a virtual device with a lamp, lamp has tohave a recipe that describes, in detail, how to actupon a particular request. A recipe consist of header,that describes a recipe and steps that device has toperform. The header consist of recipe type, serviceand context that recipe is associated with. A recipecan be associated with a person and customized forthis person.

Fig. 4. Example of a recipe in knowledgebase, expressed in RDFtriples.

An example of a recipe for a lamp expressedin RDF format is shown in Figure 4. The recipe1consists of a header and four steps. Steps in a reciperepresents a sequence of actions that the device hasto perform. If a device reach the end of a recipe, atask is finished.

A device that interprets a knowledgebase needs tobe extended with a Semantic Interpretation compo-nent (Figure 5) and use a communication link toreceive and send messages necessary to establishvirtual devices and cooperate with other devices.To influence a device’s functionality, the SemanticInterpretation component uses the Device Interfacecomponent.

Fig. 5. Device’s architecture.

The addition of components specified in Figure 5and a device specific knowledgebase enables seman-tic interoperability between devices in a smart space.Recipes are used to inform a device what exact steps

it should undertake in a particular context or whenreceiving data from different devices. Since devicesmay have many recipe , choice on which recipe torun is made by the device using the content of thereceived message. If there are more than one recipesthat are usable in a certain situation, the the messagecontent is used to further refine a recipe choice. Thisis done by matching RDF triples of the recipe to fieldsof the message. This way we can make conditionalchoices on which recipe to run and there by achievedifferent device behavior.

III. IMPLEMENTATION

The smart lighting system chosen to illustrate acontext-aware distributed system consists of a lightswitch, different kinds of light sources and sensors(Figure 6). The switch has a User Interface (UI) andit can accept requests from the user. Light sources aredifferent types of lamps, with different light emissionsource types. There can be many sensors presentin the space: light sensors, motion sensors, devicesrecognizing users’ identity (for example RFID read-ers) and any other sensors that can be beneficial forthe system. As shown on Figure 6 the system hasno central control. The request comes from the UIembedded in light switch, but this does not make thelight switch a controller, it takes the role of a requestcenter. All messages sent in the system are broadcastmessages, every device receives the message anddecides if it should accept or discard it.

Fig. 6. The lighting system architecture.

Any device that can increase or reduce light levelin the room may consider itself as a provider of alighting service. Therefore lamps are not the onlysource of the illumination in a space but also windowblinds that can block light from outside or a mirrorthat reflects exterior light onto the ceiling are alsolight sources. The idea is to integrate and engage allof those elements of the system that can influenceillumination in a particular space. Applying this ap-proach can save energy by using natural light, ratherthan using only light sources.

The simulation was implemented in JCSP (Com-municating Sequential Processes for Java) [14]. Thislanguage supports concurrent programming, it is use-ful when simulating devices running simultaneously.Devices are presented as CSP processes and commu-nication links are implemented using CSP channels.

Use of JCSP enables simulating a network of manydevices running in parallel and communicating bysending messages.

Fig. 7. Proposed scenario.

The scenario chosen to describe the system ispresented on Figure 7, which shows a model of aroom that is used as an office. The space is equippedwith a light switch with UI, ceiling lamps, lamp ona desk, automatic blinds on the window, mirror usedto reflect daylight to illuminate the room, light andmotion sensors. The user is the center of attention andcan set preferences manually in the UI. The user canset lights manually or choose a context available ina particular space. When the user indicates a need tocontrol light levels or the context changes, the switchperforms a search for devices that provide service:lighting for this user. All the lamps, blinds and mirrorresponds to this request, because all of them are ableto provide light for the space.

The other task that the proposed set-up can performis to keep the light at the same level in the wholeroom, when the outside light conditions change. Thisconfiguration uses communication between light andsensors to use as much natural light as it is possible,while attaining a stable level of light in the room.

IV. RESULTS AND CONCLUSIONS

A complex system consisting of sensors and ac-tuators was designed and implemented. The systemcontrolling a space consists of autonomous devicescollaborating to achieve lighting conditions desiredby the user. There are sixteen ceiling lamps, twolight sensors, light switch, mirror and user definedcontexts. The context plays a crucial role in thesystem, as it influences behavior of devices to suitdifferent users needs.

The simulation is run on PC with Intel Core2 CPU6300@1.86GHz, 1GB RAM and Windows XP OS.Java code with JCSP libraries is run to simulate de-vices working in parallel, knowledgebases are binaryfiles containing RDF triples.

There are two available contexts in the system:reading and meeting, devices have different recipesfor different contexts.A lamp in a simulation con-sists of Semantic Interoperability and Communication

components, device functionality and knowledgebasebinary file. A lamp has 214 triples in its knowledge-base and 5 recipes that are responsible for differentactions. During configuration of a virtual device forcontext reading, the knowledgebase in the lamp isaccessed 305 times and local memory, used to keeptemporary data, is accessed 806 times.

The simulated scenario consisting on 20 devicesruns 148 Java threads. Average number of messagessent per configuration is 53 and messages received bya device is 954. All messages are broadcast, everydevice receives all messages, but most of them aredropped and do not trigger any actions.

The simulation presented in this paper runs in realtime in a users perception. The goal of this simulationis is to show functional behavior of a distributedsystem using RDF based recipes. The numbers men-tioned here should be treated as a reference. we donot attempt to to compare these numbers to anotherimplementation.

The simulation is a proof of concept that a per-vasive system can be constructed out of autonomousdevices without any central control. Recipes are de-scribed in previous sections and in [7] can made inRDF format and influence the device’s functionalityand behavior. The interoperability is achieved at asemantic level by the use of ontologies. Ontologiesare also used to construct a knowledgebase that canbe reused or transferred and understood in differentdevices.

V. FURTHER WORK

Recipes are built according to a model taken fromcommon ontology, therefore devices can process anyrecipe following these rules. The next step in devel-oping the architecture is to include an extension thatallows new recipes to be added to existing knowl-edgebase in devices from newly installed devices.Adding recipes enables updatability, both forward andbackward. When a new device is added to a network,it can update existing devices with a recipe that willguide devices how to use a new device or component.

The adaptation to a changing environment, set ofdevices in a network and user’s needs can be doneby adding a new recipe to a knowledgebase. Moresophisticated technique can update and adapt recipesthemselves, learning to adjust devices behavior de-pending on situation and improving recipes that arein device’s memory.

ACKNOWLEDGMENT

The authors would like to acknowledge Perada,EU network of Excellence, for supporting knowledgeexchange. This work is being carried out in the Sofiaproject [8].

REFERENCES

[1] T. Strang and C. Linnhoff-Popien, “Service interoperabilityon context level in ubiquitous computing environments,”L’Aquila, Italy, 2003.

[2] R. Orfali, D. Harkey, and J. Edwards, Theessential distributed objects survival guide. JohnWiley and Sons, Inc., 1995. [Online]. Available:http://portal.acm.org/citation.cfm?id=210712

[3] E. Christensen, F. Curbera, F. Curbera, G. Meredith, andS. Weerawarana.

[4] N. Georgantas, “Semantics-based interoperability for perva-sive computing systems,” 2008.

[5] D. O’Sullivan and D. Lewis, “Semantically driven serviceinteroperability for pervasive computing,” in Proceedings ofthe 3rd ACM international workshop on Data engineering forwireless and mobile access. San Diego, CA, USA: ACM,2003, pp. 17–24.

[6] T. Gruber, “Ontolingua: A mechanism to support portableontologies,” Stanford University, Knowledge Systems Labo-ratory, Tech. Rep., 1992.

[7] A. A. Syed, J. Lukkien, and R. Frunza, “An ad hocnetworking architecture for pervasive systems based ondistributed knowledge,” in Proceedings of Date2010,Dresden, 2010. [Online]. Available: http://www.date-conference.com/proceedings/PAPERS/2010

[8] Sofia, “Smart objects for intelligent applications,” 2009.[Online]. Available: http://www.sofia-project.eu

[9] C. Masolo, S. Borgo, A. Gangemi, N. Guarino, and A. Oltra-mari, “WonderWeb deliverable d18,” 2001.

[10] R. Neches, R. Fikes, T. Finin, T. Gruber, R. Patil, T. Senator,and W. Swartout, “Enabling technology for knowledge shar-ing,” AI Magazine, vol. 12, no. 3, pp. 36–56, August 1991.

[11] O. Lassila and R. R. Swick, “Resource description framework(rdf) model and syntax specification,” 1998.

[12] L. Khan and Y. Rao, “A performance evaluation ofstoring XML data in relational database managementsystems,” in Proceedings of the 3rd international workshopon Web information and data management. Atlanta,Georgia, USA: ACM, 2001, pp. 31–38. [Online]. Available:http://portal.acm.org/citation.cfm?id=502939

[13] R. Bourret, “XML and databases,” 2005. [Online]. Available:http://www.rpbourret.com/xml/XMLAndDatabases.htm

[14] P. Welch and N. Brown, “Communicating sequential pro-cesses for java (jcsp),” 2009.