a framework for context-aware digital signage

8/14/2019 A Framework for Context-Aware Digital Signage

1/12

A Framework for Context-Aware Digital Signage

Ichiro Satoh

National Institute of Informatics

2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan

[email protected]

Abstract. A framework for building and operating context-aware multimedia

content on digital signage in public/private spaces is presented. It enables ac-

tive and multimedia content to be composed of mobile agents, which are self-

contained programmable entities that can travel from computer to computer and

provide multimedia content for advertising or user-assistant services to users byusing their own programs. The framework automatically deploys their agents at

computers close to their current positions to provide advertising or annotations

on objects or users. To demonstrate the utility of the framework, we present a

user-assistant that enables shopping with digital signage.

1 Introduction

Active media have been expected to play an important role in advertising, navigation,

assistance, or entertainment. For example, modern out-of-home (OOH) advertising hasfocused on marketing to consumers according to their contexts, e.g., when and where

they are on the go in public places, in-transit, or waiting. Ambient services are ser-

vices that are related to the surrounding physical environment of the user, and can be

considered to be a form of location-based services. For example, active media should

be selected and adapted to according to the users context.

Different kinds of ambient shopping services have been developed to aid users in-

cluding brokerage services, navigational services, comparison shopping, product

searches, location-based auctions, wireless advertising, and even information services

where a user can point a device at a particular product or shop to find out more about it[2,4,5,7,9]. Active media may be useful for advertising products at retailers and public

spaces, but they should be used for manufacturing and distributing these products and

assisting customers while they are using them and after they have bought them.

This paper presents a framework for building and managing active media in large-

scale ambient computing environments. The framework monitors contextual informa-

tion in the real world by means of sensing systems and selects and adapts active media

according to changes in the real world. It provides massive users with active media

at massive ubiquitous and mobile computing devices in their personalized forms ac-

cording to their contexts. It enables us to easily build and provide a variety of activemedia, including interactive content. The framework enables creators, who may have

no professional knowledge about active media, to compose active media from active

and programmable contents. Active media have the ability to monitor and response to

user behaviors, because advertising or user-assistant services provided as active media

should adapt to the user profiles.

N. Zhong et al. (Eds.): AMT 2011, LNCS 6890, pp. 251262, 2011.

c Springer-Verlag Berlin Heidelberg 2011


2/12


3/12

A Framework for Context-Aware Digital Signage 253

be present at the computer only while the computer needs the active media provided

by that agent. Nevertheless, mobile agent technology itself does not support any active

media or context-awareness. The framework offers several functions to enable us to

easily build and operate context-awareness active media. It manages location-sensing

systems and deploys mobile agents for active media at appropriate computers accordingto the locations of target users. Furthermore, the activation of active media is classified

into three patterns according to the time services are activated. A service is provided in

a place, i) before its target is at the location, ii) while its target is at the place, or iii)

after its target has left the location.

3 Design and Implementation

The framework consists of two parts: (1) mobile agents and (2) location informationservers, called LISs (Fig.1). The former offers application-specific active media, which

are attached to physical entities and places, as collections of mobile agents. The latter

provides a layer of indirection between the underlying locating sensing systems and

mobile agents. Each LIS manages more than one sensor and provides the agents with

up-to-date information on the state of the real world, such as the locations of people,

places, and things, and the destinations that the agents should migrate themselves to.

Eventdispatcher

Abstraction Filter

Runtime system

Peer-to-peer

communication

Agentmigration

Location-sensingsystem (Proximity)

Location-sensingsystem (Lateration)

Location-sensingsystem (Proximity)

Spot 1

LIS

Agent host

Spot 2

Agent host Agent host Agent host

Agentinformationdatabase

Hostinformationdatabase

Contextual event manager

Abstraction Filter

Spot 3 Spot 4

Abstraction Filter

Contextual event manager

Active mediaagent

Runtime system

Active mediaagent

Runtime system Runtime system

Active mediaagent

Eventdispatcher

LIS

Agentinformationdatabase

Hostinformationdatabase

Fig. 1.Architecture


4/12

254 I. Satoh

3.1 Location Information Server

Each LIS can run on a stationary or mobile computer. It manages multiple sensors that

detect the presence of tags and maintains up-to-date information on the identities of tags

that are within the zone of coverage of its sensors. This is achieved by polling sensors

or receiving the events issued by the sensors themselves. An LIS does not require any

knowledge of other LISs. To conceal the differences between the underlying locating

systems, each LIS maps low-level positional information from each of the locating sys-

tems into information in a symbolic model of location. An LIS represents an entitys

location, called a spot, e.g., space of a few feet, which distinguishes one or more por-

tions of a room or building. The current implementation assumes that RFID tags that

periodically broadcast beacons (e.g., every second), including the identifiers of the tags,

are attached to people and physical entities and each spot has an RFID reader that can

receive the identifiers of tags within its range of coverage (1-20 meters). Therefore, it

can detect the presence (or absence) of a tag attached to a person or physical entity in a

spot.

When an LIS detects a new tag in a spot, it multicasts a query that contains the

identity of the new tag and its own network address to all the agent hosts in its current

sub-network. It then waits for reply messages from the agent hosts. Here, there are two

possible scenarios: the tag may be attached to an agent host or the tag may be attached

to a person, place, or thing other than the agent host.

In the first, the newly arriving agent host will send its network address and device

profile to the LIS; the profile describes the capabilities of the agent host, e.g., itsinput devices and screen size. After the LIS receives the reply, it stores the profile

in its database and forwards the profile to all agent hosts within the spot.

In the second, agent hosts that have agents tied to the tag will send their network

addresses and the requirements of acceptable agents to the LIS; the requirements

of each agent specify the capabilities of the agent hosts that the agent can visit and

provide its active media at.

If the LIS does not have any reply messages from the agent hosts, it can multicasts a

query message to other LISs. When there are multiple candidate destinations, each of

the agents that is tied to a tag can select one destination on the basis of the profiles of

the destinations. When the absence of a tag is detected in a spot, each LIS multicasts a

message with the identifier of the tag and the identifier of the spot to all agent hosts in

its current sub-network.

3.2 Agent Host

Each agent host has two forms of functionality: one is for advertising its capabilities

and the other for executing and migrating mobile agents. When a host receives a query

message with the identifier of a newly arriving tag from an LIS, it replies with one of thefollowing three responses: (i) if the identifier in the message is identical to the identifier

of the tag to which it is attached, it returns profile information on its capabilities to the

LIS; (ii) if one of agents running on it is tied to the tag, it returns its network address and

the requirements of the agent; and (iii) if neither of the above cases applies, it ignores

the message.


5/12


Each agent host is responsible for executing and migrating agents to other runtime

systems running on different computers through a TCP channel using mobile-agent

technology (Fig. 3). It is built on the Java virtual machine (Java VM) version 1.5 or

later versions, which conceals differences between the platform architectures of the

source and destination computers. It governs all the agents inside it and maintains thelife-cycle state of each agent. When the life-cycle state of an agent changes, e.g., when it

is created, terminates, or migrates to another runtime system, its current runtime system

issues specific events to the agent. Figure 2 shows that, when a product is carried by a

customer, a mobile agent bound to the product is migrated from computer to computer.

Factory Wholesaler Retailer End-consumer

ProductAgent

migrationAgent

migrationAgent

migration

Advertising How to useItem

informationProduction

information

Fig. 2.Forwarding agents to digital signage when user moves

When an agent is transferred over the network, not only its code but also its state

is transformed into a bitstream by using Javas object serialization package and then

the bit stream is transferred to the destination. Since the package does not support the

capturing of stack frames of threads, when an agent is deployed at another computer,

its runtime system propagates certain events to instruct it to stop its active threads.

Arriving agents may explicitly have to acquire various resources, e.g., video and sound,

or release previously acquired resources.

3.3 Mobile Agent-Based Active Media

Each agent is attached to at most one visitor and maintains his/her preference infor-

mation and programs that provide annotation and navigation to him/her (Fig.3). Each

agent keeps the identifier of the tag attached to its visitor. To support user/location-

dependent active media, each agent is dynamically assembled from the content and

user-preferenceparts.

Content part: This part is responsible for selecting and playing annotations according

to the current spot and route in addition to the information stored in the user-preference

part and it plays the content in the personalized form of its user. It is defined as a set of

content-selection function and programs to play the selected content. The content part

contains a content-selection function and a content player.


6/12

256 I. Satoh

Mobile agent-based ambient media

Agent runtime system

Java VM / OS / Hardware

External

ambientmediacontent Agent migration

manager

Agent execution

manager

Agent lifecycleevent

dispatcher

Built-in serviceAPIs

Annotation part

Contentselectionfunction

Content playerprogram

Ambient media

content

Agent state

manager

User-preference part

Knowledge

Interests

RFID Tag ID

Fig. 3.Architecture of runtime system for service-provider agent

Content-selection function: This function maps more than one argument, e.g., the cur-

rent spot, the users selected route, and the number of times he/she has visited the spotinto a URL referring to the annotative content. The content can be stored in the agent,

the current runtime system, or external http servers. That is, each agent can carry a

set of its content, play the selected content at its destinations, directly play the content

stored at its destinations, or download and play the content stored in Web servers on the

Internet. The current implementation can divide this part into three sub-parts: opening,

annotations, and closing, which are played in turn.

Content player: Annotation content varies, e.g., it can be text, image, video, or sound.

The annotation part defines programs to play this content. The current implementation

supports (rich) text data, html, image data, e.g., JPEG and GIF, video data, e.g., anima-tion GIF and MPEG, and sound data, e.g., WAV and MP3. The format for content is

specified in an MIME-based attribute description. Since the annotation part is defined

as Java-based general-purpose programs, we can easily define interactions between vis-

itors and agents.

User-preference part. This is responsible for maintaining information about a visitor.

In fact, it is almost impossible to accurately infer what a visitor knows or is interested

in from data that have been measured by sensing systems. Instead, the current imple-

mentation assumes that administrators will explicitly ask visitors about their knowledgeand interests and manually input the information into this part. Nevertheless, it is still

possible to make an educated guess with some probability as to what a visitor may be

interested in, if we know which spots he/she has visited, how many he/she has visited,

and how long he/she has visited. Each agent has a mechanism to automatically record

the identifiers, the number of visits to, and the length of stays at spots by visitors.


7/12


3.4 Current Status

This section describes the current implementation of our system. It was implemented

using Suns Java Developer Kit version 1.5 or later versions.

Support for location-sensing systems: The current implementation supports two com-mercial tracking systems. The first is the Spider active RFID tag system, which is a

typical example of proximity-based tracking. It provides active RF-tags to users. Each

tag has a unique identifier that periodically emits an RF-beacon (every second) that con-

veys an identifier within a range of 1-20 meters. The second system is the Aeroscout

positioning system, which consists of four or more readers located in a room. These

readers can measure differences in the arrival times of WiFi-based RF-pulses emitted

from tags and they estimate the positions of the tags from multiple measurements of

the distance between the readers and tags; these measurement units correspond to about

two meters.

Security and privacy: The framework only maintains per-user profile information within

those agents that are bound to the user. It promote the movement of such agents to ap-

propriate hosts near the user in response to the users movements. Thus, the agents do

not leak profile information on their users to other parties and they can interact with their

mobile users in personalized form that has been adapted to respective, individual users.

The runtime system can encrypt agents to be encrypted before migrating them over a

network and then decrypt them after they arrive at their destination. Moreover, since

each mobile agent is just a programmable entity, it can explicitly encrypt its particularfields and migrate itself with these fields and its own cryptographic procedure. The Java

virtual machine can explicitly restrict agents to only access specified resources to pro-

tect hosts from malicious agents. Although the current implementation cannot protect

agents from malicious hosts, the runtime system supports some authentication mech-

anisms for agent migration so that each agent host can only send agents to and only

receive agents from the trusted hosts.

Performance evaluation: Although the current implementation was not built for perfor-

mance, we measured the cost of migrating a null agent (a 5-KB agent, zip-compressed)

and an annotation agent (1.2-MB agent, zip-compressed) from a source host to a des-tination host that was recommended by the LISs. The latency of discovering and in-

structing an agent attached to a tag after the CDS had detected the presence of the tag

was 420 ms and the respective cost of migrating the null and annotation agent between

two hosts over a TCP connection was 38 ms and 480 ms. This evaluation was carried

out with three computers (Intel Core 2 Duo 2 GHz with Windows XP Professional and

JDK 1.5) connected via a Fast Ethernet. This cost is reasonable for migrating agents

between computers that follow visitors moving between exhibits.

4 Applications

We experimented on and evaluated mobile agent-based active media for appliances,

e.g., electric lights. These were unique to other existing active media because they did

not support advertising of their target appliances but assisted users in controlling and


8/12

258 I. Satoh

disposing of them. We attached an RFID tag to an electric light and provided a mo-

bile agent as active media for the light. The active media were attached to their target

item and were deployed at computers close to the current position of the item. The cur-

rent implementation assumed that an agent to manage the active media for their target

appliance was created when the appliance was shipped from the factory.

4.1 Location-Aware Active Media

Since an agent defines programs to display three kinds of active media content inside

it, it selects them according to their spaces. It supports the lifecycles of the items from

shipment, showcasing, assembly, use through to disposal.

In warehouse: While the light is in a warehouse, its agent is deployed at a computer in

the warehouse. It notifies a server in the warehouse of its specification, e.g., its product

number, serial number, date of manufacture, size, and weight of it.

In store: While the light is being showcased in a store, its agent is deployed at a com-

puter close its target object, which displays advertising media to attract purchases by

customers who visit the store. Figures 4a) and b) have two images maintained in the

agent that display the price, product number, and manufactures name on the current

computer.

In house: When a light is bought and transferred to the house of its buyer, its agent

migrates to a computer in the house and provides instructions on how it should be

assembled. Figure4 c) has the active media for advice on assembly. The agent alsoadvises how it is to be used as shown in Fig. 4d). When it is disposed of, the agent

presents its active media to give advice on disposal. Figure 4 e) has an image that

illustrates how the appliance is to be disposed of.

4.2 Appliance Control through Active Media

We can define agents that control appliances, which may not have any network inter-

faces. In a previous project [8], we developed a generic server to control power outlets

through a commercial protocol called X10. In both the approaches we describe here,the lights are controlled by switching their power sources on or off through the X10

protocol, which involves a user-aware automatic controller and a remote controller.

User-aware automatic controller: The first can autonomously turn room lights on

whenever a user with a tagged user is sufficiently close to them. The agent attached to

the light can also work as our X10-based servers client and runs on a stationary agent

host in the room. When a tagged user approaches the light, an LIS in the room detects

the presence of his/her tag in the cell that contains the light. The LIS then moves the

agent that is bound to his/her tag to the agent host on which the lights agent is running.

The users agent then requests the lights agent to turn the light on through inter-agent

communication.

Remote controller: The second allows us to use a PDA to remotely control nearby

lights. Place-bound controller agents, which can communicate with X10-base servers

to switch lights on or off, are attached to locations with room lights in this system.


9/12


a) In-store ambient media

c) In-house ambient media for assembly guide d) In-house ambient media for using guide

e) In-house ambient media for disposal guide

b) In-store ambient media

Fig. 4.Active media for appliance

Each user has a tagged PDA, which supports the agent host with WindowsCE and a

wireless LAN interface. When a user with a PDA visits a cell that contains a light, the

framework moves a controller agent to the agent host of the visiting PDA. The agent,now running on the PDA, displays a graphical user interface to control the light. When

the user leaves that location, the agent automatically closes its user interface and returns

to its home host.

5 Related Work

There have been several active media with the aim of enabling users to watch/listen to

context-aware information, e.g., annotation about exhibits at the right time and in the

right place. Watson et al.[14] have proposed the term u-commerce(or ubiquitous com-merce), which is defined as the use of ubiquitous networks to support personalized and

uninterrupted communications and transactions between a firm and various stakehold-

ers to provide a level of value over, above and beyond traditional commerce. A number

of architectures and prototypes of u-commerce systems have been described in the lit-

erature [3]. The Shoppers Eye [2]proposed a location-aware service with wirelessly


10/12

260 I. Satoh

RFID-tag

attached to

a desklamp

RFID-readerPDA

(Agent Host)

Desklamp

X10 Appliance

Module

Controller

Agent

Fig. 5.Controlling desk lamp from PDA

enabled portable terminals, e.g., PDAs and smart phones. As a shopper travels about,

his or her personal terminal transmits messages, which include information about his

or her location, shopping goals, preferences, and related purchase history. When this

information is received, stores create a customized offer of goods and services. The Im-

pulse project [13] is a PDA-based system whereby customers may add products to a list,

indicating preferences such as warranty terms, merchant reputations, availability, timelimits for the purchases and preferred price. When a potential customer enters shopping

zones, individual stores, or shopping malls, his/her agent engages nearby merchants in

a silent exchange seeking items on the list and opens negotiations on the terms of the

sale, alerting the shopper if a deal has been agreed on. It is envisaged that the merchant

gains valuable information about customers purchasing behavior during the negotia-

tion process. The MyGROCER project [5]provided smart shopping carts, which could

automatically scan supermarket products while simultaneously providing valuable in-

formation to the customer through a display device, thus creating a fully interactive

shopping trip. The Easishop project[4] has used intelligent agents running on mobilephones to support a proximity-based reverse auction service.

The PEACH project [10] has developed and evaluated a visitor-guide system for

use in public spaces. The system supported PDAs in addition to ambient displays and

estimated the locations of visitors by using infrared light and computer vision. The

project proposed a system that enabled agents to migrate between computers [6] by

displaying an image of an avatar or character corresponding to the agent on remote

computers; however, it could not migrate agents themselves to computers. Like the

PEACH project, several existing systems have introduced the notion of agent migration,

but they have only supported the images of avatars or codes with specified pieces ofinformation, instead of the agents themselves. Therefore, their services have not been

defined within their agents independent of their infrastructures, so that they have not

been able to customize multiple services while the infrastructures were running, unlike

our system. The Virtual Anatomy Assistant Ritchie[15] was an attempt to seamlessly

integrate characters into user actions by using 3D-optical tracking systems but it focused


11/12


on conversations between individual users and agents and did not support any multiple

user settings.

We also need to discuss differences between the framework presented in this paper

and our previous frameworks. We previously presented an approach for deploying mo-

bile agents that were spatially bound to physical places and objects at computers. Theagents moved in the places or were close to the objects [11]. However, the approach was

not designed for user-navigation, unlike the framework proposed in this paper. We also

constructed a location model for ubiquitous computing environments. It represents spa-

tial relationships between physical entities (and places) as containment relationships

between their programmable counterpart objects and deployed counterpart objects at

computers according to the positions of their target objects or places [12]. This was a

general-purpose location-model for context-aware services, but was not an infrastruc-

ture for deploying and operating such services.

6 Conclusion

We designed and implemented a context-aware infrastructure to build and manage

mobile agent-based active media. It provides users and physical entities with mobile

agent-based active media to support and annotate them. Using location-tracking sys-

tems, it can migrate active media to stationary or mobile computers near the locations

of users and physical entities to which the agents are attached. In addition, in a non-

centralized manner to support large-scale context-aware systems, the system is man-

aged. This framework was originally constructed as a part of our ambient computing

project for future shopping. The project involves three research sub-projects in addition

to the framework presented in this paper: low-carbon emission logistics, RFID-enabled

returnable containers in supply chains, and RFID-enabled carbon credit trading. We

plan to integrate the framework with these projects.

Acknowledgments. This research was supported in part by a grant from the Promotion

program for Reducing global Environmental loaD through ICT innovation (PREDICT)

made by the Ministry of Internal Affairs and Communications in Japan.

References

1. Cumby, C., Fano, A., Ghani, R., Krema, M.: Building intelligent shopping assistants using

individual consumer models. In: Proceedings of International Conference on Intelligent User

Interfaces (IUI 2005), pp. 323325. ACM Press, New York (2005)

2. Fano, A.: Shoppers eye: using location-based filtering for a shopping agent in the physical

world. In: Proceedings of International Conference on Autonomous Agents, pp. 416421.

ACM Press, New York (1998)

3. Galanxhe-Janaqi, H., Nah, F.F.-H.: U-commerce: emerging trends and research issues. In-

dustrial Management and Data Systems 104(9), 744755 (2004)

4. Keegan, S., OHare, G.M.P., OGrady, M.J.: Easishop: Ambient intelligence assists everyday

shopping. Information Sciences 178, 588611 (2008)

5. Kourouthanassis, P., Roussos, G.: Developing Consumer-Friendly Pervasive Retail Systems.

IEEE Pervasive Computing 2(2), 3239 (2003)


12/12

262 I. Satoh

6. Kruppa, M., Kruger, A.: Performing Physical Object References with Migrating Virtual

Characters. In: Maybury, M., Stock, O., Wahlster, W. (eds.) INTETAIN 2005. LNCS (LNAI),

vol. 3814, pp. 6473. Springer, Heidelberg (2005)

7. Meschtscherjakov, A., Reitberger, W., Lankes, M., Tscheligi, M.: Enhanced shopping: a dy-

namic map in a retail store. In: Proceedings of the 10th International Conference on Ubiqui-

tous Computing (UBICOMP 2008), pp. 336339. ACM Press, New York (2008)

8. Nakajima, T., Satoh, I., Aizu, H.: A Virtual Overlay Network for Integrating Home Appli-

ances. In: Proceedings of International Symposium on Applications and the Internet (SAINT

2002), pp. 246253. IEEE Computer Society, Los Alamitos (2002)

9. Reitberger, W., Obermair, C., Ploderer, B., Meschtscherjakov, A., Tscheligi, M.: Enhancing

the Shopping Experience with Ambient Displays: A Field Study in a Retail Store. In: Schiele,

B., Dey, A.K., Gellersen, H., de Ruyter, B., Tscheligi, M., Wichert, R., Aarts, E., Buchmann,

A. (eds.) AmI 2007. LNCS, vol. 4794, pp. 314331. Springer, Heidelberg (2007)

10. Rocchi, C., Stock, O., Zancanaro, M., Kruppa, M., Kruger, A.: The Museum Visit: Gen-

erating Seamless Personalized Presentations on Multiple Devices. In: Proceedings of 9thInternational Conference on Intelligent User Interface, pp. 316318. ACM Press, New York

(2004)

11. Satoh, I.: SpatialAgents: Integrating User Mobility and Program Mobility in Ubiquitous

Computing Environments. Wireless Communications and Mobile Computing 3(4), 411423

(2003)

12. Satoh, I.: A Location Model for Smart Environment. Pervasive and Mobile Computing 3(2),

158179 (2007)

13. Tewari, G., Youll, J., Maes, P.: Personalized location-based brokering using an agent-based

intermediary architecture. Decision Support Systems 34(2), 127137 (2003)

14. Watson, R.E., Pitt, L.F., Berthon, P., Zinkhan, G.M.: U-commerce: expanding the universeof marketing. Journal of the Academy of Marketing Science 30(4), 333347 (2002)

15. Wiendl, V., Dorfmuller-Ulhaas, K., Schulz, N., Andre, E.: Integrating a Virtual Agent into

the Real World: The Virtual Anatomy Assistant Ritchie. In: Pelachaud, C., Martin, J.-C.,

Andre, E., Chollet, G., Karpouzis, K., Pele, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722,

pp. 211224. Springer, Heidelberg (2007)

a framework for context-aware digital signage

Documents