Assistant agents to advice users in hybrid structured 3D virtual environments

Download Assistant agents to advice users in hybrid structured 3D virtual environments

Post on 01-Apr-2017




0 download

Embed Size (px)


<ul><li><p>COMPUTER ANIMATION AND VIRTUAL WORLDSComp. Anim. Virtual Worlds 2014; 25:495504</p><p>Published online in Wiley Online Library ( DOI: 10.1002/cav.1598</p><p>SPECIAL ISSUE PAPER</p><p>Assistant agents to advice users in hybrid structured3D virtual environmentsPablo Almajano1*, Maite Lopez-Sanchez2, Inmaculada Rodriguez2 and Tomas Trescak3</p><p>1 IIIA - CSIC, University of Barcelona, Barcelona, Spain2 University of Barcelona, Barcelona, Spain3 University of Western Sydney, Sydney, Australia</p><p>ABSTRACT</p><p>Hybrid structured 3D Virtual Environments model serious activities in immersive 3D spaces, where participants are humanand software agents, and their interactions are regulated by an Organization Centered Multi-Agent System (OCMAS). Inthis context, both OCMAS social model and the tasks that users need to accomplish can be rather complex, and thus,users may benefit from having an assistance service. Hence, we propose personal assistant agents (PA), which, based onknowledge about the OCMAS specification and current system state, provide the user with an advice (a plan) to achieve heror his goal. Additionally, we implement this service with PLAN-EA, an Extension of the A algorithm that generates plansfor a user whose actions may depend on other users actions. Thus, PAs provide plans that do not only include assisted useractions but also other users ones. We illustrate our approach by means of v-mWateran online water marketand makea comparative analysis, with and without assistance, where efficiencyin terms of number of user actionsshows animprovement (7 vs 10.8), efficacypercentage of completed tasksalso improves (93% vs 77%), and assistances overallsatisfaction is positive. Copyright 2014 John Wiley &amp; Sons, Ltd.</p><p>KEYWORDS</p><p>personal assistant agents; social model; structured 3D virtual environment; OCMAS planning</p><p>*CorrespondencePablo Almajano, IIIA - CSIC, University of Barcelona, Barcelona, Spain.E-mail:</p><p>1. INTRODUCTION</p><p>The blending of digital technologies, such as artificial intel-ligence, interactive systems, 3D interfaces and the Internet,is transforming the way people interact with computers.This blending is enabling new services for users, favouredby huge advances in the development of graphics andnetworking.</p><p>In particular, multi-user online 3D virtual environments(VE) provide users with a collaborative space not onlyfor entertainment and socialisation but also for developingserious activities like learning, negotiating and shopping.</p><p>Most often, these serious activities require the execu-tion of complex tasks, involving different subtasks, andhighly regulated interactions. Nevertheless, standard 3DVE lack of mechanisms to structure (regulate) users inter-actions. An Organization Centered Multi-Agent System(OCMAS) [1] constitutes a powerful tool that can serve thispurpose. Specifically, the marriage of 3D VE and OCMASallows the participation of both human and software (SW)agents in a so-called hybrid structured system. However, its</p><p>intrinsic complexity constitutes a serious obstacle for theparticipant system comprehension, which is seldom awareof all its runtime property values.</p><p>In this context of hybrid structured VEs, we propose Per-sonal Assistant agents (PA), which, based on knowledgeabout the OCMAS specification and current state, supportparticipants to understand the social model underlying sucha specification and to perform complex tasks in a seamlessway. We describe the overall system as a Hybrid AssistedStructured 3D Virtual Environment.</p><p>Personal assistants provide different general assistanceservices. In a previous work, we presented an InformationService for SW agents [2]. This paper focuses on the for-malisation, operationalisation, and evaluation of the AdviceService for human users. Through this advice service, aPA provides the user with a plan (i.e. a set of actionscompatible with the OCMAS specification) to accomplishher or his goal (task). PA embodiment is an Angel, whichmay accompany the user during the interactive experienceand is available under request.</p><p>Copyright 2014 John Wiley &amp; Sons, Ltd. 495</p></li><li><p>Assistant agents to advice users in hybrid structured 3D VE P. Almajano et al.</p><p>We illustrate and evaluate our approach by means ofv-mWater, a regulated virtual environment for the tradingof water rights. Our evaluation results assess that, when-ever requested, this Advice Service provides comfortableand clear guidance that facilitates task accomplishment. Infact, its usage reduces both the percentage of task failuresand the number of user performed actions, when comparedto a previous system, we evaluated without assistance [3].Assuming we think in the same type of actions (i.e. theones defined in the system specification), we consider thenumber of actions an efficiency measure: the less the num-ber of actions to do the task, the more directly users reachthe goal. The smaller number of actions also implies lesscognitive load on the user, and a reduction of the taskperceived complexity.</p><p>2. RELATED WORK</p><p>A number of works in the literature focus on (intelli-gent) virtual agents providing users with assistance. In theline of recommender systems, Dong, et al. [4] propose aReviewers Assistant to give a product reviewer optionalsuggestions. This assistant is a web browser plug-inthat employs different data mining strategies. Anotherrecommender system [5] uses a multi-agent system (MAS)populated by PA agents. A rule driven PA provides the userwith talks announcements. To do so, it exploits semanticweb data and the user profile. In these works, assis-tants serve one (assisted) single-user not involved in anyorganisation. As a remarkable difference, our PAs operatein a multi-user and structured social scenario, characterisedby complex interleaved interactions.</p><p>Lujak, et al. [6] develop Emergency Medical Assistance(EMA), a medical emergencies scenario modelled as anOCMAS, which employs web services and a 2D UserInterface (UI) to assist participants (e.g. patients, ambu-lances and medical professionals) in their coordination.RADAR [7] is another MAS with a Multi-Task Coordina-tion Assistant that observes experts and learns models toassist office workers cope with email overload. Its adviceincludes task ordering suggestions and warnings when theusers behaviour differs significantly from the experts one.They propose a 2D task-management UI. Electric Elves [8]also developed specific software personal assistants (SPA),but they were for project activities coordination and exter-nal meetings organisation in a real environment. Unlike</p><p>our shared and collaborative 3D space with embodied PAs,these works feature 2D user interfaces where the PAs arenon-embodied.</p><p>A recent work has presented a coalition planning assis-tant agent in a peacekeeping problem domain [9]. In thisscenario, the user moves in an n n grid towards a prede-fined subset of goal cells, and norms represent prohibitionsand obligations on visiting a cell. The assistant agent is ableto discover the particular users goal and execute actions(e.g. send escort request) to avoid user norm violations. Asa difference, our PA provides users with a plan (actions) toachieve an indicated goal. Moreover, their agents executeindependent actions to ensure norm compliance of usermovements. Finally, assistance has been used to supportusers physical navigation in 3D virtual environments, suchas the Virtual Theatre [10], where a non-embodied agentuses environment knowledge to indicate the user places tovisit; and the Virtual Museum [11], where a 3D characterpresents the user a precomputed guided tour to follow. OurPA is an interactive 3D character that is always available tothe user and helps her or him to achieve a given task.</p><p>3. PERSONAL ASSISTANT</p><p>This section first introduces the knowledge that a PA agentneeds to provide an advice (plan) to the human user. Next,it describes the operationalisation of the Advice Service.It illustrates them with v-mWater, an example scenarioin the agriculture domain that models an electronic mar-ket for trading water rights (i.e. the right to use water forirrigation).</p><p>3.1. Agent Knowledge: Social Model</p><p>Our structured 3D VE integrates a 3D virtual world(VW) and an OCMAS. On the one hand, the 3D inter-face facilitates the interaction of human users. On theother hand, the OCMAS infrastructure (an ElectronicInstitution [12]) regulates real-time participants interac-tions, where participants can be both human and softwareagents. A PA creates a plan for the user employing, asknowledge, the static specification of the OCMAS socialmodel (Spec) and the system current (dynamic) state (Sc).Particularly, it considers the social structure and social</p><p>Figure 1. Extract of the static specification of v-mWater social model.</p><p>496 Comp. Anim. Virtual Worlds 2014; 25:495504 2014 John Wiley &amp; Sons, Ltd.DOI: 10.1002/cav</p></li><li><p>P. Almajano et al. Assistant agents to advice users in hybrid structured 3D VE</p><p>conventions (SocStr, SocConv 2 Spec) and their runtimevalues (RtSocStr, RtSocConv 2 Sc).</p><p>Social structure (SocStr) defines the properties associ-ated to user agents (AgP), the roles (Rol) that participantsenact, and their relationships (Rel). Figure 1 depicts anextract of v-mWater specification concerning seller par-ticipants. Its SocStr (at the top) specifies two partici-pant propertiestheir location (loc) and goaland twoparticipant rolesseller (s) and market facilitator (mf ).</p><p>SocConv D hActiv, ActivRel, Proti (1)</p><p>ActivRel D hTra, Movi (2)</p><p>mov D hmRol, ori, des, mTi (3)</p><p>prot D hProtP, ProtSpeci (4)</p><p>ProtSpec D hProtC, Nod, Illoi (5)</p><p>ProtC D fhrolC, min, maxig (6)</p><p>nod D hEnterRol, ExitRoli (7)</p><p>illo D hsRol, rRol, cM, ori, desi (8)</p><p>Social conventions (Equation (1)) stand for the rules ofthe game. Participants (roles) gather in activities (Activ).There, they can perform tasks by following protocols(Prot) or move to other related activities (ActivRel). In ourexample of Figure 1, seller participants can enter/leave thesystem in the Initial/Final activity, ask for market informa-tion in the Waiting&amp;Info activity and register a water rightin the Registration. Activities relationships (ActivRel) inEquation (2) constraint the flow of roles among activities.Transitions (Tra) are intermediate locations meant for usersynchronisation. Movements (mov 2 Mov, Equation (3))are participant actions that change their location. They aredefined by (i) its role (mRol); (ii) the transition and activityit connects (ori, des 2 fTra [ Activg, ori being origin anddes destination); and (iii) its type (mT 2 {new, enter,exit}, meaning create and enter a new activity, enter andexit an existing activity). Figure 1 represents a directedgraph where nodes correspond to both activities (rect-angles) and transitions (diamonds), and edges (labelledarrows) represent allowed movements between them.</p><p>Protocols structure user interactions within activities.Equation (4) defines a protocol (prot 2 Prot) as a setof properties (ProtP) and its specification (ProtSpec).ProtSpec (Equation (5)) is a finite state machine, wherenodes (Nod) correspond to the states and illocutions (Illo)to state transitions. Figure 1 depicts protocols nodes as cir-cles and illocutions as labelled arrows. Those nodes wherethe protocol execution starts and ends are named initial andfinal, respectively. Moreover, the protocol capacity (ProtCin Equation (6)) defines a set of tuples that bound the num-ber of participants that enact each role (rolC): min sets theminimum number required to open the protocol, whereasmax sets the maximum number of allowed participants in</p><p>this protocol. We can also specify, for each protocol node(nod 2 Nod) in Equation (7), the list of roles that can join(EnterRol) and leave (ExitRol) the activity when it is at thisprotocol state. For example, in the Registration activity ofFigure 1, users playing seller role are allowed to enter (Cs)and exit (s) whenever its protocol state is node n1.</p><p>Finally, illocutions are messages that participants inter-change. They act as transitions of the finite state machine,so they imply a change in the protocol execution state(node). Thus, for instance, any illocution uttered at aninitial node opens the protocol. An illocution (illo inEquation (8)) formalises a participant action by specifyingthe following: (i) the role of the sender (sRol) or all, incase anybody can send it; (ii) the role of the receiver (rRol)or all, if it is public and everybody will receive it; (iii)the message content (cM), conforming an ontology; (iv)the origin node (ori) where the message can be sent; and(v) the destination node (des), the node reached once theillocution is successfully uttered. In Figure 1, illocutionregister (s, mf , hw, pi) in Registration specifies the requestfrom a seller (s) to a market facilitator (mf ) the registrationof a water right (w) to a given price (p). It can be utteredwhen the protocols state is node n1 and, once performed,the protocol state will transit to n2.</p><p>Execution State (Sc) contains current runtime values ofdynamic organisational elements, such as actual partici-pants (RtAg 2 RtSocStr) and running activities (RtActiv 2RtSocConv).</p><p>3.2. Advice Service: Operationalisation</p><p>Activities in the social model specify user interactions sothat certain tasks can be performed (e.g. users can registerwater rights in the Registration activity). Whenever a useraims to perform one of such tasks, she or he can request anadvice service to her or his PA. This PA will then respondwith a plan (pl D fa1, : : : , amg) consisting of a sequenceof m actions (movements and illocutions). The providedadvice (plan) will conform to the social model specificationand, if executed at the current system state, will lead theuser to her or his goal of performing the task.</p><p>Equation (9) formalises this advice service. When auser request (Req) is received, the PA uses social modelspecification (Spec) and current state (Sc) to provide aresponse (Res). Both Req and Res are messages, whichcontain a sender, a receiver and a content. In the former(Equation (10)), the user asks for PAs advice to perform atask. In the latter (Equation (11)), the PA answers the userwith a plan (pl).</p><p>Advice : Req Spec Sc ! Res (9)</p><p>Req D hrtAg, PA, taski (10)</p><p>Res D hPA, rtAg, pli (11)</p><p>A PA is graphically represented as an Angel bot, aninteractive non-player 3D character. Figure 2 shows a</p><p>Comp. Anim. Virtual Worlds 2014; 25:495504 2014 John Wiley &amp; Sons, Ltd. 497DOI: 10.1002/cav</p></li><li><p>Assistant agents to advice users in hybrid structured 3D VE P. Almajano et al.</p><p>Figure 2. Marys avatar with her Personal Assistant in the 3D VW.</p><p>snapshot of our 3D UI, with user Mary (female user avatar)and her PA (Angel bot). PA always accompanies the userduring the interactive experience so it is always availablefor her. Nevertheless, she can activate (show) her PA, inter-act with it whenever she needs help or deactivate (hide) itotherwise.</p><p>User-PA interaction has been implemented in the fol-lowing way: (i) A user performs the action touch to he...</p></li></ul>