Touring Machines:Guide Agents for Sharing Stories about Digital Places

Katherine IsbisterOpen Laboratory

NTT Communication Science Laboratories2-4, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237

JAPANkatherine@digitalcity.gr.jp

Patrick Doyle Stanford University

Gates Computer Science Building 2AStanford, CA 94305-9020pdoyle@cs.stanford.edu

Abstract

Human understanding of the physical world frequently de-rives from the narratives we c onstruct about it . Analo-gously, we can incorporate narratives in virtual spaces togive visitors a sense of place and purpose. In this paper wedescribe two projects using interactive characters as a me-dium for story building and storytelling within and about thedigital places they inhabit.

IntroductionAccording to (Schank and Abelson 1995), the bulk of hu-man knowledge a nd memory is communicated and en-coded in story form. However, there is currently little ex-plicit use of story in the repertoire of interface a gents.Many of the interface agents in widespread use a re e m-ployed largely for simple data-driven tasks such as filesearches or context-sensitive help, in which the agents aresimply embodied, rather than menu- or prompt-driven,interfaces, but without possessing intelligent believability(Ball et al. 1997; Rist et al. 1997). There has also b eenactive research on pedagogical or utilitarian agents that doexhibit intelligent behavior, though they create narrativesonly in the sense that their pedagogical requirements im-pose a structured sequence of behaviors (e.g. (Rickel andJohnson 1997; Lester and Stone 1997)). Finally, there hasbeen work on the creation of stories through the interactionbetween the user and agents that are characters in the story(Bates 1992; Hayes-Roth et al. 1997). In these systems,story is both the mechanism of the interaction and its pur-pose.

In contrast, our interest is in the explicit use of narrativeto help users organize familiar ideas, learn new material,and more e ffectively engage them with this content. Inparticular, we are interested in situated stories—stories thatare tied closely to the environment at hand, which can il-lustrate it and bring it to life for the visitor. In this paper,we will discuss two p rojects that attempt t o incorporatesituated storytelling into the behavior of an agent.

Both of us are working on projects that involve creatinginterface agents that can act in the human role of “guide”to a digital location. In examining qualities a human guidemust possess, we learned that “interpretation” of the loca-

tion is one of the most important t asks. Interpreting isbringing a place to life using one’s own perspective a ndartifacts at hand, and usually involves telling stories tovisitors, to help make the account more vivid and emotion-ally engaging to them (Pond, 1993).

We will briefly summarize the structures of our projectsand how we are attempting to build guides capable of thissort of interpretation. We discuss the direction this re-search is taking, and close with a list of challenges foragent t our guide design that we hope will also serve a sguidelines for their construction.

Kyoto Digital City Tour GuideIsbister is currently part of a team that is building a digitalversion of Kyoto, Japan (http://www.digitalcity.gr.jp). Thisonline c ity is meant t o b e a n evolving resource both foroutsiders and for Kyoto residents, with a rich store of use-ful local information. It currently consists of a database ofgeographically-arranged links to local websites, with 2Dand 3D interfaces for browsing these links. The 3D inter-face to the city is being constructed in a new developmentenvironment called 3 DML. 3DML uses texture mappingand simple geometric forms to enable users to rapidly cre-ate photo-based 3 D worlds. The worlds can be e xploredusing Doom-like navigation. Visitors can also click onbuildings to open related Web links.

One of Isbister’s tasks is to ensure that the Kyoto digitalcity will be an inviting place for real people to use—a trulyuseful and viable tool. As a visitor herself, she is focusingon making the site approachable and engaging for outsiderswho might want to someday visit Kyoto. To help accom-plish this goal, she is creating an agent-led group chat tourof the city. The tour will be a point of entry to the onlineresource and to Kyoto itself, ideally increasing visitor in-terest in and use of the digital city. Isbister hopes the grouptour will also encourage dialogue and relationships amongthose who p articipate, as well as increase e xposure toKyoto’s history among those who are friends and family ofpeople who participate in the tour.

In creating the agent, Isbister is focusing on developingstorytelling strategies that will produce an engaging expe-rience for tour takers. She derived a list of target abilitiesfor the a gent by researching the behavior of actual t our

guides in Kyoto. Isbister hopes that the process of devel-oping the agent’s storytelling abilities will lead to a contri-bution to the narrative intelligence/agent research domain,specifically involving timing and d uration strategies forsituated storytelling.

Researching the Tour Guide RoleIn preparation for creating the tour guide a gent, Isbisterwent on several guided tours of Kyoto, making notes abouthow the tour guides did their work. She found that t ourguides made use of illustrative stories frequently, supple-menting the rich visual environment of Kyoto with expla-nations of how Japanese people, both past and p resent,made use of the settings that were visited. Stories includedthings such as: descriptions of how a given site was con-structed and its history of destruction and reconstruction;descriptions of peak h istoric e vents that happened at agiven site; and descriptions of seasonal events and custom-ary activities that occur at the site. Tour guides also rein-corporated material from previous tours, discussing whatother visitors said and did when they visited the sites.

The tour guides would introduce the stories upon arrivalat the site, and at specific points in the site that were di-rectly relevant to the particular story. While visitors lookedat t he buildings, and took things in v isually, the guidewould create a narrative context for the site with these sto-ries, providing v isitors with stories that t hey could sharewith their fellow tour members, as well as with peopleback home.

The tour guide would time his or her storytelling to thevisitors’ absorption with the site at hand. That is, the guidewould p rovide story context while the visitors were e n-gaged with the site, and move on when it became clear thatthe visitors were through looking at the site and were readyfor a new venue. In addition, the guide would provide fol-low-up stories to those that were met with particular inter-est by visitors.

Isbister once worked as an exhibit sign writer at a zoo,and noticed another quality that should b e c onsidered indeveloping tour guide a gents. At the z oo, parents oftenread the e xhibit t ext and then retold the c ontents of thesigns to their children on the spot. For example, childrenwere most likely to be interested in stories about how ani-mals behave or their unusual characteristics. So a parentwould find a fact like this on the sign, and turn and tell thechild this particular piece of information, omitting the rest.This kind o f content repurposing for particular audiencemembers is an important goal to aim for in designing one-on-one agent guides, and will be discussed a bit later on inthe section on the virtual art museum tour guide agent.

To summarize, Isbister observed the following qualitiesin human tour guide storytelling to groups:

1. Stories were told about particular locations, whilevisitors looked at those locations.

2. Some stories included material that t our guides rein-corporated from previous tours.

3. Stories that the guide selected seemed to be those thatwould lead to easy retelling by visitors to others.

4. Guides adjusted the timing of storytelling as well aschoice of f ollow-up stories based o n v isitor interestlevel.

Agent Construction StrategyIsbister is currently compiling a database of stories that arerelated to the sites that will be part of the Kyoto DigitalCity. She is categorizing these stories along several dimen-sions: what specific location the story is linked to, type ofstory (historic e vent, construction-based, tour anecdote,etc.), and length of story version. Isbister plans to haveshort, medium, and long versions of each story, to allowfor length adaptation by the agent to tour member interestlevel. She also hopes to tag these stories with general inter-est ratings once she has feedback from actual prototypetour takers about which stories are the most interesting tothem. Finally, stories will be grouped based on similarity,to allow the agent to tell a series of related stories.

Isbister will use this database of stories to create a nagent-led tour of the 3D interface to the digital city. Thetour will be implemented within the World Wide Web en-vironment, using I-Chat and Microsoft’s Peedy Agent (seeFigure 1). Peedy’s tour narrative will be triggered b y thegroup’s approach to each of the sites on the tour. Isbisteris currently planning to vary Peedy’s narrative to adapt toeach visitor group’s interest level on the following dimen-sions:

1. Story duration.2. Number of stories.3. Length of stay at a given site.

Figure 1. Prototype of the Kyoto Digital City Tour

Currently, plans for tracking visitor interest include not-ing the amount of conversation among visitors within the

chat environment, searching for positive key words in con-versation while at the site, as well as directly asking visi-tors in prototype tours to rate stories and to indicate thatthey would like more information and/or stories about asite. Isbister plans to develop and incorporate direct feed-back mechanisms into the final interface to the guide agent,as well.

A Virtual Art MuseumDoyle has arrived at t he same problem from a differentdirection. His recent work has focused on increasing theeffectiveness of intelligent characters in v irtual spaces byadding annotations to those e nvironments. Annotationsare declarations and procedures embedded in the environ-ment and made accessible to wandering agents in order togive those agents useful information about available con-tent and activities, as well as to assist them in the choiceand timing of their actions.

Previous explorations have taken place in text-based en-vironments called MUDs, where agents have used annota-tions to solve puzzles, play games, and enhance their emo-tional intelligence with respect to the environment (Doyleand Hayes-Roth 1998a; Doyle and Hayes-Roth 1998b).While the World Wide Web does not offer as sophisticatedor flexible a platform for agent behavior, its ubiquity andobvious amenability to simple forms of annotation make ita natural choice for exploring the annotation approach.

One of our current testbeds is a Web-based art museum.The museum consists of a set of galleries, each of whichcontains rooms filled with artworks. There a re c urrentlytwo galleries: the Nativity Exhibit houses medieval andRenaissance religious art revolving around the birth ofChrist, while the Pre-Raphaelite Exhibit features 19th cen-tury artworks in the style of that movement.

Within these galleries, annotations provide a visitingagent with details about their artworks, including theirhistorical contexts, the lives of their artists, their relation-ships to o ther artworks in the gallery, and d etails aboutspecific characteristics of the art. Any agent familiar withthe a rtwork ontology can explore the site together with auser, lecturing, providing commentary, playing games, andguiding the user according to its own lights, the site de-signer’s, or the user’s preferences.

The Web agent used in the virtual art museum is a char-acter called Merlyn, aptly named after the forgetful oldwizard of T.H. White’s The Once and Future King. Theanimation and speech synthesis systems are provided b ythe Microsoft Agent package. Figure 2 shows Merlyn inthe Briar Rose Room of the Pre-Raphaelite gallery, in theprocess of telling a story to the user. Merlyn is designed toaccompany young children, and the c ontrol i nterface andhis patter reflect this. When he is provided with prewrittentext by the Web site (as is the case with the site tours), heperforms simple syntactic manipulations on the text, suchas pauses, stutters, or interjections, to personalize its flavor.

Figure 2. Merlyn in the Art Museum

The a nnotation framework provides a simple structurefor implementing tours across different parts of the site.For these Web experiments, annotations are written in theExtensible Markup Language (XML). Each Web page inthe site has a c orresponding XML file, which the a gentrequests from the server when the page is loaded. XMLhas the advantage of being simple and highly flexible, andits growing use on the Web gives us hope that we may beable to test our agents on sites designed by others.

Figure 3 shows a fragment of a tour annotation associ-ated with one of the rooms in the museum. Tours presentlyconsist of a linear sequence of stops, at each of which thereis a set of texts available for presentation. Texts are cate-gorized according to user age or interest (e.g., “child,”“adult,” “ historian,” “artist”). Each artwork may havemultiple tours associated with it, though currently no art-work may appear within a single tour more than once.

<TOURS> <TOUR NAME="Sleeping Beauty" TYPE="story"> <TOPIC>the legend of Sleeping Beauty</TOPIC> <TEXT TARGET=”ALL”> Once upon a time, there lived a beautiful princess in a great castle… . . . </TEXT> <STOP> <NAME>Briar Rose Room South Wall</NAME> <URL>gallery/briar/briar4.html</URL> </STOP> <NEXT-STOP> <NAME>Briar Rose Room North Wall</NAME> <URL>gallery/briar/briar2.html</URL> </NEXT-STOP> </TOUR></TOURS>

Figure 3. Fragment of an XML Tour Annotation

Notice that the annotation mechanisms assume no spe-cial character-like qualities of any v isiting agent, so theycan be useful t o Web spiders, search engines, or purelyfunctional agents of other kinds. However, our interest isin embodied, personable tour guides, to which we restrictthis discussion.

Merlyn provides narrative in two ways. First, by his de-scriptions of artworks, artists and entities, which are eitheroffered autonomously or upon u ser r equest. These a reproduced with simple template-driven n atural-languagegeneration that uses the XML data as source. Merlyn has ahigh degree of control over the form of this storytelling.

Secondly, Merlyn produces narrative through the sitetours that he offers. The XML for the tours now containscomplete stories written in English. Merlyn presents thesewith superficial changes, and expands upon them withmore details using the other XML content according to theuser’s interests. Here we are sacrificing some personalityfor the sake of a well-scripted tale; ultimately we wouldlike the tour annotations represented in a deep, rather thansurface, manner, but the complexity of that problem is be-yond the scope of our work.

Currently Merlyn relies solely upon tours embedded inthe environment, though we are in the process of allowingusers to create their own stories. Using the Scrapbook,which records snapshot images and basic facts about everyartwork the user sees, it is possible for the user to b uildnew stories by specifying sequences of artworks and ap-propriate corresponding text. Merlyn will remember thesetales and o ffer them to their author or any new childrenwith whom he interacts. In this way the users enable theagent to become a more entertaining guide just as the envi-ronment does.

ChallengesWhile interactive tour guides offer the possibility of in-creased user attentiveness, retention, and enjoyment(Lester et al. 1997), these benefits require we make theseagents believable to their users, both as living entities andas expert guides. Our research suggests that the followingfour traits are critical for creating believable and compel-ling guides. The first three points have emerged fromDoyle's work on annotation-based tour guide storytelling;the last has emerged from Isbister's investigation of humantour guides lading group tours.

Intelligent reincorporation. Reincorporation in thiscontext means the reintroduction of ideas or entitiesthat have been seen earlier in the tour, with some ref-erence to how they relate to the topic under discussion.In improvisational theater ( Johnstone 1992) it is wellunderstood that reincorporation is a key to building astory satisfying to the a udience; a sequence of unre-lated events does not make a story. Similarly, we sus-pect that a key to creating an effective tour narrative is

reincorporation of earlier material. This requires thatthe agent not only track what material has been seenbut also when connections are either pedagogically ordramatically effective.

Empathy with the content. (Elliott et al. 1997; Elliott1998) has argued that understanding of and reasoningabout narrative is s trongly associated with reasoningabout emotion. If we think of stories as descriptionsof sequences of events that have emotional associa-tions for the listener, then we can enhance the signifi-cance of our tour content to the user by infusing emo-tion into the presentation. Beyond incorporating emo-tions in the presented material, we c an build o uragents to react emotionally to what they present. Thisnot only heightens the significance of these narratives,it also enhances the believability of the tour guide.

Presentation through personality. The most obviousflaw of an intelligent character that relies upon pre-written text is that the text must either be designed forthat particular personality, rendering it dangerous forother personalities to u se, or so d evoid o f characterthat the recitation seems stilted and unbelievable. Ul-timately, one would desire an annotation language andan agent powerful enough that the concepts could beexplained, reinterpreted, and formed into dialog by theagent. Unfortunately, this is s till a hard u nsolvedproblem. Syntactic sugar is a simple a pproximationthat frequently provides good results, as users come toassociate idiosyncratic verbal behaviors with theagent’s personality. Nonetheless, a more intelligentmechanism for integrating content and p resentationwill ultimately be required for widespread use.

Artful timing/delivery. As noted in the section on hu-man tour guides, successful storytelling involvesknowing when to b egin a story, how long it shouldlast, and whether one should elaborate with relatedstories. Making good decisions about t iming and d e-livery requires the a bility to d etect user interest, andreact appropriately to it. Human beings use many sub-tle cues to indicate low or high engagement, and de-tection of user interest is currently very primitive. Thiswill continue to be an important area of research, bothfor development of characters and for development ofsatisfactory and subtle interactions with interfaces, ingeneral.

Future PlansBoth Isbister and Doyle a re interested in incorporatingvisitor reactions and stories into the repertoire of tour guideagents. Isbister is interested in allowing web-site hosts(such as shop owners or temple leaders) to provide storiesthat can be retold by the tour guide agent to visitors during

the tour. Isbister is currently prototyping a web-based inputform, that will allow site hosts to add their own stories forthe agent to retell when their site is visited. She is also ex-perimenting with ways to solicit direct visitor input aboutinterest and engagement with stories, during the tour, with-out interrupting the flow of the experience.

Doyle is expanding the virtual art museum with addi-tional areas in order to more fully explore the problemsassociated with annotated environments as those environ-ments grow in size. Merlyn is being rewritten as part of aneffort to build a unified basic annotation-sensitive believ-able a gent architecture, one that can support a variety ofinterfaces and types of environmental annotations. Hehopes to employ adaptive modeling of the user both toenhance believability and improve the a gent’s effective-ness as a guide.

ConclusionsStorytelling is an important part of the human role of tourguide. By telling stories, they convey content in structuresthat give it meaning. Using reincorporation, emotion andpersonality, and artful t iming, human guides can presentthe entities their users see effectively. We anticipate thatthe lessons we are learning designing interface agents ac-cording to these principles will allow us to p roduce so-phisticated storytelling agents, and will be useful t o anydesigners building characters to interact in social settingsand tasks.

AcknowledgmentsPatrick Doyle’s work was s upported b y NSF Grant IRI-9424261, gifts from Microsoft and Intel, and an IntelFoundation Fellowship.

ReferencesBall, G.; Kurlander, D.; Miller, J.; Pugh, D.; Skelly, A.;Stankosky, A.; Thiel, D.; van Dantzich, M.; and Wax, T.1997. Lifelike computer characters: the Persona Project atMicrosoft Research. In J. Bradshaw, ed., Software Agents.Menlo Park, CA: AAAI Press.

Bates, J. 1992. The nature of characters in interactiveworlds and the Oz project. Technical Report CMU-CS-92-200, School of Computer Science, Carnegie Mellon Uni-versity.

Doyle, P., and Hayes-Roth, B. 1998a. Guided explorationof virtual worlds. In F. Sudweeks, M. McLaughlin and S.Rafaeli, eds., Network and Netplay: Virtual Groups on theInternet. New York: MIT Press.

Doyle, P., and Hayes-Roth, B. 1998b. Agents in anno-tated worlds. In Proceedings of the Second InternationalConference on Autonomous Agents. New York: ACMPress.

Elliott, C.; Lester, J.; and Rickel, J. 1997. Integrating af-fective computing into animated tutoring agents. In Notesof the IJCAI ’97 Workshop on Animated Interface Agents:Making Them Intelligent, 113-121.

Elliott, C.; Brzezinski, J.; Sheth, S.; and Salvatoriello, R.1998. Story-morphing in the Affective Reasoning para-digm: Generating stories automatically for use with “emo-tionally intelligent'' multimedia agents. In Proceedings ofthe Second International Conference on AutonomousAgents. New York: ACM Press.

Hayes-Roth, B.; van Gent, R.; and Huber, D. 1997. Act-ing in character. In R. Trappl and P. Petta, eds., CreatingPersonalities for Synthetic Actors. Berlin: Springer-Verlag.

Johnstone, K. 1992. IMPRO: Improvisation and theTheater. New York: Routledge.

Lester, J,; Converse, S.; Kahler, S.; Barlow, T.; Stone, B.;and Bhogal, R. 1997. The persona effect: affective impactof animated p edagogical agents. In Proceedings of CHI’97. New York: ACM Press.

Lester, J., and Stone, B. 1997. Increasing believability inanimated pedagogical agents. In Proceedings of the FirstInternational Conference on Autonomous Agents, 16-21.New York: ACM Press.

Pond, K. 1993. The Professional Guide: Dynamics ofTour Guiding. New York: Van Nostrand Reinhold.

Rickel, J., and Johnson, W. L. 1997. Integrating peda-gogical capabilities in a virtual environment agent. In Pro-ceedings of the First International Conference on Autono-mous Agents, 30-38. New York: ACM Press.

Rist, T.; Andr , E.; and M ller, J. 1997. Adding animatedpresentation agents to the interface. In Proceedings of the1997 International Conference on Intelligent User Inter-faces, 21-28. New York: ACM Press.

Schank, R. and Abelson, R. 1995. Knowledge and mem-ory: the real story. In R. Wyer, ed., Advances in SocialCognition, vol. VII. Hillsdale, NJ: Lawrence ErlbaumAssociates, pp. 1-86.