Socially Intelligent Agent Research at CARTE

W. Lewis Johnson

Center for Advanced Research in Technology for Education (CARTE)USC / Information Sciences Institute

4676 Admiralty WayMarina del Rey, CA 90292

johnson@isi.eduhttp://www.isi.edu/isd/carte/

AbstractThis paper gives an overview of current research onpedagogical agents at CARTE in the USC / InformationSciences Institute. Pedagogical agents, also nicknamed“guidebots,” interact with learners in order to help keeplearning activities on track. Social intelligence is required inorder to engage in tutorial interactions with the learners,properly interpret the interactions of learners with otheragents, and to model characters that can interact with thelearners in social settings. At the same time, depth ofdomain intelligence is typically required in order for theagents to work effectively as advisors or collaborators.

Introduction

The purpose of the Center for Advanced Research inTechnology for Education (CARTE) at USC / InformationSciences Institute is to develop new technologies thatpromote effective learning and increase learner satisfaction.These technologies are intended to result in interactivelearning materials that support the learning process, andthat complement and enhance existing technologiesrelevant to learning such as the World Wide Web.

Our work draws significant inspiration from humanlearning and teaching. We seek to understand and modelhow people learn from their teachers, from peers, and fromtheir environment, and draw on this knowledge to guide ourdevelopment of educational technologies. We seek a betterunderstanding of the characteristics that make learningexperiences captivating and exciting, and attempt to findways of fostering these characteristics more broadly andsystematically. Our work thus builds upon research inartificial intelligence and cognitive science, and at the sametime draws on the experience of specialists in the arts.

A Major Theme: Guidebots

A broad theme of our work has been the creation of“ guidebots,” or animated virtual guides for learners. We Copyright © 2000, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.

also refer to these guidebots by the more lengthy term of“ animated pedagogical agents” (Johnson et al 2000).Guidebots are animated characters that can interact withlearners in computer-based interactive learningenvironments, in order to stimulate and encourage learning.In their complete implementation, they have a number ofimportant features that are relevant to CARTE’s goals.They interact naturally with learners, generally in a mannerthat is inspired by the behavior of human tutors; theseinteractions include a combination of verbalcommunication and nonverbal gestures. They express boththoughts and emotions; emotional expression is importantin order to portray characteristics of enthusiasm andempathy that are important for human teachers. They areknowledgeable about the subject matter being learned, ofpedagogical strategies, and also have knowledge about howto find and obtain relevant knowledge from availableresources such as the World Wide Web.

Ultimately, we see guidebots as elements of a variety ofrich interactive experiences. In these experiences, studentshave significant freedom to explore learning materials andperform activities that reinforce their learning, eitherindividually or in groups. The guidebots serve a number ofimportant functions in these environments.

• They help keep the learning on track. People have atremendous capacity to learn, but they fail to utilizethis capacity effectively if they fail to take advantageof the resources available to them, or if they fail toapply the right metacognitive skills. Guidebots canremind learners of available resources, and canreinforce good learning habits. They can offer helpand guidance when the students get stuck, and providefeedback to the learners on their progress.

• They provide an additional communicationchannel. Guidebots are a user interface component,and play important roles within an overall educationaluser interface design. They are extremely effective atdirecting the learner’s attention to what is important inthe learning environment. As learners viewinstructional materials, guidebots can provide usefulcommentary on those materials. They can also providelearners with help in navigating complex learningenvironments. They provide both highly salient and

From: AAAI Technical Report FS-00-04. Compilation copyright © 2000, AAAI (www.aaai.org). All rights reserved.

highly nuanced interaction. Human personae havemuch greater expressive capabilities than conventionaluser interface elements, and people are well practicedin reading these expressions.

• They act as the teacher’s representatives. They areable to interact with the learners at times when ahuman teacher is not available. At the same time, theycollect data about their interactions with the learnersthat can be valuable to teachers in assessing thelearners’ skills and planning future computer-basedlearning experiences.

Guidebots play one role within the overall envisionedenvironment described here. There are other importantroles that must be supported.• Supporting Characters. Many learning experiences

include additional characters, such as synthetic teammembers or inhabitants of virtual worlds. These donot act as the learner’s guide, but they neverthelesssupport the learning objectives of the experiencethrough their interactions with the learners.

• The Director. Borrowing the dramatic metaphor, it isimportant to have a director who can guide the overallaction. In some applications a human is in the loop tohelp direct the action, and in other applications wewant the direction to be performed automatically. Thedirector needs to assess whether the current interactionor “ scene” has achieved its intended instructionalfunction, and when it is time to move to another scene.The director may influence the way the guidebots andsupporting characters interact with the learners,depending upon how the action unfolds. If the directordetermines that the current learning objectives havebeen met, it may guide the interaction towardsituations that address new learning objectives. Thedirector thus requires real-time planning andassessment capabilities. In some applications thedirector also needs to control the visual presentation ofthe scene, thus serving as cinematographer.

• The Author. Guidebot technology will becomepractical only if it becomes easy to design and createthe interactive experiences that incorporate guidebots.New kinds of authoring tools are needed that supportthe unique characteristics of guidebot-enhancedlearning experiences, and that take advantage of thelearning materials that are available on the WorldWide Web, in digital libraries, and elsewhere.

Guidebots can be integrated into a variety of interactivemedia, and the resulting interactive experiences can bedelivered in different ways. The following are the mediathat are of interest to CARTE. They currently requiredistinct technologies; however technology advances areblurring the distinctions between them.• Virtual environments. The action takes place within

a virtual world. Learners can manipulate objects in thevirtual world, and through their manipulations canpractice performing tasks. Each learner has a presencein the virtual world, in the sense that guidebots and

other participants can observe them and interact withthem. 3D worlds such as Steve’s immersive mockupsare prime examples. However, 2D interactivesimulations used by Web-based guidebots can alsoserve as virtual environments, insofar as the guidebotcan observe the learner’s manipulation of objects onthe screen and react to them.

• Web-based information spaces. The World WideWeb is a common medium of instruction, andtherefore we design guidebots to operate inconjunction with this medium. In general the Webdoes not support a strong spatial geometry, nor do thelearners have a spatial location within it. 3D spacescan be found on the Web, but they constitute a smallfraction of Web sites at this time. In order to assistWeb-based learning, guidebots receive notificationswhen learners view pages and click on links on thepages, and may respond to those actions.

• Interactive pedagogical dramas. The softwarepresents a dramatic story, using sound and images.Learners can influence what takes place in the drama,and the characters can adapt their behavior in response.If the drama is presented in a virtual environment, thenthe learners can play roles within the story themselves.However, added flexibility can be attained if there issome separation between the learner and the action,i.e., where the computer screen frames the action. Thelearner then shares duties with the automated directorin guiding the action. This structure createsopportunities to present back-story material as needed,and to control the learner’s view via cinematographictechniques. In such scenarios the guidebot typicallyacts as buddy or advisor to the character or charactersthat are being directed by the learner.

Finally, guidebots can serve useful instructional roles inenvironments whose purpose is not primarily instructional.For example, guidebots can be embedded in softwarepackages to provide just-in-time training, and in interactivegames to teach users how to play. Whenever users haveparticular tasks to perform or problems to solve, guidebotscan potentially provide assistance.

A number of CARTE projects are conducted incollaboration with other research groups, in computerscience and in educational application areas. We areworking with USC’s new Institute for CreativeTechnologies to develop more engaging immersive trainingexperiences. Other collaborators include health scienceresearch centers and teaching faculty at USC andelsewhere. These collaborations are essential both foridentifying the deficiencies in current teaching practice thatguidebots can address and assisting in the evaluation ofguidebot applications in instructional settings.

Example Guidebots

CARTE has developed a number of animated pedagogicalagents that have served as vehicles for developing and

testing key aspects of the underlying technologies. Papersdescribing these projects are listed in the references andavailable on the Web at http://www.isi.edu/isd/carte.

SteveSteve (Soar Training Expert for Virtual Environments)assists procedural skill training in immersive virtualenvironments, such as virtual mockups of engine rooms(Rickel and Johnson 1999a). Steve supports both individualand team training; he can advise learners as they performroles within teams, and he can play the role of a missingteam member (Rickel and Johnson 1999b). Steve operateswithin a networked virtual environment, interoperatingwith a collection of other components including the VistaViewer visualization system of Lockheed Martin and theVIVIDS simulation authoring tool (Johnson et al. 1998).

.

Figure 1. Steve pointing at an indicator light

Steve uses the Soar cognitive architecture (Laird et al.1987) to model adaptive task execution in dynamicenvironments. Like other intelligent tutoring systems, hecan monitor students’ actions, point out errors, and answerquestions such as “ What should I do next?” and “ Why?”However, because he has an animated body, and cohabitsthe virtual world with students, he can provide morehuman-like assistance than previous automated tutors could(Rickel & Johnson 2000). For example, he candemonstrate actions, use gaze and gestures to direct thestudent’s attention, and guide the student around the virtualworld. When playing the role of a student’s teammate,Steve’s human body allows students to track his activitiesas they would track those of a human teammate. Finally,Steve’s agent architecture allows him to robustly handle adynamic virtual world, potentially populated with peopleand other agents; he continually monitors the state of thevirtual world, always maintaining a plan for completing hiscurrent task, and revising the plan to handle unexpectedevents.

In addition to Steve’s novel instructional capabilities, hewas designed to simplify the development of new trainingapplications. Unlike many computer-aided instructionprograms, Steve’s instructional interaction with studentsneed not be scripted. Instead, Steve includes a variety of

domain-independent capabilities for interacting withstudents, such as explanation generation and studentmonitoring. Steve can immediately provide instruction ina new domain given only simple knowledge of the virtualworld and a description of the procedural tasks in thatdomain. This approach results in more robust interactionswith students than course designers could easily anticipatein scripts.

Web-Based Guidebots

Figure 2. Adele guiding a student through a clinical case

A number of guidebots for Web-based instruction havebeen developed. They rely upon a common distributedarchitecture and share component modules, whichfacilitates the creation of new guidebot applications. Allfocus on helping students acquire particular types ofproblem solving skills, such as planning or diagnosticreasoning. Students work through interactive exercises,receiving guidance, feedback, and evaluation from theguidebot as needed. Guided exercises are typicallycombined with other kinds of Web-based learning materialsto create on-line courses.

Adele (Agent for Distributed Learning Environments)supports on-line case-based problem solving, particularly inthe health sciences (Shaw et al 1999). Adele monitors thestudent as he or she works through a simulated case,downloaded to the student’s client computer. As thestudent proceeds Adele compares the student’s actionsagainst a model of how the task ought to be performed.The task model is in the form of a set of hierarchical partialorder plans, with different plans applying to differentsituations. Adele may give hints, explain the rationale forrecommended actions, point the learner to relevant on-linereferences, and intervene if the student is making seriousmistakes. The amount of intervention can be changeddepending upon the instructional objectives of the moduleand the needs of the student. During the interaction Adelecurrently responds to the learner using a combination ofverbal and nonverbal gestures. The Adele architecturesupports alternative student monitoring engines fordifferent types of problem solving, and an alternative

engine has been developed to help students with diagnosticreasoning tasks (Ganeshan et al. 2000).

PAT (Pedagogical Agent for Training) is animplementation of Steve’s procedural skill trainingrepresentation compatible with the Adele architecture.PAT incorporates a model of tutorial dialog, enabling it toplace its interactions with the learner in the context of acoherent dialog.

ALI (Automated Laboratory Instructor) applies theguidebot approach to on-line science experiments. ALImonitors students as they conduct experiments onsimulated physical systems. As the student manipulatessimulation variables, ALI interprets the simulation resultsusing a qualitative model of the system, defined usingQualitative Process Theory. ALI can then quiz the studentto see whether he or she interpreted the simulation resultsin the same manner. ALI can also critique the student’sexperimental technique, to make sure that they areexperimenting with the system thoroughly andsystematically.

Gina and Carmen

Figure 3. Carmen in Gina’s office

CARTE is also integrating guidebots into interactivepedagogical dramas. An example of an application usingthis approach is Carmen’s Bright IDEAS, an interactivemultimedia training course designed to help mothers ofpediatric cancer patients develop problem solving skills(Marsella et al. 2000). Learners are presented with amultimedia dramatic story about a mother, Carmen, whofaces a variety of problems at home and at work relating toher daughter’s illness. Gina, a clinical trainer, helpsCarmen to learn how better to address her problems and tryto solve them. In each run through the story, the user candirect the thoughts and moods of either Carmen or Gina,who then behaves in a manner consistent with thosethoughts and moods. The character that the user is notcontrolling acts autonomously in response to the othercharacter. The unfolding of the story line thus is differentfor each run through the story.

Figure 4. Carmen talking about her problems

Current ResearchThe following are some guidebot-related research topicsthat CARTE is currently investigating.

Figure 5. Improved human figure model for Steve

• Increasing realism of appearance and behavior.Currently our guidebots have a distinctly stylized andartificial appearance. In some applications such asCarmen’s Bright IDEAS this is a deliberate stylisticchoice, which can be appropriate when realism is notnecessary to help the guidebots achieve the intendedinstructional objectives. However in immersiveenvironments such as the engine room that Steveinhabits in Figure 1, a realistic appearance is importantboth to increase believability and to give the guidebot agreater ability to demonstrate skills. We are currentlyworking with USC’s new Institute for CreativeTechnologies in order to improve a new version ofSteve with highly realistic appearance and behavior.Steve’s agent “ brain” will be used to control a realistichuman figure, developed by Boston Dynamics,

coupled with a realistic face model developed byHaptek, Inc. The new Steve will be able to modelcomplex behaviors and exhibit believable facialexpressions. This will enable Steve to be employed innew types of training applications, including thosewhere social interaction skills are crucial.

• Speech communication. A number of our guidebotsemploy text to speech synthesis (TTS) in order tocommunicate with learners. Steve furthermoresupports speech recognition. TTS affords flexibilityfor interactive learning applications since the guidebotcan generate utterances dynamically and is not limitedto particular utterances that were recorded beforehand.Unfortunately, limitations in TTS technology limit itsusefulness; guidebot voices can sound monotonous andartificial, and can be difficult to understand at times.We are investigating ways of making more effectiveuse of speech processing technology. We aredeveloping models of how to employ prosodyeffectively in instructional interactions. This involvesobserving interactions between human teachers andstudents, and recording professional actors as theydeliver lines appropriate for instructional interaction.We then are modeling the prosody contours usingavailable TTS packages.

• Synthetic face-to-face interaction. Steve and Adeleboth have the ability to engage in face-to-facecommunication with learners, combining speech andgesture. Face-to-face communication provides themwith both direct and subtle means to communicate withlearners and provide them with feedback. Gaze, headposition, and body position can be used to indicate theguidebots’ focus of attention, and to indicate whenthey are communicating and to whom. They helpmake these guidebots appear more aware of theirenvironment, and hence more believable as intelligentguides. We are now interested in improving thenaturalness of our guidebots’ face-to-face interaction,using nonverbal gestures to support and regulate face-to-face communication. We also see potential forusing body language to depict the mental state ofsupporting characters and encourage empathy on thepart of the learner. This use of body language isexhibited to some extent by Carmen, as illustrated inFigure 4, and is something that we continue toinvestigate. Meanwhile we continue to developmodels of tutorial dialog that can be incorporated intosuch face-to-face interaction (Rickel et al 2000).

• Student monitoring. All of our work hinges on ouragents being able to perform plan recognition in ageneral sense, to be able to understand what strategiesstudents are following as they solve problems. Ourhypothesis is that students can be effectively monitoredacross a range of applications using a small number ofdomain-independent monitoring engines focused onparticular skills, such as diagnostic problem solving orexperimentation. We continue to develop and test this

hypothesis. For example, we are currently workingwith science faculty at California State University atLos Angeles to determine whether the ALIexperimentation model can be applied to new types ofscience experiments. In the process we are testing thelimits of Qualitative Process Theory as a method formodeling physical systems, and identifying new typesof problem solving skills that are relevant toexperimental science.

• Team performance analysis. The area of teamtraining is important, and not well supported byintelligent tutoring systems. We have developed twosystems that exhibit important technologies for teamperformance analysis. One is the PROBES system,which monitors the performance of tank platoons in asimulated armor training exercise (Marsella & Johnson1998). The other is ISAAC, which analyzes teamperformance and identifies factors that contribute toteam success or failure (Raines et al 2000). Both havea role to play in the team analysis capabilities that weenvision. PROBES is effective where a well definedmodel of team behavior and team performanceobjectives exists; this is encoded in the situation spacemodel that PROBES uses to track the learner. ISAACis more appropriate when such a model is lacking, as ithelps to acquire a model from team performance data.We are interested in integrating and developing theseteam analysis capabilities, and applying them in teamtraining exercises utilizing the new realistic version ofSteve.

• Models of emotion. People naturally read emotionsinto animated characters. Furthermore, emotionalresponse is an essential aspect of face-to-face tutorialinteraction. We must therefore design our guidebots sothat they impress the learner as having the rightemotional responses at the right time. We thereforehave been developing dynamic models of emotion, andtesting various combinations of facial expression andbody language in order to determine how best toconvey emotion to learners.

• Models of personality and character. Models ofemotion of are important for believable characters inguidebot-enhanced learning applications, but they arenot sufficient, particularly in interactive pedagogicaldramas. Pedagogical dramas require characters withpersonalities, which help to determine how characterstend to react cognitively and emotionally to differentsituations. Personality characteristics also caninfluence how characters express their emotions. Wealso require models of character development, so thatthe characters’ personality attributes can change inappropriate ways over time. We are investigatingthese issues in the context of a new pedagogical dramaproject called IMPACT, intended to foster good eatingand exercise habits in 4th and 5th graders. The IMPACTgame has several main characters, each of which has

different personality characteristics that change overtime.

• Automated direction. Carmen’s Bright IDEAS supportsnonlinear story interaction, by determining what takesplace within a scene and when scene transitions takeplace. It also provides automated cinematography,character direction, and shared directorial control. Weare now generalizing these mechanisms so that theycan be applied to a range of interactive pedagogicaldramas.

• Authoring technology. Current authoring toolresearch is exemplified by Diligent, a systemdeveloped by Richard Angros that learns proceduresfrom a combination of observing expertdemonstrations and performing experiments (Angros2000). Diligent demonstrates that interactive authoringtools can effectively exploit machine learningtechniques. Advanced authoring techniques such asthese are important in order to ensure the broadapplicability of guidebot technology; conventionalauthoring tools designed for scripted instruction is notappropriate for guidebot design. We are engaged in anumber of authoring research and developmentactivities. Andrew Scholer is building upon Angros’smodel to support authoring via interactive tutorialdialog – tutoring the agent so that the agent can thentutor students (Scholer et al. 2000a & b). A graphicalinterface to Adele’s plan representation has beendeveloped so that domain specialists can modifyAdele’s task knowledge. Finally, we are developingmethods for reusing elements of Adele’s cases, so thatnew case-based exercises can be constructed rapidly.

Acknowledgments

CARTE includes the following faculty and staff members:Jeff Rickel, Stacy Marsella, Richard Whitney, Erin Shaw,Kate LaBore, Erin Shaw, Frederica Stassi, and MarcusThiebaux, and Fanny Mak. Students and interns includeRogelio Adobbati, Shish Aikat, Paul Cho, Teresa Dey,Aaron D’Souza, Daniel Mellet D’Huart, Carlos Fernandez,Stephen Kim, Girish Narayanan, Brett Rutland, NarayananSadagopan, Girish Srinivasan, and Daisy Zhang.

The work described in this paper was conducted withsupport from the Office of Naval Research, Air ForceResearch Laboratory, US Army STRICOM, NationalCancer Institute, US Department of Agriculture, NationalInstitute of Child Health and Human Development, theNational Science Foundation, and internal grants from theUniversity of Southern California and USC / InformationSciences Institute.

References

Angros, R. (2000). An Agent that Learns to Instruct:Exploiting Demonstrations through Experimentation.Ph.D. Thesis, University of Southern California.

Ganeshan, R., Johnson, W.L., Shaw, E., & Wood, B.(2000). Tutoring diagnostic problem solving. Proceedingsof ITS 2000, Springer-Verlag, Berlin.

Johnson, W.L., Rickel, J.W., & Lester, J.C. (2000).Animated Pedagogical Agents: Face-to-Face Interaction inInteractive Learning Environments. To appear inInternational Journal of Artificial Intelligence in Education11, 47-78.

Johnson, W.L., Rickel, J., Stiles, R. and Munro, A.(1998). Integrating pedagogical agents into virtualenvironments. Presence: Teleoperators and VirtualEnvironments, 7(6):523—546.

Laird, J.E., Newell, A., and Rosenbloom, P. (1987).Soar: An architecture for general intelligence. ArtificialIntelligence 33:1-64.

Marsella, S., & Johnson, W.L. (2000). An intelligentassistant for team training in dynamic multi-agent virtualworlds. In Proceedings of ITS ‘98, 464-473, Springer-Verlag, Berlin.

Marsella, S., Johnson, W.L., and LaBore, C. (2000).Interactive Pedagogical Drama. In Proc of the 4th Intl.Conf. on Autonomous Agents, Barcelona, Spain. ACMPress, New York.

Raines, T., Tambe, M., & Marsella, S. (2000).Automated assistants to aid humans in understanding teambehaviors. Proceedings of Agents 2000, 419-426, ACMPress.

Rickel, J., Ganeshan, R., Rich, C., Sidner, C.L., & Lesh,N. (2000). Task-oriented tutorial dialogue: Issues andagents. Working Notes of the AAAI Fall Symposium onBuilding Dialogue Systems for Tutorial Applications,November 2000.

Rickel, J., and Johnson, W. L. (1999a). Animated agentsfor procedural training in virtual reality: Perception,cognition, and motor control. Applied Artificial Intelligence13:343-382.

Rickel, J. and Johnson, W.L. (1999b). Virtual humansfor team training in virtual reality. In Proceedings of theNinth World Conference on Artificial Intelligence inEducation, pages 578-585. IOS Press, 1999.

Rickel, J. and Johnson, W.L. (2000). Task-orientedcollaboration with embodied agents in virtual worlds. In J.Cassell, J. Sullivan, S. Prevost, and E.Churchill (Eds.),Embodied Conversational Agents, MIT Press.

Scholer, A., Rickel, J., Angros, R. Jr., & Johnson, W.L.(2000a). Acquiring and using task knowledge for trainingin virtual reality. IUI-2000 Workshop on Using Plans inIntelligent User Interfaces.

Scholer, A., Angros, R. Jr., Rickel, J., and Johnson, W.L.(2000b). Teaching animated agents in virtual worlds. InWorking Notes of AAAI Spring Symposium on SmartGraphics.

Shaw, E.; Johnson, W. L.; and Ganeshan, R. (1999).Pedagogical agents on the web. In Proceedings of the ThirdInternational Conference on Autonomous Agents, 283-290.ACM Press, New York.