personification in computer mediated communication

Personification: Crossover between… 1

RUNNING HEAD: Personification in Computer Mediated Communication

Personification: Crossover between Metaphor and Fictional Character in Computer Mediated Communication

Johan F. HoornFree University, Amsterdam, The Netherlands

Author’s address:Vrije Universiteit, Faculty of Sciences, Department of Computer Science, Section Information Management &

Software Engineering, Subsection Human Computer Interaction, Multimedia & Culture

[email protected]

Word count (body of text): 9,903Word count (including everything): 11,851


Abstract

This contribution defines the psychological processing of the artistic, cultural, and communication aspectsof agents in information interfaces and characters in media entertainment (i.e. video games) as a new research topicin communication studies and computer science. It offers a theory of what personifications such as (intelligent)adaptive navigation assistants for Web environments can achieve, cognitively and emotionally: Personifications canbe user-friendly, be motivating, and suggest ideas but only if their behavior is task-relevant. The personificationshould not only be more-or-less pertinent to the interface goals or story line but also be adaptive to changes in tasks,personal and business goals, and work processes as these change over time. This paper offers a multiple-meaningstheory of metaphoric communication (i.e. personifications) and a general communication model for human-agentinteractions. Although a theoretical exploration, it offers methods for empirical verification, blending technical toolsfrom EEG-measurement labs, reaction-time recordings, and structured questionnaires.


Personification: Crossover between Metaphor and Fictional Character in Computer Mediated CommunicationCommunication between humans and computers sometimes seems a battle zone of tragic heroes (us) against

invincible mechwarriors (them). Whenever emotion and intelligence meet technical rigidity, a communicationvacuum emerges in which the end-user becomes the ‘end-looser,’ suffocating in the absence of human-centeredfeedback. Designers of computer systems and communication interfaces develop embedded agents as the utmostuser-friendly software application. For example, agents supposedly help performing tasks, help achieving (business)goals, and facilitate work processes. One of the most important functions of agents is to emulate face-to-facemultimodal communication between the system and the user. Personified multimodal communication would bericher and easier to comprehend than unimodal communication such as text. Personified interfaces can provide back-channel feedback to users in a more natural and less obtrusive manner than interface widgets such as dialog boxes(cf. Mayer, 1997; 2001; Mayer, Heiser, & Lonn, 2001; Mayer, & Moreno, 1998). Although it is widely admitted thatintegrating human interpretation is paramount in interface design, yet agent theory mainly “… is concerned with thequestion of what an agent is, and the use of mathematical formalisms for representing and reasoning about theproperties of agents.” (Wooldridge & Jennings, 1995). Although that may be a bit of an overstatement (see Paiva,Machado, & Prada, 2001; Machado, Paiva, & Prada, 2001), there is a need for sophisticated theory that values the“artistic nature of the problem.” (Reilly, 1996). Reilly states that “one of the key steps in creating quality interactivedrama is the ability to create quality interactive characters (or believable agents). Two important aspects of suchcharacters will be that they appear emotional and that they can engage in social interactions.”

As much as designers look into the technical requirements of agents (software architecture, rendering), theyare less focused on the user requirements, i.e. the prerequisites of human communication. Inspection of the WorldWide Web, for example, shows us Eliza-style chatterbots that rely on pattern-matching techniques to simulateintelligent conversational behavior. However, it is much more interesting to understand the agent’s impact on theuser’s cognition. If this is well understood, then it remains an engineering challenge how to program themaccordingly. The question is whether agents as compared to dialog are the ultimate solution to all possibleapplications for which one might use computers. Probably not (cf. Craig, Gholson, & Driscoll, 2002). Yet, agents areextremely interesting communication devices. In occupying the crossroads of art, culture, psychology,communication, and technique, they have the potential to become many-sided, and therefore, mature communicationpartners. It is worthwhile investigating for which applications they are suited. To get there, we need a revision of thecomputer system-centered approach. We need to treat intelligent assistants as cultural products (i.e. as fictionalcharacters that sometimes crossover with metaphors), subjected to psychological processes of perception and(emotional) experience. Certain agents serve as metaphors for abstract ideas of, for instance, help and guidance, andtherefore, they are personifications. If we can establish a general theoretical framework for communication throughagents, empirical tests may follow that can decide on the specific requirements for particular application domains.

The present paper takes up the first part by developing a new theory of human-agent communication. Itwants to follow up on the work of Laurel (1991, p. 356), who states that:

An interface agent can be defined as a character, enacted by the computer, who acts on behalf of the user ina virtual (computer-based) environment. Interface agents draw their strength from the naturalness of theliving-organism metaphor in terms of both cognitive accessibility and communication style (p. 356).To explain and improve the supposed strength of agent communication, integration should take place

between theory of processing fictional characters and theory of processing metaphors. This kind of theory is urgentand still wanting. It is becoming very relevant to software developers. Agents, avatars, bots, virtual and electroniccreatures and icons inhabit the new generation of software in video games, active worlds and search engines on theWorld Wide Web. Our screens – and our lives (vide the rage for “tamagotchis”) – are becoming peopled by theseelectronic pets and guides. It is important to consider how they are perceived and experienced because that is theway developers can make them both attractive and user-friendly. Content providers such as publishers, media broad-casters, the entertainment industry, and cultural institutions are now preparing for distributing content online. Awealth of choices awaits the user. Any means that facilitates selection, classification, inspection, and manipulation ofthe huge supply and the retrieval and actual use of items will be welcome. With the increasing exposure totelevision, film, and digital media, emotional education and competence is acquired more and more via fictionalcharacters (Buck, 1999). In the future, this topic will become increasingly important with the growth of, for example,multimedia performances, education through virtual reality, and ‘meeting’ people on the Internet (De Kerckhove1991; 1995; 1997). Museums, cultural archives, publishing houses, and broadcasting companies employ multimediadevices (CD-ROM, Internet, and the World Wide Web) to bring culture to a larger audience. Therefore, it isnecessary to investigate how graphic design and visualization techniques can improve the transfer of culturalinformation by multimedia. The issue is also pertinent to child psychologists and to media critics. It might even


throw much needed new blood in the world of literary criticism as its theory takes some of its cues from the ancienttradition of biblical exegesis: Indeed personification is the more accessible term for classical antiquity’s “allegory.”

To what end and under what conditions can agents be put to use in communication interfaces (cf.Rickenberg & Reeves, 2000)? Are personifications appreciated differently from literal equivalents, other metaphors,or other fictional characters? Regarding cognition and emotion, is a personification differently processed from othertypes of metaphors and differently from other types of fictional characters? Is communication through personifica-tion more effective than via literal instruction, other metaphors, or other fictional characters? The theoretical aim ofthis article is to arrive at an interdisciplinary integration of literary theories on metaphor and fictional characters tohelp research into human-agent interaction with a communication model in which the cognitive psychology of usersis described while processing personifications. Theory development should be facilitated by tests on the integratedmodel with various types of stimuli and real users. The results may support the practical aim, which is to promotebetter understanding of the relation between system usability and acceptance and to develop guidelines, models, andmethods by which designers may be able to provide better human-oriented communication of computer systems.

Agents as Fictional CharactersConsider a merely voice-based information service that is accessed via a telephone line. There, the user may

well believe that s/he is talking to a real or virtual person. This situation resembles the ambiguity of perceivingmediated people such as reporters and journalists (see Konijn and Hoorn, 2003, this conference). If the user is in theillusion of talking to a real person, the psychology of interpersonal communication applies. This situation falls out ofthe scope of the present paper. However, if the user/receiver is somehow aware that this conversation is with amachine, then the agent/sender is much the same like a character in a realistic theater piece or motion picture. Thecomputer system offers a representation of a real conversation like the mediated journalist who is merely therepresentation of the real person. Therefore, the voice agent can be considered a character that is representedrealistically but is yet fictional. This example stands for various types of agents such as “embodied conversationalagents,” “life-like characters,” and “virtual humans.” They share the common approach in computer science tocapitalize on (hyper)realistic rendering with neglect of other factors that evidently contribute to the receiver/user’sinvolvement and appreciation (Konijn and Hoorn, 2003; also Hoorn, Konijn, & Van der Veer, 2002).

Figure 1. Herman the Bug, the animated pedagogical agent, teaches on plant growth.

Animation agents like Herman the Bug (Elliot, Rickel, & Lester, 1997; Rickel & Johnson, 1998; Figure 1)are typical for fictional characters that are rendered unrealistically. This is not necessarily negative because fantasyand magical features may stimulate the imagination and increase fun, especially with children. This type of characteris easier recognized as fictional but the theory to be presented in this paper pretends to account for all.

Not only will certain agents be recognized as fictional, in many cases they will be relied on as teachers,guides, darlings, and servants. The roles agents play reflect the abstract function they have for performing the user’stasks. As soon as this role or abstract function is recognized, the agent not only is a fictional character but canbecome a metaphor (e.g., a harbor pilot representing a software navigation facility). Agents that are fictionalcharacters as well as metaphors for an abstraction correspond to the (artistic) device of personification (see below).Thus, a theory of communicating with agents should allow for realistic and unrealistic representation and integrate itwith the metaphoric aspects of (certain) fictional characters. In conclusion, the theory of fictional characters should


be generic for all types of user-interface agents, animated agents, embodied characters, helper agents, tutor agents(e.g., Anderson, Corbett, Koedinger, & Pelletier, 1995), virtual product presenters (Figure 7), and automated newsreaders (Apple, 1990) as long as the receiver/user does not qualify them as real people (cf. Reeves & Nass, 1996).The theory of personification is valid for those agents that moreover have a metaphoric twist.

Interaction ContextAn approach most suited for describing the interaction context of a user (group) in communication with a

computer system is Groupware Task Analysis or GTA (Van der Veer & Van Welie, 2000). Figure 2 depicts anontology of the task world of a user. In GTA, agents can be users as well as software components (also other thanthe puppets on our screens). In our case, however, the second meaning of agent does coincide with the puppets onour screens. Users and agents perform tasks like searching, navigating, and exchanging information. They useobjects such as databases and hardware devices to perform those tasks. The roles agent and user play often is ofsender and receiver, of supply and demand, of helper and helped, respectively. However, these roles may switchduring the process when for instance the agent needs more information or the user interferes with an agent’s task.This is called interaction. All this action and interaction serves to an end, which is the goal of the user, for instance,to find information, to be entertained, to reach a business goal (cf. Marketing Information Systems), or to streamlinea work process (e.g., tips and suggestions by Microsoft’s Clippit).

Figure 2. The task world of the user (Van der Veer & Van Welie, 2000).

External and internal events may induce the need for new tasks, for setting new goals, and inventing newwork processes. By pressure from the market, for instance, a digital library can develop into an e-commercecompany. The change in business model from an educational to a commercial company has an impact on thebusiness goals that are set and the business processes that should help achieving those goals. Offering high qualitycontent and keeping an exhaustive stock may be of interest to an educational institution but may be cost-intensive inthe eyes of a commercial service. This changes the user’s requirements of a computer system or software application,in our case, the software agents. A library would want an agent to immediately point out new works that might be ofpersonal interest to the user or improve the user’s comprehension of a scientific problem. Commercial industries,however, would hope that an animated agent attracts more traffic on their pages because it is fun to interact with asomehow life-like entity. They hope for ease of communication because the agents to a certain extent emulatehuman-human communication.

Yet, Dehn and Van Mulken (2000) provide a survey on empirical work in the area and found that ananimated agent does not necessary improve a user’s comprehension or recall of information. The added value hasmore to do with motivational aspects, e.g., learners maybe willing to spent more time with the learning applicationwhen an agent is present, or may feel less anxious in a comprehension test conducted by a synthetic tutor. Whateverthe effect of an agent may be, the central issue is that the goals of the user are satisfied by the agent and that the work

Software

User


process towards those goals is facilitated. That is, the agent should be relevant to the task at hand (be it informationor entertainment seeking, communication, or composition) and should support rather than impede the work process(raising positive instead of negative valence) (cf. Frijda, 1986, p. 494, p. 463, p. 455). Commercial systems such asMicrosoft’s BOB and Clippit, kiosks such as the Postal Buddy and the anthropomorphic bank teller machines, or therecent Ananova news reader have failed because they either gave irrelevant information or slowed down computerprocessing and therefore, transaction time.

Having said this, what is considered relevant and what has positive valence depends on different classes ofusers (old vs. young), usage in different tasks, and different user personalities (e.g., internal vs. external locus).However, these are empirical questions that cannot be answered if not an overall theory of perceiving andexperiencing agents is devised first.

Agents as Cultural ProductsAgents should be conceived of as cultural products. For example, Islamic users may not appreciate an

animation agent because of religious restrictions on depicting humans (Marcus, 1993). Agents are liable toconventions of how mediated persons and fictional characters should behave. They follow cultural scripts (thebutler) and coded behavior such as politeness. In this way, agents tune in to the user’s tendency to treat computers as

social entities, maintaining gender stereotypes and social rules and rites (Reeves &Nass, 1996). Agents are often based on existing literary or historic models.Arthurian sagas model the world of fantasy games and inspired the Merlin character(http://msdn.microsoft.com/workshop/imedia/agent/). Genie has his roots in Saladin.Agents parody cultural key figures (Einstein, Shakespeare, Figure 3). Role-playinggames often are indebted to literature (e.g., American McGee’s Alice, Figure 5).Comics are translated into films, which are translated into games (e.g., X-Men) andvice versa.

Communication interface designers often look at (artistic) film to improve the story telling. For example,FitzGerald, Buie, & Cuales (1998) wonder whether “the grammar of visual storytelling presented in film offer astandard that we can borrow.” FitzGerald et al. (1998) refer to, for instance, Boorstin (1995) and Katz (1991) tomake their claim that ‘video offers nuance of emotion and that no other media can carry this sort of information sovividly.’

Agents can be anchored in conventional metaphors such as ‘understanding is light’ (The Genius carries alight bulb, Figure 3) and ‘computing is doing magic.’ Duncker (2002) argues that Maori people have difficultyaccessing information, because the Western library metaphor misfits their tribal knowledge repositories. Also Preeceet al. (1994, pp. 141-149) acknowledge the role of metaphor in intuitively understanding interfaces (e.g., “thewindows metaphor”).

Figure 3. Office Assistants follow the traditional story line of the butler: Polite greetings, asking how to be ofservice, a modest tip or suggestion, and humble retreat after assistance.

In the remainder, I sketch a model of agents with a clear embodiment (Dix, Finlay, Abowd, & Beale, 1998,p. 157) as cultural products that are emotionally and socially perceived and experienced by human observers quitethe same as in theater, books, film, and other media. The aspect of culture is most strongly denoted by the factornorm (see section Personifications are Fictional Characters). There the social and traditional values of an observer(group norm) are compared with the subjective norm while engaging in agent communication. The approach I will


advance goes back so far in the grounding of historical consciousness (e.g., gods personifying human qualities) butalso brings to bear the scientific rigor of the neurosciences, based on empirical data and leading to verifiableapplications.

Theory of PersonificationWhen in Act IV of The Winter’s Tale (Shakespeare, 1610-11) (Appendix), Time declares to “please some,

try all” and to spread “both joy and terror of good and bad,” he suggests that he may bring happiness and sorrow inview of a positive or negative prospect but ultimately leaves no one untouched. Time does not only do so because heis the ruler of our futures but also because he is a personification. A personification is a fictional character ornarrative agent who is the embodiment (objectification, substantialization) of conceptual entities such as classes,ideas, faculties of the mind, thoughts and feelings, speech acts, values, or fields of knowledge (philosophy, religion)(Paxson, 1994). Rhetorical tradition understands personification loosely as the giving of a human face or identity tosomething not human, usually abstract, such as Truth, Folly or Time (Paxson 1994, p. 1, also Figure 4).

Figure 4. Pierre Mignard (1694). Time Clipping Cupid’s Wings. Oil on canvas, France.

Navigating the World Wide Web, the unsuspecting surfer on the electronic highway may meet the virtualwizard Merlin, an animation agent who controls the confusing world of cyberspace with incomprehensible tricks.Such fictional characters are metaphors for security, knowledge, and guidance, abstract concepts that have thefeatures of living creatures in a more-or-less realistic way. Therefore, they are personifications who supposedlymake abstraction come within cognitive reach for the computer user and may even call out affective relationshipswith what actually is a complex of electronic switches. Does personification really help to better understand the useof a computer system (cf. FitzGerald et al.)? Does personification really have the cognitive significance of the“master trope” as Paxson (1994, chaps. 1-2) supposes? When Merlin stirs his caldron to express ‘processing,’ looksinto a crystal ball to indicate ‘searching,’ and when in the process of explaining concepts on plant growth, Hermanthe Bug performs a host of activities such as walking, flying, shrinking, expanding, swimming, fishing, bungeejumping, teleporting, and acrobatics (Lester et al., 1997), does this improve information transfer better than othermeans of communication (Shneiderman, 1998, pp. 380-384)? Or does personification merely teases out moreappreciation ‘because it is fun’ without stimulating communication? Van Driel, Hoorn, Oosterveld, & Vorst (1998)indicate that appreciation for a cultural product is highest when the ratio, emotion, and fantasy of observers arestimulated in equal proportions. It should be found out whether personification covers these three domains. Instimulating the fantasy more than a literal statement, personification seems to acquire less automatic cognitiveelaboration. It does so, while gaining more affective satisfaction than a ‘plain’ metaphor, because it is a fictionalcharacter. Although it seems likely that personification is a strong candidate for evoking high appreciation, it isquestionable whether high appreciation is necessary and sufficient for optimal communication (Shneiderman, 1998,pp. 380-384).

Personification is a fictional character that is used as a metaphor for an abstraction. In assembling a theoryon personification for computer mediated communication, models of human-agent interaction should synthesize theprocessing of fictional characters with the processing of metaphors. At face value, it seems that the metaphoricaspect is additional to the more comprehensive process of perceiving and experiencing fictional characters. Thissuggests that firstly, the personification is processed as fictional character, and while going through the epistemicmodule (see below), secondly as metaphor for an abstraction.


There are many variants of particularly graphical embodiments with simplistic 2D faces at the one end, allkinds of video-based and more or less photo realistic 2D or 3D graphical representations in between, andentertainment robots with physical anthropomorphic embodiments on the other end. Empirical works by, forinstance, McBreen, Shade, Jack, & Wyard (2000) suggest that form and style of embodiment have an impact on userappreciation of agents. Next, I discuss the factors of form (aesthetics), style (realistic or not), and affect (relevance-valence) that are pertinent to involvement with and appreciation for fictional characters followed by a discussion onmetaphor processing.Personifications are Fictional Characters

Although the user may feel interacting with a real person, in fact all embodied agents are fictionalcharacters. Against Reeves and Nass’s (1996) position of the computer as a real person, one may argue whetherusers actually think there is another person or just apply the same social and communication rules to the computerbecause the computer roughly fakes human interaction? Sometimes people talk to their pen or pat their car on thedashboard as a form of self expression but that does not mean they mix up fiction with reality (Hoorn, Konijn, &Van der Veer, 2002). Although FitzGerald et al. avow that “… creating audience empathy with the agent throughmanifestations of personality can create motivation in the student learner that would not otherwise exist,” mostagents do not really emotionally absorb the user but remain quite empty puppets (Bonito, Burgoon, & Bengtsson,1999). Much of what an agent should offer depends on the task at hand. I will start describing a theory of engagingagents for entertainment, evolve through education, information, and I will end with work process efficiency.

In the best tradition of role-playing games, characters for entertainment are multi-sided, have contradictoryfeatures, and cope with exciting situations. If agents should entertain and motivate, they perhaps should be designedas characters of a computer game. Would not it be exciting to discover that our animated pedagogical agent is adark-haired, gothic girl of about 18 years old, with a bloody apron covered with symbols and tied together with askull on her back (Alice, American McGee, Figure 5)? To date, agents are boring, slick good guys serving asobedient slaves. Much of the distance they evoke, comes from their stupidity in executing tasks. Konijn and Hoorn(2003), however, show that characters that should entertain raise distance that is based, for instance, on ethicalbadness. This distance, however, is evaluated positively and contributes to the overall appreciation for the character.

Figure 5. Alice, but not in Wonderland (left panel). Konoko, brutal force and youthful innocence (right panel)(http://www.3dactionplanet.com/oni/images/konoko_rogue.jpg). Two models for more motivating types of agents?

People like role-playing games because they want to escape from daily life and identify or have empathywith an incredible hero. If agents should have an entertaining side, we perhaps might want them to be empathetic.However, notions such as empathy and identification have many meanings. If we really completely identify with thecharacter, then the user suffers as much pain as the blood and gore that Hana TsuVachel experiences in Fear Effect2: Retro Helix (Kronos). Game after game, an empathetic player would experience the real terror of detectiveEdward Carnby, who finds his friend Charles Fiske being dead in Alone in the Dark 4: The New Nightmare(DarkWorks). In their theory on Perceiving and Experiencing Fictional Characters (PEFiC), hence, Konijn andHoorn (2003) rather speak of a measure of involvement with the character, which co-occurs with a measure ofdistance; a healthy distance, which protects the human emotional system against too threatening but also toodesirable situations and characters. Konijn and Hoorn provide evidence on film characters that involvement anddistance are not two mutually excluding experiences. On the contrary, high appreciation for a fictional character


flows from a ratio between involvement and distance. Such apparent contradictions often are found in game reviewsas well:

Konoko, the girly heroin of the action game Oni has everything to become a new icon in the game world.Not like Lara Croft because of her spectacular looks but exactly through the combination of youthfulinnocence and the brute violence she is capable off. That developer Bungie did not choose for the obviousway to make her super-sexy only increases the sympathy for this girl. (Bartelson, 2001) (author translationand italics)Figure 6 summarizes the PEFiC theory in explicating how involvement and distance transpire and how their

interrelationship determines the appreciation for the character. Relying on cognitive memory theories (Raaijmakers& Shiffrin, 1981) and similarity studies (e.g., Tversky, 1977), Konijn and Hoorn claim that not only recognition ofthe self in the fictional character but also the memory of one’s own vicissitudes, or situations one has seen or heardoff, bring the observer closer to the character. The measure of (inner) resemblance between observer and character,including their respective situations, partly mediates the measure of involvement. By the same token do thedifferences partly mediate distance. Partly, because it is not as simple as counting the similarities and differences. Inthe light of their personal goals and concerns, observers determine how relevant particular events concerning andfeatures of the fictional character are. That Hana is bisexual is probably more important to people with the samepreference than to others. That Masami von Weitsaecker (Ring of Red, KCE Studios) is from Japanese-Germandescent leaves everybody indifferent, unless you underwent the attack on Pearl Harbor. Subsequently, the features ofand the events concerning the fictional character are appraised for their ethics (a good hero or a bad villain) (Paiva etal., 2001), their aesthetics (is she a beauty or an ugly witch), and their epistemic value (how realistic or unrealisticare they?) (Shapiro & McDonald, 1992; Paiva et al.). Like mediated persons, voice-based information agents remainin the area of realistic representations. Programmable plush dolls and stuffed animals that control syntheticcharacters in animated environments (Umaschi, 1996; Johnson, Wilson, Blumberg, Kline, & Bobick, 1999) recede atthe unrealistic end of the scale. It is easy to imagine that the tennis star Anna Kournikova evokes involvementbecause of her attractive looks, but when she is the cover-up of a virus (subject: Here you have ;-o) in an e-mailattachment (Anna-Kournikova.jpg.vbs), she yields quite as much distance. Likewise, a character like Black MagicianVivi (Final Fantasy IX, SquareSoft) may or may not be appreciated for her highly imaginative or otherwiseunnatural peculiarities. But that’s not all there is to it. Nobody lives in isolation and no one is so self-willed that s/heis immune to the opinion of others. Social psychology (e.g., Terry and Hogg, 1996) teaches that people like tobelong to particular groups (e.g., Black versus White Hat hackers) and adhere to the (cultural) norms of those groupseven when they do not agree on them all that much (Grey Hats). The same is valid, of course, for Internet surfers,game addicts, UNIX gurus, film freaks, art lovers, and other connoisseurs. In the PlayStation camp (Sony), it is notallowed to positively value the Pokémon characters (Nintendo). As a child psychologist, you must cry out against theviolence of shooter games and yet buy your kid Point Blank 3 (Namco) for Xmas. Thus, the socially acceptedjudgment over a character (“Dan & Don are a bunch of aggressive yahoo’s”) may contest with what one silentlythinks (“Dan & Don are my quirky heroes”). Normative behavior of agents (Machado et al., 2001) and socialcompetence (Prendinger & Ishizuka, 2001), then, should be part of an agent’s design.

All those dimensions of appraisal can lead to a tendency to approach a fictional character (“youthfulinnocence” hints at a friend) and a tendency to avoid (“brute violence” hints at a foe). This simultaneous occurrenceof a positive and negative ‘valence’ (that is, ‘ambivalence’) suggests that self-recognition without more does notnecessarily awaken involvement with the character. The lesson learned from this for avatars is that mere resemblanceis insufficient for user involvement. If someone recognizes him or herself in the revenge of Edward Carnby, theresemblance may lead to involvement but that the trait is negatively appraised leads to that extraordinarycontradictory experience of distance. With the PEFiC-model, it can now be explained why, for example, the wish ‘tobe as cool as Lara Croft’ stirs up mixed emotions, which all too often may confuse young adolescents. The fact thatyou are not as cool (the differences) contributes to the experience of distance, whereas the positive appraisal of thosedifferences leads to involvement.

Leaving the traditional identification and empathy approaches, then, the PEFiC-model gives an integrativeaccount of how appreciation for fictional characters in literature, film, TV, and other media is a trade-off between theparallel processes of involvement and distance. Given the relevance and valence (positive-negative) of a situation,involvement and distance supposedly flow from ethic (good-bad), aesthetic (beautiful-ugly), and epistemic (realistic-unrealistic) appraisals of the features of the fictional character, thereby considering the (dis)similarity between thefictional character and the observer (Figure 6). Konijn and Hoorn (2002) have strong evidence on film charactersthat the factors in the PEFiC-model are all necessary to explain appreciation for a character but how those factorsinteract is harder to determine.


Norm

Epistemics

Aesthetics

Ethicsgood

beautiful

realistic

bad

ugly

unrealistic

Involvement

Distance

Appreciation

dissimilar

irrelevant

negative valence

similar

relevant

positive valence

%

%

ENCODE COMPARE RESPONDFe

ature

s of s

ituati

onan

d Fict

ional

Char

acter

Figure 6. Model for perceiving and experiencing fictional characters, including agents. Features that are relevant forthe goals and concerns of the observer evoke emotions. Distinctive features that are appraised positively andintersecting features that are appraised negatively increase both involvement and distance (see %-sign). If badfeatures are appraised positively, an approach tendency may occur, whereas a negative appraisal of good featuresmay cause an avoidance tendency. The entire process is doubled if the norms of the individual are contrasted withthe norms of the socio-cultural group (“What do the others think of it?”).

PEFiC can be considered an overall model for character engagement in virtual or fictional environments. Inthe case of agents, it is the application that matters and makes the difference. Involvement with the character may bethe goal in a game more than in a training environment such as a driving simulator, where keeping your cool(distance) is more important. In tele-presence involvement probably is more important than in remote-controlscenarios such as long-distance surgery, where a physician should keep his/her mind together. Yet, PEFiC canaccount for these different constellations of involvement-distance trade-offs. The main shifts probably can be tracedback in the change of goals for using the application, which will narrow down the relevance of the agent.

The relevance of characters in gaming probably relates to win or loose without many real-life consequences(except when there is a bet or social reward involved). Appreciation is probably induced by activity andentertainment. Agents in learning applications will have more dimensions of assessment. Yes, they should beentertaining to motivate the student but its core relevance lies in clear explanation of principles and mechanisms.Here the performance of the agent does have real-life consequences (e.g., pass or fail exams). Appreciation probablyrelates to entertainment and easy access to information. In typical scenarios with information agents (e.g., voice mailnavigators), the user may perceive the agent as being a different person. However, as soon as the user realizes to talkto a machine the relevance of the agent is restricted to the task of providing information about (acoustic) interfaceusage. Here agent performance affects real life as well. Missing an important message due to badly designednavigation may mean the difference between getting a job or not. Equating character involvement in film and


literature with character involvement in interactive computer-based applications depends upon whether the user’sactivities with the computer naturally entail interacting with or observing the character. For example, it may be thatmany users of Microsoft Office might not have much appreciation for Office Assistant regardless of its“vicissitudes,” simply because they do not need to interact with the Assistant to do what they want, and the Assistantis simply a distraction. In that case, the user performs the task better than the Assistant or the Assistant merely offersirrelevant tips and suggestions. The task-relevance of the Assistant, then, is extremely low. The purpose of theAssistant to facilitate the work process in Office turns into its opposite, slowing down PC processing time andboosting distance at the cost of involvement.Personifications are Metaphors

All embodied agents are fictional characters but not all embodied agents are personifications. Broadlyspeaking, a metaphor is a non-literal means of communication in which a semantic unit is compared to another unitof a different semantic class but with the preservation of meaning. ‘Time is duration’ is not a metaphor because bothunits belong to the same semantic class. ‘Time is a man’ is a metaphor because the units do not belong to the samesemantic class but the expression remains interpretable. In Figure 7, the tutor agents are realistically rendered(http://www.ics.uci.edu/~kobsa/courses/ICS104/course-notes/metaphors.htm). They are literal or perhapstautological representations (‘the teacher is a tutor’). The virtual product presenter also is realistically rendered buthere a metaphor might be interpreted - probably unintended by the sender (‘sales people are dogs’). Max is a mix ofrealistic (head) and unrealistic rendering (kinesthetic skeleton) (http://www.techfak.uni-bielefeld.de/ags/wbski/lehre/digiSA/Methoden_der_KI/WS0102/methki15.pdf). He has no metaphor in him (‘aconversation partner is a person’). Robby the Robot (Microsoft) is an example of an unrealistically depicted servant(walking and talking robots do not exist yet). Robby is a metaphor and a true personification because he is theanthropomorphization of hardware and software (e.g., the search engine) in one and represents an abstraction: ‘Helpis a mechanic man’ (cf. the ‘User Virtual Machine,’ Van der Veer, Tauber, Waern, & Van Muylwijk, 1985). Anothermetaphor he carries is that ‘servants are robots,’ that is, stiff and restrained people.

descriptive figurative descriptive figurative

literal metaphor

realistic

descriptive figurative descriptive figurative

literal metaphor

unrealistic

ASSOCIATION COMMUNICATION REPRESENTATION

Figure 7. Agent typology according to representation style, communication form, and association type. From left toright: Two tutors, a product presenter dog, the conversational agent Max, and Robby the Robot.

A mainstream assumption in the field is that agents with realistic features evoke more involvement andappreciation than with unrealistic ones. PEFiC explains, however, that on the epistemic dimension, judgments aboutthe reality status of the features of a fictional character (realistic-unrealistic) should be separated from theirmetaphoric status. Realistic as well as unrealistic representations may evoke literal as well as metaphoriccommunications (Figure 7). This means, that unrealistic features (the ‘wings’ of Time) may be highly appreciated or

feet feetslimy restrained

suit


require an educational extra when they evoke a figurative association (‘wings’ express ‘speed’). Yet, PEFiC gives noaccount of how metaphors that are connected to fictional characters are processed. In other words, to account forpersonifications, PEFiC should be extended with a model of metaphor processing.

The Race model of Metaphor Processing (RMP) offers a unifying account of processing metaphoriccommunication (Hoorn, 1997). At the basis lies the distinction between descriptive and figurative associations (cf.Richards, 1965). Descriptive associations of a word such as ‘dog’ are the explanatory features (‘legs,’ ‘tail,’ etc.).Figurative associations are features with a symbolic or emblematic connotation (‘drooling’ is literal for saliva,figurative for being overly enthusiastic). Figurative associations may be embedded in the culture but may also bequite personal. Three empirical findings form the underpinning of the ‘multiple-meanings theory’ that is modeled inRMP. First, metaphors (cf. the product presenter dog) awake more figurative associations than literalcommunications (cf. the animated tutor). Second, the reaction time to judge that a communication is metaphoric ismarginally longer than to judge it is literal. Third, while recording the electroencephalogram (EEG) online, theenergetic effect of the semantic surprise in metaphors is higher than in literal expressions. Van Driel et al. (1998)found that appreciation for books is highest when the ratio, emotion, and fantasy of readers were equally activated.Thus, metaphors draw on fantasy more than literal expressions because they activate more figurative associations.They take only a few milliseconds longer to process, so that the stimulation of fantasy is at the cost of little moreeffort. Moreover, the electrocortical effect of semantic surprise occurred early in processing the metaphors (385msec. after presentation). Deploying a metaphor for information transfer, then, may increase attention and may urgeto use more control processes (ratio). The emotional side of processing a personification may be covered by the factthat it is also a fictional character. In requiring more control processes (ratio), stimulating fantasy, and establishingaffective relationships, personification probably raises higher appreciation than other communication forms.

To further explore the RMP, Hoorn (2001) suggests a measuring technique that is highly sophisticated forthe literary field and computer science with stimuli that are highly uncommon in psychology. Based on the work byColes et al. (1995), Hoorn (2001) explains how the lateralized readiness potential (LRP) in the EEG can be used tosettle an old score in metaphor theory. The LRP is sensitive to the making of “partial errors,” incorrect choices thatare not executed because the user makes the right decision after all. This indicates that a process may be started (tomake the incorrect decision) but is canceled in favor of the process for the correct decision. In the actual correctdecision, this is not visible. In the accompanying LRPs, it is. Authors such as Richards (1965), Henle (1966), andBlack (1979) posit that an expression always is first interpreted literally and if the expression is a metaphor, then fig-uratively or relational. Thus, metaphors require more cognitive elaboration because they take an extra processingstage. However, Hoorn (1997, pp. 245, 281) found that literal communications and metaphors evoke parallelprocesses of descriptive and figurative information evaluation, with a slight speed advantage for literalcommunications. This suggests that literal communications and metaphors are equally difficult to process. Or arethey? Perhaps metaphors raise more partial errors than literal communications, and require more cognitive effort toreadjust the process for the correct decision (‘Robby is a metaphor!’). This can be made visible in the recordedLRPs, which has the following theoretical value. If personifications are metaphors, and when metaphors areappreciated better because they require more cognitive elaboration, then processing personifications should raisemore partial errors and evoke more LRP activity than literal agents such as the tutor (Figure 7) or other (fictional)characters. This higher appreciation, however, could be at the cost of information transfer, exactly becausepersonification may raise more partial errors and requires more effort.

Again, the tasks at hand in the interaction context (Figure 2) are decisive. If personifications as metaphorsare appreciated more because they involve more cognitive effort, this raises the question of whether the user’s taskimposes a cognitive load that results in a scarcity of cognitive resources. For example, would air traffic controllersappreciate an interface in which information necessary for time-critical decisions is presented metaphorically?Moreover, the account of cognitive processing of metaphor, if true, may depend upon the extent to which the subjectis accustomed to the metaphor. Metaphorical meanings have a tendency through use to turn into literal meanings.Thus, for example, as Web browser users become accustomed to using “bookmarks” to save links to Web pages, thenew meaning for the term “bookmark” is reinforced and the metaphor of Web as book becomes less salient. Thiscomplicates the problem of how to discriminate literal from metaphoric communication reliably in an experimentaldesign.

Communication ContextThe Director of the Advanced User Interface Design Group at Microsoft, Tandy Trower, posits that

although “there are a number of people who are working on inference engines and other smart technologies, what ismissing out of the picture is how we interact with these [intelligent agents].” (Barker, 2001). Although some workhas been done (e.g., Pollack, 1990; Lester et al.; Bonito et al., 1999; Cassell, 2000), there is no firm theoretical or


empirical ground to capitalize on the efficacy of personification and to ignore simpler forms such as literalinstructions (dialog boxes), metaphors (windows, bin, desktop), and performed characters (the tutor) (cf.Shneidermann, 1998, pp. 380-384). Figure 8 is an attempt to model the processing of cultural products (such asagents) in computer interfaces in which GTA, PEFiC, and RMP can be integrated. The central issue in GTA is thatthe agent satisfies the goals of the user, which may be entertainment, education, information, task or work processfacilitation, sales, etc. PEFiC and RMP relate to the psychological processing of communication forms such ascharacters and metaphors.

Goal: Intended contentssender = understoodcontents user

Contents: Informationto realize goal user

Goal: Entertainment,education, information,process control, etc.

Communication form:Literal, metaphor,mediated person or fictional character,personification

Medium:User-interface, literary book, theater,film, TV

Sensory channel:Visual (writing, image),acoustic (speech,sound), touch

Three domains ofthe human mind:

Ratio, emotion, fantasy,

encompassing four processes:Comprehension, motivation,

effort, and interaction

Informationprocessingby the user

Feedback:Goal reached?

Agent Message User

Figure 8. Outline of a user-centered model of human-agent communication.

Presumably, the following forms of communications mainly activate the subsequent domains of the humanmind:

Communication forms Domains involvedLiteral instructions ratio Metaphors ratio and fantasy Fictional Characters ratio and emotionPersonifications ratio, emotion, and fantasy

Van Driel et al. show empirical evidence that the highest appreciation of literary books was based on equalactivation of the three domains ratio, emotion, and fantasy. I argue that personifications may activate all threedomains of the human mind. Therefore, they are optimal communication devices and yield the highest appreciationrate. Because information transfer might be more efficient through personifications, the use of personifications toinstruct users on performing a decision task in an interactive environment should ideally lead to more ‘hits’ andfewer ‘misses’ and faster response times. Nevertheless, certain factors may obstruct such a straightforward outcome.Personifications probably excite certain information sources more strongly than other forms of communication do:

Information sources of personification Domains mainly involved


The fictional character (cf. Konijn & Hoorn, 2002) ratio and emotionThe metaphor (cf. Hoorn, 1997) ratio and fantasyThe message contents (Figure 8) ratio

Because personification is a composite of metaphor and fictional character, more information sources areprocessed and more processes are active in more domains, so that personification is harder to process than the othercommunication forms. Investing more effort leads to elongated response times and thus to delays in achieving ‘hits,’which lowers the appreciation. Personifications, then, should raise more ‘hits’ and fewer ‘misses’ in helping to reachthe user’s goals but the speed of achieving ‘hits’ might be low, since the processing of personification takes moreeffort. Thus, the appreciation of personifications is the trade-off between the profit of a higher stimulation of themind (no boredom) and the costs of investing more effort and getting delayed satisfaction.

As said, a personification is an abstraction represented by a concrete living creature, as in the case of LouisXIV, who said: “The State, that’s me.” The stimulus materials, therefore, should not consist of abstractions explainedby other abstractions, for example, “The State, that’s the government” (literal), “The State, that’s the hell”(metaphor) or by concrete but dead objects “The State, that’s a ship” (metaphor). The abstract topic of thepersonification in the present proposal is help in achieving the user’s goals. In an interface, help is successful if therequired functionality is provided to reach a certain goal (e.g., increase productivity or finding the proper e-commerce site). The stimulus materials, therefore, should meet the following constraints:

Form of communication Manipulation Examples(1) Literal instruction descriptive, non-human help is a manual(2) Fictional Character descriptive, human help is a tutor(3) Metaphor figurative, non-human help is a key(4) Personification figurative, human help is a wizard

Operationalization of the examples (1) up to (4) for user interfaces:

(1) Dialog box: “Put the cursor on the underscored word (=hyperlink) and connect to the digital document (=Web page)”

(2) Animated tutor: “Put the cursor on the underscored word (=hyperlink) and connect to the digital document (=Web page)”

(3) Dialog box: “Put the key (=cursor) into the lock (=hyperlink) and open the door (=connect) to the treasure room (=Web page)”

(4) Wizard Merlin: “Put the key (=cursor) into the lock (=hyperlink) and open the door (=connect) to the treasure room (=Web page)”

Personifications supposedly are more motivating and more fun (raise higher appreciation) because theyemploy more information sources and start more processes in more domains of the mind but they are also harder toprocess. Thus, do personifications facilitate computer mediated communication or not? Below the relevant methodsand measuring techniques are elaborated to find things out empirically. I propose three research topics that areconnected to specific methods.

Three research topics Methods1. processing fictional characters → structured questionnaires2. processing metaphors → reaction timing, physiological recordings (LRP)3. processing message contents → structured questionnaires and reaction timing of

‘hits’ and ‘misses’

The associations of the users obtain the features of the stimuli that are used in the questionnaires,behavioral, and psychophysiological tests. That Merlin can do magic is more important than that he wares a bluemantle. Features with high frequencies of mentioning serve, for example, as the subject of items in the scales of thestructured questionnaire

Concerning research topic 1, normal fictional characters are compared with personifications. The theoreticalcenter is the PEFiC-model (Konijn & Hoorn, 2002). Based on this theory, the main question is whetherpersonifications activate ratio and emotion better than other fictional characters because the involvement-distance


conflict they raise feeds the highest appreciation. The complete questionnaire for investigation of fictional charactersis developed and tested as documented in Hoorn and Konijn (2001).

In the test on the metaphor aspects (research topic 2), literal forms of communication are compared withmetaphors. Central are the ‘multiple-meanings theory’ and RMP model of Hoorn (1997). How do descriptive (ratio)and figurative (fantasy) information interact with each other to arrive at the decision ‘Merlin is a metaphor’? InHoorn’s multiple-meanings theory, metaphors evoke more meanings (i.e. descriptive and figurative) than literalcommunications, which mostly activate descriptive meanings. Therefore, metaphors should take more time toprocess than literal communications. The RT-literature on metaphors, however, shows that no evidence for responsetime differences is found. The present proposal suggests that standard RT-procedures are too coarse a measure tofind that users do employ two information sources in metaphors and only one in literal communications. To do so,Hoorn (2001) developed a complete research design to utilize the lateralized readiness potential (LRP) in the EEG.In combination with the “partial errors” that users make during a decision task, LRPs can render information on theprocessing of the descriptive and figurative information source in metaphors. The test prediction is that processingtimes may be equal but that the hesitation to decide whether an expression has descriptive or figurative meaning isgreater in metaphors than in literal communications. Hoorn (2001) proposes to confront users with mixedpresentations of literal and metaphoric communications and that they should decide for ‘literal’ or ‘metaphor’ bypressing L with the one hand or M with the other hand, respectively. For metaphors, muscle tension should be foundin preparation of erroneously pressing the L-button accompanied by heightened contralateral LRP-activity, afterwhich the error is not made and the M-button is pressed. For literal communications, such “partial errors” should notoccur. In so doing, it is made feasible that for metaphors users control a descriptive as well as a figurativeinformation source, whereas this is unnecessary for literal communications. Different from interaction with literalcommunications, then, metaphors (and thus personifications) would not only activate the ratio but also stimulatefantasy.

The next paragraphs concern the processing of the message contents during human-agent interactions(research topic 3). In the three domains of the human mind (ratio, emotion, and fantasy), four processes aresupposedly involved during user’s goal achievement through agents: Comprehension, motivation, effort, andinteraction (see Figure 8). A questionnaire should be constructed with which each communication form (literal,fictional character, metaphor, and personification) can be studied (between-subjects design). Items are statements (6indicative, 6 contra-indicative in a scale) and combine a domain with a process (see Table 1). Users rate on at 6-pointscale (not agree – agree) in how far the item reflects their opinions.

Table 1. Communication model for human-agent interaction tested by a structured questionnaire. Theexamples combine domains with processes. The symbol X should be substituted by the respective literal ormetaphoric communications, the fictional characters or personifications (e.g., Merlin). DomainProcess

Ratio Emotion Fantasy

Comprehension Because X made me think,I got the message

Because X made me smile,I understood the task

Because X made me dream,I knew what he meant

Motivation Owing to X, I told myselfto keep learning

Because X moved me, Ipushed through

The magic of X made mecurious to see more tricks

Effort Due to X, I had difficulty tokeep everything in mind

That X touched me, costme strength

The fiction around X tookmy energy

Interaction The cooperation with Xdrew upon my insight

The feedback of X affectedmy feelings

The turn taking of Xaroused my imagination

By performing a confirmative factor analysis on each of the four communication forms and fitting themodel with the questionnaire answers, the correlations between domains and processes can be analyzed. It shouldshow that personifications evoke the highest comprehension of the message contents, provoke the highestmotivation, cost most effort, but also raise the most interaction, which correlates with the activation of ratio as wellas emotion and fantasy. Metaphors, on the other hand, do not lead to the highest comprehension, provoke lessmotivation, cost less effort, and render less interaction than personifications, which correlates with the activation ofonly ratio and fantasy and insignificant correlations with emotion. The same is valid for fictional characters, but thenbecause of the correlation with only ratio and emotion and insignificant correlations with fantasy. Likewise for literal


communications, which is due to the correlations with merely ratio and the insignificant correlations with emotionand fantasy.

A forced decision reaction time test for each of the four communication forms can register the speed of‘hits’ and ‘misses,’ which should indicate the speed with which and the number of times that the user’s goal issatisfied. The task in this experiment is to follow an instruction that is presented in literal, metaphoric, fictionalcharacter, or personification form. The task is, for example, to “click on the underscored word A and connect to thedigital document” (see for different forms the stimulus materials). In addition, rivaling options are offeredsimultaneously (hyperlink B, C, D). The more processes and domains are active, the slower processing gets but alsothe fewer mistakes are made (speed-accuracy trade-off). Speed as expressed in milliseconds and error score shouldreveal the following:

Millisec.literal < fictional character ≈ metaphor < personification% errors literal > fictional character ≈ metaphor > personification

Information transfer via literal communications may be faster but through personifications more accurate.An additional questionnaire should show whether higher decision speed or lower error-percentage induce a higherlevel of satisfaction and appreciation in work process control.

In conclusion, a structured questionnaire study into the model of human-agent interaction should enlightenwhether personifications yield the highest comprehension of the message, evoke most motivation, cost most effort,and yield most interaction, which should correlate with the activation of ratio, emotion, and fantasy. Additionalreaction time tests can elucidate if information transfer through literal communications is faster and viapersonifications slower, so that the latter costs more effort but is also more accurate (more hits of the required goals).An additional questionnaire should inform about the level of satisfaction and appreciation for the alternatives ‘fast’(literal communications) versus ‘thorough’ (personification). The height of the appreciation of personifications(fictional character and metaphor in one) is explained from the special processes that are involved in perceiving andexperiencing fictional characters in combination with metaphors. A structured questionnaire study should reveal ifpersonifications compared to normal fictional characters activate ratio and emotion the strongest, because they excitean optimal ‘involvement-distance conflict’ and thus invoke the highest appreciation. With psychophysiologicalexperiments into the brain potentials in the motor cortex it can be studied if metaphors activate both descriptive andfigurative information sources and if literal communication offers only a descriptive information source. Theprediction is that literal communications only activate the ratio, whereas metaphors (and thus personifications) notonly stimulate the ratio (descriptive) but also fantasy (figurative).

DiscussionAt the outposts of what we know about communication, psychology, and the media, I have defined the

processing of the artistic and cultural aspects of personification in communication interfaces and gaming as an areaof possible research and a new topic in computer mediated communication and human-computer interaction. Thishas been skirted and alluded-to in various ways (e.g., Reilly, 1996; Paiva et al.; Machado et al.) but for the first timean integrative account is provided leading to verifiable predictions. Is it especially helpful in some way to have ideaspersonified, and perhaps in a later stage, does the value depend on the medium (e.g., interface vs. TV)? This is anovel question at a time when media are rapidly developing new formats. The leaping ahead of informationtechnology has a dazzling effect on its users, whether they are novices or experts. Users can easily conjure upcompletely new identities and become fictional characters themselves (e.g., http://secretadmirer.com for Valentine’swishes). No one can oversee the possibilities and consequences, which leaves many with feelings of insecurity andincapacity. To keep their products from being appreciated negatively, computer companies develop enrichments ofthe graphic interface of applications to humanize, for instance, Web sites. Personifications of abstractions are appliedto help users on their way through electronic wilderness. Is it worthwhile to put money and effort into thedevelopment of such ‘enrichments?’ On a deeper level, do personifications evoke more affect than other metaphors,because they are fictional characters? What kind of affective relationships (involvement and/or distance) do usersform with such fictional characters? Are users prepared to start up more control processes for understanding themetaphor behind the animated agent? Does it increase their insight into the underlying abstractions? Do personi-fications stimulate the fantasy more than literal dialog boxes? How contributes all this to appreciation? Thesequestions cannot be answered without looking into theories on perception and emotion psychology, literature, TV,film, and computer mediated interaction. The results may bear not only on communication interface design but alsoon a better understanding of how artistic stimuli are experienced and understood.


There are three novel elements in the proposed theory of personification: The singling-out of personificationfor special attention, a theory of the metaphoric character of personification (especially the role of multiple “partialerrors”), and the use of an EEG marker that is said to indicate part of comprehension. The argument aboutpersonification takes the form: We first identify a character and then later take the character to be a personification.Presumably, third, we settle on a conception of what is personified and why this is a more-or-less suitablepersonification. This is said to be the insight that should be incorporated into a theory for design of communicationinterfaces. Actually, this argument about personification is just the content or application of the theory of metaphoriccommunication that this article offers. My multiple-meanings theory is metaphoric communication triggers morethan literal communication because descriptive as well as figurative information is set free. In personification,moreover, the communication triggers both the character and the notion that the character stands for something. Thenon-metaphoric character would just trigger the character. This theory of personification is a comparatively originaldomain of application of theory of metaphoric communication, since most theories are worked out using examples ofmetaphors such as “life is a journey.” However, this use of personification is more-or-less just an area of application,and perhaps the real merit of the proposal lies in the multiple-meanings theory of metaphoric communication.

The argument is that metaphors trigger many partial errors, and more than comparable (control) literalcommunications. This hypothesis has several rivals in the field including A. the literal-first/figurative-second order-of-processing theory of Richards and Henle (mentioned and favored by me to some extent), B. a parallel-processingtheory (in the spirit of Gibbs, 1992) and also favored to some extent, and C. a recent salient-meanings-first theory ofGiora (1999). If metaphoric communication often triggers more meanings than control literal communication thiscould explain many of the findings in the technical experimental literature and deserves to be distinguished from theA, B, and C accounts above. The argument that a component of the EEG called the LRP component is sensitive tothe making of “partial errors” (interpretations which are entertained to some extent before a final interpretation isachieved), is intriguing but a long-shot. This interpretation of the LRP would require a great deal of ground-breakingwork to establish. Any research that assumes this interpretation is therefore standing on highly contentious ground.But then, what is research for? Nevertheless, I should add that one concern here is that the research would beproceeding from this LRP interpretation rather than acting as the primary navigator checking and revising it, and sothe ideas will be subject to revision as LRP research reinterprets itself. On the other hand, who does not have toassume the value of someone else’s research pro tem? The LRP/EEG method should be used in concert with othermethods to test theories of processing and multiple-meanings. They will involve use of reaction-time measures,controlled presentations, and considerable care over control materials that have “comparable meanings” toexperimental materials. In general, I must caution that each method mentioned in the proposal will give results thatwill require considerable interpretation. The materials selected for examination will have a clear impact on theresults and interpretations. Considerable care over the sampling and range of materials is needed (see Hoorn, 2001,Table 2). Further, the LRP idea about “partial errors” is just a way of testing the theory of metaphoriccommunication. In sum, this area of the proposal is actually an area of application of more basic tools.

My theoretical proposal has intra- and interdisciplinary impact as well as practical relevance. Instead of thetwo traditional routes to an understanding of the mechanisms of personification, the linguistic-rhetorical and thephilosophical-ontological, the proposed theory intends to fuse the cognitive theory on metaphor processing Hoorn(1997) with the theory on engaging fictional characters (Konijn & Hoorn, 2002). By placing personification within alarger context of basic lexical, semantic, and cognitive transfer mechanisms (Paxson, 1994), literary theory onpersonification can be understood in a much more precise and enlightening manner than hitherto done. Cognitivemetaphor theory can be extended with a context-dependent model on affective experience. The epistemic dimensionof experiencing fictional characters can be expanded with a full-fledged model on metaphor processing. Tests withreal users would not only uncover new insights for human-agent communication and the arts but also for thepsychology of complex semantic stimuli. The practical relevance lies in the investigation of whether informationtransfer by communication interfaces is more successful via metaphor than literal explanation, because fantasy andratio are stimulated more optimally. Moreover, are personifications more successful than other metaphors, becausethe emotional domain is excited more strongly? The big thing for computer science is whether “users really needextensive help and explanation facilities?” (Brazier & Van der Veer, 1994) and if they do, what kind? Byinvestigating the usefulness of agents for communication, the present proposal wants to help clarify that problem. Itprovides part of the conditions that favor the use of personifications in interface applications such as search enginesand agents for user generated personal profiling of supplies and with it, a set of mental models for these can bedefined, featuring appropriate personifications. The outcome could be helpful to graphic designers, interfaceengineers, and technologists.


Already, multi-agent systems are one of the key technologies for software development. This requires thework of researchers from the fields of logic, theoretical computer science, and multi-agent systems in order todiscuss formal techniques for specifying and verifying such systems. However, the human aspect is the domain ofcommunication studies, psychology, and the arts. If we want to catch up with technological developments andaccount for multi-agent communication, then we should start out with thorough theory of that one person incommunication with that one agent. Fictional characters such as animated agents may simultaneously be good andbad, bring joy and terror, they may please some but put into a metaphor, they try all. In view of the growing impactof the digital world on education and (cognitive) behavior, therefore, let us keep up with Time “to plant ando’erwhelm custom” in communication and IT research.

ReferencesAnderson, J. R., Corbett, A. T., Koedinger, K. R., and Pelletier, R. (1995). Cognitive tutors: Lessons learned.

Journal of Learning Science, 4, 2, 167-207.Apple Computers, Inc. (1990). Knowledge Navigator (video). Cupertino, CA: Apple Creative Services.Barker, D. (2001). The secret agent man. Microsoft’s new animated agent technology [Online]. Retrieved January

17, 2002, from http://www.botspot.com/pcai/article11.htm.Bartelson, E. (2001). Review Oni. PlayNation 3, 18, 58-61.Black, M. (1979). More about metaphor. In A. Ortony (Ed.) Metaphor and thought. Cambridge, UK: Cambridge

University Press.Bonito, J. A., Burgoon, J. K., and Bengtsson, B. (1999). The role of expectations in human-computer interaction. In

Proceedings of the international ACM SIGGROUP conference on supporting group work (pp. 229-238). NewYork: ACM Press.

Boorstin, J. (1995). Making movies work. Beverly Hills, CA: Silman-James.Brazier, F. M. T., & Van der Veer, G. C. (1994). Do users really need extensive help and explanation facilities? In P.

J. Schellens, & E. R. Seydel (Eds.), Communicatie in onderzoek (pp. 19-31) [Communication in Research].Enschede, NL: Twente University.

Buck, R. (1999). The biological affects: A typology. Psychological Review, 106, 2, 301-336.Cassell, J. (2000). Embodied conversational interface agents. Communications of the ACM, 43, 4, 70-78. Coles, M. G. H., Scheffers, M. K., & Fournier, L. (1995). Where did you go wrong? Errors, partial errors, and the

nature of human information processing. Acta Psychologica, 90, 129-144.Craig, S. D., Gholson, B., & Driscoll, D. (2002) Animated pedagogical agents in multimedia educational

environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology.94, 428-434.

De Kerckhove, D. (1991). Brainframes: Technology, mind and business. Utrecht, NL: Bosch & Keuning.De Kerckhove, D. (1995). The skin of culture. Toronto, Ca: Somerville House Publishing.De Kerckhove, D. (1997). Connected intelligence. Toronto, Ca: Somerville House Publishing.Dehn, D. M., & Van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research.

International Journal of Human-Computer Studies, 52, 1, 1-22.Dix , A. J., Finlay, J. E., Abowd, G. D., & Beale, R. (1998). Human-computer interaction. London, New York:

Prentice Hall Europe.Duncker, E. (2002). Cross-cultural usability of the library metaphor. Proceeding of the second ACM/IEEE-CS joint

conference on digital libraries 2002, Portland, Oregon, USA (pp. 223-230). New York: ACM Press.Elliott, C., Rickel, J., and Lester, J. (1997). Integrating affective computing into animated tutoring agents. In

Proceedings of the IJCAI workshop on animated interface agents: Making them intelligent (pp. 113-121).Nagoya, JP: New York: ACM Press.

FitzGerald, P. J., Buie, T., and Cuales, M. (1998). The EyeCue system: A prototype for the next generation ofeducational technology [Online]. Retrieved January 15, 2002, from http://www.ncsu.edu/meridian/jun98/feat2-1/feat2-1.html.

Frijda, N. H. (1986). The emotions. New York: Cambridge University.Gibbs, R. W. (1992). Categorization and metaphor understanding. Psychological Review, 99, 3, 572-577. Giora, R. (1999). Literal and figurative language. Journal of Pragmatics, 31, 12, 1539-1707.Henle, P. (1966). Language, thought and culture. Ann Arbor, MI: University of Michigan Press.Hoorn, J. F. (1997). Metaphor and the brain: Behavioral and psychophysiological research into literary metaphor

processing. Ann Arbor, MI: ProQuest.


Hoorn, J. F. (2001). A renaissance perspective on the empirical study of literature. An example frompsychophysiology. In D. H. Schram, & G. Steen (Eds.), The psychology and sociology of literature (pp. 129-143). Amsterdam, Philadelphia: John Benjamins.

Hoorn, J. F., and Konijn, E. A. (2001). Report on questionnaire for perceiving and experiencing film characters,Tech. Rep. [Online]. Retrieved October 30, 2002, from http://home.scw.vu.nl/~konijn.

Hoorn, J. F., Konijn, E. A., & Van der Veer, G. C. (2002). Virtual reality: Do not augment realism, augmentrelevance. In I. A. Cuevas, P. Díaz Pérez, & C. Fernández Llamas (Eds.), Actas del III congreso internacionalde interacción persona-ordenador, Leganés, May 8-10, 2002 (pp. 154-163). Leganés, ES: Universidad CarlosIII de Madrid.

Johnson, M. P., Wilson, A., Blumberg, B., Kline, C., & Bobick, A. (1999). Sympathetic interfaces: Using a plush toyto direct synthetic characters. In Proceeding of the CHI 99 conference on Human factors in computing systems:The CHI is the limit (pp. 152-158). New York: ACM Press.

Katz, S. (1991). Film directing shot by shot. San Diego, CA: Michael Weiss Publications.Konijn, E. A., and Hoorn, J. F. (2003). Ethic, aesthetic, and epistemic borderlandsLaurel, B. (1990). Interface agents: Metaphors with character. In The art of human computer interface design.

Reading, MA: Addison-Wesley. Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., and Bhoga, R. S. (1997). The persona

effect: Affective impact of animated pedagogical agents [Online]. Retrieved January 17, 2002, fromhttp://www1.acm.org/sigs/sigchi/chi97/proceedings/paper/jl.html.

Machado, I., Paiva, A., & Prada, R. (2001). Is the wolf angry or… just hungry? In Proceedings of the fifthinternational conference on autonomous agents, 2001, Montreal, Quebec, Canada (pp. 370-376). New York:ACM Press.

Marcus, A. (1993). Human communications issues in advanced UIs. Communications of the ACM, 36, 4, 100-109.Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1-19. Mayer, R.E. (2001). Multimedia learning. Cambridge, UK: Cambridge University Press. Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more

material results in less understanding. Journal of Educational Psychology, 93, 187-198. Mayer, R. E., & Moreno, R. (1998). A split attention effect in multimedia learning: Evidence for dual processing

systems in working memory. Journal of Educational Psychology, 90, 312-320. McBreen, H., Shade, P., Jack, M., Wyard, P. (2000). Experimental assessment of the effectiveness of synthetic

personae for multi-modal e-retail applications. In Proceedings of autonomous agents 2000, Barcelona, Spain,June 3-7 (pp. 39-45), New York: ACM Press.

Paiva, A., Machado, I., & Prada, R. (2001). Heroes, villians, magicians, …: Dramatis personae in a virtual storycreation environment. In Proceedings of the 6th international conference on intelligent user interfaces 2001,Santa Fe, New Mexico, USA (pp. 129-136). New York: ACM Press.

Paxson, J. J. (1994). The poetics of personification. Cambridge, UK: Cambridge University Press. Pollack, M. E., and Ringuette, M. (1990). Introducing the tile-world: Experimentally evaluating agent architectures.

In Proceedings of the conference of the American Association for Artificial Intelligence (pp. 183-189). NewYork: ACM Press.

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., Carey, T. (1994). Human-computer interaction. NewYork, Amsterdam, Tokyo: Addison-Wesley.

Raaijmakers, J. G. W., and Shiffrin, R. M. (1981). Search of associative memory. Psychological Review, 88, 93-134.Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television, and new media like

real people and places. Cambridge, UK: Cambridge University Press.Reilly, W. S. N. (1996). Believable social and emotional agents. Unpublished doctoral dissertation, Tech. Rep.CMU-CS-96-138. Pittsburgh, PA: School of Computer Science, Carnegie Mellon University.Richards, I. A. (1965). The philosophy of rhetoric. London, New York: Oxford University Press.Rickel, J. and Johnson, W. (1999). Animated agents for procedural training in virtual reality: Perception, cognition,

and motor control. Applied Artificial Intelligence, 13, 343-382.Shapiro, M. A., & McDonald, D. G. (1992). I’m not a real doctor, but I play one in virtual reality: Implications of

virtual reality for judgments about reality. Journal of Communication, 42, 4, 95-114.Shneidermann, B. (1998). Designing the user interface. Strategies for effective human-computer interaction.

Berkeley CA, Amsterdam, Tokyo: Addison-Wesley.Terry, D. J., and Hogg, M. A. (1996). Group norms and the attitude-behavior relationship: A role for group

identification. Personality and Social Psychology Bulletin, 22, 8, 776-793.


Tversky, A. (1977). Features of similarity. Psychological Review, 84, 4, 327-350.Umaschi, M. (1996). SAGE storytellers: Learning about identity, language and technology. In ICLS 96 Proceedings,

AACE, 526-531. Van der Veer, G. C., Lenting, B. F., & Bergevoet, B. A. J. (1996). GTA: Groupware task analysis - modeling

complexity. Acta Psychologica, 91, 297-322.Van der Veer, G. C., Tauber, M. J., Waern, Y., & Van Muylwijk, B. (1985). On the interaction between system and

user characteristics. Behaviour and Information Technology, 4, 289-308.Van der Veer, G. C., & Van Welie, M. (2000). Task based groupware design: Putting theory into practice. In

Conference proceedings on designing interactive systems: Processes, practices, methods, and techniques, 17-19August 2000, New York (pp. 326-337). New York: ACM Press.

Van Driel, K., Hoorn, J. F., Oosterveld, P., & Vorst, H. C. M. (1998). Beyond good and bad: The cognitiveprocessing of cultural products model. SPIEL, 17, 2, 214-231.

Wooldridge, M., & Jennings, N. R. (1995). Intelligent agents: Theory and practice. The Knowledge EngineeringReview, 10, 2, 115-152.

AcknowledgmentsThis paper is part of the Innovation Oriented research Program (IOP) for Human-Machine Interaction entitledIntegrating Design of Business Processes and Task Analysis, granted by the Senter Agency of the Dutch Ministry ofEconomic Affairs in The Hague, grant Mmi99009. I am deeply indebted to Derrick de Kerckhove of the McLuhanProgram in Culture & Technology, University of Toronto and several anonymous reviewers for their extendedcomments, clear suggestions, and thoughtful discussions. Electronic mail may be sent to [email protected].


Appendix

William Shakespeare

THE WINTER’S TALE (1610-11)

Act IV

Enter TIME, as Chorus.

Time. I,-that please some, try all; both joy and terrorOf good and bad; that make and unfold error,-Now take upon me, in the name of Time,To use my wings. Impute it not a crimeTo me or my swift passage, that I slideO’er sixteen years, and leave the growth untriedOf that wide gap, since it is in my powerTo o’erthrow law, and in one self-born hourTo plant and o’erwhelm custom. Let me passThe same I am, ere ancient’st order was,Or what is now received: I witness toThe times that brought them in; so shall I doTo the freshest things now reigning, and make staleThe glistering of this present, as my taleNow seems to it. Your patience this allowing,I turn my glass, and give my scene such growingAs you had slept between. Leontes leaving The effects of his fond jealousies, so grievingThat he shuts up himself; imagine me,Gentle observers, that I now may beIn fair Bohemia; and remember well,I mention’d a son o’ the king’s, which FlorizelI now name to you; and with speed so paceTo speak to Perdita, now grown in graceEqual with wondering: what of her ensues,I list not prophesy; but let Time’s newsBe known when ‘tis brought forth:-a shepherd’s daughter,And what to her adheres, which follows after,Is the argument of Time. Of this allow,If ever you have spent time worse than ere now;If never, yet that Time himself doth sayHe wishes earnestly you never may. [Exit.

personification in computer mediated communication

Documents