D MODELINGSOCIALLY INTELLIGENT AGENTS

BRUCE EDMONDS Centre for Policy Modelling, Manchester Metropolitan University, Manchester, United Kingdom

An approach to modeling houndedly rational agents is described. This approach uses the parallel evolution of a population of mental modelsfor each agent based on the genet ic programming paradigm. Then some of the characteristics one would expect of a society of agents (as opposed to a collect ion of interacting agents) is argued for. These include the abilit y co idencify, model, and communicate with specific other agents in a heterogeneous way. An example model is described, which extends Brian Arthurs' " £/ Faro/ Bar"' model with learning and communication. The model allows the co-evolution of the agent s"

populations of models. The results are then analyzed co show the differentiation that results between the agents. A specific case study of the situat ion at the end of the simulation is then examined in depth. This indicates that some of the problems in analyzing human communicat ion will also occur with such models and that explanations based on

W ittqensteinian lanquage qames and use in practice may he most appropriate.

In the artificial agent community there is now quite a lot of research on the phenomena that result when agents interact. Most of this research focuses on the inferential process, which involves typically either an agent inferential mechanism designed so that a community of agents can perform some speci­fied task or an inference mechanism specified so that the resulting inter­action between agents can be examined. In computational organization theory the agents are frequently allowed to learn as well as interact, but the object is to examine the effect of the given structure in which the agents operate. Consequently, the agents in these models are frequently quite simple.

This article represents a step toward modeling the learning that might occur in a collection of reasonably sophisticated agents who communicate

The original version of this article was published as Modeling Socially Intelligent Agents in Organ­isa tions, by B. Edmonds, in Socially Intelligent Agents, 37- 42, AAAI Press Technical Report FS-97-02, © 1997, American Association for Artificial Intelligence.

My thanks to Scott Moss for many discussions about social modeling, to Helen Gaylard for com­ments on the idea of doing linguistics on the communications of such agents, Steve Wallis for SDML, Edmund Chattoe for comments on a rela ted paper, and Brian Arthur for kindly supplying me with a copy of his source code. SDML has been developed in Visua!Works 2.5.1 , the Smalltalk-80 environment produced by ParcPiace-Digitalk. Free distribution of SDML for use in academic research is made pos­sible by the sponsorship of ParcPlace-Digitalk (UK) Ltd. The research reported here was funded by the Economic and Social Research Council of the United Kingdom under contract number R000236179 and by the Faculty of Management and Business, Manchester Metropolitan University.

Address correspondence to Bruce Edmonds, Centre for Policy Modelling, Manchester Metropolitan University, Aytoun Building, Aytoun Street, Manchester, M1 3GH, United Kingdom. URL : http :jjwww.cpm.mmu.ac.uk/ ~ bruce

Applied Artificial Intelligence, 12:677- 699, 1998 Copyright © 1998 Taylor & Francis

0883-9514/98 $12.00 + .00 677

678 B. Edmonds

and its effects on behavior. The knowledge of the agents thus co-evolves so that the resulting behavior is not just appropriate to the social situation but also comes from the social situation. In such a model, although the agents are each autonomous in terms of their goals, memory, models, and learning, the collection develops very much as a whole. This is in contrast to a para­digm, where to a large extent, agents are designed separately and put together to perform. It is a paradigm where the intelligence of the agents is truly grounded in their social interaction.

The article is broadly divided into two halves: the first half traces the broad aims and techniques used to capture social intelligence in a model, and the second exhibits a model that goes some way toward meeting these aims by using the described techniques.

MODELING AGENTS

This article considers modeling real agents, which can be people or other institutional units (such as firms or departments). The approach is to model these as intelligent software agents. This perspective has different concerns from those involved with designing agents or robots to meet particular goals- what might be called an "engineering perspective." In particular, it seeks veracity over efficiency. It is not claimed here that the agent architec­tures and techniques described herein result in agents, or groups of agents, that are particularly good at any specific task. It does claim that these tech­niques result in communities of agents that exhibit behaviors that character­ize boundedly rational and socially intelligent agents, i.e. , are a step toward modeling some key aspects of humans in social and organizational settings.

One corollary of this is that reactive agents are not used, since a prin­cipal concern is the nature and development of the agents' internal models as they interact with other agents and the environment (see also the reasons in the section below, entitled Social Intelligence). The purpose of modeling these agents is to discover the emergent behavior. If the agent's behavior were closely specified (for example, by compiling it down into a reactive architecture), it would needlessly delimit the behavior that might result, which would result in our being less informed about the possibilities inher­ent in a multiagent situation. Instead, the agents are endowed with flexible and open-ended cognitive processes, so that in principle, it is possible that an agent could induce almost any relevant model.

This leaves open the question of the validation and verification of these models- if you constrain them as little as possible, how do you know if they correspond to reality in any practical way? The answer is twofold. The first step is to validate the mechanics of the model by separating out clearly the implementation details from any implicit theory of cognition (Cooper et al., 1997). We do this by specifying the implementation in a language with

Modeling Socially Intelligent Agents 679

clearly known semantics (in our case, a declarative language) and by basing the agents' cognition in a known process or cognitive theory. The second step is to verify the output of the model against aspects of the real world, even if these are only expressed in terms of qualitative descriptions. This issue is discussed in greater detail by Moss et al. (1997). The type of vali-

dation and verification depends on the purpose of the model. In this article, the purpose of the model is to show the potential of a particular technique in modeling boundedly rational cognition. It is validated against the genetic programming algorithm and the "mental models" paradigm and verified against some qualitative social characteristics.

The agents' internal models are explicitly represented in a specified lan­guage, usually of a quasi-logical or functional variety. This explicit represen­tation makes it possible to limit, examine, and analyze the agents' models, as they develop. I do not claim that humans use such a language of representa­tion (this is a hotly debated issue 1

) but merely that by having such inspect­able and comprehensible models one can easily find out the state of the agent at any time. Providing agents with a suitably expressive internal mod­eling language and allowing them to develop their own models in a very flexible way means that we do not introduce any obviously inappropriate behavior into our agents. The final test of the appropriateness of an agent's behavior is domain dependent and only verifiable with respect to known properties of what is being modeled.

Thus the purpose of an agent in such a model is different from either agents that are designed with a particular purpose in mind (e.g., Bonasso et al. , 1995) or for exploration of the most effective and flexible algorithm for a set of problems. Thus I do not necessarily look to design them to be efficient, general, or consistent in their beliefs.

MODELING BOUNDEDLY RATIONAL AGENTS

In particular, I am interested in agents who

• do not have perfect information about their environment ; in general, it will only acquire information through interaction with its environment, which will be dynamically changing ;

• do not have a perfect model of their environment; • have limited computational power, so they cannot work out all the logical

consequences of their knowledge (Simon, 1972) ; • other resources, like memory are limited (so they cannot hold large popu­

lations of models) ;

In addition to these bounds on their rationality, I also add some other observed characteristics of real social agents, namely,

680 B. Edmonds

• the mechanisms of learning dominate the mechanisms of deduction in deciding their action;

• they tend to learn in an incremental, path-dependent (Arthur, 1994) (or "exploitative") way rather than attempting a global search for the best possible model (Penrose, 1959);

• even though they cannot perform inconsistent actions, they often entertain mutually inconsistent models and beliefs.

The modeled agents have distinct limitations of resources; they are boundedly rational in several respects. They have limited memory, a limit on the resources spent on searches for improved models, and a limit of their ability to make inferences from their models. Following what is known about real agents, we ensure that their search for new models is incremental, rather than global in nature. The limitations on the current memory cache, especially the agent's stock of candidate models, encodes a sharp path dependency.

The fundamental difference between these agents and, say, logic-based agents is that the updating of internal belief structures is done in a competi-

tive evolutionary manner using a continuously variable fitness measure rather than in a "crisp" consistency preserving manner. This is appropriate in situations of great uncertainty caused by a rationality that is not able to completely "cope" with its environment but is more restricted in its ability.

Framework for Boundedly Rational Agents

For the above reasons, Scott Moss and I have developed a paradigm of modeling the learning that such agents engage in, as itself a process of mod­eling by the agents. This follows the strand in cognitive modeling started by Johnson-Laird (1983). For more on this framework, see Moss and Edmonds (1998).

Although social agents primarily develop through a process of incremen­tal learning, they also use some deductive procedures. In real social agents these processes may be arbitrarily mixed as well as developed and abstracted over different layers of an organization. Here we will only look at a model that effectively separates out learning and deduction and comes from an essentially unitary agent structure.

The agent works within a given a priori body of knowledge (e.g., accounting rules). The agent may well make deductions from this in a tradi­tional way and apply these to the current hypotheses. This body of a priori knowledge may also determine the syntax of the models the agent starts with, its principal goals, default actions, fitness functions, and the operations to be applied to its models. Typically, much of this a priori knowledge can

Modeling Socially Intelligent Agents 681

be made implicit in the syntax of the genome (which is the approach we have tended to take).

The agent here has many models of its environment. Once started, the agent incrementally develops and propagates these models according to its evaluation of them, which in turn is based on its memory of past data and

effects of its actions as well as the complexity and specificity. It then selects the best such model according to that measure. The best such model and its goals determine its action. It then takes that action and notes the effects in the environment for future use. The setup is illustrated in Figure 1.

The development of these models (i.e., the learning) is modeled by a learning process on this population of internal models. Important restrictions on such agents include the fact that the agent may have gained only limited information as the result of interaction with its environment and that any action costs it so that it cannot indulge in an extensive explora­tory search without this being weighed against the benefit being gained.

Modeling Bounded Rationality Using the Genetic Programming Paradigm

An important special case of the above approach to learning is where the range of operations includes selection and some mechanism for variation, i.e., an evolutionary algorithm. There are several possible ways of using evolving populations to simulate a community of agents:

1. Each member of the evolving population could represent one agent. 2. Each agent could be modeled by a whole evolving population.

a priori l Goals J c ~

beliefs restriction .L. """' .. ""' ~ - . of models

Populations of Models Machinery (fittest in bold) to decide

Evaluation an Action of Fitness (inference)

Agent

I Memory of Past Data I and act1ons

Environment

Observations Actions

FIGURE 1. Basic structure of an agent.

682 B. Edmonds

3. The whole population could be modeled by the whole evolving popu­lation but without an individually intended agent +-+ gene correspon­dence.

Method 1 has been used in several models of agents that evolve (e.g., Holland, 1992). Here the genetic development has nothing to do with the nature of an agent's cognitive processes but helps determine its goals or strategies. Method 3 is the most popular in economics (e.g., Arifovic, 1994), but unless such a model predicts pertinent properties of real populations of agents, it represents a bit of a fudge and means that the observable behavior and content of individual entities in the model do not have a clear referent in what is being modeled. This makes it far less useful if one wants to use such models to gain detailed insight into the internal dynamics of popu­lations. Method 2 actually addresses the cognitive process as the agent cor­responds to a population of mental models. This has been done before in a limited way by Palmer et al. (1994), but here agents have a fixed menu of possible models that do not develop.

The paradigm of genetic programming (GP) (Koza, 1992) is particularly

appropriate because of the structure of the genome. These techniques, however, cannot be blindly applied. For example, the efficiency of the learn­ing process is only a secondary concern when seeking to model social agents by their software cousins, but many of the other features of this approach for modeling learning are appropriate, namely,

• the population of programs can represent a collection of multiple, com­peting models of the world with which it is concerned ;

• there is always at least one maximally fit individual model that can be used to react to events and from which appropriate deductions can be made;

• the models are incrementally developed by the learning mechanism ; • the fitness measure can be tailored to include aspects such as cost and

complexity as well as the extent of the agreement with known data; • the language of representation of the models can be fairly general and

expressive, e.g., logical expressions.

Thus the GP algorithm is a sort of stand-in for the cognitive processes of the social agent. This is not so far-fetched, given that the purpose is to model the agents at an abstract epistemological level rather than guess at the low-level processes involved in human cognition. Thus this follows the line of thought in evolutionary epistemology, in particular, that of Campbell (1960). 2 Ultimately, this use of GP will only be verified by whether it is a credible model of a suitable subset of human behavior.

Modeling Socially Intelligent Agents 683

In using the evolutionary paradigm in this sort of modeling, I

• represent the agent by a whole evolving population, each gene corre­sponding to one of its alternative models (this is the approach taken in the

example model discussed);

• model populations of agents as evolving populations (i.e., populations of populations), with an intended agent to evolving population correspon­dence;

• give the agents only small populations of models, representing limited memory;

• base the fitness function of a combination of its error compared to past data, size of model, and predictivity (precision and range of applicability);

• restrict the variation operators to generalization, specialization, averaging, combining, and mutating; and

• give an operator only a limited inferential ability to use its best model to choose its action.

This paradigm needs to be integrated with an agent-based approach and adapted to relate to credible models of economic agents. In particular, the crossover operator is somewhat arbitrary when simulating the development of models in economic agents (although undeniably efficient). This also introduces a globality to the search, which is unrealistic.

MODELING ORGANIZATIONS

One motivation of working toward capturing social behavior is that this is important in real (as opposed to planned or theoretical) organizational behavior. For example, it is known that emergent norms and other emergent social structures can greatly affect collective behavior (Latane et al., 1994) and that social factors can make the prediction of organizational behavior from the microlevel behavior very difficult for managers as well as academics (Earley & Brittain, 1992). This concern with the veracity of modeling behav­ior in organizations can be traced back to Simon's (1957) studies. Here in addition to the dimension of procedurality, we include sociality as a signifi­cant factor.

We model such organizations as populations of interacting agents in a given structure. We do not necessarily do this down to the level of individual persons but sometimes stay at the level of departments or even whole firms (if they are themselves interacting). Here a theme we are investigating is the contrast between the official (usually hierarchical) structure of the firm and the unofficial structures that emerge from the social interactions of the indi­viduals.

684 B. Edmonds

In order to study such models, we have developed a modeling language called SDML- a Strictly Declarative Modeling Language. 3 This allows the flexible and theory-free modeling of intelligent agents in a declarative frame­work with object-oriented features (Edmonds et al. , 1996). This is particu­larly well suited for modeling organizations built up in many levels in a composite manner; it allows for better structured large organizational models involving more complex cognitive agents, compared to some other systems (Moss et al., 1998).

Many models developed in organizational science have relatively simple agents but put these in complex structures. This article can be seen as a step toward developing techniques to introduce elements of social complexity into such simulations.

MODES OF COMMUNICATION IN SOCIAL INTERACTION

Communication is a special case of action and perception. It is perhaps for this reason that in many organizational models the communication is rudimentary, i.e., it is implemented on a similar basis to other modeled actions. For example, in many economic models, only the price is communi­cated, and then in a global manner to all the agents equally. However, communication is so important in social situations and potentially so com­putationally onerous that effectively it becomes a separate consideration.

One concern in this regard is the passivity of the communication in many models of interaction. One gets the impression that in many models, agents tend to make requests for information and sometimes issue orders but will not, for instance, volunteer unrequested information. Thus many models of communication correspond to a unidirectional mutual (or merely one-way) transmission of information. For example, in many agent models concerned with negotiation between agents there is an assumption that com­munication will act using the pull of information in the form of requests and replies (in this way, it extends the query-based dialog used in interrogating databases to one of interrogating other agents). This is in contrast to the mix of push and ull modes, found in social interaction, where unrequested infor­mation is frequently volunteered.

A second concern is the inability to refer to specific named individuals or process messages from different individuals in different ways. We discuss this in the next sections.

SOCIAL INTELLIGENCE

Social intelligence implies more than mere interaction with other agents plus intelligence. For example, agents might apply their intelligence to trading with other agents, without any intelligence being applied to the

Modeling Socially Intelligent Agents 685

process of relating to the others. Such an intelligence might be without the

means of recognizing and referring to other agents as individuals (or

groups). In such a case, any models it has of its social environment would be

entirely generic, so it could not form any individual social relationship with

another agent (different from its relation with any other agent it interacts

with). Such a lack of social intelligence has advantages, such as the ability to

analyze and predict their behavior in computational communities. However, if we are to model many key behaviors in organizations, we need a greater social sophistication.

In denoting the presence of a social intelligence, one would expect to see at least some of the following :

• a relative sophistication of communicative mechanisms (both in the gener­ation and interpretation of messages);

• the ability to represent aspects of other agents (individually or grouped), in order to anticipate their actions (though this need not involve the explicit representation of the other's beliefs, goals, etc.) ;

• the ability to distinguish between and refer to different agents, such that different aspects may be captured for each one (or each group), e.g., in modeling their individual reliability as an information source;

• the ability to direct messages locally to specific individuals (or groups of individuals) ;

• the presence of purely communicative (social) subgoals (or even top goals);

• the possibility for the evolution of the communicative "language"; • a sophistication of communicative forms.

The model described herein goes a little way to address all but the last two points. A slight extension of the model is currently being investigated that introduces a quoting operator to allow the evolution of more sophisti­cated message forms.

SOCIAL INTELLIGENCE AND COMPLEXITY

In addition to the aspects of social intelligence listed above, I think there is another aspect to social intelligence, namely, that of coping with the over­whelming complexity that social systems can (and do) produce. This is a complexity that seems to grow exponentially with the size of the society (Carneiro, 1987). In fact, it seems to be a hallmark of social systems that such complexity arises due to the variety of individual specializations and hence relationships that can develop. A society consisting only of homoge­neous agents only equipped with global communication mechanisms will not have the same characteristics.

686 B. Edmonds

Luhman has argued that one of our social institutions' primary functions is to filter out the complexity of the external world.4 This perspective high­lights some other important aspects of social intelligence, including

• the intelligent but restrictive selection of information sources; • the development of rules to structure social interaction either formally or

informally (e.g., emergent social norms, or formal procedure); • the development of binding long-term relationships (contracts, friendships,

etc.).

A socially intelligent agent may thus seek to use institutions that help it deal with the complexity of social reality. The institution may do this by performing considerable selection and modeling for the individual. If an appropriate institution does not exist (or is inaccessible), the agent may seek

to construct one with other agents. The institutions may also regulate the social structures within themselves by various means such as rules, pro­cedures, and sanctions. In this way, institutions can have a role not only in effectively simplifying the external reality but also in structuring and hence simplifying the social relationships that are internal to it. Such an institution may embed itself within a further institution for similar reasons, resulting in a hierarchical structure. Membership of different institutions covering differ­ent aspects of life may result in a parallel matrix structure. Such an emer­gence of social structure is thus evidence of social intelligence.

The model described below starts to address some of these concerns, in that one can start to see the elements of selection of information and social contracts emerging, as well as the overwhelming complexity inherent in the agents' "society."

EXTENDED "EL F AROL BAR" MODEL

Description of the Original Model

I have extended Arthur's (1994) El Farol Bar model to include model­based learning and communication. In the original problem, a fixed popu­lation of agents has to decide whether to go to El Farol's Bar each Thursday night or stay at home. It is generally desirable to go (apparently, they play Irish music on Thursdays) but not if it is too crowded. The bar is too crowded if more than sixty of the one hundred agents decide to go. Each week each agent has to predict if the bar will be crowded or not. If the agent predicts that the bar will be crowded, then it does not go, and if it predicts the bar will not be crowded, then it does go. The problem is set up so that if most of the agents share the same model, then this model will be self­defeating, for if most predict that the bar will be crowded, they do not go and so it will not be crowded, and vice versa.

Modeling Socially Intelligent Agents 687

Arthur modeled this by dealing each agent a fixed menu of models ran­domly allocated from a limited number of types, for example "the same as

last week," sixty minus the number who went two weeks ago," "the average

over the last four weeks," or "the number predicted by the trend over the

last three weeks." Then each week each agent evaluates all its models against

the past record of how many went and finds the best predictive model. It then uses this model to predict the number who will go this week and bases its decision on this. All the agents do this simultaneously and in parallel.

The resulting number who go each week seems to oscillate stochastically about the critical 60 percent mark (Figure 2), despite the fact that this model, once initialized, is strictly deterministic. Each agent's repertoire of models is fixed, and only the current assessment of the models changes from week to week, so that at different times, different models will be more suc­cessful and hence chosen as a basis for action. The only communication that takes place is the implicit message inherent in the number who go each week.

One of the interesting things in this model is that although each agent is dealt a different menu of models, and all decide to go or not in different combinations and at different times, at the level of the whole model, they are pretty indistinguishable in terms of their behavior.

Extending the Model

I have extended this model in several ways. Firstly, I have added a rudi­mentary social structure. A randomized "acquaintance" structure is imposed

0100 0

0 90 :J 80 ~ c 70 Cll :s: 0 60 .r. :s: 50 OJ

..0 40 E :::l z 30

20

10

0

0 10 20 30 40 50 6o 70 ao 90 100 lime (in not1onal weeks)

FIGURE 2. Number of people going to El Farol's during a representative run of Arthur's Model.

688 B. Edmonds

talk: [greaterThan [trendOverlast [2]] [divide [5] [3]]]

action: [OR [saidBy ['barGoer-3']] [ISaid]]

FIGURE 3. Simple example model.

upon the agents, limiting who they can talk to. Second, agents have a chance to communicate with acquaintances before making their decisions. Thirdly, agents are equipped with a very flexible learning mechanism based on the G P paradigm.

The agent modeling approach follows that described above and has the same basic architecture as in Figure 1. Each agent has a population of mental models, which broadly corresponds to alternative models of its world. These models are each composed of a pair of expressions : one to determine the action (whether to go or not) and a second to determine their communication with other agents. Either action or communication can be dependent upon communications received, which includes the identity of the agent the communications were received from. These internal models are initially randomly generated to a given depth. Subsequently, they are devel­oped in a slow evolutionary manner based either on the past accuracy of the models predictions or some measure of what its past success at gaining utility might be. Although the beliefs and goals of other named agents are not explicitly represented, they emerge implicitly in the agents' models.

Each notional week, a new population of models is produced by a GP algorithm. In this model we are using some tree crossover but with a high degree of propagation and a few random genes introduced each week. The models are evaluated according to a fitness function. The fitness function is measured by the utility the agents would have gained by using each model in the past. The best model is then selected and used to determine first its communicative action and, subsequently, whether to go to El Farol's or not.

The evolution of mental models based upon a GP mechanism is only a rough representation of learning. Again I stress that this is not supposed to implement an efficient learning mechanism, but one that deliberately exhibits such qualities such as "lock-in" and path dependency. The cross over operation is not very realistic for this purpose but serves as a first approximation; for a critique of crossover for these purposes, see Edmonds and Moss (1997).

Each model is composed of two parts: one determines what it says and the other whether it will go to El Farol's. These parts are expressions in a strongly typed language that is specified by the programmer at the start. 5 A simple but real example model is shown in Figure 3.

Modeling Socially Intelligent Agents 689

Translated, this example means that it will tell its "friends" that it will go to El Farol's if the trend observed over the previous two weeks predicts a number greater than five-third (the total population was five in this example); but it will actually go if it said it would go or if barGoer-3 said it

will go. The possible nodes and terminals of the action and talk models are indi­

cated in Figure 4. The computational effect of some of these (when genes are being

evaluated) is illustrated below via some examples :

[lessThan[trendOverLast [5]] [divide [times [maxPopula­tion] [3]] [5]] - the number predicted by the trend over the last five weeks is less than three-fifths of the total population;

[greaterThan [randomintegerUpTo [5]] wentLag [2] ] - a random number chosen between one and five is less than the attendance two weeks ago;

[minus [previous [IPredictedLastWeek]] [wentLastTime] ]­the error in a previous prediction;

[OR [AND [saidBy ['barGoer-1']] [NOT [saidBy ['barGoer-4' ]]] ] [AND [NOT [ saidBy [ 'barGoer-1' ]] ] saidBy [ 'barGoer-4' ] ]]] - barGoer-1 and barGoer-4 say something different;

[OR [NOT [IWentLastWeek]] [ISaidYesterday]] - I did not go last week or I said I would go.

As you can see, this language allows a great variety of possible models, including arithmetic, logic, stochastic elements, models based on what other agents have said, on what the agent itself did, or said previously, and mix­tures of all of these. Explicit names for all agents are included in the lan­guage as well as some useful arithmetic and logical constants.

The friendship structure is chosen at random (within set limits on the

possible nodes for talk gene: greaterThan less Than previous times plus minus divide averageOverlast boundedByPopulation wentlag trendOverlast randomlntegerUpTo

possible terminals for talk gene: wentlastTime maxPopulation IPredictedlastWeek 1 2 3 4 5

possible nodes for action gene: AND OR NOT saidBy

possible terminals for action gene: random Decision ISaidYesterday IWentLastWeek T F 'barGoer-1' 'barGoer-2' 'barGoer-3' 'barGoer-4' 'barGoer-5'

FIGURE 4. Possible nodes and terminals of the tree-structured genes.

690 B. Edmonds

FIGURE 5. Imposed friendship structure.

minimum and maximum number of friends each can have) at the beginning and, in this case, is as shown in Figure 5. Agents are constrained so that they only listen to what their friends are telling them. It also has an effect upon the utility they gain.

The amount of utility each agent gains is determined by its action and what the others do. The utility that agents get is 0.4 if they go when it is too crowded, 0.5 if they stay at home, and 0.6 if they go when it is not too crowded (where "too crowded" means greater than 60 percent of the total population).

This means that each agent is developing its models of what the other agents are going to do, in a competitive way. A successful agent learns some way of deciding to go when others do not and avoiding going when many others.

This model can be seen as an extension of the modeling work of Akiyama and Kaneko (1996), which examines three player games with a similar structure to this model. Their model is, however, simpler, for it involves no explicit communication or potential for modeling of other agents and so only touches on the social elements.

Analysis of the Results

I will look at the results from just one run of the model with five agents {'barGoer-1,' etc.) over a hundred iterations (notional weeks). The results here are typical of other runs; further, the model was reasonably robust to minor changes in the setup of the model (e.g., the exact terms available in the internal modeling language). I do this because the interesting behavior of the model is not easy to distinguish when statistically aggregated. At first sight, the behavior seems to be effectively stochastic with what appears to be rapid "flipping" of behavior. This is similar to the behavior in the original model. In fact, the behavior in Arthur's model is not acting as if it were stochastic, as some have claimed, but seems to be a version of globally coupled chaos (as described in Kaneko (1990)). It is thus likely that the extension of the model here described is not stochastic (even in effect) either. See Edmonds (in press) for details.

Modeling Socially Intelligent Agents 691

A result of the highly dynamic behavior that the model exhibits is that it has to be smoothed if the tendencies are to be apparent. For this reason, all attendances, truthfulness, and utility shown in this section are smoothed (usually over ten time periods).

The aggregated pattern for the number of agents attending each week (Figure 6) is similar to that of the original El Farol model (Figure 2). In particular, notice that it does not settle down to any particular regular pattern. This is unsurprising given the nature of the set up, where agents are competitively trying to out-model each other. In Figure 7, we see the utilities that the agents managed to gain. As one would expect, the utilities that the agents gain roughly converge but with continuing fluctuations as the agents try to exploit new behaviors.

When one analyzes the attendance in terms of who goes and who does not (and it is sufficiently smoothed), it can be seen that some of the agents

0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 N N M M ~ ~ ~ ~ ~ ~ ~ ~ 00 00 m m 0

Time (in notional weeks) ,.....

FIGURE 6. Number a ttending each week.

C2 0.55 t

~ ~ 0.53 ::l Ol ~oJ

"0 >­Q).o

-5 " 0.51 I 0 Q) 0 c E -(/) <1J ~ Ol 0.49 -

• barGoer-1 • • • barGoer-2

barGoer-3 --barGoer-4 --barGoer-5

t ~ , . - , ,

0.47 ' , ~ ---. ------ ~

0.45 11111 1111111111111 11 1111 11 I II II: 111 111111111 1111111111111 111111111111 111111111111111111111111 111111111 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0

~ N N M M ~ ~ ~ ~ ~ ~ ~ ~ 00 00 m m 0

Time (in notional weeksr

FIGURE 7. (Smoothed) utility gained by agents.

692 B. Edmonds

settle for a fixed strategy but some are more dynamic and constantly swap between different strategies (Figure 8).

In general, one can see that the agents differentiate into those who gener­ally go and those who generally do not. While barGoer-1 and barGoer-3 seem to settle for a policy of general attendance, barGoer-4 does the opposite.

The agents do not necessarily match what they say and what they do (as in the example shown in Figure 3), but learn what is the best strategy in this regard, both for themselves and when interpreting the utterances of other agents. Figure 9 is a graph showing the extent to which agents tell the truth. If agents do what they say they are going to do, this is shown as a value + 1.

~100% Q)

g' 90%

~ 80% (,) .... ~ 70%

...... Q) 60% (,)

@50% '0

2 40% ;; 30%

20%

10%

0%

'

' ' .. .. ..... -..... ' , -............ ,.

;

"' ~ , ..

o~~~N~~~~~o~~~N~~~~~o~~~N~~~~~o~~~ ~~~NNN~~~~~~~~~~~~~~~~~~~~~~~~

Time (in notional weeks)

FIGURE 8. (Smoothed) attendance of agents.

~

(/) "5 c t Q) 'Qi Ol 0 <1l Q)

0 '0

~ (/) 0 (/) 0 Q) 0 c ..--"5 II

::c ..--0

~ "5 ::c

~-'0 5 Q) ~ ~ 0 0

0 0 0 E ~ .e II ..--

-1

,.,#h=,..,."'~ ... ,~...,,

''"","',

' / ' barGoer-1 ' ' ---barGoer-2 • barGoer-3 , ..

--barGoer-4 I ' --barGoer-5 '

} I

/ f

f

~ -- - - ------o~~m~~oo~~~o~~mN~ro~v~o~~mN~ro~v~oMmm

~~~NNNMMMMVV~~~~~~~ID~~~rororommmm

Time (in notional weeks)

FIGURE 9. (Smoothed) truthfulness of agents.

Modeling Socially Intelligent Agents 693

If agents do the opposite of what they say, this is shown as a value of -1. I then smooth the values over time in order to make the tendencies of the agents apparent.

We can see (Figure 9) that, again, in terms of truthfulness, the agents have differentiated themselves in terms of strategy, with some learning to

always lie and others varying their strategy and only sometimes lying.

Notice that the developing clustering by attendance is different from the clustering here. To make this double differentiation easier to see, I have plotted the trajectories of the agents in a space of truthfulness against atten­dance (Figure 10). Here I have not smoothed the values quite so much, since the agents clearly differentiate in this space and hence are easier to see.

There is clearly no tendency for the agents to converge here. Thus we have a model that appears to exhibit a clear tendency for the agents to differentiate from each other. In order to check this, I will now go into more detail of what is happening in the model, based on a case study focusing on the last few iterations.

Case Study from the Results

The best (and hence active) genes of each agent at the last date are summarized in Figure 11. I have simplified each so as to indicate is logical effect only. The actual genes contain much logically redundant material. This is typical of GP algorithms (as well as real genetics).

CJJ"S c:~ Q) 'ij) O>o ro QJ

- '0 0 0 (J)O' CJJO QJO c::~

"5 II :E ~ 0 2:2 - .r:::. 'O'S Q)-= .r:::.~ 0~ Oo E ~

.f!!.. II

0%

barGoer-1 - - - barGoer-2

barGoer-3 --barGoer-4 -- barGoer-5

I

'

I

'

/

20% 40% 60% 80% Attendance (percentage)

FIGURE 10. Trajectories of agents (circle is week I; square is week 100).

100%

694 B. Edmonds

talk-1: averageOverlast(numWentlast) > previous(trendOverlast(numWentlast))

action-1 : wentlastTime

talk-2: trendOverlast(numWentlast) - 2 * numWentlag(2) > numWentlag(numWentlast)

action-2: NOT lsaid

talk-3: randomNumberUpTo(8) < 8/3

action-3: True

talk-4: average0verlast(4)/average0verlast(5) < numWentlag(15} action-4: (lsaid AND randomDecision) OR (saidBy 2)

talk-5: trand0verlast(20) < numWentlag(2) - averageOverlast(numWentlast)

action-S: randomDecision OR (saidBy 4) OR (saidBy 2)

FIGURE II. (Simplified) best talk and action genes at date 100.

The "junk" code is not useless, as it may put in an appearance in later populations due to the activity of crossover in producing later variations.

The effect of the genes is tricky to analyze even in its simplified form. For example, ba rGoe r -1 will tell others it will go to El Farol's if the average attendance over a previous number of time periods equal to the number who went last time is greater than the predicted number indicated by the trend estimated over the same number of time periods but evaluated as from the previous week! However, its rule for whether it goes is simpler- it goes if it went last week.

You can see that for only one agent does what it says indicate what it does in a positive way (barGoer-4) and that one will do the exactly the opposite of what it says (barGoer-2). It may seem that barGoer-1 and ba rGoe r- 3 are both static, but this is not so because Figure 11 only shows the fittest genes for each agent at the moment in terms of the utility they would have gained in previous weeks. During the next week, another gene may be selected as the best.

The interactions between the agents at date 100 are summarized in Figure 12, which shows the five agents as numbered circles. It has simple arrows to indicate a positive influence (i.e., if barGoer-2 says it is going, this makes it more likely that barGoer-4 would go) and crossed arrows for negative influences (e.g., if barGoer-2 says it will go, this makes it less likely it will go). The circles with an "R" represent a random input.

By the end of the run described above, barGoer-3 and barGoer-1 had developed a stand-alone repertoire of strategies that largely ignored what other agents said. Ba rGoe r- 3 had settled on a fairly fixed strategy

Mode ling Social/ y Intelligent Agents 695

FIGURE 12. Talk to action causation at week 100.

(but using different best genes at different dates), while barGoer-1 relied on largely deterministic forecasting strategies.

The other three agents had developed what might be called social stra­tegies. For example, barGoer-2 has developed its action gene so as to chaoti­cally change the number of NOTs. By date 100, it had accumulated seven such NO Ts (so that it actually read NOT [NOT [. . . NOT [ I s aid] . . . ] ] ). Its general pattern was to chaotically change the number of embedded NOTs it used, combined with a slight tendency to increase them (as shown in Figure 13). In this way, it appears that it has been able to "fool" barGoer -4 by sometimes lying and sometimes not. barGoer-4 has come to rely (at least sometimes) on what barGoer-2 says, and likewise, barGoer-5 uses what both what barGoer-2 and barGoer-4 say (although both mix this with other methods, including a degree of randomness).

8 CJl

t=- 7 0 z 0 6 0

..c g.5 0

4

3

2 0

0 0 0 00 0 0 0 0 0 0

0 -KIDo--- - - -10---<:>-- --t---(:>--- -+------t-----a::H

0 20 40 60 80 100 Time (in notional weeks)

FIGURE 13. Depth of negation embedding in barGoer-2 ' s action gene (line is the smoothed trend).

696 B. Edmonds

Thus although all agents were indistinguishable at the start of the run in terms of their resources and computational structure, they evolved not only different models but also very distinct strategies and roles. They certainly do not all converge to the game-theoretic mixed-strategy mentioned above (but a few do). Thus, allowing social aspects to emerge has resulted in a clear difference in the behavior of the model compared to Arthur's original, where although each was dealt a different menu of models, agents did not act in a qualitatively differentiable way.

This reinforces the results of Akiyama and Kaneko (1996), who find in a much simpler three-player model that the ability to distinguish between the other two players allows behavior of greater complexity to emerge.

Issues and Interpretations

There are some very interesting problems that arise when we try to inter­pret what occurred. Even given that we can look inside the agents' "heads," one comes across some of the same problems that philosophers, psychol­ogists, and social scientists encounter in trying to account for human com­munication. The web of cause and effect can be so complex as to impede a "brute force" analysis of the complete pattern of cause and effect-just as seems to occur in the human case.

One issue in particular is the question of the "meaning" of the agents' utterances to each other. Their utterances do have a "meaning" for each other, otherwise they would quickly select out action genes that included "saidBy" clauses. However, these meanings are not at all obvious to us. One cannot assume that a message of "yes" means "going to the bar" or even its opposite, "I am not going." There is nothing built into the agent's cognition to force a particular interpretation (expect maybe a slight bias to those corresponding to simpler interpretation trees). The generation of such messages by each agent is not determined solely by its own model structures. In fact, most of the computation that results in them is distributed across several agents. Thus it may be more sensible to attribute the utterance to their role in a language game, whose ultimate grounding is to the practice of such communication in relation to actual decisions. The language game is the attempt to get the others to go and come when it is useful for each agent, and the ultimate meaning of these messages is grounded in their (current) usage in terms of what each agent tends to say before it takes each action. For this reason, it seems that an approach that follows (Wittgenstein, 1953) the picture of language in terms of language-games rooted in its practice, and describing the state of affairs in terms of "speech acts" (Searle, 1969) may make for a simpler and more appropriate model of the situation than a traditional AI belief and inference model. If one attempted to account for cause and effect in terms of the beliefs of a single agent in terms of its

Modeling Socially Intelligent Agents

Week 98 Week 99

I I I

'0 I

I I

0 __ / I I

0 __ / Talk Action Talk Action

I

0 I

I I

FIGURE 14. Analysis of specific causation over the last three weeks.

Talk

Week 100

'\

\

I I

Action

697

environment, one would have to read a great deal into each agent's model as implicitly representing a very complex web of causation. To illustrate the complexity of this causal structure, Figure 14 shows the specific (as in spe­cific agent to specific agent) causation in the last three weeks of the model. Even this does not include much of the complexity, as it omits the causation via the input to the talk genes in terms of past attendance statistics (numWent, numWentLag [2], a v erageOverLast, etc .... ) as well as the causation implicit in how each population of genes evolves due the evalu­ations of model fitness.

Thus it seems that the pragmatics of this situation are the most impor­tant for determining meaning, followed by a semantics grounded in the effects of their actions, leaving the syntax to merely distinguish between the two possible messages. This case illustrates Peter Gardenfors observation about human language:

Action is primary, pragmatics consists of the rules for linguistic actions, semantics is conventionalised pragmatics and syntax adds markers to help disambiguation (when context does not suffice). [Gardenfors, 1997]

CONCLUSION

I have suggested that to model social behavior, one should include the ability to distinguish, identify, model, and address other agents individually in a flexible manner, because a social intelligence needs to be grounded in these abilities.

698 B. Edmonds

I have exhibited an extension the El Farol model, where some of these elements are included. In this model we see the evolution of a society where the intelligence seems to reside, at least somewhat, beyond each agent indi­vidually in the web of their interactions-the society of agents. This seems to be due to the co-evolution of the structure together. Such a case illustrates the potential difference between modeling truly social agents, where the agents are truly "bound up" with their society and modeling their cognition individually and then placing them together in a situation where they then interact.

When such social behavior does occur, we may well find ourselves with many of the same difficulties that other social scientists have, namely, the tracing of very complex chains of causation if one works in detail and the problem of the meaning and use of our descriptive terms if we attempt a macroscopic approach.

NOTES l. See, for example, the debate at Chicago Philosophy Project (http://csmaclab-www.uchicago.edu/

philosophy Project/philos.html). 2. It also approaches the suggestions made by Dennett (1995).

3. For more on SDML, see the URL at http:jjwww.cpm.mmu.ac.uk/sdml. 4. This aspect of Luhman's thought is summarized in English by Bednarz (1984). 5. This typing means that this strictly corresponds to a strongly typed genetic programming algorithm

(Montana, 1995).

REFERENCES Akiyama, E., and K. Kaneko. 1996. Evolution of cooperation, differentiation, complexity, and diversity in

an iterated three-person game. Artificial Life 2:293- 304. Arifovic, J. 1994. Genetic algorithm learning and the cobweb model. Journal of Economic Dynamics and

Control18 :3- 28. Arthur, B. 1994. Inductive reasoning and bounded rationality. American Economic Association Papers

84 :406- 411. Bednarz, J. 1984. Complexity and intersubjectivity. Human Studies 7:55-70. Bonasso, R. P., D. Kortenkamp, D. P. Miller, and M. Slack. 1995. Experience with an architecture for

intelligent reactive agents. Lecture Notes in Artificial Intelligence 1037:187- 202. Campbell, D. T. 1960. Blind variation and selective retention in creative thought as in other knowledge

processes. Psychological Review 67:380- 400. Carneiro, R. L. 1987. The evolution of complexity in human societies and its mathematical expression.

International Journal of Comparative Sociology 28 :111- 128. Cooper, R., J. Fox, J. Farringdom, and T. Shallice. 1997. A systematic methodology for cognitive model­

ing. Artificial Intelligence 85:3- 44. Dennett, D. C. 1995. Consciousness explained. Philosophy and Phenomenological Research 53:889- 892. Earley, P. C., and J. Brittain. 1992. Cross-level analysis of organizations- social resource-management

model. Research in Organizational Behaviour 15:357- 408. Edmonds, B., and S. J. Moss. 1997. Modeling bounded rationality using evolutionary techniques. Lecture

Notes in Computer Science 1305 :31- 42. Edmonds, B., S. J. Moss, and S. Wallis. 1996. Logic, reasoning and a programming language for simulat­

ing economic and business processes with artificial intelligent agents. In Artificial Intelligence in Economics and Management, ed. P. Ein-Dor, 221- 230. Boston, MA : Kluwer Academic.

Modeling Socially Intelligent Agents 699

Edmonds, B. In press. Modelling bounded rationality in agent-based simulations using the evolution of mental models. In Computational Techniques for Modelling Learning in Economics, ed. T. Brenner. Boston, MA: Kluwer Academic.

Gardenfors, P. 1997. The pragmatic role of modality in natural language. Presented at 20th Wittgenstein

Symposium, Kirchberg am Weshel, Lower Austria.

Holland, J. H. 1992. Adaptation in natural and artificial systems, 2d ed. Cambridge, Mass.: MIT Press.

Johnson-Laird, P. N. 1983. Mental models. Cambridge University Press. Kaneko, K. 1990. Globally coupled chaos violates the law of large numbers but not the central limit

theorem. Phys. Rev. Lett. 65:1391- 1394. Koza, J. R. 1992. Genetic Programming: On the programming of computers by means of natural selection.

Cambridge, MA: MIT Press. Latane, B., A. Nowak, and J. H. Liu. 1994. Measuring emergent social phenomena-dynamism, polariza­

tion and clustering as order parameters of social systems. Behavioral Science 39:1- 24. Montana, D. J. 1995. Strongly typed genetic programming. Evolutionary Computation 3:199-230. Moss, S. J., and B. Edmonds. 1998. Modeling economic learning as modeling. Systems and Cybernetics

29:5- 37. Moss, S. J. , B. Edmonds, and S. Wallis. 1997. Validation and verification of computational models with

multiple cognitive agents. Centre for Policy Modeling Report 97-25. Manchester Metropolitan Uni­versity, Manchester, UK. (http://www.cpm.mmu.ac.uk/cpmrep25.html)

Moss, S. J., H., Gaylard, S. Wallis, and B. Edmonds. 1998. SDML : A multi-agent language for organiz­ational modeling. Computational and Mathematical Organization Theory 4 :43- 69.

Palmer, R. G., et al. 1994. Artificial economic life- A simple model of a stockmarket. Physica D 75:264-274.

Penrose, E. T. 1959. The theory of the growth of the firm. Oxford: Blackwells. Searle, J. R. 1969. Speech acts: An essay in the philosophy of language. London: Cambridge University

Press. Simon, H. 1957. Administrative behavior: A study of decision-making processes in administrative organiz­

ation. New York: Macmillan. Simon, H. A. 1972. Theories of bounded rationality. In Decision and organization, eds. C. B. McGuire and

R. Radner. Amsterdam: North-Holland. Wittgenstein, L. 1953. Philosophical investigations. Oxford: Blackwells.