ResearchFrontier

Introduction and Background

One of the most fascinating andpromising research areas in eco-nomics is the recent integration of

the following three fields of economics:experimental economics, computational economicsand neuroeconomics (Figure 1). Due to theirmethodologically interdisciplinary nature,the development of each of the threeshould interest computer scientists andengineering people [1]. The relationshipamong experimental economics, agent-based computational economics and neu-roeconomics is, in essence, a relationshipbetween human agents and software agents.Software agents can be regarded as theeffective abstraction or model of humanagents which we learn from experimentaleconomics or neural economics.

Experimental Economics and Agent-Based Computational Economics

Experiments withHuman SubjectsAmong the three, experimental eco-nomics is the oldest one and has a sixty-year history. The first paper waspublished in 1948 by Edward Cham-berlin [2], and his student VernonSmith, a Nobel Laureate in 2002, con-tinued the laboratory studies withhuman subjects in the 1960s. Theadvantage of experimental economics isthat it allows us to observe humanbehavior in a highly controlled envi-ronment so that we can give the eco-

nomic theory a sharper test,i.e., a test based on less-pollut-ed or less noisy data.

However, the disadvantageis that using human agents canbe quite costly, considering thepecuniary incentive paid to thehuman agents and the physicalspace required to accommodatetheir bodies. Therefore, scaling-upbecomes a severe constraint to experi-mental economics, and it is hard toconduct experiments with a largenumber of agents for many iterations.In other words, size is not really a con-trol variable in experiments. Hence,one may question to what extent theresults obtained from small experimentscan be extended or generalized.

Simulationswith Software AgentsGiven the constraint above,it is preferable if the experi-

ments with human subjectscan be replaced by simulations

with software agents, which isthe essence of agent-basedcomputational economics

(ACE). The idea of agent-based modeling in economics also has along history. Thomas Schelling, aNobel Laureate in 2005, is well knownfor his work on the segregation model(or spatial proximity model), whichappeared in the late 1960s [3], [4].However, the term “agent-based com-putational economics” (ACE) did notexist in economics until Leigh

Software-Agent Designs in Economics: An Interdisciplinary Framework

Shu-Heng ChenNational Chengchi University, TAIWAN

FIGURE 1 Economic software agents in an interdisciplinary framework.

Agent-BasedComputational

EconomicsExperimentalEconomics

Cognitive Capacity(IQ)

Software Agents

Personality(Big 5)

ComputationalIntelligence

Culture

Neuroeconomics

© EYEWIRE

18 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | NOVEMBER 2008 1556-603X/08/$25.00©2008IEEE

Digital Object Identifier 10.1109/MCI.2008.929844

Tesfatsion in the mid-1990s formally gaveand defined such a name with the estab-lishment of the website, http://www.econ.iastate.edu/tesfatsi/ace.htm.

By using software agents, we can easi-ly conduct market experiments (simula-tions) with 100,000 agents. Thescaling-up problem is less severe. Never-theless, there is another issue; namely,how these agents can be programmed sothat they can reasonably replace or repre-sent their human counterparts. There isno definite answer to this question; thehuman’s decision-making process islargely a black box. For example, in anauction experiment, we can certainlyobserve how agents bid or ask, but itwould be hard to infer their bidding orasking strategies from their behavior.Hence, the direct programming of humanagents becomes difficult if not impossible.

Turing Test: Instead, what mostACE economists engage in is indirectprogramming. The early ACE studies areclearly motivated by using softwareagents to mimic the behavior of humanagents observed in the laboratory. Thefamous Turing test serves as the bestillustration. [5] points out that thedevelopment of social science theoriescan be likened to the task of building acomputer to mimic human behavior.Thus, a social science theory can bedeemed to be successful when it is nolonger possible for a computer judge totell the difference between behaviorgenerated by humans and that generatedby a machine.

Genetic Algorithms and Soft-ware Agents in Market Experi-ments: In this regard, the two CItools, namely, genetic algorithms (GAs)and genetic programming (GP) are fre-quently used to build software agentssuch that their collective behavior canmirror the laboratories with human sub-jects. [6] applied GAs to study marketdynamics. The results of the simulationsshow that software agents driven by GAcan capture several features of theexperimental behavior of human sub-jects better than other statistical oreconometric learning algorithms.

Genetic Prorgamming and Soft-ware Agents in Market Experi-

ments: In addition to genetic algo-rithms, genetic programming is alsoextensively applied to build systems ofsoftware agents which are able to repli-cate the laboratory results with humansubjects. [7] studied bargaining behaviorobserved in the double-auction labora-tory markets with human subjects. Allbuyers and sellers in [7] are artificial adap-tive agents, with each artificial adaptiveagent being built upon genetic program-ming. The architecture of genetic pro-gramming used is what is known asmulti-population genetic programming(MGP). Briefly, they viewed or mod-eled an agent as a population of bargainingstrategies. Genetic programming wasthen applied to evolving each populationof bargaining strategies. This architec-ture is shown in Figure 2.

Figure 3 demonstrates a typicalresult observed in this agent-based dou-ble auction market. The left panel ofthe figure is the simulated market envi-ronment defined by the demand andsupply schedule, whereas the rightpanel of the figure gives the pricedynamics resulting from the bargainingbehavior generated by MGP agents. Aswe can see from Figure 3, marketprices in this case quickly move towardthe equilibrium price (or price interval),and then slightly fluctuate around there.This result is basically consistent withwhat we learned from experimentaleconomics with human subjects [8].

Cognition, Personality and CultureOne of the recent developments in

experimental economics involves plac-ing more emphasis on behavioral exper-iments which take into account the roleof cognitive capability, personality traitsand culture. Given the already estab-lished connection between ACE andEE (Section 2) and the relationshipbetween human agents and softwareagents, it is expected that one of thenext steps in ACE is to design econom-ic software agents with these interdisci-plinary backgrounds, from psychologyto anthropology (Figure 1).

Software Agents with Different “IQ”Human agents are heterogeneous inintelligence or cognitive ability. Currentbehavioral genetics enhance our under-standing of the heterogeneity in humancognitive ability [9]. This simple fact hasalready caught the attention of experi-mental economists. For example, theRaven intelligence test [10] has beenadopted by experimental economists aspart of their experimental designs.

However, from a pure engineeringviewpoint, there would be a tendencyto make software agents as smart as pos-sible, and usually equally smart. Thisdesign principle, therefore, obviouslycontradicts our understanding of humanagents. Fortunately, these biases can beadjusted. Taxonomies of CI tools basedon the degree of cognitive constraintsare possible, while not perfect [11], [12].Therefore, a market composed of agentswith heterogeneity in intelligence canbe considered to be a market com-posed of some “less smart” agents,

FIGURE 2 Double auction market.

Software AgentCl, GP, ....

Buyer 1

Buyer 2

Buyer 3

Buyer N

Seller 1

Seller 2

Seller 3

Seller N

Software AgentCl, GP, ....

Software AgentCl, GP, ....

Software AgentCl, GP, ....

Software AgentCl, GP, ....

Software AgentCl, GP, ....

Software AgentCl, GP, ....

Software AgentCl, GP, ....

DoubleAuctionMarket

NOVEMBER 2008 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 19

20 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | NOVEMBER 2008

whose adaptive behavior is driven byreinforcement learning, and some“smart” agents, whose adaptive behav-ior is driven by evolutionary computa-tion. With this mixture, a few CI toolscan interact with each other in the sameeconomic environment. [13] is probablythe first study of this kind.

Financial Agents with Heteroge-neous Intelligence: [13] considers threedifferent types of agents, namely, themomentum traders, empirical Bayesiantraders and k-nearest-neighbor traders,each of which represents an opportunity-seeking trader with different degrees ofsmartness. The efficient market hypothe-sis implies that there are no profitablestrategies, and hence intelligence, regard-less of its formalism, does not matter. As aresult, the three types of traders shouldbehave equally well, at least in the longrun. However, when the market is notefficient, and intelligence may matter, it isexpected that smarter agents can takeadvantage of dumber agents. In their sim-ulations, [13] found that momentumtraders, who never learn, performedworst during the transition period whenthe market is not efficient.

In addition to using different CI toolsto characterize heterogeneity in intelli-gence, it is also possible to use differentparameters of the same CI tool to distin-guish different degrees of smartness. Inthis case, instead of the interaction ofseveral different CI tools, what weobserve is the interaction of the same CItools, but which differ owing to differ-ent parameters.

Smarter Agents in Auctions: Ina study of the agent-based double auc-tion market, [14] examine how theco-evolution of agents’ strategies willchange with the agents’ level ofsmartness, and the associated micro-and-macro correspondence. Theiragent-based double auction market issimulated with genetic programming(GP). They vary their GP traders so asto have different population sizes. Asmaller population size assumes alower degree of smartness, whereas alarger population size implies a higherdegree of smartness. It is found thatthe level and the composition of theintelligence of traders can impact therealized social welfare, a measure ofmarket efficiency.

Agents with PersonalityPersonality refers to the sets of pre-dictable behaviors by which humansare recognized and identified. In psy-chology, cognitive ability and personalitytraits are distinguished, while they arenot independent of each other andtheir distinctions are not easy. Person-ality psychologists have developedmeasurement systems for personalitytraits which economists have begun touse. Among these, the most prominentis the “Big Five.” Using the Big FiveModel, labor economists have alreadystarted to explore the relevance of per-sonality to economics [15].

Emotional Agents: Various archi-tectures of software agents with person-ality have been proposed. The

development of the emotional agent isprobably the most evident one. In late1990, emotions started to draw econo-mists’ attention [16], [17]. They havebeen further incorporated into experi-mental economics as evidence thatemotions play an important role in realdecision making [18].

Agent-Based ComputationalLottery Markets: However, in agent-based computational economics,autonomous agents have rarely beenclothed with personality traits. In anagent-based modeling of lottery mar-kets, [19] makes their lottery partici-pants able to feature a different degreeof aversion to regret. Basically, when themass media intensively reports the win-ners with their gigantic prizes, it maymake those people who did not gamblefeel regret. People with this personalitytrait would have different decisions onlottery participation. Since personalitytraits are more malleable than cognitiveability over the life cycle, [19] goes onestep further to make the parameter ofaversion to regret endogenously deter-mined through genetic algorithms.With this design, not only can personal-ity traits affect human economic behav-ior, but the market forces can generatefeedback to affect human personalitytraits as well.

Financial Agents with DifferentRisk Attitudes: Personality traits arelikely to prove useful in economic mod-els of decision making under uncertain-ty. People more open to experience mayhave different observed risk attitudes

FIGURE 3 Agent-based double auction market simulation with MGP agents.

2,2002,1002,0001,9001,8001,7001,6001,5001,4001,3001,2001,1001,000

900800700600500400300200100

0 21 3 4 5 6 7 8 9 10 11 12 13 14 15 16

2,050

1,550

1,050

550

501 501 1,001 1,501 2,001 2,501

NOVEMBER 2008 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 21

than other people. While economistsclearly acknowledge the effect of riskattitude on investors’ decision-making,few have ever studied the significance ofrisk attitude from a survival viewpoint.In an agent-based artificial stock market,[20] simulated the asset markets com-posed of investors with different riskattitudes using genetic algorithms. Theyfound that risk attitude can play an evenmore determining role in the investors’wealth share than forecasting accuracy.

Cultural AgentsWhat has also been neglected in thecurrent design of economic softwareagents is the culture factor. In this area,experimental economics also has some-thing to teach ACE. Using experimen-tal results from ultimatum bargaininggames, [21] conducted a cross-culturalstudy in fifteen small-scale societies inLatin America and Africa. They foundenormous variation in behavior acrosscommunities. Some societies closelyconformed to the game-theoretic pre-diction, while others made extremelygenerous offers. They are able to relatethis difference to interactional patternsof everyday life and the social normsoperating in these various communities.

Software Agents with SocialPreferences: There are many differentways in which cultural factors can bebuilt into software agents. Socialpreference, for example, is a key attribute.An agent’s preference is personal if itdepends only on his own consumptionor received income; otherwise, it issocial. [22] has a simple operationalmodel for agents with social preferences.Consider the following utility functionof player i in a two-person game,

ui(π i, π j)

={

π i − α i(π j − π i) if π i ≤ π j,

π i − β i(π i − π j) if π i ≥ π j,

(1)

where π i and π j are payoffs received byplayer i and j respectively, and0 ≤ β i ≤ α i . The special case ofα i = β i = 0 provides personal prefer-ences, whereas α i ≥ β i > 0 provides the

social preferences. For the latter, playeri still likes high monetary payoffs, but isnow also averse to inequality, beingespecially uneasy with inequality forwhich he receives lower payoffs(π i ≤ π j ). Cultures which supportcooperation or sharing, like Achehunters in Paraguay (see [21]), can beconsidered to be agents with a suffi-ciently large value of β , whereasMachiguenga in Peru are associatedwith a low value of β . Then manyfamiliar games that are studied using CIcan then be extended to agents withsocial preferences so as to further simu-late the possible influence of culture onhuman behavior.

Kuznar [23] provides another illus-tration of cultural agents. Using anthro-pological data on individual men’swealth and political affiliations fromKapauku, a New Guinea tribal village,he tested 24 software agents (decisionalgorithms) from 7 different paradigmsby simulating men’s decisions with the-orized decision rules and examiningwhich rules produce Kapauku-likealliances. The attempt is, therefore,made to match software agents with realhuman agents in Kapauku, and, throughthe exploratory modeling, to shed lighton the cultural characteristics of theKapauku people.

While in ACE there are alreadymany studies comparing different soft-ware algorithms in terms of their abilityto fit outcomes of experiment outcomeswith human subjects, cultural elementshave been generally ignored. This is sobecause most experiments which theACE researchers considered mirroringhave no cultural elements. Therefore,this makes the cultural aspects of thesedifferent algorithms rather implicit. Forexample, reinforcement learning is anindividual learning, which may applywell for societies whose people have lit-tle experience regarding sharing; on theother hand, the genetic algorithm is aform of social learning, which mayapply to the opposite situation. So, forthe same experiments, the softwareagents that are fitting for Machiguengahunters may be inapplicable to Lamelarawhale-hunters [21].

NeuroeconomicsNow we move to the last part ofFigure 1: neuroeconomics. Neuroeco-nomics is young but has been growingvery rapidly. Given the page limitationsof the paper, we do not attempt to givea review or even an introduction to thisarea. The interested reader is referred to[24], [25] and [26]. What we, however,want to address here is the general ideaon how ACE and NE can be related,with a specific focus on software agents.

One common feature shared by theagent models inspired by cognitiveability, personality traits and culture(Section 3) is that they are all concernedwith the mechanisms which generatehuman decisions and behaviors. Thesemechanisms, to a large extent and for along time, have been known as a blackbox. Software-agent designs are purport-ed to make this black box transparent byproposing “act as if ” mechanisms,which, however, may only be artificial.Nonetheless, by integrating knowledgefrom cognitive psychology, personalitypsychology and anthropology withcomputational intelligence tools, wemay hopefully come to a better approx-imation of the underlying mechanisms.

Neuroscience provides the directmeasurement of the activity in this blackbox from the molecular level to themacroscopic level. One ambitious goalof neuroeconomics, while looking quitefar-fetched, is to pass the Turing test. Infact, it can be considered to be a robotequipped with some brain-scanningmachines, such as fMRI, which are con-nected to a human body and can even-tually perform “horrible” mind-readingtasks. This “mind-reading machine”,regardless of whether it will pass theTuring test, can be regarded as a soft-ware agent. This software agent may bemore complex than just algorithms(codes), since some of its operations mayinvolve being embedded in an externalenvironment which facilitates comput-ing with natural materials, such as DNA,quantum, or chemical computing.

There are already a lot of economicbehavioral models that have been stud-ied by neuroeconomists, including theexpected utility computation, prospect

22 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | NOVEMBER 2008

theory, risk-adjusted rewards, reinforce-ment learning, and the balance betweenexploitation and exploration. Thisresearch trend can be generally thoughtof as a mapping from the brain space tothe decision or behavioral space in thesense that a single decision or type ofbehavior can be mapped to a sequenceof temporal and space images of brains.What software-agent designs attempt todo is not very much different exceptthat the mapping is from the decision orbehavior space to a space of algorithms.Hence, ACE, EE and NE are closelyrelated as shown in Figure 1.

Conclusions: The Future of Agents in Economics and the Social SciencesBased on the interdisciplinary frame-work as summarized in Figure 1, weshow how the integration of the threebranches of economics can constitute anew framework for the next step on theeconomic research agenda. Part of thisnew framework is a laboratory com-posed of both human agents and softwareagents and allowing for the interplaybetween the two as illustrated in [27].In this lab, computational intelligencestill plays the role of designing softwareagents, just as it has done for ACEbefore. Nevertheless, the integratingframework may change the nature ofthe design. Even though intelligence orintelligent behavior remains an impor-tant goal of the design, it is not the onlyconsideration. In some cases, agentswith emotional designs or emotionalagents can be just as important as intelli-

gent agents. In other cases, even thoughagents are all intelligently designed, weneed to differentiate them in terms oftheir degree of intelligence as humanagents are heterogeneous in their intelli-gence quotient (IQ). Therefore, thenew framework enriches the design ofagents. It is no longer just narrowlyrestricted to computer science, butmore broadly connected to cognitivepsychology, behavioral genetics, neuralsciences, and social sciences. Theseextended software agents provide uswith better robustness tests of variousinstitutional or market designs [28].

References[1] J. Grossklags, “Experimental economics and experimen-tal computer science: A survey,” Proceedings of the 2007.Workshop on Experimental Computer Science, Article no.11, 2007.[2] E. Chamberlin, “An experimental imperfect market,”Journal of Political Economy, vol. 56, pp. 95–108, 1948.[3] T. Schelling, “Models of segregation,” American EconomicReview, Papers and Proceedings, vol. 59, pp. 488–493, 1969.[4] T. Schelling, “Dynamic models of segregation,” Journal ofMathematical Sociology, vol. 1, pp. 143–186, 1971.[5] J. Arifovic, R. McKelvey, and S. Pevnitskaya, “An initialimplementation of the Turing tournament to learning inrepeated two-person games,” Games and Economic Behavior,vol. 57, no. 1, pp. 93–122, 2006.[6] J. Arifovic, “Genetic algorithm learning and the cob-web model,” Journal of Economic Dynamics and Control,vol. 18, no. 1, pp. 3–28, 1994.[7] S.-H. Chen and C.-C. Tai, “Trading restrictions, pricedynamics, and allocative efficiency in double auction markets:Analysis based on agent-based modeling and simulations,”Advances in Complex Systems, vol. 6, no. 3, pp. 283–302,2003.[8] V. Smith, Bidding and auctioning institutions: Experimentalresults. In: Smith V (ed), Papers in experimental economics.Cambridge University Press, Cambridge, pp. 106–127,1991.[9] G. McClearn, B. Johansson, S. Berg, N. Pedersen, F.Ahern, S. Petrill, and R. Plomin, “Substantial genetic influ-ence on cognitive abilities in twins 80 or more years old,”Science, vol. 276, pp. 1560–1563, 1997.[10] J. Raven, Advanced progressive matrices: Sets I and II. H.K.Lewis, London, 1962.[11] T. Brenner, “Agent learning representation: Advice onmodeling economic learning,” In: Tesfatsion L, Judd K

(eds), Handbook of computational economics: Agent-based compu-tational economics, vol. 2. Elsevier, Oxford, PP. 895–947,2006.[12] J. Duffy, “Agent-based models and human subject experi-ments,” In: Tesfatsion L, Judd K (eds), Handbook of com-putational economics: Agent-based computationaleconomics, vol. 2. Elsevier, Oxford, pp. 949–1011, 2006.[13] N. Chan, B. LeBaron, A. Lo, and T. Poggio, Informa-tion dissemination and aggregation in asset markets with simpleintelligent traders. Working paper, MIT. 1999.[14] S.-H. Chen, R.-J. Zeng, and T. Yu, “Co-evolvingtrading strategies to analyze bounded rationality in doubleauction markets,” In: Riolo R (ed.), Genetic programming the-ory and practice VI, Springer, 2008, pp. 195–213.[15] L. Borghans, A. Duckworth, J. Heckman, and B. Weel,The economics and psychology of personality traits. IZA DP No.3333, 2008.[16] J. Elster, “Emotions and economic theory,” Journal ofEconomic Literature, vol. 36, pp. 47–74, 1998.[17] G. Loewenstein, “Emotions in economic theory andeconomic behavior,” American Economic Review, Papers andProceedings, vol. 90, pp. 426–432, 2000.[18] R. Bosman and F. van Winden, “Emotional hazard in apower-to-take experiment,” Economic Journal, vol. 112,147–169, 2002.[19] S.-H. Chen and B.-T. Chie, “Lottery markets design,micro-structure, and macro-behavior: An ACE approach,”Journal of Economic Behavior & Organization, vol. 67, no. 2,pp. 463–480, 2008.[20] S.-H. Chen and Y.-C. Huang, “Risk preference,forecasting accuracy and survival dynamics: Simulationsbased on a multi-asset agent-based artificial stock market,”Journal of Economic Behavior and Organization, vol. 67, no. 3,pp. 702–717, 2008.[21] J. Henrich, R. Boyd, S. Bowles, C. Camerer, E. Fehr,and H. Gintis, (eds.) Foundations of human sociality: Economicexperiments and ethnographic evidence from fifteen small-scale soci-eties. Oxford University Press, 2004.[22] E. Fehr and K. Schmidt, “A theory of fairness, com-petition, and cooperation,” Quarterly Journal of Economics,vol. 114, no. 3, pp. 817–68, 1999.[23] A. Kuznar, “Boundaries in decision theory-understand-ing tribal politics,” Proceedings of the Agent 2007 Conference onComplex Interaction and Social Emergence, 2007, pp. 433–442.[24] C. Camerer, G. Loewenstein, and D. Prelec, “Neu-roeconomics: How neuroscience can inform economics,”Journal of Economic Literature, vol. 43, pp. 9–64, 2005.[25] A. Rustichini, “Neuroeconomics: Present and future,”Games and Economic Behavior, vol. 52, pp. 201–212. 2005.[26] A. Sanfey, G. Loewenstein, S. McClure, and J. Cohen,“Neuroeconomics: Cross-currents in research on decision-mak-ing,” Trends in Cognitive Sciences, vol. 10, pp. 108–116, 2006.[27] S.-H. Chen and C.-C. Tai, “On the selection of adap-tive algorithms in ABM: A computational-equivalenceapproach,” Computational Economics,” vol. 28, no. 1, pp. 51–69. 2006.[28] R. Marks, “Market design using agent-based models,” In:Tesfatsion L, Judd K (eds), Handbook of Computational Econom-ics, vol. 2, Chapter 27, pp, 1339–1380, Elsevier. 2006.

IntroductionComputational intelligence (CI) tech-niques become more and more impor-tant for seeking solutions, strategies, or

aggregate behaviours in economicgames. By economic games, we meangames that have their foundations ingame theory with an economic settingor market models in micro-economicsthat can be formulated as (elaborate)

games. Examples of the former arenegotiation, auction, and the prisoner’sdilemma, while instances of the latterare basic models like the Cournot oli-gopoly game or elaborate models forelectricity and labor markets.

Computational Intelligence in Economic Games and Policy Design

Herbert Dawid, University of Bielefeld, GERMANY, Han La Poutré, NationalResearch Centre for Mathematics andComputer Science, THE NETHERLANDS,and Xin Yao, University of Birmingham, U.K.

Digital Object Identifier 10.1109/MCI.2008.929845