collaboration rules for autonomous software agents

Ž .Decision Support Systems 24 1999 269–278

Collaboration rules for autonomous software agents

Sarosh N. Talukdar )

Carnegie Mellon UniÕersity, Department of Electrical and Computer Engineering, Pittsburgh, PA 15213, USA

Abstract

Can autonomous software agents that are distributed over a computer network collaborate effectively? Both empiricalevidence and theory suggest that they can. Moreover, there seem to be simple rules for designing problem-solving

Žorganizations in which collaboration among such agents is automatic and scale-effective adding agents tends to improve.solution-quality; adding computers tends to improve solution-speed . This paper develops some of these rules for off-line

Ž .problems and argues that they can be extended for the on-line real-time control of power systems. q 1999 Elsevier ScienceB.V. All rights reserved.

Keywords: Autonomous agents; Collaboration; Multi-agent systems; Organizations

1. Introduction

This paper deals with the skills that unsupervisedŽ .autonomous software agents must have if they areto collaborate effectively. This section explains theterminology that will be used, formulates the collab-oration problem and outlines an approach to itsresolution.

1.1. Terminology

Let a ‘software agent’ be any encapsulated pieceof computer code, such as a program or a subroutine.Functionally, such an agent can be thought of as a

w xbundle of skills 5 that can be divided into threecategories:Ø problem solving skills which determine what the

Žagent can do particularly, what computational.tasks it can perform ;

Ø social or self-management skills which determineŽwhat the agent chooses to do particularly, how it

) E-mail: [email protected]

chooses to collaborate with the other information.processing agents in its environment ;

Ø and learning skills which determine how well theagent transforms its experiences into new skills.

Let:Ø ‘An agent’s environment’ mean the set of all the

things that the agent can affect or by which it isŽaffected The environment of a software agent is

mediated by a computer network and may includeother types of information-processing agents, par-

.ticularly, humans .Ø ‘An off-line problem’ mean a computational

problem without hard constraints on solution-time,as is the case with many design, planning,scheduling, optimization and diagnosis problems.ŽIn other words, dealing with an off-line probleminvolves two objectives: maximizing solution-qu-ality and maximizing solution-speed. In contrast,dealing with an on-line or real-time problemsinvolves only a single objective: maximizing so-lution-quality, but this must be done subject todeadlines, that is, hard constraints on solution-

.speed.

0167-9236r99r$ - see front matter q 1999 Elsevier Science B.V. All rights reserved.Ž .PII: S0167-9236 98 00070-0

( )S.N. TalukdarrDecision Support Systems 24 1999 269–278270

Ø ‘Collaboration’ mean the exchange of data amonginformation-processing agents, regardless ofwhether the exchange is productive or not.

Ø ‘An organization’ mean a triple whose elementsare: a set of software agents, a network of com-puters for the agents to use, and a prescription for

Žthe agents’ collaborations. Organizations can beŽ .of two types: hierarchic and non-hierarchic flat .

In a hierarchic organization, the prescription forcollaboration relies for its effectiveness on thedelegation of responsibility: the agents are ar-ranged in layers; those in a higher layer are maderesponsible for, and given supervisory authorityover those in the layers below. In a flat organiza-tion, there are no supervisors. Rather, the agentsare autonomous and the effectiveness of theircollaborations are determined solely by their so-

.cial skills.Ø An ‘open organization’ mean an organization

whose size can easily be changed by the additionor removal of agents and computers.

Ø A ‘scale effective organization’ mean an organi-zation whose performance improves with size.Specifically, solution-quality improves with theaddition of agents and solution-speed improveswith the addition of computers.

1.2. A design problem

Existing software agents tend to be rich in prob-lem-solving skills but poor in social and learningskills. Computer networks make it possible to inter-connect very large numbers of software agents, andthereby, to amass enormous pools of raw problem-solving capability. How can all this capability beexploited? Of the many issues implied by this ques-tion, the subset that will be considered here is asfollows.

Given:Ø an instance of an off-line problem, andØ a set of software agents distributed over a

computer network,Design:Ø an open, scale-effective organization for solv-

ing the off-line problem.In other words, search through the space of allpossible organizations for one whose size is easilyincreased and whose performance, on the given off-line problem, improves with size.

The advantages of such an organization are easygrowth and fault tolerance. Costly demolition andreengineering become unnecessary for performanceimprovements. Instead, the problem of improvingperformance reduces to one of finding which agentsand computers to add. Conversely, the loss of agentsor computers is likely to cause a gradual rather thanprecipitous decrease in performance.

1.3. A subspace of flat organizations

Rather than solving the design problem above, theapproach taken here is to replace it with an easierproblem. Specifically, the space of all possible orga-nizations is replaced by a much smaller subspacewith a much higher concentration of open and scale-effective organizations. This subspace contains onlya certain type of flat organization called an asyn-chronous team. These teams are always open and areoften scale effective.

The subspace of asynchronous teams is definedby a constructive grammar. The two most importantcomponents of such a grammar are a set of primi-tives and a set of rules for composing organizationsfrom the primitives.

The primitives of the grammar were obtained byexamining a variety of existing organizations andextracting their better features. These primitives pro-

Žvide for agent autonomy as enjoyed by the membersw x.of some insect societies 20 , creation and destruc-

Žtion processes of adjustable strengths as in somew x.cellular communities 15 , populations of trial-solu-

Ž w x.tions as in genetic algorithms 9 , shared memoriesŽ w x. Žas in blackboards 19 , randomized selection as in

w x.simulated annealing 16 , lists of elements to beŽ w x.avoided as in tabu search 12,13 , and uninhibitedŽ w x.creation as in brainstorming 21 .

The rules of the grammar ensure openness andease of communication. Specifically, these rules per-mit only completely autonomous agents that collabo-rate only by modifying one another’s results. There-fore, the agents have no need for external supervi-sion; no centralized control or planning systems needbe built, nor are there are any managerial layers toimpede the addition or removal of agents. Moreover,the needs for commonality of expression and repre-sentation are minimal. To illustrate, consider an agentthat specializes in repairing one part of the results

( )S.N. TalukdarrDecision Support Systems 24 1999 269–278 271

generated by other agents. This agent needs to knowonly how this part is represented. It interacts with theother agents only through its repairs.

1.4. Demonstrations

One might think that agents that are autonomousand know virtually nothing about one another, wouldtend to work at cross purposes. Nevertheless, effec-tive asynchronous teams have been developed for avariety of off-line problems, including nonlinear

w xequation solving 10,27 , traveling salesman prob-w x w xlems 11 , high-rise building design 22 , reconfig-

w xurable robot design 18 , diagnosis of faults in elec-w x w xtric networks 7 , control of electric networks 1,25 ,

w xjob-shop-scheduling 8 , steel and paper mill schedul-w x w xing 4,17,23 , train-scheduling 28 , and constraint

w xsatisfaction 14 . Not only do these asynchronousteams find good solutions, but they appear to achievescale effectiveness through fairly simple mecha-nisms. The succeeding material explains how andwhy. Specifically, Section 2 develops a frameworkfor describing and analyzing collaborative effortsamong autonomous software agents. Section 3 dis-tills the descriptive elements of this framework into aset of primitives, and the prescriptive elements into aset of rules, of a grammar for asynchronous teamsfor off-line problems. Section 4 argues for the exten-

Ž .sion of this grammar to on-line real-time problems,particularly, distributed control and mixed initiativeproblems for electric power systems.

2. Definitions and models

This section develops a framework for analyzingcollaborations among cyber agents, a class of agentsthat includes software agents as well as humans, andargues that scale-effectiveness is likely to occur inmany asynchronous teams.

2.1. Cyber agents

The environment of the any agent can be dividedinto two spaces-one that the agent perceives or sensesŽ . Žits input-space , the other, that it affects its output-

.space . Of course, these spaces may overlap.

Definition 1. A cyber agent is an agent whose input-and output-spaces are maintained by computers.ŽThus, both software agents and humans who happento be engaged in computer-mediated work, qualify as

.cyber agents.

Since a space is a set of objects, any computermaintained space can be thought of as a set ofmemories for symbolic objects. Therefore, every cy-ber agent in any organization of cyber agents can bemodeled as:Ø a set of computer-maintained memories from

Ž .which the agent can read the agent’s input space ,Ø a set of computer-maintained memories to which

Ž .the agent can write the agent’s output space ,ŽØ an operator embodying the agent’s problem-solv-

.ing skills that can copy objects from the inputmemories, transform them, and write the resultsto one or more of the output memories, and

Ø a control system consisting of the agent’s ownsocial skills together with any external controls,such as reporting requirements, imposed by theorganization.

Note that this model captures all the operating possi-bilities for the software agent. But it is less completefor humans in that it allows only for computer-medi-ated exchanges of information.

Definition 2. A cyber agent is autonomous if itscontrol system is completely self-contained, that is,if its social skills are its only controls.

As such, an autonomous cyber agent can do whatit wants when it wants. In particular, autonomousagents can choose to work in parallel all the time, ifenough computers are available.

Definition 3. A cyber agent is static if it remains atthe same location in its computer network, that is, ifit does not switch from one set of input or outputmemories to another.

Thus, there are only two tasks for the controlŽsystem of a static cyber agent: selection choosing

.the objects to be read from its input-memories andŽscheduling determining when the operator will work

on the selected objects and which of the available.computers it will use for this work .


Definition 4. The work-cycle of a cyber agent con-Ž .sists of the following sequence: read copy a set of

objects from its input-memories, modify these ob-jects, and write the results to one or more of itsoutput-memories.

In some cases, the set of objects read by a cyberagent might be empty. But the work that a cyberagent does always causes the population of objects inat least one of its output-memories to change.

Definition 5. A cyber agent is destructive if it erasesobjects from the populations in its output-memories.Otherwise, it is constructive.

The purpose of a destructive agent is to eliminatethe mistakes and potential mistakes of its construc-tive counterparts by erasing outputs they should nothave produced and inputs they should not consider.

Notice that all the intelligence of an autonomousdestroyer is in its controls-its operator has only toperform the trivial task of erasing those objects itsselector has chosen.

2.2. Collaboration

Definition 6. Two autonomous cyber agents areconnected if they can exchange data, that is, if anoutput memory of one is an input memory of theother.

Definition 7. Two autonomous cyber agents collabo-rate whenever they actually exchange data, whetherthe exchange is productive or not.

Any collaborative arrangement among two ormore completely autonomous cyber agents can be

Ž .visualized as a ‘data flow’ Fig. 1 .

Definition 8. A data flow is a directed hypergraph inwhich each node is a Venn diagram of overlappinginput- and output-memories, and each arc representsthe operator and social skills of an agent. A dataflow is strongly cyclic if every one of its arcs is in aclosed loop. A data flow is unary if it has one andonly one node representing a single memory that isshared by all the agents in the data flow.

Fig. 1. A data flow in which M , M , M , M are memories, C ,1 2 3 4 1

C , C , C are construction agents, and D , D are destruction2 3 4 1 2

agents. M holds a population of trial solutions to the problem-to-1

be-solved. M and M hold populations of trial-solutions to2 3

related problems. M is the union of M and M .4 1 3

Definition 9. An asynchronous team is a stronglycyclic data flow, that is, a set of autonomous cyberagents, connected so their outputs can circulate, andrestricted to interacting only by modifying one an-other’s outputs.

2.3. Families of problems

Experience with asynchronous teams suggests thatthere are advantages to having them work on fami-lies of problems that include CP, the computationalproblem-to-be-solved, and some of its relatives. Per-haps progress on some of the easier problems in sucha family catalyzes progress in the more difficultones.

Ž .Definition 10. Two problems are related if i goodsolutions to one provide parts of, bounds for, or

Ž .other clues to good solutions of the second, or iisolutions of one influence solutions of the second.Two or more related problems constitute a family.

In general, a family of problems is more than ahierarchical decomposition of the problem-to-be-solved, and can include members whose only rela-tionship is that the solutions of some influence thesolutions of others. For instance, the two problems:design a car and design a process for manufacturingit, constitute a family, because the solution to thefirst influences the solution to the second.


The obvious way to deploy an asynchronous teamon a family of problems is to dedicate each of theteam’s memories to one of the problems. This isachieved by arranging for the memory to hold apopulation of trial-solutions to its problem, and usinga representation that is understood by all the agentsthat read from or write to that memory.

2.4. EffectiÕeness

An asynchronous team is started by placing apopulation of solutions in each of its memories. Theagents then go to work on changing these popula-

Žtions the constructors add new solutions while the.destroyers erase old solutions . Under what condi-

tions will good solutions appear in a population? Toanswer this question, we will model all stronglycyclic data flows by unary data flows, and all off-linecomputational problems by single objective opti-mization problems. The justifications are as follows.First, a node in a data flow represents several over-lapping memories. These can always be lumped intoa single equivalent memory and every disjoint sub-graph that starts and ends at the node can always belumped into a single equivalent agent, to yield aunary data flow. Since the ‘lumping’ involves only achange of name and no change of structure, thedynamics of the original node are preserved by itsequivalent. Second, every computational problem canbe expressed as a multi-objective, constrained opti-mization problem, which in turn, can be approxi-mated by a sequence of single-objective, uncon-strained optimization problems.Let:

M be the memory of a unary data flowŽ .S, q be the single-objective optimization prob-lem associated with M, where S is a set of allpossible solutions to the problem, and q is a scalarmeasure of solution-quality. An optimal solution isa member of S that maximizes q.G be the subset of S that contains only solutionsq

of quality q or better.Ž .P t be the population of trial-solutions in M at

Ž Ž .time t. The initial value of this population, P 0 ,.is assumed to be a randomly chosen subset of S .

C, D be the sets of construction and destructionagents that work on M, causing P to change withtime.

Ž . Ž .T q be the expected value of t for which P tand G first develop a non-zero intersection, thatq

Ž .is, the expected time for P t to evolve at leastone member of quality q or better.

Ž .Definition 11. G is reachable if T q is finite, thatq

is, if solutions of quality q or better will appear in Min an amount of time whose expected value is finite.

Definition 12. The effectiveness of M is the double:� Ž .4q , T q , where q is the largest value of qmax max max

such that G is reachable.q

Definition 13. A unary data flow is scale-effective ifadding agents increases q , and adding computersmax

Ž .decreases T q . A node in a strongly cyclic datamax

flow is scale-effective if its equivalent unary dataflow is scale-effective. An asynchronous team isscale-effective if any of its primary nodes is scale-ef-fective. A node is a primary node if it is dedicated toCP, the problem-to-be-solved, instead of to one of itsrelatives.

2.5. Construction space

Ž .Construction space, S, g , is the space of solu-tions, S, together with an integer-valued function, g,that measures the separation in construction-work-cycles between subsets of S. The following defini-tion of g is given in terms of the two subsets of

Ž .principal concern: P t and G .q

Ž Ž . .Definition 14. g P t , G , the separation of the setqŽ .P t from the set G , is the distance of the closestq

Ž . Ž Ž . . � Žpoint in P t to G , that is, g P t , G sMin f p,q q.4 Ž . Ž .G , pgP t , where f p, G , the distance of anyq q

point p from the set G , is the minimum number ofq

work-cycles by construction agents necessary to im-prove the quality of p to a level of q or better; thatis, to modify p till it becomes a member of G .q

Separation measured in this way has two notewor-thy properties. First, solution-quality and separationare not directly related; the highest quality member

Ž .of P t is not necessarily its closest member to G .q

Second, separation depends on the skills of the oper-ators in C, the set of construction agents. To illus-


Ž .trate, consider two peaks maxima in the qualitysurface, s and s . Suppose that the quality of s is1 2 1

only slightly lower than that of s . Suppose that C2

contains only greedy, hill-climbing operators that areunable to descend from a high point in order to reachan even higher point. Then, there is no path from s1

to s . In other words, the distance from s to s is2 1 2

infinite. However, if C is expanded to include opera-tors that can go down-hill, then the distance betweens and s will become finite. In general, as agents1 2

with new and useful problem-solving capabilities areadded to C, points of lesser quality draw closer topoints of greater quality.

2.6. ConÕergence conditions

Let:t , t , . . . be the discrete points in time at which1 2Ž .P t changes.

Ž .P , P , . . . , P be a sequence trajectory of0 1 N

populations that reaches G in N steps, whereqŽ .P sP t .n n

� 4s , s , . . . , s be the trial-solutions contained in1 2 J

P .n

s) be the solution in P that is closest to G .n q

g be the distance of P from G , that is, g sn n q nŽ . Ž .g P , G s f s), Gn q q

a be the event: g -g .n nq1 n

b be the event: g )g .n nq1 nŽ .Prob x be the probability of event x.

Definition 15. The drift, l , at the n-th point of an

population-trajectory is:

l sProb a yProb b 1Ž . Ž . Ž .n n n

In words, the drift at any point of the population-trajectory is the difference between two probabilities.The first is the probability that the next populationwill be closer to the goal, G , that is, the probabilityq

of one of the constructors selecting s) and improv-ing it. The second is the probability that the nextpopulation will be further from G , that is, theq

probability that one of the destroyers will mistakenlyerase s).

Theorem 1. If the actions of all the agents, particu-larly the destroyers, are reÕersible, if g is finite and0

if l is positiÕe for all n, then the expected Õalue ofn

N is finite. In other words, sufficient conditions forobtaining solutions of arbitrarily high quality are:( )a one of the constructors must be able to replace

( )solutions erased by the destroyers, b at least onemember of the initial population must be at a finite

( )distance from the desired solutions, and c the driftat all points along the population-trajectory must bepositiÕe.

Under the reversibility condition, the population-trajectory becomes a Markov chain, and the proof ofthe theorem follows directly from the properties of

w xthese chains. The details can be found in Ref. 26 .

2.7. Drift

What affects the drift, l ? Some insights aren

obtained by considering the following simplified sit-uation:Ø The agents work in sequence and each agent

modifies only one solution per work cycle. Inother words, P is produced from P by anq1 n

single agent. If this agent is a constructor, it addsone solution to P , if it is a destroyer, it erasesn

one solution from P .n

Ø The quality of each solution reflects its probabil-Ž .ity of being the best closest solution, that is

Prob d sq 2Ž .Ž .j j

where d is the event: s ss), and q is thej j j

quality of s , scaled so q qq q . . . qq s1.j 1 2 J

Ø All the agents use the same selection mechanism,‘quality based selection,’ and the same schedulingmechanisms, ‘perfect sequential scheduling.’

To define these mechanisms and analyze the simpli-fied situation, suppose there are K constructors, Kdestroyers and the solutions in P are labelled son

q Fq F . . . Fq . Now, consider the transition1 2 J

from P to P and the following events:n nq1

c : the transition is caused by the k-th constructor.k

h : the k-th constructor selects the j-solution, s .k j j

d : s is s), the best solution in terms of distance.j j

g : the k-th constructor is able to improve s).k

d : the transition is caused by the k-th destroyer.k

m : the k-th destroyer selects and erases thek j

j-solution, s .j


Definition 16. In quality-based selection, construc-tors and destroyers select randomly from the popula-tion of solutions, such that:

Prob h sq 3Ž . Ž .k j j

Prob m sq 4Ž . Ž .k j Jq1yj

Thus, constructors are more likely to select the higherquality solutions while destroyers are more likely to

Žselect lower quality solutions As such, quality-basedselection is a symmetrical adaptation of thesolution-rejection strategy used in simulated anneal-

.ing.

Definition 17. In perfect sequential scheduling, con-structors and destroyers schedule themselves so that:

Prob c s0.5 if Prob g )Prob g for all i/kŽ . Ž . Ž .k k i

s0 otherwise. 5Ž .Prob d s0.5rK 6Ž . Ž .k

Thus, a constructor is as likely as a destroyer tocause the transition from P to P . And all but then nq1

Žmost capable constructor with the largest value ofŽ ..Prob g refrain from scheduling themselves.k

Theorem 2. In the simplified situation the drift, l ,n

is positiÕe, if:

2Max Prob g ) S q q r S q 7� 4Ž . Ž . Ž .k j j Jq1yj j j

in other words, sufficient conditions for l to ben

positive are that there be at least one constructionagent with a nonzero probability of being able toimprove the best current solution, and furthermore,this probability must be greater than the right hand

Ž .side of 7 . Note that this right hand side can beconsiderably smaller than unity. For instance, withthree solutions of quality q s0.1, q s0.2 and1 2

Ž .q s0.7, the right hand side of 7 is only 1r3.3

Ž . Ž .Proof. l sProb a yProb b by definition. Ton n nŽ .determine the value of Prob a , note that the eventsn

c , c , . . . , c , d , d , . . . ,d are mutually exclusive,1 2 k 1 2 k

and a can occur only if the agent that works on Pn n

is a constructor. Therefore:

<Prob a sS Prob a c Prob c 8Ž . Ž . Ž .Ž .n k n k k

where S means ‘summation over all k.’ But:k

<Prob a c sS Prob h ld lgŽ . Ž .n k j k j j k

sS Prob h Prob d Prob gŽ . Ž .Ž .j k j j k

9Ž .

Ž . Ž . Ž .Combining 2 , 3 and 9 gives:

<Prob a c sProb g S y y 10Ž . Ž .Ž .n k k j j j


2Prob a s0.5 Max Prob g S q 11� 4Ž . Ž . Ž . Ž .n k j j

when scheduling is perfect. Now, to determine theŽ .value of Prob b , note that b can occur only if then n

agent that works on P is a destroyer and only if s)n

is unique. Therefore:

<Prob b sS Prob b d Prob d 12Ž . Ž . Ž .Ž .n j n k k

and

<Prob b d FS Prob m ldŽ . Ž .n k j k j j

FS Prob m Prob d 13Ž . Ž .Ž .j k j j


<Prob b d FS y y 14Ž .Ž .n k j Jq1yj j


Prob b F0.5 S q q 15Ž . Ž .n j j Jq1yj

Ž . Ž .l is positive if Prob a , as given by 11 , is largern nŽ . Ž .than Prob b , as given by 15 , completing then

proof.

Conjecture 1: Construction and destruction are du-als:adept destruction agents can compensate for ineptconstruction agents and vice versa. In other words,all the benefits of adding construction agents withnew and useful skills can also be obtained by addingdestruction agents with new and useful skills.

As yet no experiments have been conducted toclearly demonstrate the validity of this conjecture.But a strong argument for its validity is made in Ref.w x26 .

Conjecture 2: The mix of agents can include hu-mans without any deleterious consequences.


By way of justifying this conjecture, note that heconvergence conditions do limit the type of agent, aslong as the agent is autonomous and interacts withother agents only by modifying the trial-solutionsthey produce. Therefore, complex software agents

Žcan be mixed with simple software agents whichwhas been experimentally demonstrated 1,7,8,10,

x.11,14,17,18,22,23,25,27,28 , and in principle soft-Žware agents can be mixed with humans experiments

are now underway to examine this part of the conjec-w x.ture 24 .

2.8. Remarks

The key points made by the above analysis are:Ø Construction and destruction are duals in that any

effects obtained from adding construction agentscan be replicated by adding suitable destructionagents.

Ø As construction agents with new and useful prob-lem-solving skills are added, construction spacecontracts, that is, solutions of lesser quality drawcloser to the solutions of greatest quality.

Ø If the drift is kept positive, then solutions ofincreasingly high quality become reachable asconstruction space contracts, that is, as new con-struction agents are added.

Ø Drift is likely to be positive if quality-basedselection and perfect sequential scheduling areused.But perfect sequential scheduling is simulated if

Žall the agents work in parallel which happens auto-matically if the agents are autonomous and enough

.computers are available . Also, solution-speed maybe expected to increase as computers are added, atleast to the extent that the work cycles of the agentsare reduced these additions. Thus, the conditions forscale-effectiveness is asynchronous teams are notoverly restrictive, and one may expect it to appear inmany, if not most, asynchronous teams. The experi-

wmental evidence obtained so far 1,7,8,10,11,14,x17,18,22,23,25,27,28 bears out this conclusion.

3. A grammar for asynchronous teams

The preceding section provides a general descrip-tion of the structure and properties of asynchronousteams. However, in designing these teams, it is

convenient to have a more compact description. Sucha description is provided by a grammar whose primi-tives and rules are a distillation of Definitions 1–17.The main elements of this grammar are given below,

w xthe formal details are developed in Ref. 24 .The purpose of the grammar is to provide a means

for constructing all asynchronous teams that mightbe used in solving a given instance of a family ofoff-line problems. In other words, the grammar con-structively defines the space that must be searched ifan asynchronous team that is good at solving thegiven problem-instance is to be found.

The primitives of the grammar are:Ø sharable memories, each dedicated to a member

of the family-of-problems, and designed to con-tain a population of trial-solutions to its problem.

Ø operators for modifying trial-solutions.Ø selectors for picking trial-solutions.Ø schedulers for determining when selectors and

operators are to work.Ø Loosely speaking, the rules of the grammar are:Ø Form autonomous agents by packaging an opera-

tor with a selector and a scheduler.Ø Use quality-based-selection and completely paral-

Žlel execution all the agents running all the time,or as close to all the time as the available com-

.puter resources will allow as the default selectionand scheduling strategies.

Ø Connect the agents and memories to form astrongly cyclic data flow.

Ø Compensate for construction deficiencies withskilled destruction.

Ø Mix agents as needed without regard to theircomplexity or phylla, that is big and small soft-ware agents may be combined with humans, pro-vided only that the humans subscribe to the com-munication and selection conditions prescribedfor the software agents.

4. Some research issues: real-time power systemcontrol and learning

The typical power system contains thousands ofŽdistributed, mechanical decision makers control de-

.vices , such as relays and voltage regulators. Therestructuring of power systems that is now under-way, will undoubtedly introduce entirely new classesof control devices, such as FACTS-controllers and


automatic agents to trade in energy on behalf ofcustomers and independent generators. What are thebest strategies for these old and new controllers toemploy?

The main obstacles to good automatic-decision-making in distributed systems, such as electric powersystems, are:

Ž .1. The context location and view of the system ofeach decision-maker is unique. Therefore, theideal strategy for each decision-maker is alsounique.

2. The system changes with time. Therefore, thestrategy used by each decision-maker also needsto evolve with time.

3. Many of the rules that should be included in astrategy, such as the rules for when to switchfrom the normal mode of control to an the emer-gency mode of control, are not explicitly known.Because of these obstacles, it is impractical to

manually program the ideal strategy into each con-troller. An alternative is to provide these deviceswith the capabilities for automatic, context-depen-

Ždent and lifelong learning by which we mean thecontinual transformation of experience-actual or sim-ulated operating data-into location-specific, strategy

.improvements.Existing learning models, such as Neural Nets,

Bayesian Nets and Reinforcement Learning, can dealwith very large volumes of numerical data. But theycannot deal with data of high dimension, such as thestate information of an entire power system. There-fore, the successful application of learning technol-ogy to a power system is contingent on decomposingthe system into much smaller subsystems, each witha much smaller state-space. This decomposition canbe either hierarchical or flat. But, as has been pointedout before, flat decompositions have profound advan-tages: they tend to be much more open and faulttolerant. Realizing on these advantages will requireextending the off-line technology described earlier toreal-time situations, and adding learning skills toautonomous agents. Then, the mechanical decisionmaking can be handled by these autonomous agents,each collecting only locally available data on thestate of the power system, making up for its lack of aglobal view by collaborating with its immediateneighbors, and continually improving its perfor-mance through automatic learning.

w xFor quasi-repetitive problems 6 , that is, differentinstances of essentially the same general problem, ithas been demonstrated that the autonomous agents inan asynchronous team can learn to collaborate more

w xeffectively 2,3 . Now, more powerful learning tech-niques need to be developed and the issue of hardconstraints on solution-speed taken into account.

Acknowledgements

The work reported here was supported in part bythe National Science Foundation under Grant Num-ber ECS-9615599, and by DARPA through ContactNumber ONR Grant Number N00014-96-1-0854.

References

w x1 P. Avila-Abascal, S.N. Talukdar, Cooperative algorithms andabductive causal networks for the automatic generation ofintelligent substation alarm processors, Proceedings of IS-CAS-96.

w x2 L. Baerentzen, S.N. Talukdar, Improving cooperation amongautonomous agents in asynchronous teams, Journal of Com-putational and Mathematical Organizational Theory, submit-ted.

w x3 L. Baerentzen, P. Avila, S.N. Talukdar, Learning networkdesigns for asynchronous teams, in: Lecture Notes in Artifi-cial Intelligence 1237: Multi-Agent Rationality, Proc. of 8thEuropean Workshop on Modeling Autonomous Agents in aMulti-Agent World, MAAMAW’97, Springer, Ronneby,Sweden, May 1997.

w x4 G. Bassak, Bringing in the A-teams, IBM Research, No. 2,1996.

w x5 K. Carley, A. Newell, The nature of the social agent, JournalŽ . Ž .of Mathematical Sociology 19 4 1994 221–262.

w x6 K.M. Carley, Computational and mathematical organizationtheory: perspectives and directions, Journal of Computational

Ž . Ž .and Mathematical Organizational Theory 1 1 1995 .w x7 C.L. Chen, Bayesian Nets and A-Teams for Power System

Fault Diagnosis, PhD Dissertation, Electrical and ComputerEngineering Department, Carnegie Mellon University, Pitts-

Ž .burgh, PA, 1992 .w x8 S.Y. Chen, S.N. Talukdar, N.M. Sadeh, Job-shop-scheduling

by a team of asynchronous agents, IJCAI-93 Workshop onKnowledge-Based Production, Scheduling and Control,Chambery, France, 1993.

w x Ž .9 L. Davis Ed. , Handbook of Genetic Algorithms, Van Nos-Ž .trand Reinhold 1991 .


w x10 P.S. deSouza, S.N. Talukdar, Genetic algorithms in asyn-chronous teams, Proceedings of the Fourth International Con-ference on Genetic Algorithms, Morgan Kaufmann, Los Al-

Ž .tos, CA 1991 .w x11 P. de Souza, Asynchronous Organizations for Multi-Al-

gorithm Problems, PhD Dissertation, Dept. of Electrical andComputer Engineering, Carnegie Mellon University, Pitts-

Ž .burgh, PA 1993 .w x12 F. Glover, Tabu Search—Part I, ORSA Journal of Comput-

Ž . Ž .ing 1 3 1989 .w x13 F. Glover, Tabu Search—Part II, ORSA Journal of Comput-

Ž . Ž .ing 2 1 1990 .w x14 S.R. Gorti, S. Humair, R.D. Sriram, S. Talukdar, S. Murthy,

Solving constraint satisfaction problems using A-teams, toappear in AI-EDAM.

w x Ž .15 A. Kerr Jr., in: C.C. Thomas Ed. , Subacute BacterialŽ .Endocardites, Springfield, IL 1955 .

w x16 S. Kirkpatrick, C.D. Gelatt, M.P. Cecchi, Optimization byŽ . Ž .simulated annealing, Science 220 4598 1983 .

w x17 H. Lee, S. Murthy, W. Haider, D. Morse, Primary productionscheduling in steelmaking industries, IBM Report, 1995.

w x18 S. Murthy, Synergy in Cooperating Agents: Designing Ma-nipulators from Task Specifications, PhD dissertation, De-partment of Electrical and Computer Engineering, Carnegie

Ž .Mellon University, Pittsburgh, PA 1992 .w x19 H.P. Nii, Blackboard systems: the blackboard model of

problem solving and the evolution of blackboard architec-Ž .tures, Parts I and II, AI Magazine, 7:2 and 7:3 1986 .

w x20 G.F. Oster, E.O. Wilson, Caste and Ecology in the SocialŽ .Insects, Princeton University Press, Princeton, NJ 1978 .

w x21 S. Pugh, Total Design, Addison Wesley, 1990.w x22 R.W. Quadrel, Asynchronous Design Environments: Archi-

tecture and Behavior, PhD Dissertation, Department of Ar-

chitecture, Carnegie Mellon University, Pittsburgh, PAŽ .1991 .

w x23 J. Rachlin, F. Wu, S. Murthy, S. Talukdar, M. Sturzenbecker,R. Akkiraju, R. Fuhrer, A. Aggarwal, J. Yeh, R. Henry, R.Jayaraman, Forest view: a system for integrated schedulingin complex manufacturing domains, IBM Report, 1996.

w x24 S. Sachdev, Modular Optimization, PhD Proposal, CarnegieMellon University, Feb. 1998.

w x25 S.N. Talukdar, V.C. Ramesh, A parallel global optimizationalgorithm and its application to the CCOPF problem, Pro-ceedings of the Power Industry Computer Applications Con-ference, Phoenix, May, 1993.

w x26 S.N. Talukdar, L. Baerentzen, A. Gove, P.S. deSouza, Asyn-chronous teams: cooperation schemes for autonomous agents,to appear in the Journal of Heuristics, and visible at:www.ece.cmu.edurafsrecerusrrtalukdarrheuristics.ps.

w x27 S.N. Talukdar, S.S. Pyo, T. Giras, Asynchronous proceduresŽ . Ž .for parallel processing, IEEE Trans. on PAS 102 11 1983 .

w x28 C.K. Tsen, Solving Train Scheduling Problems Using A-Teams, PhD dissertation, Electrical and Computer engineer-ing Department, CMU, Pittsburgh, 1995.

Sarosh Talukdar is a Professor of Electrical and Computer Engi-Ž .neering at Carnegie Mellon University CMU . Before joining

CMU, he worked as a senior staff engineer at McGraw-Edison-amanufacturer of electrical equipment. His undergraduate degree isfrom the Indian institute of Technology, Madras, and his PhD isfrom Purdue University. Talukdar is known for his work insimulation, optimization and knowledge-based systems for electricpower networks, and autonomous software agents for difficultcomputational problems. He is a fellow of the IEEE and has beenthe recipient of the Eta Kappa Nu award for excellence inundergraduate teaching.

collaboration rules for autonomous software agents

Documents