multi-agent systems (chapter 9) adapted with permission from adina magda florea adina@wpi

Slides from Sobah Abbas Peterson 1

Multi-Agent Systems(Chapter 9)

Adapted with permission from Adina Magda Florea

[email protected]


Benevolent vs.. self-interested agents

• Benevolent: cooperative distributed systems. (CDPS) Simplifies the task enormously.

• Self-interested- potential for conflict


Distributed problem solvingDistributed problem solving

• Group coherenceGroup coherence - agents want to work together - cooperative agents

• CompetenceCompetence - agents must find ways to work together - coordinate to cooperate

• Task and result sharing - an agent has many tasks to do and asks other agents to do some of its tasks; then it should integrate the results

• Distributed planning - the problem to be solved is to design and execute a plan in a distributed manner, by many agents


Distributed Problem Solving

• Motivations:– Speed up through parallelization– Distribution of expertise– Distribution of Data, features change– Problem is inherently distributed– Distribution of Results

• General Steps– Task decomposition– Task allocation– Exchange sub problem solutions– Task accomplishment– Results Synthesis (make whole)


Task Decomposition• Partitioning of a task into sub-tasks for possible

allocation to another agent• Goal is to make sub-tasks independent Minimize

coordination (so communication costs don’t outweigh gain)– Minimize shared data

– Minimize share resources

• Task decomposition is a hard problem and generally performed a priori by system designers.


Task Allocation• Homogenous Systems

– Agents identical, allocation simple since each is equally qualified to work on sub-tasks

• Heterogeneous Systems– Sub-task requirements - matched to agent skills– Potentially difficult problem (perfect match

problem)


Which kind of system to build?

• Homogenous systems are simpler– Only one kind of agent to build

– Don’t have to consider agent skills when distributing sub-tasks

• Homogenous systems considered unsuitable for complex problems– Low overall utilization of skills and resources


Agent Roles in Task Allocation• Agents can assume two roles

– Servers: Agents capable of providing a service– Clients: Agents requiring a service

• Agents can be both – I.e. An agent may use the services of other

agents to complete a service is to providing to another agent

• Task allocation systems must provide a way to match clients with servers


Centralized Allocation Systems• Centralized

– 3rd party manages client-server matching– Hierarchical Subordination

• Superior agents order subordinates to carry out task.• Typically a static, pre-defined agent organization

– “Egalitarian” - all agents considered “equal” • Requires special “broker” or “trader” agents to

manage client requests and server bids• Allows centralized allocation techniques


Egalitarian Allocation System

ServersTraderClient

A B

C

D

RequestA

RequestA

RejectC

AcceptD

AcceptD


Distributed Allocation Systems• Each agent individually attempts to obtain required services

• Acquaintance Network– Direct Allocation

• Agents can only use the services of the agents it knows about– Potentially serious scalability issues

– Delegated Allocation• Agents can ask other agents to use their acquaintances to find an agent

capable of providing a particular service– Requires strongly connect acquaintance network

– Both methods require accurate knowledge of agent skills• May use various “caching” strategies to maintain and age acquaintance

information


Distributed Allocation Systems (cont)• Contract Net

– “Market Place” approach• Clients issue description of tasks

• Servers reply with bids

• Client chooses the best bidder

• Server affirms its commitment

– Proven approach from other disciplines/simple

– Well suited for dynamic environments

– Concurrent and many-to-many nature of the protocol creates challenging race conditions


Task Allocation System Tradeoffs

• Benefits– Coherence

• Drawbacks– Bottleneck– Fault

Intolerance

• Benefits– No Bottleneck

– Fault tolerance

• Drawbacks– Coherence

– Scalability

– Latency

• Benefits– Proven/Simple

– Flexibility

• Drawbacks– Message volume

– Temporal & Spatial Ignorance

Distributed

Acquaintance Contract Net

Centralized

Trader


Types of Tasks• Independent

– Tasks are self-contained

– Can be performed in any order and concurrently

• Interdependent– The solutions of some sub-tasks are required for the solution

of other sub-tasks

– Coordination possible if dependencies known before

– Possible dependencies only become apparent at runtime

– A Results Sharing mechanism is needed to solve these dependencies


Motivations for Results Sharing• Confidence:

– Independent derivations affirm/challenge previous results leading to more confidence

• Completeness:– Combination of partial results leads to a larger set of

results

• Precision:– Sharing of results allows for iterative refinement

(agents come to see interface)

• Timeliness:– Obvious performance benefits via parallel processing


Result Sharing

• Problem solving proceeds by agents cooperatively

exchanging information as the solution is developed.

• Results may be shared:

– proactively - one agent sends another agent some information

because it believes that the other will be interested in it.

– reactively – an agent sends information to another in response to a

request.

A1 A2 A3


Result Sharing Benefits

• Confidence (checking solutions)

• Completeness/precision: share local views

• Timeliness: may get results faster (even if agent could do it himself)


What about inconsistency?

• Ignore it – but are you throwing away the true information (the part that doesn’t fit the expectation)?

• Resolve it through negotiation• Degrade gracefully

– progress opportunistically (not in strict predetermined order)

– communicate high level results, not raw data– inconsistency resolved as you go (not at end)– no single solution route (if one is problematic, try

another)


The Coordination Problem

• Managing the

interdependencies between the

activities of agents. e.g.

– You and I both want to leave the

room. We independently walk

towards the door, which can only

fit one of us. I graciously permit

you to leave first.


Coordination Techniques

• Organisational Structures

• Multi-agent Planning

• Norms and social laws

• Coordination Models based on human teamwork:

– Joint commitments (Jennings)

– Mutual Modelling


Organizational Structuring

• Organizes agents into an organization– May be based on how the task was decomposed

• Agents use knowledge of the organization to– Determine with whom to communicate– Prioritize tasks

• Agents only need to know about the local organizational structure (coherence)

• Choosing an organization structure can, itself, be a difficult problem!


Organizational Structuring

Geographicallydistributed“cells”


Organizational Structures

• A pattern of information and control relationships between

individuals.

• Responsible for shaping the types of interactions among the agents.

• Aids coordination by specifying which actions an agent will

undertake.

• Organizational structures may be:

– Functional (based on skills)

– Spatial (based on physical location)

– Temporal (based on time relationship)


Organizational Structure Models

• A pattern for decision-making and communication

among a set of agents who perform tasks in order

to achieve goals. e.g.

– Automobile industry

• Has a set of goals: To produce different lines of cars

• Has a set of agents to perform the tasks: designers, engineers,

salesmen

Reference: Malone 1987


Alternative Coordination Structures 1Product Hierarchy

Designer

Product Manager I

SalesmanEngineer Designer

Product Manager 2

SalesmanEngineer


Product Manager (several products)

Alternative Coordination Structures 2Functional Hierarchy

Designers

DesignManager

Salesmen

SalesManager

Engineers

EngineeringManager


Alternative Coordination Structures 3Centralised Market

Product Manager 2

Designers

DesignManager

Salesmen

SalesManager

Engineers

EngineeringManager

Product Manager 1 Product Manager 3

FunctionalManagers


Alternative Coordination Structures 4Decentralised Market

Product Manager 2

Designers SalesmenEngineers

Product Manager 1 Product Manager 3


Comparison of Organization Structures – the Issues!

Production

cost

Coordination

cost

Vulnerability

cost

Product

hierarchyH L H-

Funtional

hierarchyL M- H+

Centralised

marketL M+ H-

Decentralised

marketL H L


Organizational Structures - Critique

• Useful when there are master/slave relationships in the

MAS.

• Control over the slaves actions – mitigates against benefits

of DAI such as reliability, concurrency.

• Presumes that atleast one agent has global overview – an

unrealistic assumption in MAS.


Partial Global Planning (PGP)

• A DAI testbed – Distributed Vehicle Monitoring Testbed

(DVMT) – to successfully track a number of vehicles that

pass within the range of a set of distributed sensors

(agents).

• Each agent monitors a

dedicated area

• There could be overlapping

areas

Overlappingarea

Agenti

Agentj

Vehicletrack



• Main principle: cooperating agents exchange information

in order to reach common conclusions about the problem

solving process.

• Why is planning partial?

– The system does not generate a plan for the entire problem.

• Why is planning global?

– Agents form non-local plans by exchanging local plans and

cooperating to achieve a non-local view of problem solving.



• Starts with the premise that tasks are inherently decomposed.

• Assumes that an agent with a task to plan for might be unaware as

to what tasks other agents might be planning for and how those

tasks are related to its own.

• No individual agent might be aware of the global tasks or states.

• Purpose of coordination is to develop sufficient awareness.



• PGP involves 3 iterated stages:

1. Each agent decides what its own goals are and

generates short-term plans in order to achieve them.

2. Agents exchange information to determine where

plans and goals interact.

3. Agents alter local plans in order to better coordinate

their own activities.



• Partial Global Plan: a cooperatively generated

datastructure containing the actions and interactions of a

group of agents.

• Contains:

– Objective – the larger goal of the system.

– Activity map – what agents are actually doing and the results

generated by the activities.

– Solution construction graph – a representation of how the agents

ought to interact in order to successfully generate a solution.



• A DAI testbed – revisited.

Overlappingarea

Agenti

Agentj

Vehicletrack

ji




Multi-agent Planning







• Agents generate, exchange and synchronise explicit plans

of actions to coordinate their joint activity.

• They arrange apriori precisely which tasks each agent will

take on.

• Plans specify a sequence of actions for each agent.

• It is a trade-off between specificity and reactive.



• Two basic approaches:

1. Centralised – plans of individual agents analysed by a

central coordinator to identify interactions.

2. Distributed – a group of agents cooperate to form a:

1. Centralized plan

2. Distributed plan

Big difference between them!



• Distributed Planning for centralised plans:

– e.g. Air traffic control domain (Cammarata)

• Aim: Enable each aircraft to maintain a flight plan that will

maintain a safe distance with all aircrafts in its vicinity.

• Each aircraft send a central coordinator information about its

intended actions. The coordinator builds a plan which specifies

all of the agents’ actions including the ones that they should

take to avoid collision.



• Distributed Planning for distributed

plans:

– Individual plans of agents, coordinated dynamically.

– No individual with a complete view of all the agents’

actions.

– More difficult to detect and resolve undesirable

interactions.

Slides from Sobah Abbas Peterson 42 42

2 Distributed planning2 Distributed planning• What can be distributed:

The process of devising a plan is distributed among agents

Execution is distributed among agents

PlanningPlanningState representation and plan representationSearch vs. planning

• representation of changes to the world state

• representation of and reasoning about the plan (steps/actions)

Planning Planning SearchSearch


2.1 Centralized planning for distributed plans2.1 Centralized planning for distributed plans Operators move(b,x,y) move b from x to y movetotable(b,x)

Precond: on(b,x) clear(b) clear(y) Precond: on(b,x) clear(b)Postcond: on(b,y) clear(x) Postcond: ob(b,T) clear(x) on(b,x)on(b,x) clear(y)

43

A

B D

C E

F

Sinit

C

A

E

B F

D

Sfinal

I'm BillAgent1

I'm TomAgent2

on(A,B) on(C,D) on(E,F)on(B,T) on(D,T) on(F,T)

on(B,A) on(F,D)on(A,E) on(D,C)on(E,T) on(C,T)

on(B,A)

S1: move(B,T,A)

on(B,T) clear(B) clear(A)

movetotable(A,B) move(A,B,y)

S2: move(A,B,E)

clear(A) clear(E) on(A,B)…………..………….

on(E,T)

S3: movetotable(E,F)

1. Given a goal description,a set of operators,and an initial state descriptiongenerate a partial order plan

work backward from each “on” goal


S1: move(B,T,A) To satisfy the preconditions, we have:

S2: move(A,B,E) S2 < S1, S3 < S4

S3:movetotable(E,F) S6 < S4, S6 < S5

S4: move(F,T,D) Also

S5: move(D,T,C) S2 threat to S3 S3 < S2

S6: movetotable(C,D) S4 threat to S5 S5 < S4

Then the partial ordering is: S3 < S2 < S1

S6 < S5 < S4

S3 < S4

Any total ordering that satisfies this partial ordering is a good plan for Agent1

44

2. Decompose the plan into sub problems so as to minimize order relations across plans 3. Insert synchronization

4. Allocate sub plans to agents


What if we have 2 agents?

DECOMP1

Subplan1 S3 < S2 < S1


and S3 < S4

Agent1 S3 < send(clear(F)) < S2 < S1

Agent2 S6 < S5 < wait(clear(F)) < S4


S3: movetotable(E,F) S2: move(A,B,E) S1: move(B,T,A)

S6: movetotable(C,D) S5: move(D,T,C) S4: move(F,T,D)

DECOMP2



and S3 < S2 and S6 < S5

Agent1 S3 < send(don't_care(E)) < wait(clear(D)) < S5 < S4

Agent2 S6 < wait(don't_care(E)) < wait(clear(D)) < S2 < S1

Obviously, DECOMP2 has more order relations among sub plans than DECOMP1 Therefore, we choose DECOMP1

S3 < send(clear(F)) < S2 < S1

S6 < S5 < wait(clear(F)) < S4

But

then back to DECOMP2

46

< <

4. If failure to allocate sub plans

then redo decomposition (2)

If failure to allocate sub plans with

any decomposition

then redo generate plan (1)

5. Execute and monitor sub plans

I know howto move only

D, E, F

I know howto move only

A, B, C


2.2 Distributed planning for centralized plans2.2 Distributed planning for centralized plans

generate separate plans, then merge parallel result sharing may involve negotiation

Agent 1 - is specialized in doing movetotable(b,x)

Agent 2 - is specialized in doing move(b,x,y)

PAgent1 = { S3: movetotable(E,F) satisfies on(E,T)

S6: movetotable(C,D) satisfies on(C,T)

no ordering }

PAgent 2 = { S1: move(B,T,A), S2: move(A,B,E) satisfies on(B,A) on(A,E)

S4: move(F,T,D), S5: move(D,T,C) satisfies on(F,D) on(D,C)

ordering S2 < S1 and S5 < S4 }

• Merge PAgent1 with PAgent2 by checking preconditions and threats

• S3 < S2, S6 < S5, S3 < S4, S2 < S1 and S5 < S4

• one agent executes (as is centralized)

47


• The problem is decomposed , given to specialize

• similar to task sharing may involve backtracking

Agent 1 - knows only how to deal with 2-block stacks

Agent 2 - knows only how to deal with 3-block stacks

48

C

A

E

B F

DSf

C

A

E B F

D

A

B D

C E

F

Si


2.3 Distributed planning for distributed plans2.3 Distributed planning for distributed plansa) Plan merginga) Plan merging How much effort on coordinating issues? Agents formulate local plans to satisfy their goals Local plans are exchanged Local plans are combined analyzing for positive and negative interaction Add messages and/or timing commitments to resolve negative plan interactions

and to exploit positive plan interactions

Interacting situations• Positive interactions between plans

– redundant actions – beneficial actions

• Negative interactions between plans– harmful actions– exclusive actions– incompatible actions

49


movehigh(b,x,y)

Precond: have_lifter clear(b) clear(y) on(y,z) z T

Postcond: on(b,y) clear(x) on(b,x) clear(y) free_lifter pick_lifter

Precond: free_lifter

Postcond: have_lifter free_lifter

Agent1: { S1:move(B,T,A) < S2: pick_lifter < S3: movehigh(E,T,B) }

Agent2: { R1:move(C,T,D) < R2: pick_lifter < R3: movehigh(F,T,C) }

50

A B D CEF

Si

D

A

B

E F

CSf

S1

S2

S3

R1

R3

R2

need_l

free_l

B

A

B

CSf1

Negative interactionswhat type?

if both select same lifter


Give examples of positive interactions

• redundant

• beneficial

Problems with the approach?

51

Positive interactions

b) Iterative plan formationb) Iterative plan formation• build all feasible plans• build partial order plans to facilitate plan

merging• build abstract plans to be iteratively refined


c) Hierarchical distributed planningc) Hierarchical distributed planning• Each agent stores plans on several levels of abstraction

• Use abstract plans (hides details)

• Abstract operator - a kind of macro-operator = sequence of applicable operators

Write paper

Readreferences

Organizeideas

Typecontent

Locate Computer

Edittext

Editfigures

Checkfor errors

…..


Hierarchical behavior-space search algorithm1. Level 0 (current level of abstraction), Agent_List = {Agent1, …, AgentN}

2. for i=1,N do

if Pi is compatible with {PJ}, j=1,N, ji

then Agenti removes itself from Agent_List (no problems)

3. if Agent_list = { } then exit

4. Let N be the new number of agents in Agent_List

4.1 Determine conflicts between {Pi}

4.2 if conflicts to be resolved at a lower level

then (a) Level Level + 1 (b) go to step 2

5. 5.1 Sort agents in Agent_List

5.2 for i=1,N-1, in current ordering do

(a) make Agenti the current superior

(b) send Pi to each AgentJ, j=i+1, N

(c) for j=i+1, N do

- AgentJ checks compatibility of PJ with Pi and replan

- AgentJ checks compatibility with PK, k=1,i-1 and replan

A kind of CSP: - backward checking - forward checking Ordering: - what heuristic?

Add exit conditionfor no solution


2.4 Distributed planning and execution2.4 Distributed planning and executionReal world: incomplete and incorrect information

a) Contingency planninga) Contingency planning• Conditional planning - constructing a conditional plan that

accounts for each possible situation or contingency that could arrive

54

move(A,B,C)

Start

Ask Ag2 to move(A,B,C)

Checkarm(Ag1)

Finish

Negotiate with Ag2for it to achieve move

Context:armbroken(Ag1)

armbroken(Ag1) armbroken(Ag1)

on(B,A)on(A,C)

on(A,B)clear(C)clear(A)A

B C

A

C

B

… Plan to achieve on(B,A)



• Critique:

– Agents share and process a huge amount of information.

– Requires more computing and communication resources.

• Difference between multi-agent planning and PGP:

– PGP does not require agents to reach mutual agreements

before they start acting.



• Sometime Plans can also become obsolete very quickly.

i.e. Short life-span.


Let’s take a minute……

• Can you think of a situation where multi-agent

planning will not be appropriate?

• Discuss with your neighbours.


Comparing Common Coordination Techniques

A Look at the Issues

Organisation

Structures

Multi-agent

Planning

low low less

high high more

Predictability

Reactivty

Info

Exchange





Norms and social laws





Social Norms and Laws

• Norm: an established, expected pattern of behaviour.

– e.g. To queue when waiting for the bus (not always in Norway!!)

• Social laws: similar to Norms, but carry some authority.

– e.g. Traffic rules.

• Social laws in an agent system can be defined as a set of constraints:

– Constraint => E’, ,

• E’ E is a set of environment states

Ac is an action, (Ac is the finite set of actions possible for an agent)

if the environment is in some state e E’, then the action is forbidden.


Social Norms and Laws

• Example: Feature

interaction in

telecommunications

• Uses deontic logic

(model obligations)

Process incoming

call

Incomingcall screening

Incomingcall answer

Forwardcall

Acceptcall

Recall

Forward #1 Forward #1

obliged obliged

obliged

obliged

forbidden forbidden

forbidden

obliged






Coordination Models based on human teamwork:




Coordination & Cooperation 1

• Can we have coordination without

cooperation?

– ”A group of people are sitting in a park. As a

result of a sudden downpour, all of them run to

a tree in the middle of the park because it is the

only source of shelter.”


• How does an individual intention towards a goal

differ from being a part of a team (a collective

intention towards a goal)?

Responsibility

– e.g. You and I are lifting a heavy object.

Individual goal team responsibility

Coordination & Cooperation 2


Coordination Based on Human Teamwork

• Some agent coordination models are inspired by human

teamwork models, e.g. Joints intentions (Jennings).

• Intentions are central to the concept of practical reasoning.

Practical reasoning = deliberation + means-end reasoning

– Deliberation – deciding what state of affairs to achieve

– Means-end reasoning – deciding how to achieve these states of

affairs


Mutual Modelling

• Build a model of the other agents – their beliefs

and intentions.

Put ourselves in the place of the other

• Coordinate own activities based on this model.

• Coordination without cooperation – game-thoery

can be used.


Joint Intentions

• Proposed by Jennings

• Based on human teamwork models

– ”When a group of agents are engaged in a cooperative activity,

they must have a joint commitment to the overall aim as well as

their individual commitments.”

• Distinguishes between the commitment that underpins an

intention and the associated convention.


Joint Commitments

• Commitment – a pledge or promise (e.g. to lift the heavy

object).

– Commitment persists – if an agent adopts a commitment, it is not

dropped until for some reason it becomes redundant.

– Commitments may change over time, e.g. due to a change in the

environment

– Main problem with joint commitment:

• Hard to be aware of each others states at all times


Conventions

• Convention – means of monitoring a commitment

– e.g. specifies under what circumstances a commitment can be

abandoned.

• Need conventions to describe when to change a

commitment:

1. When to keep a commitment (retain)

2. When to revise a commitment (rectify)

3. When to remove a commitment (abandon)


Convention - Example

• Reasons for terminating a Commitment:

– Commitment Satisfied

– Commitment Unattainable

– Motivation for commitment no longer present

• Rule R1:– If Commitment Satisfied OR

Commitment Unattainable OR

Motivation for Commitment no longer present

then

terminate Commitment.


Social Conventions

• Conventions describe how an agent should monitor its

commitments, but not how it should behave towards other

agents.

– Asocial

– Sufficient for goals that are independent.

• For inter-dependent goals:

– Need social conventions

• Specify how to behave with respect to the other members of the team.


Teamwork Definition

• American Heritage Dictionary

– Cooperative effort by the members of a

team to achieve a common goal.


Teamwork Example

Two vehicles travelling in a convoy:

Consider two agents Bob and Alice. Bobs wants to drive

home, but does not know his way. He knows that Alice is

going near there and that she does know the way. Bob

talks to Alice and they both agree that he follows her

through traffic and that they drive together.

Ref: Cohen & Levesque, 1991


Teamwork 1

• Important distinction:

– Coordinated action that is not cooperative, e.g

• Individual drivers in traffic following traffic rules

– Coordinated cooperative action, e.g

• A convoy of drivers


Teamwork 2

• How does an individual intention towards a particular

goal differ from being a part of a team with a

collective intention towards a goal?

– Responsibility towards the other members of the team.G

g2 g3g1

i j k

• Agents i, j and k are a team and have a

common goal G.


Teamwork 3

• Joint action by a team involves more than just the

union of simultaneous individual actions.

- Joint intentions and mutual beliefs (Cohen &

Levesque, 1991)

- Joint commitment (Jennings, 1996)

• When a group of agents are engaged in a cooperative

activity, they must have:

• Joint commitment to the overall activity

• Individual commitment to the specific task that they have been

assigned to

G

g2 g3g1

i j k


Joint Intentions (Jennings) RevisitedSocial Conventions

• Team members must be aware of the convention that govern

their interactions. e.g.

G

g1 g2AND

Ai Aj

G

g1 g2OR

Ai Aj

• Both Ai and Aj must fulfill their commitments

to achieve G.

• Either Ai or Aj must fulfill their commitment.

There is a need for all agents in a team to

inform other members of the status of their

commitments!


Teamwork Model Based on CDPS

1. Recognition

• Agent has a goal and recognises the potential for cooperative

action.

2. Team Formation

• Finds a group of agents that have a commitment to joint action.

3. Plan Formation

• Agree upon course of action, (through a process of negotiation).

4. Team Action

• Execute agreed plan of joint action.

G

G

g2 g3g1


Team Selection

• ”The process of selecting a group of agents that

have complimentary skills to achieve a given

goal(s).” (Ref: Tidhar et. al., 1996)

– Agents exchange their skills, goals, plans,

current beliefs.

– Done at runtime.

multi-agent systems (chapter 9) adapted with permission from adina magda florea adina@wpi

Documents

sobah abbas petersonbenevolent

service agents

trader agents

agents capable

kind of agent

allocation simple

possible allocation

agenttask allocation