multiagent systems...multiagent system? coherence : how well does the system behave as a unit ?...

MultiagentSystems

B. Nebel,C. Becker-Asano,S. Wöl�

Motivation

Task sharing

Coordination

Models forCoordination

Multiagent Systems7. Working Together: Methods for Coordination

B. Nebel, C. Becker-Asano, S. Wöl�

Albert-Ludwigs-Universität Freiburg

June 4, 2014

1 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Motivation

2 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Cooperative Distributed Problem Solving (CDPS)

Basic question:

How can a loosely coupled network of problem solvers worktogether to solve problems that are beyond their individualcapabilities? (Durfee et al., 1989, after Wooldridge, 2009)

Case one:Agents are benevolent, share a common goal and no potentialof con�ict between them.⇒ greatly simpli�es designer's task

Case two:Societies of self-interested agents don't share a common goal.⇒ designed by di�erent individuals or organizations to reachgoals through cooperation (which is nevertheless inevitable)

Second case is more challenging and interesting!3 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


CDPS versus parallel problem solving

How is CDPS di�erent from parallel problem solving (PPS)?

PPS exploits parallelism to solve problems

Computational components are processors, not agents

Central node decomposes overall problem intosubcomponents

Processors independently provide subsolutions

Subsolutions are assembled into overall solution by centralnode

PPS was synonym for CDPS in �early days of multiagentsystems�, two �elds are now quite separate.

4 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


How to evaluate a MAS

What criteria can be used to evaluate the �success� of amultiagent system?

Coherence: How well does the system �behave as a unit�?Criteria: solution quality, e�ciency of resource usage,conceptual clarity of operation, performance in thepresence of uncertainty/failure

Coordination: the degree to which agents can avoidine�ectual activitiesCriteria: amount of communication, task sharing, resultsharing

5 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Main issues in CDPS

Main issues to be addressed in CDPS include:

How can a problem be divided into smaller tasks fordistribution among agents?

How can problem solution be e�ectively synthesized

from subproblem results?

How to optimize overall problem-solving activities ofagents so as to produce solution to maximize coherencemetric?

What techniques can be used to coordinate activity ofagents, thus avoiding destructive interactions, andmaximizing e�ectiveness?

6 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


The three stages of CDPS

(Wooldridge, 2009, p. 154)

7 / 37

MultiagentSystems


Motivation

Task sharing

Contract Netprotocol

BlackboardArchitecture

Coordination


Task sharing

8 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Task sharing

�How can a problem be divided into smaller tasks fordistribution among agents?�⇒ Task-sharing

(Wooldridge, 2009, p. 156)A task is decomposed into subproblems that are allocated toagents.

9 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Task sharing in the Contract Net

The Contract Net (CNET) protocol (Smith 1977, 1980):

high-level protocol for achieving e�cient cooperationthrough task sharingbasic metaphor: contracting

10 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Contract Net protocol

One of the oldest, most widely used agent interactionprotocols

A manager agent announces one or several tasks, agentsplace bids for performing them

Task is assigned by manager according to evaluationfunction applied to agents' bids (e.g., choose cheapestagent)

Idea of exploiting local cost function (agents' privateknowledge) for distributed optimal task allocation

Even in purely cooperative settings, decentralization canimprove global performance

A typical example of �how it can make sense to agentify asystem�

Successfully applied to di�erent domains (e.g. transportlogistics)

11 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Contract-net protocolAgent-Based Systems

Contract-net protocol

Initiator Participant

cfp

refuse

not−understood

propose

reject−proposal

accept−proposal

failure

inform−ref

inform−done

24 / 25(FIPA Contract Net Interaction Protocol Speci�cation, 2000) 12 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Marginal cost of task

How should a potential contractor decide whether or not to bidfor a task?Idea: marginal cost for task (Sandholm 1999)

Let t be the time, i be the contractor, τ ti be the taskscheduled for i at time t

The endowment of contractor i is ei and an announcementts is made with τ(ts) denoting the task speci�ed by ts

Let cti(τ) denote the cost to agent i of carrying out the setof task τ at time t

Then, the marginal cost of carrying out tasks τ is denoted by:

µi(τ(ts) | τ ti ) = ci(τ(ts) ∪ τ ti )− ci(τ ti )

⇒ di�erence between cost of carrying out old plus new tasksand only carrying out previously agreed tasks

13 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Blackboard architecture

Inspired by human problem solving in a team:�Imagine a group of human specialists seated next to a largeblackboard. The specialists are working cooperatively tosolve a problem, using the blackboard as the workplace fordeveloping the solution.Problem solving begins when the problem and initial data arewritten onto the blackboard. The specialists watch theblackboard, looking for an opportunity to apply theirexpertise to the developing solution. When a specialist �ndssu�cient information to make a contribution, she records thecontribution on the blackboard, hopefully enabling otherspecialists to apply their expertise. This process of addingcontributions to the blackboard continues until the problemhas been solved.� (Corkill, 1991)

14 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Blackboard metaphor

15 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Blackboard architecture characteristics

Eight characteristics of blackboard architectures (Corkill, 1991):

Independence of expertise (I think, therefore I am.)

Diversity in problem-solving techniques (I don't thinklike you do.)

Flexible representation of blackboard information (Ifyou can draw it, I can use it.)

Common interaction language (What'd you say?)

Positioning metrics (You could look it up.)

Event-based activation (Is anybody there?)

Need for control (It's my turn.)

Incremental solution generation (Step by step. . . )

16 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Basic Blackboard System

A basic blackboard system (Corkill, 1991):

KS: static Knowledge SourceKS Activations: Combination of KS knowledge and speci�c�triggering context� ⇒ the proactive �agent�

17 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


Blackboard architecture components

The simple blackboard system consists of two majorcomponents:

1 The blackboard as global structure available to all KSs,serves as:

a community memory of raw input data; partial solutions,alternatives, and �nal solutions; and control informationa communication medium and bu�era KS trigger mechanism

2 Control component as explicit control mechanism,chooses course of action based on state of blackboard andset of triggered KSs

Incremental reasoning style with the solution being built onestep at a time. At each step:

execute any triggered KS

choose a di�erent focus of attention, on the basis of thestate of the solution

18 / 37

MultiagentSystems


Motivation

Task sharing



Coordination


When to use Blackboard Problem-Solving Approach

Situations suitable for the blackboard approach (Corkill, 1991):

diverse, specialized knowledge representations

as an integration framework for heterogeneousproblem-solving representations and expertise (e.g.,diagnostic systems)

software development in teams of independent developers(i.e. as software-engineering metaphor)

When �uncertain knowledge or limited data inhibitsabsolute determination of a solution�

To realize multilevel reasoning or �exible, dynamiccontrol of problem-solving activities

The �Cognitive Agent Architecture � Cougaar� (DARPA) isbased on a blackboard architecture, but not FIPA-compliant. Itis used for the development of applications in military logistics.

19 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Coordination

20 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Methods for coordination

Coordination

Coordination is the process of managing inter-dependenciesbetween agents' activities

Coordination is a special case of interaction in whichagents are aware how they depend on other agents andattempt to adjust their actions appropriately.

Actually this only covers agent-based coordination, butthere can also be centralised mechanisms

In contrast to cooperation, coordination is also necessary innon-cooperative systems (unless agents ignore each other)

21 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Coordination relationship typology

A typology in the context of multiagent planning is provided by(von Martial, 1990):

22 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


von Martial's typology

von Martial's typology distinguishes:

Positive relationships: relationships between two agents'plans for which bene�t will be derived for at least oneagent if plans are combined

Requests: explicitly asking for help with own activities

Non-requested: pareto-like implicit relationships

action equality relationships: su�cient if one agentperforms action both agents needconsequence relationships: side e�ects of agent's planachieve other's goalsfavor relationships: side e�ects of agent's plan make goalachievement for other agent easier

Basic di�erence to traditional computer systems⇒ coordination is achieved at run time rather than designtime

23 / 37

MultiagentSystems


Motivation

Task sharing

Coordination


Partial globalplanning

Jointintentions

Mutualmodeling

Thanks


24 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Partial global planning

Partial global planning (PGP): exchange information to reachcommon conclusions about problem-solving process

Partial ⇒ individual agents don't generate plan for entireproblem

Global ⇒ agents use information obtained from others toachieve non-local view of problem

Three iterated stages:1 Agents deliberate locally and generate short-term plans for

goal achievement2 They exchange information to determine where plans and

goals interact3 Agents alter local plans to better coordinate their activities

Meta-level structure guides the coordination process,dictates information exchange activities

25 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Partial global planning

Central data structure is the partial global plan containing:

Objective: larger goal of the system

Activity maps: describe what agents are doing and theresults of these activities

Solution construction graph: describes how agents shouldinteract and exchange information to achieve larger goal

26 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

PGP extension

The Partial Global Planning framework was extended to theGeneralized PGP (GPGP), which introduces �ve techniquesfor coordinating activities, i.e. strategies for:

updating non-local viewpoints (share all/no/some inform.)

communicating results

handling simple (action) redundancy

handling hard (�negative�) coordination relationships(mainly by means of rescheduling)

handling soft (�positive�) coordination relationships(rescheduling whenever possible, but not �mission critical�)

27 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Joint intentions

Discussed intentions in practical (single-agent) reasoning:

Intentions also provide stability and predictabilitynecessary for social interaction⇒ intentions also signi�cant for coordination, especiallyteamwork

Helps to distinguish between non-cooperative and

cooperative coordinated activity

Basic question:In which way are individual intentions di�erent from (and whatrole do they play in) collective intentions?

Remember Cohen and Levesque's theory of intentions?They extended it to teamwork situations, introducing a notionof �responsibility�

28 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Joint intentions: example

Imagine: We try to lift a stone together, and I discover itwon't work⇒ individually rational behavior: drop the stone!However, this is not really cooperative (we should at leastinform other)Two important notions:

1 commitments (pledges or promises to underpin anintention)

2 conventions (mechanisms for monitoring commitment,mechanics of adopting/abandoning commitments)

Agents commit themselves to actions or states of a�airsCommitments are persistent, i.e. they are not droppedunless special circumstances ariseConventions de�ne these circumstances, e.g., motivationfor goal is no longer present, or it is / can never beachieved

29 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Teamwork-based model of CDPS

Teamwork-based model of CDPS (Wooldridge & Jennings,1994, 1999): a practical model of how CDPS can operate usinga teamwork approach

Stage 1: Recognition of a goal that can be achievedthrough cooperation (e.g. an agent can't do it (e�ciently)on his own)Stage 2: Team formation, i.e. assistance solicitation

if successful, this results in nominal commitment tocollective actiondeliberation phase, ends in agreement on ends (not onmeans)rationality plays a role in deciding whether to form a group

Stage 3: Plan formation involves joint means-endsreasoning, e.g. through negotiation or argumentation

Stage 4: Team action plan of joint action is executed byagents based on mutual convention

30 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Mutual modeling

Basic idea: Putting oneself in the shoes of the other.

Involves modeling others' beliefs, desires, and intentions

. . . and coordinating own actions depending on resultingpredictions

Explicit communication unnecessary

MACE one of the �rst systems (mid 1980s) to useacquaintance models for this purpose

Acquaintance knowledge involves information about othersName unique to every agentClass of group the agent belongs toRoles played by an agent in a classSkills as the capabilities of the modeled agentGoals that the modeled agent wants to achievePlans as the agent's means to achieve an end

Agent explicitly models itself!31 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Norms and social laws

Norms are established patterns of expected behavior, sociallaws often add some authority to that (can be enforced or not)⇒ Balance autonomy with goals of entire society

Features of norms and social laws in terms of conventions forMAS:

conventions make decision making easier for agents

conventions can be designed o�ine (simpler) or emerge

from within the system (more �exible)

Hard to predict at design time which norm will be optimal

Also hard to derive global conventions from agents' localpoint of view

32 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Emergent social norms and laws I

Example: the t-shirt game

agents wear red or blue t-shirt (initially at random), goal is foreveryone to wear the same color

agents are randomly paired in each round of the game, get tosee other's t-shirt color, and then may decide to switch color

Problem: agent must decide which convention to adoptalthough no global information is availablePossible update functions (=decision rules based on history):

Simple majority: agent chooses color observed most often

Simple majority with agent types: agents con�de in certain otheragents and exchange memory with them to inform their decision

Simple majority with communication on success: agents willcommunicate (successful part of) memory if success rate exceeds athreshold

Highest cumulative reward: uses strategy that has had the highestcumulative reward so far

33 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Emergent social norms and laws II

Properties of update functions:

All update functions converged to some convention

Measure: time taken to converge

Memory restarts were investigated to model �new ideas�

But also stability important (we don't want society tochange conventions all the time)

Basic result: for highest cumulative reward rule, for any0 ≤ ε ≤ 1 agents will reach agreement within n roundswith probability 1− εAlso, once reached, the convention will be stable

Convention is e�cient, i.e. it guarantees payo� no worsethan that obtainable from sticking to initial choice

Note that change of norm may be expensive in practice!

34 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

O�ine design

O�ine design is closely related to mechanism designFormally, remembering our agent model Ag : RE → Ac we cande�ne constraints 〈E′, α〉 where:

E′ ⊆ E, a constraint on the environment

α ∈ Ac, actions α as before

A social law is a set of such constraints ⇒ agents/plans arelegal if they never attempt to perform forbidden actions

useful social law problem

Given a set F ⊆ E of focal states (states that should alwaysbe allowed), a �useful social law problem� is to �nd a social lawthat allows agents to legally visit any state in F .

General problem NP-complete, tractable cases �do not appearto correspond to useful real-world cases.�

35 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Summary

Cooperative Distributed Problem Solving (CDPS)

Coherence & Coordination

Task sharing: Contract Net & Blackboard

Coordination

Partial global planning: PGP & GPGP

Joint intention: commitments & conventions

Mutual modeling: Norms, social laws, & conventions

⇒ next time: Multiagent Interactions

36 / 37

MultiagentSystems


Motivation

Task sharing

Coordination



Jointintentions

Mutualmodeling

Thanks

Acknowledgments

These lecture slides are based on the following resources:

Dr. Michael Rovatsos, The University of Edinburghhttp://www.inf.ed.ac.uk/teaching/courses/abs/

abs-timetable.html

Michael Wooldridge: An Introduction to MultiAgent

Systems, John Wiley & Sons, 2nd edition 2009.

Corkill, Daniel D. �Blackboard systems.� AI expert 6.9(1991): 40-47.

37 / 37

http://www.inf.ed.ac.uk/teaching/courses/abs/abs-timetable.html

http://www.inf.ed.ac.uk/teaching/courses/abs/abs-timetable.html

multiagent systems...multiagent system? coherence : how well does the system behave as a unit ?...

Documents