topic 7: collaborative distributed problem solving

78
Topic 7: Collaborative Distributed Problem Solving application domain type defining a MAS cooperation strategies

Upload: redford

Post on 23-Jan-2016

44 views

Category:

Documents


0 download

DESCRIPTION

Topic 7: Collaborative Distributed Problem Solving. application domain type defining a MAS cooperation strategies. Managing the interdependencies between the activities of agents. e.g. You and I both want to leave the room. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Topic 7: Collaborative Distributed Problem Solving

Topic 7: Collaborative Distributed Problem Solving

application domain type defining a MAS cooperation strategies

Page 2: Topic 7: Collaborative Distributed Problem Solving

The Coordination Problem

Managing the interdependencies between the activities of agents. e.g. You and I both want to

leave the room.

We independently walk towards the door, which can only fit one of us.

I graciously permit you to leave first.

Page 3: Topic 7: Collaborative Distributed Problem Solving

http://media.funmansion.com/funmansion/player/player.php?url=http://media.funmansion.com/content/flv/d4282006/crazy_foreign_traffic.flv

Page 4: Topic 7: Collaborative Distributed Problem Solving

1. Understand application domain type (requirements) application characteristics, requirements, constraints “domain theory”

TOD Task-Oriented Domains WOD Worth-Oriented Domains SOD State-Oriented Domains

2. Define a MAS for the particular system objectives various aspects design issues / considerations interaction situations

3. Define cooperation strategies / examples AGV case examples

1. simple task allocation, task (re)allocation, jointly lifting, traffic mgt, …

Page 5: Topic 7: Collaborative Distributed Problem Solving

1. Understand application domain type- Domain Theory

Application requirements obviously crucial for defining MAS-based software architecture

Domain theory: study of types / domains of MAS

Task Oriented Domains Agents have tasks to achieve Task (re)distribution

State Oriented Domains Goals specify acceptable final states Side effects Joint plan and schedules

Worth Oriented Domains Function rating states’ acceptability Joint plan, schedules, and goal relaxation

Page 6: Topic 7: Collaborative Distributed Problem Solving

1.1 Task Oriented Domains

“Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994)

characteristics a set of agents a set of tasks to achieve a set of resources

agents can achieve tasks without help or interference from each other

however, agents may benefit by sharing some tasks

Page 7: Topic 7: Collaborative Distributed Problem Solving

Task-oriented Domain: Example

Imagine you have 3 children, each of whom needs to be delivered to 3 different schools each morning. Your neighbour has 4 children who also need to be taken to school. Delivery of each child is a task. Assume that one of your children and one of your neighbour’s children both go to the same school. It obviously makes sense for both children to be taken together and only you or your neighbour needs to make the trip.

Page 8: Topic 7: Collaborative Distributed Problem Solving

Task-oriented Domain: Example

Imagine you have 3 children, …

Post menPost OfficePost Office

a

c

d e

21

b

f

Page 9: Topic 7: Collaborative Distributed Problem Solving

Task-oriented Domain: Example

Imagine you have 3 children, … Post men

Database queries Common DatabaseCommon Database

“All female employeeswith more than threechildren.”

2

1

“All female employeesmaking over $50,000 ayear.”

Page 10: Topic 7: Collaborative Distributed Problem Solving

1.2 State-Oriented Domain

State-Oriented Domains actions lead to changes of system state

System objective==> attain a particular system state

either pre-defined/static, either evolving

typical additional requirements (e.g. efficiency, …)

Page 11: Topic 7: Collaborative Distributed Problem Solving

11 22 33

11 22 33

2

1

e.g. slotted blocks worlde.g. furniture moving

State-Oriented Domain

Page 12: Topic 7: Collaborative Distributed Problem Solving

1.3 Worth-Oriented Domains

Worth Oriented Domains actions lead to utility:

function rating states’ acceptability

System objective==> maximize utility

Joint plan, schedules, and goal relaxation

Page 13: Topic 7: Collaborative Distributed Problem Solving

Worth-Oriented Domains

2 22

2

55

34

AB tilehole

obstacle

agents

example: tile world maximize rewards

example: traffic lights maximize traffic throughput

Page 14: Topic 7: Collaborative Distributed Problem Solving

2. Define a MASfor particular system objectives

identify various ‘elements’ of MAS

design issues / considerations

interaction situations

Page 15: Topic 7: Collaborative Distributed Problem Solving

2. Defining a MASfor particular System objectives

identify various aspects in defining the MAS

starting fromsystem objectives / requirements / constraints

define MAS for reaching the objective(s) identify agents identify individual goals identify individual skills (actions, comm, capabilities) identify their relation to resources (insufficient, ...) identify their cooperation

e.g. task allocation, coordination over resources, ...

identify suitable agent architecture

creative

task

Page 16: Topic 7: Collaborative Distributed Problem Solving

2. Defining a MASfor particular System objectives

identify various aspects in defining the MAS

design issues / considerations within application boundaries obviously, i.e. if appl. allows

heterogeneity? all agents can perform tasks, vs. not

splittable tasks? specialization? grouping? distributed planning? task distribution predictiveness (anticipatory vs. in situ)

e.g. for avoiding conflicts over resources e.g. road load, deadlocks, ...

Page 17: Topic 7: Collaborative Distributed Problem Solving

2. Defining a MASfor particular System objectives

identify various aspects in defining the MAS design issues / considerations

interaction situations what is an interaction ?

“dynamic relationship through a set of reciprocal actions” i.e. stem from actions whose consequences have an influence on

the future behaviour of the agents

e.g. agents helping each other data exchange between servers use of a printer by two programs simultaneously

Page 18: Topic 7: Collaborative Distributed Problem Solving

Types of interaction situations

components / criteria for categorization of interaction situations types

compatible and incompatible goals relation to resources agent capacities / skills

Page 19: Topic 7: Collaborative Distributed Problem Solving

Components of interaction situations

compatible and incompatible goals concurrent goals ? or contradictory or even opposed ?

incompatible antagonist situation achieving one goal other goal cannot be achieved

compatible cooperation situation

Page 20: Topic 7: Collaborative Distributed Problem Solving

Components of interaction situations …

relation to resources resources: environmental elements which can be used in carrying out an

action objects, tools, space, time, …

limited resources conflicts agents needing the same tool at the same place in the same time programs sharing a CPU vehicles on busy roads

conflict resolution many methods

law of the strongest negotiation

e.g. insurances and court houses to resolve conflicts of interest due to accident

coordinating actions devices, regulations, supplementary actions e.g. traffic lights / highway code / … to avoid conflicts

resource spaceresources needed by A

resources needed by B

Page 21: Topic 7: Collaborative Distributed Problem Solving

Components of interaction situations

capacities of agents in relation to tasks can an agent carry out a task alone ? or does it need others ?

particular tasks can be carried out either by a single agents

e.g. moving a block, doing a calculation, reading a file, … better by several agents / only by several agents

e.g. heavy blocks, different expertise, …

beneficial interactions

result can be more than sum of parts

Page 22: Topic 7: Collaborative Distributed Problem Solving

Goals Resources(each has)

Skills Types of situation Involvement

Compatible Sufficient Sufficient Independence Indifference

Compatible Sufficient Insufficient Simple collaboration  

Compatible Insufficient Sufficient Obstruction Cooperative

Compatible Insufficient Insufficient Coordinated collaboration

 

Incompatible Sufficient Sufficient Pure individual competition

 

Incompatible Sufficient Insufficient Pure collective competition

Antagonism

Incompatible Insufficient Sufficient Individual conflict over resources

 

Incompatible Insufficient Insufficient Collective conflict over resources

 

Types of interaction situations

Page 23: Topic 7: Collaborative Distributed Problem Solving

Types of interaction situations:Examples

Compatible goals, insufficient resources, sufficient skills

obstruction e.g. motorway queue

Incompatible goals, insufficient resources, sufficient skills

individual conflict over resources e.g. only one liter of water, promotion, ...

...

Page 24: Topic 7: Collaborative Distributed Problem Solving

3. Cooperation strategies - illustrated in AGV case

application domain type task-oriented domain

defining a MAS AGV agents, transport agents individual goals

AGV agents: perform tasks transport agents: ensure tasks are performed

skills AGV agents: jobs/actions/communication/… transport agents: communication/…

resources warehouse (lanes, stock) communication medium

cooperation …

Page 25: Topic 7: Collaborative Distributed Problem Solving

3. Cooperation strategies - illustrated in AGV case (cont.)

interaction situations (in)compatible goals (in)sufficient skills (in)sufficient resources

Page 26: Topic 7: Collaborative Distributed Problem Solving

3. Cooperation strategies - illustrated in AGV case (cont.)

1. (simple) task allocation CNET Gradient fields DynCNET

2. task (re)distribution negotiation

3. resource allocation - traffic management CNET

4. collision avoidance coordination protocol hull projection

5. joint action - jointly lift a (heavy) pallet joint intentions

6. batch order management1. team formation2. distributed planning

Page 27: Topic 7: Collaborative Distributed Problem Solving

3.1 Simple task allocation

situationa task consists of

- a pick job (pick up pallet)- a move job (move towards destination)- a drop job (place pallet on target location)

a task can be executed by any AGVa task needs to be assigned / allocated to an AGV

characteristics“delayed commencement”dynamiccommunication overhead?

Page 28: Topic 7: Collaborative Distributed Problem Solving
Page 29: Topic 7: Collaborative Distributed Problem Solving

Simple task allocation (cont.)

situation

collaboration approaches

“Classic approach”: agents coordinate byexchanging messages

Contract NetDynCNet

Exploit the agent environment

Gradient Fields

Page 30: Topic 7: Collaborative Distributed Problem Solving

Simple task allocation (cont.) situation

collaboration approaches1. Contract Net

transport agent issues request for bids(local broadcast)

if no response: stronger signalAGV agents can place bids

based oncurrent location (distance)other tasksbattery level…

transport agent assigns task to best bid

characteristics…

Page 31: Topic 7: Collaborative Distributed Problem Solving

Simple task allocation (cont.) situation

collaboration approaches

1. Contract Net…

characteristics

communication overhead limited

does not take dynamics/opportunities into account

no rejection of assigned task

no re-allocation if better-suited AGV becomes available

issues

how should an AGV agent deal with multiple concurrent requests?

bid? value of bid? consequences?

ok approach for indiviual tasks, what for large numbers of tasks?

one task at a time

Page 32: Topic 7: Collaborative Distributed Problem Solving

Simple task allocation (cont.) situation

collaboration approaches2. Dynamic Contract Net (DynCNET)

contract net, but taking into account dynamics

characteristicscommunication overheadcan take dynamics/opportunities into account

rejection of assigned taskre-allocation if better-suited AGV becomes available

issueshow should an AGV agent deal with multiple concurrent requests?

bid? value of bid? consequences?ok approach for indiviual tasks, what for large numbers of tasks?one task at a time

Page 33: Topic 7: Collaborative Distributed Problem Solving

New Task Becomes Available

Page 34: Topic 7: Collaborative Distributed Problem Solving

New AGV Becomes Available

Page 35: Topic 7: Collaborative Distributed Problem Solving

DynCNET = Dynamic Contract Net

Page 36: Topic 7: Collaborative Distributed Problem Solving

Simple task allocation (cont.) situation

collaboration approaches3. Gradient Fields

coordination through the environment

physical environment Restricts how agents can exploit the environment

agent environment virtual environment layer enables agents to share information and coordinate their

behavior

characteristicscfr. DynCNETreduced complexity of AGV agent!

Page 37: Topic 7: Collaborative Distributed Problem Solving

Agent Environment for Agents

Page 38: Topic 7: Collaborative Distributed Problem Solving

Exploit the Agent Environment Task Assignment (Field-Based Approach)

Page 39: Topic 7: Collaborative Distributed Problem Solving

Exploit the Agent Environment

Task Assignment

Page 40: Topic 7: Collaborative Distributed Problem Solving

Task Allocation experiments -Test Setting AGV System

AGV system “fish handling” (real scenario) 134 x 134 m; 56 pick/drop locations 14 AGVs; 0.7m/s; load manipulation 5s 140 transports/hour (standard test profile)

Tests Communication load Average waiting time Stress test: perform fixed number of transports as quick as possible

(e.g. handle the arrival of a truck with loads)

Page 41: Topic 7: Collaborative Distributed Problem Solving

Communication Load Average Waiting Time

Test Results

Page 42: Topic 7: Collaborative Distributed Problem Solving

Finished Transports in Stress Test

Page 43: Topic 7: Collaborative Distributed Problem Solving

Two Qualities: Flexibility and Robustness

Flexibility (adapt to changes in the environment) FiTA: implicitly represented in the fields DynCNET: explicit points of choice

Robustness (to message loss) FiTA: graceful degradation DynCNET: requires additional functionality

Page 44: Topic 7: Collaborative Distributed Problem Solving

3.2 Task (re)distribution

situationAGV agents have several tasks assigned to thema set of AGV agents could redistribute tasks amongst

each other for better results

e.g. after using several CNET protocols, the following assignmentexists:

AGV1: task 1 (NE corner), task 2 (SW corner) AGV2: task 3 (NE corner), task 4 (SW corner)

redistributing the tasks could seriously enhance task execution time

AGV1: task 1 (NE corner), task 3 (NE corner) AGV2: task 2 (SW corner), task 4 (SW corner)

Page 45: Topic 7: Collaborative Distributed Problem Solving

Task (re)distribution (cont.)

situation

typical situation for negotiation in Task-Oriented Domains…

Page 46: Topic 7: Collaborative Distributed Problem Solving

Task (re)distribution (cont.)

Task-Oriented Domains (TODs) Defined

a TOD is a triple<T, Ag, c>

where T is the (finite) set of all possible tasks Ag = {1,…,n} is the set of participating agents c = (T) + defines the cost of executing each subset of tasks

an encounter is a collection of tasks<T1,…,Tn>

where Ti T for each i Ag

Page 47: Topic 7: Collaborative Distributed Problem Solving

Deals in TODs

given encounter <T1, T2>, a deal is an allocation of the tasks T1 T2 to the agents 1 and 2

the cost to i of deal = <D1, D2> is c(Di), and will be denoted costi()

the utility of deal to agent i is:utilityi() = c(Ti ) – costi()

the conflict deal, , is the deal <T1, T2> consisting of the tasks originally allocated.note that utilityi() = 0 for all i Ag

deal is individual rational if it weakly dominates the conflict deal

Page 48: Topic 7: Collaborative Distributed Problem Solving

Negotiation

”The process of several agents searching for an agreement”

Reaching consensus

”Rules of Encouter” by Rosenchein and Zlotskin, 1994

Page 49: Topic 7: Collaborative Distributed Problem Solving

Complexity of Negotiations

Some attributes that make the negotiation process complex are:

Multiple attributes: Single attribute (price) – symmetric scenario. Multiple attributes – several inter-related attributes, e.g. buying a

car.

The number of agents and the way they interact: One-to-one, e.g. single buyer and single seller. Many-to-one, e.g. multiple buyers and a single seller, auctions. Many-to-many, e.g. multiple buyers and multiple sellers.

Page 50: Topic 7: Collaborative Distributed Problem Solving

Negotiation Components

Any negotiation setting will have 4 components: Negotiation set: represents the space of possible proposals

that agents can make

Protocol: defines the legal proposals that agents can make

Collection of strategies: (one for each agent) determines what proposals the agent will make

Rule: to determine when an agreement has been reached

Page 51: Topic 7: Collaborative Distributed Problem Solving

The Negotiation Set

the set of deals over which agents negotiate are those that are: individual rational pareto efficient

Page 52: Topic 7: Collaborative Distributed Problem Solving

The Negotiation Set Illustrated

Page 53: Topic 7: Collaborative Distributed Problem Solving

Mechanisms, Protocols, Strategies

Negotiation is governed by a mechanism or a protocol: defines the ”rules of encounter” between the agents the public rules by which the agents will come to

agreements.

Given a particular protocol, how can a particular strategy be designed that individual agents can use?

Page 54: Topic 7: Collaborative Distributed Problem Solving

Monotonic Concession Protocol

Negotiation proceeds in rounds

On round 1, agents simultaneously propose a deal from the negotiation set.

Agreement is reached if one agent finds that the deal proposed by the other agent is at least as good or better than its proposal.

Ai best deal Aj best deal

Page 55: Topic 7: Collaborative Distributed Problem Solving

Monotonic Concession Protocol

If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals.

In round u+1, no agent is allowed to make a proposal that is less preferred by the other agent than the deal proposed at time u.

If neither agent concedes, then negotiation terminates with a conflict deal.

Page 56: Topic 7: Collaborative Distributed Problem Solving

Monotonic Concession Protocol

Advantages: Symmetrically distributed (no agent plays a special role) Ensures convergence It will not go on indefinitely

Disadvantages: Agents can run into conflicts Inefficient – no quarantee that an agreement will be reached

quickly

Page 57: Topic 7: Collaborative Distributed Problem Solving

Key Questions

3 key questions to be answered: What should an agent’s first proposal be?

It’s most preferred deal.

On any given round, who should concede?The agent least willing to risk conflict.

If an agent concedes, then how much should it concede?Just enough to change the balance of risk.

Page 58: Topic 7: Collaborative Distributed Problem Solving

The Risk Factor

One way to think about which agent should concede is to consider how much each has to loose by running into conflict at that point.

Ai best deal Aj best deal

Conflict deal

How much am I willing to risk a conflict?

Maximum loss from conflict

Maximum loss from concession

Page 59: Topic 7: Collaborative Distributed Problem Solving

The Zeuthen Strategy

Uses the risk evaluation strategy

Suppose you have conceded a lot. Then: Your proposal is now close to conflict deal. You are more willing to risk conflict.

An agent will be more willing to risk conflict if the difference in utility between its current proposal and the conflict deal is low.

Degree of willingness to risk a conflict can be defined as:

utility i loses by conceding and accepting j’s offerutility i loses by not conceding and causing conflictRiskt

i =

Page 60: Topic 7: Collaborative Distributed Problem Solving

Nash Equilibrium…

the Zeuthen strategy is in Nash equilibrium under the assumption that one agent is using the strategy the

other can do no better than use it himself…

interesting to the designer of automated agents publicly known strategy

Page 61: Topic 7: Collaborative Distributed Problem Solving

About MCP and Zeuthen Strategies

Advantages: Simple and reflects the way human negotiations work. Stability – in Nash equilibrium – if one agent is using the

strategy, then the other can do no better than using it him/herself.

Disadvantages: Computationally expensive – players need to compute the entire

negotiation set. Communication burden – negotiation process may involve

several steps.

Page 62: Topic 7: Collaborative Distributed Problem Solving

3.3 Resource allocation - traffic management (cont.)

situation

approach

negotiation protocols

delegate MAS - see later

Page 63: Topic 7: Collaborative Distributed Problem Solving

3.4 Collision avoidance

situation

AGV autonomously follow their own routes towards

source / target locations

warehouse layout may contain locations where

only 1 AGVs can pass at a time

e.g. cross roads, close lanes

Page 64: Topic 7: Collaborative Distributed Problem Solving

Collision avoidance (cont.)

situation

approaches collision avoidance communication protocols

many possibilities local decision e.g. based on priority rules (priority to the right) CNET for bidding on ‘critical section’

quite complex in non-trivial situations n vehicles complex warehouse layouts dynamic set of AGVs to take into account needs to be integrated seamlessly in agent architecture… deadlock?

Page 65: Topic 7: Collaborative Distributed Problem Solving

Collision avoidance (cont.)

situation

approaches collision avoidance communication protocols

hull projection environment as controlling entity environment mediates between AGV requests to use a lane

more easily extensible for more vehicles / complex layouts more easily integrated in agent architecture

Page 66: Topic 7: Collaborative Distributed Problem Solving

Hull projection through the virtual environment

Page 67: Topic 7: Collaborative Distributed Problem Solving

3.5 Joint action- jointly lift a (heavy) pallet

situation heavy pallet needs to be transported heavy = requires two AGVs

AGVs need to agree on this AGVs need to stick to the agreement

an AGV cannot suddenly drop the pallet because it sees another opportunity (e.g. another job)

Page 68: Topic 7: Collaborative Distributed Problem Solving

Joint action - jointly lift a (heavy) pallet (cont.)

situation

approach joint intentions (Jennings, Tambe)

Recall intentions important for individual practical reasoning

Now: intentions also important for coordination distinguish

individual intentions (that may be coordinated) from intentions to cooperatively and coordinatedly achieve a goal as a team

commitment associated with an intention future directed, persitent, should not be dropped for no reason conventions exist that regulate when it‘s appropriate to drop an intention e.g. when lifting a heavy pallet together

Page 69: Topic 7: Collaborative Distributed Problem Solving

Coordination through joint intentions (cont.)

Agents need individual commitments, and joint commitment to overall goal

State of joint commitment is distributed over agents

Convention regulates e.g. when joint commitment may be dropped and how other agents need to be informed about change of mind.

More formally: Joint persistant goals (JPGs)

JPG = (goal φ, motivation for goal ψ)

e.g.: φ = „having heavy object lifted onto truck“; ψ = „later transportation“

Page 70: Topic 7: Collaborative Distributed Problem Solving

Conventions next to intentions

Initially each agent believes φ has not been satsified, and believes possibleToDo φ

Until termination condition is met, each agent has goal φ (more precisely intention φ)

Termination condition: It is mutually believed that either φ is satisfied φ is impossible ψ is no longer valid

Page 71: Topic 7: Collaborative Distributed Problem Solving

Until termination condition is met:

if an agent believes that either the goal is met or the goal is impossible or the motivation no longer holds

it has the goal to make this mutually believed (the goal to convince all others about that)

Page 72: Topic 7: Collaborative Distributed Problem Solving

Examples for JPG-like architectures: ARCHON (Jennings e.a.) Steam (Tambe)

Both encode conventions formally as rules, thus integrating them into the „normal“ reasoning process

Page 73: Topic 7: Collaborative Distributed Problem Solving

3.6 Batch order handling

situation peek load in warehouse e.g. several loaded trucks need to be cleared

10s of pallets need to be transportedsome pallets should stay together (e.g. same colour

material)

complex tasks may need to be decomposed? stack of pallets --> smaller stacks or individual pallets

narrow unload location - coordination required

humans would make a team for efficiency? unloading chain?

Page 74: Topic 7: Collaborative Distributed Problem Solving

Batch order handling (cont.)

situation approach

organisation-based approach coordinated planning

Page 75: Topic 7: Collaborative Distributed Problem Solving

Batch order handling (cont.)

situation approach

organisation-based approach inspired by human organisations define roles, assign/negotiate about roles commitment

e.g. action chain roles:

head of chaintail of chainbody of chaintransport

head (unload)

body (pass on)

body (pass on)

tail (drop off)

transport

transport

Page 76: Topic 7: Collaborative Distributed Problem Solving

Batch order handling (cont.)

situation approach

organisation-based approach

coordinated planning centralized (locally or globally)

plan building and distribution plan merging

distributed (G)PGP - (Generalized) Partial Global Planning (Durfee e.a.) SharedPlans (Grosz, Kraus)

Page 77: Topic 7: Collaborative Distributed Problem Solving

Coordinated planning Centralized Planning

plan building and distribution plan merging

each agent makes local, partial plan (PP) the PPs are communicated to central coordinator

it resolves the conflicts and generates a conflict-free global plan (GP). GP is communicated to all agents. e.g. Georgeff in AAAI conference 1983 (National Conf on AI)

Distributed Planning no central controller each agent has some knowledge or model of other agents It has the capability to piece the PPs together into an agreeable GP e.g. Victor Lesser and group (University of Massachusettes, 1981)

Critique: agents share and process a huge amount of information requires lots of computing and communication resources centralised approaches introduce a single point of failure sometimes plans can also become obsolete very quickly

Page 78: Topic 7: Collaborative Distributed Problem Solving

Conclusion

collaborative distributed problem solving at the heart of multi-agent systems

just a small sample of possible situations / cooperation strategies here …

distributed problem solving solves problems but creates new challenges many coordination situations many possible solutions

in the end: integration in one agent architecture coordination and coherence !!

coordinationManaging inter-dependencies between the activities of agents

coherence:“how well the MAS behaves as a unit along some dimension

of evaluation” (Bond et al.)