lecture 6 multi-agent systems lecture 6 university “politehnica” of bucarest 2005 - 2006 adina...
Embed Size (px)
TRANSCRIPT

Multi-Agent SystemsLecture 6Lecture 6
University “Politehnica” of Bucarest2005 - 2006
Adina Magda [email protected]
http://turing.cs.pub.ro/blia_06

Working togetherWorking togetherLecture outlineLecture outline
1 Coordination strategies1 Coordination strategies
1 Distributed problem solving1 Distributed problem solving
2 Distributed planning2 Distributed planning
3 An example: Partial global planning3 An example: Partial global planning

3
1 Coordination strategies1 Coordination strategies Coordination Coordination = the process by which an agent reasons about
its local actions and the (anticipated) actions of others to try to ensure the community acts in a coherent manner
CoordinationCoordination
CollectivelyCollectivelymotivated agentsmotivated agentscommon goals
Self-interestedSelf-interestedagentsagents
own goals
Cooperation toCooperation toachieve common goalachieve common goal
Coordination forCoordination forcoherent behaviorcoherent behavior
Neutral to one anotherNeutral to one anotherdisjunctive goals
CompetitiveCompetitiveconflicting goals

4
Perfect coordination ??? Centralized coordination Distributed coordination
Model Protocol Communication
Tightly coupled interactions - distributed search
Cognitive agents – DPS (distributed planning, task sharing, resource sharing)
Heterogeneous agents - interaction protocols: Contract Net, KQML conversations, FIPA protocols
Dynamic interactions – Shared mental states, commitments and conventions
Complex interactions - organizational structure to reduce complexity
Unpredictable interactions - social laws
Conflict of interests - interaction protocols: voting, auctions, bargaining, market mechanisms, extended Contract Net, coalition formation
CooperativeCooperative
Neutral orNeutral orcompetitivecompetitive

5
2. Distributed problem 2. Distributed problem solvingsolving
Distributed planning - the problem to be solved is to design and execute a plan in a distributed manner, by many agents – discussed here
Task and result sharing - an agent has many tasks to do and asks other agents to do some of its tasks; then it should integrate the results – discussed at negotiation

6
3 Distributed planning3 Distributed planning What can be distributed:
The process of coming out with a plan is distributed among agents
Execution is distributed among agents
PlanningPlanning State representation and plan representation Search vs planning
• representation of changes to the world state• representation of and reasoning about the plan
(steps/actions)
Linear planning Partial order planning Hierarchical planning Conditional planning
Planning Planning Search Search

3.1 Planificare monoagent
Operatori de plan – planificare liniara- Actiune.
- Lista Preconditiilor
- Lista Adaugarilor
- Lista Eliminarilor Operatori de plan – planificare neliniara
- Actiune.
- Lista Preconditiilor
- Lista Postconditiilor
7

Operatori de planSTACK(x,y), UNSTACK(x,y), PICKUP(x), PUTDOWN(x)
Predicate:
ON(x,y), ONTABLE(x), CLEAR(x), HOLD(x), ARMEMTY
Axiome:
8
( x) (HOLD(x)) ~ ARMEMPTY
( x) (ONTABLE(x) ~ ( y) (ON(x,y)))
( x) (~ ( y) (ON(y,x)) CLEAR(x))
STRIPS – planificare liniara

Reprezentarea STRIPS Operatori de plan
9
LP:LE: CLEAR(y)HOLD(x)LA: ON(x,y)ARMEMPTY
LP:LE:LA:
PICKUP(x) LP:LE:LA:
PUTDOWN (x) LP:LE:LA:
STACK(x,y) CLEAR(y) HOLD(x)
UNSTACK (x,y) ON(x,y) CLEAR(x) ARMEMPTY ON(x,y) ARMEMPTY
HOLD(x) CLEAR(y)
CLEAR(x) ONTABLE (x) ARMEMPTY ONTABLE(x) ARMEMPTYHOLD(x)
HOLD(x)HOLD(x)
ONTABLE(x) ARMEMPTY

TWEAK – planificare neliniaraReprezentare
10
ActiunePreconditii:Postconditii:
Actiune: PICKUP(x)Preconditii:Postconditii:
STACK(x,y)
CLEAR(y) HOLD(x)ARMEMPTY ON(x,y) CLEAR(y) HOLD(x) ~ ~
CLEAR(x) ONTABLE (x) ARMEMPTY HOLD(x) ONTABLE (x) ARMEMPTY ~ ~

TWEAK – planificare neliniara
11
Operatii de modificare a planului:(1) adaugarea de pasi este operatia prin care se creaza noi pasi care se adauga la plan;(2) promovarea este operatia de stabilire a unei ordonari (temporale) intre doi pasi de plan;(3) legarea simpla este operatia de atribuire de valori variabilelor pentru a valida preconditiile unui pas de plan;(4) separarea este operatia de impiedicare a atribuirii anumitor valori unei variabile;(5) eliminarea destructivitatii este operatia de introducere a unui pas S3 (un pas deja existent in plan sau un pas nou) intre pasii S1 si S2, in scopul de a adauga un fapt invalidat de pasul S1 si necesar in pasul S2.

3.1 Centralized planning for distributed plans3.1 Centralized planning for distributed plans Operators move(b,x,y) movetotable(b,x)
Precond: on(b,x) clear(b) clear(y) Precond: on(b,x) clear(b)Postcond: on(b,y) clear(x) Postcond: on(b,T) clear(x) on(b,x)on(b,x) clear(y)
12
A
B D
C E
F
Si
C
A
E
B F
DSf
I'm BillAgent1
I'm TomAgent2
on(A,B) on(C,D) on(E,F)on(B,T) on(D,T) on(F,T)
on(B,A) on(F,D)on(A,E) on(D,C)on(E,T) on(C,T)
on(B,A)
S1: move(B,T,A)
on(B,T) clear(B) clear(A)
movetotable(A,B) move(A,B,y)
S2: move(A,B,E)
clear(A) clear(E) on(A,B)…………..………….
on(E,T)
S3: movetotable(E,F)
1. Given a goal description,a set of operators,and an initial state descriptiongenerate a partial order plan

S1: move(B,T,A) To satisfy the preconditions, we have:
S2: move(A,B,E) S2 < S1, S3 < S4
S3:movetotable(E,F) S6 < S4, S6 < S5
S4: move(F,T,D) Also
S5: move(D,T,C) S2 threat to S3 S3 < S2
S6: movetotable(C,D) S4 threat to S5 S5 < S4
Then the partial ordering is: S3 < S2 < S1
S6 < S5 < S4
S3 < S4
S3: movetotable(E,F) S2: move(A,B,E) S1: move(B,T,A)
S6: movetotable(C,D) S5: move(D,T,C) S4: move(F,T,D)
Any total ordering that satisfies this partial ordering is a good plan for Agent1
What if we have 2 agents?
DECOMP1
Subplan1 S3 < S2 < S1
Subplan2 S6 < S5 < S4
and S3 < S4
Agent1 S3 < send(clear(F)) < S2 < S1
Agent2 S6 < S5 < wait(clear(F)) < S413
< <
2. Decompose the plan intosubproblems so as to minimizeorder relations across plans
3. Insert synchronization
4. Allocate subplans to agents

S3: movetotable(E,F) S2: move(A,B,E) S1: move(B,T,A)
S6: movetotable(C,D) S5: move(D,T,C) S4: move(F,T,D)
DECOMP2
Subplan1 S3 < S5 < S4
Subplan2 S6 < S2 < S1
and S3 < S2 and S6 < S5
Agent1 S3 < send(on(E,T)) , wait(clear(D)) < S5 < S4
Agent2 S6 < send(clear(D)), wait(on(E,T)) < S2 < S1
Obviously, DECOMP2 has more order relations among subplans than DECOMP1
Therefore, we choose DECOMP1
S3 < send(clear(F)) < S2 < S1
S6 < S5 < wait(clear(F)) < S4
But
then back to DECOMP214
< <
4. If failure to allocate subplans
then redo decomposition (2)
If failure to allocate subplans with
any decomposition
then redo generate plan (1)
5. Execute and monitor subplans
I know howto move only
D, E, F
I know howto move only
A, B, C

2.2 Distributed planning for centralized plans2.2 Distributed planning for centralized plans Each of the planning agents generate a partial plan in parallel
then merge these plans into a global plan parallel to result sharing may involve negotiation
Agent 1 - is specialized in doing movetotable(b,x)
Agent 2 - is specialized in doing move(b,x,y)
Agent 1 - based on Sf it comes out with the partial plan
PAgent1 = { S3: movetotable(E,F) satisfies on(E,T)
S6: movetotable(C,D) satisfies on(C,T)
no ordering }
Agent 2 - based on Sf it comes out with the partial plan
PAgent 2 = { S1: move(B,T,A), S2: move(A,B,E) satisfies on(B,A) on(A,E)
S4: move(F,T,D), S5: move(D,T,C) satisfies on(F,D) on(D,C)
ordering S2 < S1 and S5 < S4 }
Merge PAgent1 with PAgent2 by checking preconditons and threats
Establish thus order S3 < S2, S6 < S5, S3 < S4 + order of PAgent2
Then give any instance of this partial plan to an execution agent to carry it out15

The problem is decomposed and distributed among various planning specialists, each of which proceeds then to generate its portion of the plan
similar to task sharing may involve backtracking
Agent 1 - knows only how to deal with 2-block stacks
Agent 2 - knows only how to deal with 3-block stacks
16
C
A
E
B F
DSf
C
A
E B F
D
A
B D
C E
F
Si

2.3 Distributed planning for distributed plans2.3 Distributed planning for distributed plansa) Plan merginga) Plan merging Agents formulate local plans to satisfy their goals Local plans are exchanged Local plans are combined analyzing for positive and negative interaction Add messages and/or timing commitments to resolve negative plan
interactions and to exploit positive plan interactions
Interacting situations Positive interactions between plans
– redundant actions • static detection: sequencing
– favour actions• dynamic detection: incorporation
Negative interactions between plans– harmful actions– exclusive actions– incompatible actions
17

movehigh(b,x,y)
Precond: have_lifter clear(b) clear(y) on(y,z) z T
Postcond: on(b,y) clear(x) on(b,x) clear(y) free_lifter pick_lifter
Precond: free_lifter
Postcond: have_lifter free_lifter
Agent1: { S1:move(B,T,A) < S2: pick_lifter < S3: movehigh(E,T,B) }
Agent2: { R1:move(C,T,D) < R2: pick_lifter < R3: movehigh(F,T,C) }
18
A B D CEF
Si
D
A
B
E F
CSf
S1
S2
S3
R1
R3
R2
need_l
free_l
B
A
B
CSf1
Negative interactionswhat type?

19
b) Iterative plan formationb) Iterative plan formation build all feasible plans build partial order plans to facilitate plan
merging build abstract plans to be iteratively refined
- see next section and PGP section

20
c) Hierarchical distributed planningc) Hierarchical distributed planning Design plans on several levels of abstraction Use abstract plans Abstract operator - a kind of macro-operator = sequence of
applicable operators
Write paper
Readreferences
Organizeideas
Editcontent
Findeditor
Edittext
Editfigures
Checkfor errors
…..

21
Hierarchical behavior-space search algorithm1. Level 0, Agent_List = {Agent1, …, AgentN}
2. for every Agenti in Agent_List do
2.1 Agenti sends description of Gi and Pi to every Agentj, j=1,N, ji
2.2 Agenti gets Gj, Pj from Agentj, j=1,N, ji
2.3 if Pi is compatible with {Pj}, j=1,N, ji
then Agenti removes itself from Agent_List
3. if Agent_list = { } then exit
4. Be N the new number of agents in Agent_List
5. Sort agents in Agent_List
6. for i=1,N-1, cf. ordering do
6.1 make Agenti the current superior
6.2 Agenti determines conflicts between {Pi}
6.3 if conflicts to be resolved at a lower level
then (a) Level Level + 1
(b) Agent_List = {Agenti+1, …, AgentN}
(c) go to step 2
6.4 send Pi to each Agentj, j=i+1, N
6.5 for j=i+1, N do
- Agentj checks compatibility of Pj with Pi and replan, if nec.
A kind of CSP: Ordering: - what heuristic?
Add exit conditionfor no solution

2.4 Distributed planning and execution2.4 Distributed planning and executionReal world: incomplete and incorrect information
a) Contingency planninga) Contingency planning Conditional planning - deals with incomplete information by
constructing a conditional plan that accounts for each possible situation or contingency that could arrive
sensing actions a context of a plan step, i.e., a union of conditions on the environment that must
hold in order for a step to be executed introduces disjunctive steps + conditional links among plan steps
22
move(A,B,C)
Start
Ask Ag2 to move(A,B,C)
Checkarm(Ag1)
Finish
Negotiate with Ag2for it to achieve move
Context:armbroken(Ag1)
armbroken(Ag1) armbroken(Ag1)
on(B,A)on(A,C)
on(A,B)clear(C)clear(A)A
B C
A
C
B
… Plan to achieve on(B,A)

b) Execution monitoringb) Execution monitoring The agent does not execute the plan with "its eyes closed" - It
monitors what is happening while it executes the plan and it can do replanning to achieve a goal in a new situation
Conditional planning = thinks before to several alternatives Monitoring and replanning = defers the job; I shall see what to do if
new conditions occur
c) Social lawsc) Social laws What actions are legal to be executed in a certain context Find conflicting situations, analyze what concurrent actions lead to
these situations and prohibit such concurrent actions by social laws It is fit, in general, for loosely coupled subproblems / subplans
23

24
3 Partial Global Planning3 Partial Global Planning Initially applied in the Distributed Monitoring VehicleDistributed Monitoring Vehicle (DVM)
Testbed
Extended to be domain independent
Integrates planning and execution
Coordination by means of partial plans exchange
Partial plans: abstract plans + partial ordering plan merging
The domain - unpredictable, unreliable information
The tasks are inherently distributed; each agent performs its own task
The agents are not aware of the global state of the system; however there is a common goal: converge on a consistent map of vehicle movements by integrating the partial tracks formed by different agents into a single complete map or into a consistent set of local maps distributed among agents
Cooperative agents (collectively motivated)

3.1 Aircraft monitoring scenario3.1 Aircraft monitoring scenario each type of aircraft produces a characteristic spectrum of acoustic frequencies
signals may be improperly sensed, there is ghosting and environmental noise
there are two agents A and B whose regions of interest overlap; each agent receives data only about its own region, from its acoustic sensor
the goal is to identify any aircraft that is moving through the region of interest, determine their types and track them through regions
converge on a consistent map of vehicle movements by integrating the partial tracks formed by different agents into a single complete map or into a consistent set of local maps distributed among agents
25Final solutionData input

3.2 Agent functioning3.2 Agent functioning
1. Represent its own expected activity by a set of local (tentative) plans, at two levels: higher level (abstract plans) and detailed level; local plans may involve alternative actions depending on the result of previous actions and changes in the environment
conditional plans; hierarchical plans
2. Communicate abstract local plans to the other agents and get from them such plans another form of communication
3. Model collective activity of the agents by forming Partial Global Plans and finding out how they can be improved for better coordination– identify when the goals of one or more agents can be considered subgoals
of a single global goal partial global goal
– construct a PGP and identify opportunities for improved coordination
– search for an improved PGP
4. Based on 3, propose changes to one or more agents' plans
negotiation
5. Modify its local plan according to the proposal and plan what and when results will be communicated to the other agents 26

2 types of problem-solving activities: task-level activities - build a map of vehicle movements meta-level activities - decide how and with whom to coordinate
Result sharing - agents exchange appropriate results at the right time
Task sharing - allow agents to propose potential plans that involve the transfer of tasks among them
27
A: Process 1/2 data Who?: Process 1/2 data
A: Process 1/3 data B: Process 1/3 data Who?: Process 1/3 data

3.3 Plan representation3.3 Plan representation
A plan represents future activity at two levels of detail: at the higher level it outlines the major steps it expects to take to
achieve its goal - abstract plan at a detailed level it specifies primitive actions to achieve the next
step in the abstract plan; as the plan is executed, new details are added incrementally
actionPrec – preconditions for the actionPost – results of the actionD - the set of data to be processed by the actionP - the set of procedures to be applied to the data
Tstart - the estimated start time of the action
Tend - the estimated end time of the actionabres - an estimate of the characteristics of and confidence in the abstract partial result that will be developed as conclusion of action
28

3.4 PGP formation and coordination3.4 PGP formation and coordination(1) Task decomposition
(2) Local plan formation
(3) Local plan abstraction
(4) Communication about local abstract plans Meta-Level Organization: specifies roles and controls communication For each agent, the MLO specifies:
- the agents it has authority over
- the agents that have authority over it
- the agents that have equal authority
(5) Partial global goal identification Set of operators that generate global goals based on local goals
29

(6) Partial global plan construction and modification partial global goal plan-activity-map = plan actions to be executed
concurrently by itself and the other agents, including costs and expected results of actions – PGP
PGP – hill-climbing algorithm Criteria for rating the actions (eval function):
the action extends a partial result (vehicle tracking hypothesis)
the action produces a partial result that might help some other agents in forming partial results
how long the action is expected to take
30

Algorithm for PGP plan step reordering For the current ordering, rate individual actions and
sum the rating
For each action, examine the later actions for the same agent and find the most highly rated one. If it is higher rated, then swap the actions
If the new ordering is more highly rated than the current one, then replace the current ordering with the new one and Go to step 2
Return the current ordering
31

(7) Communication planning From the modified plan-activity-map, the agent builds a
solution-construction-graph = how the agents should interact, including specifications about what partial results to exchange and when to exchange them
(8) Translate to local level the activities in the revised plan
(9) If authority, send PGP to the other agents
32

ReferencesReferences E.H. Durfee. Distributed problem solving and planning. In Multiagent
Systems - A Modern Approach to Distributed Artficial Intelligence, G. Weiss (Ed.), The MIT Press, 2001, p.121-164.
V.R. Lesser. A retrospective view of FA/C distributed problem solving. IEEE Trans. On Systems, Man, and Cybernetics, 21(6), Nov/Dec 1991, p.1347-1362.
E.D. Durfee, V.R. Lesser Partial global planning: A coordination framework for distributed hypothesis formation. IEEE Trans. On Systems, Man, and Cybernetics, 21(5), Sept. 1991, p.1167-1183.
K.S. Decker, V.R. Lesser. Generalizing the partial global planning algorithm. International Journal of Intelligent Cooperative Information Systems, 1(2), 1992, p. 319-346.
S. Russell, P. Norvig. Artificial Intelligence: A Modern Approach. Prentice hall, 1995, Ch. 11, 12, 13.
33