Multi-Agent Systems

University “Politehnica” of BucarestSpring 2011

Adina Magda Florea

http://turing.cs.pub.ro/mas_11curs.cs.pub.ro

Lecture 2: Agent architectures

Simple agent models Cognitive agent architectures Reactive agent architectures Layered architectures

An agent perceives its environment through sensors and acts upon the environment through effectors.

Aim: design an agent program = a function that implements the agent mapping from percepts to actions.

Agent = architecture + program

A rational agent has a performance measure that defines its degree of success.

A rational agent has a percept sequence and, for each possible percept sequence the agent should do whatever action is expected to maximize its performance measure.

3

1. Simple agent models

E = {e1, .., e, ..}P = {p1, .., p, ..}A = {a1, .., a, ..}

Reactive agentsee : E Paction : P Aenv : E x A E(env : E x A P(E))Environment

env (E)

Decisioncomponent

action

Executioncomponent

action

Perceptioncomponent

see

Agent

4

P A

Reactive agent modelReactive agent model

Several reactive agentsseei : E Pi

actioni : Pi Ai

env : E x A1 x … An P(E)

Environmentenv

Decisioncomponent

action

Executioncomponent

action

Perceptioncomponent

see

Agent (A1)

Agent (A2)

Agent (A3)

5

A1,…, Ai,..P1,…, Pi,..(usually the same)

Reactive agent modelReactive agent model

E = {e1, .., e, ..}P = {p1, .., p, ..}A = {a1, .., a, ..}S = {s1, .., s, ..}

State Agentsee : E Pnext : S x P Saction : S Aenv : E x A P(E)Environment

env (E)

Decisioncomponent

action, next

Executioncomponent

action

Perceptioncomponent

see

Agent

6

P A

S

Cognitive agent modelCognitive agent model

Several cognitive agentsseei : E Pi

nexti : Si x P Si

actioni : Si x I Ai

interi : Si I env : E x A1 x … An P(E)

Environmentenv

Decisioncomponent

action, next

Executioncomponent

action

Perceptioncomponent

see

Agent (A1)

Agent (A2)

Agent (A3)

Interactioncomponent

inter

7

S1,…, Si,..A1,…, Ai,..P1,…, Pi,..(not always the same)

I = {i1, .., ik,…}

Cognitive agent modelCognitive agent model

Agent with states and goals

goal : E {0, 1}

Agents with utility

utility : E R

Nondeterministic environment

env : E x A P(E)

Utility theory = every state has a degree of usefulness, to an agent, and that agent will prefer states with higher utility

Decision theory = an agent is rational if and only if it chooses the actions that yields the highest expected utility, averaged over all possible outcomes of actions - Maximum Expected Utility 8

Cognitive agent modelCognitive agent model

The expected probability that the result of an action (a) executed in e to be the new state e’

The expected utility of a action a in a state e, from the point of view of the agent

Maximum Expected Utility (MEU)

prob(ex(a, e) e') 1e'env( e,a)

9

Cognitive agent modelCognitive agent model

),('

)'(*)'),((),(aeenve

eutilityeeaexprobeaU

Example: getting out of a maze

– Reactive agent

– Cognitive agent

– Cognitive agent with utility

3 problems:

– what action to choose if several available

– what to do if the outcomes of an action are not known

– how to cope with changes in the environment10

2. Cognitive agent architectures

Rational behaviour: AI and Decision theory AI = models of searching the space of possible actions

to compute some sequence of actions that will achieve a particular goal

Decision theory = competing alternatives are taken as given, and the problem is to weight these alternatives and decide on one of them (means-end analysis is implicit in the specification of competing alternatives)

Problem = deliberation/decision– The agents are resource bounded

– The environment may change during deliberation

11

General cognitive agent architectureGeneral cognitive agent architecture

12

Information aboutitself- what it knows- what it believes- what is able to do- how it is able to do- what it wantsenvironment andother agents- knowledge- beliefs

Communication

Interactions

Control

Output

Input

Otheragents

Environment

Scheduler&Executor State

Planner

Reasoner

FOPL models of agency Symbolic representation of knowledge + use inferences in FOPL -

deduction or theorem proving to determine what actions to execute

Declarative problem solving approach - agent behavior represented as a theory T which can be viewed as an executable specification

(a) Deduction rules

At(0,0) Free(0,1) Exit(east) Do(move_east)

Facts and rules about the environment

At(0,0) Wall(1,1) x y Wall(x,y) Free(x,y)

Automatically update current state and test for the goal state

At(0,3) or At(3,1)

(b) Use situation calculus =describe change in FOPL

Function Result(Action,State) = NewState

At((0,0), S0) Free(0,1) Exit(east) At((0,1), Result(move_east,S0))

Try to prove the goal At((0,3), _) and determines actions that lead to it

- means-end analysis 13

Advantages of FOPL

- simple, elegant

- executable specifications

Disadvantages

- difficult to represent changes over time

other logics

- decision making is deduction and selection of a strategy

- intractable

- semi-decidable

14

BDI (Belief-Desire-Intention) architectures

High-level specifications of an architecture for a resource-bounded agent.

Beliefs = information the agent has about the world

Desires = state of affairs that the agent would wish to bring about

Intentions = desires the agent has committed to achieve

BDI - a theory of practical reasoning - Bratman, 1988

Intentions play a critical role in practical reasoning - limits options, DM simpler

15

16

BDI particularly compelling because: philosophical component - based on a theory of

rational actions in humans software architecture - it has been implemented and

successfully used in a number of complex fielded applications– IRMA (Intelligent Resource-bounded Machine Architecture)– PRS - Procedural Reasoning System

logical component - the model has been rigorously formalized in a family of BDI logics– Rao & Georgeff, Wooldrige– (Int i ) (Bel i )

17

BDI ArchitectureBelief revision

BeliefsKnowledge

Deliberation process

percepts

DesiresOpportunityanalyzer

Intentions

Filter

Means-endreasonner

Intentions structured in partial plans

PlansLibrary of plans

Executor

B = brf(B, p)

I = options(D, I)

I = filter(B, D, I)

= plan(B, I)

actions

BDI Agent control loopBDI Agent control loop

B = B0 I = I0 D = D0

while true do

get next perceipt p

B = brf(B,p)

D = options(B, D, I)

I = filter(B, D, I)

= plan(B, I)

execute()

end while18

Commitment strategies The option chosen by the agent as an

intention = the agent is committed to that intention

Commitments imply temporal persistence of intentions

Question: How long is an agent committed to an intention

Blind commitment Single minded commitment Open minded commitment

19

B = B0

I = I0 D = D0

while true do

get next perceipt p

B = brf(B,p)

D = options(B, D, I)

I = filter(B, D, I)

= plan(B, I)

while not (empty() or succeeded (I, B)) do

= head()

execute()

= tail()

get next perceipt p

B = brf(B,p)

if not sound(, I, B) then

= plan(B, I)

end while

end while 20

Reactivity, replan

BDI Agent control loopBDI Agent control loopBlind commitment

B = B0

I = I0 D = D0

while true do

get next perceipt p

B = brf(B,p)

D = options(B, D, I)

I = filter(B, D, I)

= plan(B, I)

while not (empty() or succeeded (I, B) or impossible(I, B)) do

= head()

execute()

= tail()

get next perceipt p

B = brf(B,p)

if not sound(, I, B) then

= plan(B, I)

end while

end while 21

Reactivity, replan

Dropping intentions that are impossibleor have succeeded

BDI Agent control loopBDI Agent control loopSingle minded commitment

B = B0

I = I0 D = D0

while true doget next perceipt pB = brf(B,p)D = options(B, D, I)I = filter(B, D, I) = plan(B, I)while not (empty() or succeeded (I, B) or impossible(I, B)) do

= head()execute()

= tail()get next perceipt pB = brf(B,p)

D = options(B, D, I) I = filter(B, D, I)

= plan(B, I)end while

end while 22

Replan

if reconsider(I, B) then

BDI Agent control loopBDI Agent control loopOpen minded commitment

BDI architecture – most popular There is no unique BDI architecture

PRS - Procedural Reasoning System (Georgeff) dMARS UMPRS si JAM – C++ JADE (http://jade.tilab.com/ ) JACK (http://aosgrp.com/index.html)

JADEX – XML si Java (http://vsis-www.informatik.uni-hamburg.de/projects/jadex/) JASON – Java (http://jason.sourceforge.net/)

23

3. Reactive agent architectures

Subsumption architecture - Brooks, 1986 DM = {Task Accomplishing Behaviours} A TAB is represented by a competence module (c.m.) Every c.m. is responsible for a clearly defined, but not

particular complex task - concrete behavior The c.m. are operating in parallel Lower layers in the hierarchy have higher priority and are

able to inhibit operations of higher layers The modules located at the lower end of the hierarchy are

responsible for basic, primitive tasks The higher modules reflect more complex patterns of

behaviour and incorporate a subset of the tasks of the subordinate modules subsumtion architecture

24

Module 1 can monitor and influence the inputs and outputs of Module 2

M1 = wonders about while avoiding obstacles M0M2 = explores the environment looking for

distant objects of interests while moving around M1

Incorporating the functionality of a subordinated c.m. by a higher module is performed using suppressors (modify input signals) and inhibitors (inhibit output) 25

CompetenceModule (1)Move around

CompetenceModule (0)Avoid obstacles

Inhibitor nodeSupressor node

Sensors

CompetenceModule (2)Investigate env

CompetenceModule (0)Avoid obstacles

EffectorsCompetenceModule (1)Move around

Input(percepts)

Output(actions)

More modules can be added:• Replenishing energy• Optimising paths• Making a map of territory• Pick up and put down objects• Investigate environment• Move around• Avoid obstacles

Behavior (c, a) – pair of condition-action describing behavior

Beh = { (c, a) | c P, a A} R = set of behavior rules

R x R - binary inhibition relation on the set of behaviors, total ordering of R

function action( p: P)var fired: P(R)begin

fired = {(c, a) | (c, a) R and p c}for each (c, a) fired doif (c', a') fired such that (c', a') (c, a) then return areturn null

end 26

1990 - Brooks extends the architecture to cope with a large number of c.m. - Behavior Language

Indirect communication

Advantages of reactive architectures

Disadvantages

27

4. Layered agent architectures

Combine reactive and pro-active behavior At least two layers, for each type of behavior

Horizontal layering - i/o flows horizontally

Vertical layering - i/o flows vertically

29

Layer n

…

Layer 2

Layer 1

perceptualinput

Actionoutput

Layer n

…

Layer 2

Layer 1

Layer n

…

Layer 2

Layer 1

Actionoutput

Actionoutput

perceptualinput

perceptualinput

HorizontalVertical

Layered agent architectures

TouringMachine Horizontal layering - 3 activity producing layers, each layer produces

suggestions for actions to be performed reactive layer - set of situation-action rules, react to precepts from the

environment planning layer

- pro-active behavior

- uses a library of plan skeletons called schemas

- hierarchical structured plans refined in this layer modeling layer

- represents the world, the agent and other agents

- set up goals, predicts conflicts

- goals are given to the planning layer to be achieved Control subsystem

- centralized component, contains a set of control rules

- the rules: suppress info from a lower layer to give control to a higher one

- censor actions of layers, so as to control which layer will do the actions30

InteRRaP Vertically layered two pass agent architecture Based on a BDI concept but concentrates on the dynamic control

process of the agent

Design principles the three layered architecture describes the agent using various

degrees of abstraction and complexity both the control process and the KBs are multi-layered the control process is bottom-up, that is a layer receives control

over a process only when this exceeds the capabilities of the layer beyond

every layer uses the operations primitives of the lower layer to achieve its goals

Every control layer consists of two modules:

- situation recognition / goal activation module (SG)

- planning / scheduling module (PS) 31

32

Cooperativeplanning layer SG

SG PS

SG PS

PS

Localplanning layer

Behaviorbased layer

World interface Sensors Effectors Communication

Social KB

Planning KB

World KB

actions percepts

InteRRaP

33

Beliefs

Social model

Mental model

World model

Situation

Cooperative situation

Local planning situation

Routine/emergency sit.

Goals

Cooperative goals

Local goals

Reactions

Options

Cooperative option

Local option

Reaction

Operational primitive

Joint plans

Local plans

Behavior patterns

Intentions

Cooperative intents

Local intentions

Response

Sensors

Effectors

BDI model in InteRRaPBDI model in InteRRaP

filter

plan

options

PS

SG

Muller tested InteRRaP in a simulated loading area. A number of agents act as automatic fork-lifts that move in the loading

area, remove and replace stock from various storage bays, and so compete with other agents for resources

34