topic 4: agent architectures general agent architectures deductive reasoning agents practical...

Topic 4: Agent architectures

general agent architectures

deductive reasoning agents practical reasoning agents reactive agents hybrid agents

Agents: definition

M. Wooldridge

An agent is a computer system … … that is situated in some environment, … … capable of autonomous action in this environment … … in order to meet its design objectives.

environment

actionobservationagent

agent properties

reactivityreacts to stimuli (changes in env., communication, …)

autonomydoes not require user interaction

pro-activenessaims to achieve its own goals, therefore initiates appropriate actions

social abilitycooperates / coordinates / communicates / …

embodiedsituated in the environment

mobilemoves around network sites

learninglearn from past experiences

…

essential

extra

Agents versus Objects

Objects (Java / C++ / C# / Smalltalk / Eiffel / …)

encapsulate state “attributes” / “data members” / … behaviour “operations” / “methods” / …

represent real-world entity own responsibility

Agents versus Objects (cont.) differences

autonomy: who decides to execute a particular “action”

objects have control over state (through operations)objects do not have control over their behaviour

any object can call any public operation of the object object cannot decide when to execute its behaviour

agents request an action from another agent […] control lies entirely within receiving agent

cfr. humans “objects do it for free, agents do it for money”

because they want to”

action

agent 1 agent 2

Agents versus Objects (cont.)

differences (cont.) … behaviour architecture: integration of flexible autonomous behaviour

objects operations to offer behaviour

agents integrate

reactive behavioursocial behaviourproactive behaviour…

cfr. humans

agent

Agents versus Objects (cont.)

differences (cont.) … inherently multi-threaded

objects no separate thread of control … active objects

agents conceptually different threads of control

agent

Agents versus Expert Systems

Expert systems e.g. MYCIN act as computerized consultant for physicians

MYCIN knows about blood diseases in humans a wealth of knowledge about blood diseases, in the form of rules a doctor can obtain expert advice about blood diseases by giving MYCIN

facts, answering questions, and posing queries

differences

inherently disembodied do not operate in an environment

no reactive/proactive behaviour user-controlled

no social behaviour as in cooperation / coordination / negotiation / …

Agent architecture:how to do the right thing ?

Pattie Maes [1991]

‘[A] particular methodology for building [agents]. It specifies how . . . the agent can be decomposed into the construction of a set of component modules and how these modules should be made to interact. The total set of modules and their interactions has to provide an answer to the question of how the sensor data and the current internal state of the agent determine the actions . . . and future internal state of the agent.’

model of agent machinery

abstract architecture

elements

E set of environment states Ac set of actions

Ag: E* Ac (mapping)

agents with state

I internal agent state

see E Per

action I Ac

next I x Per I

environment

actobserve

agent

see action

nextstate

Concrete agent architectures

1. Deductive reasoning agents 1956 – present “Agents make decisions about what to do via symbol manipulation. Its purest

expression, proposes that agents use explicit logical reasoning in order to decide what to do.”

2. Practical reasoning agents 1990 – present “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs /

desires / intentions.”

3. Reactive agents 1985 – present “Problems with symbolic reasoning led to a reaction against this — led to the

reactive agents movement.”

n Hybrid agents 1989 – present “Hybrid architectures attempt to combine the best of reasoning and reactive

architectures.”

1. Deductive reasoning agents

architectures based on ideas of “symbolic AI” symbolic representation

environment behaviour goals …

representation: logical formulae syntactic manipulation: logical deduction / theorem proving

Δ “deliberative agent”

Deductive reasoning agents:Agents as theorem provers

deliberative agents databases of

beliefs are specified using formulae of first-order predicate logic e.g. open (door1)

closed (door2)visible (wall32)…

but: just beliefs … not certified knowledge … semantic not specified …

open (door) may mean something exotic

to the agent designer

set of deduction rules

Deductive reasoning agents:Agents as theorem provers (cont.)

agent’s action selection function

for each action a

if precondition-for-action a can be inferred from current beliefs

return a

for each action a

if precondition-for-action a is not excluded by current beliefs

return a

return null


vacuum example [Russell & Norvig, 1995]

robot

(0,2) (1,2) (2,2)

(0,1) (1,1) (2,1)

(0,0) (1,0) (2,0)

percept: dirt or null.

actions: forward, suck, or turn.

goal: transverse room, remove dirt

domain predicatesIn (x,y) agent is at (x,y)Dirt (x,y) there is dirt at (x,y)Facing (d) agent is facing direction dWall (x,y) there is a wall at (x,y)

deduction rulesx y Wall(x,y) Free(x,y)

cleaning action rule (will take priority)In (x,y) Dirt (x,y) Do (suck)

if agent is at location (x,y) and perceives dirt: remove dirtotherwise

transverse the world

For example...In(0,0) Facing(north) Free (0,1) Dirt(0,0) Do(forward)In(0,1) Facing(north) Free (0,2) Dirt(0,1) Do(forward)In(0,2) Facing(north) Dirt(0,2) Do(turn)In(0,2) Facing(east) Free (1,2) Do(forward)…


“calculative rationality”

the selected action is the result of decision making on state in the beginning of the process of decision making

not acceptable in environments that change faster than decision making


Advantages clean logical semantics expressive well-researched domain of logic

Problems

how to build internal representation from percepts e.g. image logical formulae

inherent computational complexity of theorem proving timely functions !

many (most) search-based symbol manipulation algorithms of interest are highly intractable


1. Deductive reasoning agents 1956 – present “Agents make decisions about what to do via symbol manipulation. Its purest expression,

proposes that agents use explicit logical reasoning in order to decide what to do.”

2.2. Practical reasoning agentsPractical reasoning agents 1990 – present “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires /

intentions.”

3. Reactive agents 1985 – present “Problems with symbolic reasoning led to a reaction against this — led to the reactive agents

movement.”

4. Hybrid agents 1989 – present “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”

2. Practical Reasoning Agents

what is practical reasoning ?

“reasoning directed towards actions”

“practical reasoning is a matter of weighing conflicting considerations for and against competing options, where the relevant considerations are provided by what the agent desires/values/cares about and what the agent believes.” [Bratman]

distinguish practical reasoning from theoretical reasoning: theoretical reasoning is directed towards beliefs practical reasoning is directed towards actions

BDI architectures

BDI - a theory of practical reasoning - Bratman, 1988 for “resource-bounded agent”

includes means-end analysis weighting of competing alternatives interactions between these two forms of reasoning

Core concepts Beliefs = information the agent has about the world Desires = state of affairs that the agent would wish to bring about Intentions = desires (or actions) that the agent has committed to achieve

BDI particularly compelling because: philosophical component - based on a theory of rational actions in

humans software architecture - it has been implemented and successfully

used in a number of complex fielded applications IRMA - Intelligent Resource-bounded Machine Architecture PRS - Procedural Reasoning System

logical component - the model has been rigorously formalized in a family of BDI logics Rao & Georgeff, Wooldrige (Int Ai ) (Bel Ai )

Practical Reasoning Agents (cont.)

human practical reasoningpractical reasoning = deliberation + means-ends reasoning

deliberation

deciding what state of affairs you want to achieve

the outputs of deliberation are intentions

means-ends reasoning

deciding how to achieve these states of affairs

the outputs of means-ends reasoning are plans


1. deliberation intentions

2. means-ends reasoning planning

3. architecture

Practical Reasoning Agents:1. Deliberation: Intentions and Desires

intentions are stronger than desires

“My desire to play basketball this afternoon is merely a potential influencer of my conduct this afternoon. It must vie with my other relevant desires [. . . ] before it is settled what I will do. In contrast, once I intend to play basketball this afternoon, the matter is settled: I normally need not continue to weigh the pros and cons. When the afternoon arrives, I will normally just proceed to execute my intentions.” [Bratman, 1990]

Practical Reasoning Agents: Intentions

1. agents are expected to determine ways of achieving intentions If I have an intention to Φ, you would expect me to devote resources to deciding how to bring about

Φ

2. agents cannot adopt intentions which conflict1. If I have an intention to Φ , you would not expect me to adopt an intention Ψ that was incompatible

with Φ

n agents are inclined to try again if their attempts to achieve their intention fail1. If an agent’s first attempt to achieve Φ fails, then all other things being equal, it will try an alternative

plan to achieve Φ

1. agents believe their intentions are possible1. That is, they believe there is at least some way that the intentions could be brought about.

2. agents do not believe they will not bring about their intentions1. It would not be rational of me to adopt an intention to Φ if I believed that I would fail with Φ

n under certain circumstances, agents believe they will bring about their intentions If I intend Φ, then I believe that under “normal circumstances” I will succeed withΦ

1. agents need not intend all the expected side effects of their intentions1. I may believe that going to the dentist involves pain, and I may also intend to go to the dentist — but

this does not imply that I intend to suffer pain!

planner

intention(goal / task)

beliefs(state of

environment)

possibleactions

plan toachieve goal

means-endsreasoning

Practical Reasoning Agents:2. Means-ends Reasoning

The Blocks World

illustrate means-ends reasoning with reference to the blocks world

Contains a robot arm, 3 blocks (A, B, and C) of equal size, and a table-top

A

B C

The Blocks World

Here is a representation of the blocks world described above:

Clear(A)On(A, B)OnTable(B)OnTable(C)

Use the closed world assumption: anything not stated is assumed to be false

A

B C

The Blocks World

A goal is represented as a set of formulae Here is a goal:

OnTable(A) OnTable(B) OnTable(C)

AB C

The Blocks World Actions are represented using a technique that was

developed in the STRIPS planner

Each action has: a name

which may have arguments a pre-condition list

list of facts which must be true for action to be executed a delete list

list of facts that are no longer true after action is performed an add list

list of facts made true by executing the action

Each of these may contain variables

The Blocks World Operators

Example 1:The stack action occurs when the robot arm places the object x it is holding is placed on top of object y.

Stack(x, y)pre Clear(y) Holding(x)del Clear(y) Holding(x)add ArmEmpty On(x, y)

A

B

The Blocks World Operators Example 2:

The unstack action occurs when the robot arm picks an object x up from on top of another object y.

UnStack(x, y)pre On(x, y) Clear(x) ArmEmptydel On(x, y) ArmEmpty add Holding(x) Clear(y)

Stack and UnStack are inverses of one-another.

A

B

The Blocks World Operators

Example 3:The pickup action occurs when the arm picks up an object x from the table.

Pickup(x)pre Clear(x) OnTable(x) ArmEmptydel OnTable(x) ArmEmpty add Holding(x)

Example 4:The putdown action occurs when the arm places the object x onto the table. Putdown(x)

pre Holding(x)del Holding(x) add Clear(x) OnTable(x) ArmEmpty

A Plan

What is a plan?A sequence (list) of actions, with variables replaced by constants.

I G

a1

a17

a142

3. BDI ArchitectureBelief revision

Deliberation process

percepts

DesiresOpportunityanalyzer

Intentions

Filter

Means-endsreasoner

Plans

Intentions structured in partial plans

Executor

B = brf(B, p)

D = options(B, D, I)

I = filter(B, D, I)

= plan(B, I)

Library of plans

actions

BeliefsKnowledge


agent control loop

while true

observe the world;

update internal world model;

deliberate about what intention to achieve next;

use means-ends reasoning to get a plan for the intention;

execute the plan

end while

- when to reconsider intentions !?

- what are the options (desires) ?- how to choose an option ?- incl. filter- chosen option intention …

Implementing Practical Reasoning Agents

Let’s make the algorithm more formal:

Implementing Practical Reasoning Agents

this version: optimal behaviour if

deliberation and means-ends reasoning take a vanishingly small amount of time;or

the world is guaranteed to remain static while the agent is deliberating and performing means-ends reasoning;or

an intention that is optimal when achieved at time t0 (the time at which the world is observed) is guaranteed to remain optimal until time t2 (the time at which the agent has found a course of action to achieve the intention).

Deliberation

The deliberate function can be decomposed into two distinct functional components:

option generationin which the agent generates a set of possible alternatives;Represent option generation via a function, options, which takes the agent’s current beliefs and current intentions, and from them determines a set of options (= desires)

filteringin which the agent chooses between competing alternatives, and commits to achieving them.In order to select between competing options, an agent uses a filter function.

Deliberation


If an option has successfully passed trough the filter function and is chosen by the agent as an intention, we say that the agent has made a commitment to that option

Commitment implies temporal persistence of intentions; once an intention is adopted, it should not be immediately dropped out.

Question: How committed an agent should be to its intentions? degrees of commitments

blind commitment ≈ fanatical commitment: continue until achieved

single-minded commitment continue until achieved or no longer possible

open-minded commitment continue until no longer believed possible

Commitment Strategies

An agent has commitment both to ends (i.e., the wishes to bring about) and means (i.e., the mechanism via which the agent wishes to

achieve the state of affairs)

current version of agent control loop is overcommitted, both to means and ends

modification: replan if ever a plan goes wrong

Reactivity, replan

“Blind commitment”

Commitment Strategies

this version still overcommitted to intentions: never stops to consider whether or not its intentions are

appropriate

modification: stop for determining whether intentions have succeeded or whether they are impossible:

“Single-minded commitment”

Single-minded Commitment

Dropping intentions that are impossibleor have succeeded

Reactivity, replan

Intention Reconsideration

Our agent gets to reconsider its intentions when: it has completely executed a plan to achieve its current intentions; or it believes it has achieved its current intentions; or it believes its current intentions are no longer possible.

This is limited in the way that it permits an agent to reconsider its intentions

modification: Reconsider intentions after executing every action

“Open-minded commitment”

Open-minded Commitment


But intention reconsideration is costly!A dilemma: an agent that does not stop to reconsider its intentions sufficiently often

will continue attempting to achieve its intentions even after it is clear that they cannot be achieved, or that there is no longer any reason for achieving them

an agent that constantly reconsiders its attentions may spend insufficient time actually working to achieve them, and hence runs the risk of never actually achieving them

Solution: incorporate an explicit meta-level control component, that decides whether or not to reconsider

meta-level control

Possible Interactions

The possible interactions between meta-level control and deliberation are:


Situations In situation (1), the agent did not choose to deliberate, and as consequence, did not

choose to change intentions.Moreover, if it had chosen to deliberate, it would not have changed intentions. the reconsider(…) function is behaving optimally.

In situation (2), the agent did not choose to deliberate, but if it had done so, it would have changed intentions. the reconsider(…) function is not behaving optimally.

In situation (3), the agent chose to deliberate, but did not change intentions. the reconsider(…) function is not behaving optimally.

In situation (4), the agent chose to deliberate, and did change intentions. the reconsider(…) function is behaving optimally.

An important assumption: cost of reconsider(…) is much less than the cost of the deliberation process itself.

Optimal Intention Reconsideration

Kinny and Georgeff’s experimentally investigated effectiveness of intention reconsideration strategies

Two different types of reconsideration strategy were used: bold agents

never pause to reconsider intentions, and cautious agents

stop to reconsider after every action

Dynamism in the environment is represented by the rate of world change,

Optimal Intention Reconsideration

Results (not surprising):

If is low (i.e., the environment does not change quickly),bold agents do well compared to cautious ones. cautious ones waste time reconsidering their commitments while bold

agents are busy working towards — and achieving — their intentions.

If is high (i.e., the environment changes frequently),cautious agents tend to outperform bold agents. they are able to recognize when intentions are doomed, and also to

take advantage of serendipitous situations and new opportunities when they arise.

Implemented BDI Agents: IRMA

IRMA – Intelligent Resource-bounded Machine Architecture – Bratman, Israel, Pollack

IRMA has four key symbolic data structures: a plan library explicit representations of

beliefs: information available to the agent — may be represented symbolically, but may be simple variables

desires: those things the agent would like to make true — think of desires as tasks that the agent has been allocated;

intentions: desires that the agent has chosen and committed to

IRMA

Additionally, the architecture has: a reasoner

for reasoning about the world; an inference engine a means-ends analyzer

determines which plans might be used to achieve intentions an opportunity analyzer

monitors the environment, and as a result of changes, generates new options

a filtering process determines which options are compatible with current intentions

a deliberation process responsible for deciding upon the ‘best’ intentions to adopt

Practical Reasoning Agents:Procedural Reasoning System (PRS)

“BDI-architecture” (beliefs / desires / intentions) explicit data structures for b/d/i

planning no “on-the-fly” planning plan libraries

a plan: goal (post-condition)

context (pre-condition)

body(sequence of actions

/ subgoals)

intention stackbeliefs plans

desires intentions

agent

interpreter

sensor input

action

Implemented BDI Agents: PRS

Another BDI-based agent architecture PRS – Procedural Reasoning System

(Georgeff, Lansky)

each agent is equipped with a plan library, representing that agent’s procedural knowledge: knowledge about the mechanisms that can be used by the agent in order to realize its intentions

the options available to an agent are directly determined by the plans an agent has: an agent with no plans has no options

in addition, agents have explicit representations of beliefs, desires, and intentions, as above


further implementations Jadex (java) Jason (java – sourceforge.net) JAM (java) JACK (java) …




2. Practical reasoning agents 1990 – present “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires /

intentions.”

3.3. Reactive agentsReactive agents 1985 – present “Problems with symbolic reasoning led to a reaction against this — led to the reactive agents

movement.”

4. Hybrid agents 1989 – present “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”

“symbolism” “intelligence requires a symbolic representation of the world”

“connectionism”

intelligent behaviour is product of interaction with environment and emerges from simple behaviours

example: social behaviour (e.g. social insects)

3. Reactive Agents

environment

Brooks’ subsumption architecture PENGI Situated Automata RULER / GAPPS Pattie Maes: “behaviour network architecture” Free-flow architecture

Rodney Brooks two basic ideas

situatedness and embodiment “real” intelligence is situated in the world, not in disembodied systems such

as theorem provers or expert systems

intelligence and emergence “intelligent” behaviour arises as a result of an agent’s interaction with its

environment intelligence is ‘in the eye of the beholder’, it is not an innate, isolated

property

two key theses intelligence without representation

intelligent behaviour can be achieved without explicit representations of the kind that symbolic AI proposes

intelligence without reasoning intelligent behaviour can be achieved without explicit abstract reasoning of

the kind that symbolic AI proposes

Rodney Brooks:Subsumption Architecture

subsumption architecture “a hierarchy of task-accomplishing behaviours”

each rule / behaviour: if situation then action map perception input directly to actions

each behaviour ‘competes’ with others to exercise control over the agent

actions selection: by organising rules in layers

lower layers inhibit (“subsume”) higher layers lower layers represent more primitive kinds of behaviour (such as avoiding obstacles),

and have precedence over layers further up the hierarchy

computationally: very simple

“some of the robots do tasks that would be impressive if they were accomplished by symbolic AI systems”

Rodney Brooks:Subsumption Architecture (cont.)

a behaviour is a pair (c,a)where c P is a set of percepts called the conditionand a A is an action

a behaviour (c,a) can fire when the environment is in state s S iff see(s) c

Beh = {(c,a) ¦ c P and a A} be the set of all such rules

associated with an agent’s set of behaviour rules R Beh is a binary inhibition relation on the set of behaviours < R RThis relation is a total ordering on R

b1 < b2 iff (b1,b2) <

“b1 inhibits b2” “b1 subsumes b2” (b1 is lower in the hierarchy than b2, and will get priority over b2)

the “action selection function” is then defined as follows...

Rodney Brooks:Subsumption Architecture (cont.)

function action (p : P) : Avar fired : (R)var selected : Abegin

fired := {(c,a) ¦ (c,a) R and p c}for each (c,a) fired doif ((c’,a’) fired such that (c’,a’) < (c,a)) then

return aend-if

end for return nullend function action

compute all behaviours that can fire

determine if some behaviours subsume others

return appropriate action or null

Subsumption Architecture:example – Mars explorer

Mars explorer (L. Steels) objective

to explore a distant planet, and in particular, to collect sample of a precious rockthe location of the samples is not known in advance, but it is known that they tend to be clustered

mother ship broadcasts radio signal weakens with distance

no map available collaborative

Mother ship

autonomous vehicle

precious rock

Subsumption Architecture:example – Mars explorer (cont.)

single explorer solution:

behaviours / rules1. if obstacle then change direction

2. if carrying samples and at basethen drop them

3. if carrying samples and not at basethen travel up the gradient

field of base’s signal

4. if detect sample then pick it up

5. if true then walk randomly

total order relation 1 < 2 < 3 < 4 < 5


multiple explorer solution ?

think about it …

if one agent found a cluster of rocks – communicate ? range ? position ? how to deal with such messages ? may be far off …

indirect communication: each agent carries “radioactive crumbs”, which can be dropped,

picked up and detected by passing robots

communication via environment is called stigmergy


solution inspired by ant foraging behaviour agent creates a “trail” of radioactive crumbs back to the mother ship

whenever it finds a rock sample if another agent comes across a trail, it can follow it to the sample

cluster

refinement: agents following trail to the samples picks up some crumbs to make

the trail fainter the trail leading to the empty cluster will finally be removed


modified rule set

1. if detect an obstacle then change direction2. if carrying samples and at the base

then drop samples3. if carrying samples and not at the base

then drop 2 crumbs and travel up gradient4. if detect a sample then pick up sample5. if sense crumbs then pick up 1 crumb and travel down gradient6. if true then move randomly (nothing better to do)

1. order relation: 1 < 2 < 3 < 4 < 5 < 6

2. achieves near optimal performance in many situations3. cheap solution and robust (the loss of a single agent is not critical).

4. L. Steels argues that (deliberative) agents are “entirely unrealistic” for this problem.


advantages simple economic computationally tractable robust against failure

disadvantages agents act short-term since they use only local information no learning how to engineer such agents ? Difficult if more than 10 rules interact no formal tools to analyse and predict

Maes’ Behaviour Network for Situated Agents

observations “deliberative approach does not work in real dynamic

environment” brittleness, inflexibility, slow response time

goal-oriented behaviour explicit notion of goal (does not imply planning!) inertia in behaviour

agents: multiple behaviours at a time in a given situation parallel

behaviours conflict use same mechanism (action) or shared resource

agents have “competing” behaviours or actions

Maes’ Behaviour Network for Situated Agents (cont.)

an agent is a set of “competence modules” (behaviours)

each competence module: pre-condition (situation) post-condition (addition / deletion) activation level (~ relevance in current situation)

competence modules are linked according to pre-/post-conditions activation levels change activation levels influence action selection


behaviour network: a graph nodes: behaviours edges: three types of links

successor predecessor conflicter

B1abc

wy

B3wxy

rs

B2cde

xz


which action (behaviour) to perform ?

“activation energy” flow

global goals built-in source of motivation environment situational relevance behaviours (forward) activation by successors

(backward) activation by predecessors

inhibition by conflicters


pick-uppaint brush

paintput-down

paint brush

put-downhammer

backward

inhibitor

backward

forward

backward

current situation agent has hammer in hand the paint brush lies in front of the table

state

state

goal

= energy flow

forward


action selection algorithm

do forever

add external activation energy from goals & environment

spread activation/inhibition among behaviours forward activation via successor links backward activation via predecessor links backward inhibition via conflicter links

decay: total activation in system is constant

behaviour is selected if it is executable (all preconditions are satisfied) its activation level is over a threshold (theta) its activation is the highest among all executable/activated

behaviours

if one behaviour executes its activation is set to zero. threshold value is reset to default.

if no behaviour executes, reduce threshold value by x% agent “thinks” for a round and tries again next cycle


tuning the dynamics

action selection emerges from the dynamics of activation spreading

tuneable parameters:

amount of activation injected by environment amount of activation energy injected by goals the threshold value


evaluation / characteristics:

goal-oriented reactive and fast situation-oriented and opportunistic somewhat inert: biased to ongoing goal/plan goals interact and avoid conflicts robust

Free-flow architecture

esp. suitable for multi-objective behaviour Tyrell / Bryson used in AGV case study

proceed to location obstacle avoidance collision avoidance

principle tree structure of behaviour / sub-behaviour / primitive actions “energy flow”, from:

fixed amount for each action selection perceptions

primitive actions: winner-takes-all

target


a1 a2 a3 a4 a5 a6 a7 a8 a9 a10

live

find target avoid obstacles


a1 a2 a3 a4 a5 a6 a7 a8 a9 a10

live

find target avoid obstacles

need energydistance to obstacle

perc.x

perc.y




2. Practical reasoning agents 1990 – present “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires /

intentions.”

3. Reactive agents 1985 – present “Problems with symbolic reasoning led to a reaction against this — led to the reactive agents

movement.”

4.4. Hybrid agentsHybrid agents 1989 – present “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”

4. Hybrid agents

best of both worlds ? deliberative reactive

obvious approach: build an agent out of two (or more) subsystems:

a deliberative one, containing a symbolic world model, which develops plans and makes decisions in the way proposed by symbolic AI

a reactive one, which is capable of reacting to events without complex reasoning

mostly the reactive component is given precedence over the deliberative one

this kind of structuring leads naturally to the idea of a layered architecture

Hybrid agents (cont.)

horizontal layering layers are directly connected to sensory input and action output

vertical layering sensory input and action output are each dealt with by at most one

layer each

m possible actions suggested by each layer, n layers

mn interactions m2(n-1) interactions

introduces bottleneckin central control system

not fault-tolerant to layer failure

Horizontally Layered Architecture:Ferguson – TouringMachines

perception and action subsystemsinterface directly with the agent’s environment

three activity producing layers reactive layer

+- immediate response to changes in the environmentsituation-action rules

planning layer pro-active behaviour uses a library of plan skeletons (schemas)

modelling layer represents the various entities in the world predicts conflicts and generates new goals to resolve these conflicts

a control subsystem,a set of control rules for deciding which of the layersshould have control over the agent

modelling layer

planning layer

reactive layer

control subsystem

action subsystem

perceptionsubsystem

sensorinput

actionoutput

reactive layer a set of situation-action rules

example: rule-1: kerb-avoidance

if is-in-front (Kerb, Observer) and speed (Observer) > 0 and separation (Kerb, Observer) < KerbThreshHoldthen change-orientation (KerbAvoidanceAngle)

planning layer plans and selects actions in order to achieve agent’s goals

modelling layer symbolic repr. of ‘cognitive state’ of other entities in the agent’s environment

layers communicate with each other and are embedded in a control framework, which use control rules

example:censor-rule-1:

ifentity(obstacle-6) in perception-buffer

thenremove-sensory-record(layer-R, entity(obstacle-6))

Horizontally Layered Architecture (cont.)

advantages conceptual simplicity

if we need an agent to exhibit n different types of behaviour, then we implement n different layers!

disadvantages competing layers no guarantee for coherent behaviour ?

to avoid this: mediator function makes decisions about which layer has control at a given time

the need for such “central control” can be problematic agent designer must potentially consider all possible interactions

between layers!

Vertically Layered Architecture:Müller –InteRRaP

vertically layered, two-pass architecture

planning knowledge

cooperation layer

plan layer

behaviour layer

social knowledge

world model

world interface

perceptual input action output

Vertically Layered Architecture:Müller –InteRRaP (cont.)

three layers behaviour based layer reactive behaviour local planning layer for everyday planning co-operative planning for social interactions

control between layers: bottom-up activation

if layer not competent to deal with situation: pass control to higher layer top-down execution

when a higher layer makes use of the facilities provided by a lower layer to achieve one of its goals

agent control flow

perceptual input arrives at the lowest layer

if reactive layer can deal with this input: it will do sootherwise: bottom-up activation (control passed

to local planning layer)

if local planning layer can handle the situation: it will do so (making use of top-

down execution)otherwise: bottom-up activation to pass control to

the highest layer

Vertically Layered Architecture (cont.)

advantage complexity of interactions between layers is reduced

n-1 interfaces between n layers

if each layer is capable of suggesting m actionsat most m2 (n-1) interactions

much simpler than the horizontal case (mn interactions)

disadvantages this simplicity comes at the cost of some flexibility

control must pass between each different layernot fault tolerant:

failures in any one layer can have serious consequences for agent performance

Conclusion

general: agent architectures: engines for agent behaviour

families of agent architectures1. Deductive reasoning agents

“Agents make decisions about what to do via symbol manipulation. Its purest expression, proposes that agents use explicit logical reasoning in order to decide what to do.”

2. Practical reasoning agents “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs /

desires / intentions.”

3. Reactive agents “Problems with symbolic reasoning led to a reaction against this — led to the

reactive agents movement.”

4. Hybrid agents “Hybrid architectures attempt to combine the best of reasoning and reactive

architectures.”

topic 4: agent architectures general agent architectures deductive reasoning agents practical...

Documents

agent control

model of agent

objects cont

behaviour architecture

building agents

differences cont

blood diseases

operations objects