ROBOTICS

COE 584

Deliberative & Hybrid Control

Lecture Outline Lecture Outline Deliberative control Hybrid control Types of layer organization

selection advising adaptation postponement

Examples of hybrid control AuRA, Atlantis SSS, PRS

Deliberative Systems Deliberative Systems

Purely deliberative systems are considered the classical control architecture, since they were the first to be tried

In AI, classical deliberative, planner-based architectures were used for reasoning about actions in various non-physical domains, such as chess

As a result, the same architectures were applied to robotics as well

In the 1960’s: Shakey In the 1960’s: Shakey

In the late 1960's, the state-of-the-art in machine vision was used to process visual information on a robot called Shakey, the forerunner of many AI-inspired robotics projects.

Shakey used a classical planner as the underlying structure to decide what to do.

What is planning?

Planning as Search Planning as Search Planning is looking ahead, searching The goal is a state The robot's entire state space is enumerated, and

searched, from the current state to the goal state Different paths are tried until one is found that

reaches the goal

If the optimal path is desired, then all possible paths must be considered in order to find the best one

SPA = Planner-based SPA = Planner-based Planner-based (deliberative) architectures

typically involve three generic sequential steps or functional modules: 1) sensing (S) 2) planning (P) 3) acting (A), executing the plan

Thus, they are called SPA architectures SPA has serious drawbacks What are they?

Problem 1: Time Problem 1: Time Complex state spaces:

very slow plan generation

Dynamic worlds: out of date plans (latency)

Problem 2: Space Problem 2: Space Representation of state space may

be very large Search tree (intermediate plan data)

may be very large Modern machines have virtual

memory (page to disk), but swapping is very slow

Problem 3: Representation Problem 3: Representation

Representation for planning has two parts: Knowing the state of the world Predicting the outcome of actions

State representation assumed to be: complete accurate current predictable

Problem 3: Representation Problem 3: Representation

Sensors have: noise inaccuracies aliasing (partial observability)

Effectors are: unpredictable unreliable

None of the assumptions are valid!

Problem 4: Execution Problem 4: Execution Execution is assumed to be:

sequential reliable unique (one actor)

But: blind execution of long sequences of

unreliable actions will fail E.g., p(success | 1 action) = 0.90 => p(success | 10 actions) = 0.35

Deliberative Summary Deliberative Summary In short, deliberative (SPA)

approaches: require search (slow) require representations (hard) encourage open-loop execution

(dangerous)

Opposition to SPA Opposition to SPA As a consequence, much opposition

from real robot practitioners mounted against SPA architectures

In the early/mid 1980's alternatives were proposed reactive systems hybrid systems

What happened to purely deliberative systems?

Role of Pure Deliberation Role of Pure Deliberation

Pure deliberation is alive and well in other domains, like game playing (chess, go, etc.) and other static worlds with plenty of time to plan

Planners Live On in Robotics Planners Live On in Robotics

The SPA approach has not been abandoned, it has been expanded

Given the two fundamental problems with purely deliberative approaches, we can augment them: search/planning is slow, so save/cache

important and/or urgent decisions; open-loop plan execution is bad, use

closed-loop feedback, and be ready to respond or re-plan when the plan fails.

Reusing Plans Reusing Plans Some frequently useful planned

decisions may need to be reused, so to avoid planning, an intermediate layer may cache and look those up

These can be intermediate-level actions (ILAs) macro operators: plans compiled into

more general operators for future use

Universal Plans Universal Plans Suppose for a given problem, all

possible plans are generated for all possible situations in advance, and stored

If for each situation a robot has a pre-existing optimal plan, it can react optimally, be reactive and optimal

It has a universal plan (These are complete reactive

mappings)

Viability of Universal Plans Viability of Universal Plans

A system with a universal plan is reactive; the planning is done at compile-time, not at run-time

Universal plans are not viable in most domains, because they require that: the world must be deterministic the world must not change the goals must not change

The world is too complex (state space is too large)

Situated Automata Situated Automata A formal notion of finite state machines

whose inputs are connected to sensors and whose outputs are connected to effectors are called situated automata.

Situated means existing in and interacting with a complex world, and automata is the formal name for FSMs (formally: finite state automata).

Situated automata are used to create reactive principled control systems.

Control w/ Situated AutomataControl w/ Situated Automata

Situated automata can be constructed in two basic ways: By hand (i.e., the designer puts FSMs

together), as in the Subsumption Architecture). By pre-compiling a complete plan (similar to

Universal Plans, but reduced down to circuits of FSMs). This requires the use of a special programming language that implements the right semantics and compiles down into FSM circuitry, as Rex and Gapps.

Domain Knowledge Domain Knowledge A key advantage of pre-compiled systems

is that domain knowledge, i.e., information that the designer has about the environment, the robot, and the task, can be embedded into the system in a principled way

Then, the system is compiled into a reactive circuit, so the knowledge does not have to be reasoned about (or planned with) explicitly, in real-time

Disadvantages Disadvantages A key disadvantage of pre-compiled

systems is that it quickly becomes prohibitively large to enumerate the state space of a real robot, and thus pre-compiling generally does not scale up to complex systems

Another disadvantage is common to compiled or hard-wired systems: the result is not flexible in the presence of changing environments, tasks or goals

Inventing Hybrid ControlInventing Hybrid Control The basic idea is simple: we want the

best of both worlds (if possible) The goal is to combine closed-loop

and open-loop execution That means to combine reactive and

deliberative control This implies combining the different

time-scales and representations This mix is called hybrid control

Organizing Hybrid Systems Organizing Hybrid Systems

A hybrid system typically consists of three components: a reactive layer a planner a layer that puts the two together

Hybrid architectures are often called three-layer architectures (TLA)

The planner and the reactive system are both standard, as we have covered them so far

The Magic Middle The Magic Middle The middle layer has a hard job:

1) compensate for the limitations of both the planner and the reactive system

2) reconcile their different time-scales 3) deal with their different

representations 4) reconcile any contradictory

commands between the two This is the challenge of hybrid

systems

Interaction of Layers Interaction of Layers

Hierarchical integrationPlanning guides reaction

Coupled planning & reacting

Dynamic Re-planning Dynamic Re-planning

Reaction can influence planning Any "important" changes discovered

by the low-level controller are passed back to the planner in a way that the planner can use to re-plan

The planner is interrupted when even a partial answer is needed in real-time

The reactive controller (and thus the robot) is stopped if it must wait for the planner to tell it where to go.

Planner-Driven Reaction Planner-Driven Reaction

Planning can influence reaction Any "important" optimizations the

planner discovers are passed down to the reactive controller

The planner’s suggestions are used if they are possible and safe

Who has priority, planner or reactor?

Types of Interaction Types of Interaction

Selection: Planning is viewed as configuration

Advising: Planning is viewed as advice giving

Adaptation: Planning is viewed as adaptation of controller

Postponing: Planning is viewed as a least commitment process

Selection Example: AuRA Selection Example: AuRA

R. Arkin (1986)

Planning is viewed as configuration Initial A* planner integrated with

schema-based controller Provides modularity, flexibility, and

adaptability

AuRA Schematic AuRA Schematic

Advising Example: Atlantis Advising Example: Atlantis

E. Gat (1991) (JPL) Three layers: controller, sequencer, deliberator Asynchronous, heterogeneous: reactivity and

deliberation Implemented in ALFA (A Language for Action) Planning as advice giving, not decree Notion of cognizant failure Tested on NASA rovers

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Rocky 4

Atlantis Schematic Atlantis Schematic

Adaptation Example: Planner-ReactorAdaptation Example: Planner-ReactorD. Lyons (1992)Continuous modification of a reactive control

systemPlanning is a form of reactor adaptationAdaptation is on-line rather than off-line

deliberationPlanning is used to remove performance

errors when they occurUses a particular underlying mathematical

model called a process algebraTested in both assembly cell and grasp

planning

Planner-Reactor ArchitecturePlanner-Reactor Architecture

REACTOR

PLANNER WORLDPERCEPTIONS

REACTIONS

ADAPTION ACTION

SENSINGPERCEPTION

GOALS

Postponing Example: PRSPostponing Example: PRS PRS = Procedural Reasoning System Georgeff and A. Lansky (1987)

Least commitment via plan elaboration postponement

Tested on SRI Flakey

Flakey the robot Flakey the robot

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

PRS Schematic PRS Schematic

Another Example: SSS Another Example: SSS J. Connell (1992)

SSS = Servo Subsumption Symbolic 3 layers: servo, subsumption, symbolic World models are a convenience, not a

necessity Symbolic: where-to-next (discrete time) Subsumption: where-to-go-now Servo: making it go (continuous time)

Tested on TJ

SSS Implementation: T J SSS Implementation: T J

More Examples More ExamplesSOMASS hybrid assembly system

C. Malcolm and T. Smithers (Edinburgh U.)

cognitive/subcognitive components planning as configuration

Agent architecture B. Hayes-Roth (Stanford) physical and cognitive levels functional boundary blurry

Multi-valued logic Saffiotti, Konolige, Ruspini (SRI)

Even More Examples Even More Examples

Supervenience L. Spector (1992, U. of Maryland)

Multiple levels of abstraction

Teleo-reactive agent architecture Benson and N. Nilsson (1995, Stanford)

Planning yields TR operator tree

Reactive Deliberation M. Sahota (1993, U. of British Columbia)

Robosoccer

Still More Examples Still More Examples Theoagent

T. Mitchell (CMU, 1990)Reacts when it can plans when it mustEmphasis on learning

Generic Robot ArchitectureNoreils and Chatila (1995, France)3 levels: planning, control system, functional

Dynamical Systems ApproachSchoner and Dose (1992)Planning is selecting and parameterizing

behavioral fields

Behaviors use vector summation

And Still More Examples And Still More Examples

Integrated path planning and dynamic steering control Krogh and C. Thorpe (1986, CMU)

Relaxation over grid-based model with potential fields controller

Planner generated waypoints for controller

Many others (including several for UUVs)

Hybrids Everywhere? Hybrids Everywhere?

Hybrid systems are the most popular alternative for single-robot control

Behavior-based systems are not used by quite as many researchers, but have more specialized niches (e.g., multi-robot systems) and more practical applications

Textbook Readings Textbook Readings MM 13, 15