robotic paradigms and control architectures · 2017-12-06 · robotic paradigms and control...

Robotic Paradigms and ControlArchitectures

Jan Faigl

Department of Computer ScienceFaculty of Electrical Engineering

Czech Technical University in Prague

Lecture 02

B4M36UIR – Artificial Intelligence in Robotics

Jan Faigl, 2017 B4M36UIR – Lecture 02: Robotic Paradigms 1 / 46

Overview of the Lecture

Part 1 – Robotic Paradigms and Control Architectures

Robotics Paradigms

Hierarchical Paradigm

Reactive Paradigm

Hybrid Paradigm

Example of Collision Avoidance

Robot Control


Robotics Paradigms Hierarchical Paradigm Reactive Paradigm Hybrid Paradigm Example of Collision Avoidance Robot Control

Part I

Part 1 – Robotic Paradigms and ControlArchitectures



Outline

Robotics Paradigms


Reactive Paradigm

Hybrid Paradigm


Robot Control



RobotA robot perceive an environment using sensors to control its actuators

Sensor Controller

Actuators

The main parts of the robot correspond to the primitives of robotics:Sense, Plan, and ActThe primitives form a control architecture that is called roboticparadigm



Robotic Paradigms

Primitives of robotics are: Sense, Plan, and ActRobotic paradigms – define relationship between the primitivesThree fundamental paradigms have proposed

Hierarchical paradigm – purely deliberative system

SENSE ACTPLAN

Reactive paradigm – reactive control

SENSE ACT

Hybrid paradigm – reactive and deliberative

SENSE

PLAN

ACT



Outline

Robotics Paradigms


Reactive Paradigm

Hybrid Paradigm


Robot Control




The robot sense the environment and create the “world model”A ”world model” can also be an a priori available, e.g., prior map

Then, the robot plans its action and execute it

SENSE ACTPLAN

The advantage is in ordering relationship between the primitivesIt is a direct “implementation” of the first AI approach to robotic

Introduced in Shakey, the first AI robot (1967-70)It is deliberative architecture

It use a generalized algorithm for planningGeneral Problem Solver – Strips

It works under the closed world assumptionThe world model contains everything the robot needs to know



Disadvantages of Hierarchical ModelDisadvantages are related to planning – Computational requirements

Planning can be very slow and the “global world” representation hasto contain all information needed for planning

Sensing and acting are always disconnected

The “global world” representation has to be up to dateThe world model used by the planner has to be frequently updatedto achieve a sufficient accuracy for the particular task

A general problem solver needs many facts about the world to searchfor a solutionSearching for a solution in huge search space is quickly computation-ally intractable and this problem is related to the frame problem

Even simple actions need to reason over all (irrelevant) details

Frame problem – a problem of representing the real-word situa-tions to be computationally tractable

Decomposition of the world model into parts thatbest fit the type of actions



Examples of Hierarchical Models

Despite of drawbacks of the hierarchical paradigm, it has been de-ployed in various systemsAn example are Nested Hierarchical Controller and NIST RealtimeControl System

It has been used until 1980 when the focus has been changedon the reactive paradigm

The development of hierarchical models further exhibit additionaladvancements, e.g., to address the frame problemThey also provide a way how to organize the particular blocks ofthe control architectureFinally, the hierarchical model represents an architecture that sup-port evolution and learning systems towards fully autonomous con-trol



Nested Hierarchical ControllerDecomposition of the planner into three different subsystems:Mission Planner, Navigation, Pilot

Navigation – planning a path as a sequence of waypointsPilot generates an action to follow the path

It can response to sudden objects in the navigation course. The planexists and it is not necessary to perform a complete planning.

SensorSensor

Navigator

Plan

Act

SenseMissionPlanner

Low-levelController

DriveSensor

WorldModel

Pilot

Steer



NIST Real-time Control System (RCS)

Motivated to create a guide for manufactures for addingintelligence to their robotsIt is based on NHC and the main feature it introduces is a set ofmodels for sensory perceptionIt introduces preprocessing step between the sensory perceptionand a world modelThe sensor preprocessing is called as feature extraction

E.g., extraction of the relevant information for creating a model ofthe environment such as salient objects utilized for localization

It also introduced the so called Value Judgment moduleAfter planing, it simulates the plan to ensure its feasibility

Then, the plan is passed to Behavior Generation module toconvert the plans into actions that are performed (ACT).

The “behavior” is further utilized in reactive and hybrid architectures



Overview of the Real-time Control System (RCS)

Key featuresSensor preprocessing, plan simulator for evaluation, and behavior gener-ator

Plan

Act

Sense

changes and

events

observedinput

perception,focus of

attention

plans,state ofactions

simulatedplans

tasksgoals

commandedactions

BehaviorGeneration

ValueJudgment

SensoryPerception

WorldModeling

KnowledgeDatabase



Hierarchical Paradigm – SummaryHierarchical paradigm represents deliberative architecture also calledsense-plan-actThe robot control is decomposed into functional modules that aresequentially executed

The output of sense module is input of the plan module, etc

Centralized representation and reasoningMay need extensive and computationally demanding reasoningEncourage open loop execution of the generated plansSeveral architectures have been proposed, e.g., using STRIP plannerin Shakey, Nested Hierarchical Controller (NHC), NIST RealtimeControl System (RCS)

NIST – National Institute of Standards and Technology

Despite of the drawbacks, hierarchical architectures tend to supportthe evolution of intelligence from semi-autonomous control to fullyautonomous control

Navlab (1996), 90% of autonomous steering from Washington DC to Los Angeles



Outline

Robotics Paradigms


Reactive Paradigm

Hybrid Paradigm


Robot Control



Reactive Paradigm

The reactive paradigm is a connection of sensing with acting

SENSE ACT

It is biological inspired as humans and animals provide an evidenceof intelligent behavior in an open world, and thus it may be possibleto over come the close world assumptionInsects, fish, and other “simple” animals exhibit intelligent behaviorwithout virtually no brainThere must be same mechanism that avoid the frame problemFor a further discussion, we need some terms that to discuss prop-erties of “intelligence” of various entity



Agent and Computational-Level Theory

Agent is a self-contained and independent entityIt can interact with the world to make changes and sense the worldIt has self-awareness

The reactive paradigm is influenced by Computational-Level Theo-riesD. Marr a neurophysiologist working computer vision techniques inspired by biological vision processes

Computational Level – What? and Why?What is the goal of the computation and why it is relevant?

Algorithmic level – How?Focus on the process rather the implementation

How to implement the computational theory? What is the rep-resentation of input and output? What is the algorithm for thetransformation of input to output?

Physical level – How to implement the process?How to physically realize the representation and algorithm?



Behaviors

Behavior – mapping of sensory inputs to pattern of motor actionSensory-Motor Pattern

Patternof motoraction

SensorInput Behavior

Behaviors can be divided into three categoriesReflexive behaviors are “hardwired” stimulus-response (S-R)

Stimulus is directly connected to the response – fastest response time

Reactive behaviors are learned and they are then executed withoutconscious thought

E.g., Behaviors based on “muscle memory” such as biking, skiing are reactive behaviors

Conscious behaviors are deliberative as a sequence of the previouslydeveloped behaviors

Notice, in ethology, the reactive behavior is the learned behavior whilein robotics, it connotes a reflexive behavior.



Reflexive Behaviors

Reflexive behaviors are fast “hardwired” if there is sense, it producethe actionIt can categorized into three types1. Reflexes – the response lasts only as long as the stimulus

The response is proportional to the intensity of the stimulus

2. Taxes – the response to stimulus results in a movement towards oraway of the stimulus,

E.g., moving to light, warm, etc.

3. Fixed-Action Patterns – the response continues for a longer dura-tion than the stimulus

The categories are not mutually exclusiveAn animal may keep its orientation to the last sensed location of thefood source (taxis) even when it loses the “sight” of it (fixed-actionpatterns)



Four Ways to Acquire a Behavior

Ethology provides insights how animals might acquire and organizebehaviors Konrad Lorenz and Niko Tinbergen

1. Innate – be born with a behavior, e.g., be pre-programmed2. Sequence of innate behaviors – be born with the sequence

The sequence is logical but importantEach step is triggered by the combination of internal state and theenvironment

It is similar to the Finite State Machine

3. Innate with memory – be born with behaviors that need initial-ization E.g., a bee does not born with the known location of the hive. It has

to perform some initialization steps to learn how the hive looks like.

Notice, S-R types of behaviors are simple to pre-program, but itcertainly should not exclude usage of memory

4. Learn – to learn a set of behaviors



Releasing Behavior – When to Stop/Suppress the BehaviorThe internal state and/or motivation may release the behavior

Being hungry results in looking for food

Behaviors can be sequenced into complex behaviorInnate releasing mechanism is a way to specify when a behaviorgets turned on and offThe releaser acts as a control signal to activate a behavior

If the behavior is not released, it does not respond to sensoryinputs and it does not produce the motor outputs

Patternof motoraction

SensorInput Behavior

Releaser

The releaser filters the perception

Notice, the releasers can be compound, i.e., a multiple conditionshave to be satisfied to release the behavior



Concurrent Behaviors

Behaviors can execute concurrently and independently which mayresults into different interactions

Equilibrium – the behaviors seems to balance each other outE.g., Undecided behaviour of squirrel whether to go for a food or rather run

avoiding humanDominance of one – winner takes all as only one behavior canexecute and not both simultaneouslyCancellation – the behaviors cancel each other outE.g., one behavior going to light and the second behavior going out the light

It is not known how different mechanisms for conflicting behaviorsare employedHowever, it is important to be aware how the behaviors will interactin a robotic system



Behaviors Summary

Behavior is fundamental element in biological intelligence and is alsofundamental component of intelligence in robotic systemsComplex actions can be decomposed into independent behaviorswhich couple sensing and actingBehaviors are inherently parallel and distributedStraightforward activation mechanisms (e.g., boolean) may be usedto simplify control and coordination of behaviorsPerception filters may be used to simply sensing that is relevant tothe behavior (action-oriented perception)Direct perception reduces computational complexity of sensing

Allows actions without memory, inference or interpretation

Behaviors are independent, but the output from one behaviorCan be combined with another to produce the outputMay serve to inhibit another behavior



Reactive ParadigmReactive paradigm originates from dissatisfaction with hierarchicalparadigm (S-P-A) and it is influenced by ethology

ActuatorsSensors

Build map

Explore

Wander

Avoid Collisions

Sense Act

Contrary to S-P-A, which exhibit horizontal decomposition, thereactive paradigm (S-A) provides vertical decomposition

Behaviors are layered, where lower layers are “survival” behaviorsUpper layers may reuse the lower, inhibit them, or create paralleltracks of more advanced behaviors

If an upper layer fails, the bottom layers would still operate



Multiple, Concurrent Behaviors

Strictly speaking, one behavior does not know what another behav-ior is doing or perceiving

Behavior

Behavior

Behavior

SENSE ACT

Mechanisms for handling simultaneously active multiple behaviorsare needed for complex reactive architecturesTwo main representative methods have been proposed in literature

Subsumption architecture proposed by Rodney BrooksPotential fields methodology studied by Ronald Arkin, David Pay-ton, et al.



Characteristics of Reactive Behaviors1. Robots are situated agents operating in an ecological niche

Robot has its own intentions and goals, it changes the world by itsactions, and what it senses influence its goals

2. Behaviors serve as the building blocks for robotic actions and theoverall all behavior of the robot is emergent

3. Only local, behavior-specific sensing is permitted – usage of explicitabstract representation is avoided – ego-centric representation

E.g., robot-centric coordinates of an obstacle are relative and not inthe world coordinates

4. Reactive-based systems follow good software design principles –modularity of behaviors supports decomposition of a task into par-ticular behaviors

Behaviors can be tested independentlyBehaviors can be created from other (primitive) behaviors

5. Reactive-based systems or behaviors are often biologically inspiredUnder reactive paradigm, it is acceptable to mimic biological intelligence



An Overview of Subsumption ArchitectureSubsumption architecture has been deployed in many robots thatexhibit walk, collision avoidance, etc. without the “move-think-move-think” pauses of ShakeyBehaviors are released in a stimulus-response wayModules are organized into layers of competence1. Modules at higher layer can override

(subsume) the output from the behaviorsof the lower layerWinner-take-all – the winner is the higher layer

Level 0Sensors Actuators

Level 2

Level 1

Level 3

2. Internal states are avoidedA good behavioral design minimizes the internal states, that can be,e.g., used in releasing behavior

3. A task is accomplished by activating the appropriate layer thatactivities a lower layer and so on

In practice, the subsumption-based system is not easily taskableIt needs to be reprogrammed for a different task



An Example of Subsumption Architecture

Avoid Objects

Sensors Actuators

Explore

Wander Around

Environment

Further reading: R. Murphy, Introduction to AI Robotics



Outline

Robotics Paradigms


Reactive Paradigm

Hybrid Paradigm


Robot Control



Hybrid Paradigm

The main drawback of the reactive-based architectures is a lack ofplanning and reasoning about the world

E.g., a robot cannot plan an optimal trajectory

Hybrid architecture combines the hierarchical (deliberative)paradigm with the reactive paradigm Beginning of the 1990’s

SENSE

PLAN

ACT

Hybrid architecture can be described as Plan, then Sense-ActPlanning covers a long time horizon and it uses global world modelSense-Act covers the reactive (real-time) part of the control



Characteristics of Reactive Paradigm in Hybrid Paradigm

Hybrid paradigm is an extension of the Reactive paradigmThe term behavior in hybrid paradigm includes reflexive, innate, andlearned behaviors In reactive paradigm, it connotes purely reflexive behaviors

Behaviors are also sequenced over timed and more complex emer-gent behaviors can occurBehavioural management – planning which behavior to use re-quires information outside the particular model (a global knowledge)

Reactive behavior works without any outside knowledge

Performance monitor evaluates if the robot is making progress toits goal, e.g., whether the robot is moving or stucked

In order to monitor the progress, the program has to know whichbehavior the robot is trying to accomplish



Components of Hybrid Deliberative/Reactive Paradigm

Sequencer – generates a set of behaviors to accomplish a subtaskResource Manager – allocates resources to behaviors, e.g., a se-lection of the suitable sensors

In reactive architectures, resources for behaviors are usually hardcoded.

Cartographer – creates, stores, and maintains map or spatial in-formation, a global world model and knowledge representation

It can be a map but not necessarily

Mission Planner – interacts with the operator and transform thecommands into the robot term

Construct a mission plan, e.g., consisting of navigation to some placewhere a further action is taken

Performance Monitoring and Problem Solving – it is a sort ofself-awareness that allows the robot to monitor its progress



Existing Hybrid Architectures

Managerial architectures use agents for high level planning at thetop, then there are agents for plan refinement to the reactive be-haviors at the lowest level

E.g., Autonomous Robot Architecture and Sensor Fusion Effects

State-Hierarchy architectures organize activity by scope of timeknowledge E.g., 3-Tiered architectures

Model-Oriented architectures concentrate on symbolic manipulationaround the global world E.g., Saphira

Task Control Architecture (TCA) – layered architectureSequencer Agent, Resource Manager – Navigation LayerCartographer – Path-Planning LayerMission Planner – Task Scheduling LayerPerformance Monitoring Agent – Navigation, Path-Planning, Task-SchedulingEmergent Behavior – Filtering



Task Architecture

EffectorsSensors

Mission Planner

Deliberative Layer

Obstacle Avoidance(CVM - Curvature Velocity Method)

Cartographer

Sequencer,Resource Manager

Reactive Layer

Navigation(POMDP - Partially Observable Markov Decision Process)

Path Planning

Task Scheduling(PRODIGY)

Global

World

Models



Outline

Robotics Paradigms


Reactive Paradigm

Hybrid Paradigm


Robot Control



Example of Reactive Collision Avoidance

Biologically inspired reactive architecture with vision sensor and CPGNotice, all is hardwired into the program and the robot goes ’just’ahead with avoiding intercepting obstacles

CPG-based locomotion control can beparametrized to steer the robot mo-tion to left or right to avoid collisionswith approaching objects

Avoiding collisions with obstacles andintercepting objects can be basedon the visual perception inspired bythe Lobula Giant Movement Detector(LGMD)

LGMD is a neural network detectingapproaching objects

Camera - Image L

Left LGMDRight LGMD

P P P P

I I I IE E E E

S S S S

LGMD

Pf (x, y) = Lf (x, y)− Lf−1(x, y)

Ef (x, y) = abs(Pf (x, y))

If (x, y) = conv2(Pf (x, y),wI)

wI =

0.125 0.250 0.1250.250 0 0.250.125 0.250 0.125

Sf (x, y) = Ef (x, y)− abs(If (x, y))

Uf =k∑

x=1

l∑y=1

abs(Sf (x, y))

uf =

(1 + exp

Uf

kl

)−1

∈ [0.5, 1]LSTM IN1 IN2

...

OUT

CPG locomotioncontroll – turn

Actuators

Čížek, Milička, Faigl (IJCNN 2017)



LGMD-based Collision Avoidance – Control Rule

Input image

Left image

Right image

Left LGMD

Right LGMD

uleft

uright

LGMD differencee = uleft − uright

turn← Φ(e)

CPG

A mapping function: Φ : from the output of the LGMDvision system to the turn parameter of the CPG

Φ(e) =

{100/e for abs(e) ≥ 0.210000 · sgn(e) for abs(e) < 0.2




Example of LGMD-based Collision Avoidance

x[m]

2.5

Collision avoidance experiment - hallway

2

1.5

1

0.5

0-1y[m]

-0.50

0.4

0.2

0z[m

]

t1

t2

t3

t4

t5

obstacle

LGMD output together with the proposedmapping function provide a smooth mo-tion of the robot




Outline

Robotics Paradigms


Reactive Paradigm

Hybrid Paradigm


Robot Control



A Control Schema for a Mobile RobotA general control schema for a mobile robot consists of Perception Mod-ule, Localization and Mapping Module, Path Planning Module, andMotion Control Module

Actuatorscommands

PathExecution

Acting

PathPlanning

Missioncommands

"Position", Global Map

Path

Rawdata

InformationExtraction andInterpretation

Sensing

LocalizationMap Building

Environment ModelLocal Map

Real Environment

KnowledgeData Base

Perception Motion Control

In B4M36UIR, we focus on Path Planning ModuleJan Faigl, 2017 B4M36UIR – Lecture 02: Robotic Paradigms 40 / 46


Motion Control

An important part of navigation is execution of the planned pathMotion control module is responsible in path realization

Position control – aims to navigate the robot to the desired locationPath-Following – the controller aims to navigate the robot alongthe given pathTrajectory-Tracking – it differs from the path-following in that thecontroller forces the robot to reach and follow a time parametrizedreference (path) E.g., a geometric path with an associated timing law

The controller can be realized as one of two typesFeedback controllerFeedforward controller



FeedBack Controller

The difference between the goal pose and the distance traveled sofar is the error used to control the motorsThe controller commands the motors (actuators) which change thereal robot poseSensors, such as encoders for a wheeled robot, provide the informa-tion about the traveled distance

Sensors Actuators

ControllerMotor commands

Input

Output"Current Pose"

+

-"Goal Pose"Feedback"Distance Traveled"

Notice, the robot may stuck, but it is not necessarilydetected by the encoders



Feed-Forward Controller

In feed-forward controller, there is not a feedback from the real wordexecution of the performed actionsInstead of that, a model of the robot is employed in calculation ofthe expected effect of the performed action

Model

Motor commands

Input

Output"Current Pose"

+"Goal Pose" ActuatorsController+

Feedforward

In this case, we fully rely on the assumption that theactuators will performed as expected



Temporal Decomposition of Control Layers

The robot control architecture typically consists of several modules (be-haviors) that may run at different frequencies

Low-level control is usually the fastest one, while path planning is sloweras the robot needs some time to reach the desired location

An example of possible control frequencies of different control layers

0.001 Hz

1 Hz

10 Hz

Range-based obstacle avoidance

Emergency stop

Path planning

PID speed control 150 Hz

Adapted from Introduction to Autonomous Mobile Robots, R. Siegwart et al.


Topics Discussed

Summary of the Lecture


Topics Discussed

Topics Discussed

Robotic ParadigmsHiearchical paradigmReactive paradigmHybrid Hiearchical/Reactive paradigm

Example of Reactive architecture – collision avoidanceRobot Control

Next: Path and Motion Planning


robotic paradigms and control architectures · 2017-12-06 · robotic paradigms and control...

Documents