Toward Versatile Robotic Assistants for Security and Service Applications

Monica N. Nicolescu

Department of Computer Science and Engineering

University of Nevada, Reno

http://www.cs.unr.edu/~monica

2/34

Overview

Goals:

Integrate robots in human society

Increase the utility of autonomous robots and their ability to

function in dynamic, unpredictable environments

Facilitate interaction with robots

Motivation:

Accessibility to a large range of users

3/34

Application Domains

Security

Scenario: security checkpoint

Task: threat detection

Service

Scenario: office/home robot assistant

Task: service multiple user requests

4/34

Research Problems

Robot control:

Support for frequent human-robot interactions, include the

human in the loop

Communication:

Engage in sustained interactions with people

Express/understand intent

Learning:

Program robots using an accessible method

5/34

Approach

Behavior-based control with a particular behavior

representation Frequent, sustained interactions with people

Understanding intent

Learning by demonstration Natural robot programming

Understanding (malevolent) activity/intent

Communication through actions Expressing intent

Long-term: integrate with neuroscience and cognitive

science approaches

6/34

Robot Control

A control architecture that provides support for frequent

human-robot interactions

Modularity, robustness and real-time response, support

for learning

Automatic reusability of existing components

Ability to encode complex task representations

Run-time reconfiguration of controllers

Behavior-based control as underlying control

architecture

7/34

Behavior-Based Robot Control

Behaviors Goal-driven, time-extended control processes, running in

parallel, connecting sensors and effectors

Highly effective in unstructured, dynamic environments

Usually invoked by reactive conditions

Built-in task specific information

BehaviorInput ActionsSensors Actuators

Behavior

Inp

ut

Input Actions

8/34

Hierarchical Abstract Behavior Architecture

Extended behavior-based architecture

Representation & execution of complex, sequential,

hierarchically structured tasks

Flexible activation conditions for behavior reuse

Representation of tasks as (hierarchical) behavior

networks

Sequential & opportunistic execution

Support for automated generation (task learning) Environment

sensory input

M. N. Nicolescu, M. J Matarić, “A hierarchical architecture for behavior-based robots", International Conference of Autonomous Agents and Multiagent Systems, July 15- July 19, 2002.

9/34

The Behavior Architecture

Goals Behi

{1/0}

Abstract/primitive behaviorstructure

Primitive behavior

Perform actions

Abstract behavior

Test world preconditions

Task specific preconditionsif met

Goals Beh1…k {1/0} Abstract Behavior Embeds representations of the

preconditions & goals

Primitive Behavior Performs actions, achieve goals

Representation and execution of behavioral sequences

Flexible activation conditions behavior reuse

Representation of tasks as behavior networks

10/34

Task Representation: Behavior Networks

GoTo(Source)

PickUp(Box)

Follow(Wall)

GoTo(Dest)

Drop(Box)

A

A

A

A

A

A

A

A

A

A

Abstract behaviors

Primitive behaviors

Links represent task relevant precondition-postcondition dependencies

Permanent preconditionEnabling preconditionOrdering precondition

11/34

Layers of Abstraction: Network Abstract Behaviors

Abstracts existing networks into a single component

Use NAB’s as parts of other behavior networks

Allows for a hierarchical representation of increasingly complex tasks

Upon activation, enable their components

12/34

The Robot Testbed

Pioneer 2DX mobile robot Pan-tilt-zoom camera Laser range-finder Gripper 2 rings of 8 sonars PC104 stack Logitech cordless headset IBM ViaVoice speech software Implementation in Ayllu

Picking up, dropping objects (PickUp, Drop) and tracking targets (Track)

13/34

Experimental Validation

Sequential & opportunistic execution Object transport &

visit targets subtasks

Hierarchical representation

14/34

Results

Yellow Orange Pink Light-GreenTrial 5

Yellow Light-Green Orange PinkTrial 4

Light-Green Yellow Orange PinkTrial 3

Pink Yellow Orange Light-GreenTrial 2

Orange Pink Light-Green YellowTrial 1

Order of target visits

M. N. Nicolescu, M. J Matarić, “A hierarchical architecture for behavior-based robots", International Conference of Autonomous Agents and Multiagent Systems, July 15- July 19, 2002.

15/34

Human-Robot Interaction – Proposed Work

Goal:

Include support for frequent human-robot interactions

Issues:

Handle interruptions

Switching between different activities (idle, task execution,

learning, dialog)

Approach:

Incorporate awareness of human presence

Incorporate a model of activity control (situations and associated

responses) with our behavior based architecture

16/34

Communication – Proposed Work

Goal:

Understanding intent from simple behavior

Approach:

Match high-level perceptions of the robot with the known

goals of the robot’s behaviors

Applications:

Service: achieve better cooperation by understanding

human intentions (e.g., giving/taking a tool/object)

17/34

Learning

Learn a high-level task

representation, from a set of

underlying capabilities

already available to the robot

Approach:

Learning by experience (teacher following) Active participation in the demonstration

Mapping between observations and the skills that achieve the same

observed effects

18/34

Learning by Demonstration Framework

Inspiration:

Human-like teaching by demonstration

Multiple means for interaction and learning: concurrent use of

demonstration, verbal instruction, attentional cues, gestures, etc.

Solution: Instructive demonstrations, generalization and practice

GIVEDEMONSTRATION

TASK REPRESENTATION

FIRST?YES

EXECUTETASK

OK?YES

DONE

NONOGENERALIZE

GENERALIZED REPRESENTATION

EXECUTETASK

REFINED TASK REPRESENTATION

19/34

Instruction Stage: Teacher’s Perspective

The teacher is aware of:

Robot skills

What observations/features the robot could detect

Instructions for the robot

Informative cues:

“HERE” – moments of time relevant to the task

The teacher may give simple instructions:

“TAKE”, “DROP” – pick-up, drop objects

“START”, “DONE” –beginning/end of a demonstration

20/34

Instruction Stage: Learner’s Perspective

Teacher-following strategy (laser rangefinder &

camera)

Abstract behaviors (perceptual component)

continuously monitor their goals:

Ability to interpret high-level effects (e.g. approaching a target,

being given/taken an object)

Goals Met Abstract Behavior signals

observation-behavior

mapping

Compute the values of behavior parameters gathered

through its own sensors

21/34

Learning an Object-Transport Task

Human demonstration

Robot demonstration

Learned topologyEnvironment

All observations relevantNo trajectory learning

Not reactive policy

22/34

Generalization

Hard to learn a task from only one trial: Limited sensing capabilities, quality of teacher’s demonstration,

particularities of the environment

Main learning inaccuracies: Learning irrelevant steps (false positives)

Omission of steps that are relevant (false negatives)

Approach: Demonstrate the same task in different/similar environments

Construct a task representation that: Encodes the specifics of each given example

Captures the common parts between all demonstrations

M. N. Nicolescu, M. J Matarić, ”Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice", Second International Joint Conference on Autonomous Agents and Multi-Agent Systems , July 14-18, 2003

23/34

Generalization

Task: Go to either the Green or Light Green targets, pick up the Orange box, go between the Yellow and Red targets, go to the Pink target, drop the box there, go to the Light Orange target and come back to the Light Green target

None of the demonstrations corresponds to the desired task Contain incorrect steps and inconsistencies

24/34

Generalization Experiments

3rd Human demonstration

Robot performance

3rd 2nd 1st

25/34

Refining Task Representation Through Practice

Practice allows more accurate refining of the learned tasks Unnecessary task steps (“bad”) Missing task steps (”come” ”go”)

A

B

A C

Deleteunnecessarysteps

A

B

A C

M

N

Include newlydemonstratedsteps

A

C

B

F

A C

BAD

BAD

COME

M

N

GO

A

B

A C

M. N. Nicolescu, M. J Matarić, ”Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice", Second International Joint Conference on Autonomous Agents and Multi-Agent Systems , July 14-18, 2003

26/34

Practice and Feedback Experiments

3rd demonstration Practice run & feedback

Robot performance

Topology refinement

27/34

Practice and Feedback Experiments

Practice run & feedback1st demonstration

Robot performancePractice run

Topology refinement

28/34

Humandemonstration

Robotexecution Learned network

Gate Traversing Task

Learning from Robot Teachers

M. N. Nicolescu, M. J Matarić, "Experience-based representation construction: learning from human and robot teachers", IEEE/RSJ International Conference on Intelligent Robots and Systems, Pages 740-745, Oct. 29 – Nov 3, 2001

29/34

Learning – Proposed Work

Goal: learn a larger spectrum of tasks

Repetitive tasks: “repeat-until”

Conditioned tasks: “if-then”

Time relevant information: “do-until”

Trajectory learning: Turn(Angle), MoveForward(Distance)

Approach: Use an increased vocabulary of instructional cues (repeat,

until, if, etc.)

30/34

Support for Communication – Proposed Work

Goal:

Understanding intentions from complex activity

Approach:

Use learning from demonstration to train the robot patterns of

activity, and

Understand activity by observing/following people and mapping the

observations to its learned database of activities

Applications:

Security: detect suspicious behavior (e.g. passing around a

checkpoint area)

31/34

Communicating Through Actions – Proposed Work

Goal:

Natural communication & engaging in interactions

with people

Approach:

Use actions as vocabulary for communicating

intentions

Understanding exhibited behavior is natural: actions

carry intentional meanings

32/34

Communication – Preliminary Work

If in trouble, try to get help from a human assistant perform “dog-like” actions to get a human’s attention perform the actions that failed in front of the helper to express

intentions

Traverse a blocked gate

Pick up an inaccessible object

33/34

Communication – Proposed Work

Goal:

Understanding intent from interaction

Approach: Engaging in interactions with people can expose

underlying intentions

Applications: Security: uncooperative person could potentially have

malicious intentions Service: learn about cooperative/uncooperative users

34/34

Summary

Proposed framework for the development of

autonomous, interactive robotic systems

Behavior-based control with a particular behavior

representation Frequent, sustained interactions with people

Understanding intent

Learning by demonstration Natural robot programming

Understanding (malevolent) activity/intent

Communication through actions Expressing intent, engaging in interactions with people