Topics: Introduction to Robotics

CS 491/691(X)

Lecture 13

Instructor: Monica Nicolescu

CS 491/691(X) - Lecture 13 2

Review

• Hybrid control

– Selection, Advising, Adaptation, Postponing

– AuRA, Atlantis, Planner-Reactor, PRS, many others

• Adaptive behavior

– Adaptation vs. learning

– Challenges

– Reinforcement learning, examples (learning to walk,

learning to push)

CS 491/691(X) - Lecture 13 3

Supervised Learning

• Supervised learning requires the user to give the

exact solution to the robot in the form of the error

direction and magnitude

• The user must know the exact desired behavior for

each situation

• Supervised learning involves training, which can be

very slow; the user must supervise the system with

numerous examples

CS 491/691(X) - Lecture 13 4

Neural Networks

• One of the most used supervised learning methods

• Used for approximating real-valued and vector-

valued target functions

• Inspired from biology: learning systems are built

from complex networks of interconnecting neurons

• The goal is to minimize the error between the

network output and the desired output

– This is achieved by adjusting the weights on the network

connections

CS 491/691(X) - Lecture 13 5

Training Neural Networks

• Hebbian learning

– Increases synaptic strength along neural pathways

associated with a stimulus and a correct response

• Perceptron learning

– Delta Rule: for networks without hidden layers

– Back-propagation: for multi-layer networks

CS 491/691(X) - Lecture 13 6

Perceptron Learning

Repeat

• Present an example from a set of positive and negative

learning experiences

• Verify the output of the network as to whether it is correct or

incorrect

• If it is incorrect, supply the correct output at the output unit

• Adjust the synaptic weights of the perceptrons in a manner

that reduces the error between the observed output and the

correct output

Until satisfactory performance (convergence or stopping

condition is met)

CS 491/691(X) - Lecture 13 7

ALVINN• ALVINN (Autonomous Land

Vehicle in a Neural Network)

• Dean Pomerleau (1991)

• Pittsburg to San Diego: 98.2%

autonomous

CS 491/691(X) - Lecture 13 8

Learning from Demonstration & RL

• S. Schaal (’97) • Pole balancing, pendulum-swing-up

CS 491/691(X) - Lecture 13 9

Learning from Demonstration

Inspiration:

• Human-like teaching by demonstration

Demonstration Robot performance

CS 491/691(X) - Lecture 13 11

Learning from Robot Teachers

• Transfer of task knowledge from humans to robots

Human demonstration Robot performance

CS 491/691(X) - Lecture 13 12

Classical Conditioning

• Pavlov 1927

• Assumes that unconditioned stimuli (e.g. food)

automatically generate an unconditioned

response (e.g., salivation)

• Conditioned stimulus (e.g., ringing a bell) can,

over time, become associated with the

unconditioned response

CS 491/691(X) - Lecture 13 13

Darvin VII

• G. Edelman et. Al.

• Darvin VII Sensors

– CCD Camera

– Gripper that senses

conductivity

– IR sensors

• Darvin VII Actuators

– PTZ camera

– Wheels

– Gripper

• Low reflectivity walls, floor

• Two types of stimulus blocks

– 6cm metallic cubes

– Blobs: low conductivity (“bad

taste”)

– Stripes: high conductivity (“good

taste”)

CS 491/691(X) - Lecture 13 14

Darvin’s Perceptual Categorization

• Instead of hard-wiring stimulus-response rules,

develop these associations over time

Early training After the 10th stimulus

CS 491/691(X) - Lecture 13 15

Genetic Algorithms

• Inspired from evolutionary biology

• Individuals in a populations have a particular fitness

with respect to a task

• Individuals with the highest fitness are kept as

survivors

• Individuals with poor performance are discarded: the

process of natural selection

• Evolutionary process: search through the space

of solutions to find the one with the highest fitness

CS 491/691(X) - Lecture 13 16

Genetic Operators

• Knowledge is encoded as bit strings: chromozome– Each bit represents a “gene”

• Biologically inspired operators are applied to yield

better generations

CS 491/691(X) - Lecture 13 17

Classifier Systems

• ALECSYS system

• Learns new behaviors and

coordination

• Genetic operators act upon a

set of rules encoded by bit

strings

• Demonstrated tasks:

– Phototaxis

– Coordination of approaching,

chasing and escaping

behaviors by combination,

suppression and sequencing

CS 491/691(X) - Lecture 13 18

Evolving Structure and Control

• Karl Sims 1994

• Evolved morphology and control

for virtual creatures performing

swimming, walking, jumping,

and following

• Genotypes encoded as directed graphs are used to produce

3D kinematic structures

• Genotype encode points of attachment

• Sensors used: contact, joint angle and photosensors

CS 491/691(X) - Lecture 13 19

Evolving Structure and Control

• Jordan Pollak

– Real structures

CS 491/691(X) - Lecture 13 20

Fuzzy Control

• Fuzzy control produces actions using a set of fuzzy

rules based on fuzzy logic

• In fuzzy logic, variables take values based on how

much they belong to a particular fuzzy set:

– Fast, slow, far, near – not crisp values!!

• A fuzzy logic control system consists of:

– Fuzzifier: maps sensor readings to fuzzy input sets

– Fuzzy rule base: collection of IF-THEN rules

– Fuzzy inference: maps fuzzy sets to other fuzzy sets

according to the rulebase

– Defuzzifier: maps fuzzy outputs to crisp actuator commands

CS 491/691(X) - Lecture 13 21

Examples of Fuzzy Control

• Flakey the robot:– Behaviors are encoded as collections of fuzzy rules

IF obstacle-close-in-front AND NOT obstacle-close-on-left

THEN turn sharp-left

– Each behavior may be active to a varying degree

– Behavior responses are blended smoothly

– Multiple goals can be pursued

• Systems for learning fuzzy rules have also been developed

CS 491/691(X) - Lecture 13 22

Where Next?

CS 491/691(X) - Lecture 13 23

Fringe Robotics: Beyond Behavior

Questions for the future

• Human-like intelligence

• Robot consciousness

• Complete autonomy of complex thought and action

• Emotions and imagination in artificial systems

• Nanorobotics

• Successor to human beings

CS 491/691(X) - Lecture 13 24

A Robot Mind

• The goal of AI is to build artificial minds

• What is the mind?

• “The mind is what the brain does.” (M. Minsky)

• The mind includes

– thinking

– feeling

CS 491/691(X) - Lecture 13 25

Computational Thought

• What does it mean for a machine to think?

• Bellman

– Thought is not well defined, so we cannot ascribe/judge it

– Computers can perform processes representative of human

thought: decision making/learning

• Albus

– For robots to understand humans, they must be

indistinguishable from humans in bodily appearance, physical

and mental development

• Brooks:

– Thought and consciousness need not be programmed in: they

will emerge

CS 491/691(X) - Lecture 13 26

The Turing Test

• Developed by the mathematician Alan Turing

Original version of Turing Test:

• Two people (a man and a woman) are put in

separate closed rooms. A third person can interact

with each of the two through writing (no voices).

• Can the 3rd person tell the difference between the

man and the woman?

CS 491/691(X) - Lecture 13 27

The Turing Test

AI version of the Turing Test:

• A person sits in front of two terminals: at one end is

a human at the other end is a computer. The

questioner is free to ask any questions to the

respondents at the other end of the terminals

• If the questioner cannot tell the difference between

the computer and the human subject, the computer

has passed the Turing Test!

CS 491/691(X) - Lecture 13 28

The Turing Test

• The Turing Test contest is performed annually, and it

carries a $100,000 award for anybody who passes it

• No computer so far has truly passed the Turing Test

• Is this a good test of intelligence?

– Thought is defined based on human fallibility rather than on

machine consciousness

• Many researchers oppose to using this test as a proof

of intelligence

CS 491/691(X) - Lecture 13 29

Penrose’s Critique

• Roger Penrose (Emperor’s new Mind, Shadows of the Mind), a British physicist, is a famous critic of AI

• Intelligence is a consequence of neural activity and interactions in the brain

• Computers can only simulate this activity, but this is not sufficient for true intelligence

• Intelligence requires understanding, and understanding requires awareness, an aspect of consciousness

• Many refuting arguments have been given

CS 491/691(X) - Lecture 13 30

“They're Made Out Of Meat“"They're made out of meat.“

"Meat?“

"Meat. They're made out of meat.“

"Meat?“

"There's no doubt about it. We picked several from different

parts of the planet, took them aboard our recon vessels,

probed them all the way through. They're completely meat.“

"That's impossible. What about the radio signals? The

messages to the stars.“

"They use the radio waves to talk, but the signals don't come

from them. The signals come from machines.“

"So who made the machines? That's who we want to contact."

Terry Bisson

CS 491/691(X) - Lecture 13 31

“They're Made Out Of Meat“

"They made the machines. That's what I'm trying to tell you. Meat made the machines.“

That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat.“

"I'm not asking you, I'm telling you. These creatures are the only sentient race in the sector and they're made out of meat.“

"Maybe they're like the Orfolei. You know, a carbon-based intelligence that goes through a meat stage.“

"Nope. They're born meat and they die meat. We studied them for several of their life spans, which didn't take too long. Do you have any idea what’s the life span of meat?“

"Spare me. Okay, maybe they're only part meat. You know, like the Weddilei. A meat head with an electron plasma brain inside."

Terry Bisson

CS 491/691(X) - Lecture 13 32

“They're Made Out Of Meat“

"Nope. We thought of that, since they do have meat heads like the Weddilei. But I told you, we probed them. They're meat all the way through.“

"No brain?“

"Oh, there is a brain all right. It's just that the brain is made out of meat!“

"So... what does the thinking?"

"You're not understanding, are you? The brain does the thinking. The meat.“

"Thinking meat! You're asking me to believe in thinking meat!“

"Yes, thinking meat! Conscious meat! Loving meat. Dreaming meat. The meat is the whole deal! Are you getting the picture?"

Terry Bisson

CS 491/691(X) - Lecture 13 33

Conclusion

Lots of remaining interesting problems to explore!

Get involved!

CS 491/691(X) - Lecture 13 34

Readings

• Lecture notes