IVR Lecture1 2007

1

Introduction to Vision & Roboticsby Bob Fisher       rbf@inf.ed.ac.uk   

Introduction to Robotics

Introduction

Some definitions

Applications of robotics and vision

The challenge: a demonstration

Historical highlights

IVR Lecture1 2007

2

Vision and Robotics: some definitions

Connecting the computer to the “raw unwashed world” (Russell & Norvig)“ t [f 2 d i ] t t ti f th“create [from 2‐d image] an accurate representation of the three‐dimensional world and its properties, then using this information we can perform any visual task” (Aloimonos& Rosenfeld)Vision is the direct extraction of affordances from the optic array ‐ GibsonA robot is:“A programmable multi‐function manipulator designed to move material, parts, or specialised devices g p pthrough variable programmed motions for the performance of a variety of tasks” (Robot Institute of America)“Robotics is the intelligent connection of perception to action’’ (Brady)

Applications: dull, dirty or dangerousVisual inspection of parts

Detecting crime on CCTV

Welding on carsN.B. Overlap with automation

IVR Lecture1 2007

3

Applications: dull, dirty or dangerousRobot vacuum cleaners

Cleaning nuclear plants

Robot sewer inspection

N.B. Overlaps with teleoperation

Applications: dull, dirty or dangerousVisual aids for driving

Demining

Space exploration

IVR Lecture1 2007

4

Applications:  also...?Entertainment industry

Service industry

Science

A challenging problem

We donʹt have much introspective insightWe don t have much introspective insight into how we see or how we control actionBuilding vision and robot systems involves a variety of interacting technology domains:– Mechanical, electrical, digital, computational...This has proved to be a hard problem for AIThis has proved to be a hard problem for AI– Can beat the human grandmaster at chess– Canʹt replace a house cleaner

IVR Lecture1 2007

5

Vision and robotics uses all areas of AI:

Problem solving, planning, search, inference, knowledge representation, learning etc...g p , gBut we canʹt just plug sensors and effectors onto an AI simulation and expect it to work Have constraints such as:– Limited, noisy, raw information– Continuous dynamic problem space– Time, power, cost and hardware limitationsOften solutions grounded in these constraints do not resemble conventional AI approaches

Ancient Greek hydraulic and mechanical automata

H fHero of Alexandria

AD 100

IVR Lecture1 2007

6

Renaissance optics:

The algorithmic connection between the world and the image - Durer c.1500

18th century clockwork animals

Vaucanson’s duck

IVR Lecture1 2007

7

Early 20th century

Electronic devices for remote control – Tesla

Methods for transducing images into electrical signals

‘Robot’ used to describe tifi i l h id lartificial humanoid slaves

in Capek’s play “Rossum’s Universal Robots” 1920

1940s –1950s

Development of electronic computer and control theoryand control theory

Used for artificial t W lt ’creatures e.g. Walter’s

‘tortoise’ and John Hopkins’ ‘beast’

IVR Lecture1 2007

8

1960s

Industrial robot arms:arms:

Unimation

Methods for image enhancement and pattern recognitionp g

1970s

Work on systems in restricted domains

e.g. Shakey in blocks world

Freddy assembly task

IVR Lecture1 2007

9

1980s

Tackling more realistic problems:problems:

Natural scene analysis

Face recognition

Dynamic locomotion

Significant impact in manufacturing

Active vision

Recent highlights:

Leg Lab - MIT

1980 onward

1995 – biped acrobatics

IVR Lecture1 2007

10

(Leg lab continued) 2000 – complex biped

Recent highlights:

NavLab

CMU 1987 onwards

1995 ‘No hands across America’ drive from Pittsburgh to SanDiegoSanDiego

98.2% autonomous

IVR Lecture1 2007

11

Military – Predator UAV

Walking Reactive “Insects”

Atilla & Ghengis – MIT Brooks Lab c. 1990

IVR Lecture1 2007

12

Barrett Gripper

Classical Control Paradigm: “SPA”

•• speed and efficiencyspeed and efficiency•• flexibility and adaptivityflexibility and adaptivity•• modularity and scalabilitymodularity and scalability•• errorerror--tolerance and robustnesstolerance and robustness

SPA lacksSPA lacks•• serialserial•• ad hoc ad hoc •• analyticalanalytical•• assumptousassumptous

SPA isSPA is

IVR Lecture1 2007

13

Quotations by R. Brooks”fast, cheap, and out of control”

Planning is just a way of avoiding figuring out what to do next.The world is its own best modelThe world is its own best modelComplex behavior need not necessarily be the product of a complex control systemSimplicity is a virtue Robots should be cheapAll on‐board computation is importantS t h ld b b ild i t llSystems should be build incrementallyIntelligence is in the eye of the observerNo representation, no calibration, no complex computers

Subsumption ArchitectureEvaluation of progressEvaluation of progress

Scheduling of subtasksScheduling of subtasksif notif not

Scheduling of subtasksScheduling of subtasks

(sub(sub--) Goal approach) Goal approach

Path planningPath planning

SelfSelf localization &localization & calibrationcalibrationif notif not

if notif not

if notif not

SelfSelf--localization & localization & --calibrationcalibration

Obstacle avoidanceObstacle avoidance

Move when clearMove when clearif notif not

if notif not

sensesense actact

IVR Lecture1 2007

14

Evaluation of the Subsumption Architecture

“I wouldn’t want one to be my chauffeur”(C. Torpe)

Modifications at low‐levels affect higher levelsOften there the hierarchy is not strictPriorities rather than inhibitionRepresentations, plans, and models do helpReproducibility is a virtuep ySPA is top‐down, SubsArc is bottom‐up“neats vs. scruffies”

Sensing the world

Keypoints:Why robots need sensingFactors that affect sensing capabilityContact sensingProximity and range sensingSensing light

IVR Lecture1 2007

15

Why robots need sensing 

• For a robot to act successfully in the real world it needs to be able to perceive the world and itselfit needs to be able to perceive the world, and itself in the world.

• Can consider sensing tasks in two broad classes:– Finding out what is out there: e.g. is there a goal; is this a team-mate; is there danger? = Recognition– Finding out where things are: e.g. where is the ball andFinding out where things are: e.g. where is the ball and how can I get to it; where is the cliff-edge and how can I avoid it? = Location

• But note that this need not be explicit knowledge

Sensing capability depends on a number of factors:

1 What signals are available?g

Light Pressure & Sound Chemicals

IVR Lecture1 2007

16

N.B. Many more signals in world than humans usually sense: e.g. Electric fish generate electric field and detect distortiondetect distortion

Sensing capability depends on a number of factors:

1 What signals are available?g2 What are the capabilities of the sensors?

Distance

Vision

Contact

Taste

Internal

BalanceHearingSmell

PressureTemperature

Actuator position/

movementPain/damage

IVR Lecture1 2007

17

Note this differs across animals: e.g. Bees see ultraviolet light

Need to choose what to build in to robot – options and costsp

Visible UltravioletMore like a target?

Sensors perform transductionTransduction: transformation of energy from one form to another (typically, into electrical signals)( yp y, g )

IVR Lecture1 2007

18

Sensors perform transductionSensor characteristics mean there is rarely an isomorphic mapping between the environment and the internal signal, e.g:g , g

- Most transducers have a limited range

- Most transducers have a limited resolution, accuracy, and repeatability

- Most transducers have lags or sampling delays

- Many transducers have a non-linear responseMany transducers have a non linear response

- Biological transducers are often adaptive

- Good sensors are usually expensive in cost, power, size…

Sensing capability depends on a number of factors:

1 What signals are available?2 Wh t th biliti f th ?2 What are the capabilities of the sensors?3 What processing is taking place?

E.g. extracting useful information from a sound signal is difficult:

IVR Lecture1 2007

19

•Sound sources cause air ib ivibration

•Diaphragm (ear drum or microphone) has complex pattern of vibration in response to sound

• Usually analysed by separating frequencies and grouping through harmonic/temporal cues

time

freq

uenc

y

Sensing capability depends on a number of factors:

1 What signals are available?g2 What are the capabilities of our sensors?3 What processing is taking place?4 What is the task?

IVR Lecture1 2007

20

Trans-duction

Internal modelProcessing Actuators

‘Classical’ view

Motor commands

Plan of Action

Decision on Action

Task

Alternative viewTask specific t d ti

Task specific i

Task specific titransduction processing action

Task 2 specific transduction

Task 2 specific processing

Task 2 specific action

Task 3 specific transduction

Task 3 specific processing

Task 3 specific action

Contact sensors

Principal function i l tiis location– E.g. bump switch or pressure sensor: is the object contacting this part of the robot?

IVR Lecture1 2007

21

Contact sensors

Principal function i l tiis location– E.g. bump switch or pressure sensor: is the object contacting this part of the robot?

– Antennae: extend the range with flexible element

Contact sensors

Can also use forCan also use for recognition e.g. – Is it moving or are you?– Human touch can distinguish shape, force, slip, surface texture…p,

– Rat whiskers used to distinguish textures

IVR Lecture1 2007

22

‘Contact’ sensors

Note these kinds of sensors canNote these kinds of sensors can also be used to detect flow e.g. wind sensors

Proximity and Range Sensors

Again main function isAgain main function is position: distance to object at specific angle to robotTypically works by emitting signal and dete ti efle tiodetecting reflectionShort‐range = proximity sensor, e.g. IR

IVR Lecture1 2007

23

Proximity and Range SensorsOver longer distance = range sensors e.g. Sonar: emit sound 

and detect reflection

IVR Lecture1 2007

24

a. Sonar reflection time gives rangegives range

b. Can only resolve objects of beam width

c. Apparent range shorter than axial range

d. Angle too large so wall invisible

e. Invisible corner

f. False reflection makes apparent range greater

Using sonar to construct an occupancy grid 

Robot wants to know about free Robot wants to know about freespaceMap space as gridEach element has a value which is the probability it contains an obstacleUpdate probability estimates from sonar readings

IVR Lecture1 2007

25

Learning the mapAssuming robot knows where it is in grid, sensory input provides noisy information about obstacles, e.g. for sonar

⎤⎡ ⎞⎛⎞⎛

R

βα r

IIIIII

s

Probability p(z|O) of grid element z=(r,α) in region I if occupied (O) given measurement s

Using Bayesian approachwhere p(O)will depend on previous measurements

⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ −+⎟

⎠⎞

⎜⎝⎛ − 2

βαβλ

RrR

)(~)|~()()|()()|()|(

OpOzpOpOzpOpOzpzOp

+=

Sample occupancy grid 

Noisy fusion of multiple sonar observations

IVR Lecture1 2007

26

Proximity and Range Sensors

More accurate information (same principle) from laser range finderrange finder

•Either planar or scanning

•1,000,000 pixels per second

•Range of 30 metres

•Accuracy of few mms

Sample Laser Scan

IVR Lecture1 2007

27

Light sensorsWhy is it so useful to detect light?

Straight lines mean the rays reflected from objects can beStraight lines mean the rays reflected from objects can be used to form an image, giving you ‘where’.Very short wavelengths gives detailed structural information (including reflectance properties of surface, seen as colour) to determine ‘what’.Very fast, it is especially useful over large distances.But requires half our brain to do vision…q

Conclusions

Robots need sensing: location objectsRobots need sensing: location, objects, obstaclesCommonly used sensors: laser range, sonar, contact, proprioceptic, GPS (outdoors), markersGeneral scene scanning vs affordancesGeneral scene scanning vs affordances