1

PanoramicUniversity of Amsterdam

Informatics Institute

2

Localization

3

with a Sony Aiboby Jürgen Sturm

4

RoboCup 4-Legged League

Sony Aibo Robots

4 vs. 4 robots play fully autonomously

Soccer Games

Con

text

: M

obile

Rob

ots

5

RoboCup @ home

real-world applications human-machine interaction

Fully autonomous robots have to master challenges in unknown & unstructured environments

Follow a human, navigate, etc.

6

7

Traditional approaches

• Aibos / 4-Legged league uses landmarks with

known positions,known shape andknown color (manually calibration taking

hours)

• General solutions (SLAM) use better hardware

• Laser range finders• Omnidirectional cameras• Robots with better odometry (wheels)

The

pro

blem

:M

obile

rob

ot lo

caliz

atio

n(e

stim

atin

g th

e ro

bot’s

pos

ition

)

8

Features of new approach

• Real-time localization on a Sony Aibo• Take advantage of natural features of a

room– Independency of artificial landmarks– Auto-calibrating in new environments

• Idea:– Learn a panoramic model of the surroundings

of the robot for localization

9

Color clustering

Collect interesting colors

(around the robot)

Determine 10 most characteristic colors(using an EM clustering

algorithm)

Raw image(208x160,

YCbCr)

10

Sector appearance

Divide in vertical slices, called sectors

(360° correspond to 80 sectors)

Count color transitionsper sector

(between the 10 most char-acteristic colors of the scene)

App

roac

h B

uild

ing

an v

irtua

l pan

oram

ic w

all

Raw image(208x160,

YCbCr)

11

Learning the panorama model

Image features(10-12 sectors/image,

10x10 frequencies/sector)

Learn panorama model(estimate frequency distributions

per sector)

Panorama model(80 sectors,

10x10 distributions, each defined by 5 bins)

12

Alignment and Localization

Image features(10-12 sectors/image,

10x10 frequencies/sector)

Align with stored panorama model

(find shortest path)

Output(Rotational estimateSignal-to-noise ratioConfidence range)

After learning from 131 frames Robot rotated 45° to the left

13

Experiments in human environments

• Rotational test in living room (at night)

Res

ults

L

earn

ing

of th

e ap

pear

ance

of u

nkno

wn

& u

nstr

uctu

red

envi

ronm

ents

14

4-Legged soccer field, indoors,single learned spot

Translational test on soccer fields

Human soccer field, outdoors,single learned spot

15

Multi-spot learning

• Aibo trained on 4 different spots, yielding 4 different panoramas

• Aibo kidnapped and placed back at arbitrary positions on the field

• Aibo tries to walk back to center spot

16

Possibilities for the 4-Legged league

• Getting rid of all artificial landmarks

• 11 vs. 11 games (bigger field)

• Outdoor demonstrations become possible

Con

clus

ions

17

Possible usage for theRoboCup @ home league

• Distinguish living room from kitchen or garden

• Rough but quick map building

• Find relative position of the TV/stove/etc on this map

18

Other applications

• CareBot: navigation in a closed indoor environment

• Mobile applications (for example on cellular phones) for quick positional estimates (tourism)

19

Rotational estimate and Confidence range in numbers

0

22.5

45

67.5

90

0 50 100 150 200 250

Distance from learned spot (mm)

de

gre

es

confidence range

error in rotation estimate

20

Architecture Overview

Camera images

Odometry updatesd

Sector-based feature

extraction

and SNR

Align with panorama model

0º

90º

180º

270º

Orientation buffer

+ confidence range

.. ..

21

Conclusions

• Accurate estimate of the rotation from a single learned spot (up to 40 meters)

• A good estimate of the relative distance from a single learned spot (up to 40 meters)

• Rough estimate of the absolute position from multiple trained spots

22

University of AmsterdamInformatics Institute

Panoramic Localization with a Sony Aiboby Jürgen Sturm

User manual• Head button always resets robot and triggers autoshutter & color clustering• Press front button to manually trigger color clustering

In training mode:• Press middle button to start learning of the first spot• Press middle button again to continue learning on more spots• Press back button to switch to localization mode

In localization mode:• Press front button to switch between rotational and translational mode• Press middle button to reset panorama and start learning• Press back button to switch between find and set-reference mode

1

2

1 5

3

1

4

7

8 9

6

Fullly working memorystick image can be downloaded fromhttp://staff.science.uva.nl/~jsturm/panorama/panorama-release.zip