statistical learning in robotics state-of-the-art, challenges and opportunities

Sebastian ThrunCarnegie Mellon University

Statistical Learning in RoboticsState-of-the-Art, Challenges and Opportunities

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

7 Open Problems

Estimation andLearning In

Robotics

RoboticsResearch Today

RoboticsResearch Today

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics Yesterday

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics Today

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics Tomorrow?

Thanks toT. Dietterich

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics @ CMU, 1992

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics @ CMU, 1994

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics @ CMU 1996

With: RWI / iRobot, Hans Nopper

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics @ CMU/UBonn, 1997

with W. Burgard, A.B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D. Schulz, W. Steiner

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Robotics @ CMU, 1998

with M. Beetz, M. Bennewitz, W. Burgard, A.B. Cremers, F. Dellaert, D. Fox, D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, D. Schulz

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

7 Open Problems

RoboticsResearch Today

Estimation andLearning In

Robotics

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

The Robot Localization Problem

• Position tracking (error bounded)• Global localization (unbounded error)• Kidnapping (recovery from failure)

?

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Probabilistic Localization

p(z0 | x, m)

p(x0 | z0, m)

p(x1|u1,z0,m)

[Simmons/Koenig 95][Kaelbling et al 96][Burgard et al 96][Thrun et al 96]

p(z1 | x, m)

p(x1| ,z1 ,u1,z0,m)

p(x0 | m)

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Probabilistic Localization

),,|(),,,|( 010010 muzxpmuzxzp ttttttt Bayes

[Kalman 60, Rabiner 85]

x = statet = timem = mapz = measurementu = control

Markov),,|(),|( 010 muzxpmxzp ttttt

101010101 ),,|(),,,|(),|( tttttttttt dxmuzxpmuzxxpmxzp

Markov

1101011 ),,|(),|(),|( ttttttttt dxmuzxpuxxpmxzp

),,|( 00 muzxp ttt

laser data p(z|x,m)map m

xt-1

ut

p(xt|xt-1,ut)

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

What is the Right Representation?

Multi-hypothesis

[Weckesser et al. 98], [Jensfelt et al. 99]

Particles

[Kanazawa et al 95] [de Freitas 98][Isard/Blake 98] [Doucet 98]

Kalman filter

[Schiele et al. 94], [Weiß et al. 94], [Borenstein 96], [Gutmann et al. 96, 98], [Arras 98]

[Nourbakhsh et al. 95], [Simmons et al. 95], [Kaelbling et al. 96], [Burgard et al. 96], [Konolige et al. 99]

Histograms(metric, topological)

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Monte Carlo Localization (MCL)

p(z0 | x, m)

p(x0 | z0, m)

p(x1|u1,z0,m)

p(z1 | x, m)

p(x1| ,z1 ,u1,z0,m)

p(x0 | m)

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Monte Carlo Localization (MCL)

With: Wolfram Burgard, Dieter Fox, Frank Dellaert

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Implications for Planning & Control

MDP Planner POMDP Planner

With N. Roy

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Monte Carlo Localization

With:

Frank

Dellaert

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Learning Mapsaka Simultaneous Localization and Mapping (SLAM)

70 m

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Learning Maps

110101100 ),,|(),|(),|(),,|( tttttttttttt dxmuzxpuxxpmxzpmuzxp

106 dimensions 3 dimensions

Localization:

111010111100 ),|,(),,|,(),|(),|,( tttttttttttttttttt dmdxuzmxpumxmxpmxzpuzmxp

110101100 ),|,(),|(),|(),|,( tttttttttttt dxuzmxpuxxpmxzpuzmxp

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Learning Maps with Extended Kalman Filters

2

2

2

2

2

2

2

1

11

21

21

21

21

2222221

1111211

,),|,(

yxlll

yyxyylylyl

xxyxxlxlxl

lylxllllll

lylxllllll

lylxllllll

Nttt

N

N

N

NNNNNN

N

N

y

x

l

l

l

uzmxp

[Smith, Self, Cheeseman, 1990]

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Kalman Filter Mapping: O(N2)

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Can We Do the Same WithParticle Filters?

robot poses and maps

),|,( 00 ttt uzmxp

sample map + pose

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

N

ntttntttttt uzsmpuzspuzsmp

1000000000 ),,|(),|(),|,(

Mapping: Structured Generative Model

s1 s2 st

u2 ut

m2

m1

z1

z2

s3

u3

z3

zt

. . .

Landmark

robot pose

control

measurement

With K. Murphy, B. Wegbreit and D. Koller

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Rao-Blackwellized Particle Filters

landmark n=2

…

),|(~ 00][

ttti

t uzxpx

landmark n=Nlandmark n=1

landmark n=2

…landmark n=Nlandmark n=1

),,|( 00][

tti

tn uxmp

robot poses

[Murphy 99, Montemerlo 02]

N

ntttntttttt uzsmpuzspuzsmp

1000000000 ),,|(),|(),|,(

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Ben Wegbreit’s Log-Trick

8,87,7

k 3 ?

FT

6,65,5

k 1 ?

FT

4,43,3

k 3 ?

FT

2,21,1

k 1 ?

FT

k 6 ?

FT

k 2 ?

FT

k 4 ?FT

[i][i] [i][i] [i][i] [i][i] [i][i] [i][i] [i][i][i][i] 8,87,7

n 7 ?

FT

6,65,5

n 5 ?

FT

4,43,3

n 3 ?

FT

2,21,1

n 1 ?

FT

n 6 ?

FT

n 2 ?

FT

n 4 ?FT

[i] [i][i] [i][i] [i][i] [i][i] [i][i] [i][i][i]

3,3

n 3 ?

FT

n 2 ?

FT

n 4 ?

F

T

[i][i]

new particle

old particle

Michael Montemerlo, Ben Wegbreit, Daphne Koller & Sebastian Thrun

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Advantage of Structured PF Solution

Kalman: O(N2)

500 features

1,000,000 features

Moore’s Theorem: logN 30Experimental: M=250

Rao-B’ PFs: O(MlogN)

+ global uncertainty, multimodal+ non-linear systems+ sampling over data associations

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

3 Examples

Particles +Kalman filters Particles +

Particles

Particles +Point Estimators

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Outdoor Mapping (no GPS)

With Juan Nieto, Jose Guivant, Eduardo Nebot, Univ of Sydney

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

With Juan Nieto, Jose Guivant, Eduardo Nebot, Univ of Sydney

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Tracking Moving Features

With: Michael Montemerlo

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Tracking Moving Entities Through Map Differencing

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Map-Based People Tracking

With: Michael Montemerlo

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Autonomous People Following

With: Michael Montemerlo

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Indoor Mapping

Map: point estimators (no uncertainty) Lazy

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Importance of Probabilistic Component

Non-probabilistic Probabilistic, with samples

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Multi-Robot Exploration

DARPA TMR MarylandDARPA TMR Texas

With: Reid Simmons and Dieter Fox

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Learning Object Models

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Nearly Planar Maps

Idea: Exploit fact that buildings posses many planar surfaces Compacter models Higher Accuracy Good for capturing environmental change

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Online EM and Model Selection

mostly planar mapraw data

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Online EM and Model Selection

CMU Wean Hall Stanford Gates Hall

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

3D Mapping Result

With: Christian Martin

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Combining Tracking and Mapping

With Dirk Hähnel, Dirk Schulz and Wolfram Burgard

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Combining Tracking and Mapping

With Dirk Hähnel, Dirk Schulz and Wolfram Burgard

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Underwater Mapping (with University of Sydney)

With: Hugh Durrant-Whyte, Somajyoti Majunder, Marc de Battista, Steve Scheding

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

RoboticsResearch Today

Estimation andLearning In

Robotics

7 Open Problems

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Can We Learn Better Maps?

Stationary objects and moving objects, people

Motion characteristics, relational knowledge Less structured environments (jungle, underwater) In real-time

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Can We Learn Control?

1t0..t0..t u)u,z|(: txp

Not an MDP Not discrete or low-dimensional Not knowledge-free

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

How Can We Learn in Context?

Goal of robotics is not … mapping classification clustering density estimation reward prediction …

But simply: Doing the right thing.

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

How can we exploit Domain Knowledge in Learning?

Test: Is hypothesis consistent with laws of geometry? laws of physics? conventional wisdom? …

Domain knowledge is your friend! ILP? “Lifelong” learning?

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Can we Integrating Learning and Programming?

LearningProgramming

prob<int> x = {{10, 0.2}, {11, 0.8}};

prob<int> y = {{20, 0.5}, {21, 0.5}};

prob<int> z = x + y;

prob<double> f = neuroNet(y);

with Frank Pfenning, CMU

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

What Can We LearnFrom Biology?

Courtesy of Bill Skaggs, University of Pittsburgh

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

University of Pittsburgh School of Nursing

Prof. Jackie Dunbar-JacobProf. Sandy EngbergProf. Margo HolmProf. Deb LewisProf. Judy MatthewsProf. Barbara Spier

School of MedicineProf. Neil ResnickProf. Joan Rogers

Intelligent SystemsProf. Don Chiarulli

University of Pittsburgh Computer Science Prof. Martha Pollack

Carnegie Mellon University Computer Science, Robotics

Prof. Sebastian Thrun

Prof. Geoff Gordon

Human Computer Interaction Prof. Sara Kiesler

Financial Support National Science Foundation

$1.4M ITR Grant $3.2M ITR Grant

…And Can We Actually DoSomething Useful?

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

The Nursebot Project

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Haptic Interface (In Development)

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Wizard of Oz Studies

By Sara Kiesler, Jenn Goetz

Sebastian Thrun Carnegie Mellon University ICML July 10-12, 2002

Truly Useful….?