long-term autonomy in everyday environments - bcs.org · haus der barmherzigkeit, vienna, austria...

Nick Hawes

Long-Term Autonomy in Everyday Environments

School of Computer Science, University of Birmingham, UK

A New Challenge for AI and Robotics

http://nickhaw.es @hawesie

http://nickhaw.es

Long-Term Autonomy in Everyday

Environments

http://strands-project.eu


Robust, intelligent,

autonomous behaviour

Long run-times in everyday

environments

Novel opportunities

to learn structure

environment

Exploitation of structure for

improved performance



environments




Meta-room mapping Desktop observations

Object presence checks Door checks

G4S Technology, UK

Haus der Barmherzigkeit,

Austria

Information provision Object presence checks

Door checks

G4S Technology, Challenge House, Tewkesbury, UK 690m3

Haus der Barmherzigkeit, Vienna, Austria 1030m3

Task ActionTask Action

Continuous

Topological

Monitoring

Nav Learning

Routine

Task Executor

Task Action

Scheduler

Task Action

Localisation& Navigation

ExecutiveControl

ApplicationSpecific

Continuous

Continuous

Topological

Continuous

Topological

Monitoring

Continuous

Topological

Monitoring

Nav Learning

Continuous

Topological

Monitoring

Nav Learning

Task Executor

Continuous

Topological

Monitoring

Nav Learning

Task Executor

Routine

Check fire doors Check fire extinguisher

Check all doors Observe desks Patrol corridors

Check fire doors Map offices

Check all doors Observe desks Patrol corridors

Charge

Upload data Replicate database

Process maps

From 9:00 to 17:00 Weekdays, except 26/5/14

Continuous

Topological

Monitoring

Nav Learning

Task Executor

Routine

Schedulertask

task

task

task

task

Localisation& Navigation

ExecutiveControl

ApplicationSpecific

Task ActionTask Action

Continuous

Topological

Monitoring

Nav Learning

Routine

Task Executor

Task Action

Scheduler

Task Action

Care Security

Deployment 14/5/14 to 4/6/14 22/5/14 to 12/6/14

Working Hours Weekdays, 8.00 to 17.00 Weekdays, 8.45am to 17.45

Distance 27.94km 20.64km

Tasks Completed 1985 963

Autonomous Time 48h 53m 17s 26h 18m 51s

System Lifetime

Max SL 171h 0m (7d 3h 0m) 91h 0m (3d 19h 0m)

Max SL working 48h 40m (2d 0h 40m) 39h 30m (1d 15h 30m)

wait object check door check metric map desktop perception

wait patrol object check idle/engagement door check

G4S Technology, UK

Haus der Barmherzigkeit,

Austria


environments




mean time from robot

straight line time

Best 8 matches between straight-line and recorded times

mean time from robot

straight line time

Worst 8 matches between straight-line and recorded times

W1 W2

W3

0.9

action goto W2 from W1

0.1

cost mean time from all attempts

W1 W2

W3

0.9

e.g. (F W2) (eventually reach W2)

0.1

express navigation goals in Linear Temporal Logic

W1 W2

W3

0.9

0.1

W2¬W2 true

╳

B. Lacerda, D. Parker, and N. Hawes. Optimal and Dynamic Planning for Markov Decision Processes with Co-Safe LTL Specifications. In: IROS 2014.

Qualitative Spatial Relations (QSRs)

Akshaya Thippur et al. KTH-3D-TOTAL: A 3D Dataset for Discovering Spatial Structures for Long-Term Autonomous Learning. In SAIS’14.

Lars et al. Bootstrapping probabilistic models of qualitative spatial relations for active visual object search. In AAAI SS 2014 on Qualitative Representations for Robots

Object Presence Probability Pr

obab

ility

0

0.25

0.5

0.75

1

Mon

itor

Keyb

oard

Mou

se

Telep

hone

Cup/

Mug

Pen/

Penc

il

Book

Bottl

e

Desk

top

PC

Stap

ler

Lapt

op

Mob

ile p

hone

Lam

p

Calcu

lator

Keys

Head

phon

e

Glas

s

Hole

punc

h

0.0

0.5

1.0

left right front behind close distant

0.0

0.5

1.0


0.0

0.5

1.0


0.0

0.5

1.0


0.0

0.5

1.0


book wrt. monitor

mug wrt. monitor

PC wrt. monitor

keyboard wrt. monitor

mouse wrt. monitor

Position of cup relative to monitor

Position of cup relative to keyboard

Supporting planes vs QSRs 10 trials 3 out of 8 tables choose 1/500 sim. desks

L. Kunze, K. K. Doreswamy and N. Hawes. Using Qualitative Spatial Relations for Indirect Object Search. In ICRA’14.

0.0

17.5

35.0

52.5

70.0

0

2.5

5

7.5

10

Random Views Supporting Planes Correct QSRs Partially Correct QSRs Misleading QSRs

Objects Found (/10) Time (secs) Poses

3.23.1

1.1

2.3

4.8

65.0

55.0

15.6

33.6

68.5

6

8

1010

6

Search Results (Simulation)

Search Results (Robot)

0.0

17.5

35.0

52.5

70.0

0

2.5

5

7.5

10

Supporting Planes Correct QSRs

Objects Found (/10) Time (secs) Poses

1.1

2.2

33.4

69.5 10

9

Qualitative Spatial Relations (QSRs)

train: 19 desks, 3 scenes per desk = 57 scenes test: 1 desk, 3 scenes per desk = 3 scenes

Lars Kunze et al. Combining Top-down Spatial Reasoning and Bottom-up Object Class Recognition for Scene Understanding. In IROS ’14.

0.0

25.0

50.0

75.0

100.0

No Relations Learnt Metric Relations Ternary Point Calculus Ternary Point Calculus Ternary Point Calculus Distance

Relative Size

Ternary Point Calculus Distance

Relative Size Connectivity

88.9490.98

54.72

45.38

95.65

0

89.992.3

65.059.2

96.0

59.2

With Visual Classification Without Visual Classification

Classification Results (Robot)

Rares Ambrus et al. Meta-Rooms: Building and Maintaining Long Term Spatial Models in a Dynamic World. In IROS ’14.


environments




Robust, intelligent,

autonomous behaviour


environments

Novel opportunities

to learn structure

environment




Nick Hawes

Long-Term Autonomy in Everyday Environments

School of Computer Science, University of Birmingham, UK


http://nickhaw.es @hawesie

http://nickhaw.es

long-term autonomy in everyday environments - bcs.org · haus der barmherzigkeit, vienna, austria...

Documents