fp7-proviscoutrobotics.estec.esa.int/astra/astra2011/presentations... · 2011. 5. 12. · page 2 11...
TRANSCRIPT
![Page 1: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/1.jpg)
Pa
ge
1
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
FP7-PRoViScout – Planetary Robotics Vision Scout Gerhard Paar(1), Mark Woods (2) & the PRoViScout Team
(1) Joanneum Research, Graz, Austria
(2) Scisys Ltd., Bristol, UK
11th ASTRA Conference, ESTEC, The Netherlands,
April 12-14, 2011
![Page 2: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/2.jpg)
Pa
ge
2
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
FP7-SPACE Call 2008 (ITT): The Objectives
— Call Topic: „Space Exploration―
— Optimum preparation of scientific payloads
— Effective scientific exploitation of future space missions data
— Enhance effectiveness & productivity in usage of data,
including archived data
— Improve capabilities to move, to select and collect and finally
return samples to Earth
— Increase public awareness
— Complementary to and in close co-operation with ESA
![Page 3: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/3.jpg)
Pa
ge
3
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Heritage: EUROBOT, ExoMars, PRoVisG, CREST, MER – 12 Partners
MSSL/UCL: Exobiology & WALI
DLR: Geology
JR: 3D Vision & PR
AU: Robotics & Platform / IP
JPL: Autonomous Science on MER
SciSys: Autonomous Scientist & SW Framework
CTU: Vision & www
GMV: Navigation
CSEM: Zoom 3D-TOF & RGB Cam
TRS: Simulation
UoS: Decision Module
ULeic: Geology
Science
![Page 4: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/4.jpg)
Pa
ge
4
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Objectives: Efficient Scout Mission
© Scisys
![Page 5: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/5.jpg)
Pa
ge
5
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Target2: Sample Fetching Rover
©
See http://www.youtube.com/watch?v=uCQw5dcqUaI
![Page 6: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/6.jpg)
Pa
ge
6
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Where to go for Optimum Result
Identify / select targets
and reach the best one !
![Page 7: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/7.jpg)
Pa
ge
7
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Technical Approach: PRoViSC
!!
Planetary Robotics Vision Scouting Chain
Planetary Robotics
Vision Monitoring
& PRoViM
![Page 8: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/8.jpg)
Pa
ge
8
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Work Packages
PROVISC &
On-Board System
WP 4
Science Target
Selection
WP 3
Consolidation
WP 2
Simulation & Testing
WP 6
Exploitation
WP 7
Www.proviscout.eu
Spin Off
Science
Expertise
Missions
Expertise
Tools
PROVIM
WP 5
Publication
Workshops
Field Test
Vision Scientists
Planetary Scientists
Management
WP 1
Meetings,
Technical & Scientific Mgt
EC
Financial Reporting
214 PM
Funding:
• 50%... Industry
• 75%... R&D
• 100 %... Management
• 100 %... Dissemination
30 Months
April 2010 – Sept 2012
![Page 9: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/9.jpg)
Pa
ge
9
Ve
rsio
n:
13
.04
.20
11
G
erh
ard
.pa
ar@
joa
nn
eu
m.a
t
PRoViScout
Planetary Science Aspects
— Site selection & Assessment
— Criteria
• Accessibility
• Science Richness
• (Weather) Safety
• Traversability
• Lack of vegetation
— Clarach Bay, Ceredigion
— Morocco
— Tenerife
— Science Requirements
— Training Image Definition & Masks
© Univ. Leicester
© Univ. Leicester
![Page 10: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/10.jpg)
Pa
ge
10
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Technical Approach: Platforms & Sensors
© Aberystwyth University
© Aberystwyth University
![Page 11: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/11.jpg)
Pa
ge
11
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
© CSEM / Zurich
Innovation in Space Robotics: Zoomed3D-TOF & RGB
— 20W optical peak power @ 15 MHz Modulation
— Distance range: 1m – 10m
— FoV (Diagonal) 7° (zoomed) – 40° (Wide Angle)
— Real-Time Mapping
— Full Sunlight resolution (st.dev)
0.01 m@ 1m (full field)
0.2 m@ 5m (full field)
0.003 m@ 1m (full zoom)
0.02 m@ 5m (full zoom)
— RGB 1 Mpix camera (IR filter, optional)
— Common beam path of TOF & RGB
— Drawbacks: Dimensions / Weight (& Eye Safety)
![Page 12: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/12.jpg)
Pa
ge
12
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Software Components
© Scisys
Problem:
High effort into
integration of
heterogeneous
components
![Page 13: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/13.jpg)
Pa
ge
13
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
3D Vision Processing Chain
Ortho image DEM
Hazard map Slope map
Path Planning
![Page 14: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/14.jpg)
Pa
ge
14
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Example: 3D Vision Server
![Page 15: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/15.jpg)
Pa
ge
15
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Navigation
© GMV, 2011
All rights reserved
![Page 16: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/16.jpg)
Pa
ge
16
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Classification Scheme 1 Feature Extraction (SIFT) Classifier (NN)
Representative Test images using the same classifier:
![Page 17: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/17.jpg)
Pa
ge
17
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Training Accumulation ROIs
SIFT / SURF Hits
![Page 18: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/18.jpg)
Pa
ge
18
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Science Assessment Interface Function Description Parameters (in/out) Constraints Data Size
(Large/Small) Required By
(Called from)
Determine
science targets
in 3D (*)
Controls the detection of
science targets in imagery.
Provides the high level
Interface to other modules.
RGB image [in];
3D science target
coordinates (X,Y,Z of
type double),
science target
classes, weights
(double) [out]
Large bandwidth for
reading [in] images
and the scientific
database description.
Little bandwidth for
output [out]
Executive
Determine Local
Interest
Candidates
Inspects an RGB image to
determine particular cues
for the detection of science
targets in 2d image space,
assigning weights prop. to
each candidate.
RGB Image [in]
Local Interest
Candidates (2D
positions in image
and priorities) [out]
Candidate coordinates
within image dimension,
priorities 0..100 (0 low)
Internal (*)
Determine
Region of
interest (ROI) for
science targets
Weighted integration of
local interest candidates in
order to make a decision for
a (2D) segmentation into
science target(s). The
integration step will utilize
the scientific database for
matching candidates.
Local Interest
Candidates (2D
positions in image
and priorities) [in]
Requests on science
database (description of
features, etc.). Calls the
Target Localization
Module finally to return
3D position of the
determined scientific
target.
Internal (*)
Multiple cues for the Decision Module
![Page 19: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/19.jpg)
Pa
ge
19
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Multi – Cue Information & Scale
Context…
Hit of
Scientific
Interest
![Page 20: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/20.jpg)
Pa
ge
20
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Multi – Cue Information & Scale
ROI Context…
![Page 21: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/21.jpg)
Pa
ge
21
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Multi – Cue Information & Scale
ROI…
![Page 22: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/22.jpg)
Pa
ge
22
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Multi – Cue Information & Scale
Target Context…
![Page 23: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/23.jpg)
Pa
ge
23
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Multi – Cue Information & Scale
Target
& 3D Target
![Page 24: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/24.jpg)
Pa
ge
24
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Technical Approach: PRoViM - To understand system behavior
©: all Trasys
Heritage:
3D-ROV
![Page 25: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/25.jpg)
Pa
ge
25
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
PRoViM – to – www
Window monitor
Window monitor
Window monitor
Screen-shot memory
(file system, RAM, …)
www server
© Trasys; CTU Prague
![Page 26: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/26.jpg)
Pa
ge
26
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Status as from April 2011 (M13)
— Design & 70% Prototype components finished
— 3D-TOF & RGB to be assembled
— Science Requirements finished
— Test / training data & samples forthcoming
— Aerobots mapping test successful
— Field Assessment in Tenerife June 2011
— Hints welcome !
— First SW Integration Campaign June 2011
— HW Components Integration Campaign October 2011
— Field Test Summer 2012 @ Tenerife
![Page 27: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/27.jpg)
Pa
ge
27
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Other (&) Innovative Aspects (Selection) — 3D-TOF & Zoom & RGB & Space Application
— WALI (Wide Angle Laser Imager) as Flourescent Organics cue
— SpaceMaster Programme Link
(Students – into – Project)
— Combination Science / Navigation
Decision System
— Aerobot & Rover Combination (Offline ?)
— Science & System Integrated
— Towards MSR
— Towards Planetary Science Agent Scenarios
— Geologists – in – the – loop
— Compare with Human Spaceflight (Costs vs. Performance Issues)
![Page 28: FP7-PRoViScoutrobotics.estec.esa.int/ASTRA/Astra2011/Presentations... · 2011. 5. 12. · Page 2 11 t PRoViScout FP7-SPACE Call 2008 (ITT): The Objectives —Call Topic: „Space](https://reader035.vdocuments.net/reader035/viewer/2022071019/5fd2a2d0b190261ee73cf539/html5/thumbnails/28.jpg)
Pa
ge
28
V
ers
ion
: 1
3.0
4.2
01
1
Ge
rha
rd.p
aa
r@jo
an
ne
um
.at
PRoViScout
Related Presentations @ ASTRA — M. Woods: High-Level Autonomy
Wed 10:25, Session 1A
— A. Medina: Heritage EUROBOT
Thu 12:10, Session 6A
— L. Tyler: Aerobot
Thu 14:00, Session 7A
www.proviscout.eu
© SciSys
© gmv
© Aberystwyth University