Mauro Dalla Mura, Michele Zanin, Claudio Andreatta, Paul Chippendale,
Fondazione Bruno Kessler, Trento, Italy
Augmented Reality: Fusing the Real and
Synthetic Worlds
2 M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale
Morphological and Attribute Profiles Spectral/spatial analysis
IEEE IGARSS’12
What is Augmented Reality?
3 M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale
Morphological and Attribute Profiles Spectral/spatial analysis
IEEE IGARSS’12
What is Augmented Reality?
Can it be useful for the GRS community?
Outline
1. Introduction to Augmented Reality
2. AR and GRS: the proposed concept
3. Augmenting the user’s view
4. Augmenting geo-data
5. Discussion and conclusion
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmented Reality
Augmented Reality (AR) blends virtual and real scenes in both time and space
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmented Reality
Augmented Reality (AR) blends virtual and real scenes in both time and space
1950s 1966
Nowadays, AR technology starts to run on consumer level mobile devices (e.g.,
smartphones and tablets) in real-time
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmented Reality
Augmented Reality (AR) blends virtual and real scenes in both time and space
1950s 1966 2000
Nowadays, AR technology starts to run on consumer level mobile devices
(e.g., smartphones and tablets) in real-time
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmented Reality
Augmented Reality (AR) blends virtual and real scenes in both time and space
1950s 1966 2000 2008
Nowadays, AR technology starts to run on consumer level mobile devices
(e.g., smartphones and tablets) in real-time
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmented Reality
Augmented Reality (AR) blends virtual and real scenes in both time and space
1950s 1966 2000 2008 2014
Nowadays, AR technology starts to run on consumer level mobile devices
(e.g., smartphones and tablets) in real-time
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmented Reality
Augmented Reality (AR) blends virtual and real scenes in both time and space
1950s 1966 2000 2008 2014 . . .
Nowadays, AR technology starts to run on consumer level mobile devices
(e.g., smartphones and tablets) in real-time
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
AR applications
Gaming
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
AR applications
Gaming Entertainment
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
AR applications
Gaming Entertainment Sport
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
AR applications
Gaming Entertainment Sport Commercial
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
AR applications
Gaming Entertainment Sport Commercial
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Urban planning
AR applications
Gaming Entertainment Sport Commercial
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Urban planning Tourism
AR applications
Gaming Entertainment Sport Commercial
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Urban planning Tourism Navigation
AR applications
Gaming Entertainment Sport Commercial
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Urban planning Tourism Navigation . . .
. . .
Augmented Reality for GRS Augmented Reality exploiting geo-data
Availability of massive archives of geo-referenced data (e.g., remote sensing,
geo-referenced social photo websites, GIS, geo-data)
A possible exploitation of geo-data
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmentation of a 2D map superimposing the corresponding 3D model
Effective integration of AR and geo-data is becoming feasible on consumer-
level portable devices
Augmented Reality for GRS Available applications showing geo-data on smart-phones
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Layar (since Jun 2009) Peak.AR (since Oct 2009)
Wikitude (since May 2009)
SwissPeaks (since Nov 2009)
Peaks Mixare (since Mar 2010)
Device-world registration is usually not accurate (e.g., errors in horizontal orientation of 30 deg if using the on-board compass)
Photo-world registration Accurate alignment of the device and the
surrounding panorama is necessary
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Captured Photograph DEM
2D Rendered DEM
Aligned Photograph
Salient feature extraction
Photo registration
Camera position (ϕ0, λ0, h0) and pose (α, β, γ) should be estimated
Vision based approaches operates by finding the best match between visual features and synthetic ones (from a 3D model)
Hardware information (e.g., magnetometer, compass, GPS) can be used as initial clue of the pose
After photo-world registration, each pixel in the image can be associated to geographic coordinates
Panorama rendering
AR for GRS: the proposed concept
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Reality augmentation
Augmenting the user’s view
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmenting the user’s view Once registered, geo-data can
be delivered to the user Geographical toponyms GIS layers Topographic maps 3D building models . . .
New modality to query and access geo-data
Fast interactive modality (e.g., touchscreen)
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmenting the geoscientist’s view
Support tool for field campaigns: Real-time guidance for field surveys (e.g., ground truth collection) In-situ validation of remote sensing data and products (e.g., land cover maps)
Aid the interpretation of geo-referenced RS data data not in the visible range time series
Forest regions from Corine land
cover 2000 (25m resolution)
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Topographic map
Reality augmentation
Augmenting the geo-data archives
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Augmenting the geo-data archives
Users’ portable devices become
ground-based sensors
r c
ϕ
λ
Digital Elevation Model
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
position pose focal length
(r, c) (ϕ, λ)
f(F, ϕ0, λ0, h0, α, β, γ, DEM)
Ground-based sensing
Extract Information on the land cover
November snowline
>1785 m
December snowline >1212 m
Oct. 2007
Nov. 2007
Dec. 2007
Snow cover map
0
0.2
0.4
0.6
0.8
1
86
0
10
60
12
60
14
60
16
60
18
60
20
60
22
60
24
60
26
60
28
60
30
60
Altitude (m)
Sno
w (
pe
rce
nta
ge)
Sept Nov Jan
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Ground-based sensors
Extract Information on the land cover Availability of multiple images on the
same area 3D models with photogrammetry Analysis of the land cover evolution
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Ground-based sensors
Extract Information on the land cover Availability of multiple images on the
same area 3D models with photogrammetry Analysis of the land cover evolution
Capture vertical surfaces Massive databases of geo-tagged images
(for free!) Near-live acquisition (e.g., for damage
assessment)
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Ground-based sensors
Extract Information on the land cover Availability of multiple images on the
same area 3D models with photogrammetry Analysis of the land cover evolution
Capture vertical surfaces Massive databases of geo-tagged images
(for free!) Near-live acquisition (e.g., for damage assessment)
Limited spatial coverage Uneven resolution Occlusions
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Discussion and conclusion
The effectiveness, quality and accuracy of the AR result depends on many factors: Geo-data projection and rendering are demanding operations and should be
performed fast Estimation of the camera pose and position for achieving a precise match for
accurately overlay virtual information on the scene and for acquiring information on the landscape
High-speed wireless transfer of data (i.e., optimize network transfers)
Conclusion Hardware and software specification of consumer-level portable devices are
constantly improving The exploitation of portable devices for AR applications has just started We proposed a concept that exploiting AR technology for GRS applications Thanks to an accurate registration of the device to the world it is possible to
perform visualization of geo-data on the mobile device interactively exploitation of the mobiles’ camera as a ground-based imaging sensors
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]
Introduction Discussion and conclusion Morphological and Attribute Profiles Spectral/spatial analysis
from http://www.gls-tedesco-a-berlino.de/2065.html
Thanks for your attention!
https://venturi.fbk.eu/ http://tev.fbk.eu/marmota/
M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale IEEE IGARSS’12 [email protected]