motion from image and inertial measurements dennis strelow honeywell advanced technology lab
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/1.jpg)
Motion from image and inertial measurements
Dennis Strelow
Honeywell Advanced Technology Lab
![Page 2: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/2.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 2
On the web
Related materials:
these slides
related papers
movies
VRML models
at:
http://www.dennis-strelow.com/umn
![Page 3: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/3.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 3
Introduction (1)
From an image sequence, we can recover:
6 degree of freedom (DOF) camera motion
without knowledge of the camera’s surroundings
without GPS
![Page 4: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/4.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 4
Introduction (2)
Fitzgibbon
![Page 5: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/5.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 5
Introduction (3)
Potential applications include:
modeling from video
Yuji Uchida
![Page 6: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/6.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 6
Introduction (4)
micro air vehicles (MAVs)
AeroVironment Black Widow AeroVironment Microbat
![Page 7: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/7.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 7
Introduction (5)
rover navigation
Hyperion Nister, et al.
![Page 8: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/8.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 8
Introduction (6)
search and rescue robots
Rhex (movies: http://ai.eecs.umich.edu/Rhex/Movies)
![Page 9: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/9.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 9
Introduction (7)
NASA Personal Satellite Assistant (PSA)
![Page 10: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/10.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 10
Introduction (8)
For these problems, we want:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
![Page 11: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/11.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 11
Introduction (8)
For these problems, we want:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
using small, light, and cheap sensors
![Page 12: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/12.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 12
Introduction (8)
For these problems, we want:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
using small, light, and cheap sensors
over the long term
![Page 13: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/13.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 13
Introduction (9)
Long-term motion estimation:
absolute distance or time is long
only a small fraction of the scene is visible at any one time
![Page 14: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/14.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 14
Introduction (10)
given these requirements, cameras are promising sensors…
…and many algorithms for motion from images already exist
![Page 15: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/15.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 15
Introduction (11)
But, where are the systems for estimating the motion of:
over the long term?
![Page 16: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/16.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 16
Introduction (12)
…and for automatically modeling
rooms
buildings
cities
from a handheld camera?
![Page 17: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/17.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 17
Introduction (13)
Motion from images suffers from some long-standing difficulties
This work attacks these problems by…
exploiting image and inertial measurements
robust image feature tracking
recognizing previously mapped locations
exploiting omnidirectional images
![Page 18: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/18.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 18
Outline
Motion from images
refresher
bundle adjustment
difficulties
Motion from image and inertial measurements
Robust image feature tracking
Long-term motion estimation
Conclusion
![Page 19: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/19.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 19
Motion from images: refresher (1)
A two-step process is common:
sparse feature tracking
estimation
![Page 20: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/20.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 20
Motion from images: refresher (1)
A two-step process is common:
sparse feature tracking
estimation
Sparse feature tracking:
inputs: raw images
outputs: projections
![Page 21: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/21.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 21
Motion from images: refresher (2)
![Page 22: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/22.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 22
Motion from images: refresher (3)
Template matching:
correlation tracking
Lucas-Kanade (Lucas and Kanade, 1981)
Extraction and matching:
Harris features (Harris, 1992)
Scale Invariant Feature Transform (SIFT) keypoints (Lowe, 2004)
![Page 23: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/23.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 23
Motion from images: refresher (4)
The second step is estimation:
inputs:
projections
outputs:
6 DOF camera position at the time of each image
3D position of each tracked point
![Page 24: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/24.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 24
Motion from images: refresher (5)
![Page 25: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/25.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 25
Motion from images: refresher (6)
bundle adjustment (various, 1950’s)
Kalman filtering (Broida, Chandrashekhar, and Chellappa, 1990)
variable state dimension filter (VSDF) (McLauchlan, 1996)
two- and three-frame methods(Hartley and Zisserman, 2000; Nister, et al. 2004)
![Page 26: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/26.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 26
Motion from images: bundle adjustment (1)
From tracking, we have the image locations xij for each point j and each image i
![Page 27: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/27.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 27
Motion from images: bundle adjustment (2)
Suppose we also have estimates of:
the camera rotation ρi and translation ti at time of each image
3D point positions Xj of each tracked point
Then, we can compute reprojections:
))(( iji tXR
![Page 28: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/28.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 28
Motion from images: bundle adjustment (3)
![Page 29: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/29.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 29
Motion from images: bundle adjustment (4)
![Page 30: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/30.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 30
Motion from images: bundle adjustment (5)
So, minimize:
with respect to all the ρi, ti, Xj
)),)(((,
image ijijji
i xtXRDE
![Page 31: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/31.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 31
Motion from images: bundle adjustment (5)
So, minimize:
with respect to all the ρi, ti, Xj
)),)(((,
image ijijji
i xtXRDE
![Page 32: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/32.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 32
Motion from images: difficulties (1)
Estimation step can be very sensitive to…
incorrect or insufficient image feature tracking
camera modeling and calibration errors
outlier detection thresholds
sequences with degenerate camera motions
![Page 33: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/33.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 33
Motion from images: difficulties (2)
Iterative batch methods have poor convergence or may fail to converge if:
observations are missing
the initial estimate is poor
![Page 34: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/34.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 34
Motion from images: difficulties (3)
Recursive methods suffer from:
poor prior assumptions on the motion
poor approximations in state error modeling
![Page 35: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/35.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 35
Motion from images: difficulties (4)
Resulting errors are:
gross local errors
long term drift
![Page 36: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/36.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 36
Motion from images: difficulties (5)
![Page 37: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/37.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 37
Motion from images: difficulties (6)
151 images, 23 pointsmanually corrected Lucas-Kanade
![Page 38: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/38.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 38
Motion from images: difficulties (7)
squares: ground truth points dash-dotted line: accurate estimate solid line: image-only, bundle adjustment estimate
![Page 39: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/39.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 39
Outline
Motion from images
Motion from image and inertial measurements
inertial sensors
algorithms and results
related work
Robust image feature tracking
Long-term motion estimation
Conclusion
![Page 40: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/40.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 40
Motion from image and inertial measurements: inertial sensors (1)
inertial sensors can be integrated to estimate six degree of freedom motion
![Page 41: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/41.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 41
Motion from image and inertial measurements: inertial sensors (2)
But many applications require small, light, and cheap sensors
![Page 42: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/42.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 42
Motion from image and inertial measurements: inertial sensors (3)
Integrating the outputs of these low grade sensors will produce drifting motion because of:
noise
unmodeled nonlinearities
![Page 43: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/43.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 43
Motion from image and inertial measurements: inertial sensors (4)
And, we can’t even integrate until we can separate the effects of…
rotation ρ
gravity g
acceleration a
slowly changing bias ba
noise n
…in the accelerometer measurements
![Page 44: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/44.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 44
Motion from image and inertial measurements: inertial sensors (5)
Image and inertial measurements are highly complementary
With inertial measurements we can:
decrease sensitivity in image-only estimates
establish two rotation angles without drift
establish the global scale
![Page 45: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/45.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 45
Motion from image and inertial measurements: inertial sensors (5)
Image and inertial measurements are highly complementary
With inertial measurements we can:
decrease sensitivity in image-only estimates
establish two rotation angles without drift
establish the global scale
…even with our low-grade sensors
![Page 46: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/46.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 46
Motion from image and inertial measurements: inertial sensors (6)
With image measurements, we can:
reduce the drift in integrating inertial data
distinguish between… rotation ρ
gravity g
acceleration a
bias ba
noise n
…in accelerometer measurements
![Page 47: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/47.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 47
Motion from image and inertial measurements: algorithms and results (1)
This work has developed both:
batch
recursive
algorithms for motion from image and inertial measurements
![Page 48: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/48.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 48
Motion from image and inertial measurements: algorithms and results (2)
Gyro measurements:
ω’, ω: measured and actual angular velocity
bω: gyro bias
n: gaussian noise
nb '
![Page 49: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/49.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 49
Motion from image and inertial measurements: algorithms and results (3)
Accelerometer measurements:
ρ: rotation
a’, a: measured and actual acceleration
g: gravity vector
ba: accelerometer bias
n: gaussian noise
nbgaRa' aT )((
![Page 50: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/50.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 50
Motion from image and inertial measurements: algorithms and results (4)
batch algorithm minimizes a combined error:
inertialimagecombined EEE
![Page 51: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/51.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 51
Motion from image and inertial measurements: algorithms and results (5)
image term Eimage is the same as before
![Page 52: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/52.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 52
Motion from image and inertial measurements: algorithms and results (6)
inertial error term Einertial is:
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
![Page 53: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/53.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 53
Motion from image and inertial measurements: algorithms and results (6)
inertial error term Einertial is:
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
![Page 54: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/54.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 54
Motion from image and inertial measurements: algorithms and results (6)
inertial error term Einertial is:
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
![Page 55: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/55.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 55
Motion from image and inertial measurements: algorithms and results (7)
,...))(,( 1n,translatio itii tItDE
timeτi-1 (time of image i - 1)
ti-1
ti
I(ti-1, …)
τi (time of image i)
tran
slat
ion
( : translation estimate for image i – 1)
( : translation estimate for image i)
( : translation integrated from previous estimate)
![Page 56: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/56.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 56
Motion from image and inertial measurements: algorithms and results (8)
timeτ0
tran
slat
ion
τ1 τ2 τ5 τ3 τ4 τf-3 τf-2 τf-1
1
1n,translationtranslatio
f
iiEE
![Page 57: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/57.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 57
Motion from image and inertial measurements: algorithms and results (9)
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
![Page 58: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/58.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 58
Motion from image and inertial measurements: algorithms and results (10)
It(τi-1, τi ,…, ti-1) depends on:
τi-1, τi (known)
all inertial measurements for times τi-1< τ < τi (known)
ρi-1, ti-1
g
bω, ba
camera linear velocities: vi
![Page 59: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/59.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 59
Motion from image and inertial measurements: algorithms and results (12)
dash-dotted line: batch estimate from image and inertial solid line: image-only, bundle adjustment estimate squares: ground truth points
![Page 60: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/60.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 60
Motion from image and inertial measurements: algorithms and results (13)
IEKF for the same sensors, unknowns dash-dotted line: batch estimate solid line: IEKF estimate
![Page 61: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/61.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 61
Motion from image and inertial measurements: algorithms and results (14)
Difficulties with IEKF for this application:
prior assumptions about motion smoothness
cannot model relative error between adjacent camera positions
So, converting the batch algorithm into a variable state dimension filter (VSDF) is a promising future direction
![Page 62: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/62.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 62
Motion from image and inertial measurements: algorithms and results (15)
IEKF assumptions on motion smoothness dash-dotted line: batch estimate solid line: IEKF estimate
right: IEKF propagation variances too strict
left: IEKF propagation variances just right
![Page 63: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/63.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 63
Motion from image and inertial measurements
Recap:
image, gyro, and accelerometer measurements
batch algorithm
recursive algorithm
experiments
evaluate batch and recursive algorithms
establish basic facts about motion from image and inertial measurements
![Page 64: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/64.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 64
Outline
Motion from images
Motion from image and inertial measurements
Robust image feature tracking
Lucas-Kanade and real sequences
The “smalls” tracker
Long-term motion estimation
Conclusion
![Page 65: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/65.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 65
Robust image feature tracking: Lucas-Kanade and real sequences (1)
Combining image and inertial measurements improves our situation, but…
we still need accurate feature tracking tracking
some sequences do not come with inertial measurements
![Page 66: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/66.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 66
Robust image feature tracking: Lucas-Kanade and real sequences (2)
better feature tracking for improved 6 DOF motion estimation
remaining results will be image-only
![Page 67: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/67.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 67
Robust image feature tracking: Lucas-Kanade and real sequences (3)
Lucas-Kanade has been the go-to feature tracker for shape-from-motion
minimizes a correlation-like matching error
using general minimization
evaluates the matching error at only a few locations
subpixel resolution
![Page 68: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/68.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 68
Robust image feature tracking: Lucas-Kanade and real sequences (4)
Additional heuristics used to apply Lucas-Kanade to shape-from-motion:
task: heuristic:
choose features to track high image texture
identify mistracked, occluded, no-longer-visible
convergence, matching error
handle large motions image pyramid
![Page 69: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/69.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 69
Robust image feature tracking: Lucas-Kanade and real sequences (5)
But Lucas-Kanade performs poorly on many real sequences…
![Page 70: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/70.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 70
Robust image feature tracking: the “smalls” tracker (1)
smalls is a new feature tracker targeted at 6 DOF motion estimation
exploits the rigid scene assumption
eliminates the heuristics normally used with Lucas-Kanade
SIFT is an enabling technology here
![Page 71: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/71.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 71
Robust image feature tracking: the “smalls” tracker (2)
First step: epipolar geometry estimation
use SIFT to establish matches between the two images
get the 6 DOF camera motion between the two images
get the epipolar geometry relating the two images
![Page 72: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/72.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 72
Robust image feature tracking: the “smalls” tracker (3)
![Page 73: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/73.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 73
Robust image feature tracking: the “smalls” tracker (4)
![Page 74: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/74.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 74
Robust image feature tracking: the “smalls” tracker (5)
Second step: track along epipolar lines
use nearby SIFT matches to get initial position on epipolar line
exploits the rigid scene assumption
eliminates heuristic: pyramid
![Page 75: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/75.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 75
Robust image feature tracking: the “smalls” tracker (6)
Third step: prune features
geometrically inconsistent features are marked as mistracked and removed
clumped features are pruned
eliminates heuristic: detecting mistracked features based on convergence, error
![Page 76: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/76.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 76
Robust image feature tracking: the “smalls” tracker (7)
Fourth step: extract new features
spatial image coverage is the main criterion
required texture is minimal when tracking is restricted to the epipolar lines
eliminates heuristic: extracting only textured features
![Page 77: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/77.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 77
Robust image feature tracking: the “smalls” tracker (8)
![Page 78: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/78.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 78
Robust image feature tracking: the “smalls” tracker (9)
left: odometry only right: images only
average error: 1.74 m
maximum error: 5.14 m
total distance: 230 m
![Page 79: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/79.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 79
Robust image feature tracking: the “smalls” tracker (10)
Recap:
exploits the rigid scene and eliminates heuristics
allows hands-free tracking for real sequences
can still be defeated by textureless areas or repetitive texture
![Page 80: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/80.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 80
Outline
Motion from images
Motion from image and inertial measurements
Robust image feature tracking
Long-term motion estimation
proof of concept system
experiment
Conclusion
![Page 81: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/81.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 81
Long-term motion estimation: proof of concept system (1)
Image-based motion estimates from any system will drift:
if the features we see are always changing
given sufficient time
if we don’t recognize when we’ve revisited a location
![Page 82: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/82.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 82
Long-term motion estimation: proof of concept system (2)
![Page 83: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/83.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 83
Long-term motion estimation: proof of concept system (3)
To limit drift:
recognize when we’ve returned to a previous location
exploit the return
A proof of concept system demonstrates these capabilities
![Page 84: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/84.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 84
Long-term motion estimation: proof of concept system (4)
“smalls” tracker state: 2D feature history for images in I
variable state dimension filter (VSDF) state for images in I: 6 DOF camera positions, covariances for images in I 3D positions for features visible in I
SIFT keypoints for image in
system state S
image indices: I = {i1, …, in}
![Page 85: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/85.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 85
Long-term motion estimation: proof of concept system (5)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2} {0, 1, …, 8}
![Page 86: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/86.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 86
rollback
Long-term motion estimation: proof of concept system (6)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2} {0, 1, …, 8}
non-rollback States:
![Page 87: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/87.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 87
rollback
Long-term motion estimation: proof of concept system (7)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2} {0, 1, …, 8}
8
non-rollback States:
![Page 88: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/88.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 88
rollback
Long-term motion estimation: proof of concept system (8)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2}
8
{0, 1, 2, 3, 8}
non-rollback pruned States:
![Page 89: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/89.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 89
rollback
Long-term motion estimation: proof of concept system (9)
0 1 2 3 4 5 6 7 8
8 9 10 11
11 12 13 14
14 15 16 17
17 18 19 20
{0, …, 6, 11, 12, 17, …, 20}
non-rollback pruned States:
![Page 90: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/90.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 90
Long-term motion estimation: proof of concept system (10)
When to “roll back”?
examine the camera covariances for the current state and the candidate rollback state
check the number of SIFT matches
extend from the candidate state
examine the camera covariances for the current state and the resulting extended state
![Page 91: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/91.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 91
Long-term motion estimation: experiment (1)
CMU FRC highbay views; 945 images total
![Page 92: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/92.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 92
Long-term motion estimation: experiment (2)
CMU FRC highbay
![Page 93: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/93.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 93
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first forward pass: images 0-213)
![Page 94: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/94.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 94
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first forward pass: images 0-213)
![Page 95: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/95.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 95
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first forward pass: images 0-213)
![Page 96: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/96.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 96
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first backward pass: images 214-380)
![Page 97: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/97.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 97
rollback
Long-term motion estimation: experiment (3)
0 1 2 3 4 5 6 7 8
8 9 10 11
11 12 13 14
14 15 16 17
17 18 19 20
non-rollback pruned States:
normally, the system produces a general tree of states
![Page 98: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/98.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 98
Long-term motion estimation: experiment (4)
…0 1 2 3 4 5 6 7
13 14 15 14
16 17 18 17
non-rollback rollback pruned States:
for this example, the “rollback” states are restricted to the first forward pass
8 9
10 11 12 14
14
213
![Page 99: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/99.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 99
Long-term motion estimation: experiment (5)
movie…bottom half is smalls output:
![Page 100: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/100.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 100
Long-term motion estimation: experiment (6)
movie…top half is motion estimates:
![Page 101: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/101.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 101
Long-term motion estimation: experiment (7)
movie…top half is motion estimates:
![Page 102: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/102.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 102
Outline
Motion from images
Motion from image and inertial measurements
Robust image feature tracking
Long-term motion estimation
Conclusion
remaining issues
![Page 103: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/103.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 103
Conclusion: remaining issues
all: system is experimental, not optimized for speed
image and inertial: VSDF
“smalls”: integration of gyro, more robustness to poor texture needed
long-term: “roll back” space, computation grow with sequence length
![Page 104: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/104.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 104
Thanks!
Related materials:
these slides
related papers
movies
VRML models
at:
http://www.dennis-strelow.com/umn
![Page 105: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/105.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 105
Motion from omnidirectional images (1)
![Page 106: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/106.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 106
Motion from omnidirectional images (2)
![Page 107: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/107.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 107
Motion from omnidirectional images (3)
![Page 108: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/108.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 108
Motion from omnidirectional images (4)
![Page 109: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/109.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 109
Motion from omnidirectional images (5)
left: non-rigid camera right: rigid camera
squares: ground truth points solid: image-only estimates
dash-dotted: image-and-inertial estimates
![Page 110: Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab](https://reader037.vdocuments.net/reader037/viewer/2022103123/56649d3e5503460f94a1799d/html5/thumbnails/110.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 110
Motion from omnidirectional images (6)
In this experiment:
omni images
conventional images + inertial
have roughly the same advantages
But in general:
inertial has some advantages that omni images alone can’t produce
omni images can be harder to use