robot vision part 2: visual odometry - mcgill...
TRANSCRIPT
![Page 1: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/1.jpg)
Robot Vision Part 2: Visual Odometry
Guest Lecture by David MegerMcGill CS 417
November 6, 2013
![Page 2: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/2.jpg)
Last Time: Object Recognition
![Page 3: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/3.jpg)
Today: Visual Odometry
● Videos...
![Page 4: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/4.jpg)
Visual odometry problem
● Input: ● Images from a moving camera: Robot's-eye-video,
cell phone video feed, automobile safety system feed, GoPro, tourist photos
● Output: ● The location of the camera when each image was
taken● (optional) information about the geometry of the
world
![Page 5: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/5.jpg)
How?
![Page 6: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/6.jpg)
Pinhole Camera Model
C
(R,t)
![Page 7: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/7.jpg)
Epipolar Geometry
(R,t)
![Page 8: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/8.jpg)
Solving for E from images
● E holds the geometric information that we require for our visual odometry solution
● m'Em=0 for all corresponding points● Each pair of points constrains possible values
for E● Constructing a linear system from 5 or more
matches allows solving for E● How can we find good matches?
![Page 9: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/9.jpg)
Several ways to find candidate matches
● Interest point matching in every image
● Use temporal consistency between frames in video:● Find longer feature tracks● Simpler processing to find
the “flow”
![Page 10: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/10.jpg)
RANSAC
•Repeat:•pick K matches•solve for E •find # inliers•If #inliers > threshold, break
•Output E as odometry estimate
![Page 11: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/11.jpg)
RANSAC
![Page 12: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/12.jpg)
RANSAC
![Page 13: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/13.jpg)
RANSAC
![Page 14: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/14.jpg)
Optical Flow
![Page 15: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/15.jpg)
Optical Flow
![Page 16: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/16.jpg)
Optical Flow
![Page 17: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/17.jpg)
Putting it together: Visual Odometry Solution
● Running libviso2 example code...
![Page 18: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/18.jpg)
Visual Odometry on real robots
● Camera is usually not the only sensor:● Combine visual odometry with compass,
accelerometer, gyroscope to get intial guesses at motion
● Maps help things stay consistent over long periods:● Can save key-frame images, perhaps tagged with
GPS coordinates and match to those sporadically
![Page 19: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/19.jpg)
Related Problem: Bundle Adjustment
![Page 20: Robot Vision Part 2: Visual Odometry - McGill CIMyiannis/417/2013/LectureSlides/CS417_visodom_2013.pdf · Visual odometry problem Input: Images from a moving camera: Robot's-eye-video,](https://reader034.vdocuments.net/reader034/viewer/2022042811/5fa041da7964b03cb45c3065/html5/thumbnails/20.jpg)
Questions
20 / 20