structure-from-motion algorithm · 2011-10-13 · © 2006 noah snavely reproduced with permission...
TRANSCRIPT
© 2006 Noah Snavely
Structure-from-Motion Algorithm
Bundle Adjustment
© 2006 Noah Snavely
15,464
76,389
37,383
© 2006 Noah Snavely
Reproduced with permission of Yahoo! Inc. © 2005 by Yahoo! Inc.
YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.
© 2006 Noah Snavely
Structure from motion
© 2006 Noah Snavely
Photo Tourism overview
Scene
reconstruction
Photo
Explorer Input photographs Relative camera
positions and orientations
Point cloud
Sparse correspondence
© 2006 Noah Snavely
Scene reconstruction
• Automatically estimate
– position, orientation, and focal length of cameras
– 3D positions of feature points
Feature detection
Pairwise
feature matching
Incremental
structure
from motion
Correspondence
estimation
© 2006 Noah Snavely
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
© 2006 Noah Snavely
Feature detection
• Detect features using SIFT [Lowe, IJCV 2004]
© 2006 Noah Snavely
Pairwise feature matching
• Match features between each pair of images
© 2006 Noah Snavely
Feature matching
Refine matching using RANSAC [Fischler & Bolles 1987]
to estimate fundamental matrices between pairs
© 2006 Noah Snavely
Structure from motion
Camera 1
Camera 2
Camera 3
R1,t1
R2,t2
R3,t3
p1
p4
p3
p2
p5
p6
p7
minimize
f (R, T, P)
© 2006 Noah Snavely
Minimize the re-projection error
Estimate:-M = Projection matrix, X = 3D points
© 2006 Noah Snavely
Levenberg–Marquardt algorithm
Minimize
Where, , J = Jacobian matrix
At its minimum, the sum of squares, S(β), the gradient of S with respect to δ will be zero.
The above first-order approximation of gives
In the vector notation,
Taking the derivative with respect to δ and setting the result to zero gives,
© 2006 Noah Snavely
Incremental structure from motion
© 2006 Noah Snavely
Incremental structure from motion
© 2006 Noah Snavely
3D reconstruction
© 2006 Noah Snavely
Demo
© 2006 Noah Snavely
• Advantages
Handle large number of views
Handle missing data
• Limitations
Large minimization problem (parameters grow
with number of views) – Takes lot of time
requires good initial condition
© 2006 Noah Snavely
REFERENCES:
Noah Snavely, Steven M. Seitz, Richard Szeliski. Photo Tourism: Exploring image
collections in 3D. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006),
2006.
Noah Snavely, Steven M. Seitz, Richard Szeliski. Modeling the World from Internet
Photo Collections. International Journal of Computer Vision, 2007.
Source Code: http://phototour.cs.washington.edu/bundler/#S4
Youtube Link: http://www.youtube.com/watch?v=9M4KWgRGNa0
Slides Courtesy by :- Noah Snavely, Assistant Professor, Cornell University.