© 2005 martin bujňák, [email protected], dataexpert.sk martin bujňák [email protected]...
TRANSCRIPT
![Page 1: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/1.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Martin Bujňá[email protected]
Supervisor : RNDr. Martin Samuelčík
On-line Structure from Motion
![Page 2: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/2.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Aim
• Reconstruction of sparse 3D model of the scene using video sequence
• Input : video sequence captured with standard hand-held camera
• As small as possible user interactions
• Output :– Sparse 3D model– Camera calibration parameters
• Intrinsic
• Extrinsic
![Page 4: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/4.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Feature detection
• Harris based detector– Removing features that can be
interchanged with its neighborhood
– Introducing feature orientation
![Page 5: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/5.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Features matching
• Find the nearest (distance) similar (correlation) feature in the next frame
• Matching ambiguities– two neighboring frames don’t
differ a lot – „outliers“ are effectively
removed in further processes (up to 30% of bad pairs is expected)
??
![Page 6: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/6.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Guided searching
• F-matrix : two view geometry– Restricts searching for
matching pair to 1D search region in the second frame
– Using “stereo” principle• Epipolar line ordering constrain• Ratio of corresponding
segments is near 1 for small motion between two frames
?x
![Page 8: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/8.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Projective reconstruction• Canonical camera pair from
F-matrix– F-matrix calculated using
RANSAC paradigm– 4 free parameters calculated
so that camera pair satisfies calibration conditions
• Principal point at image center• Aspect ration of 1• Zero skew• Unknown varying focal length
![Page 9: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/9.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Projective reconstruction
• sparse 3D reconstruction obtained from camera pair– Intersection of sightlines passing
through known feature pairs
• Quasi calibration– Angles are near to “calibrated”
case• Estimation of precision of the 3D
point (size of area where 3D point can be put)
![Page 10: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/10.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Structure from motion (SfM)
• Merging camera pairs– Merge using common 2D-
3D feature points– Search for projective
transformation H • Find H or H-1 depending
on quality of 3D points• All measurements are
performed in 2D• RANSAC paradigm
– Linear solution• New more precise SfM
– Recalculating structure– Recalculating camera
motionNew pair
Scene
4x4 homography H
H-1
3px noise added to images
![Page 11: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/11.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Self-Calibration
**
projection
constraints
• Pinhole camera :– Expressed by 3x4 matrix– Can be decomposed into calibration
3x3 (K), rotation 3x3 (R) and translation (T) 3x1 matrix
• Self-calibration– Means searching for calibration
parameters – only from input images– Math : Dual Absolute Quadrics(Ω*)
• KKT = P(H Ω* HT)PT
KKT = (PH) Ω* (HTPT)
– Linear solution with one cubical constrain
![Page 12: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/12.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Experiment
• Detection and correction of lens radial distortion– Polynomial model :
X1 = (1 + a(x02+y0
2))* x0
Y1 = (1 + a(x02+y0
2))* y0
– Search for ‘a’ using F-matrix
• minimizing distance of matching feature points from their corresponding epipolar lines
– Numerical minimization • simulated annealing
![Page 13: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/13.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Conclusion
• Contribution– New feature tracker– Enhanced guided matching – Sequential SfM without need of searching for initial
frame– Lens radial distortion correction
• Results– Video
• Current state– video
![Page 14: © 2005 Martin Bujňák, MartinB@DataExpert.sk, DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr](https://reader035.vdocuments.net/reader035/viewer/2022070415/56649f175503460f94c2d294/html5/thumbnails/14.jpg)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research