triangulation and multi-view geometry class 9 read notes section 3.3, 4.3-4.4, 5.1 (if interested,...
Post on 21-Dec-2015
215 views
TRANSCRIPT
![Page 1: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/1.jpg)
Triangulation and Multi-View Geometry
Class 9
Read notes Section 3.3, 4.3-4.4, 5.1(if interested, read Triggs’s paper on MVG using tensor notation, see
http://www.unc.edu/courses/2004fall/comp/290/089/papers/Triggs-ijcv95.pdf
)
![Page 2: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/2.jpg)
Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do
Step 3.1 select minimal sample (i.e. 7 matches)
Step 3.2 compute solution(s) for F
Step 3.3 determine inliers
until (#inliers,#samples)<95%
samples#7)1(1
matches#inliers#
#inliers 90%
80%
70% 60%
50%
#samples
5 13 35 106 382
Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches
(generate hypothesis)
(verify hypothesis)
Automatic computation of F
![Page 3: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/3.jpg)
Abort verification early
Given n samples and an expected proportion of inliers p, how likely is it that I have observed less than T inliers?abort if P<0.02 (initial sample most probably contained outliers)
(inspired from Chum and Matas BMVC2002)
OOOOOIOOIOOOOOIOOOOOOOIOOOOOIOIOOOOOOOOOIOIIIIOIIIOIOIIIIOOIOIIIIOIOIOIIIIIIII
(use normal approximation to binomial)
To avoid problems this requires to also verify at random!
(but we already have a random sampler anyway)
![Page 4: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/4.jpg)
restrict search range to neighborhood of epipolar line (e.g. 1.5 pixels)
relax disparity restriction (along epipolar line)
Finding more matches
![Page 5: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/5.jpg)
• Degenerate cases• Planar scene• Pure rotation
• No unique solution• Remaining DOF filled by noise• Use simpler model (e.g. homography)
• Solution 1: Model selection (Torr et al., ICCV´98, Kanatani, Akaike)
• Compare H and F according to expected residual error (compensate for model complexity)
• Solution 2: RANSAC• Compare H and F according to inlier count
(see next slide)
Degenerate cases:
![Page 6: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/6.jpg)
RANSAC for (quasi-)degenerate cases
• Full model (8pts, 1D solution)
Sample for out of plane points among outliers
closest rank-6 of Anx9 for all plane inliers
(accept inliers to solution F)
(accept inliers to solution F1,F2&F3)
• Planar model (6pts, 3D solution)
Accept if large number of remaining inliers• Plane+parallax model (plane+2pts)
80% in plane 2% out plane18% outlier
![Page 7: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/7.jpg)
• Absence of sufficient features (no texture)• Repeated structure ambiguity
(Schaffalitzky and Zisserman, BMVC‘98)
• Robust matcher also finds Robust matcher also finds support for wrong hypothesissupport for wrong hypothesis• solution: detect repetition solution: detect repetition
More problems:
![Page 8: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/8.jpg)
RANSAC for ambiguous matching
• Include multiple candidate matches in set of potential matches
• Select according to matching probability (~ matching score)
• Helps for repeated structures or scenes with similar features as it avoids an early commitment, but also useful in general
(Tordoff and Murray ECCV02)
![Page 9: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/9.jpg)
geometric relations between two views is fully
described by recovered 3x3 matrix F
two-view geometry
![Page 10: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/10.jpg)
Triangulation (finally!)
C1x1
L1
x2
L2
X
C2
Triangulation
- calibration
- correspondences
![Page 11: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/11.jpg)
Triangulation• Backprojection
• Triangulation
Iterative least-squares
• Maximum Likelihood Triangulation
C1 x1L1
x2
L2
X
C2
![Page 12: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/12.jpg)
Optimal 3D point in epipolar plane
• Given an epipolar plane, find best 3D point for (m1,m2)
m1
m2
l1 l2l1m1
m2l2
m1´
m2´
Select closest points (m1´,m2´) on epipolar lines
Obtain 3D point through exact triangulationGuarantees minimal reprojection error (given this epipolar plane)
![Page 13: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/13.jpg)
Non-iterative optimal solution
• Reconstruct matches in projective frame by minimizing the reprojection error
• Non-iterative methodDetermine the epipolar plane for reconstruction
Reconstruct optimal point from selected epipolar plane Note: only works for two views
2222
11 ,, MPmMPm DD
(Hartley and Sturm, CVIU´97)
2222
11 ,, lmlm DD (polynomial of degree 6)
m1
m2
l1 l2
3DOF
1DOF
![Page 14: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/14.jpg)
Backprojection
• Represent point as intersection of row and column
Useful presentation for deriving and understanding multiple view geometry(notice 3D planes are linear in 2D point coordinates)
• Condition for solution?
![Page 15: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/15.jpg)
Multi-view geometry
(intersection constraint)
(multi-linearity of determinants)
(= epipolar constraint!)
(counting argument: 11x2-15=7)
![Page 16: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/16.jpg)
Multi-view geometry
(multi-linearity of determinants)
(= trifocal constraint!)
(3x3x3=27 coefficients)
(counting argument: 11x3-15=18)
![Page 17: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/17.jpg)
Multi-view geometry
(multi-linearity of determinants)
(= quadrifocal constraint!)
(3x3x3x3=81 coefficients)
(counting argument: 11x4-15=29)
![Page 18: Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, 4.3-4.4, 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see](https://reader035.vdocuments.net/reader035/viewer/2022081603/56649d5a5503460f94a3a8d0/html5/thumbnails/18.jpg)
Next class: rectification and stereo
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y),y)