structure from motion multi-view geometry affine structure from motion projective structure from...
DESCRIPTION
Epipolar Constraint: Calibrated Case Essential Matrix (Longuet-Higgins, 1981)TRANSCRIPT
![Page 1: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/1.jpg)
Structure from motion
• Multi-view geometry• Affine structure from motion• Projective structure from motion
Planches :– http://www.di.ens.fr/~ponce/geomvis/lect4.ppt – http://www.di.ens.fr/~ponce/geomvis/lect4.pdf
![Page 2: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/2.jpg)
Epipolar Constraint
• Potential matches for p have to lie on the corresponding epipolar line l’.
• Potential matches for p’ have to lie on the corresponding epipolar line l.
![Page 3: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/3.jpg)
Epipolar Constraint: Calibrated Case
Essential Matrix(Longuet-Higgins, 1981)
![Page 4: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/4.jpg)
Properties of the Essential Matrix
• E p’ is the epipolar line associated with p’.
• E p is the epipolar line associated with p.
• E e’=0 and E e=0.
• E is singular.
• E has two equal non-zero singular values (Huang and Faugeras, 1989).
T
T
![Page 5: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/5.jpg)
Epipolar Constraint: Small MotionsTo First-Order:
Pure translation:Focus of Expansion
![Page 6: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/6.jpg)
Epipolar Constraint: Uncalibrated Case
Fundamental Matrix(Faugeras and Luong, 1992)
![Page 7: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/7.jpg)
Properties of the Fundamental Matrix
• F p’ is the epipolar line associated with p’.
• F p is the epipolar line associated with p.
• F e’=0 and F e=0.
• F is singular.
T
T
![Page 8: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/8.jpg)
The Eight-Point Algorithm (Longuet-Higgins, 1981)
|F | =1.
Minimize:
under the constraint2
![Page 9: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/9.jpg)
Non-Linear Least-Squares Approach (Luong et al., 1993)
Minimize
with respect to the coefficients of F , using an appropriate rank-2 parameterization.
![Page 10: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/10.jpg)
The Normalized Eight-Point Algorithm (Hartley, 1995)
• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’.
• Use the eight-point algorithm to compute F from thepoints q and q’ .
• Enforce the rank-2 constraint.
• Output T F T’.T
i i i i
i i
![Page 11: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/11.jpg)
Data courtesy of R. Mohr and B. Boufama.
![Page 12: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/12.jpg)
With
out n
orm
aliz
atio
nW
ith n
orm
aliz
atio
nMean errors:10.0pixel9.1pixel
Mean errors:1.0pixel0.9pixel
![Page 13: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/13.jpg)
Trinocular Epipolar Constraints
These constraints are not independent!
![Page 14: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/14.jpg)
Trinocular Epipolar Constraints: Transfer
Given p and p , p can be computed
as the solution of linear equations.
1 2 3
![Page 15: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/15.jpg)
Trifocal Constraints
![Page 16: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/16.jpg)
Trifocal Constraints
All 3x3 minorsmust be zero!
Calibrated Case
Trifocal Tensor
![Page 17: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/17.jpg)
Trifocal ConstraintsUncalibrated Case
Trifocal Tensor
![Page 18: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/18.jpg)
Trifocal Constraints: 3 Points
Pick any two lines l and l through p and p .Do it again.
2 3 2 3T( p , p , p )=01 2 3
![Page 19: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/19.jpg)
Properties of the Trifocal Tensor
Estimating the Trifocal Tensor
• Ignore the non-linear constraints and use linear least-squaresa posteriori.
• Impose the constraints a posteriori.
• For any matching epipolar lines, l G l = 0.
• The matrices G are singular.
• They satisfy 8 independent constraints in theuncalibrated case (Faugeras and Mourrain, 1995).
2 1 3T i
1i
![Page 20: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/20.jpg)
For any matching epipolar lines, l G l = 0. 2 1 3T i
The backprojections of the two lines do not define a line!
![Page 21: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/21.jpg)
Multiple Views (Faugeras and Mourrain, 1995)
![Page 22: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/22.jpg)
Two Views
Epipolar Constraint
![Page 23: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/23.jpg)
Three Views
Trifocal Constraint
![Page 24: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/24.jpg)
Four Views
Quadrifocal Constraint(Triggs, 1995)
![Page 25: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/25.jpg)
Geometrically, the four rays must intersect in P..
![Page 26: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/26.jpg)
Quadrifocal Tensorand Lines
![Page 27: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/27.jpg)
Scale-Restraint Condition from Photogrammetry
![Page 28: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/28.jpg)
The Euclidean (perspective) Structure-from-Motion Problem
Given m calibrated perspective images of n fixed points Pj we can write
Problem: estimate the m 3x4 matrices Mi = [Ri ti] and
the n positions Pj from the mn correspondences pij .
2mn equations in 11m+3n unknowns
Overconstrained problem, that can be solvedusing (non-linear) least squares!
![Page 29: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/29.jpg)
The Euclidean Ambiguity of Euclidean SFM
If Ri, ti, and Pj are solutions,
So are Ri’, ti’, and Pj’, where
In fact, the absolute scale cannot be recovered since:
When the intrinsic and extrinsic parameters are known
Euclidean ambiguity up to a similarity transformation.
![Page 30: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/30.jpg)
The Affine Structure-from-Motion Problem
Given m images of n fixed points P we can write
Problem: estimate the m 2x4 matrices M andthe n positions P from the mn correspondences p .
ij ij
2mn equations in 8m+3n unknowns
Overconstrained problem, that can be solvedusing (non-linear) least squares!
j
![Page 31: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/31.jpg)
The Affine Ambiguity of Affine SFM
If M and P are solutions, i j
So are M’ and P’ wherei j
and
Q is an affinetransformation.
When the intrinsic and extrinsic parameters are unknown
![Page 32: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/32.jpg)
The Affine Epipolar Constraint
Note: the epipolar lines are parallel.
![Page 33: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/33.jpg)
Affine Epipolar Geometry
![Page 34: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/34.jpg)
The Affine Fundamental Matrix
where
![Page 35: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/35.jpg)
With
out n
orm
aliz
atio
nW
ith n
orm
aliz
atio
nMean errors:10.0pixel9.1pixel
Mean errors:1.0pixel0.9pixel
Perspective case..
![Page 36: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/36.jpg)
Mean errors: 3.24 and 3.15pixel (without normalization160.92 and 158.54pixel).
Affine case..
![Page 37: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/37.jpg)
An Affine Trick..
![Page 38: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/38.jpg)
The Affine Epipolar Constraint
Note: the epipolar lines are parallel.
![Page 39: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/39.jpg)
An Affine Trick.. Algebraic Scene Reconstruction Method
![Page 40: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/40.jpg)
Affine reconstruction. Mean relative error: 3.2%
![Page 41: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/41.jpg)
The Affine Structure of Affine Images
Suppose we observe a static scene with m fixed cameras..
The set m-tuples of allimage points in a sceneis a 3D affine space!
![Page 42: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/42.jpg)
When do m+1 points define a p-dimensional subspace Y of ann-dimensional affine space X equipped with some coordinateframe basis?
Writing that all minors of size (p+2)x(p+2) of D are equal tozero gives the equations of Y.
Rank ( D ) = p+1, where
has rank 4!
![Page 43: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/43.jpg)
From Affine to Vectorial Structure
Idea: pick one of the points (or their center of mass)as the origin.
![Page 44: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/44.jpg)
Singular Value Decomposition
![Page 45: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/45.jpg)
Singular Value Decomposition square roots of
![Page 46: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/46.jpg)
Singular Value Decomposition
![Page 47: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/47.jpg)
Singular Value Decomposition
![Page 48: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/48.jpg)
What if we could factorize D? (Tomasi and Kanade, 1992)
Affine SFM is solved!
Singular Value Decomposition
We can take
![Page 49: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/49.jpg)
Affine reconstruction. Mean relative error: 2.8%
![Page 50: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/50.jpg)
Back to perspective:Euclidean motion from E (Longuet-Higgins, 1981)
• Given F computed from n > 7 point correspondences, and its SVD F= UWVT, compute E=U diag(1,1,0) VT.
• There are two solutions t’ = u3 and t’’ = -t’ to ETt=0.
• Define R’ = UWVT and R” = UWTVT where
(It is easy to check R’ and R” are rotations.)
• Then [tx’]R’ = -E and [tx’]R” = E. Similar reasoning for t”.
• Four solutions. Only two of them place the reconstructedpoints in front of the cameras.
100001010
W
![Page 51: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/51.jpg)
Euclidean reconstruction. Mean relative error: 3.1%
![Page 52: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/52.jpg)
A different view of the fundamental matrix
• Projective ambiguity ! M’Q=[Id 0] MQ=[A b].
• Hence: zp = [A b] P and z’p’ = [Id 0] P, with P=(x,y,z,1)T.
• This can be rewritten as: zp = ( A [Id 0] + [0 b] ) P = z’Ap’ + b.
• Or: z (b x p) = z’ (b x Ap’).
• Finally: pTFp’ = 0 with F = [bx] A.
![Page 53: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/53.jpg)
Projective motion from the fundamental matrix
• Given F computed from n > 7 point correspondences, compute b as the solution of FTb=0 with |b|2=1.
• Note that: [ax]2 = aaT - |a|2Id for any a.
• Thus, if A0 = - [bx] F,
[bx] A0 = - [bx]2 F = - bbTF + |b|2 F = F.
• The general solution is M = [A b] with
A = A0 + ( b | b | b).
![Page 54: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/54.jpg)
Two-view projective reconstruction. Mean relative error: 3.0%
![Page 55: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/55.jpg)
Bundle adjustment
Use nonlinear least-squares to minimize:
![Page 56: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/56.jpg)
Bundle adjustment. Mean relative error: 0.2%
![Page 57: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/57.jpg)
Projective SFM from multiple images
z11p11 … z1np1n
… … …zm1pm1 … zmnpmn
M1
…Mm
P_1 … P_n= , D = MP
• If the zij’s are known, can be done via SVD. In principlethe zij’s can be found pairwise from F (Triggs 96).
• Alternative, eliminate zij from the minimization of E=|D-MP|2
• This reduces the problem to the minimization ofE = ij |pij x MiPj|2
under the constraints |Mi|2=|Pj|2=1 with |pij|2=1.
• Bilinear problem.
![Page 58: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/58.jpg)
Bilinear projective reconstruction. Mean relative error: 0.2%
![Page 59: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/59.jpg)
From uncalibrated to calibrated cameras
Weak-perspective camera:
Calibrated camera:
Problem: what is Q ?
Note: Absolute scale cannot be recovered. The Euclidean shape(defined up to an arbitrary similitude) is recovered.
![Page 60: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/60.jpg)
Reconstruction Results (Tomasi and Kanade, 1992)
Reprinted from “Factoring Image Sequences into Shape and Motion,” by C. Tomasi andT. Kanade, Proc. IEEE Workshop on Visual Motion (1991). 1991 IEEE.
![Page 61: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/61.jpg)
What is some parameters are known?
Weak-perspective camera:
Zero skew:
Problem: what is Q ?
0
Self calibration!
![Page 62: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/62.jpg)
П1
Chasles’ absolute conic: x12+x2
2+x32 = 0, x4 = 0.
Kruppa (1913); Maybank & Faugeras (1992)
Triggs (1997);Pollefeys et al. (1998,2002)
, u0, v0
The absolute quadric u0 = v0 = 0The absolute quadratic complex 2 = 2, = 0
u0
v0
kl
f
x’ ≈ P ( H H-1 ) xH = [ X y ]
![Page 63: Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : ponce/geomvis/lect4.ppt ponce/geomvis/lect4.ppt](https://reader036.vdocuments.net/reader036/viewer/2022070610/5a4d1b9e7f8b9ab0599c68d3/html5/thumbnails/63.jpg)
Relation between K, , and *