two-view geometry - tohoku university official english websitetwo-view geometry • also known as...
TRANSCRIPT
Two-view geometry• epipolar points/lines• fundamental matrix• essential matrix• estimation of relative camera pose
Two-view geometry• Also known as epipolar geometry
𝐱𝐱′
𝐞𝐞′
𝐗
𝐥′𝐥
e, e0
l, l0
: epipoles(the images of camera centers)
: epipolar lines(intersections of the plane XCCʼwith the image planes)
C
C 0
Two-view geometry• Specifying , you have
C
C 0
x l0
𝐱 𝐱′𝐞
𝐞′
𝐗
𝐥′
Two-view geometry
A case when the epipole is at infinity
[Hartley-Zisserman03]
x
l0
• Specifying , you havex
l0
Fundamental matrix• The epipolar line specified by is given as
• is a constant 3x3 matrix defined for a pair of views, which is called the fundamental matrix
x
l0
l
0 = Fx
F
𝐱𝐱′
𝐞′
𝐥′
𝛑
CC 0
Suppose a plane that does not pass or
⇡C C 0
Then there is a 2D projective transformation
x
0 / Hx
is given by l0l
0 = [e0]⇥x0
= [e0]⇥Hx
= ([e0]⇥H)x
F = ([e0]⇥H)
A matrix representing vector cross product• For a 3-vector , is defined to be a 3x3 matrix satisfying
• It is given by
• is a skew-symmetric matrix, i.e., • has rank , because
v [v]⇥
v ⇥ x = [v]⇥x
2 [v]⇥v = 0
[v]⇥ =
2
40 �v3 v2v3 0 �v1�v2 v1 0
3
5
[v]⇥[v]⇥
[v]>⇥ = �[v]⇥
Properties of fundamental matrix• gives the relation for the reverse order of views:
• , because• and , because any epipolar line passes ;
thus , and ; this should hold for any• has rank 2, because
• Any matrix of rank 2 can be a fundamental matrix; proof omitted
• The DoF of is seven; 3 x 3 – 1(scaling) – 1(rank=2) = 7 • represents the geometric relation of a pair of uncalibrated
cameras in a complete and concise manner
F>
l
0 = Fx l = F>x0
x
0>Fx = 0x
0>l
0 = 0
Fe = 0 F>e0 = 0 l el>e = 0 x
0>Fe = 0 x
0
F F = [e0]⇥H
F
F
Deriving camera matrices from • Proposition: Given a fundamental matrix of two views, the
camera matrices of the two views are given as
• where is any skew-symmetric matrix and is the epipole:
• Remark: the above gives a projective reconstruction
• Lemma of the proposition: If is a fundamental matrix of two views having camera matrices and , then is skew-symmetric, and vice versa
F
F
P =⇥I 0
⇤P0 =
⇥SF e0
⇤
S e0
e0>F = 0>
FP P0 P0>FP
A> = �ASkew-symmetric:
Deriving camera matrices from • Proof of Lemma: If a matrix is skew-symmetric, then it holds
that for any , and vice versa• Therefore, if is skew-symmetric, for
any , and vice versa • We may set and , resulting in
• Proof of proposition: We only need to show is skew-symmetric; this is done as follows
F
P0>FP
x
>Ax = 0
A
x
X
x
0 / P0Xx / PX x
0>Fx = 0
P0>FP =⇥SF e0
⇤>F⇥I 0
⇤
=
�F>SF 0e0>F 0
�=
�F>SF 00> 0
�
P0>FP
X>P0>FPX = 0
Essential matrix• is called the essential matrix
• gives a two-view relation similar to when the camera(s) are calibrated• Substituting and
into , we have
• Properties:• Denote the coordinate trans. between the two camera coord.
by ; then • DoF of is five (rotation + translation – scaling)• A 3x3 matrix is an essential matrix if and only if the two of its
singular values are identical and the rest is zero; proof omitted
x
0>Fx = 0
E F
E
E
E = [t]⇥R
x / KX̃ = K⇥X Y Z
⇤>
x
0>Fx = X̃
0>K0>FKX̃ = X̃
0>(K0>FK)X̃ = 0
x
0 / K0X̃0 = K0⇥X 0 Y 0 Z 0⇤>
E ⌘ K0>FK
X̃0 = RX̃+ t
Essential matrix• Proof of
K�1x
(t)
(RK�1x)
CC 0
(K0�1x
0)
X(X0)
x
x
0
E = [t]⇥R
The three colored vectors lie on a plane:
Coordinate trans. from 1st
to 2nd camera:
(K0�1x
0)>(t⇥ RK�1x) = 0
x
0>K0�>t⇥ RK�1
x = 0
E = [t]⇥R
x
0>K0�>[t]⇥RK�1
x = 0
X̃0 = RX̃+ t
Essential matrix• If is an essential matrix, then its singular values is
• is of rank 2 just like
• Given , we can obtain and as shown below
• The SVD of is given as
• Suppose the camera matrix of 1st view to be
• Then, 2nd camera matrix should be one of the four matrices:
[s, s, 0]E
E F
E t R
EE = U
2
41
10
3
5 V>
P = K⇥I 0
⇤
P0 = K0⇥UWV> u3
⇤
P0 = K0⇥UWV> �u3
⇤
P0 = K0⇥UW>V> u3
⇤
P0 = K⇥UW>V> �u3
⇤ W ⌘
2
4�1
11
3
5 u3 = U
2
4001
3
5
where
Obtaining R and t from E• Four solutions and relative camera poses
• A single solution is physically possible: 3D points will be in front of both the cameras for only one case (“chirality check”)
1 2
12 1 2
12
chirality
Application of epipolar (two-view) geometry• Finding point correspondences between two images
• Starting from (seven point matches) or (five)• Once epipolar geometry is obtained, you need only to search along
the epipolar line for the corresponding point • Robust correspondence estimation: RANSAC etc.
• Reconstructing 3D structure or camera poses• Projective reconstruction from
• Similarity reconstruction from
EF
x
l0
P =⇥I 0
⇤P0 =
⇥SF e0
⇤F
E
Summary: matrix decompositions• SVD; singular value decomposition
• Any matrix can be decomposed as• and are orthogonal:• is diagonal whose diagonal entries are called singular values• Unique if singular values are sorted in descending order
• QR decomposition• Any matrix can be decomposed as• is orthogonal ( ) and is an upper-right triangular matrix• Decomposition is unique
• Cholesky decomposition• Any positive definite symmetric matrix can be
decomposed as• L is a lower-left triangular matrix
=
=
=
m⇥ n
m⇥m
A = UWV>
m⇥m
V>V = IU>U = IU V
W
A = QR
A
A
Q Q>Q = I R
AA = LL>