descriptive geometry meets computer vision the geometry …table of contents 1. remarks on linear...
TRANSCRIPT
![Page 1: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/1.jpg)
Descriptive Geometry Meets Computer Vision
The Geometry of Two Images (# 82)
Hellmuth Stachel
[email protected] — http://www.geometrie.tuwien.ac.at/stachel
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
![Page 2: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/2.jpg)
Table of contents
1. Remarks on linear images
2. Geometry of two images
3. Numerical reconstruction of two images
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 1
![Page 3: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/3.jpg)
1. Remarks on linear images
linear image nonlinear (curved) image
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 2
![Page 4: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/4.jpg)
Central projection
The central projection (according to A. Durer)
can be generalized by a central axonometry.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 3
![Page 5: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/5.jpg)
Central axonometric principle
in space E3:
PSfrag replacements
O
E1
E2
E3
U1
U2
U3
cartesian basis O; E1, E2, E3
and points at infinity U1, U2, U3
PSfrag replacements
U c1
U c2
U c3
Ec1
Ec2
Ec3
Oc
in the image plane E2:
central axonometric reference systemOc; Ec
1, Ec2, E
c3; U
c1 , U c
2 , U c3
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 4
![Page 6: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/6.jpg)
Definition of linear images
There is a unique collinear transformation
κ : E3 → E
2 mit O 7→ Oc, Ei 7→ Eci , Ui 7→ U c
i , i = 1, 2, 3.
Any two-dimensional image of E3 under a collinear transformation is called linear.
=⇒
{collinear points have collinear or coincident imagescross-ratios of any four collinear points are preserved.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 5
![Page 7: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/7.jpg)
Definition of linear images
There is a unique collinear transformation
κ : E3 → E
2 mit O 7→ Oc, Ei 7→ Eci , Ui 7→ U c
i , i = 1, 2, 3.
Any two-dimensional image of E3 under a collinear transformation is called linear.
=⇒
{collinear points have collinear or coincident imagescross-ratios of any four collinear points are preserved.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 5
![Page 8: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/8.jpg)
Central projection in coordinates
Notation:
Z . . . center
H . . . principal point
d . . . focal length
x1, x2, x3 . . .camera frame
x′1, x
′2 . . . imagecoordinate frame
PSfrag replacementsimage plane
vanishing planeΠΠ
v
x1
x2
x3
X
Z H
d
Xc
x′1
x′2
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 6
![Page 9: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/9.jpg)
Central projection in coordinates
(x′
1
x′2
)
=d
x3
(x1
x2
)
, or homogeneous
ξ′
0
ξ′
1
ξ′
2
=
0 0 0 10 d 0 00 0 d 0
ξ0...
ξ3
.
Transformation from the camera frame (x1, x2, x3) into arbitrary world coordinates(x1, x2, x3) and translation from the particular image frame (x′
1, x′2) into arbitrary
(x′1, x
′2) gives in homogeneous form
ξ′0ξ′1ξ′2
=
1 0 0h′
1 d f1 0h′
2 0 d f2
0 0 0 10 1 0 00 0 1 0
1 0 0 0o1...
o3
R
︸ ︷︷ ︸
matrix A
ξ0...ξ3
.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 7
![Page 10: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/10.jpg)
Central projection in coordinates
(x′
1
x′2
)
=d
x3
(x1
x2
)
, or homogeneous
ξ′
0
ξ′
1
ξ′
2
=
0 0 0 10 d 0 00 0 d 0
ξ0...
ξ3
.
Transformation from the camera frame (x1, x2, x3) into arbitrary world coordinates(x1, x2, x3) and translation from the particular image frame (x′
1, x′2) into arbitrary
(x′1, x
′2) gives in homogeneous form
ξ′0ξ′1ξ′2
=
1 0 0h′
1 d f1 0h′
2 0 d f2
0 0 0 10 1 0 00 0 1 0
1 0 0 0o1...
o3
R
︸ ︷︷ ︸
matrix A
ξ0...ξ3
.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 7
![Page 11: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/11.jpg)
Central projection in coordinates
Left hand matrix: (h′1, h
′2) are image coordinates of the principal point H,
(f1, f2) are possible scaling factors, and d is the focal length.
These parameters are called the intrinsic calibration parameters.
Right hand matrix: R is an orthogonal matrix.
The position of the camera frame with respect to the world coordinates definesthe extrinsic calibration parameters.
Photos with known interior orientation are called calibrated images, others (likecentral axonometries) are uncalibrated.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 8
![Page 12: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/12.jpg)
Central projection in coordinates
Left hand matrix: (h′1, h
′2) are image coordinates of the principal point H,
(f1, f2) are possible scaling factors, and d is the focal length.
These parameters are called the intrinsic calibration parameters.
Right hand matrix: R is an orthogonal matrix.
The position of the camera frame with respect to the world coordinates definesthe extrinsic calibration parameters.
Photos with known interior orientation are called calibrated images, others (likecentral axonometries) are uncalibrated.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 8
![Page 13: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/13.jpg)
Unknown interior calibration parameters
ZZZZZZZZZZZZZZZZZ
PSfrag replacements
collinear
bundle tran
sformation
ZZZZZZZZZZZZZZZZZ
the bundles Z and Zof the rays of sight arecollinear
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 9
![Page 14: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/14.jpg)
2. Geometry of two images
Given: Two linear images or two photographs.
Wanted: Dimensions of the depicted 3D-object.
Historical ‘Stadtbahn’ station Karlsplatz in Vienna (Otto Wagner, 1897)
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 10
![Page 15: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/15.jpg)
2. Geometry of two images
The geometry of two images is a classical subject of Descriptive Geometry.Its results have become standard (Finsterwalder, Kruppa, Krames,Wunderlich, Hohenberg, Tschupik, Brauner, Havlicek, H.S., . . . ).
Why now ? Advantages of digital images:
• less distorsion, because no paper prints are needed,
• exact boundary is available, and
• precise coordinate measurements are possible using standard software.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 11
![Page 16: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/16.jpg)
2. Geometry of two images
The geometry of two images is a classical subject of Descriptive Geometry.Its results have become standard (Finsterwalder, Kruppa, Krames,Wunderlich, Hohenberg, Tschupik, Brauner, Havlicek, H.S., . . . ).
Why now ? Advantages of digital images:
• less distorsion, because no paper prints are needed,
• exact boundary is available, and
• precise coordinate measurements are possible using standard software.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 11
![Page 17: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/17.jpg)
Computer Vision
Why now ?
The geometry of two images is important for Computer Vision, a topic with themain goal to endow a computer with a sense of vision.
Basic problems:
• Which information can be extracted from digital images ?
• How to preprocess and represent this information ?
Sensor-guided robots, automatic vehicle control, ‘Big Brother’, . . .
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 12
![Page 18: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/18.jpg)
Computer Vision
Why now ?
The geometry of two images is important for Computer Vision, a topic with themain goal to endow a computer with a sense of vision.
Basic problems:
• Which information can be extracted from digital images ?
• How to preprocess and represent this information ?
Sensor-guided robots, automatic vehicle control, ‘Big Brother’, . . .
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 12
![Page 19: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/19.jpg)
Computer Vision
Recent textbooks:
Yi Ma, St. Soatto, J. Kosecka, S.S.Sastry: An Invitation to 3-D Vision.Springer-Verlag, New York 2004
R. Hartley, A. Zisserman:Multiple View Geometry in ComputerVision. Cambridge University Press 2000
Fortunately the authors in the cited bookrefer to some of these standard results(Krames, Kruppa, Wunderlich)
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 13
![Page 20: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/20.jpg)
Geometry of two images (epipolar geometry)
viewing situation
collinear transformations
two images
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12
zzzzzzzzzzzzzzzzz
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXX
δXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2
l2l2
l2l2l2l2
l2l2
l2l2
l2l2l2l2
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1π′1 π′′
2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2π′′2
γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1
γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2
X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′
X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′
l′l′l′
l′l′
l′l′l′l′
l′l′
l′l′
l′l′l′l′
l′′l′′l′′
l′′l′′l′′l′′l′′l′′
l′′l′′l′′l′′
l′′l′′l′′l′′Z ′
2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2Z ′2
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1Z ′′1
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 14
![Page 21: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/21.jpg)
Geometry of two images (epipolar geometry)
Notations:
line z = Z1Z2 . . . baseline,
Z ′2, Z
′′1 . . . epipoles
(German: Kernpunkte),
δX . . . epipolar plane (it is twiceprojecting),
l′, l′′ . . . pair of epipolar lines(German: Kernstrahlen),
(X ′, X ′′) . . . corresponding views.
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12
zzzzzzzzzzzzzzzzz
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXX
δXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2
l2l2
l2l2l2l2
l2l2
l2l2
l2l2l2l2
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1π′1 π′′
2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2π′′2
γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1γ1
γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2γ2
X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′X ′
X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′X ′′
l′l′l′
l′l′
l′l′l′l′
l′l′
l′l′
l′l′l′l′
l′′l′′l′′
l′′l′′l′′l′′l′′l′′
l′′l′′l′′l′′
l′′l′′l′′l′′Z ′
2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2
Z ′2Z ′2Z ′2
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1
Z ′′1Z ′′1Z ′′1
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 15
![Page 22: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/22.jpg)
Epipolar constraint
Theorem (synthetic version): For any two linear images of a scene, there is aprojectivity between two line pencils
Z ′2(δ
′X) ∧− Z ′′
1 (δ′′X)
such that the points X ′, X ′′ are corresponding ⇐⇒ they are located on(corresponding =) epipolar lines.
Theorem (analytic version): Using homogeneous coordinates for both images,there is a bilinear form β of rank 2 such that two points X ′ = x
′R = (ξ′0 : ξ′1 : ξ′2)
and X ′′ = x′′R = (ξ′′0 : ξ′′1 : ξ′′2 ) are corresponding
⇐⇒ β(x′,x′′) =
2∑
i,j=0
bij ξ′i ξ′′j = (ξ′0 ξ′1 ξ′2)·
(bij
)
0
@
ξ′′0
ξ′′1
ξ′′2
1
A = x′T · B · x′′ = 0 .
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 16
![Page 23: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/23.jpg)
Epipolar constraint
Theorem (synthetic version): For any two linear images of a scene, there is aprojectivity between two line pencils
Z ′2(δ
′X) ∧− Z ′′
1 (δ′′X)
such that the points X ′, X ′′ are corresponding ⇐⇒ they are located on(corresponding =) epipolar lines.
Theorem (analytic version): Using homogeneous coordinates for both images,there is a bilinear form β of rank 2 such that two points X ′ = x
′R = (ξ′0 : ξ′1 : ξ′2)
and X ′′ = x′′R = (ξ′′0 : ξ′′1 : ξ′′2 ) are corresponding
⇐⇒ β(x′,x′′) =
2∑
i,j=0
bij ξ′i ξ′′j = (ξ′0 ξ′1 ξ′2)·
(bij
)
0
@
ξ′′0
ξ′′1
ξ′′2
1
A = x′T · B · x′′ = 0 .
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 16
![Page 24: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/24.jpg)
Epipolar constraint
Proof (analytic version): Using homogeneous line coordinates, the projectivitybetween the line pencils can be expressed as
β : (u′1λ1 + u
′2λ2)R 7→ (u′′
1λ1 + u′′2λ2)R for all (λ1, λ2) ∈ R
2 \ {(0, 0)}.
x′ and x
′′ are corresponding ⇐⇒ there is a nontrivial pair (λ1, λ2) such that
(u′1λ1 + u
′2λ2)· x
′ = 0
(u′′1λ1 + u
′′2λ2)· x
′′ = 0 .
These two linear homogeneous equations in the unknowns (λ1, λ2) have anontrivial solution ⇐⇒ the determinant vanishes, i.e.,
β(x′,x′′) := (u′1·x
′)(u′′2 ·x
′′) − (u′2·x
′)(u′′1 ·x
′′) =∑2
i,j=0 bij ξ′i ξ′′j = 0.
There are singular points of this correspondance: Z ′2 corresponds to all X ′′, and
vice versa all points X ′ correspond to Z ′′1 =⇒ rk(bij) = 2 .
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 17
![Page 25: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/25.jpg)
Epipolar constraint in the calibrated case
Theorem: In the calibrated casethe essential matrix B = (bij) is theproduct of a skew symmetric matrixand an orthogonal one, i.e.,
B = S ·R .
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1z′
z′
z′
z′z′
z′
z′
z′z′
z′z′
z′
z′
z′
z′
z′
z′
Z21Z21Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21Z21Z21
Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2x
′x′
x′
x′x′
x′
x′
x′x′
x′x′
x′
x′
x′
x′
x′
x′
x′′
x′′
x′′
x′′x′′
x′′
x′′
x′′x′′
x′′x′′
x′′
x′′
x′′
x′′
x′′
x′′
Proof: We use both camera frames and the homogeneous coordinates
x′ =
−−−→Z1X
′, x′′ =
−−−→Z2X
′′.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 18
![Page 26: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/26.jpg)
Epipolar constraint in the calibrated case
For transforming the coordinates from the second camera frame into the first one,there is an orthogonal matrix R such that
x′′1 = z
′ + R·x′′ with RT = R−1 and z′ = (z′1, z′2, z′3)
T =−−−→Z1Z2.
The points X1, X2, Z1,Z2 are coplanar ⇐⇒ the tripleproduct of the vectors x
′, z′ and
x′′1 = Z1X2 vanishes, i.e.,
det(x′, z′,x′′1) = x
′ · (z′×x′′1) = 0.
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1z′
z′
z′
z′z′
z′
z′
z′z′
z′z′
z′
z′
z′
z′
z′
z′
Z21Z21Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21Z21Z21
Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2x
′x′
x′
x′x′
x′
x′
x′x′
x′x′
x′
x′
x′
x′
x′
x′
x′′
x′′
x′′
x′′x′′
x′′
x′′
x′′x′′
x′′x′′
x′′
x′′
x′′
x′′
x′′
x′′
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 19
![Page 27: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/27.jpg)
Epipolar constraint in the calibrated case
For transforming the coordinates from the second camera frame into the first one,there is an orthogonal matrix R such that
x′′1 = z
′ + R·x′′ with RT = R−1 and z′ = (z′1, z′2, z′3)
T =−−−→Z1Z2.
The points X1, X2, Z1, Z2
are coplanar ⇐⇒ the tripleproduct of the vectors x
′, z′ and
x′′1 = Z1X2 vanishes, i.e.,
det(x′, z′,x′′1) = x
′ · (z′×x′′1) = 0.
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1z′
z′
z′
z′z′
z′
z′
z′z′
z′z′
z′
z′
z′
z′
z′
z′
Z21Z21Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21Z21Z21
Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2x
′x′
x′
x′x′
x′
x′
x′x′
x′x′
x′
x′
x′
x′
x′
x′
x′′
x′′
x′′
x′′x′′
x′′
x′′
x′′x′′
x′′x′′
x′′
x′′
x′′
x′′
x′′
x′′
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 19
![Page 28: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/28.jpg)
Epipolar constraint in the calibrated case
We replace the vector product (z′×x′′1) by
z′×(z′ + R·x′′) = z
′×R·x′′ = S ·R·x′′ mit S =
0
@
0 −z′3 z′
2
z′3 0 −z′
1
−z′2 z′
1 0
1
A.
Matrix S is skew symmetric and R is orthogonal.
Hence, the coplanarity of x′, x
′′ and z′ is equivalent to
0 = x′ · (z′×x
′′1) = x
′T · S ·R︸︷︷︸B
·x′′, also B = S ·R .
The decomposition of the fundamental matrix B into these two factors definesthe relative position of the second camera frame against the first one !
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 20
![Page 29: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/29.jpg)
Epipolar constraint in the calibrated case
We replace the vector product (z′×x′′1) by
z′×(z′ + R·x′′) = z
′×R·x′′ = S ·R·x′′ mit S =
0
@
0 −z′3 z′
2
z′3 0 −z′
1
−z′2 z′
1 0
1
A.
Matrix S is skew symmetric and R is orthogonal.
Hence, the coplanarity of x′, x
′′ and z′ is equivalent to
0 = x′ · (z′×x
′′1) = x
′T · S ·R︸︷︷︸B
·x′′, also B = S ·R .
The decomposition of the fundamental matrix B into these two factors definesthe relative position of the second camera frame against the first one !
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 20
![Page 30: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/30.jpg)
Essential matrix
Theorem:The essential matrix B has two equalsingular values σ := σ1 = σ2.
Proof: We have B = S ·R withorthogonal R. The vector
S ·x = z′×x
is orthogonal zu the orthogonal viewx
n, where
‖z′×x‖ = | sin ϕ| ‖x‖ ‖z′‖ =
= ‖xn‖ ‖z′‖ = σ ‖xn‖.
PSfrag replacementsz′
x
xn
z′×x
ϕ
Π ⊥ z′
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 21
![Page 31: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/31.jpg)
Singular value decomposition
Theorem: [Singular value decomposition]
Any matrix A ∈ M(m, n; R) can be decomposed into a product
A = U ·D ·V T with orthogonal U, V and D = diag(σ1, . . . , σp)
with D ∈ M(m, n; R), σi ≥ 0, and p = min{m,n}.
The positive entries in the main diagonal of D are called singular values of A.
The singular values of A can be seen as principal distortion factors of the affinetransformation represented by A, i.e., the semiaxes of the affine image of the unitsphere.
Hence the singular values of an orthogonal projection are (1, 1).
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 22
![Page 32: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/32.jpg)
Singular value decomposition
Theorem: [Singular value decomposition]
Any matrix A ∈ M(m, n; R) can be decomposed into a product
A = U ·D ·V T with orthogonal U, V and D = diag(σ1, . . . , σp)
with D ∈ M(m, n; R), σi ≥ 0, and p = min{m,n}.
The positive entries in the main diagonal of D are called singular values of A.
The singular values of A can be seen as principal distortion factors of the affinetransformation represented by A, i.e., the semiaxes of the affine image of the unitsphere.
Hence the singular values of an orthogonal projection are (1, 1).
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil 22
![Page 33: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/33.jpg)
Singular value decomposition
LinAlg
LinAlg
PSfrag replacements
a0a1
a2 xA
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
23
![Page 34: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/34.jpg)
Singular value decomposition
LinAlg
LinAlg
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0a1
a2 xA
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
rotation ↓ V T rotation ↑ U
LinAlgLinAlg
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scaling23
![Page 35: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/35.jpg)
What means ‘reconstruction’
Given: Two either calibratedor uncalibrated images.
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling π′1π′1π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1
π′1π′1π′1π′1π′1 π′′
2π′′2π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2
π′′2π′′2π′′2π′′2π′′2
X ′1X ′1X ′1X ′1
X ′1X ′1
X ′1X ′1
X ′1X ′1
X ′1X ′1
X ′1X ′1X ′1X ′1X ′1 X ′′
1X ′′1X ′′1X ′′1
X ′′1X ′′1
X ′′1X ′′1
X ′′1X ′′1
X ′′1X ′′1
X ′′1X ′′1X ′′1X ′′1X ′′1
X ′2X ′2X ′2X ′2
X ′2X ′2
X ′2X ′2
X ′2X ′2
X ′2X ′2
X ′2X ′2X ′2X ′2X ′2
X ′′2X ′′2X ′′2X ′′2
X ′′2X ′′2
X ′′2X ′′2
X ′′2X ′′2
X ′′2X ′′2
X ′′2X ′′2X ′′2X ′′2X ′′2
Wanted: ‘viewing situation’,i.e., determine
• the relative position of thetwo camera frames, and
• the location of any spacepoint X from its images(X ′, X ′′).
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1
Z21Z21Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21Z21Z21
Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12
zzzzzzzzzzzzzzzzz
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scaling24
![Page 36: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/36.jpg)
First fundamental theorem
Theorem:From two uncalibrated images with given projectivity between epipolar lines thedepicted object can be reconstructed up to a collinear transformation.
Sketch of the proof:The two images can be placedin space such that pairs ofepipolar lines are intersecting.Then for arbitrary Z1, Z2 on thebaseline z = Z2
1Z12 there is a
reconstructed 3D object.
Any other choice of theviewing situation gives a collineartransform of the 3D object.
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1
Z21Z21Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21Z21Z21
Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12
zzzzzzzzzzzzzzzzz
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scaling25
![Page 37: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/37.jpg)
Second fundamental theorem
Theorem (S. Finsterwalder, 1899):From two calibrated images with given projectivity between epipolar lines thedepicted object can be reconstructed up to a similarity.
Sketch of the proof:Now in the two bundles of raysthe pencils of epipolar planesδX are congruent, and they canbe made coincident by a rigidmotion. Then relative to the firstbundle Z1 for any Z2 ∈ z thereis a reconstructed 3D object.
Any other choice of Z2 gives asimilar 3D object.
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1π1
π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2π2
Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2Z2 Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1Z1
Z21Z21Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21
Z21Z21Z21Z21Z21
Z12Z12Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12
Z12Z12Z12Z12Z12
zzzzzzzzzzzzzzzzz
X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1X1
X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2X2
XXXXXXXXXXXXXXXXXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδXδX
l1l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2l2
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scaling26
![Page 38: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/38.jpg)
Determination of epipoles — geometric meaning
Problem of Projectivity:
Given: 7 pairs of corresponding points (X ′1, X
′′1 ), . . . , (X ′
7, X′′7 ).
Wanted: A pair of points (S′, S′′) (= epipoles) such that there is a projectivity
S′([S′X ′1], . . . , [S
′X ′7]) ∧− S′′([S′X ′′
1 ], . . . , [S′′X ′′7 ]).
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1 X ′
2
X ′3X ′
4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′ 27
![Page 39: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/39.jpg)
Determination of epipoles — geometric meaning
Problem of Projectivity:
Given: 7 pairs of corresponding points (X ′1, X
′′1 ), . . . , (X ′
7, X′′7 ).
Wanted: A pair of points (S′, S′′) (= epipoles) such that there is a projectivity
S′([S′X ′1], . . . , [S
′X ′7]) ∧− S′′([S′X ′′
1 ], . . . , [S′′X ′′7 ]).
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1 X ′
2
X ′3X ′
4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′π′
π′′
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 27
![Page 40: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/40.jpg)
Determination of epipoles — analytic solution
Theorem: If 7 pairs of corresponding points (X ′1, X
′′1 ), . . . , (X ′
7, X′′7 ) are given,
the determination of the epipoles is a cubic problem.
Proof: 7 pairs of corresponding points give 7 linear homogeneous equations
β(x′i,x
′′i ) = x
Ti · B · x′′
i = 0, i = 1, . . . , 7,
for the 9 entries in the (3×3)-matrix B = (bij) — called essential matrix.
det(bij) = 0 gives an additional cubic equation which fixes all bij up to a commonfactor.
For noisy image points it is recommended to use more than 7 points and methodsof least square approximation for obtaining the ‘best fitting matrix’ B:
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 28
![Page 41: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/41.jpg)
Determination of epipoles — analytic solution
Theorem: If 7 pairs of corresponding points (X ′1, X
′′1 ), . . . , (X ′
7, X′′7 ) are given,
the determination of the epipoles is a cubic problem.
Proof: 7 pairs of corresponding points give 7 linear homogeneous equations
β(x′i,x
′′i ) = x
Ti · B · x′′
i = 0, i = 1, . . . , 7,
for the 9 entries in the (3×3)-matrix B = (bij) — called essential matrix.
det(bij) = 0 gives an additional cubic equation which fixes all bij up to a commonfactor.
For noisy image points it is recommended to use more than 7 points and methodsof least square approximation for obtaining the ‘best fitting matrix’ B:
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 28
![Page 42: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/42.jpg)
Determination of epipoles — analytic solution
1) Let A denote the coefficient matrix in the linear system for the entries of B.Then the ‘least square fit’ for this overdetermined system is an eigenvector forthe smallest eigenvalue of the symmetric matrix AT · A.
2) As an essential matrix needs to have rank 2, we use the ’projection into theessential space’. This means, the singular value decomposition of B gives arepresentation
B = U · diag(σ1, σ2, σ3) · VT with orthogonal U, V and σ1 ≥ σ2 ≥ σ3 .
Then in the uncalibrated case B = U ·diag(σ1, σ2, 0) ·V is optimal (with respectto the Frobenius norm) and in the calibrated case
B = U · diag(σ, σ, 0) · V T with σ1 = (σ1 + σ2)/2.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 29
![Page 43: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/43.jpg)
Determination of epipoles — analytic solution
1) Let A denote the coefficient matrix in the linear system for the entries of B.Then the ‘least square fit’ for this overdetermined system is an eigenvector forthe smallest eigenvalue of the symmetric matrix AT · A.
2) As an essential matrix needs to have rank 2, we use the ’projection into theessential space’. This means, the singular value decomposition of B gives arepresentation
B = U · diag(σ1, σ2, σ3) · VT with orthogonal U, V and σ1 ≥ σ2 ≥ σ3 .
Then in the uncalibrated case B = U · diag(σ1, σ2, 0) · V is optimal (withrespect to the Frobenius norm) and in the calibrated case
B = U · diag(σ, σ, 0) · V T with σ1 = (σ1 + σ2)/2.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 29
![Page 44: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/44.jpg)
3. Numerical reconstruction of two images
Step 1: Specify at least 7 reference points
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
11111111111111111
22222222222222222
33333333333333333 44444444444444444
55555555555555555
6666666666666666677777777777777777
88888888888888888
99999999999999999
1010101010101010101010101010101010
1111111111111111111111111111111111
1212121212121212121212121212121212
13131313131313131313131313131313131414141414141414141414141414141414
1515151515151515151515151515151515
1616161616161616161616161616161616
1717171717171717171717171717171717
1818181818181818181818181818181818
1919191919191919191919191919191919
2020202020202020202020202020202020
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
11111111111111111
22222222222222222
33333333333333333 44444444444444444
55555555555555555
66666666666666666
7777777777777777788888888888888888
99999999999999999
1010101010101010101010101010101010
1111111111111111111111111111111111
1212121212121212121212121212121212
13131313131313131313131313131313131414141414141414141414141414141414
1515151515151515151515151515151515
1616161616161616161616161616161616
1717171717171717171717171717171717
1818181818181818181818181818181818
1919191919191919191919191919191919
2020202020202020202020202020202020
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 30
![Page 45: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/45.jpg)
Step 2: Compute the essential matrix
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
Step 2: Compute the essential matrix B — including the pairs of epipolar lines
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 31
![Page 46: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/46.jpg)
Step 3: Factorize B = S.R
Theorem: There are exactly two ways of decomposing B = U ·D ·V T withD = diag(σ, σ, 0) into a product S ·R with skew-symmetric S and orthogonal R :
S = ±U ·R+·D ·UT and R = ±U ·RT+·V T with R+ =
0
@
0 −1 0
1 0 0
0 0 1
1
A.
Proof:
a) It is sufficient to factorize U ·D = S ·R′ which implies B = S · (R′·V T ), i.e.,R = R′·V T .
b) D represents the product of the orthogonal projection into the x1x2-plane andthe scaling with factor σ . The rotation U transforms the x1x2-plane into theimage plane of U · D.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 32
![Page 47: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/47.jpg)
Step 3: Factorize B = S.R
Theorem: There are exactly two ways of decomposing B = U ·D ·V T withD = diag(σ, σ, 0) into a product S ·R with skew-symmetric S and orthogonal R :
S = ±U ·R+·D ·UT and R = ±U ·RT+·V T with R+ =
0
@
0 −1 0
1 0 0
0 0 1
1
A.
Proof:
a) It is sufficient to factorize U ·D = S ·R′ which implies B = S · (R′·V T ), i.e.,R = R′·V T .
b) D represents the product of the orthogonal projection into the x1x2-plane andthe scaling with factor σ . The rotation U transforms the x1x2-plane into theimage plane of U · D.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 32
![Page 48: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/48.jpg)
Step 3: Factorize B = S.R
c) Any skew symmetric matrix S represents the product of an orthogonal projectionparallel to z
′, a 90◦-rotation about z′ and a scaling with factor ‖z′‖.
d) R+ · D is skew-symmetric with z′ = (0, 0, σ). We transform it by U to obtain
the required position, i.e., S = ±U ·(R+·D)·UT .
R+ commutes with D, =⇒ U ·D =[±U ·R+·D ·UT
]
︸ ︷︷ ︸S
·[±U ·RT
+
]
︸ ︷︷ ︸
R′
.
e) B represents an orthogonal axonometry; its column vectors are images ofan orthonormal frame. We know from Descriptive Geometry that apart fromtranslations there are not more than two different frames with given images.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 33
![Page 49: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/49.jpg)
Step 3: Factorize B = S.R
c) Any skew symmetric matrix S represents the product of an orthogonal projectionparallel to z
′, a 90◦-rotation about z′ and a scaling with factor ‖z′‖.
d) R+ · D is skew-symmetric with z′ = (0, 0, σ). We transform it by U to obtain
the required position, i.e., S = ±U ·(R+·D)·UT .
R+ commutes with D, =⇒ U ·D =[±U ·R+·D ·UT
]
︸ ︷︷ ︸S
·[±U ·RT
+
]
︸ ︷︷ ︸
R′
.
e) B represents an orthogonal axonometry; its column vectors are images ofan orthonormal frame. We know from Descriptive Geometry that apart fromtranslations there are not more than two different frames with given images.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 33
![Page 50: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/50.jpg)
Summary of algorithm
1) Specify n > 7 pairs (X ′i, X
′′i ), i = 1, . . . , n.
2) Set up linear system of equations for the essential matrix B and seek bestfitting matrix (eigenvector of the smallest eigenvalue).
3) Compute the closest rank 2 matrix B with two equal singular values.
4) Factorize B = S · R ; this reveals the relative position of the two cameraframes.
5) In one of the frames compute the approximate point of intersection betweencorresponding rays.
6) Transform the recovered coordinates into world coordinates.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 34
![Page 51: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/51.jpg)
Remaining problems
• Analysis of precision,
• automated calibration (autofocus and zooming change the focal distance d),
• critical configurations.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 35
![Page 52: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/52.jpg)
The solution
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
PSfrag replacements
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
A
α(a0)
α(a1)
α(a2)
α(x)
A′
D−→
scaling
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
X ′1
X ′2
X ′3
X ′4
X ′5
X ′6
X ′7
X ′′1
X ′′2
X ′′3
X ′′4
X ′′5
X ′′6
X ′′7
S′
S′′
π′
π′′
11111111111111111
22222222222222222
33333333333333333 44444444444444444
55555555555555555
6666666666666666677777777777777777
88888888888888888
99999999999999999
1010101010101010101010101010101010
1111111111111111111111111111111111
1212121212121212121212121212121212
13131313131313131313131313131313131414141414141414141414141414141414
1515151515151515151515151515151515
1616161616161616161616161616161616
1717171717171717171717171717171717
1818181818181818181818181818181818
1919191919191919191919191919191919
2020202020202020202020202020202020
original image
PSfrag replacements
a0
a1
a2
x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0
a1
a2
x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7
S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′
1
2
3 4
56
78
9
10
11
121314
15
16
17
18
19
20
the reconstruction (M ∼ 1 : 100)
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 36
![Page 53: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/53.jpg)
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′
1
2
3 4
56
78
9
9
10
11
12
12
13
1314
15
16
17
18
18
19
20
Z1
Z1
Z2
Z2
Position of centers
relative to the depicted object
front view
top view
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 37
![Page 54: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/54.jpg)
Literatur
• H. Brauner: Lineare Abbildungen aus euklidischen Raumen. Beitr. AlgebraGeom. 21, 5–26 (1986).
• O. Faugeras: Three-Dimensional Computer Vision. A Geometric Viewpoint.MIT Press, Cambridge, Mass., 1906 .
• O. Faugeras, Q.-T. Luong: The Geometry of Multiple Images. MITPress, Cambridge, Mass., 2001.
• R. Harley, A. Zisserman: Multiple View Geometry in Computer Vision.Cambridge University Press 2000.
• H. Havlicek: On the Matrices of Central Linear Mappings. Math. Bohem.121, 151–156 (1996).
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 38
![Page 55: Descriptive Geometry Meets Computer Vision The Geometry …Table of contents 1. Remarks on linear images 2. Geometry of two images 3. Numerical reconstruction of two images 12th International](https://reader033.vdocuments.net/reader033/viewer/2022042018/5e764b6c5d927665472d4564/html5/thumbnails/55.jpg)
• E. Kruppa: Zur achsonometrischen Methode der darstellenden Geometrie.Sitzungsber., Abt. II, osterr. Akad. Wiss., Math.-Naturw. Kl. 119, 487–506(1910).
• Yi Ma, St. Soatto, J. Kosecka, S. Sh. Sastry: An Invitation to 3-DVision. Springer-Verlag, New York 2004.
• H. Stachel: Zur Kennzeichnung der Zentralprojektionen nach H. Havlicek.Sitzungsber., Abt. II, osterr. Akad. Wiss., Math.-Naturw. Kl. 204, 33–46(1995).
• J. Szabo, H. Stachel, H. Vogel: Ein Satz uber die Zentralaxonometrie.Sitzungsber., Abt. II, osterr. Akad. Wiss., Math.-Naturw. Kl. 203, 3–11 (1994).
• J. Tschupik, F. Hohenberg: Die geometrische Grundlagen derPhotogrammetrie. In Jordan, Eggert, Kneissl (eds.): Handbuch derVermessungskunde III a/3. 10. Aufl., Metzlersche Verlagsbuchhandlung,Stuttart 1972, 2235–2295.
12th International Conference on Geometry and Graphics, August 6–10, 2006, Salvador/Brazil
PSfrag replacements
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
U ·D·V T
A−→
a0a1a2x
Aα(a0)α(a1)α(a2)α(x)
A′
D−→
scalingX′
1X′
2X′
3X′
4X′
5X′
6X′
7X′′
1X′′
2X′′
3X′′
4X′′
5X′′
6X′′
7S′
S′′
π′
π′′
X′1
X′2
X′3
X′4
X′5
X′6
X′7
X′′1
X′′2
X′′3
X′′4
X′′5
X′′6
X′′7
S′
S′′
π′
π′′ 39