![Page 1: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/1.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Geometry for Computer VisionLecture 4a
Calibration and OrientedEpipolar Geometry
1
Per-Erik Forssén
![Page 2: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/2.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Overview1. Lens effects (distortion, vignetting)2. Extrinsic and intrinsic camera parameters3. Zhang’s camera calibration4. Calibrated epipolar geometry (intro)5. Oriented epipolar geometry
Break
2
![Page 3: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/3.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
The pin-hole camera
A brightly illuminated scene will be projected onto a wall opposite of the pin-hole.
The image is rotated .
3
180�
![Page 4: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/4.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
The pin-hole camera
• From similar triangles we get:
4
x = fX
Zy = f
Y
Z
![Page 5: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/5.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
The pin-hole camera
• From similar triangles we get:
5
�
�
⇤xy1
⇥
⌅ =
�
⇤f 0 00 f 00 0 1
⇥
⌅
�
⇤XYZ
⇥
⌅
![Page 6: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/6.jpg)
• More generally, we write:
• f-focal length, s-skew, a-aspect ratio,(cx,cy)-projection of optical centre
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
The pin-hole camera
6
�
�
⇤xy1
⇥
⌅ =
�
⇤f 0 00 f 00 0 1
⇥
⌅
�
⇤XYZ
⇥
⌅
�
�
⇤xy1
⇥
⌅ =
�
⇤f s cx
0 fa cy
0 0 1
⇥
⌅
�
⇤XYZ
⇥
⌅
![Page 7: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/7.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
�
�
⇤xy1
⇥
⌅
⌥⌃⇧�x
=
�
⇤f s cx
0 fa cy
0 0 1
⇥
⌅
⌥ ⌃⇧ �K
�
⇤XYZ
⇥
⌅
⌥ ⌃⇧ �X̃
The pin-hole camera
x � KX̃
7
![Page 8: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/8.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
�
�
⇤xy1
⇥
⌅
⌥⌃⇧�x
=
�
⇤f s cx
0 fa cy
0 0 1
⇥
⌅
⌥ ⌃⇧ �K
�
⇤XYZ
⇥
⌅
⌥ ⌃⇧ �X̃
The pin-hole camera
Motivation:
f-focal length, s-skew, a-aspect ratio,(cx,cy)-projection of optical centre
x � KX̃
8
![Page 9: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/9.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Thin Lens CameraReal cameras use lenses, not pin-holes!
9
![Page 10: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/10.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Thin Lens CameraA thin lens is a (positive) lens with
Parallel rays converge at the focal pointsRays through the optical centre are not
refracted
d << f
10
![Page 11: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/11.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Thin Lens Camera
Thin lens relation (from similar triangles):1f
=1Z
+1l
11
![Page 12: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/12.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Thin Lens Camera
- Focus at one depth only. - Objects at other depths are blurred.
12
![Page 13: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/13.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Thin Lens Camera
An aperture increases the depth-of-field, the range which is sharp in the image.
A compromise between pinhole and thin lens.13
![Page 14: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/14.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Lens effects
• Radial distortion• For zoom lenses: Barrel at wide FoV
pin-cushion at narrow FoV
Barrel distortionCorrect Pin-cushion distortion
14
![Page 15: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/15.jpg)
)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Lens effects
• Modelling• Used in optimisation such as BA
DistortedCorrect image
15
x ⇠ Kf(u,⇥0)
![Page 16: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/16.jpg)
)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Lens effects
• Rectification• Used in dense stereo
Distorted image Correct
16
x
0 ⇠ f�1(x,⇥)
![Page 17: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/17.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Distorsion polynomials• Different models for different classes
of cameras• Radial model for normal and telecentric
lenses with moderate distortion
• Also model centre of distortion17
r =qx
21 + x
22 ' = atan2(x2, x1)
x
01 = r
0cos'
0x
02 = r
0sin'
0r0 = ✓1r + ✓2r3 . . . '0 = '
![Page 18: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/18.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Distorsion polynomials• For better accuracy:• Tangential distorsion• Rational model [Claus & Fitzgibbon 05]
• Specialised models:• wide-angle cameras [Kannala&Brandt 06]• catadioptric cameras [Micusik&Pajdla 03]• most simple is the FoV model [Devernay &
Faugeras 2001]:
18
r0 = atan(r✓1)/✓1
![Page 19: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/19.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Lens effects
• Vignetting and cos4-law• Stronger effects in wide FoV
Darkened peripheryCorrect
19
![Page 20: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/20.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Lens effectsVignetting
20
![Page 21: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/21.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Lens effectsVignetting
cos4-lawdampening withcos4(w)
http://software.canon-europe.com/files/documents/EF_Lens_Work_Book_10_EN.pdf
21
![Page 22: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/22.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
For a general position of the world coordinate system (WCS) we have:
x � K
�
⇤r11 r12 r13 t1r21 r22 r23 t2r31 r32 r33 t3
⇥
⌅
�⌥ ⌦[R|t]
�
⇧⇧⇤
XYZ1
⇥
⌃⌃⌅
�⌥ ⌦X
22
Camera parameters
![Page 23: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/23.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
For a general position of the world coordinate system (WCS) we have:
K contains the intrinsic parameters[R | t] contain the extrinsic parameters
x � K
�
⇤r11 r12 r13 t1r21 r22 r23 t2r31 r32 r33 t3
⇥
⌅
�⌥ ⌦[R|t]
�
⇧⇧⇤
XYZ1
⇥
⌃⌃⌅
�⌥ ⌦X
23
Camera parameters
![Page 24: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/24.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera parametersMetric points transformed to the camera’s
coordinate system are called normalised image coordinates
In contrast to regular image coordinates
K contains the intrinsic parameters[R | t] contain the extrinsic parameters
24
x ⇠ K [R|t]X
x̂ ⇠ [R|t]X
x = Kx̂
![Page 25: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/25.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationZhang’s camera calibration (A flexible new
technique for camera calibration, PAMI 2000)
In OpenCV, and in Matlab toolboxFinds K from 3 or more photos of a planar
calibration targetModerate lens
distorsion can alsobe estimated.
25
![Page 26: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/26.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationWe now imagine a world coordinate
system fixed to the planar target
�
�
⇤xy1
⇥
⌅ = K
�
⇤r11 r12 r13 t1r21 r22 r23 t2r31 r32 r33 t3
⇥
⌅
�
⇧⇧⇤
XYZ1
⇥
⌃⌃⌅
26
![Page 27: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/27.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationWe now imagine a world coordinate
system fixed to the planar target
�
�
⇤xy1
⇥
⌅ = K
�
⇤r11 r12 t1r21 r22 t2r31 r32 t3
⇥
⌅
�
⇤XY1
⇥
⌅ =
�
⇤h11 h12 h13
h21 h22 h23
h31 h32 h33
⇥
⌅
⌥ ⌃⇧ �H
�
⇤XY1
⇥
⌅
27
![Page 28: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/28.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
If we estimate a homography between the image and the model plane (lecture 3) we know H
We also know that and
28
rT1 r2 = 0 rT
1 r1 = rT2 r2
H = [h1 h2 h3] = K [r1 r2 t]
![Page 29: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/29.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
If we estimate a homography between the image and the model plane (lecture 3) we know H
We also know that and
29
rT1 r2 = 0 rT
1 r1 = rT2 r2
H = [h1 h2 h3] = K [r1 r2 t]
hT1 K�T K�1h2 = 0
hT1 K�T K�1h1 = hT
2 K�T K�1h2
)
![Page 30: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/30.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
For a K of the formIt can be shown that (use e.g. Maple)
30
K�T K�1 = B =
2
64
1↵2 � �
↵2�v0��u0�
↵2�
� �↵2�
�2
↵2�2 + 1�2 ��(v0��u0�)
↵2�2 � v0�2
v0��u0�↵2� ��(v0��u0�)
↵2�2 � v0�2
(v0��u0�)2
↵2�2 + v20
�2 + 1
3
75
K =
2
4↵ � u0
0 � v0
0 0 1
3
5
![Page 31: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/31.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
For a K of the formIt can be shown that (use e.g. Maple)
Remember our constraints and
31
K�T K�1 = B =
2
64
1↵2 � �
↵2�v0��u0�
↵2�
� �↵2�
�2
↵2�2 + 1�2 ��(v0��u0�)
↵2�2 � v0�2
v0��u0�↵2� ��(v0��u0�)
↵2�2 � v0�2
(v0��u0�)2
↵2�2 + v20
�2 + 1
3
75
K =
2
4↵ � u0
0 � v0
0 0 1
3
5
hT1 Bh2 = 0 hT
1 Bh1 � hT2 Bh2 = 0
![Page 32: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/32.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
As B is symmetric
32
B =
2
4b1 b2 b4
b2 b3 b5
b4 b5 b6
3
5
![Page 33: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/33.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
As B is symmetric
If we now defineThe constraints can be written as
33
B =
2
4b1 b2 b4
b2 b3 b5
b4 b5 b6
3
5
b =⇥b1 b2 b3 b4 b5 b6
⇤T
vT
12
(v11 � v22)T
�b = 0
vij = [hi1hj1, hi1hj2 + hi2hj1, hi2hj2, hi3hj1 + hi1hj3, hi3hj2 + hi2hj3, hi3hj3]T
![Page 34: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/34.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
Each view of the plane gives us two rows in the system:
As b has 6 unknowns, we need 3 views of the plane.
Two views can also work if we require
34
Vb = 0
� = 0
![Page 35: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/35.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationOnce b has been estimated, we can extract the
parameters in K according to
35
v0 = (b2b4 � b1b5)/(b1b3 � b22)
� = b6 � (b23 + v0(b2b4 � b1b5)/b1
� = �b2↵2�/�
u0 = �v0↵� b4↵2/�
↵ =p
�/b1
� =q
�b1/(b1b3 � b22)
![Page 36: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/36.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationOnce b has been estimated, we can extract the
parameters in K according to
The H&Z book instead suggests Cholesky factorisation36
v0 = (b2b4 � b1b5)/(b1b3 � b22)
� = b6 � (b23 + v0(b2b4 � b1b5)/b1
� = �b2↵2�/�
u0 = �v0↵� b4↵2/�
↵ =p
�/b1
� =q
�b1/(b1b3 � b22)
![Page 37: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/37.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationCholesky factorisation of B(b)
Gives us which is invertible.
37
B(b) = K�1TK�1
K�1
![Page 38: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/38.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
Once K is computed we can also find the extrinsic camera parameters R,t for each image:
( )
38
r1 = �K�1h1 r2 = �K�1h2 r3 = r1 ⇥ r2
R = [r1 r2 r3] t = �K�1h3
� = 1/||K�1h1|| = 1/||K�1h2||
![Page 39: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/39.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
Once K is computed we can also find the extrinsic camera parameters R,t for each image:
Finally, K, Ri, ti are refined using ML (minimising the cost function)
39
r1 = �K�1h1 r2 = �K�1h2 r3 = r1 ⇥ r2
R = [r1 r2 r3] t = �K�1h3
arg minnX
i=1
mX
j=1
||xij � x̂(K,Ri, ti,Xj)||2
![Page 40: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/40.jpg)
𝚯
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
Once K is computed we can also find the extrinsic camera parameters R,t for each image:
Optionally, all of K, ,Ri, ti are refined using ML:
40
r1 = �K�1h1 r2 = �K�1h2 r3 = r1 ⇥ r2
R = [r1 r2 r3] t = �K�1h3
argminnX
i=1
mX
j=1
kxij � x̂(K,⇥,Ri, ti,Xj)k2
![Page 41: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/41.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
So what about the initial homographies?
Assign each point a WCS value
41
H = K [r1 r2 t]X = [x y 0]T
![Page 42: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/42.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibration
So what about the initial homographies?
Assign each point a WCS value Do we need to know which point is the upper left one on the checker-board? Why not?
42
H = K [r1 r2 t]X = [x y 0]T
![Page 43: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/43.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationCan we use any combination images of the calibration plane?
43
![Page 44: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/44.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Camera calibrationCan we use any combination images of the calibration plane?
The constraints used: andhave to be linearly independent.
Planes must not be parallel!
44
rT1 r2 = 0 rT
1 r1 = rT2 r2
)
![Page 45: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/45.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometry
Recall the epipolar constraint
45
x
T1 Fx2 = 0
![Page 46: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/46.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometry
Recall the epipolar constraint...and the normalised image coordinates
We can instead express the epipolar constraint in normalised coordinates
The matrix E is called the essential matrix.It has some interesting properties...
46
x
T1 Fx2 = 0
x = Kx̂
ˆ
x
T1 K
T1 FK2ˆ
x2 = 0 or
ˆ
x
T1 E
ˆ
x2 = 0
![Page 47: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/47.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometryIn lecture 2 we saw that for cameras P1 and P2:
Now, if
We get and
47
P+2 =
K�1
2
0T
�
F = [e12]⇥P1P+2
P2 = K2 [I|0] and P1 = K1 [R|t]
F = [K1t]⇥K1RK�12
e12 = P1O2
![Page 48: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/48.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometryUsing the cross-product-commutator rule:
(A4.3)
on
...we may express F as either of
48
F = K�T1 [t]⇥RK�1
2 F = K�T1 R
⇥RT t
⇤⇥K�1
2
F = K�T1 R [t2]⇥K�1
2
[b]⇥A = det(A)A�T⇥A�1b
⇤⇥
F = [K1t]⇥K1RK�12
![Page 49: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/49.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometryThis gives us the essential matrix
expressions:
E has only 5 dof (3 from R, 2 from t)recall that F has 7
A necessary and sufficient condition on E is that it has the singular values [a,a,0](see 9.6.1 in the H&Z book for proof)
49
E = [t]⇥R = R⇥RT t
⇤⇥
![Page 50: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/50.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometryThis gives us the essential matrix
expressions:
We can extract R and t (up to scale) from Eif we also make use of one point correspondence (a 3D point known to be in front of both cameras). See 9.6.2 in the H&Z book.
50
E = [t]⇥R = R⇥RT t
⇤⇥
![Page 51: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/51.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Calibrated epipolar geometry4 cases for R and t, just one has point in front
of both cameras.
51
![Page 52: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/52.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Oriented epipolar geometryThe regular epipolar constraint
ignores the knowledge that points are in frontof the camera.
52
x
T1 Fx2 = 0
![Page 53: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/53.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Oriented epipolar geometryIn oriented projective geometry a (visible)
point in front of the camera is defined as having a projection
and a (hidden) point behind the camera has a projection
53
x = �
⇥x1 x2 1
⇤Twith � > 0
x = �
⇥x1 x2 1
⇤Twith � < 0
![Page 54: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/54.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Oriented epipolar geometryThe oriented epipolar constraint properly
distinguishes points in front of and behind the camera
54
�e1 ⇥ x1 = Fx2 , � 2 R+
![Page 55: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/55.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Oriented epipolar geometryThe oriented epipolar constraint can be
interpreted as comparing oriented lines and
(NB! image planes drawn in front of cameras)55
�e1 ⇥ x1 Fx2
![Page 56: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/56.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Oriented epipolar geometryLine normalisation is not unique
The extra information in the sign can be used to encode the line orientation.
56
normD(l) =⇥cos↵ sin↵ �⇢
⇤T
normD(�l) =⇥� cos↵ � sin↵ ⇢
⇤T
![Page 57: Geometry for Computer Vision - Linköping University · May 20, 2014 Computer Vision lecture 4a Computer Vision Laboratory Camera calibration Zhang’s camera calibration (A flexible](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0dfc8ee260c8e61afd1/html5/thumbnails/57.jpg)
May 20, 2014 Computer Vision lecture 4a
Computer Vision Laboratory
Oriented epipolar geometryUsage:The oriented epipolar constraint can be used
to quickly reject a hypothesized F inside a RANSAC loop.
See today’s paper: Chum, Werner and Matas, Epipolar Geometry Estimation via RANSAC benefits from the Oriented Epipolar Constraint, ICPR04
57