![Page 1: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/1.jpg)
Camera Calibration
Sebastian Thrun, Gary Bradski, Daniel RussakoffStanford CS223B Computer Vision
http://robots.stanford.edu/cs223b
(with material from David Forsyth, James Rehg and Allen Hanson)
![Page 2: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/2.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
A Quiz
How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10:
How Many Corner Points do we need in Total?
1: 2: 3: 4: 10: 20:
![Page 3: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/3.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Example Calibration Pattern
![Page 4: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/4.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Perspective Camera Model
WX
WY
WZ
CX
CY
Object Space
CZ
![Page 5: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/5.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (extrinsic)
?),(
W
W
W
C
C
Z
Y
X
fY
X
![Page 6: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/6.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Experiment 1: Parallel Board
![Page 7: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/7.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
30cm10cm 20cm
Projective Perspective of Parallel Board
![Page 8: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/8.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Experiment 2: Tilted Board
![Page 9: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/9.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
30cm10cm 20cm
500cm50cm 100cm
Projective Perspective of Tilted Board
![Page 10: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/10.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (extrinsic)
),,,,( T
Z
Y
X
fY
X
W
W
W
C
C
rotation
translation (3D)
![Page 11: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/11.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (extrinsic)
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
01
sin0cos
100
0cossin
0sincos
~
~
~
C
C
CC
C
Y
X
Z
f
Y
X~
~
~
Homogeneous Coordinates
![Page 12: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/12.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Homogeneous Coordinates
Idea: Most Operations Become Linear!
Extract Image Coordinates by Z-normalization
C
C
CC
C
Y
X
ZY
X~
~
~1
C
C
C
Z
Y
X
~
~
~
1
12
10
000,1
200,1
000,10
5.1
18
15
1
12
10
![Page 13: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/13.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Advantage of Homogeneous C’s
Z
Y
X
W
W
W
C
C
T
T
T
iZ
iY
iX
iY
iXi
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0
0
cos
sin
sin
cos
][~
][~
][
i-th data point
parametersrotation in nonlinear but
][][Equation Quadratici ibXiAX T
![Page 14: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/14.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (intrinsic)
xs
ysPixel size
C
C
CC
C
Y
X
Z
f
Y
X~
~
~
Focal length
Image center
yx oo ,
![Page 15: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/15.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
5.~
~
5.~
~
yC
C
y
xC
C
x
im
im
oZ
Y
s
f
oZ
X
s
f
y
x
Intrinsic Transformation
![Page 16: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/16.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0cossin
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0sincos
y
Z
Y
X
W
W
W
Z
Y
X
W
W
W
y
x
Z
Y
X
W
W
W
Z
Y
X
W
W
W
x
im
im
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
y
x
Plugging the Model Together!
![Page 17: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/17.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Summary Parameters
Extrinsic
– Rotation
– Translation
Intrinsic
– Focal length
– Pixel size
– Image center coordinates
– (Distortion coefficients - see JYB’s tutorial )
,.
T
f
),( yx oo
,...1k
),( yx ss
![Page 18: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/18.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Q: Can We recover all Intrinsic Params?
No
![Page 19: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/19.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Summary Parameters, Revisited
Extrinsic
– Rotation
– Translation
Intrinsic
– Focal length
– Pixel size
– Image center coordinates
– (Distortion coefficients - see JYB’s tutorial )
,,
T
f
),( yx oo
,...1k
),( yx ssFocal length, in pixel units
Aspect ratioy
x
s
s
xs
f
![Page 20: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/20.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Trucco
Substitute
Advantage: Equations are linear in params
If over-constrained, minimize Least Mean Square fct
One possible solution:
Enforce constraint that R is rotation matrix
Lots of considerations to recover individual params…
cossin0
sincos0
001
cos0sin
01
sin0cos
100
0cossin
0sincos
333231
232221
131211
rrr
rrr
rrr
bAX
bAAAX TT 1
min)()( bAXbAX T
},,,,{ yxx
oos
fTRX
![Page 21: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/21.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0cossin
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0sincos
y
Z
Y
X
W
W
W
Z
Y
X
W
W
W
y
x
Z
Y
X
W
W
W
Z
Y
X
W
W
W
x
im
im
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
y
x
),,,,,,,( yxxW
W
W
im
im oos
fT
Z
Y
X
gy
x
![Page 22: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/22.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet, cont’d
Calibration Examples:
),,,,,,,
]1[
]1[
]1[
(]1[
]1[yx
xW
W
W
im
im oos
fT
Z
Y
X
gy
x
),,,,,,,
][
][
][
(][
][yx
xW
W
W
im
im oos
fT
NZ
NY
NX
gNy
Nx
…),,,,,,,
]2[
]2[
]2[
(]2[
]2[yx
xW
W
W
im
im oos
fT
Z
Y
X
gy
x
![Page 23: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/23.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet, cont’d
Least Mean Square
min),,,,,,,
][
][
][
(][
][
2
iyx
xW
W
W
im
im oos
fT
iZ
iY
iX
giy
ixJ
![Page 24: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/24.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet, cont’d
Least Mean Square
Gradient descent:J
},,,,,,{ yxx
oos
fTX
0X
0X
J
)(1.0 11
kkk XX
JXX
min),,,,,,,
][
][
][
(][
][
2
iyx
xW
W
W
im
im oos
fT
iZ
iY
iX
giy
ixJ
![Page 25: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/25.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Trucco Versus Bouguet
Trucco: Mimization of Squared
distance in parameter space
Bouguet Minimization of Squared
distance in Image space
)}ˆ
ˆ
ˆ
ˆ()
ˆ
ˆ
ˆ
ˆ(
2
1exp{
|2|
1)
ˆ
ˆ|
ˆ
ˆ( 1
5.
edge
edge
edge
edgeT
edge
edge
edge
edge
edge
edge
edge
edge
y
x
y
x
y
x
y
x
y
x
y
xp
)ˆ
ˆ
ˆ
ˆ()
ˆ
ˆ
ˆ
ˆ(
2
1)
ˆ
ˆ|
ˆ
ˆ(log 1
edge
edge
edge
edgeT
edge
edge
edge
edge
edge
edge
edge
edge
y
x
y
x
y
x
y
xconst
y
x
y
xp
![Page 26: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/26.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Q: How Many Images Do We Need?
Assumption: K images with M corners each 4+6K parameters 2KM constraints 2KM 4+6K M>3 and K 2/(M-3) 2 images with 4 points, but will 1 images with 5
points work? No, since points cannot be co-planar!
![Page 27: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/27.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Nonlinear Distortions
Barrel and Pincushion Tangential
![Page 28: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/28.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Barrel and Pincushion Distortion
telewideangle
![Page 29: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/29.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Models of Radial Distortion
)1( 42
21 rkrk
y
x
y
x
d
d
distance from center
![Page 30: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/30.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Tangential Distortion
cheapglue
cheap CMOS chipcheap lense image
cheap camera
![Page 31: Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from](https://reader034.vdocuments.net/reader034/viewer/2022051417/56649f345503460f94c51d09/html5/thumbnails/31.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Image Rectification