3d camera models navab
TRANSCRIPT
-
8/13/2019 3D Camera Models Navab
1/56
3D Computer Vision II
ReminderCamera Models
Nassir Navabbased on a course given at UNC by Marc Pollefeys & the book Multiple View Geometry by Hartley & Zisserman
October 27, 2009
-
8/13/2019 3D Camera Models Navab
2/56
Outline Camera Models
# Geometric Parameters of a Finite Camera# Projective Camera Model
!# Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray!# Decomposition of Camera Matrix
# Cameras at Infinity# Other Camera Models (Pushbroom and Line Cameras)
23D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
3/56
Pinhole Camera Model
# Mapping between 3D world and 2D image# Central projection# Models are described in matrices with particular properties
33D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
4/56
Homogeneous Coordinates
43D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
5/56
Central Projection
53D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
6/56
are the coordinates of the principal point
Principal Point Offset
where
principal point(perpendicular intersection point of
principal axis and image plane)
63D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
7/56
is called camera calibration
matrix
Principal Point Offset
where
73D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
8/56
Inhomogeneous
coordinates
where represents the point in
world coordinates
represents the same point in
camera coordinates
represents the coordinates of the camera
origin in the world coordinate frame
Camera Rotation and Translation
83D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
9/56
Homogeneous
coordinates
projection to image plane from camera coordinates
projection to image plane from world coordinates
Camera Rotation and Translation
3D Computer Vision II - Camera Models 9
-
8/13/2019 3D Camera Models Navab
10/56
where
projection matrix of a general
pinhole camera with 9 DOF
intrinsic camera parameters with 3 DOF
extrinsic camera parameters with each 3 DOF
(camera orientation, position in world coordinates)
Extrinsic and Intrinsic Parameters
3D Computer Vision II - Camera Models 10
-
8/13/2019 3D Camera Models Navab
11/56
No explicit
camera center
where
from
Camera Rotation and Translation
3D Computer Vision II - Camera Models 11
-
8/13/2019 3D Camera Models Navab
12/56
CCD Cameras Non-Square Pixels
number of pixels per unit distance
4 DOF
10 DOF
123D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
13/56
Skew Parameter
skew parameter
5 DOF
finite projective camera with11 DOF
133D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
14/56
non-singular
Finite Projective Camera Summary
projection matrix
11 DOF (5+3+3)
143D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
15/56
decompose projection matrix P in K,R,C
Finite Projective Camera Decomposition of P
non-singular 3x3 matrix (8 DOF)
QR matrix
decomposition
153D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
16/56
Finite Projective Camera Summary
# Camera matrices Pare identical with the set of homogeneous 3x4matrices for which the left 3x3 sub-matrix is non-singular
# If rank(P) = 3, but rank(M) < 3, then camera at infinity# if rank(P) < 3 the matrix mapping will be a line or a point and not a
plane (not a 2D image)
where
163D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
17/56
Outline Camera Models
# Geometric Parameters of a Finite Camera# Projective Camera Model
!# Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray!# Decomposition of Camera Matrix
# Cameras at Infinity# Other Camera Models (Pushbroom and Line Cameras)
173D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
18/56
Camera Anatomy
# Camera center# Column vectors# Principal plane# Axis plane# Principal point# Principal ray
183D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
19/56
P has a 1D null-space
Finite cameras:
Infinite cameras:
Camera Center
we will prove that the 4-vector C is the camera center
points on a line through A and C
since
All 3D points on the line are mapped on the same 2D image point, and
thus the line is a ray through the camera center
193D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
20/56
Column vectors are the image points which project the
axis directions (X,Y,Z) and the origin
Column Vectors
Example for the image
of the y-axis
is the image of the world origin
203D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
21/56
Row Vectors
Represent geometrically particular world planes.
row vectors
column vectors
213D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
22/56
Row Vectors of the Projection Matrix
p1is defined by the camera center and the
line x=0 on the image.
p2is defined by the camera center and the
line y=0 on the image.
Example p2
respectively for p1
223D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
23/56
Principal Plane
Plane through camera center and
parallel to the image plane.
points Xare imaged on the line at
infinity
if X is on the principle plane
especially
233D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
24/56
The line through camera center and
perpendicular to principal plane is the
principal axis.
The intersection of the principal
axis with the image plane is the
principal point.
normal direction to the principal plane
principal point
where
third row of Mand
Principal Point
3D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
25/56
Ambiguity that principal axis points
towards the front of the camera (positive
direction)
direction unaffected by scaling
since
Principal Axis Vector
towards the front of the camera
253D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
26/56
Maps a point in space on the image plane
Forward Projection
Vanishing points
Only M affects the projection of vanishing points
263D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
27/56
-
8/13/2019 3D Camera Models Navab
28/56
w can be interpreted
as the dot product of
the ray CX with the
principal ray direction
(PC=0)
If ,
then m3is a unit vector pointing in positive axis direction
Suppose . Then
Depth of Points
283D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
29/56
Depth of Points: Examples
293D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
30/56
Outline Camera Models
# Geometric Parameters of a Finite Camera# Projective Camera Model
!# Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray!#
Decomposition of Camera Matrix
# Cameras at Infinity# Other Camera Models (Pushbroom and Line Cameras)
303D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
31/56
Finding the camera center C
numerically: find right null-space by SVD of P
Algebraically:
where
Camera Matrix Decomposition
313D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
32/56
Finding the camera center C
Any plane !going through C will be a linear combination of the three
planes defined by the rows of P. Therefore:
where
Camera Matrix Decomposition
323D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
33/56
Finding the camera orientation and internal parameters
using RQ decomposition
Q R=( )-1= -1 -1QR
Decompose
Ambiguity removed by enforcing positive diagonal entries
Camera Matrix Decomposition
333D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
34/56
1
arctan(1/s)
for CCD/CMOS, always s=0
Image from image, s"0 possible
(non coinciding principal axis)
resulting camera:
When is Skew Non-zero?
where H is a 3x3 homography
343D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
35/56
General projective interpretation
# Meaningful decomposition in K,R,t requiresEuclidean image and space
#Camera center is still valid in projective space#Principal plane requires affine image and space#Principal ray requires affine image and Euclidean space
Euclidean vs. Projective Spaces
353D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
36/56
Outline Camera Models
# Geometric Parameters of a Finite Camera# Projective Camera Model
!# Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray!#
Decomposition of Camera Matrix
# Cameras at Infinity# Other Camera Models (Pushbroom and Line Cameras)
363D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
37/56
Cameras with their center lying at infinity
Two types of cameras at infinity:Affine and non-affine cameras
Cameras at Infinity
M is singular
373D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
38/56
Definition:
An affine camera is a camera with a camera matrix P in which the
last row p3T is of the form (0,0,0,1)T.
Affine Cameras
Points at infinity are mapped to points at infinity
383D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
39/56
Affine Cameras
393D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
40/56
modifying p34corresponds to moving along principal ray
Affine Cameras
distance of the world origin from the camera center indirection of the principal ray
403D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
41/56
Combine tracking back and zooming
Affine Cameras
magnification factor k=dt/d0
remains fixed
413D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
42/56
point on plane parallel with principal plane and
through origin, then
general points (not on the parallel plane) with
distance!from the plane
Error in Employing Affine Cameras
423D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
43/56
Error in Employing Affine Cameras
3D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
44/56
Approximation should only cause small error
1.# "much smaller than d02.# points close to principal ray (i.e. small field of view)
Affine Imaging Conditions
443D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
45/56
absorb d0 in K2x2
alternatives, because 8dof (3+3+2), not more
Decomposition of P#
453D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
46/56
-
8/13/2019 3D Camera Models Navab
47/56
orthographic projection
(5dof)
Hierarchy of Affine Cameras
dropping the z-coordinate
473D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
48/56
scaled orthographic projection
(6dof)
Hierarchy of Affine Cameras
483D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
49/56
weak perspective projection
(7dof)
Hierarchy of Affine Cameras
493D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
50/56
Affine camera
(8dof)
full generality of an affine camera
"# Affine camera is a projective camera with principal plane at infinity"# Affine camera maps parallel world lines to parallel image lines"# No center of projection, but direction of projection PAD=0
Hierarchy of Affine Cameras
503D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
51/56
M is singular, but last row not zero
"# Camera center is on plane at infinity"# Principal plane is not plane at infinity"# Images of points at infinity are in general not mapped to infinity
on the image plane
General Camera at Infinity
3D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
52/56
Outline Camera Models
# Geometric Parameters of a Finite Camera# Projective Camera Model
!# Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray!# Decomposition of Camera Matrix
# Cameras at Infinity# Other Camera Models (Pushbroom and Line Cameras)
523D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
53/56
Straight lines are not mapped to straight lines!
(otherwise it would be a projective camera)
(11dof)
Pushbroom Cameras
533D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
54/56
(5dof)
Null-space PC=0 yields camera center
Also decomposition
Line Cameras
543D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
55/56
Summary Camera Models
# Photometric and radiometric properties of a camera# Geometric parameters of a finite camera# Projective cameras
# Camera anatomy (camera center, principle plane, principle point, and principle axis) # Camera matrix decomposition (camera center, orientation, and intrinsic parameter
# Cameras at infinity# Affine cameras# Non-affine cameras
# Alternative models (pushbroom cameras, line cameras)
553D Computer Vision II - Camera Models
-
8/13/2019 3D Camera Models Navab
56/56
Literature on Camera Models
# Chapter 6 in R. Hartley and A. Zisserman, Multiple View Geometry, 2ndedition, Cambridge University Press, 2003.
# Chapter 3 in O. Faugeras, Three-dimensional Computer Vision, MIT Press,1993.
# Chapter 2 in E. Trucco and A. Verri, Introductory Techniques for 3-DComputer Vision, Prentice Hall, 1998.
# H. Gernsheim, The Origins of Photography, Thames and Hudson, 1982.# A. Shashua. Geometry and Photometry in 3D Visual Recognition, Ph.D.
Thesis, MIT, Nov. 1992. AITR-1401.