Download - Camera Models and Imaging
Camera Models and Imaging
Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition
CS4243 Camera Models and Imaging 1
Through our eyes… Through our eyes we see the world.
CS4243 Camera Models and Imaging 2
CS4243 Camera Models and Imaging 3
lens
pupil
cornea
iris
retina
optic nerves
focus light
control amount of light entering the eye
captures image
transmit image
Through their eyes… Camera is computer’s eye.
CS4243 Camera Models and Imaging 4
Camera is structurally the same the eye.
CS4243 Camera Models and Imaging 5
lens lens | cornea
sensor | retina
CS4243 Camera Models and Imaging 6
lens | cornea
lens
aperture | pupil
CS4243 Camera Models and Imaging 7
aperture | pupil
aperture plates | iris
Imaging Images are 2D projections of real world scene. Images capture two kinds of information:
Geometric: positions, points, lines, curves, etc. Photometric: intensity, colour.
Complex 3D-2D relationships. Camera models approximate relationships.
CS4243 Camera Models and Imaging 8
Camera Models Pinhole camera model Orthographic projection Scaled orthographic projection Paraperspective projection Perspective projection
CS4243 Camera Models and Imaging 9
Pinhole Camera Model
CS4243 Camera Models and Imaging 10
image plane pinhole virtual image
plane 3D
object
focal length
2D image
Math representation
CS4243 Camera Models and Imaging 11
world frame / camera frame
image frame
camera centre
image centre
optical axis
projection line
By default, use right-handed coordinate system.
CS4243 Camera Models and Imaging 12
3D point P = (X, Y, Z)T projects to 2D image point p = (x, y)T .
By symmetry,
i.e.,
Simplest form of perspective projection.
CS4243 Camera Models and Imaging 13
Orthographic Projection 3D scene is at infinite distance from camera. All projection lines are parallel to optical axis. So,
CS4243 Camera Models and Imaging 14
Scaled Orthographic Projection
CS4243 Camera Models and Imaging 15
Scene depth << distance to camera. Z is the same for all scene points, say Z0
Perspective Projection
CS4243 Camera Models and Imaging 16
camera frame
world frame
image frame is parallel to camera’s X-Y plane
Intrinsic Parameters Pinhole camera model
Camera sensor’s pixels not exactly square x, y : coordinates (pixels) k, l : scale parameters (pixels/m) f : focal length (m or mm)
CS4243 Camera Models and Imaging 17
f, k, l are not independent. Can rewrite as follows (in pixels):
So,
Image centre or principal point c may not be at origin. Denote location of c in image plane as cx, cy. Then,
Along optical axis, X = Y = 0, and x = cx, y = cy.
CS4243 Camera Models and Imaging 18
Image frame may not be exactly rectangular. Let θ denote skew angle between x- and y-axis. Then,
Combine all parameters yield
CS4243 Camera Models and Imaging 19
homogeneous coordinates
Intrinsic parameter matrix
Denoting ρ = Z, we get
Some books and papers absorb ρ into :
giving Be careful.
A simpler form of K uses skew parameter s:
CS4243 Camera Models and Imaging 20
Extrinsic Parameters Camera frame is not aligned with world frame. Rigid transformation between them:
Coordinates of 3D scene point in camera frame. Coordinates of 3D scene point in world frame. Rotation matrix of world frame in camera frame. Position of world frame’s origin in camera frame.
CS4243 Camera Models and Imaging 21
frame
object
Translation matrix
Rotation matrix
CS4243 Camera Models and Imaging 22
Rotation matrix is orthonormal:
So, . In particular, let
Then,
CS4243 Camera Models and Imaging 23
Combine intrinsic and extrinsic parameters:
Simpler notation:
CS4243 Camera Models and Imaging 24
Lens Distortion Lens can distort images,
especially at short focal length.
CS4243 Camera Models and Imaging 25
barrel pin-cushion fisheye
Radial distortion is modelled as
with . Actual image coordinates
CS4243 Camera Models and Imaging 26
distorted coordinates
undistorted coordinates
distortion parameters
Camera Calibration Compute intrinsic / extrinsic parameters. Capture calibration pattern from various views.
CS4243 Camera Models and Imaging 27
Detect inner corners in images.
Run calibration program (available in OpenCV). CS4243 Camera Models and Imaging 28
Homography A transformation between projective planes. Maps straight lines to straight lines.
CS4243 Camera Models and Imaging 29
Homography equation:
Affine is a special case of homography:
CS4243 Camera Models and Imaging 30
image point
image point homography
Scaling H by s does not change equation:
Homography is defined up to unspecified scale. So, can set h33 = 1. Expanding homography equation gives
CS4243 Camera Models and Imaging 31
Assembling equations over all points yields
System of linear equations. Easy to solve.
CS4243 Camera Models and Imaging 32
Example Circles: input corresponding points. Black dots: computed points, match input points.
CS4243 Camera Models and Imaging 33
Before homographic transform
CS4243 Camera Models and Imaging 34
After homographic transform
CS4243 Camera Models and Imaging 35
Summary Simplest camera model: pinhole model. Most commonly used model: perspective model. Intrinsic parameters:
Focal length, principal point.
Extrinsic parameters: Camera rotation and translation.
Lens distortion Homography: maps lines to lines.
CS4243 Camera Models and Imaging 36
Further Reading Camera models: [Sze10] Section 2.1.5, [FP03]
Section 1.1, 1.2, 2.2, 2.3. Lens distortion: [Sze10] Section 2.1.6, [BK08]
Chapter 11. Camera calibration: [BK08] Chapter 11, [Zha00]. Image undistortion: [BK08] Chapter 11.
CS4243 Camera Models and Imaging 37
References Bradski and Kaehler. Learning OpenCV. O’Reilly, 2008.
D. A. Forsyth and J. Ponce. Computer Vision: A Modern Approach. Pearson Education, 2003.
R. Szeliski. Computer Vision: Algorithms and Applications. Springer, 2010.
Z. Zhang. A flexible new technique for camera calibration. IEEE Trans. PAMI, 22(11):1330–1334, 2000.
CS4243 Camera Models and Imaging 38