multi view geometry (spring '08)

1

Camera Models 1

2008-1

Multi View Geometry (Spring '08)

Prof. Kyoung Mu LeeSoEECS, Seoul National University

Camera Models

Camera Models 2

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Camera Models – Finite Cameras• Pinhole camera model:

In homogeneous coordinates

ZYfy

ZY

fy

=⇒=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

10100

ZYX

ff

ZfYfX

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

1010101

1ZYX

ff

ZfYfX

PXx = [ ]0|I)1,,(diagP ff=

2

Camera Models 3


Finite cameras

• Principal point offset model:Set the coordinates of the principal point (image center) beThen the mapping becomes

Or in homogeneous representation

Tyx pp ),(

calibration matrix

Tyx

T pZfYpZfXZYX )/,/(),,( ++a

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛++

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

10100

1ZYX

pfpf

ZZpfYZpfX

ZYX

y

x

x

x

a

[ ] camX0|IKx =⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

1y

x

pfpf

K

Camera Models 4


Finite cameras

• Camera Rotation and Translation:

center camera the of scoordinate ousinhomogene :~

scoordiante camera ofmatrix rotation :scoordinate camera ousinhomogene:~

scoordinate worldousinhomogene:~)~~~

cam

cam

C

RX

X

CXR(X −=

world coordinatescamera coordinates

X10RCR

110

C~RRXcam ⎥⎦

⎤⎢⎣

⎡ −=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎥⎦

⎤⎢⎣

⎡ −=

ZYX

[ ] camX0|IKx = [ ]XC~|IKRx −= PXx =[ ]t|RKP =

C~Rt −=

3

Camera Models 5


[ ]C~|IKRP −=

Finite cameras

• CCD Cameras:Non-square pixels

10 DOF

• Finite projective camera:Including skew parameter s (zero for most normal cameras)

11 DOF (5+3+3)

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

1yyy

xxx

pmfmpmfm

M: 3x3 nonsingular

[ ]44 ][ pM|IMp|MP 1−==

{finite cameras}={P4x3 =[M|p4]|det M≠0}

If mx and my pixels per unit distancein x and y directions

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

10

0

yx

x

x

αα

K⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

11y

x

x

x

pfpf

mm

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

1yx

xx

pp

αα

K

Camera Models 6


Camera Models - The projective camera

• General projective cameras: [ ]4p|MP =

C~

4

Camera Models 7


The projective camera

• Camera center:is the null-space of the camera projection matrix

For all A, all points on the line AC project on image of A, therefore C is the camera centerImage of camera center is (0,0,0)T, i.e. undefined

Finite Camera case:

Infinite Camera case:

0PC =CAX λ)(1λ −+=

PAPCPAPXx λλ)(1λ =−+==

⎟⎟⎠

⎞⎜⎜⎝

⎛−=

−

14

1pMC

0,0

=⎟⎟⎠

⎞⎜⎜⎝

⎛= Md

dC

C~

Camera Models 8



• Column vectors of P:[ ]

[ ] origin world1000 , While,

. and , toSimilarilyon point axis,-X ofdirection 0001 ,

4

3

1

⇒==

⇒== ∞

Too

TXX

DPDp

ppDPDp

2

π

5

Camera Models 9



• Row vectors of P:

The principal plane: qthe plane passing C parallel to the image plane, set of X, s.t.,

Axis plane:qPoints on P1,

qSimilarly, P2 is a plane defined by C and the line y=0

plane principal thengrepresenti vector theis 0)0,,(

3

3

PXPPX

⇒

=⇒= TTyx

0 line theand by defined plane a is Thus,on lies 0

axis-yon points),,0(0

1

11

1

=

⇒=

⇒=⇒=

x

yT

TT

CPPCCP

PXXP ω

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=T

T

T

3

2

1

ppp

P

Camera Models 10



• C is the intersection of P1, P2 and P3: 0CPPP

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

T

T

T

2

2

1

Principal axis

Camera center C

Principal point

6

Camera Models 11



• The principal point:

P3

Normal direction of P3

= (p31,p32,p33)T

x0

[ ] 334

30

ˆ|ˆ MmPPMPPx ===

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

0

ˆ33

32

31

3

ppp

P

∞π

Point on π∞

Camera Models 12



• Principal axis vector:There is an ambiguity between m3 and –m3

[ ] [ ]

cameratheoffront thetowarddirection theis on t independen isdirection

)det( )det(ˆ)ˆdet(ˆ

ˆ|ˆ~|for

then,)det(Let

434

343

4

3

v

vmMmKRmMv

pMCRRKP

mMv

⇒⇒

==

==

=−=

=

kkk

k

k 3m

7

Camera Models 13


Action of projective camera on point

• Forward projection:

Projection of vanishing point:

• Back-projection:

Finite camera case:

D=(dT,0)T

1)( −+ = TT PPPP

A ray passing through C and P+x

41~ pMC −−=

CBack projected point on π∞

xPX +=

IPP =+

Camera Models 14



• Depth of points:Consider the projection of a space point X onto an image point x

. ofdirection (positive) in the from ofdepth the

represent ,1 and 0 det if ,Thus

)~~()(

Then,

1

~

1

3

3

333

3

2

1

mCX

mM

CXmCXPXP

X

PPP

PXx

ω

ω

ω

=>

−=−==

⎥⎦

⎤⎢⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

==⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

TTT

T

T

T

yx

Since PC=0

dot product of the ray from C to X, with the principal ray direction

8

Camera Models 15



• In general, for

( )3

)sign(det;depthm

MPXT

ω=

( )TX X,Y,Z,T=

Camera Models 16


The projective cameras - Decomposition of P

• Finding the camera center:

• Finding the camera orientation and internal parameters:

Decompose M as M = KR using RQ factorization methodK: Upper triangular matrix R: rotation matrix (orthogonal) Keep K to have positive diagonals to resolve ambiguity

SVD using of space null),,,( , PC0PC →== TTZYX

p. 67

9

Camera Models 17


Decomposition of P

• When is skew non-zero?

for CCD/CMOS, always s=0Image from image, s≠0 possible (non coinciding principal axis)

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

1yx

xx

pps

αα

K1 γ

arctan(1/s)

Camera Models 18


Decomposition of P

• Euclidean vs. projectivegeneral projective interpretation

Meaningfull decomposition in K,R,t requires Euclidean image and spaceCamera center is still valid in projective spacePrincipal plane requires affine image and spacePrincipal ray requires affine image and Euclidean space

[ ] [ ]homography 44010000100001

homography 33 ×⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡×=P

10

Camera Models 19


Cameras at infinity – Affine cameras

• Camera at infinity Cameras center lying on the plane at infinityaffine and non-affine cameras

• Affine cameras: the last row of P, P3T is the form of (0,0,0,1)Thus, points at infinity are mapped to points at infinityConsider moving back a finite projection camera while zooming tokeep the size of object be the same

[ ]

0singular is

4

=

=

PCM

p|MP

ray principal theofdirection in thecenter camera thefromorigin world theof distance the:~

ray principal ofdirection the:3

0

3

Cr

rTd −=

Camera Models 20


Affine cameras

• Moving back at unit speed for a time t, so that

• Next, by zooming with a zooming factor so that the image size remains fixed. Then,

3~~ rCC t−→

ttd T

t

at timeray principal theofdirection in thecenter camera thefromorigin world theof distance the:~3 +−= Cr

0/ ddk t=

11

Camera Models 21


Affine cameras

Camera Models 22


Affine cameras

• As dt tends to infinity,

Affine camera

12

Camera Models 23


Affine cameras-Error

• For point on plane parallel with principal plane and through the world origin,

• For general points

⎟⎟⎠

⎞⎜⎜⎝

⎛ +=

1βα 21 rr

X XPXPXP ∞== t0

⎟⎟⎠

⎞⎜⎜⎝

⎛ Δ++=

1βα 321 rrr

X

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

+==

Δ

~~

0

0proj

dyx

KXPx

projxaffinex

0x

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛== ∞

0

affine~~

dyx

KXPx

( )0proj0

00affine

~-~~-~ xxxxd

d Δ+=

Camera Models 24


Affine cameras-Error

• Thus, the error in employing an affine camera:

• Approximation error is small ifΔ is small compared to the average depth d0

The distance of the point from the principal ray is small (i.e. small field of view)

( )0proj0

projaffine~-~~-~ xxxx

dΔ

=

13

Camera Models 25


Affine cameras-Decomposition

• Decomposition of

or

Thus,

∞P

0x0 =~for

⎥⎦

⎤⎢⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡= ×

1100000100001

1ˆˆ

22TT 0

tR0

0K

Camera Models 26


Affine cameras

• Summary of parallel projection

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=∞

100000100001

P canonical representation

⎥⎦

⎤⎢⎣

⎡= ×

100K 22K calibration matrix

principal point is not defined

14

Camera Models 27


Hierarchy of affine cameras

• Orthographic projection:

• Incorporating rotation and translation

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11

100000100001

1YX

ZYX

yx

[ ]tM0

rr

0tR

P =⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎦

⎤⎢⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

11

100000100001

22

11

tt

T

T

T

unit norm

5 DOF

Camera Models 28



• Scaled orthographic projection:

• Weak perspective projection:

6 DOF

7 DOF

15

Camera Models 29



• Affine camera:

• In inhomogeneous coordinates

⎥⎦

⎤⎢⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡= ×

1100000100001

1ˆˆ

22TT 0

tR0

0K

3x3 affine 4x4 affine

3x4 affine

linear mapping

8 DOF

Camera Models 30



• Properties of the affine camera:

Any projective camera with the principal plane is the plane at infinity is an affine cameraThe camera center is also lies on the plane at infinityParallel lines in space are mapped to parallel image linesThe vector d satisfying M2x3d=0 is the direction of parallel projection, and (dT,0)T is the camera center.

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

00

YX

ZYX

APplane at infinity

= principal plane

point at infinity

0d

P =⎥⎦

⎤⎢⎣

⎡0A

16

Camera Models 31


General cameras at infinity

• Non-affine camera at infinity

[ ]

0zeronot is rowlast but thesingular is

4

=

=

PCM

p|MP

Camera Models 32


Other camera models

• Pushbroom cameras:• Linear Pushbroom (LP) camera – satellite sensor, SPOT

(11dof)

17

Camera Models 33


Other camera models

• straight lines are not mapped to straight lines

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡→

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

ωω

/~~

1y

xyx

ZYX

yx

P

hyperbola ~~~~ ⇒=+++⇒ oyxyx δγβα

Camera Models 34


Other camera models

• Line cameras:2D to 1D projection

5 DOF

0PC =

camera center

multi view geometry (spring '08)

Documents