cs 395/495-25: spring 2003 ibmr: week 6b r 3 r 2 and p 3 p 2 the camera matrix jack tumblin...

CS 395/495-25: Spring 2003CS 395/495-25: Spring 2003

IBMR: Week 6B IBMR: Week 6B

RR33RR22 and P and P33PP22

The Camera MatrixThe Camera Matrix

Jack TumblinJack Tumblin

[email protected]@cs.northwestern.edu

RemindersReminders

• ProjA graded: Good Job!ProjA graded: Good Job!• ProjB being graded...ProjB being graded...• MidTerm Due today, Thurs May 8: Solutions!MidTerm Due today, Thurs May 8: Solutions!

• Watson’s Late Policy: Grade -(3Watson’s Late Policy: Grade -(3nn) points;) points;

n=# of class meetings late n=# of class meetings late• ProjC posted tonight; Due next Thurs, May 15ProjC posted tonight; Due next Thurs, May 15• ProjD coming Thurs May 15, due Thurs May 29ProjD coming Thurs May 15, due Thurs May 29• Take-Home Final Exam coming June 5, dueTake-Home Final Exam coming June 5, due

Cameras RevisitedCameras Revisited

• Goal: Formalize projective 3DGoal: Formalize projective 3D2D mapping 2D mapping

• Homogeneous coords handles infinities wellHomogeneous coords handles infinities well– Projective cameras ( convergent ‘eye’ rays)Projective cameras ( convergent ‘eye’ rays)– Affine cameras (parallel ‘eye’ rays)Affine cameras (parallel ‘eye’ rays)– Composed, controlled as matrix productComposed, controlled as matrix product

• But First: Cameras as Euclidean But First: Cameras as Euclidean RR33RR22::

x’ = fx’ = f x / zx / zy’ = fy’ = f y / zy / z yy

y’y’

ffzzCC


Plenty of Terminology:Plenty of Terminology:

• Image Plane or Focal PlaneImage Plane or Focal Plane

• Focal Distance fFocal Distance f

• Camera Center CCamera Center C

• Principal PointPrincipal Point

• Principal AxisPrincipal Axis• Principal Plane Principal Plane (?!?! DOESN’T touch principle point!?!?)(?!?! DOESN’T touch principle point!?!?)

• Camera Coords (xCamera Coords (xcc,y,ycc,z,zcc))

• Image Coords (x’,y’)Image Coords (x’,y’)

yycc

zzcc

xxcc




• Focal Distance Focal Distance ff

• Camera Center CCamera Center C





ff

yycc

zzcc

xxcc





• Camera Center Camera Center CC





ffCC

yycc

zzcc

xxcc




• Focal DistanceFocal Distance f f


• Principal Point Principal Point pp




ffCC

yycc

zzcc

xxcc

pp






• Principal PointPrincipal Point pp




ffCC

yycc

zzcc

xxcc

pp







• Principal AxisPrincipal Axis• Principal Plane Principal Plane (?!?! DOESN’T touch principle point (?!?! DOESN’T touch principle point PP !?!?) !?!?)



ffCC

yycc

zzcc

xxcc

pp







• Principal AxisPrincipal Axis• Principal PlanePrincipal Plane (?!?! DOESN’T touch principle point!?!?)(?!?! DOESN’T touch principle point!?!?)

• Camera Coords Camera Coords (x(xcc,y,ycc,z,zcc))


ffCC

yycc

zzcc

xxcc

pp







• Principal AxisPrincipal Axis• Principal PlanePrincipal Plane (?!?! DOESN’T touch principle point!?!?)(?!?! DOESN’T touch principle point!?!?)



ffCC

yycc

zzcc

xxcc

pp

Homogeneous Coords: RHomogeneous Coords: R33PP22

• Basic Camera as a 3x4 matrix:Basic Camera as a 3x4 matrix:

• Tricky!Tricky! XX is in augmented is in augmented RR33, not P, not P33 (yet) (yet)

xx is in is in PP22 space space• As shown: origin of (As shown: origin of (x’,y’x’,y’) is principal point ) is principal point pp,,

but pixel counting starts at corners…but pixel counting starts at corners…

x’x’y’y’z’z’

ff 0 0 0 0 000 0 ff 0 0 000 0 0 0 11 0 0

xxcc

yycc

zzcc

11

==

PP00 X = x X = x

yyy’y’

ffzz

CC

yycc

xxcc

zzcc

pp



• Translate Translate imageimage coords ( coords (x’,y’x’,y’):):Shifts principal point Shifts principal point pp from ( from (0,00,0) to () to (ppxx,p,pyy))

– is is NOTNOT obvious: scales obvious: scales zzcc

– does does NOTNOT use ‘homogeneous’ use ‘homogeneous’ 11 term in term in X X

x’x’y’y’z’z’

xxcc

yycc

zzcc

11

==

PP00 X = x X = x

yyy’y’

ffzz

CC

yycc

xxcc

zzcc

pp

ff 0 0 ppxx 00

0 0 ff ppyy 00

0 0 0 0 11 0 0



Less common adjustments:Less common adjustments:

• Non-square pixels? change scaling Non-square pixels? change scaling ((xx, , yy))

• Parallelogram pixels? set nonzero skew Parallelogram pixels? set nonzero skew ss

K matrixK matrix: “(internal) camera calib. matrix”: “(internal) camera calib. matrix”

xxyyzz

xxcc

yycc

zzcc

11

==

PP00 X = x X = x

yyy’y’

ffzz

CC

yycc

xxcc

zzcc

pp

xxff ss ppxx 00

0 0 yyff ppyy 00

0 0 0 0 11 0 0

K K (3x3 submatrix)(3x3 submatrix)

Matrix Form: Euclidean RMatrix Form: Euclidean R33RR22

• K matrixK matrix: “: “internalinternal camera calib. matrix” camera calib. matrix”

• RR··T matrixT matrix: “: “externalexternal camera calib. matrix” camera calib. matrix” – T matrix: Translate world to cam. originT matrix: Translate world to cam. origin– R matrix: 3D rotate world to fit cam. axesR matrix: 3D rotate world to fit cam. axes

Combine: writeCombine: write

(P(P00··RR··T)T)··X = xX = x

asas

PP··X = xX = x

Output: Output: xx2D Camera Image 2D Camera Image

Input:Input: XX3D World Space 3D World Space

zz

yy

xx

XX (world space)(world space)

zz cc

yy cc

xx cc

CC

xx(c

amer

a sp

ace)

(cam

era

spac

e)

ENDEND

••••••00

••••••11

Uses for Camera Matrix Uses for Camera Matrix PP

• PP’s Null Space: camera center ’s Null Space: camera center CC in world space! in world space!P C = 0 P C = 0 (solve for C. SVD works, but answer matches (solve for C. SVD works, but answer matches

these:)these:)

– Finite Camera: Finite Camera: C = C = Affine Camera: Affine Camera: C =C =

• Surprise!Surprise!(minor)(minor) works projectively too: works projectively too: PP33PP22

(see book pg. 152-3 for restrictions—almost none!)(see book pg. 152-3 for restrictions—almost none!)

xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or

~~~~

PP = =

• • • •• • • •• • • •• • • •• • • •• • • •

MM pp44

--MM-1-1··pp44 dd(where (where Md=0)Md=0)

~~

~~


ColumnsColumns of P matrix: Points in image-space: of P matrix: Points in image-space: • PP11,P,P22,P,P33 == image of x,y,z axis vanishing points == image of x,y,z axis vanishing points

– Proof: let D = [1 0 0 0]Proof: let D = [1 0 0 0]TT = point on x axis, at inifinity = point on x axis, at inifinity– PD = 1PD = 1stst column of P. Repeat for y and z axes column of P. Repeat for y and z axes

• PP44 == image of the world-space origin pt. == image of the world-space origin pt.– Proof: let Proof: let DD = [0 0 0 1]T = world origin = [0 0 0 1]T = world origin– PDPD = 4 = 4thth column of column of PP = image of origin pt. = image of origin pt.

xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or PP = =• • •• • • ••• • •• • • ••• • • • • • • •

PP11 PP22 PP33

ff CC

yycc

zzcc

xxcc

pp

PP44


RowsRows of P matrix: planes in world space of P matrix: planes in world space• row 1 = row 1 = pp1T1T = image x-axis plane = image x-axis plane• row 2 = row 2 = pp2T2T = image y-axis plane = image y-axis plane• row 1 = row 1 = pp3T3T = camera’s principal plane = camera’s principal plane

xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or PP = =• • •• • • ••• • •• • • ••• • • • • • • •

pp1T1T

pp2T2T

pp3T3T

ff CC

yycc

zzcc

xxcc

pp


RowsRows of P matrix: planes in world space of P matrix: planes in world space• row 1 = row 1 = pp11 = image x-axis plane = image x-axis plane• row 2 = row 2 = pp22 = image y-axis plane = image y-axis plane• row 1 = row 1 = pp33 = camera’s principal plane = camera’s principal plane

xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or PP = =• • •• • • ••• • •• • • ••• • • • • • • •

pp1T1T

pp2T2T

pp3T3T

ff CC

yycc

zzcc

xxcc

pp


RowsRows of P matrix: planes in world space of P matrix: planes in world space• row 1 = row 1 = pp11 = image x-axis plane = image x-axis plane• row 2 = row 2 = pp22 = image y-axis plane = image y-axis plane• Careful!Careful! Shifting image origin Shifting image origin

shifts the x,y axis planes!shifts the x,y axis planes!

xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or PP = =• • •• • • ••• • •• • • ••• • • • • • • •

pp1T1T

pp2T2T

pp3T3T

ff CC

yycc

zzcc

xxcc

pp



xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or PP = =• • •• • • ••• • •• • • ••• • • • • • • •

pp1T1T

pp2T2T

pp3T3T

ff CC

yycc

zzcc

xxcc

pp



– princip. plane princip. plane pp33= = [p[p3131 p p3232 p p3333 p p3434]]TT

– its normal direction: its normal direction: [p[p3131 p p3232 p p33 33 0 ]0 ]TT

– Why is it normal? It’s the world-space Why is it normal? It’s the world-space endpoint of endpoint of zzcc axis (at infinity ) axis (at infinity )

xxcc

yycc

zzcc

xxww

yyww

zzww

ttww

==• • • •• • • •• • • •• • • •• • • •• • • •

PPPP··X = xX = x, or, or PP = =• • •• • • ••• • •• • • ••• • • • • • • •

pp1T1T

pp2T2T

pp3T3T

ff CC

yycc

zzcc

xxcc

pp

Principal Principal plane pplane p33

Chapter 6 Chapter 6 In Just One Slide:In Just One Slide:

Given point correspondence sets Given point correspondence sets (x(xii X Xii)), How , How do you find camera matrix do you find camera matrix P ? P ? (full 11 DOF)(full 11 DOF)

Surprise! You already know how !Surprise! You already know how !• DLT method: DLT method:

-rewrite H x = x’ as Hx -rewrite H x = x’ as Hx x’ = 0 x’ = 0--rewrite rewrite P X = xP X = x as as PX PX x = 0 x = 0-vectorize, stack, solve Ah = 0 for h vector-vectorize, stack, solve Ah = 0 for h vector-vectorize, stack, solve -vectorize, stack, solve Ap = 0Ap = 0 for for pp vector vector

--Normalizing step removes origin dependenceNormalizing step removes origin dependence

• More data More data better results (at least 28 point pairs) better results (at least 28 point pairs)• Algebraic & Geometric Error, Sampson Error…Algebraic & Geometric Error, Sampson Error…

cs 395/495-25: spring 2003 ibmr: week 6b r 3 r 2 and p 3 p 2 the camera matrix jack tumblin...

Documents

z c camera coords x

z c image coords x

f y z y y f z c slide

focal planeimage plane

f x z y

yimage coords x

doesnt touch principle

d mapping homogeneous