perspective projection - eskisehir.edu.treem.eskisehir.edu.tr/cihant/eem 561/icerik/lec02_ct... ·...

Perspective projection

Albrecht Dürer, Mechanical creation of a perspective image, 1525

Slides Credit: Prof. David Forsyth, Prof. Svetlana Lazebnik

Overview of next two lectures

• The pinhole projection model• Qualitative properties

• Perspective projection matrix

• Cameras with lenses• Depth of focus

• Field of view

• Lens aberrations

• Digital cameras• Sensors

• Color

• Artifacts

Let’s design a camera

Idea 1: put a piece of film in front of an object

Do we get a reasonable image?

Slide by Steve Seitz

Pinhole camera

Add a barrier to block off most of the rays


Pinhole camera

• Captures pencil of rays – all rays through a single

point: aperture, center of projection, optical center,

focal point, camera center

• The image is formed on the image plane


Camera obscura

• Latin. camera: chamber

or room,

obscura: darkened

• Basic principle known to

Mozi (470-390 BCE),

Aristotle (384-322 BCE)

• Drawing aid for artists:

described by Leonardo

da Vinci (1452-1519)

Gemma Frisius, 1558

Source: A. Efros

Turning a room into a camera obscura

Abelardo Morell, Camera Obscura Image of Manhattan View Looking South in Large

Room, 1996

http://www.abelardomorell.net/camera_obscura1.html

From Grand Images Through a Tiny Opening, Photo

District News, February 2005

http://www.abelardomorell.net/camera_obscura1.html

Turning a room into a camera obscura

A. Torralba and W. Freeman, Accidental Pinhole and Pinspeck Cameras, CVPR 2012

http://people.csail.mit.edu/torralba/research/accidentalcameras/

Pinhole cameras everywhere

Tree shadow during a solar eclipsephoto credit: Nils van der Burg

http://www.physicstogo.org/index.cfm


http://www.physicstogo.org/index.cfm

Point of observation

Figures © Stephen E. Palmer, 2002

Dimensionality reduction: from 3D to 2D

3D world 2D image

What properties of the world are preserved?• Straight lines, incidence

What properties are not preserved?• Angles, lengths Slide by A. Efros

Single-view Geometry

How tall is this woman?

Which ball is closer?

How high is the camera?

What is the camera

rotation?

What is the focal length of

the camera?

Modeling projection

• To compute the projection P’ of a scene point P, form

the visual ray connecting P to the camera center O and

find where it intersects the image plane

• All scene points that lie on this visual ray have the same

projection in the image

• Are there scene points for which this projection is undefined?

P

O

Modeling projection

f z

P

?O

P’

The coordinate system• The optical center (O) is at the origin

• The image plane is parallel to xy-plane or perpendicular to the z-axis,

which is the optical axis

Projection equations• Derived using similar triangles ),(),,(

z

yf

z

xfzyx

y

Point of observation

Fronto-parallel planes

• What happens to the projection of a pattern

on a plane parallel to the image plane?• All points on that plane are at a fixed depth z

• The pattern gets scaled by a factor of f / z, but angles and

ratios of lengths/areas are preserved

),(),,(z

yf

z

xfzyx

Fronto-parallel planes

• What happens to the projection of a pattern

on a plane parallel to the image plane?• All points on that plane are at a fixed depth z

• The pattern gets scaled by a factor of f / z, but angles and

ratios of lengths/areas are preserved

Jan Vermeer, The Music Lesson, 1662-1665Piero della Francesca, Flagellation of Christ, 1455-1460

What about non-fronto-parallel planes?

Jan Vermeer, The Music Lesson, 1662-1665Piero della Francesca, Flagellation of Christ, 1455-1460

Projection can be tricky…Slide source: Seitz

Projection can be tricky…Slide source: Seitz

Making of 3D sidewalk art:

http://www.youtube.com/watch?v=3SNYtd0Ayt0

http://www.youtube.com/watch?v=3SNYtd0Ayt0

Vanishing points

• All parallel lines converge to a vanishing point

• Each direction in space is associated with its own vanishing point

• Exception: directions parallel to the image plane

Constructing the vanishing point of a line

image plane

cameracenter

line in the scene

vanishing point


Perspective cues


Vanishing points and lines

Vanishingpoint

Vanishingline

Vanishingpoint

Vertical vanishingpoint

(at infinity)

Slide from Efros, Photo from Criminisi

Vanishing points and lines

Photo from online Tate collection

Note on estimating vanishing points

Use multiple lines for better accuracy

… but lines will not intersect at exactly the same point in practice

One solution: take mean of intersecting pairs

… bad idea!

Instead, minimize angular differences

Vanishing objects

Vanishing lines of planes

Image source: S. Seitz

How do we construct the vanishing line of a plane?

Vanishing lines of planes


cameracenter

plane in the scene

• Horizon: vanishing line of the ground plane

– All points at the same height as the camera project to the

horizon

– Points higher (resp. lower) than the camera project

above (resp. below) the horizon

– Provides way of comparing height of objects

Comparing heights

Vanishing

Point


Measuring height

1

2

3

4

55.4

2.8

3.3

Camera height

What is the height of the camera?


Perspective distortion

• Are the widths of the projected columns equal?• The exterior columns are wider

• This is not an optical illusion, and is not due to lens flaws

• Phenomenon pointed out by Da Vinci

Source: F. Durand


• What is the shape of the projection of a sphere?

Image source: F. Durand


• What is the shape of the projection of a sphere?

Perspective distortion: People

Some Applications

• Various applications of perspective geometry

& vanishing points

• Credits: David Forsyth & Derek Hoiem

Inverse Perspective Transformation

Object Recognition (CVPR 2006)

Reconstruction

Single-view reconstruction (SIGGRAPH 2005)

Secene Geometry Estimation

Getting spatial layout in indoor scenes (ICCV

2009)

Augmented Reality

Inserting synthetic objects into images

http://vimeo.com/28962540

3D Scene Reconstruction

Creating detailed and complete 3D scene models from a

single view (ongoing)

Lane Detection

B. Benligiray, C. Topal, C. Akinlar, Video-based lane detection using a fast vanishing point estimation

method, IEEE Int. Symposium Multimedia (ISM), 2012.

Modeling projection

Projection equation:

Source: J. Ponce, S. Seitz

),(),,(z

yf

z

xfzyx

x

y

z

f

Homogeneous coordinates

Is this a linear transformation?

Trick: add one more coordinate:

homogeneous image

coordinates

homogeneous scene

coordinates

Converting from homogeneous coordinates

• no—division by z is nonlinear


),(),,(z

yf

z

xfzyx

Projection: world coordinates image coordinates

Camera

Center

(tx, ty, tz)

Z

Y

X

P.

.

. f Z

Y

v

up

.

Optical

Center

(u0, v0)

v

u

X

Homogeneous coordinates

Invariant to scaling

Point in Cartesian is ray in Homogeneous

w

y

wx

kw

ky

kwkx

kw

ky

kx

w

y

x

k

Homogeneous Coordinates

Cartesian Coordinates

Basic geometry in homogeneous coordinates

Line equation: ax + by + c = 0

Append 1 to pixel coordinate to get homogeneous

coordinate

Line given by cross product of two points

Intersection of two lines given by cross product of the lines

1

i

i

i v

u

p

jiij ppline

jiij linelineq

i

i

i

i

c

b

a

line

Another problem solved by homogeneous coordinates

Cartesian: (Inf, Inf)

Homogeneous: (1, 1, 0)

Intersection of parallel lines

Cartesian: (Inf, Inf)

Homogeneous: (1, 2, 0)

divide by the third

coordinate

Perspective Projection Matrix

Projection is a matrix multiplication using homogeneous

coordinates

z

yf

xf

z

y

x

f

f

10100

000

000

),(z

yf

z

xf

In practice: lots of coordinate transformations…

World to

camera coord.

trans. matrix

(4x4)

Perspective

projection matrix

(3x4)

Camera to

pixel coord.

trans. matrix

(3x3)

=2D

point

(3x1)

3D

point

(4x1)

X0IKx

10100

000

000

1z

y

x

f

f

v

u

w

K

Slide Credit: Saverese

Projection matrix

Intrinsic Assumptions

• Unit aspect ratio

• Optical center at (0,0)

• No skew

Extrinsic Assumptions• No rotation

• Camera at (0,0,0)

Remove assumption: known optical center

X0IKx

10100

00

00

1

0

0

z

y

x

vf

uf

v

u

w

Intrinsic Assumptions

• Unit aspect ratio

• No skew



Remove assumption: square pixels

X0IKx

10100

00

00

1

0

0

z

y

x

v

u

v

u

w

Intrinsic Assumptions• No skew



Remove assumption: non-skewed pixels

X0IKx

10100

00

0

1

0

0

z

y

x

v

us

v

u

w

Intrinsic Assumptions Extrinsic Assumptions• No rotation


Note: different books use different notation for parameters

Degrees of freedom

XtRKx

1100

0

1 333231

232221

131211

0

0

z

y

x

trrr

trrr

trrr

v

us

v

u

w

z

y

x

5 6

Vanishing Point = Projection from Infinity

R

R

R

z

y

x

z

y

x

z

y

x

KpKRptRKp

0

R

R

R

z

y

x

vf

uf

v

u

w

100

0

0

1

0

0 0uz

fxu

R

R

0vz

fyv

R

R

Orthographic Projection

Special case of perspective projection• Distance from center of projection to image plane is infinite

• Also called “parallel projection”

Image World





• What’s the projection matrix?

Image World


perspective projection - eskisehir.edu.treem.eskisehir.edu.tr/cihant/eem 561/icerik/lec02_ct... ·...

Documents