Computer Vision

Spring 2006 15-385,-685

Instructor: S. Narasimhan

Wean 5403

T-R 3:00pm – 4:20pm

Lecture #13

Announcements

Homework 4 will be out today. Due 4/4/06.Please start early.

Midterm stats: A range 40+, B range 30+

40+ 13 students30+ 9 studentsBelow 30 6 students

Shape from Shading

Lecture #13

Image Intensity and 3D Geometry

• Shading as a cue for shape reconstruction• What is the relation between intensity and shape?

– Reflectance Map

Reflectance Map - RECAP

• Relates image irradiance I(x,y) to surface orientation (p,q) for given source direction and surface reflectance

• Lambertian case: yxI ,

s vni

: source brightness

: surface albedo (reflectance)

: constant (optical system)

k

c

Image irradiance:

sn kckcI i

cos

Let 1kc

then sn iI cos

• Lambertian case qpR

qpqp

qqppI

SS

ssi ,

11

1cos

2222

sn

Reflectance Map(Lambertian)

cone of constant i

Iso-brightness contour

Reflectance Map - RECAP

• Lambertian case

0.1

3.0

0.0

9.08.0

7.0, qpR

p

q

90i 01 SS qqpp

SS qp ,

iso-brightnesscontour

Note: is maximum when qpR , SS qpqp ,,

Reflectance Map - RECAP

Shape from a Single Image?

• Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object?

• Given R(p,q) ( (pS,qS) and surface reflectance) can we determine (p,q) uniquely for each image point?

NO

p

q

Solution

• Take more images– Photometric stereo (previous class)

• Add more constraints– Shape-from-shading (this class)

Photometric Stereo

p

q

11 ,SS

qp

22 ,SS

qp

33 ,SS

qp

• We can write this in matrix form:

Image irradiance:

1kc

11 snI1s

n

v

2s

22 snI3s

33 snI

n

s

s

s

T

T

T

I

I

I

3

2

2

2

1 1

Lambertian case:

sn

ikcI cos

Photometric Stereo

Solution

• Take more images– Photometric stereo (previous class)

• Add more constraints– Shape-from-shading (this class)

Human Perception

by V. Ramachandran

• Our brain often perceives shape from shading. • Mostly, it makes many assumptions to do so.

• For example:

Light is coming from above (sun).

Biased by occluding contours.

See Ramachandran’s work on Shape

from Shading by Humans

http://psy.ucsd.edu/chip/ramabio.html

Stereographic Projection

y

z

x

1z

q

p

1

s n

NS

s n

1

1z

g

f

1n

s

(p,q)-space (gradient space)

y

z

x

(f,g)-space

Problem (p,q) can be infinite when 90

2211

2

qp

pf

2211

2

qp

qg

Redefine reflectance map as gfR ,

Occluding Boundaries

ne

ve

n

venvnen , e and are knownv

The values on the occluding boundary can be used as the boundary condition for shape-from-shading

n

Image Irradiance Constraint

• Image irradiance should match the reflectance map

dxdygfRyxIei

2

image

,,

Minimize

(minimize errors in image irradiance in the image)

Smoothness Constraint

• Used to constrain shape-from-shading

• Relates orientations (f,g) of neighboring surface points

y

gg

x

gg

y

ff

x

ff yxyx

,,,

gf , : surface orientation under stereographic projection

es fx2 fy

2 gx2 gy

2 image

dxdy

Minimize

(penalize rapid changes in surface orientation f and g over the image)

Shape-from-Shading

• Find surface orientations (f,g) at all image points that minimize

is eee

smoothnessconstraint

weight

image irradianceerror

dxdygfRyxIggffe yxyx

2

image

2222 ,, Minimize

Numerical Shape-from-Shading

• Smoothness error at image point (i,j)

si, j 1

4f i1, j f i, j 2

f i, j1 f i, j 2 gi1, j gi, j 2

gi, j1 gi, j 2 Of course you can consider more neighbors (smoother results)

• Image irradiance error at image point (i,j)

ri, j Ii, j R f i, j ,gi, j 2

Find and that minimize

f i, j

gi, j

e si, j ri, j j

i

(Ikeuchi & Horn 89)

Find and that minimize

f i, j

gi, j

e si, j ri, j j

i

If and minimize , then lkf , lkg , 0,0,,

lklk g

e

f

ee

0,22,

,,,,,,

lkf

lklklklklklk f

RgfRIff

f

e

0,22,

,,,,,,

lkg

lklklklklklk g

RgfRIgg

g

e

g k,l 1

8gi1, j gi, j1 gi 1, j gi, j 1

f k,l 1

8f i1, j f i, j1 f i 1, j f i, j 1

where and are 4-neighbors average around image point (k,l)lkf , lkg ,

Numerical Shape-from-Shading

(Ikeuchi & Horn 89)

• Use known values on the occluding boundary to constrain the solution (boundary conditions)

• Compare with for convergence test

• As the solution converges, increase to remove the smoothness constraint

0,22,

,,,,,,

lkf

lklklklklklk f

RgfRIff

f

e 0,22,

,,,,,,

lkg

lklklklklklk g

RgfRIgg

g

e

lk

lklk

f

lklklknn

f

RgfRIff

,

,, ,,,1 ,

lk

lklk

g

lklklknn

g

RgfRIgg

,

,, ,,,1 ,

Update rule

gf ,

1,

1, , n

lknlk gf n

lknlk gf ,, ,

Numerical Shape-from-Shading

(Ikeuchi & Horn 89)

Calculus of Variations

e F f ,g, fx, fy,gx,gy image

dxdy

F fx2 fy

2 gx2 gy

2 I x,y R f ,g 2

Minimize

0,0

yxyx gggfff F

yF

xFF

yF

xF

Euler equations for F

g

RgfRyxIg

f

RgfRyxIf

,,,,, 22 Euler equations for shape-from-shading

Solve this coupled pair of second-order partial differential equationswith the occluding boundary conditions!

(read Horn A.6)

Results

Results

Next Two Classes

• Binocular Stereo

• Reading: Horn, Chapter 13.