freddy.cs.technion.ac.il...title: a direct differential approach to photometric stereo with...

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

SIAM J. IMAGING SCIENCES c© 2014 Society for Industrial and Applied MathematicsVol. 7, No. 2, pp. 579–612

A Direct Differential Approach to Photometric Stereo with Perspective Viewing∗

Roberto Mecca†, Ariel Tankus‡, Aaron Wetzler§, and Alfred M. Bruckstein§

Abstract. Shape from shading and photometric stereo are two fundamental problems in computer vision aimedat reconstructing surface depth given either a single image taken under a known light source or mul-tiple images taken under different illuminations from the same viewing angle. Whereas the formeruses partial differential equation techniques to solve the image irradiance equation, the latter can beexpressed as a linear system of equations in surface derivatives when three or more images are given.Therefore, it seems that current photometric stereo techniques do not extract all possible depthinformation from each image by itself. Extending our previous results on this problem, we considerthe more realistic perspective projection of surfaces during the photographic process. Under thisassumption, there is a unique weak solution (Lipschitz continuous) to the problem at hand, solvingthe well-known convex/concave ambiguity of the shape from shading problem. The main contribu-tion of this paper is based on a new differential approach for multi-image photometric stereo. Mostof the existing works on this topic do not directly address this problem. The common approach is toestimate the gradient field of the surface by minimizing some functional and integrate it afterwardsto find the depth and hence the geometry of the object. Our new differential approach allows usto solve the problem directly, while dealing with images having missing parts. The mathematicalwell-posedness of the new formulation allows a fast numerical algorithm based on a combination offast marching and fast sweeping methods.

Key words. perspective shape from shading, photometric stereo, nonlinear PDE system, unique weak Lipschitzsolution, finite difference upwind schemes, semi-Lagrangian schemes

AMS subject classifications. 68T45, 35A02, 65N21, 65N12, 65M25

DOI. 10.1137/120902458

1. Introduction. Reconstruction of three-dimensional (3D) surfaces from images is oneof the most fundamental problems in computer vision. Two reconstruction approaches, bothdating back to the 1970s, are shape from shading [15, 17, 6] and photometric stereo [41, 42].Shape from shading is aimed at solving the image irradiance equation, which relates thereflectance map to image intensity. Photometric stereo is a monocular 3D shape reconstructionmethod based on several images of a scene taken from an identical viewpoint under differentillumination conditions. The most common approach in the field divides the task into twosteps: first, recovery of surface gradients, and later, integration of the resultant gradient fieldto determine the 3D surface itself. The goal of the first part is to solve a system of image

∗Received by the editors December 13, 2012; accepted for publication (in revised form) January 6, 2014; publishedelectronically April 1, 2014. This research was partly supported by Broadcom Foundation.

http://www.siam.org/journals/siims/7-2/90245.html†Faculty of Computer Science, Technion University, Haifa 32000, Israel ([email protected]), and Pattern

Analysis and Computer Vision, Istituto Italiano di Tecnologia, 16163 Genova, Italy.‡Department of Neurosurgery and Department of Neurology, Tel Aviv Sourasky Medical Center, Tel Aviv 64239,

Israel ([email protected]).§Faculty of Computer Science, Technion University, Haifa 32000, Israel ([email protected], freddy@cs.

technion.ac.il).

579

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

http://www.siam.org/journals/siims/7-2/90245.html

mailto:[email protected]






580 R. MECCA, A. TANKUS, A. WETZLER, AND A. M. BRUCKSTEIN

irradiance equations. When given three or more images, this system becomes linear. As such,the gradient field can be recovered analytically. For this reason, shape from shading andphotometric stereo have very diverse methodologies, even though the latter is a generalizationof the former.

Recent research in the field of shape from shading has focused on advanced propagationand partial differential equation (PDE) methods. Among these are the method of level sets ofBruckstein [6] and Kimmel and Bruckstein [17], the Hamilton–Jacobi-based method of Rouyand Tourin [30], later extended by Prados, Faugeras, and Rouy [28], a control theory–basedalgorithm of Oliensis [25], and a finite element method by Falcone and Sagona [12]. A veryefficient single-pass algorithm is the fast marching method [18]; see [11] for a recent survey.

A more recent development in the field of shape from shading is the transition from theassumption of an orthographic projection of the photographed surface onto the image planeto an assumption of a perspective projection [39, 36, 37, 38, 27, 29].

Photometric stereo research has focused on surface reconstruction from three or moreimages (see [1] for a review). We denote the problem with n images by PSn. Conditions onthe illumination and surface reflectance required to obtain uniqueness of solution for three-light-source photometric stereo are described by Okatani and Deguchi [24]. Even when thelight source intensity and directions are unknown, Shashua [32] has shown that three or moreimages provide enough information to determine the scaled surface normals of an object up toan unknown linear transformation, which allows the reconstruction of the surface also underunknown lighting conditions (assuming distant light sources) [3].

Most of the studies which have addressed the shape from photometric stereo problemusing multiple images employ a two-step approach [34]:

1. estimation of the first derivatives of the surface (usually via some minimization algo-rithms);

2. recovery of the height from the gradient field (see, e.g., [10, 9, 2]) by integration or byfunctional minimization.

With the aim of computing the height of the surface directly, several variational approacheshave been studied using the Euler–Lagrange equations associated to functionals that involvethe given images and iterate based on some prior information on the surface. Such PDEs haveseveral drawbacks; most important are the high order of the partial derivatives involved andthe impossibility of having solution with regularity less than a C2 function [16].

Due to the analytic solution for PS3, only a few studies have investigated two-imagephotometric stereo, PS2 (for example, [26, 22, 20, 40]). A comprehensive work on existence anduniqueness in PS2 is that of Kozera [19], which retraces the study of Onn and Bruckstein [26].In PS2, the authors of [26, 21] showed that it is impossible to recover the gradient field locallyeven if the albedo is known, since an ambiguity remains. The local ambiguity is shown to beremovable in most cases by global integrability constraints. Mecca and Falcone [22] extendedthese works, proving a uniqueness result for weak (Lipschitz continuous) solutions in theabsence of shadows. In particular, the proof is based on the complete knowledge of boundarycondition. This is not a reasonable assumption as this information is not directly available inthe pictures of the object. They also proposed two approximation schemes for the numericalsolution of this problem: an upwind finite difference scheme and a semi-Lagrangian scheme.

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php


PERSPECTIVE AND MULTIPLE-IMAGE PHOTOMETRIC STEREO 581

Tankus and Kiryati [35] changed the common orthographic projection assumption andfar away light sources in photometric stereo to a perspective one (similar to Tankus, Sochen,and Yeshurun [39] in shape from shading). By doing that, they found an analytic linearsolution for the gradient field of a three-image perspective photometric stereo problem. Yoon,Prados, and Sturm [43] employed a variational framework in their perspective photometricstereo algorithm and demonstrated it using a large set of input images (≥ 16).

Whereas orthographic PS2 has been investigated for extracting more information fromeach equation using shape from shading techniques, and three-image perspective photometricstereo has an analytical solution for the gradient field, no information is available on thePS2 problem under the perspective projection model. The goal of this research is to usenumerical schemes commonly used in the shape from shading realm [5] also for PS2 under theperspective projection assumption, thus extracting additional information from each givenimage. We write the “photometric PDEs” with the same form for both the perspective andthe orthographic cases, thus “merging” the mathematical treatment of the two. We prove auniqueness result for weak (Lipschitz continuous) solutions under the perspective projectionmodel and propose two numerical approximation schemes: an upwind finite difference schemeand a semi-Lagrangian scheme. Based on the unified PDEs, the convergence of the twonumerical schemes can be easily deduced from [21]. This paper thus extends and combinesthree research directions, those by Mecca and Falcone [22], Tankus and Kiryati [35], and Onnand Bruckstein [26].

In this paper, we employ the well-posed differential formulation by Mecca and Falcone [22]for the PS3 problem with missing data (e.g., due to shadows), which adds further complexityto surface recovery. Let us emphasize that our formulation does not use geometry informationdue to shadows [8]. Our new approach uses the new PS3 formulation in regions of the imagewhere all the data is available (i.e., no shadows) and the PS2 problem in regions with missingparts. We thus formulate a new way of recovering the shape by orienting a characteristic fieldin order to avoid the supplementary need of depth information as boundary condition andbypass the additional problem of missing parts in the images. We prove that this new way of“manipulating” the characteristic field easily handles the case where images contain occlusions.A second advantage is that this new technique uses the fast marching method, so convergencecan theoretically be reached in just one iteration (in practice, the algorithm stops after veryfew iterations). We prove that such characteristic-field manipulation is well-posed.

Although our hypotheses are slightly weaker than those assumed in [14], which addressedthe same problem, we consider a two-step procedure and regularization terms for smoothsurfaces.

The paper is organized as follows. Section 2 introduces the perspective in the new pho-tometric stereo model: after the description of notation and assumptions, we formulate thenew differential model for the photometric stereo problem (section 2.1). We then prove theuniqueness of weak solution for the new differential model (section 2.2). Section 3 is ded-icated to the multi-image photometric stereo differential model: using an on/off weightedapproach (section 3.1), offering interesting remarks about this new approach (section 3.2),and studying the stability perturbed by noise in the images (section 3.3). After recalling thesemi-Lagrangian and upwind schemes used in this paper (section 4), we describe how to usethem for the surface recovery without taking into account boundary conditions (section 5).D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



ξ

η

(x, y, z)

(ξ, η,−f)

x

y

z

Figure 1. Schematic representation of a surface taken under perspective viewing. The point of the realsurface (x, y, z) is projected in the perspective domain in the point (ξ, η) of the focal plane (in blue), parallel tothe optical plane (in red) at a focal distance f .

Finally, we conclude with numerical tests on synthetic and real data (section 6). Concludingremarks appear in section 7.

2. Perspective photometric stereo: Notation and assumptions. The main ingredientsfor the formulation of the model for the perspective shape from shading for a Lambertiansurface presented in [39] are the following:

• The light source is given by a unit vector ω = (ω1, ω2, ω3) (with ω3 < 0).• The surface in the real world is given by the analytical function h(x, y) = (x, y, z(x, y))(where the point (x, y) is in the image domain Ω = Ω ∪ ∂Ω on the optical plane).

• The associated perspective surface is given by the function k(ξ, η) = (ξ, η, z(ξ, η)),where the point (ξ, η) is in the perspective image domain Ω

p= Ωp ∪ ∂Ωp on the focal

plane (shown in blue in Figure 1), parallel to the optical plane (in red in Figure 1) ata focal distance f .

• The transformation used to pass from one point in the optical plane (x, y) to therespective point in the focal plane is ξ = − x

z(x,y)f , η = − yz(x,y)f . Then we have

k(ξ, η) = (ξ, η, z(ξ, η)) = (− xz(x,y)f,− y

z(x,y)f, z(x, y)).

Let us assume that the perspective surface z(ξ, η) has no points where the view is occludedby the surface itself; this means that, once we start to consider a surface z(x, y), we have tochoose a focal length f > 0 such that

(2.1) z

(x− t

xf

z(x, y), y − t

yf

z(x, y)

)< z(x, y) +

z(x, y)√x2 + y2

∀t > 0, ∀(x, y) ∈ Ω.

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



However, it is always possible to take the domain Ω small enough to have no occludedpoints.

2.1. The new photometric stereo differential model. In this section we exploit theresults obtained in [22] generalizing the differential model of the orthographic case to theperspective case. We consider the irradiance equation given by the inner product between thelight source ω and the normal vector to the surface k(ξ, η). We have the following PDE in theperspective domain:

(2.2) ρ(ξ, η)−fzξω1 − fzηω2 − (z + ξzξ + ηzη)ω3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2= I(ξ, η) for (ξ, η) ∈ Ωp,

where the unknown ρ(ξ, η) is the albedo which represents the percentage of the light reflectedby the Lambertian object. The reflectance equation (2.2) brings us to the following well-knowndifferential problem (nonlinear PDE + Dirichlet boundary condition) [39]:

(2.3)

⎧⎪⎨⎪⎩ρ(ξ, η)

−zξ(fω1 + ξω3)− zη(fω2 + ηω3)− zω3√f2(z2ξ + z2η) + (z + ξzξ + ηzη)2

= I(ξ, η) on Ωp,

z(ξ, η) = g(ξ, η) on ∂Ωp,

which has no unique solution (even in the viscosity solution sense).

Let us try to overcome the problem of uniqueness of solution by considering the photo-metric stereo approach using two light sources defined by the unit vectors ω′ = (ω′

1, ω′2, ω

′3)

and ω′′ = (ω′′1 , ω

′′2 , ω

′′3 ) (with ω′

3, ω′′3 < 0).

Using the information obtained by both images, we can couple the two equations relatedto the irradiance equation (2.2), obtaining the following system of nonlinear PDEs:

(2.4)

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

ρ(ξ, η)−zξ(fω

′1 + ξω′

3)− zη(fω′2 + ηω′

3)− zω′3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2

= I1(ξ, η) on Ωp,

ρ(ξ, η)−zξ(fω

′′1 + ξω′′

3)− zη(fω′′2 + ηω′′

3 )− zω′′3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2

= I2(ξ, η) on Ωp,

z(ξ, η) = g(ξ, η) on ∂Ωp.

Now, observing that the nonlinearity in the denominator and the albedo of both equationsare the same (i.e., independent of the light sources) and obviously always different from zero,we can merge the nonlinear equation according to the following chain of equalities:

Second Equation︷︸︸︷−zξ(fω

′1 + ξω′

3) − zη(fω′2 + ηω′

3) − zω′3

I1(ξ, η)=

√f2(z2

ξ+ z2η) + (z + ξzξ + ηzη)2

ρ(ξ, η)︸︷︷︸First Equation

=−zξ(fω

′′1 + ξω′′

3 ) − zη(fω′′2 + ηω′′

3 ) − zω′′3

I2(ξ, η).

(2.5)

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Considering only the left- and right-hand sides of (2.5), we obtain the linear equation

(2.6) [(fω′1 + ξω′

3)I2(ξ, η) − (fω′′1 + ξω′′

3)I1(ξ, η)]∂z

∂ξ(ξ, η)

+ [(fω′2 + ηω′

3)I2(ξ, η)− (fω′′2 + ηω′′

3 )I1(ξ, η)]∂z

∂η(ξ, η)

+ (ω′3I2(ξ, η) − ω′′

3I1(ξ, η))z(ξ, η) = 0,

which leads to the linear problem

(2.7)

{b(ξ, η) · ∇z(ξ, η) = s(ξ, η)z(ξ, η) on Ωp,z(ξ, η) = g(ξ, η) on ∂Ωp,

where

b(ξ, η) =((fω′

1 + ξω′3)I2(ξ, η) − (fω′′

1 + ξω′′3)I1(ξ, η),

(fω′2 + ηω′

3)I2(ξ, η) − (fω′′2 + ηω′′

3 )I1(ξ, η))(2.8)

and

(2.9) s(ξ, η) = ω′′3I1(ξ, η) − ω′

3I2(ξ, η).

Note that the functions b(ξ, η) and s(ξ, η) generalize the orthographic case. Using thenotation of [22], the model for the orthographic case is

(2.10)

{borth(x, y) · ∇z(x, y) = sorth(x, y) on Ω,

z(x, y) = gorth(x, y) on ∂Ω,

where

(2.11) borth(x, y) =(ω′1I2(x, y)− ω′′

1I1(x, y), ω′2I2(x, y)− ω′′

2I1(x, y))

and

(2.12) sorth(x, y) = ω′3I2(ξ, η)− ω′′

3I1(ξ, η).

The right-hand sides of the orthographic and perspective equations differ just by sign due tothe different parametrization of the light sources, since ω′

3, ω′′3 < 0 in the perspective case.

The vector field b(ξ, η) has an additional component in the perspective case.

2.2. Uniqueness of weak solution for the new differential model. As stated in the intro-duction, we focus on the regularity of z to allow for weak solutions, e.g., Lipschitz continuoussolutions. Following the same strategy of [22] we shall show that b(ξ, η) guarantees the prop-agation of the information using the characteristic strip expansion. In order to do so we startby proving the following lemma.

Lemma 1. If there are no points (ξ, η) ∈ Ωpof black shadows for the image functions (i.e.,

I1(ξ, η) �= 0 and I2(ξ, η) �= 0), we have that ||b(ξ, η)|| �= 0.Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Proof. Let us prove this result by contradiction considering that there exists a point(ξ, η) ∈ Ω

psuch that

(2.13)

{(fω′

1 + ξω′3)I2(ξ, η)− (fω′′

1 + ξω′′3 )I1(ξ, η) = 0,

(fω′2 + ηω′

3)I2(ξ, η)− (fω′′2 + ηω′′

3 )I1(ξ, η) = 0.

Since we want to consider the dependence of the image functions I1(ξ, η) and I2(ξ, η) onall the other model coefficients, we make these functions explicit using the irradiance equation(2.2), obtaining the following nonlinear system:

(2.14) (fω′1 + ξω′

3)ρ(ξ, η)−zξ(fω

′′1 + ξω′′

3 )− zη(fω′′2 + ηω′′

3)− zω′′3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2

− (fω′′1 + ξω′′


′1 + ξω′

3)− zη(fω′2 + ηω′

3)− zω′3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2= 0,

(2.15) (fω′2 + ηω′


′′1 + ξω′′

3)− zη(fω′′2 + ηω′′

3)− zω′′3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2

− (fω′′2 + ηω′′


′1 + ξω′

3)− zη(fω′2 + ηω′

3)− zω′3√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2= 0.

Noting that the common quantity

(2.16)ρ(ξ, η)√

f2(z2ξ + z2η) + (z + ξzξ + ηzη)2

does not vanish, we can remove it to obtain

(2.17) (fω′′1 + ξω′′

3)[zξ(fω′1 + ξω′

3) + zη(fω′2 + ηω′

3) + zω′3]

− (fω′1 + ξω′

3)[zξ(fω′′1 + ξω′′

3 ) + zη(fω′′2 + ηω′′

3) + zω′′3 ] = 0,

(2.18) (fω′′2 + ηω′′

3 )[zξ(fω′1 + ξω′

3) + zη(fω′2 + ηω′

3) + zω′3]

− (fω′2 + ηω′

3)[zξ(fω′′1 + ξω′′

3 ) + zη(fω′′2 + ηω′′

3) + zω′′3 ] = 0.

This system can be written as

ξ(ω′2ω

′′3 − ω′′

2ω′3)zη + η(ω′′

1ω′3 − ω′

1ω′′3)zη = (ω′

1ω′′3 − ω′′

1ω′3)z + f(ω′

1ω′′2 − ω′′

1ω′2)zη ,(2.19)

ξ(ω′2ω

′′3 − ω′′

2ω′3)zξ + η(ω′′

1ω′3 − ω′

1ω′′3 )zξ = (ω′′

2ω′3 − ω′

2ω′′3)z + f(ω′

1ω′′2 − ω′′

1ω′2)zξ ,(2.20)

and it gives an explicit formula for the first partial derivative as follows:

zη =(ω′

1ω′′3 − ω′′

1ω′3)z

ξ(ω′2ω

′′3 − ω′′

2ω′3) + η(ω′′

1ω′3 − ω′

1ω′′3)− f(ω′

1ω′′2 − ω′′

1ω′2),(2.21)

zξ =(ω′′

2ω′3 − ω′

2ω′′3 )z

ξ(ω′2ω

′′3 − ω′′

2ω′3) + η(ω′′

1ω′3 − ω′

1ω′′3)− f(ω′

1ω′′2 − ω′′

1ω′2).(2.22)

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Δ

z

ω′ ω′′

π′′π′

x

y

Figure 2. The connection between the degenerate case of the linear system made by (2.19) and (2.20) andthe line (Δ) of intersection between the two light planes.

Replacing (2.21) and (2.22) in the irradiance equations of (2.4), we get I1 = I2 = 0, whichis in contradiction with the assumption of absence of shadows. Let us consider the otherparticular cases as follows:

(2.23) Case A ≡⎧⎨⎩

ω′2ω

′′3 − ω′′

2ω′3 = 0,

ω′′1ω

′3 − ω′

1ω′′3 = 0,

ω′1ω

′′2 − ω′′

1ω′2 = 0,

Case B ≡

⎧⎪⎪⎨⎪⎪⎩ω′2ω

′′3 − ω′′

2ω′3 = 0,

ω′′1ω

′3 − ω′

1ω′′3 = 0,

ω′1ω

′′2 − ω′′

1ω′2 �= 0,

zξ = zη = 0.

From a geometric point of view, the configurations leading to these equations are impos-sible. Let us consider the light vectors as generators of the light planes, namely π′ and π′′

which are orthogonal to the respective vectors ω′ and ω′′ (see Figure 2):

(2.24)π′ : ω′

1x+ ω′2y + ω′

3z = 0,

π′′ : ω′′1x+ ω′′

2y + ω′′3z = 0.

Once we define the planes π′ and π′′, we can easily compute the vector of the straight lineΔ generated by the intersection of these two planes simply by the cross-product between ω′

and ω′′:

(2.25) ω′ × ω′′ =

⎛⎜⎝ ω′2ω

′′3 − ω′′

2ω′3

ω′′1ω

′3 − ω′

1ω′′3

ω′1ω

′′2 − ω′′

1ω′2

⎞⎟⎠ =

⎛⎝ lmn

⎞⎠ .

Now, the only possibility of verifying Case A is to choose ω′ ≡ ω′′, which does not respectthe main photometric stereo assumption. If we consider Case B, it is clear that the conditionl = 0, m = 0, and n �= 0 (i.e., the straight line Δ is vertical) can be reached only if ω′

3 = ω′′3 = 0,

which contradicts the assumption that ω′3, ω

′′3 < 0.D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



We will see that Lemma 1 will be useful in two important points in what follows: the firstis for the proof of the uniqueness of solution of the problem, and the second is related to animportant remark about the semi-Lagrangian numerical scheme.

In order to complete the theoretical analysis, we will extend the uniqueness results ofthe differential problem (2.7) for the case of a weak solution as in the orthographic case.Discussion of depth discontinuities and multiple objects is beyond the scope of this paper.Our purpose is to prove the uniqueness of solution of (2.7) in the Lipschitz function space viathe characteristic method. The meaning of weak solution here is intended as a combinationof classical solutions, each defined on a different domain. These domains are then going tobe patched together in such a way that, across the boundaries between domains on whichthere are discontinuities in some derivatives, (2.7) is satisfied. Let us recall that the pointswhere the surface z is nondifferentiable are the same points where the functions b and s arediscontinuous (jump discontinuity) [22]. We assume that the points of discontinuity are givenas a family of regular curves (γ1(t), . . . , γk(t)), where t is the argument of the parametricrepresentation. It is clear that this curve also contains the points of discontinuity of the imagefunctions I1(ξ, η) and I2(ξ, η) (see Figure 3 of Example 1 and Figure 4 of Example 2). Now,since the functions b(ξ, η) and s(ξ, η) depend directly on I1(ξ, η) and I2(ξ, η), the same familyof curves represents the discontinuity also for these coefficients of the PDE in (2.7).

Another property of the discontinuity type of b(ξ, η) and s(ξ, η) is due to the fact that welook for a Lipschitz solution, so the discontinuity must be a jump discontinuity.

Example 1. With this example we show how the point of discontinuity in the imagefunctions can be interpreted as points where the surface is nondifferentiable. Let us simplifythe geometry of the example by considering the following function in dimension one:

(2.26) z(x) =

{3− x, x ≥ 0,3 + 2x, x < 0,

defined in the domain Ω = [a, b] = [−1, 1]. Once we fix the focal length, for example, f = 3(see Figure 3), we can easily compute the perspective domain

(2.27) Ωp = [ap, bp] =

[− b

z(b)f,− a

z(a)f

]=

[− 3

2, 3

].

In this particular case, the perspective function can be computed accurately. Exploitingthe knowledge of z and that z(0) = z(0) = 3 we can compute z(ξ) since it consists of a coupleof straight lines (see Figure 3). In fact,

(2.28) z(ξ) =

{r1 straight line passing through (−3

2 , 2) and (0, 3),

r2 straight line passing through (0, 3) and (3, 1),

that is,

(2.29) z(ξ) = −2

3|ξ|+ 3, ξ ∈

[− 3

2, 3

].

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



z

x

ξ

f

z

Ωp

Ω−1 1

−3

23

ω

ξ

Ωp

−3

23

1

I(ξ)

Figure 3. A schematic example shows that the points where the image function I(ξ) is discontinuous (inthe focal plane) are in one-to-one correspondence with the points of the optical plane for which the solutionz(x) is nondifferentiable. On the left, the Lipschitz “surfaces” in the real world z(x) (in red) and the associatedperspective transformation z (in blue). We emphasize also the two domains Ω and Ωp using the same respectivecolors. On the right is the associated image through the light ω.

If we consider the inner product between the normal field to this function with a nonverticallight vector ω = (ω1, ω2), it is clear that the image function has a jump discontinuity at thesame point where the function z is nondifferentiable. We highlight the coincidence for whichthe discontinuity of the image function and the nondifferentiability of z are at the same point(see Figure 3). This depends only on the fact that this point remains fixed with respect tothe perspective transformation.

Example 2. Now let us consider another example in order to give a better explanationabout the position of the points of nondifferentiability. Let us examine a more complexfunction defined in Ω = [−2, 4] (see Figure 4) as follows:

(2.30) z(x) =

⎧⎨⎩|x+ 1|+ 6, x ∈ [−2, 1],−2x+ 10, x ∈ [1, 3],x+ 1, x ∈ [3, 4].

Let us list the coordinates on the x-axis where the function is nondifferentiable in thisorder: p1 = −1, p2 = 1, and p3 = 3. With respect to this function, we have that theperspective domain (considering a focal length f = 3) is restricted to Ωp =

[ − 125 ,

67

]. The

associated perspective function can be explicitly written as follows:

(2.31) z(ξ) =

⎧⎪⎪⎪⎨⎪⎪⎪⎩−20

3 ξ − 16, ξ ∈ [−125 ,−9

4 ],3215ξ +

13215 , ξ ∈ [−9

4 ,−38 ],

−167 ξ +

507 , ξ ∈ [−3

8 ,12 ],

145 ξ +

235 , ξ ∈ [12 ,

67 ]

(see Figure 4).

Looking at the image function I(ξ) (obtained with light source direction ω, both shownin Figure 4), we can see that the points of discontinuity are shifted with respect to the pointsD

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



z

x

ξ

f

z

Ωp

Ω

ω

−12

5

−2 4

1

I(ξ)

ξΩp−12

5−9

4

6

76

7

1

2−3

8

p1 p2 p3

pp1pp2pp3

Figure 4. Left: the real (red) and the perspective (blue) Lipschitz functions used to explain how the nondif-ferentiable points for z shift according to the perspective transformation z. It is also possible to note the differentdomains in the respective colors. Right: the image I(ξ) associated to the perspective function z obtained withthe light source ω represented in the left-hand picture.

where the function is nondifferentiable in the real world. In fact, according to the perspectivetransformation the points where the surface z(ξ) is nondifferentiable are now shifted as follows:

(2.32)

p1 → pp1 = −94 ,

p2 → pp2 = −38 ,

p3 → pp3 =12 .

Theorem 2. Let γ(t) be a curve of discontinuity for the function b(ξ, η) (and s(ξ, η)), andlet p = (ξ, η) be a point of this curve. Let n(ξ, η) be the outgoing normal with respect to theset Ωp

+; then we have

(2.33)

⎡⎢⎢⎣ lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

b(ξ, η) · n(ξ, η)

⎤⎥⎥⎦⎡⎢⎢⎣ lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

−

b(ξ, η) · n(ξ, η)

⎤⎥⎥⎦ ≥ 0.

Proof. Let us start by defining the following quantities:

I+1 = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

I1(ξ, η) = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

ρ(ξ, η)−∇z(ξ, η) · (fω′ + (ξ, η)ω′

3)− z(ξ, η)ω′3√

f |∇z(ξ, η)|2 + (z(ξ, η) + (ξ, η) · ∇z(ξ, η))2

= ρ+−∇z+ · (fω′ + pω′

3)− zω′3√

f |∇z+|2 + (z + p · ∇z+)2,

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



I−1 = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

−


−

ρ(ξ, η)−∇z(ξ, η) · (fω′ + (ξ, η)ω′

3)− z(ξ, η)ω′3√

f |∇z(ξ, η)|2 + (z(ξ, η) + (ξ, η) · ∇z(ξ, η))2

= ρ−−∇z− · (fω′ + pω′

3)− zω′3√

f |∇z−|2 + (z + p · ∇z−)2,

I+2 = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+


+

ρ(ξ, η)−∇z(ξ, η) · (fω′′ + (ξ, η)ω′′

3 )− z(ξ, η)ω′′3√

f |∇z(ξ, η)|2 + (z(ξ, η) + (ξ, η) · ∇z(ξ, η))2

= ρ+−∇z+ · (fω′′ + pω′′

3)− zω′′3√

f |∇z+|2 + (z + p · ∇z+)2,

I−2 = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

−


−

ρ(ξ, η)−∇z(ξ, η) · (fω′′ + (ξ, η)ω′′

3 )− z(ξ, η)ω′′3√

f |∇z(ξ, η)|2 + (z(ξ, η) + (ξ, η) · ∇z(ξ, η))2

= ρ−−∇z− · (fω′′ + pω′′

3)− zω′′3√

f |∇z−|2 + (z + p · ∇z−)2,

where z = z(ξ, η) is a well-defined quantity since we assume that the surface z is Lipschitzcontinuous and

ρ+ = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

ρ(ξ, η), ρ− = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

−

ρ(ξ, η),

∇z+ = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

∇z(ξ, η), ∇z− = lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

−

∇z(ξ, η).

Now, replacing the previous limit values for the image functions in the limit definition forb, we have

(b+1 , b+2 ) = lim

(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

b(ξ, η)(2.34)

= ((fω′1 + ξω′

3)I+2 − (fω′′

1 + ξω′′3)I

+1 , (fω′

2 + ηω′3)I

+2 − (fω′′

2 + ηω′′3 )I

+1 ),

(b−1 , b−2 ) = lim


−

b(ξ, η)(2.35)

= ((fω′1 + ξω′

3)I−2 − (fω′′

1 + ξω′′3)I

−1 , (fω′

2 + ηω′3)I

−2 − (fω′′

2 + ηω′′3 )I

−1 ).

If we denote the two components of the normal vector n(ξ, η) by n1 and n2 (both fixedbut dependent on (ξ, η)), we can write (2.33) as follows:

(b+1 n1 + b+2 n2)(b−1 n1 + b−2 n2) ≥ 0,

b+1 b−1 n

21 + b+2 b

−2 n

22+(b+1 b

−2 + b−1 b

+2 )n1n2 ≥ 0.

(2.36)

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Using (2.34) and (2.35) we have((fω′

1 + ξω′3)I

+2 − (fω′′

1 + ξω′′3)I

+1

)((fω′

1 + ξω′3)I

−2 − (fω′′

1 + ξω′′3)I

−1

)n21

+((fω′

2 + ηω′3)I

+2 − (fω′′

2 + ηω′′3)I

+1

)((fω′

2 + ηω′3)I

−2 − (fω′′

2 + ηω′′3)I

−1

)n22

+[((fω′

1 + ξω′3)I

+2 − (fω′′

1 + ξω′′3)I

+1

)((fω′

2 + ηω′3)I

−2 − (fω′′

2 + ηω′′3)I

−1

)+((fω′

1 + ξω′3)I

−2 − (fω′′

1 + ξω′′3)I

−1

)((fω′

2 + ηω′3)I

+2 − (fω′′

2 + ηω′′3)I

+1

)]n1n2 ≥ 0,

(2.37)

and splitting the computation into three parts with respect to the coefficients of n21, n

22, and

n1n2, we have the following three results, respectively:

(2.38) I+1 I−1(fω′′

1 + ω′′3ξ)2

+ I+2 I−2(fω′

1 + ω′3ξ)2

− (I+1 I−2 + I−1 I+2 )(f2ω′

1ω′′1 + fω′′

1ω′3ξ + fω′

1ω′′3ξ + ω′

3ω′′3ξ

2)= I+1 I−1

(f2(ω′′

1 )2 + 2fω′′

1ω′′3ξ + (ω′′

3 )2ξ

2)+ I+2 I−2

(f2(ω′

1)2 + 2fω′

1ω′3ξ + (ω′

3)2ξ

2)−I+1 I−2

(f2ω′

1ω′′1+fω′′

1ω′3ξ+fω′

1ω′′3ξ+ω′

3ω′′3ξ

2)−I−1 I+2(f2ω′

1ω′′1+fω′′

1ω′3ξ+fω′

1ω′′3ξ+ω′

3ω′′3ξ

2),

(2.39) I+1 I−1(fω′′

2 + ω′′3η)2

+ I+2 I−2(fω′

2 + ω′3η)2

− (I+1 I−2 + I−1 I+2 )(f2ω′

2ω′′2 + fω′′

2ω′3η + fω′

2ω′′3η + ω′

3ω′′3η

2)

= I+1 I−1(f2(ω′′

2 )2 + 2fω′′

2ω′′3η + (ω′′

3)2η2

)+ I+2 I−2

(f2(ω′

2)2 + 2fω′

2ω′3η + (ω′

3)2η2

)−I+1 I−2

(f2ω′

1ω′′2+fω′′

2ω′3η+fω′

2ω′′3η+ω′

3ω′′3η

2)−I−1 I+2

(f2ω′

2ω′′1+fω′′

2ω′3η+fω′

2ω′′3η+ω′

3ω′′3η

2),

(2.40) 2I+1 I−1(f2ω′′

1ω′′2 + fω′′

1ω′′3η + fω′′

2ω′′3ξ + (ω′′

3 )2ξη

)+ 2I+2 I−2

(f2ω′

1ω′2 + fω′

1ω′3η + fω′

2ω′3ξ + (ω′

3)2ξη

)− (I+1 I−2 + I−1 I+2 )

(f2ω′′

1ω′2 + f2ω′

1ω′′2 + fω′′

1ω′3η + fω′

1ω′′3η + fω′′

2ω′3ξ + fω′

2ω′′3ξ + 2ω′

3ω′′3ξη

).

The image functions I+1 I−1 , I+1 I−2 , I−1 I+2 , and I+2 I−2 appearing in (2.38), (2.39), and (2.40)have the common quantity

(2.41)ρ+ρ−√

f2|∇z+|2 + (z + p · ∇z+)2√

f2|∇z−|2 + (z + p · ∇z−)2

that can be eliminated since it is always strictly positive. Only the following quantities canthen be considered:

(2.42)i+1 = −∇z+ · (fω′ + (ξ, η)ω′

3)− zω′3, i−1 = −∇z− · (fω′ + (ξ, η)ω′

3)− zω′3,

i+2 = −∇z+ · (fω′′ + (ξ, η)ω′′3)− zω′′

3 , i−2 = −∇z− · (fω′′ + (ξ, η)ω′′3)− zω′′

3 .

This allows us to write the grayscale value of one image with respect to the grayscale value ofthe same image but taken on the other side, that is,

(2.43) i+1 = i−1 + λ · (fω′ + (ξ, η)ω′3), i+2 = i−2 + λ · (fω′′ + (ξ, η)ω′′

3),Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



where

(2.44) λ = (∇z− −∇z+) = (λ1, λ2) =

(∂z−

∂x− ∂z+

∂x,∂z−

∂y− ∂z+

∂y

).

Now we substitute these quantities in (2.38), (2.39), and (2.40), obtaining the following threecoefficients for n2

1, n22, and n1n2:

(2.45) (i−1 + λ · (fω′ + (ξ, η)ω′3))i

−1

(f2(ω′′

1 )2 + 2fω′′

1ω′′3ξ + (ω′′

3 )2ξ

2)+ (i−2 + λ · (fω′′ + (ξ, η)ω′′

3 ))i−2

(f2(ω′

1)2 + 2fω′

1ω′3ξ + (ω′

3)2ξ

2)− (i−1 + λ · (fω′ + (ξ, η)ω′

3))i−2

(f2ω′

1ω′′1 + fω′′

1ω′3ξ + fω′

1ω′′3ξ + ω′

3ω′′3ξ

2)− i−1 (i

−2 + λ · (fω′′ + (ξ, η)ω′′

3 ))(f2ω′

1ω′′1 + fω′′

1ω′3ξ + fω′

1ω′′3ξ + ω′

3ω′′3ξ

2),

(2.46) (i−1 + λ · (fω′ + (ξ, η)ω′3))i

−1

(f2(ω′′

2 )2 + 2fω′′

2ω′′3η + (ω′′

3)2η2

)+ (i−2 + λ · (fω′′ + (ξ, η)ω′′

3))i−2

(f2(ω′

2)2 + 2fω′

2ω′3η + (ω′

3)2η2

)− (i−1 + λ · (fω′ + (ξ, η)ω′

3))i−2

(f2ω′

1ω′′2 + fω′′

2ω′3η + fω′

2ω′′3η + ω′

3ω′′3η

2)

− i−1 (i−2 + λ · (fω′′ + (ξ, η)ω′′

3))(f2ω′

2ω′′1 + fω′′

2ω′3η + fω′

2ω′′3η + ω′

3ω′′3η

2),

(2.47) 2(i−1 + λ · (fω′ + (ξ, η)ω′3))i

−1

(f2ω′′

1ω′′2 + fω′′

1ω′′3η + fω′′

2ω′′3ξ + (ω′′

3)2ξη

)+ 2(i−2 + λ · (fω′′ + (ξ, η)ω′′

3 ))i−2

(f2ω′

1ω′2 + fω′

1ω′3η + fω′

2ω′3ξ + (ω′

3)2ξη

)− [(i−1 + λ · (fω′ + (ξ, η)ω′

3))i−2 + i−1 (i

−2 + λ · (fω′′ + (ξ, η)ω′′

3 ))](f2ω′′

1ω′2 + f2ω′

1ω′′2 + fω′′

1ω′3η + fω′

1ω′′3η + fω′′

2ω′3ξ + fω′

2ω′′3ξ + 2ω′

3ω′′3ξη

).

After some computation we have

(2.48)

(2.45)n21 + (2.46)n2

2 + (2.47)n1n2

=[f(i−2 (n1ω

′1 + n2ω

′2)− i−1 (n1ω

′′1 + n2ω

′′2))+ (i−2 ω

′3 − i−1 ω

′′3)(n1ξ + n2η)

][− f2(n1λ2 − n2λ1)(ω′′1ω

′2 − ω′

1ω′′2) + (i−2 ω

′3 − i1−ω′′

3)(n1ξ + n2η)

+f(i−2 (n1ω

′1 + n2ω

′2)− i−2 (n1ω

′′1 + n2ω

′′2)

−(n1λ2 − n2λ1)(ηω′′1ω

′3 − ηω′

1ω′′3 − ξω′′

2ω′3 + ξω′

2ω′′3 ))]

∗=[f(i−2 (n1ω

′1 + n2ω

′2)− i−1 (n1ω

′′1 + n2ω

′′2))+ (i−2 ω

′3 − i−1 ω

′′3 )(n1ξ + n2η)

]2 ≥ 0,

where in ∗ we are using that

(2.49) (n1λ2 − n2λ1) ≡ 0,

exploiting the continuity of the surface z. In fact, using (2.44), we can write the previousDow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



γ(t) γ(t) γ(t) γ(t)

Ωp+

Ωp+ Ωp

+ Ωp+

Ωp−

Ωp− Ωp

− Ωp−

(ξ, η) (ξ, η) (ξ, η) (ξ, η)

Figure 5. All the possible behaviors of the characteristic field close to the discontinuity curve γ(t). Theonly admissible cases (that permit information to travel along the characteristic curves) are the first two (fromthe left).

equivalence as follows:

(2.50) n1λ2 − n2λ1 = n1

(∂z−

∂η− ∂z+

∂η

)− n2

(∂z−∂ξ

− ∂z+

∂ξ

)= −∂z−

∂ξn2 +

∂z−

∂ηn1 −

(− ∂z+

∂ξn2 +

∂z+

∂ηn1

)= lim


−

∇(−n2,n1)z(ξ, η) − lim(ξ,η)→(ξ,η)(ξ,η)∈Ωp

+

∇(−n2,n1)z(ξ, η) =∂z−

∂t(ξ, η)− ∂z+

∂t(ξ, η) = 0,

where in the last passage we have used the orthogonality of (−n2, n1) and the normal vector(n1, n2) of the discontinuity curve γ(t). This means that we shall consider the gradient in

the tangential direction with respect to γ(t) for both sides Ωp+ and Ωp

−, that is,∂z+

∂t and ∂z−∂t .

Now, since we are assuming continuity for surface z, these two derivatives are equal for all thepoints (ξ, η).

Theorem 2 explains the behavior of b(ξ, η) near a curve of discontinuity γ. Summarizing,it says that this curve is not a barrier for the vector field (see Figure 5). We will use Theorem 2in the next theorem showing that the information can travel the complete domain withoutobstacle using the characteristic strip expansion method.

In order to simplify the analysis of the numerical schemes we use in this paper, we modifythe differential problem (2.7), dividing the differential equation by z(ξ, η) and considering thebijective mapping ln(z(ξ, η)) = u(ξ, η). We finally get the following problem:

(2.51)

{b(ξ, η) · ∇u(ξ, η) = s(ξ, η) on Ωp,

u(ξ, η) = eg(ξ,η) on ∂Ωp.

The substitution makes the differential problem equivalent to the orthographic problem [22].Theorem 3. Let us consider the problem

(2.52)

{b(ξ, η) · ∇u(ξ, η) = s(ξ, η) a.e. (ξ, η) ∈ Ωp,

u(ξ, η) = eg(ξ,η) ∀(ξ, η) ∈ ∂Ωp,

where the functions b(ξ, η) and s(ξ, η) are defined in (2.8) and (2.9), respectively.Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Let us suppose that (γ1(t), . . . , γk(t)), the family of discontinuity curves for b(ξ, η) ands(ξ, η), are not characteristic curves (with respect to the previous problem). Then there existsa unique Lipschitz solution of the problem.

We omit the proof because Lemma 1 and Theorem 2 allow us to build the unique solutionby following the same logical steps of the orthographic case [22], that is, proving that theordinary differential equation (ODE) system

(2.53)

⎧⎨⎩(a) u(r) = −s(ξc(r), ηc(r)),

(b) ξc(r) = b1(ξc(r), ηc(r)),(c) ηc(r) = b2(ξc(r), ηc(r)),

where r is the parameter of the projected characteristic (ξc(r), ηc(r)), with initial condition⎧⎨⎩(a0) u(ξc(0), ηc(0)) = eg(ξ0,η0),(b0) ξc(0) = ξ0,(c0) ηc(0) = η0

with (ξ0, η0) ∈ ∂Ωp, has a unique solution.

3. Direct surface reconstruction using multiple images. When using multiple lightsources we have an additional requirement on the lighting directions. A further assumptionneeds to be made when more than two images are taken into account. By considering the re-flectance equation for Lambertian surfaces as a linear function with respect to the light sourcevector, we are constrained to consider only noncoplanar light sources. This inconveniencehas been studied in [23] with respect to PS3. It is well known that an image obtained withlinearly dependent light sources with respect to the other images does not add any additionalinformation if shadows do not occur [31, 4]. In the following sections, we will focus our studyon the orthographic case, which can be easily extended to the perspective case by repeatingsimilar steps. In the following subsection, we denote the unknown by z even if consideringorthographic viewing geometry. The extension to the perspective case follows trivially fromanalogous steps.

3.1. Weighted photometric stereo for multiple images with missing parts. If we havethree linearly independent images obtained by individually shining three parallel-ray lightsources in noncoplanar directions, then we can consider the set of unique image pairs andhave the system of linear PDEs

(3.1)

⎧⎪⎨⎪⎩b(1,2)(x, y) · ∇z(x, y) = s(1,2)(x, y),

b(1,3)(x, y) · ∇z(x, y) = s(1,3)(x, y),

b(2,3)(x, y) · ∇z(x, y) = s(2,3)(x, y),

of the same type as (2.10), where

(3.2) b(h,k)(x, y) = (Ik(x, y)ωh1 − Ih(x, y)ω

k1 , Ik(x, y)ω

h2 − Ih(x, y)ω

k2 )

and

(3.3) s(h,k)(x, y) = Ik(x, y)ωh3 − Ih(x, y)ω

k3D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



with (h, k) being the combination of two of the first three natural integers without repetitions.We can now describe our novel contribution, which is to ensure the well-posedness of

the PSn problem by exploiting the linearity of the basic differential formulation (2.10) andreducing it to a single PDE which can handle shadowed regions in a natural fashion. Since(2.10) does not lose the well-posedness if we multiply the equations by a function q(x, y) onboth sides (i.e., b(x, y) and f(x, y)), we are able to define the ingredients of a weighted PSn(W-PSn) model by considering the functions

(3.4) bwn (x, y) =∑

p∈( [n]2)

qp(x, y)bp(x, y)

and

(3.5) swn (x, y) =∑

p∈( [n]2)

qp(x, y)sp(x, y),

where ( [n]2 ) is the set of pairs of integer indices with no repetition. For example, if n = 3, we

have ( [3]2 ) = {(1, 2), (1, 3), (2, 3)}.The complete construction of the W-PSn formulation is therefore

(3.6)

{bwn (x, y) · ∇z(x, y) = swn (x, y) a.e. (x, y) ∈ Ω,z(x, y) = g(x, y) ∀(x, y) ∈ ∂Ω.

Next, we explain how shadows influence the definition of the weights (hence of bwn and swn ).A key observation we can make is that it is possible to use weight functions qp that are

vanishing while preserving the well-posedness of the problem. Obviously the main importanceis related to the set of points where the functions qp are null, not to the signs of qp.

Let us observe that the well-posedness of the differential formulation is guaranteed forimage pixels lit in at least two images and is preserved if the same condition holds in themulti-image, weighted case.

Since we want to exploit the photometric stereo technique, we assume that each pixel isilluminated in at least two images, thereby avoiding reduction to a PS1 problem. Our aim isto consider the weights as switches, able to locally nullify the involvement of an image pairin the summations (3.4) and (3.5) when the functions bp and sp for that pair do not containrelevant information due to the presence of shadows in the images involved. Since no ambientlight is assumed in our setup, we consider the point (x, y) ∈ Ω shadowed in the ith imagewhen Ii(x, y) = 0. Now, by using the Heaviside function we can easily define the weights asfollows:

(3.7) q(h,k) = H(Ih(x, y))H(Ik(x, y)).

These functions are not continuous; therefore, to not complicate (3.6) by adding suchdiscontinuous functions, we consider a regularization of (3.7). By assuming the shadows to beopen sets, we can regularize the weights using cutoff functions as constructed in [13].

The uniqueness for the problem (3.6) can be shown following the same steps used for thetwo-image case. That is, the following results hold:D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



1. from Lemma 1, the absence of a critical point for the projected characteristic field, i.e.,bw3 (x, y) �= (0, 0);

2. from Theorem 2, the propagation of the information from the boundary is not pre-vented between two sets separated by discontinuity.

3.2. Some remarks on the uniqueness of solution. Let us emphasize the following sym-metry property of the previous equations:

(3.8) b(h,k)(x, y) · ∇z(x, y) = f (h,k)(x, y) ⇔ b(k,h)(x, y) · ∇z(x, y) = f (k,h)(x, y)

due to the structure of b(h,k)(x, y) and f (h,k)(x, y), obeying

(3.9) b(h,k)(x, y) = −b(k,h)(x, y) and f (h,k)(x, y) = −f (k,h)(x, y) ∀(x, y) ∈ Ω.

In this section we want to focus our attention on some issues regarding the sum of thethree linear PDEs in (3.1). Let us sum them as follows:

(3.10)(b(1,2) − b(1,3) + b(2,3)

) · ∇z(x, y) =(f (1,2) − f (1,3) + f (2,3)

),

which is the same as

(3.11)(b(1,2) + b(3,1) + b(2,3)

) · ∇z(x, y) =(f (1,2) + f (3,1) + f (2,3)

)simply using (3.8) and (3.9).

If we write explicitly the sum of the functions involved in the previous equations we obtain

(3.12) b(1,2) + b(3,1) + b(2,3) = ω′(I3 − I2) + ω′′(I1 − I3) + ω′′′(I2 − I1)

and

(3.13) f (1,2) + f (3,1) + f (2,3) = ω′3(I3 − I2) + ω′′

3(I1 − I3) + ω′′′3 (I2 − I1).

Since every term is multiplied by the difference between one image and another, it is clearthat both functions vanish simultaneously at all the points (x, y) ∈ Ω for which

(3.14) I1(x, y) = I2(x, y) = I3(x, y).

Obviously in this case the uniqueness result is compromised since Lemma 1 does not hold.We now show that condition (3.14) can be a concrete problem since the points (x, y) can beexplicitly characterized in some illumination configurations. Let us consider a particular setupfor the direction of the light sources. It is based on some optimality conditions studied in [33],where the light sources are taken in the same rings. In other words, comparing with our lightsources definition, we have to take the unit vectors ω′, ω′′, and ω′′′ such that

(3.15) ω′3 = ω′′

3 = ω′′′3 .D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



Now the special points (x, y) can easily be found depending on the surface z. In fact, thesystem

I3(x, y)− I2(x, y) =−∇z(x, y) · ω′′′ + ω′′′

3√1 + ||∇z||2 − −∇z(x, y) · ω′′ + ω′′

3√1 + ||∇z||2

=−∇z(x, y) · ω′′′ +∇z(x, y) · ω′′√

1 + ||∇z||2 = 0,

(3.16)

I1(x, y)− I3(x, y) =−∇z(x, y) · ω′ + ω′

3√1 + ||∇z||2 − −∇z(x, y) · ω′′′ + ω′′′

3√1 + ||∇z||2

=−∇z(x, y) · ω′ +∇z(x, y) · ω′′′√

1 + ||∇z||2 = 0,

(3.17)

I2(x, y)− I1(x, y) =−∇z(x, y) · ω′′ + ω′′

3√1 + ||∇z||2 − −∇z(x, y) · ω′ + ω′

3√1 + ||∇z||2

=−∇z(x, y) · ω′′ +∇z(x, y) · ω′√

1 + ||∇z||2 = 0

(3.18)

holds if and only if ∇z(x, y) = 0 since ω′ �= ω′′ �= ω′′′ is the main assumption for the photomet-ric stereo technique. This means that the differential equation (3.10) (together with the sameboundary condition of (3.1)) does not have a unique smooth solution if it admits stationarypoints.

It is clear that, in fact, the set of the points where(b(1,2) − b(1,3) + b(2,3)

)(or

(f (1,2) −

f (1,3)+ f (2,3))) vanishes can also be used to find the stationary points of the unknown surface

z. In this section, we obtained results comparable with those of [7], and merging both theorieswould be possible for future works.

3.3. Numerical stability in the presence of images with noise. As seen in the previoussection, the way to solve the differential problem requires one to propagate the informationstarting from the boundary condition. Exploiting the characteristic strip expansion, it ispossible to give a constructive proof of the existence of a unique Lipschitz solution. Since weconsider numerical schemes that mimic the characteristic method, it is important to study theconvergence analysis for a general one-step numerical scheme solving the characteristic ODEsystem perturbed for a noisy image. Let us then consider the following Cauchy problem (i.e.,characteristic problem):

(3.19)

⎧⎪⎨⎪⎩(a) xc(s) = bw,1

3 (xc(s), yc(s)),

(b) yc(s) = bw,23 (xc(s), yc(s)),

(c) ˙z(s) = sw3 (xc(s), yc(s)),

where s is the variable of the parameterization and z(s) is the value of the surface on theprojected characteristic (that is, z(s) = z(xc(s), yc(s))).D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



The initial condition required to integrate the previous system of ODEs is taken from thevalues of the function z known on the boundary

(3.20) Γin =

{(x, y) ∈ ∂Ω : ν(x, y) · lim

(x,y)→(x,y)(x,y)∈Ω

bw3 (x, y) ≤ 0

},

where ν(x, y) represents the outgoing normal to the boundary ∂Ω. It is clear that, in theprevious definition, the limit is taken, because a point of discontinuity may belong to theboundary. Then ⎧⎨⎩

(a0) xc(0) = x0,(b0) yc(0) = y0,(c0) z(xc(0), yc(0)) = g(x0, y0)

with (x0, y0) ∈ ∂Ω.Let us consider bw3 , s

w3 perturbed as follows:

bw,13 = q(1,2)

((I2 + η2)ω

′1 − (I1 + η1)ω

′′1

)+ q(1,3)

((I3 + η3)ω

′1 − (I1 + η1)ω

′′′1

)+ q(2,3)

((I3 + η3)ω

′′1 − (I2 + η2)ω

′′′1

),

bw,23 = q(1,2)

((I2 + η2)ω

′2 − (I1 + η1)ω

′′2

)+ q(1,3)

((I3 + η3)ω

′2 − (I1 + η1)ω

′′′2

)+ q(2,3)

((I3 + η3)ω

′′2 − (I2 + η2)ω

′′′2

),

sw3 = q(1,2)((I2 + η2)ω

′3 − (I1 + η1)ω

′′3

)+ q(1,3)

((I3 + η3)ω

′3 − (I1 + η1)ω

′′′3

)+ q(2,3)

((I3 + η3)ω

′′3 − (I2 + η2)ω

′′′3

),

(3.21)

where η1, η2, and η3 are the additive noise at point (xc(s), yc(s)).For simplicity, let us study the three lighted point situation, which is where q(1,2) = q(1,3) =

q(2,3) = 1. Exploiting the linearity of the images (i.e., of the noise) in the previous quantities,we have

bw,13 = bw,1

3 − η1(ω′′1 + ω′′′

1 ) + η2(ω′1 − ω′′′

1 ) + η3(ω′1 + ω′′

1),

bw,23 = bw,2

3 − η1(ω′′2 + ω′′′

2 ) + η2(ω′2 − ω′′′

2 ) + η3(ω′2 + ω′′

2),

sw3 = sw3 − η1(ω′′3 + ω′′′

3 ) + η2(ω′3 − ω′′′

3 ) + η3(ω′3 + ω′′

3 );

(3.22)

that is, the additive noise of each image brings us to consider an additive perturbation of(2.53). Rewriting (3.19) in a compact form, we have

(3.23)

{Z(s) = k(s) + η,

Z(0) = g(x0, y0) + η0,

where Z and k take values in R3 and η0 is the error on the boundary condition. Let us

emphasize that, since (3.19) admits a unique solution, we can assume k to be a Lipschitzfunction. Let us recall that the strategy for proving the uniqueness of a weak solution is toconsider the characteristic method from the boundary till the first curve of discontinuity γ(t)and to start again with another boundary problem. Of course, the Lipschitz property of khas to be taken into account locally, excluding the points of discontinuity for k.D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



Let us focus our attention on a general way of writing a one-step numerical scheme whichsolves (3.23):

(3.24)

{un+1 = un +ΔΦ(sn, un, k(sn),Δ) + ηn,u0 = g(x0, y0) + η0,

where Δ is the uniform discretization step and Φ is a general function of discretization for thefirst derivative. As usual, un = u(sn) is the approximation of the solution Zn = Z(sn), andηn = η(sn) is the perturbation at point sn. We want to provide an estimation of the error atpoint sn+1, that is,

(3.25) en+1 = Zn+1 − un+1 = (Zn+1 − u∗n+1) + (u∗n+1 − un+1),

where u∗n+1 is the error computing the value of the approximation starting from correct data(see Figure 6).

Zn+1

Zn

un

un+1

u∗n+1

Δτ(Δ)en+1

rn rn+1

Z(r)

ζn

Figure 6. Geometric interpretation of the local and global errors in sn+1 in a perturbed configuration.

We have

(3.26) Zn+1 − u∗n+1 = Δτ(Δ),

which represents the consistency error of the method plus the perturbation in sn+1.The other part of the right-hand side of (3.25) becomes

u∗n+1 − un+1 = Zn +ΔΦ(sn, Zn, k(sn),Δ)− un −ΔΦ(sn, un, k(sn),Δ)− ηn

= en +Δ[Φ(sn, Zn, k(sn),Δ)− Φ(sn, un, k(sn),Δ)

]− ηn;(3.27)

that is, we have

||en+1|| ≤ Δ||τ(Δ)||+ ||ηn||+ ||en||+Δ||Φ(sn, Zn, k(sn),Δ)− Φ(sn, un, k(sn),Δ)||≤ Δ||τ(Δ)||+ ||ηn||+ ||en||+ΔL||Zn − un|| = Δ||τ(Δ)||+ ||ηn||+ (1 + ΔL)||en||,

(3.28)

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



exploiting the local Lipschitzianity of Φ (i.e., dependence on the numerical schemes) inducedby the k local regularity. For example, if Φ represents the Euler discretization, then L isexactly the Lipschitz constant of k.

Repeating the same reasoning for ||en|| and so on, we have the following chain:(3.29)

||en+1|| ≤ Δ||τ(Δ)||[1 + (1 + ΔL) + (1 + ΔL)2 + · · ·+ (1 + ΔL)n]

+ ||ηn||+ (1 + ΔL)||ηn−1||+ (1 +ΔL)2||ηn−2||+ · · · + (1 + ΔL)n−1||η1||+ (1 + ΔL)n||η0||≤ [

Δ||τ(Δ)|| +Θ](1 + (1 + ΔL) + (1 + ΔL)2 + · · ·+ (1 + ΔL)n)

= (1+ΔL)n+1−1hL

[Δ||τ(Δ)|| +Θ

],

where Θ = max0≤i≤n ||ηi||. Now, since 1 + ΔL ≤ eΔL and (n + 1)Δ = sn+1 − s0, we cancontinue the estimation as follows:

||en+1|| ≤ eL(sn+1−s0) − 1

hL

[Δ||τ(Δ)|| +Θ

](3.30)

≤ eL(sn+1−s0) 1

L

(||τ(Δ)||+ Θ

Δ

)− 1

ΔL

[Δ||τ(Δ)|| +Θ

]≤ beL(sn+1−s0) 1

L

(||τ(Δ)||+ Θ

Δ

).

It follows that we cannot have the convergence when Δ → 0 because of the presence of thenoise. Of course, there exists an optimal step for which the global error is minimum. Underthis value the noise effects become dominant in the truncation error.

4. Numerical schemes. Next, we consider the numerical methods approximating thesolution on the image domain restricted to the set [−1, 1]2 with a discretization space stepΔx and Δy [22]. We use these methods because they converge very fast and are easy toimplement.

4.1. Forward numerical schemes. Now we want to recall the numerical schemes used forthe forward approximation of (2.7) where the propagation of the information starts from theinflow part of the boundary Γin. We will use the orthographic variables (x, y), but the formu-lation can be easily reinterpreted for the perspective case just by using (ξ, η). Furthermore,simplify the notation by denoting b(xi, yj) by bi,j = (b1i,j , b

2i,j) and s(xi, yj) by si,j.

Forward upwind scheme:

(4.1) RFi,j =

Δy|b1i,j|RFi−sgn(b1i,j),j

+Δx|b2i,j|RFi,j−sgn(b2i,j)

+ΔxΔysi,j

|b1i,j|Δy + |b2i,j|Δx.

Since the numerical schemes are applied to digital images, clearly Δx = Δy = Δ.Forward semi-Lagrangian scheme:

(4.2) rFi,j = rF (xi − hα1i,j, yj − hα2

i,j) +si,j|bi,j|h,

where αi,j =bi,j|bi,j | and the parameter h > 0 is assumed equal to the size of the grid Δ in order

to reach the highest order of convergence equal to 1 [22].Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



4.2. Backward numerical schemes. The backward numerical schemes are based on ap-proximation of the surface by propagating the information stored on the outflow part of theboundary

(4.3) Γout = ∂Ω \ Γin.

Backward upwind scheme:

(4.4) RBi,j =

Δy|b1i,j|RBi+sgn(b1i,j),j

+Δx|b2i,j|RBi,j+sgn(b2i,j)

−ΔxΔysi,j

|b1i,j|Δy + |b2i,j|Δx.

Backward semi-Lagrangian scheme:

(4.5) rBi,j = rB(xi + hα1i,j, yj + hα2

i,j)−si,j|bi,j|h.

5. W-PSn with no boundary condition. In sections 2 and 3 we extended the photometricstereo model by assuming knowledge of the boundary condition g(x, y). Clearly such a hypoth-esis compromises the use of that model in most real applications. It is therefore important tofind a way to solve the PSn problem without requiring knowledge of the boundary condition.

To solve this problem we design a numerical strategy which involves selecting a singlearbitrarily valued initial seed point within the reconstruction domain and robustly manipu-lating the path of the characteristics. We do this in order to numerically integrate the lineardifferential problem (3.6) so as to let the information travel in the most convenient directionsfor the whole domain.

5.1. Controlling the characteristic field. On the way to defining a numerical strategywe will need to manipulate the path along which the information travels. To do this we willemploy the following result.

Theorem 4. Let bp(x, y) be the vector field of (3.2), where p ∈ ( [n]2 ). Then, ∀p1, p2 ∈ ( [n]2 )and ∀(x, y) ∈ Ω we have

(5.1) bp1(x, y) · bp2(x, y) �= ±|bp1(x, y)||bp2(x, y)|.Proof. For simplicity, let us fix the indices p1 and p2 as (1, 2) and (1, 3), respectively. In

order to prove by contradiction that b(1,2) and b(1,3) are never parallel, we assume that thereexists a point (x, y) ∈ Ω such that

(5.2) b(1,2)(x, y) · b(1,3)(x, y) = ±|b(1,2)(x, y)||b(1,3)(x, y)|.For the sake of clarity we omit the dependence on (x, y). Now, by squaring both sides we have

(5.3)[b(1,2)1 b

(1,3)1 + b

(1,2)2 b

(1,3)2

]2=[(b(1,2)1

)2+(b(1,2)2

)2][(b(1,3)1

)2+(b(1,3)2

)2],

and by writing b(1,2) and b(1,3) explicitly from (3.2) we get

(5.4)[I2I3(ω

′1)

2− I1I2ω′1ω

′′′1 − I1I3ω

′1ω

′′1 + I21ω

′′1ω

′′′1 + I2I3(ω

′2)

2− I1I2ω′2ω

′′′2 − I1I3ω

′2ω

′′2 + I21ω

′′2ω

′′′2

]2=[(I2ω

′1 − I1ω

′′1)

2 + (I2ω′2 − I1ω

′′2)

2][(I3ω

′1 − I1ω

′′′1 )

2 + (I3ω′2 − I1ω

′′′2 )

2].

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Now, let us write the reflectance function by simplifying the notation as

(5.5) Ij = ρ(x, y)ij(x, y)√

1 + |∇z(x, y)|2 , j = 1, 2, 3,

and substituting it into (5.4). We note that the quantity

(5.6)ρ(x, y)√

1 + |∇z(x, y)|2

is nonvanishing and therefore can be eliminated from both sides of (5.4). Finally we can write(5.4) as

(5.7)[i2i3(ω

′1)

2 − i1i2ω′1ω

′′′1 − i1i3ω

′1ω

′′1 + i21ω

′′1ω

′′′1 + i2i3(ω

′2)

2 − i1i2ω′2ω

′′′2 − i1i3ω

′2ω

′′2 + i21ω

′′2ω

′′′2

]2=[(i2ω

′1 − i1ω

′′1)

2 + (i2ω′2 − i1ω

′′2)

2][(i3ω

′1 − i1ω

′′′1 )

2 + (i3ω′2 − i1ω

′′′2 )

2],

and after some algebraic manipulation we get

(5.8) i1(ω′′′1 ω

′′2ω

′3 − ω′′

1ω′′′2 ω

′3 − ω′′′

1 ω′2ω

′′3 + ω′

1ω′′′2 ω

′′3 + ω′′

1ω′2ω

′′′3 − ω′

1ω′′2ω

′′′3 ) = 0.

Assuming that we are considering a nonshadowed point for the first image (i.e., i1 > 0), wehave that (5.8) is satisfied only if the light sources are collinear since (5.8) is equivalent to

(5.9) det

⎛⎝ω′1 ω′

2 ω′3

ω′′1 ω′′

2 ω′′3

ω′′′1 ω′′′

2 ω′′′3

⎞⎠ = 0,

which is in contradiction with the photometric stereo assumption.In other words, this theorem says that two different vector fields bp1 and bp2 cannot be

parallel. We will use this fact to control the direction of the characteristics for the case whenn = 3, which can be easily generalized to any number of images. Let us introduce the set ofthree-lighted pixels M through its indicator function as follows:

(5.10) �M(x, y) = H(I1(x, y))H(I2(x, y))H(I3(x, y)).

We would like to control the summation functions qp in (3.4) and (3.5) such that we cantake a linear weighted combination as given by the following two indexes:

(5.11) (p∗1, p∗2) = argmin

(p1,p2)∈( [3]2 )

|bp1 · bp2 |

with the aim of spanning the set of all possible directions of the derivatives by using the twoleast ill-conditioned directions bp

∗1 and bp

∗2 . This is permitted due to the linearity of the basic

differential formulation (2.10).Since qp(x, y) ≡ 1, ∀(x, y) ∈ M and ∀p ∈ ( [3]2 ), we can sum the equations in (3.1) as

follows:

(5.12)(αbp

∗1 + βbp

∗2) · ∇z = αfp∗1 + βfp∗2 ,D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



(1

0

)

(0

1

)(1

1

)

(1

−1

)(

0

−1

)(−1

−1

)

(−1

0

)

(−1

1

)

Figure 7. The main directions for the integration strategy on an axis-aligned grid.

where α and β are real coefficients.We can now fully control the direction in which we compute the first derivatives of z

independently at any pixel, provided that no shadows are involved. We will choose (α, β) inorder to control the characteristic direction at a pixel to be in the most favorable direction asrequired by some integration strategy. On an axis-aligned discretization grid we define eightprimary directions of integration (Figure 7).

We recognize these as the integration directions resulting from the possible locations acces-sible by the numerical schemes derived in the next section. Once we have chosen a particulardirection d at a point (x, y) ∈ M, we can compute values for (α, β) so that

(5.13)

(bp∗11 (x, y) b

p∗21 (x, y)

bp∗12 (x, y) b

p∗22 (x, y)

)(αβ

)=

(d1(x, y)d2(x, y)

).

Provided that the light sources used to produce the three images are noncoplanar, Theo-rem 4 guarantees that the vectors bp

∗1 and bp

∗2 are not parallel, and thus this 2 × 2 system is

guaranteed to have a unique solution. Furthermore, by virtue of (5.11) we have ensured thatit is well-conditioned. Thus for each point (x, y) ∈ M we can compute coefficients (α, β) suchthat

(5.14)(αbp

∗1 + βbp

∗2) · ∇z = αfp∗1 + βfp∗2 ,

which is now a pixel specific version of (3.6).

5.2. Integration strategy for W-PS3. We aim to minimize the accumulation of errorduring the numerical integration and therefore define a strategy for recovering the surfacewithout the use of a priori boundary conditions.

We have the following steps inspired by the wavefront expansion principle of fast marchingmethods over flat domains:

1. Fix an arbitrary value of z for a point toward the center of the image domain not inshadow, namely (x, y) (see the orange point in Figure 8) and add all of that point’sneighbors to a list of pixels to be visited.

2. Traverse the list of pixels to be visited and update the value of z for each one bycalculating p∗1, p∗2, α, and β after determining what information is available as requiredby the forward scheme (4.1) or (4.2) derived in the next section.D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



Figure 8. We fix an arbitrary point (in orange) and perform surface recovery along an expanding wavefrontof visited pixels. When shadows occur, we orient the vector field in order to avoid them when their directionsare inconsistent with the direction of expansion of the wavefront.

3. For each newly visited pixel add its unvisited neighbors to the list of pixels to bevisited.

4. In the case of shadows we can change the wavefront propagation direction in order tosurround the shadow sets (i.e., by computing the boundary condition) as shown bythe red arrows in Figure 8 and then solve the appropriate equation in (3.1) in the setof shadowed pixels when the wavefront expansion direction is in agreement with theshadowed pixel’s characteristic direction.

5. The above steps are repeated until some stopping condition on convergence is fulfilled.

The main idea that we are proposing is that for all nonshadow pixels we can orient thecharacteristic field so that its direction is convenient for use in our integration strategy. Thisis only possible using our new formulation in (3.6). In this way the advancing wavefrontconstantly makes new pixels available to be updated even when they are in the shadow setbecause at some point the direction of advance of the wavefront will be consistent with thedirection of a pixel’s characteristic which sits along the border of the shadow set. This pixelis then added to the set of pixels to be visited and in turn will act as a seed pixel that enablesother points in the shadow set to be updated in the next incremental advance of the wavefront.In such a way the entire set of pixels can be visited and correctly updated.

6. Numerical tests. This section describes the experiments conducted with the proposednumerical schemes: the semi-Lagrangian and the upwind finite difference scheme, each in itsforward and backward formulations.

For the numerical tests we utilized two surfaces, each with different geometrical and ana-lytical characteristics. The surface (Figure 9 left)

zreg(x, y) = 10 + 0.1 sin(x2 + y2)− e−((3(x−0.2))2+(3(y+0.2))2)

+ e−((3(x+1.6))2+(3(y−2.1))2) + e−((3(x+0.9))2+(3(y+1.1))2)

(z(x, y) ∈ [9.0110, 11.088]) is a regular surface with three extremes that guarantee some veryDow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



zreg(x, y) zirreg(x, y)

Figure 9. The surfaces used for the numerical tests, each with different geometrical and analytical properties.

steep slopes. The surface (Figure 9 right)

zirreg(x, y) = 10− ∣∣0.4 sin(x2 + y2)− e−((3(x−0.2))2+(3(y+0.2))2)

+ e−((3(x+1.3))2+(3(y−1.4))2) + e−((3(x+0.9))2+(3(y+1.1))2)∣∣

(z(x, y) ∈ [8.635, 10]) has some curves of nondifferentiability. In particular, its boundary isnondifferentiable everywhere.

For each of these surfaces, we computed the perspective image under two light sourcedirections according to the procedure described by Tankus, Sochen, and Yeshurun [39] (Figure10). We used a constant focal length f = 1 for all images. To demonstrate reconstructionunder a nonconstant, discontinuous albedo function, we artificially generated dark stripes inthe images (see Figure 10). These stripes are of albedo ρ = 0.5, whereas ρ = 1 in all otherparts of the test images. We repeated the experiments with images of several sizes: 100× 100,200 × 200, 400× 400, and 800× 800 pixels.

Reconstruction by the suggested perspective semi-Lagrangian and upwind schemes ishighly accurate, as demonstrated in Figure 11. We see that the suggested perspective methodsfaithfully reconstructed the surfaces, despite the irregularities.

Let us first show that the numerical schemes converge despite the irregularities and exam-ine their rates of convergence. Tables 1 and 2 examine the convergence rate of the numericalschemes using the L∞ norm in the perspective coordinate system (ξ, η, z(ξ, η)). Indeed, bothperspective schemes converge linearly (i.e., with order 1), because doubling the number of gridnodes halves the error. Thus, the theoretical convergence can be obtained in practice, and ina relatively short run time.

Having demonstrated convergence, let us now explore the accuracy of surface reconstruc-tion. This is quantified by the root mean square error (RMSE). We compute the RMSE inboth the perspective coordinate system (ξ, η, z(ξ, η)) and the real-world coordinate system(x, y, z(x, y)). The RMSE measures have the same units as surface depth and can thereforebe directly compared to the aforementioned ranges of z(x, y) values. As the original depthvalues are around 10 and reconstruction error is generally less than 0.1 (and sometimes eventwo orders of magnitude less than this), the mean error is less than 1% in these examples,generally speaking.

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



z reg

z irreg

ω′ : ϕ1 = 0.1, θ1 = 0.0 → (I1) ω′′ : ϕ2 = 0.1, θ2 = 3π4

→ (I2)

Figure 10. Perspective images of the respective surfaces of Figure 9, used as inputs to the algorithms. Thelight source directions appear on the bottom: ω = (sin(ϕ) cos(θ), sin(ϕ) sin(θ), cos(ϕ)). The albedo on the darkstripe of each image is ρ = 0.5; otherwise, ρ = 1.

zreg zirreg

Figure 11. Reconstruction by the proposed perspective backward semi-Lagrangian scheme, using the inputimages of Figure 10. The surfaces are flipped compared to the original ones (Figure 9) because of the perspectiveprojection. The original surfaces were faithfully recovered.

Table 1Convergence and accuracy of the forward numerical schemes for each surface of Figure 9. For each surface

we examined images of size Δ ×Δ pixels and computed three error measures: the L∞ norm in the perspectivecoordinate system, RMSE in the perspective system, and RMSE in the real-world coordinate system. The L∞

norm shows that convergence is linear (i.e., order 1). The RMSE measures quantify the accurate reconstructionwith respect to the original surface.

Semilag forward Upwind forwardΔ L∞ MSE-persp MSE-real L∞ MSE-persp MSE-real

z reg

100 7.582 × 10−1 0.079965 0.08006 6.780 × 10−1 0.073121 0.073157200 3.543 × 10−1 0.048369 0.04849 3.245 × 10−1 0.046625 0.046775400 1.733 × 10−1 0.027498 0.027577 1.631 × 10−1 0.02839 0.028495800 8.567 × 10−2 0.014932 0.014977 8.121 × 10−2 0.016325 0.016389

z irreg

100 6.726 × 10−1 0.11174 0.10957 4.693 × 10−1 0.11507 0.11299200 4.977 × 10−1 0.067081 0.068095 3.925 × 10−1 0.078578 0.080503400 3.381 × 10−1 0.037985 0.039874 2.664 × 10−1 0.051528 0.054701800 2.174 × 10−1 0.020888 0.02254 1.590 × 10−1 0.035682 0.038287

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Table 2Convergence and accuracy of the backward numerical schemes for each surface of Figure 9. The table is

organized similarly to Table 1. The rate of convergence of the backward algorithm is the same as that of theforward schemes (order 1). Accurate reconstruction is also achieved by the backward schemes.

Semilag backward Upwind backwardΔ L∞ MSE-persp MSE-real L∞ MSE-persp MSE-real

z reg

100 7.582 × 10−1 0.024478 0.025474 2.399 × 10−1 0.0094924 0.011658200 1.789 × 10−1 0.013978 0.014514 9.918 × 10−2 0.0061304 0.007485400 8.494 × 10−2 0.007591 0.0078847 4.662 × 10−2 0.003698 0.0045026800 4.113 × 10−2 0.003997 0.0041618 2.279 × 10−2 0.0021912 0.0026803

z irreg

100 2.929 × 10−1 0.040556 0.042861 1.673 × 10−1 0.029009 0.028976200 2.952 × 10−1 0.020974 0.023794 1.025 × 10−1 0.019232 0.019162400 2.014 × 10−1 0.012261 0.014336 8.878 × 10−2 0.015983 0.016061800 1.697 × 10−1 0.0072217 0.0086446 8.930 × 10−2 0.014743 0.015029

I1 I2 I3

Figure 12. Set of synthetic images used in the tests in which the albedo is composed of diagonal stripes.5% Gaussian noise and exaggerated black patches representing shadows are used to corrupt the images.

6.1. Numerical tests for the direct reconstruction. In the first part of the numericaltests we show a numerical analysis of convergence of the numerical schemes introduced in theprevious section. In this synthetic case we take into account a Lipschitz surface that, besidespoints of discontinuity, presents a high slope (i.e., a large Lipschitz constant). The initialimages shown in Figure 12 are synthesized using an analytical function of a surface z in thedomain Ωd = [−1, 1]2.

The images include missing image regions together with a nonconstant albedo mask. Wecorrupt the image data with 5% Gaussian noise to further improve the realism of the simulation.Table 3 displays the error together with the measured run time at convergence. For this workthe parallelism available to an implementation of the integration strategy has not been utilized.Despite this limitation the algorithm runs at 6 frames per second for images of size 500× 500.It should be noted, though, that this is a synthetic case and clearly the presence of noiseinfluences the convergence of the numerical schemes and so they do not preserve the samerate of convergence with respect to the images without noise and thus have longer run times.

Figure 13 shows the surfaces recovered from noisy images with four megapixels. Even ifthe reconstruction accurately preserves the shape of the surface, it produces some artifacts

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Table 3In order to show that the order of convergence of the numerical schemes is 1, we compute the L∞ error

doubling the size of the images (also adding 5% of Gaussian noise) starting from data of 500×500 to 2000×2000pixels.

Upwind Semi-LagrangianΔ L∞ time (sec) L∞ time (sec)

500 3.539 × 10−2 0.162 2.332 × 10−2 0.1711000 2.185 × 10−2 0.561 1.166 × 10−2 0.6232000 1.368 × 10−2 2.783 6.248 × 10−3 3.029

5%

500 6.635 × 10−2 0.161 5.855 × 10−2 0.2741000 3.578 × 10−2 0.553 3.698 × 10−2 1.2012000 3.917 × 10−2 2.684 3.916 × 10−2 4.354

(original) (upwind) (semi-Lagrangian)

Figure 13. All the surfaces shown here are related to the 2000 × 2000 pixel images with 5% of Gaussiannoise.

due to the noise and the rectangular occlusions. In this algorithm no regularization procedurehas been adopted to make the surface smoother.

In the first row of Figure 14 we consider the problem of the reconstruction of the well-known Beethoven bust after complicating the shape recovery by adding black occlusions. Asshown in Figure 15, the computed shape does not have evident artifacts even in zones wherethe information has been removed.

As a last example we consider the reconstruction of a coin using the images in the secondrow of Figure 14. These images have the shadows on different zones around the border of thehuman figure. The reconstruction in Figure 16 shows that the human shape is well computedand, furthermore, that the flat region around this shape has been well preserved.

7. Conclusions. This study utilized numerical schemes commonly employed in the shapefrom shading literature for the PS2 problem under the perspective projection assumption.We proved the uniqueness of the solution in the class of Lipschitz continuous surfaces givenDirichlet boundary conditions. We then extended the two numerical methods of Mecca andFalcone [22], the upwind finite difference scheme and the semi-Lagrangian scheme, for thesolution of the two-image perspective photometric stereo problem. As the method of Meccaand Falcone in [22, 20] can also reconstruct the albedo in a manner similar to the suggestedD

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



I1 I2 I3

Figure 14. Images of the Beethoven bust and coin with occlusions.

Figure 15. Novel views of the reconstruction of Beethoven using W-PS3.

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Figure 16. A novel view of the reconstruction of a coin model using W-PS3.

perspective case, the inaccurate orthographic reconstruction is not due to the nonconstantalbedo, but rather to a result of the unrealistic set of assumptions of a perspective projectionin the proposed algorithms.

REFERENCES

[1] V. Argyriou, M. Petrou, and W. H. Peter, Photometric stereo: An overview, in Advances in Imagingand Electron Physics, Vol. 156, Elsevier, Amsterdam, 2009, pp. 1–54.

[2] S. Barsky and M. Petrou, The 4-source photometric stereo technique for three-dimensional surfaces inthe presence of highlights and shadows, IEEE Trans. Pattern Anal. Mach. Intell., 25 (2003), pp. 1239–1252.

[3] R. Basri, D. Jacobs, and I. Kemelmacher, Photometric stereo with general, unknown lighting, Int. J.Comput. Vis., 72 (2007), pp. 239–257.

[4] P. N. Belhumeur and D. J. Kriegman, What is the set of images of an object under all possible lightingconditions?, in Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition(CVPR ’96), IEEE Computer Society, Washington, DC, 1996, pp. 270–277.

[5] M. Breuß, E. Cristiani, J.-D. Durou, M. Falcone, and O. Vogel, Perspective shape from shading:Ambiguity analysis and numerical approximations, SIAM J. Imaging Sci., 5 (2012), pp. 311–342.

[6] A. M. Bruckstein, On shape from shading, Comput. Vision Graph. Image Process., 44 (1988), pp. 139–154.

[7] M. Chandraker, J. Bai, and R. Ramamoorthi, On differential photometric reconstruction for un-known, isotropic BRDFs, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013), pp. 2941–2955.

[8] M. K. Chandraker, S. Agarwal, and D. J. Kriegman, ShadowCuts: Photometric stereo with shadows,in Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp.1–8.

[9] J.-D. Durou, J.-F. Aujol, and F. Courteille, Integrating the normal field of a surface in the presenceof discontinuities, in Energy Minimization Methods in Computer Vision and Pattern Recognition,Lecture Notes in Comput. Sci. 5681, D. Cremers, Y. Boykov, A. Blake, and F. R. Schmidt, eds.,Springer, Berlin, Heidelberg, 2009, pp. 261–273.

[10] J.-D. Durou and F. Courteille, Integration of a normal field without boundary condition, in Proceed-ings of the 11th IEEE International Conference on Computer Vision, First Workshop on PhotometricAnalysis for Computer Vision, 2007.

[11] J.-D. Durou, M. Falcone, and M. Sagona, Numerical methods for shape-from-shading: A new surveywith benchmarks, Comput. Vis. Image Underst., 109 (2008), pp. 22–43.D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



[12] M. Falcone and M. Sagona, An algorithm for the global solution of the shape-from-shading model, inImage Analysis and Processing, Lecture Notes in Comput. Sci. 1310, A. Del Bimbo, ed., Springer,Berlin, Heidelberg, 1997, pp. 596–603.

[13] K. O. Friedrichs, The identity of weak and strong extensions of differential operators, Trans. Amer.Math. Soc., 55 (1944), pp. 132–151.

[14] C. Hernandez, G. Vogiatzis, and R. Cipolla, Shadows in three-source photometric stereo, in Proceed-ings of the 10th European Conference on Computer Vision, Part I, Springer-Verlag, Berlin, Heidelberg,2008, pp. 290–303.

[15] B. K. P. Horn, Image intensity understanding, Artificial Intelligence, 8 (1977), pp. 201–231.[16] B. K. P. Horn and M. J. Brooks, The variational approach to shape from shading, Comput. Vision

Graph. Image Process., 33 (1986), pp. 174–208.[17] R. Kimmel and A. M. Bruckstein, Tracking level sets by level sets: A method for solving the shape

from shading problem, Comput. Vis. Image Underst., 62 (1995), pp. 47–58.[18] R. Kimmel and J. A. Sethian, Optimal algorithm for shape from shading and path planning, J. Math.

Imaging Vision, 14 (2001), pp. 237–244.[19] R. Kozera, Existence and uniqueness in photometric stereo, Appl. Math. Comput., 44 (1991), pp. 1–103.[20] R. Mecca, Uniqueness for shape from shading via photometric stereo technique, in Proceedings of the

18th IEEE International Conference on Image Processing, B. Macq and P. Schelkens, eds., 2011,pp. 2933–2936.

[21] R. Mecca and J.-D. Durou, Unambiguous photometric stereo using two images, in Proceedings of the16th International Conference on Image Analysis and Processing (Ravenna, Italy), Springer-Verlag,Berlin, Heidelberg, 2011, pp. 286–295.

[22] R. Mecca and M. Falcone, Uniqueness and approximation of a photometric shape-from-shading model,SIAM J. Imaging Sci., 6 (2013), pp. 616–659.

[23] R. Mecca and S. Tozza, Shape reconstruction of symmetric surfaces using photometric stereo, in In-novations for Shape Analysis, Math. Vis., M. Breuß, A. Bruckstein, and P. Maragos, eds., Springer,Heidelberg, 2013, pp. 219–243.

[24] T. Okatani and K. Deguchi, On uniqueness of solutions of the three-light-source photometric stereo:Conditions on illumination configuration and surface reflectance, Comput. Vis. Image Underst., 81(2001), pp. 211–226.

[25] J. Oliensis, Uniqueness in shape from shading, Int. J. Computer Vis., 6 (1991), pp. 75–104.[26] R. Onn and A. M. Bruckstein, Integrability disambiguates surface recovery in two-image photometric

stereo, Int. J. Comput. Vis., 5 (1990), pp. 105–113.[27] E. Prados and O. Faugeras, “Perspective shape from shading” and viscosity solutions, in Proceedings of

the Ninth IEEE International Conference on Computer Vision, IEEE Computer Society, Washington,DC, 2003, pp. 826–831.

[28] E. Prados, O. Faugeras, and E. Rouy, Shape from shading and viscosity solutions, in Proceedings ofthe 7th European Conference on Computer Vision, Vol. II, A. Heyden, G. Sparr, M. Nielsen, and P.Johansen, eds., Copenhagen, Denmark, 2002, pp. 790–804.

[29] E. Prados and S. Soatto, Fast marching method for generic shape from shading, in Variational, Ge-ometric, and Level Set Methods, Lecture Notes in Comput. Sci. 3752, N. Paragios, O. Faugeras,T. Chan, and C. Schnorr, eds., Springer, Berlin, Heidelberg, 2005, pp. 320–331.

[30] E. Rouy and A. Tourin, A viscosity solutions approach to shape-from-shading, SIAM J. Numer. Anal.,29 (1992), pp. 867–884.

[31] A. Shashua, Illumination and view position in 3D visual recognition, in Advances in Neural InformationProcessing Systems 4, Morgan Kaufmann, San Mateo, CA, 1992, pp. 404–411.

[32] A. Shashua, On photometric issues in 3D visual recognition from a single 2D image, Int. J. Comput.Vis., 21 (1997), pp. 99–122.

[33] A. D. Spence and M. J. Chantler, Optimal illumination for three-image photometric stereo usingsensitivity analysis, IEE Proc. Vis. Image Signal Process., 153 (2006), pp. 149–159.

[34] K. Sunkavalli, T. Zickler, and H. Pfister, Visibility subspaces: Uncalibrated photometric stereo withshadows, in Proceedings of the 11th European Conference on Computer Vision: Part II, Springer-Verlag, Berlin, Heidelberg, 2010, pp. 251–264.

[35] A. Tankus and N. Kiryati, Photometric stereo under perspective projection, in Proceedings of the TenthIEEE International Conference on Computer Vision, Beijing, China, 2005, pp. 611–616.D

ownl

oade

d 12

/29/

17 to

132

.68.

36.1

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p



[36] A. Tankus, N. Sochen, and Y. Yeshurun, A new perspective [on] shape-from-shading, in Proceedingsof the Ninth IEEE International Conference on Computer Vision, Vol. 2, Nice, France, 2003, pp. 862–869.

[37] A. Tankus, N. Sochen, and Y. Yeshurun, Perspective shape-from-shading by fast marching, in Proceed-ings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,Vol. I, 2004, pp. 43–49.

[38] A. Tankus, N. Sochen, and Y. Yeshurun, Reconstruction of medical images by perspective shape-from-shading, in Proceedings of the 17th International Conference on Pattern Recognition, Vol. 3,Cambridge, UK, 2004, pp. 778–781.

[39] A. Tankus, N. Sochen, and Y. Yeshurun, Shape-from-Shading under perspective projection, Int. J.Comput. Vis., 63 (2005), pp. 21–43.

[40] L. B. Wolff and E. Angelopoulou, 3-D stereo using photometric ratios, in Computer Vision—ECCV’94, Springer, Berlin, Heidelberg, 1994, pp. 247–258.

[41] R. J. Woodham, Photometric stereo: A reflectance map technique for determining surface orientationfrom a single view, in Image Understanding Systems and Industrial Applications, Proceedings of theSPIE 22nd Annual Technical Symposium, San Diego, CA, 1978, pp. 136–143.

[42] R. J. Woodham, Photometric method for determining surface orientation from multiple images, Opt.Eng., 19 (1980), pp. 139–144.

[43] K.-J. Yoon, E. Prados, and P. Sturm, Generic scene recovery using multiple images, in Proceedingsof the 2nd International Conference on Scale Space and Variational Methods in Computer Vision(SSVM’09), Lecture Notes in Comput. Sci. 5567, Springer-Verlag, Berlin, Heidelberg, 2009, pp. 745–757.

Dow

nloa

ded

12/2

9/17

to 1

32.6

8.36

.165

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

freddy.cs.technion.ac.il...title: a direct differential approach to photometric stereo with...

Documents