flexible depth of field photography

Flexible Depth of Field PhotographySujit Kuthirummal, Member, IEEE, Hajime Nagahara,

Changyin Zhou, Student Member, IEEE, and Shree K. Nayar, Member, IEEE

Abstract—The range of scene depths that appear focused in an image is known as the depth of field (DOF). Conventional cameras

are limited by a fundamental trade-off between depth of field and signal-to-noise ratio (SNR). For a dark scene, the aperture of the lens

must be opened up to maintain SNR, which causes the DOF to reduce. Also, today’s cameras have DOFs that correspond to a single

slab that is perpendicular to the optical axis. In this paper, we present an imaging system that enables one to control the DOF in new

and powerful ways. Our approach is to vary the position and/or orientation of the image detector during the integration time of a single

photograph. Even when the detector motion is very small (tens of microns), a large range of scene depths (several meters) is captured,

both in and out of focus. Our prototype camera uses a micro-actuator to translate the detector along the optical axis during image

integration. Using this device, we demonstrate four applications of flexible DOF. First, we describe extended DOF where a large depth

range is captured with a very wide aperture (low noise) but with nearly depth-independent defocus blur. Deconvolving a captured

image with a single blur kernel gives an image with extended DOF and high SNR. Next, we show the capture of images with

discontinuous DOFs. For instance, near and far objects can be imaged with sharpness, while objects in between are severely blurred.

Third, we show that our camera can capture images with tilted DOFs (Scheimpflug imaging) without tilting the image detector. Finally,

we demonstrate how our camera can be used to realize nonplanar DOFs. We believe flexible DOF imaging can open a new creative

dimension in photography and lead to new capabilities in scientific imaging, vision, and graphics.

Index Terms—Imaging geometry, programmable depth of field, detector motion, depth-independent defocus blur.

Ç

1 DEPTH OF FIELD

THE depth of field (DOF) of an imaging system is therange of scene depths that appear focused in an image.

In virtually all applications of imaging, ranging fromconsumer photography to optical microscopy, it is desirableto control the DOF. Of particular interest is the ability tocapture scenes with very large DOFs. DOF can be increasedby making the aperture smaller. However, this reduces theamount of light received by the detector, resulting in greaterimage noise (lower SNR). This trade-off gets worse withincrease in spatial resolution (decrease in pixel size). Aspixels get smaller, DOF decreases since the defocus bluroccupies a greater number of pixels. At the same time, eachpixel receives less light and, hence, SNR falls as well. Thistrade-off between DOF and SNR is one of the fundamental,long-standing limitations of imaging.

In a conventional camera, for any location of the imagedetector, there is one scene plane—the focal plane—that isperfectly focused. In this paper, we propose varying theposition and/or orientation of the image detector during theintegration time of a photograph. As a result, the focal planeis swept through a volume of the scene, causing all points

within it to come into and go out of focus while the detectorcollects photons.

We demonstrate that such an imaging system enablesone to control the DOF in new and powerful ways:

. Extended Depth of Field: Consider the case where adetector with a global shutter (all pixels are exposedsimultaneously and for the same duration) is movedwith uniform speed during image integration. Then,each scene point is captured under a continuousrange of focus settings, including perfect focus. Weanalyze the resulting defocus blur kernel and showthat it is nearly constant over the range of depths thatthe focal plane sweeps through during detectormotion. Consequently, the captured image can bedeconvolved with a single, known blur kernel torecover an image with significantly greater DOFwithout having to know or determine scene geome-try. This approach is similar in spirit to Hausler’swork in microscopy [1]. He showed that the DOF ofan optical microscope can be enhanced by moving aspecimen of depth range d, a distance 2d along theoptical axis of the microscope, while filming thespecimen. The defocus of the resulting capturedimage is similar over the entire depth range of thespecimen. However, this approach of moving thescene with respect to the imaging system is practicalonly in microscopy and is not suitable for generalscenes. More importantly, Hausler’s derivation as-sumes that defocus blur varies linearly with scenedepth which is true only for imaging systems that aretelecentric on the object side, such as microscopes.

. Discontinuous Depth of Field: A conventionalcamera’s DOF is a single fronto-parallel slab locatedaround the focal plane. We show that by moving aglobal-shutter detector nonuniformly, we can capture

58 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 33, NO. 1, JANUARY 2011

. S. Kuthirummal is with Sarnoff Corporation, W356, 201 WashingtonRoad, Princeton, NJ 08540. E-mail: [email protected].

. H. Nagahara is with the Graduate School of Engineering Science, OsakaUniversity, 1-3, Machikaneyama, Toyonaka, Osaka 560-8531, Japan.E-mail: [email protected].

. C. Zhou and S.K. Nayar are with the Department of Computer Science,Columbia University, 1214 Amsterdam Avenue, MC 0401, New York, NY10027. E-mail: {changyin, nayar}@cs.columbia.edu.

Manuscript received 18 Jan. 2009; revised 28 July 2009; accepted 4 Dec. 2009;published online 1 Mar. 2010.Recommended for acceptance by K. Kutalakos.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log NumberTPAMI-2009-01-0039.Digital Object Identifier no. 10.1109/TPAMI.2010.66.

0162-8828/11/$26.00 � 2011 IEEE Published by the IEEE Computer Society

images that are focused for certain specified scenedepths, but defocused for in-between scene regions.Consider a scene that includes a person in theforeground, a landscape in the background, and adirty window in between the two. By focusing thedetector on the nearby person for some duration andthe faraway landscape for the rest of the integrationtime, we get an image in which both appear fairlywell focused, while the dirty window is blurred outand, hence, optically erased.

. Tilted Depth of Field: Most cameras can only focuson a fronto-parallel plane. An exception is the viewcamera configuration [2], [3], where the imagedetector is tilted with respect to the lens. When thisis done, the focal plane is tilted according to the well-known Scheimpflug condition [4]. We show that byuniformly translating an image detector with a rollingelectronic shutter (different rows are exposed atdifferent time intervals but for the same duration),we emulate a tilted image detector. As a result, wecapture an image with a tilted focal plane and,hence, a tilted DOF.

. Nonplanar Depth of Field: In traditional cameras,the focal surface is a plane. In some applications, itmight be useful to have a curved/nonplanar scenesurface in focus. We show that by nonuniformly (withvarying speed) translating an image detector with arolling shutter, we emulate a nonplanar imagedetector. As a result, we get a nonplanar focalsurface and, hence, a nonplanar DOF.

An important feature of our approach is that the focalplane of the camera can be swept through a large range ofscene depths with a very small translation of the imagedetector. For instance, with a 12.5 mm focal length lens, tosweep the focal plane from a distance of 450 mm from thelens to infinity, the detector has to be translated only about360 microns. Since a detector only weighs a few milligrams,a variety of micro-actuators (solenoids, piezoelectric stacks,ultrasonic transducers, and DC motors) can be used to moveit over the required distance within very short integrationtimes (less than a millisecond if required). Note that suchmicro-actuators are already used in most consumer camerasfor focus and aperture control and for lens stabilization. Wepresent several results that demonstrate the flexibility of oursystem to control DOF in unusual ways.

This is the extended version of a paper that appeared in [5].

2 RELATED WORK

In microscopy, Hausler [1] demonstrated that DOF can beextended by changing the focus during image integration—by moving the specimen. We also propose changing thefocus during image integration, but by moving the imagedetector. We show that for conventional imaging geome-tries, a particular detector motion—constant velocity—enables us to realize extended DOF. As mentioned above,Hausler’s work assumes that defocus blur varies linearlywith scene depth, which is true for imaging systems thatare telecentric on the object side, like microscopes. On theother hand, our approach for conventional (nontelecentric)imaging geometries assumes that defocus blur varieslinearly with the translation of the detector. Note that

though the two approaches are for different imaginggeometries, they make the same underlying assumptionthat defocus blur varies linearly with axial translation of aparticular element—the scene (in Hausler’s work) or thedetector (in ours). While constant velocity detector motionenables extended DOF, we also show how other detectormotions enable different DOF manipulations like discon-tinuous, tilted, and nonplanar DOFs.

A promising approach to extended DOF imaging iswavefront coding, where phase plates placed at theaperture of the lens cause scene objects within a certaindepth range to be defocused in the same way [6], [7], [8].Thus, by deconvolving the captured image with a singleblur kernel, one can obtain an all-focused image. Theeffective DOF is determined by the phase plate used and isfixed. On the other hand, in our system, the DOF can bechosen by controlling the motion of the detector.

Recently, Levin et al. [9] and Veeraraghavan et al. [10]have used masks at the lens aperture to control the propertiesof the defocus blur kernel. From a single captured image,they aim to estimate the structure of the scene and then usethe corresponding depth-dependent blur kernels to decon-volve the image and get an all-focused image. However, theyassume simple layered scenes and their depth recovery is notrobust. In contrast, our approach is not geared toward depthrecovery, but can significantly extend DOF. Also, the masksused in these previous works attenuate some of the lightentering the lens, while our system operates with a clear andwide aperture. All-focused images can also be computedfrom an image captured using integral photography [11],[12], [13]. However, since these cameras make spatioangularresolution trade-offs to capture 4D lightfields in a singleimage, the computed images have much lower spatialresolution when compared to our approach.

A related approach is to capture many images to form afocal stack [14], [15], [16]. An all-in-focus image as well asscene depth can be computed from a focal stack. However,the need to acquire multiple images increases the totalcapture time, making the method suitable for only quasi-static scenes. An alternative is to use very small exposuresfor the individual images. However, in addition to thepractical problems involved in reading out the manyimages quickly, this approach would result in under-exposed and noisy images that are unsuitable for depthrecovery. Recently, Hasinoff and Kutulakos [17] haveproposed a technique to minimize the total capture timeof a focal stack, given a desired exposure level, using acombination of different apertures and focal plane loca-tions. The individual photographs are composited togetherusing a variant of the Photomontage method [18] to create alarge DOF composite. As a by-product, they also get acoarse depth map of the scene. Our approach does notrecover scene depth, but can produce an all-in-focusphotograph from a single, well-exposed image.

There is similar work on moving the detector during imageintegration [19]. However, their focus is on handling motionblur, for which they propose moving the detector perpendi-cular to the optical axis. Some previous works have also variedthe orientation or location of the image detector. Krishnanand Ahuja [3] tilt the detector and capture a panoramic imagesequence, from which they compute an all-focused panoramaand a depth map. For video superresolution, Ben-Ezra et al.

KUTHIRUMMAL ET AL.: FLEXIBLE DEPTH OF FIELD PHOTOGRAPHY 59

[20] capture a video sequence by instantaneously shiftingthe detector within the image plane in between theintegration periods of successive video frames.

Recently, it has been shown that a detector with a rollingshutter can be used to estimate the pose and velocity of afast moving object [21]. We show how a rolling shutterdetector can be used to focus on tilted scene planes as wellas nonplanar scene surfaces.

3 CAMERA WITH PROGRAMMABLE DEPTH OF FIELD

Consider Fig. 1a, where the detector is at a distance v from alens with focal length f and an aperture of diameter a. A scenepoint M is imaged in perfect focus at m if its distance u fromthe lens satisfies the lens law:

1

f¼ 1

uþ 1

v: ð1Þ

As shown in the figure, if the detector is shifted to adistance p from the lens (dotted line), M is imaged as ablurred circle (the circle of confusion) centered around m0.The diameter b of this circle is given by

b ¼ avjðv� pÞj : ð2Þ

The distribution of light energy within the blur circle isreferred to as the point spread function (PSF). The PSF canbe denoted as P ðr; u; pÞ, where r is the distance of an imagepoint from the center m0 of the blur circle. An idealizedmodel for characterizing the PSF is the pillbox function:

P ðr; u; pÞ ¼ 4

�b2�

r

b

� �; ð3Þ

where �ðxÞ is the rectangle function, which has a value 1, ifjxj < 1=2 and 0 otherwise. In the presence of opticalaberrations, the PSF deviates from the pillbox functionand is then often modeled as a Gaussian function:

P ðr; u; pÞ ¼ 2

�ðgbÞ2exp � 2r2

ðgbÞ2

!; ð4Þ

where g is a constant.We now analyze the effect of moving the detector during

an image’s integration time. For simplicity, consider thecase where the detector is translated along the optical axis,

as in Fig. 1b. Let pðtÞ denote the detector’s distance from thelens as a function of time. Then, the aggregate PSF for ascene point at a distance u from the lens, referred to as theintegrated PSF (IPSF), is given by

IP ðr; uÞ ¼Z T

0

P ðr; u; pðtÞÞ dt; ð5Þ

where T is the total integration time. By programming thedetector motion pðtÞ—its starting position, speed, andacceleration—we can change the properties of the resultingIPSF. This corresponds to sweeping the focal plane throughthe scene in different ways. The above analysis onlyconsiders the translation of the detector along the opticalaxis (as implemented in our prototype camera). However,this analysis can be easily extended to more generaldetector motions, where both its position and orientationare varied during image integration.

Fig. 1c shows our flexible DOF camera. It consists of a1=300 Sony CCD (with 1;024� 768 pixels) mounted on aPhysik Instrumente M-111.1DG translation stage. This stagehas a DC motor actuator that can translate the detectorthrough a 15 mm range at a top speed of 2.7 mm/sec and canposition it with an accuracy of 0.05 microns. The translationdirection is along the optical axis of the lens. The CCD shownhas a global shutter and was used to implement extendedDOF and discontinuous DOF. For realizing tilted andnonplanar DOFs, we used a 1=2:500 Micron CMOS detector(with 2;592� 1;944 pixels) which has a rolling shutter.

The table in Fig. 2 shows detector translations (thirdcolumn) required to sweep the focal plane through variousdepth ranges (second column), using lenses with twodifferent focal lengths (first column). As we can see, thedetector has to be moved by very small distances to sweepvery large depth ranges. Using commercially availablemicroactuators, such translations are easily achieved with-in typical image integration times (a few milliseconds to afew seconds).

It must be noted that when the detector is translated, themagnification of the imaging system changes.1 The fourthcolumn of the table in Fig. 2 lists the maximum change in


Fig. 1. (a) A scene point M, at a distance u from the lens, is imaged in perfect focus by a detector at a distance v from the lens. If the detector isshifted to a distance p from the lens, M is imaged as a blurred circle with diameter b centered around m0. (b) Our flexible DOF camera translates thedetector along the optical axis during the integration time of an image. By controlling the starting position, speed, and acceleration of the detector, wecan manipulate the DOF in powerful ways. (c) Our prototype flexible DOF camera.

1. Magnification is defined as the ratio of the distance between the lensand the detector and the distance between the lens and the object. Bytranslating the detector, we are changing the distance between the lens andthe detector, and, hence, changing the magnification of the system duringimage integration.

the image position of a scene point for different translationsof a 1;024� 768 pixel detector. For the detector motions werequire, these changes in magnification are very small. Thisdoes result in the images not being perspectively correct,but the distortions are imperceptible. More importantly, theIPSFs are not significantly affected by such a magnificationchange since a scene point will be in high focus only for asmall fraction of this change and will be highly blurred overthe rest of it. We verify this in the next section.

4 EXTENDED DEPTH OF FIELD (EDOF)

In this section, we show that we can capture scenes withEDOF by translating a detector with a global shutter at aconstant speed during image integration. We first show thatthe IPSF for an EDOF camera is nearly invariant to scenedepth for all depths swept by the focal plane. As a result,we can deconvolve a captured image with the IPSF toobtain an image with EDOF and high SNR.

4.1 Depth Invariance of IPSF

Consider a detector translating along the optical axis withconstant speed s, i.e., pðtÞ ¼ pð0Þ þ st. If we assume that thePSF of the lens can be modeled using the pillbox function in(3), the IPSF in (5) becomes

IP ðr; uÞ ¼ uf

ðu� fÞ�asT�0 þ �T

r� 2�0

bð0Þ �2�TbðT Þ

� �; ð6Þ

where bðtÞ is the blur circle diameter at time t, and �t ¼ 1 ifbðtÞ � 2r and 0 otherwise. If we use the Gaussian function in(4) for the lens PSF, we get

IP ðr; uÞ

¼ uf

ðu� fÞffiffiffiffiffiffi2�p

rasTerfc

rffiffiffi2p

gbð0Þ

!þ erfc

rffiffiffi2p

gbðT Þ

! !:

ð7Þ

Figs. 3a and 3c show 1D profiles of a normal camera’s PSFs forfive scene points with depths between 450 and 2,000 mm froma lens with focal length f ¼ 12:5 mm and f=# ¼ 1:4,computed using (3) and (4) (with g ¼ 1), respectively. In thissimulation, the normal camera was focused at a distance of750 mm. Figs. 3b and 3d show the corresponding IPSFs of anEDOF camera with the same lens,pð0Þ ¼ 12:5 mm, s ¼ 1 mm/sec, and T ¼ 0:36 sec, computed using (6) and (7),respectively. As expected, the normal camera’s PSF varies

dramatically with scene depth. In contrast, the IPSFs of theEDOF camera derived using both pillbox and Gaussian PSFmodels look almost identical for all five scene depths, i.e.,the IPSFs are depth invariant.

The above analysis is valid for a scene point that projectsto the center pixel in the image. For any other scene point,due to varying magnification, the location of its image willchange during image integration (see Fig. 2). However, ascene point will be in high focus for only a short duration ofthis change (contributing to the peak of the PSF), and behighly blurred the rest of the time (contributing to the tail ofthe PSF). As a result, the approximate depth invariance ofthe PSF can be expected to hold over the entire image.

To empirically verify this, we measured a normalcamera’s PSFs and the EDOF camera’s IPSFs for severalscene depths by capturing images of small dots placed atdifferent depths. Both cameras have f ¼ 12:5 mm, f=# ¼ 1:4,and T ¼ 0:36 sec. The detector motion parameters for theEDOF camera are pð0Þ ¼ 12:5 mm and s ¼ 1 mm/sec. Thiscorresponds to sweeping the focal plane from 1 to446.53 mm. Fig. 4a shows the measured PSF at the centerpixel of the normal camera for five scene depths; the camerawas focused at a distance of 750 mm. (Note that the scale ofthe plot in the center row is 50 times that of the other plots.)Fig. 4b shows the IPSFs of the EDOF camera for five differentscene depths and three different image locations. We can seethat, while the normal camera’s PSFs vary widely with scenedepth, the EDOF camera’s IPSFs appear almost invariant toboth scene depth and spatial location. This also validates ourclaim that the small magnification changes that arise due todetector motion do not have a significant impact on the IPSFs.

To quantitatively analyze the depth and space invar-iances of the IPSF, we use the Wiener reconstruction errorwhen an image is blurred with kernel k1 and thendeconvolved with kernel k2. In order to account for the factthat in natural images all frequencies do not have the sameimportance, we weigh this reconstruction error to get thefollowing measure of dissimilarity of two PSFs k1 and k2:


Fig. 2. Translation of the detector required for sweeping the focal planethrough different scene depth ranges. The maximum change in theimage position of a scene point that results from this translation, when a1;024� 768 pixel detector is used, is also shown.

Fig. 3. Simulated (a, c) normal camera PSFs and (b, d) EDOF camera

IPSFs, obtained using pillbox and Gaussian lens PSF models for five

scene depths. Note that the IPSFs are almost invariant to scene depth.

dðk1; k2Þ

¼X!

jK1ð!Þ �K2ð!Þj2

jK1ð!Þj2 þ �þ jK1ð!Þ �K2ð!Þj2

jK2ð!Þj2 þ �

!jF ð!Þj2;

ð8Þ

where Ki is the Fourier transform of ki, ! represents 2Dfrequency, jF j2 is a weighting term that encodes the powerfalloff of Fourier coefficients in natural images [22], and � isa small positive constant that ensures that the denominatorterms are nonzero. Fig. 5a shows a visualization of thepairwise dissimilarity between the normal camera’s PSFsmeasured at the center pixel, for five different scene depths.Fig. 5b shows a similar plot for the EDOF camera’s IPSFsmeasured at the center pixel, while Fig. 5c shows thepairwise dissimilarity of the IPSFs at five different imagelocations but for the same scene depth. These plots furtherillustrate the invariance of an EDOF camera’s IPSF. We haveempirically observed that the approximate invariance holdsreasonably well for the entire range of scene depths sweptby the focal plane during the detector’s motion. However,the invariance is slightly worse for depths that correspond

to roughly 10 percent of the distance traveled by thedetector at both the beginning and end of its motion.

Hausler’s work [1] describes how depth independent blurcan be realized for object-side telecentric imaging systems.The above results demonstrate that changing the focus bymoving the detector at a constant velocity, during imageintegration, yields approximate depth independent blur for


Fig. 4. (a) The measured PSF of a normal camera shown for five different scene depths. Note that the scale of the plot in the center row is 50 timesthat of the other plots. (b) The measured IPSF of our EDOF camera shown for different scene depths (vertical axis) and image locations (horizontalaxis). The EDOF camera’s IPSFs are almost invariant to scene depth and image location.

Fig. 5. (a) Pairwise dissimilarity of a normal camera’s measured PSFs atthe center pixel for five scene depths. The camera was focused at adistance of 750 mm. (b) Pairwise dissimilarity of the EDOF camera’smeasured IPSFs at the center pixel for five scene depths. (c) Pairwisedissimilarity of the EDOF camera’s measured IPSFs at five differentlocations along the center row of the image, for scene points at adistance of 750 mm. (0,0) denotes the center of the image.

conventional imaging geometries. However, it should benoted that Hausler’s analysis is more rigorous—he uses amore accurate model for defocus [23] than ours. Also, byvirtue of the imaging system being object-side telecentric, hisanalysis did not have to model magnification. In ourapproach, though magnification changes during imageintegration, we have not explicitly modeled its effects.

4.2 Computing EDOF Images Using Deconvolution

Since the EDOF camera’s IPSF is invariant to scene depth andimage location, we can deconvolve a captured image with asingle IPSF to get an image with greater DOF. A number oftechniques have been proposed for deconvolution, Richard-son-Lucy and Wiener [24] being two popular ones. For ourresults, we have used the approach of Dabov et al. [25],which combines Wiener deconvolution and block-baseddenoising. In all our experiments, we used the IPSF shown inthe first row and second column of Fig. 4 for deconvolution.

Fig. 6a shows images captured by our EDOF camera.They were captured with a 12.5 mm Fujinon lens with f=1:4and 0.36 second exposures. Notice that the captured imageslook slightly blurry, but high frequencies of all sceneelements are captured. These scenes span a depth range ofapproximately 450 to 2,000 mm—10 times larger than theDOF of a normal camera with identical lens settings. Fig. 6bshows the EDOF images computed from the capturedimages, in which all scene elements appear focused.2

Fig. 6c shows images captured by a normal camera withthe same f=# and exposure time. Scene elements at thecenter depth are in focus. We can get a large DOF imageusing a smaller aperture. Images captured by a normalcamera with the same exposure time, but with a smalleraperture of f=8 are shown in Fig. 6d. The intensities of theseimages were scaled up so that their dynamic range matchesthat of the corresponding computed EDOF images. All sceneelements look reasonably sharp, but the images are verynoisy, as can be seen in the insets (zoomed). The computedEDOF images have much less noise, while having compar-able sharpness, i.e., our EDOF camera can capture sceneswith large DOFs as well as high SNR. Fig. 7 shows anotherexample of a scene captured outdoors at night. As we cansee, in a normal camera, the trade-off between DOF and SNRis extreme for such dimly lit scenes. Our EDOF cameraoperating with a large aperture is able to capture somethingin this scene, while a normal camera with a comparable DOFis too noisy to be useful. Several denoising algorithms havebeen proposed and it is conceivable that they can be used toimprove the appearance of images captured with a smallaperture, like the images in Fig. 6d. However, it is unlikelythat they can be used to restore images like the one in Fig. 7d.High resolution versions of these images as well as otherexamples can be seen at [27].

Since we translate the detector at a constant speed, theIPSF does not depend on the direction of motion—it is thesame whether the detector moves from a distance a fromthe lens to a distance b from the lens or from a distance bfrom the lens to a distance a from the lens. We can exploitthis to get EDOF video by moving the detector alternatelyforward one frame and backward the next. Fig. 8a shows aframe from a video sequence captured in this fashion and

Fig. 8b shows the EDOF frame computed from it. In thisexample, we were restricted by the capabilities of ouractuator and were able to achieve only 1.5 frames/sec. Toreduce motion blur, the camera was placed on a slowlymoving robotic arm. For comparison, Figs. 8c and 8d showframes from video sequences captured by a normal cameraoperating at f=1:4 and f=8, respectively.

4.3 Analysis of SNR Benefits of EDOF Camera

We now analyze the SNR benefits of using our approach tocapture scenes with extended DOF. Deconvolution usingDabov et al.’s method [25] produces visually appealingresults, but since it has a nonlinear denoising step, it is notsuitable for analyzing the SNR of deconvolved capturedimages. Therefore, we performed a simulation that usesWiener deconvolution [24]. Given an IPSF k, we convolve itwith a natural image I, and add zero-mean white Gaussiannoise with standard deviation �. The resulting image isthen deconvolved with k to get the EDOF image I. Thestandard deviation � of ðI � IÞ is a measure of the noise inthe deconvolution result when the captured image hasnoise �.

The degree to which deconvolution amplifies noisedepends on how much the high frequencies are attenuatedby the IPSF. This, in turn, depends on the distance throughwhich the detector moves during image integration—as thedistance increases, so does the attenuation of high frequen-cies. This is illustrated in Fig. 9a, which shows (in red) themagnitude of the Fourier transform (MTF) for a simulatedIPSF k1, derived using the pillbox lens PSF model. In thiscase, we use the same detector translation and parametersas in our EDOF experiments (Section 4.1). The MTF of theIPSF k2 obtained when the detector translation is halved(keeping the midpoint of the translation the same) is alsoshown (in blue). As expected, k2 attenuates the highfrequencies less than k1.

We analyzed the SNR benefits for these two IPSFs fordifferent noise levels in the captured image. The table inFig. 9b shows the noise produced by a normal camera fordifferent aperture sizes, given the noise level for the largestaperture, f=1:4. (Image brightness is assumed to lie between0 and 1.) The last two rows show the effective noise levelsfor EDOF cameras with IPSFs k1 and k2, respectively. Thelast column shows the effective DOFs realized; the normalcamera was assumed to be focused at a distance of 750 mmwith a maximum permissible circle of confusion of 14:1 �mfor a 1=300 sensor. Note that, as the noise level in thecaptured image increases, the SNR benefits of EDOFcameras increase.3 As an example, if the noise of a normalcamera at f=1:4 is 0.01, then the EDOF camera with IPSF k1

has the SNR of a normal camera operating at f=2:8, but hasa DOF that is greater than that of a normal camera at f=8.

In the above analysis, the SNR was averaged over allfrequencies. However, it must be noted that SNR isfrequency dependent—SNR is greater for lower frequencies


2. The computed EDOF images do have artifacts, like ringing, that aretypical of deconvolution [26].

3. For low noise levels, instead of capturing a well-exposed image withan EDOF camera, one could possibly use a normal camera to capturemultiple images with very low exposures (so that the total exposure time isthe same) and form a focal stack like [14], [15], [16], [17], provided thecamera is able to change focus and capture the images fast enough.However, as the noise level in captured images increases, the SNR benefitsof EDOF cameras clearly increase.

than for higher frequencies in the deconvolved EDOF

images. Hence, high frequencies in an EDOF image would

be degraded compared to the high frequencies in a perfectly

focused image. However, in our experiments, this degrada-

tion is not strong, as can be seen in the insets of Fig. 6 and the

full resolution images at [27]. Different frequencies in the

image having different SNRs illustrates the trade-off that our

EDOF camera makes. In the presence of noise, instead of

capturing with high fidelity, high frequencies over a small

range of scene depths (the depth of field of a normal camera),

our EDOF camera captures with slightly lower fidelity, high

frequencies over a large range of scene depths.


Fig. 6. (a) Images captured by our EDOF camera (f=1:4). (b) EDOF images computed from the captured images. (c) Images captured by a normalcamera (f=1:4, center focus). (d) Images captured by a normal camera (f=8, center focus) with scaling. All images were captured with an exposuretime of 0.36 seconds.

5 DISCONTINUOUS DEPTH OF FIELD

Consider the image in Fig. 10a, which shows two toys (cow

and hen) in front of a scenic backdrop with a wire mesh in

between. A normal camera with a small DOF can capture

either the toys or the backdrop in focus, while eliminating

the mesh via defocusing. However, since its DOF is a single

continuous volume, it cannot capture both the toys and the

backdrop in focus and at the same time eliminate the mesh.If we use a large aperture and program our camera’sdetector motion such that it first focuses on the toys for apart of the integration time, and then moves quickly toanother location to focus on the backdrop for the remainingintegration time, we obtain the image in Fig. 10b. While thisimage includes some blurring, it captures the high frequen-cies in two disconnected DOFs—the foreground and thebackground—but almost completely eliminates the wiremesh in between. This is achieved without any postproces-sing. Note that we are not limited to two disconnected DOFs;by pausing the detector at several locations during imageintegration, more complex DOFs can be realized.

6 TILTED DEPTH OF FIELD

Normal cameras can focus on only fronto-parallel sceneplanes. On the other hand, view cameras [2], [3] can be madeto focus on tilted scene planes by adjusting the orientation ofthe lens with respect to the detector. We show that ourflexible DOF camera can be programmed to focus on tiltedscene planes by simply translating (as in previous applica-tions) a detector with a rolling electronic shutter. A largefraction of CMOS detectors are of this type—while all pixelshave the same integration time, successive rows of pixels areexposed with a slight time lag. If the exposure time issufficiently small, then, up to an approximation, we can saythat the different rows of the image are exposed indepen-dently. When such a detector is translated with uniformspeed s, during the frame read-out time T of an image, weemulate a tilted image detector. If this tilted detector makesan angle � with the lens plane, then the focal plane in thescene makes an angle �with the lens plane, where � and � arerelated by the well-known Scheimpflug condition [4]:

� ¼ tan�1 sT

H

� �and

� ¼ tan�1 2f tanð�Þ2pð0Þ þH tanð�Þ � 2f

� �:

ð9Þ

Here, H is the height of the detector. Therefore, bycontrolling the speed s of the detector, we can vary thetilt angle of the image detector and, hence, the tilt of thefocal plane and its associated DOF.

Fig. 11 shows a scene where the dominant scene plane—atable top with a newspaper, keys, and a mug on it—isinclined at an angle of approximately 53 degrees with thelens plane. As a result, a normal camera is unable to focus onthe entire plane, as seen in Fig. 11a. By translating a rollingshutter detector (1=2:500 CMOS sensor with a 70 msecexposure lag between the first and last row of pixels) at2.7 mm/sec, we emulate a detector tilt of 2.6 degrees. Thisenables us to achieve the desired DOF tilt of 53 degrees(from (9)) and capture the table top (with the newspaper andkeys) in focus, as shown in Fig. 11b. Observe that the top ofthe mug is not in focus, but the bottom appears focused,illustrating the fact that the DOF is tilted to be aligned withthe table top. Note that there is no postprocessing here.

7 NONPLANAR DEPTH OF FIELD

In the previous section, we have seen that, by uniformlytranslating a detector with a rolling shutter, we can emulate


Fig. 7. (a) Image captured by our EDOF camera (f=1:4). (b) ComputedEDOF image. (c) Image from normal camera (f=1:4, near focus). (d)Image from normal camera (f=8, near focus). with scaling. All imageswere captured with an exposure time of 0.72 seconds.

a tilted image detector. Taking this idea forward, if wetranslate such a detector in some nonuniform fashion(varying speed), we can emulate a nonplanar imagedetector. Consequently, we get a nonplanar focal surfaceand, hence, a nonplanar DOF. This is in contrast to a normalcamera which has a planar focal surface and whose DOF isa fronto-parallel slab.

Fig. 12a shows a scene captured by a normal camera. It hascrayons arranged on a semicircle with a price tag in themiddle placed at the same depth as the leftmost andrightmost crayons. Only the two extreme crayons on eitherside and the price tag are in focus; the remaining crayons aredefocused. Say we want to capture this scene so that the DOF

is “curved”—the crayons are in focus while the price tag isdefocused. We set up a nonuniform motion of the detector toachieve this desired DOF, which can be seen in Fig. 12b.

8 EXPLOIT CAMERA’S FOCUSING MECHANISM TO

MANIPULATE DEPTH OF FIELD

Till now we have seen that by moving the detector duringimage integration, we can manipulate the DOF. However, itmust be noted that whatever effect we get by moving thedetector, we can get exactly the same effect by moving thelens (in the opposite direction). In fact, cameras already havemechanisms to do this; this is what happens during focusing.


Fig. 9. (a) MTFs of simulated IPSFs, k1 and k2, of an EDOF camera corresponding to the detector traveling two different distances during imageintegration. (b) Comparison of the effective noise and DOF of a normal camera and an EDOF camera with IPSFs k1 and k2. The image noise of anormal camera operating at f=1:4 is assumed to be known.

Fig. 8. An example that demonstrates how our approach can be used to capture EDOF video and its benefits over a normal camera. These videoscan be seen at [27]. (a) Video frame captured by our EDOF camera (f=1:4). (b) Computed EDOF frame. (c) Video frame from normal camera (f=1:4).(d) Video frame from normal camera (f=8) with scaling.

Hence, we can exploit the camera’s focusing mechanism tomanipulate DOF. Fig. 13a shows an image captured by anormal SLR camera (Canon EOS 20D with a Sigma 30 mmlens) at f=1:4, where only the near flowers are in focus. Tocapture this scene with an extended DOF, we manuallyrotated the focus ring of the SLR camera lens uniformlyduring image integration. For the lens we used, uniformrotation corresponds to moving the lens at a roughlyconstant speed. Fig. 13b shows an image captured in thisfashion. Fig. 13c shows the EDOF image computed from it, inwhich the entire scene appears sharp and well focused. Fordeconvolution, we used the analytic PSF given by (6). Theseimages as well as other examples can be seen at [27].

9 COMPUTING AN ALL-FOCUSED IMAGE FROM A

FOCAL STACK

Our approach to extended DOF also provides a convenientmeans to compute an all-focused image from a focal stack.Traditionally, given a focal stack, for every pixel we have todetermine in which image that particular pixel is in-focus[28], [29]. Some previous works have tackled this as a

labeling problem, where the label for every pixel is theimage where the pixel is in-focus. The labels are optimizedusing a Markov Random Field that is biased towardpiecewise smoothness [18], [17].

We propose an alternate approach that leverages ourobservations in Section 4.1. We propose to compute aweighted average of all the images of the focal stack(compensating for magnification effects if possible), wherethe weights are chosen to mimic changing the distancebetween the lens and the detector at a constant speed. FromSection 4.1, we know that this average image would haveapproximately depth independent blur. Hence, deconvolu-tion with a single blur kernel will give a sharp image in whichall scene elements appear focused. Figs. 14a, 14b, and 14cshow three of the 28 images that form a focal stack. Thesewere captured with a Canon 20D SLR camera with a Sigma30 mm lens operating at f=1:4. Fig. 14d shows the all-focusedimage computed from the focal stack using this approach.We are not claiming that this technique is the best forcomputing an all focused image from a focal stack. As notedearlier, deconvolution artifacts could appear in the resultingimages and high frequencies would be captured with lower


Fig. 11. (a) An image captured by a normal camera (f=1:4, T ¼ 0:03 sec) of a table top inclined at 53 degrees with respect to the lens plane. (b) Animage captured by our flexible DOF camera (f=1:4, T ¼ 0:03 sec) where the DOF is tilted by 53 degrees. The entire table top (with the newspaperand keys) appears focused. Observe that the top of the mug is defocused, but the bottom appears focused, illustrating that the focal plane is alignedwith the table top. Three scene regions of both of the images are shown at a higher resolution to highlight the defocus effects.

Fig. 10. (a) An image captured by a normal camera with a large DOF. (b) An image captured by our flexible DOF camera (f=1:4), where the toy cowand hen in the foreground and the landscape in the background appear focused, while the wire mesh in between is optically erased via defocusing.

fidelity. This example illustrates how our approach can be

used to realize a simpler (possibly slightly inferior) solution

to this problem than conventional approaches.

10 DISCUSSION

In this paper, we have proposed a camera with a flexible

DOF. DOF is manipulated in various ways by controlling

the motion of the detector during image integration. We

have shown how such a system can capture scenes withextended DOF while using large apertures. We have alsoshown that we can create DOFs that span multipledisconnected volumes. In addition, we have demonstratedthat our camera can focus on tilted scene planes as well asnonplanar scene surfaces. Finally, we have shown that wecan manipulate DOF by exploiting the focusing mechanismof the lens. This can be very convenient and practical,especially for camera manufacturers.


Fig. 13. (a) Image captured by a Canon EOS 20D SLR camera with a Sigma 30 mm lens operating at f=1:4, where only the near flowers are in focus

(T ¼ 0:6 sec). (b) Image captured by the camera when the focus ring was manually rotated uniformly during image integration (f=1:4, T ¼ 0:6 sec).

(c) Image with extended DOF computed from the image in (b).

Fig. 12. (a) An image captured by a normal camera (f=1:4; T ¼ 0:01 sec) of crayons arranged on a semicircle with a price tag in the middle placed atthe same depth as the leftmost and rightmost crayons. Only the price tag and the extreme crayons are in focus. (b) An image captured by our flexibleDOF camera (f=1:4; T ¼ 0:01 sec) where the DOF is curved to be aligned with the crayons—all of the crayons are in focus, while the price tag isdefocused. Four scene regions of both the images are shown at a higher resolution to highlight the defocus effects.

Effects at occlusion boundaries. For our EDOF camera,we have not explicitly modeled the defocus effects atocclusion boundaries. Due to defocus blur, image points

that lie close to occlusion boundaries can receive light fromscene points at very different depths. However, since theIPSF of the EDOF camera is nearly depth invariant, theaggregate IPSF for such an image point can be expected tobe similar to the IPSF of points far from occlusionboundaries. In some of our experiments, we have seenartifacts at occlusion boundaries. These can possibly beeliminated using more sophisticated deconvolution algo-rithms such as [26], [30]. In the future, we would like toanalyze in detail the effects at occlusion boundaries, similarto works like [31] and [32]. Note that in tilted and nonplanarDOF examples occlusion boundaries are correctly captured;there are no artifacts.

Effects of scene motion. The simple off-the-shelf actuatorthat we used in our prototype has low translation speedsand so we had to use exposure times of about 1=3rd of asecond to capture EDOF images. However, we have notobserved any visible artifacts in EDOF images computed forscenes with typical object motion (see Fig. 6). With fasteractuators, like piezoelectric stacks, exposure times can bemade much smaller and thereby allow captured scenes to bemore dynamic. However, in general, motion blur due tohigh-speed objects can be expected to cause problems. In thiscase, a single pixel sees multiple objects with possiblydifferent depths and it is possible that neither of the objectsare imaged in perfect focus during detector translation. Intilted and nonplanar DOF applications, fast moving scenepoints can end up being imaged at multiple image locations.All images of a moving scene point would be in-focus if itscorresponding 3D positions lie within the (planar/nonpla-nar) DOF. These multiple image locations can be used tomeasure the velocity and pose of the scene point, as wasshown by [21].

Using different actuators. In our prototype, we haveused a simple linear actuator whose action was synchro-nized with the exposure time of the detector. However,other more sophisticated actuators can be used. As men-tioned above, faster actuators like piezoelectric stacks candramatically reduce the time needed to translate a detectorover the desired distance and so enable low exposure times.This can be very useful for realizing tilted and nonplanarDOFs, which need low exposure times. In an EDOF camera,an alternative to a linear actuator is a vibratory actuator—theactuator causes the detector to vibrate with an amplitudethat spans the total desired motion of the detector. If thefrequency of the vibration is very high (around 100 timeswithin the exposure of an image), then one would not needany synchronization between the detector motion and theexposure time of the detector; errors due to lack ofsynchronization would be negligible.

Robustness of EDOF camera PSF. In our experience,the EDOF camera’s PSF is robust to the actual motion ofthe detector or the lens. This is illustrated by the fact thatwe are able to capture scenes with large DOFs even whenthe motion realized is only approximately uniform (ex-ample in Section 8). Since this approach does not seemsusceptible to small errors in motion, it is particularlyattractive for practical implementation in cameras.

Realizing arbitrary DOFs. We have shown how we canexploit rolling shutter detectors to realize tilted andnonplanar DOFs (Sections 6 and 7). In these detectors, if


Fig. 14. (a)-(c) Three out of 28 images that form a focal stack. The

images were captured with a Canon 20D camera with a Sigma 30 mm

lens operating at f=1:4. (d) The all-focused image computed from the

focal stack images using the approach described in Section 9.

the exposure time is sufficiently small, then we canapproximately say that the different rows of the imageare exposed independently. This allows us to realizeDOFs where the focal surfaces are swept surfaces. It isconceivable, that in the future, we might have detectorsthat provide pixel-level control of exposure—we canindependently control the start and end time of theexposure of each pixel. Such control coupled with asuitable detector motion would enable us to indepen-dently choose the scene depth that is imaged in-focus atevery pixel, yielding arbitrary DOF manifolds.

Practical implementation. All DOF manipulationsshown in this paper can be realized by moving the lensduring image integration (Section 8 shows one example).Compared to moving the detector, moving the lens wouldbe more attractive for camera manufacturers since camerasalready have actuators that move the lens for focusing. Allthat is needed is to expose the detector while the focusingmechanism sweeps the focal plane through the scene.Hence, implementing these DOF manipulations would notbe difficult and can possibly be realized by simply updatingthe camera firmware.

We believe that flexible DOF cameras can open up a newcreative dimension in photography and lead to newcapabilities in scientific imaging, computer vision, andcomputer graphics. Our approach provides a simple meansto realizing such flexibility.

ACKNOWLEDGMENTS

The authors would like to acknowledge grants from the USNational Science Foundation (IIS-04-12759) and the USOffice of Naval Research (N00014-08-1-0329 and N00014-06-1-0032) that supported parts of this work. Thanks also toMarc Levoy for his comments related to the application ofHausler’s method [1] to microscopy.

REFERENCES

[1] G. Hausler, “A Method to Increase the Depth of Focus by TwoStep Image Processing,” Optics Comm., vol. 6, no. 1, pp. 38-42,1972.

[2] H. Merklinger, “Focusing the View Camera,” 1996.[3] A. Krishnan and N. Ahuja, “Range Estimation from Focus

Using a Non-Frontal Imaging Camera,” Int’l J. Computer Vision,vol. 20, no. 3, pp. 169-185, 1996.

[4] T. Scheimpflug, “Improved Method and Apparatus for theSystematic Alteration or Distortion of Plane Pictures and Imagesby Means of Lenses and Mirrors for Photography and for OtherPurposes,” GB patent, 1904.

[5] H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, “FlexibleDepth of Field Photography,” Proc. European Conf. ComputerVision, pp. 60-73, 2008.

[6] E.R. Dowski and W. Cathey, “Extended Depth of Field ThroughWavefront Coding,” Applied Optics, vol. 34, pp. 1859-1866, 1995.

[7] N. George and W. Chi, “Extended Depth of Field Using aLogarithmic Asphere,” J. Optics A: Pure and Applied Optics, vol. 5,pp. 157-163, 2003.

[8] A. Castro and J. Ojeda-Castaneda, “Asymmetric Phase Masks forExtended Depth of Field,” Applied Optics, vol. 43, pp. 3474-3479,2004.

[9] A. Levin, R. Fergus, F. Durand, and B. Freeman, “Image andDepth from a Conventional Camera with a Coded Aperture,”Proc. ACM SIGGRAPH, 2007.

[10] A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J.Tumblin, “Dappled Photography: Mask Enhanced Cameras forHeterodyned Light Fields and Coded Aperture,” Proc. ACMSIGGRAPH, 2007.

[11] E. Adelson and J. Wang, “Single Lens Stereo with a PlenopticCamera,” IEEE Trans. Pattern Analysis and Machine Intelligence,vol. 14, no. 2, pp. 99-106, Feb. 1992.

[12] R. Ng, M. Levoy, M. Brdif, G. Duval, M. Horowitz, and P.Hanrahan, “Light Field Photography with a Hand-Held PlenopticCamera,” technical report, Stanford Univ., 2005.

[13] T. Georgiev, C. Zheng, B. Curless, D. Salesin, S.K. Nayar, and C.Intwala, “Spatio-Angular Resolution Tradeoff in Integral Photo-graphy,” Proc. Eurographics Symp. Rendering, pp. 263-272, 2006.

[14] T. Darrell and K. Wohn, “Pyramid Based Depth from Focus,” Proc.IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 504-509,1988.

[15] S.K. Nayar, “Shape from Focus System,” Proc. IEEE CS Conf.Computer Vision and Pattern Recognition, pp. 302-308, 1992.

[16] M. Subbarao and T. Choi, “Accurate Recovery of Three-Dimen-sional Shape from Image Focus,” IEEE Trans. Pattern Analysis andMachine Intelligence, vol. 17, no. 3, pp. 266-274, Mar. 1995.

[17] S.W. Hasinoff and K.N. Kutulakos, “Light-Efficient Photography,”Proc. European Conf. Computer Vision, pp. 45-59, 2008.

[18] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A.Colburn, B. Curless, D. Salesin, and M. Cohen, “Interactive DigitalPhotomontage,” Proc. ACM SIGGRAPH, pp. 294-302, 2004.

[19] A. Levin, P. Sand, T.S. Cho, F. Durand, and W.T. Freeman,“Motion-Invarient Photography,” Proc. ACM SIGGRAPH, 2008.

[20] M. Ben-Ezra, A. Zomet, and S. Nayar, “Jitter Camera: HighResolution Video from a Low Resolution Detector,” Proc. IEEE CSConf. Computer Vision and Pattern Recognition, pp. 135-142, 2004.

[21] O. Ait-Aider, N. Andreff, J.-M. Lavest, and P. Martinet,“Simultaneous Object Pose and Velocity Computation Using aSingle View from a Rolling Shutter Camera,” Proc. European Conf.Computer Vision, pp. 56-68, 2006.

[22] D. Field, “Relations between the Statistics of Natural Images andthe Response Properties of Cortical Cells,” J. Optical Soc. of Am.,vol. 4, pp. 2379-2394, 1987.

[23] H. Hopkins, “The Frequency Response of a Defocused OpticalSystem,” Proc. Royal Soc. of London Series A, Math. and PhysicalSciences, vol. 231, pp. 91-103, 1955.

[24] P.A. Jansson, Deconvolution of Images and Spectra. Academic Press,1997.

[25] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “ImageRestoration by Sparse 3D Transform-Domain CollaborativeFiltering,” Proc. SPIE Electronic Imaging, 2008.

[26] L. Yuan, J. Sun, L. Quan, and H.-Y. Shum, “Progressive Inter-Scaleand Intra-Scale Non-Blind Image Deconvolution,” Proc. ACMSIGGRAPH, 2008.

[27] www.cs.columbia.edu/CAVE/projects/flexible_dof, 2010.[28] P. Burt and R. Kolczynski, “Enhanced Image Capture through

Fusion,” Proc. Fourth IEEE Int’l Conf. Computer Vision, pp. 173-182,1993.

[29] P. Haeberli, “Grafica Obscura,” www.sgi.com/grafica/, 1994.[30] Q. Shan, J. Jia, and A. Agarwala, “High-quality Motion Deblurring

from a Single Image,” Proc. ACM SIGGRAPH, 2008.[31] N. Asada, H. Fujiwara, and T. Matsuyama, “Seeing Behind the

Scene: Analysis of Photometric Properties of Occluding Edges bythe Reversed Projection Blurring Model,” IEEE Trans. PatternAnalysis and Machine Intelligence, vol. 20, no. 2, pp. 155-167, Feb.1998.

[32] S. Bhasin and S. Chaudhuri, “Depth from Defocus in Presence ofPartial Self Occlusion,” Proc. Eighth IEEE Int’l Conf. ComputerVision, pp. 488-493, 2001.

Sujit Kuthirummal received the BTech and MSdegrees in computer science from the Interna-tional Institute of Information Technology, Hy-derabad, in 2002 and 2003, respectively, andthe PhD degree in computer science fromColumbia University in 2009. Since 2009, hehas been a member of the technical staff atSarnoff Corporation. His research interestsinclude computational photography and 3Dreconstruction. He is a member of the IEEE.


Hajime Nagahara received the BE and MEdegrees in electrical and electronic engineeringfrom Yamaguchi University in 1996 and 1998,respectively, and the PhD degree in systemengineering from Osaka University in 2001. He isan assistant professor at the Graduate School ofEngineering Science, Osaka University, Japan.He was a research associate of the JapanSociety for the Promotion of Science (2001-2003) and the Graduate School of Engineering

Science, Osaka University (2003-2006). He was a visiting associateprofessor at CREA University of Picardie Jules Verns, France, in 2005.He was a visiting researcher at Columbia University in 2007-2008. Imageprocessing, computer vision, and virtual reality are his research fields.He received an ACM VRST2003 Honorable Mention Award in 2003.

Changyin Zhou received the BS degree instatistics and the MS degree in computer sciencefrom Fudan University in 2001 and 2007,respectively. He is currently a doctoral studentin the Computer Science Department of Colum-bia University. His research interests includecomputational imaging and physics-based vi-sion. He is a student member of the IEEE and theIEEE Computer Society.

Shree K. Nayar received the PhD degree inelectrical and computer engineering from theRobotics Institute at Carnegie Mellon Universityin 1990. He is currently the T.C. ChangProfessor of Computer Science at ColumbiaUniversity. He codirects the Columbia Visionand Graphics Center. He also heads theColumbia Computer Vision Laboratory (CAVE),which is dedicated to the development ofadvanced computer vision systems. His re-

search is focused on three areas; the creation of novel cameras, thedesign of physics-based models for vision, and the development ofalgorithms for scene understanding. His work is motivated by applica-tions in the fields of digital imaging, computer graphics, and robotics. Hehas received best paper awards at ICCV 1990, ICPR 1994, CVPR 1994,ICCV 1995, CVPR 2000 and CVPR 2004. He is the recipient of theDavid Marr Prize (1990 and 1995), the David and Lucile PackardFellowship (1992), the National Young Investigator Award (1993), theNTT Distinguished Scientific Achievement Award (1994), the KeckFoundation Award for Excellence in Teaching (1995), the ColumbiaGreat Teacher Award (2006), and the Carnegie Mellon UniversityAlumni Achievement Award (2009). In February 2008, he was elected tothe National Academy of Engineering. He is a member of the IEEE.

. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.


flexible depth of field photography

Documents