CSE 597F : Computational Photography

Spring 2010 Fridays 1:25-3:55 in 002 DeikeInstructors: R.Collins, D.Capel and Y.Liu

Credits: 3 (will count as a 598 for fulfilling graduation requirements).

Course Description

• Computational photography combines elements of optics, graphics, and computer vision to enhance or extend the capabilities of digital photography.

• We will learn about this new visual medium through lectures, readings and hands-on photographic assignments, culminating in a final course project.

Prerequisites

• knowledge of Matlab; • either CMPEN454 (Vision) or 455 (Image Processing)

or equivalents;

The following are helpful but not required:• CMPSC 458 (Computer Graphics); • access to a digital camera that allows manual control

of shutter and aperture.

Digital Cameras and Optics

Image Alignment and Mosaicing

• sample paper: Brown and Lowe. “Automatic Panoramic Image Stitching using Invariant Features “, IJCV 2006.

Image Alignment and Mosaicing

gain compensation and blending

automatic alignment of photo collections

Pyramid-based Blending

= +

+

+

+

Laplacian Image Pyramid

• sample paper: Burt and Adelson, “A multiresolutionspline with application to image mosaics,” ACM Trans on Graphics, 1983.

Pyramid-based Blending

(c) David MartinBurt and Adelson

Gradient-Domain Blendingimage compositing by “cloning” and blending

• sample paper: Perez et.al. “Poisson Image Editing”, Siggraph 2003.

Gradient-Domain Blending

naive cut and paste result Poisson blending result

Tone Mapping and HDR Imaging

overexposed

underexposed collect a range of exposures

• sample paper: Debevec and Malik. “Recovering High Dynamic Range Radiance Maps from Photographs.” Siggraph 1997.

Tone Mapping and HDR Imagingcombine exposures to yielda high dynamic range image

Flash / No-Flash Photography

• sample paper: Petschnigg et.al., “Digital Photography with Flash and No-Flash Image Pairs.”, Siggraph 2004.

low light picture, no flash- very noisy- warm natural lighting

picture with flash- less noise, more detail- but cold and unnatural lighting

Flash / No-Flash Photography

combined image- good smoothing- details transferred from flash- retains the warm, natural lighting

Coded-Aperture Imaging

• sample paper: Levin et.al., “Image and Depth from a Conventional Camera with a Coded Aperture”, Siggraph 2007.

depth estimationfrom single image

deblurring /refocusing

note difference in the Bokeh!

This one preserveshigh frequency info.

Raskar, ACCV07 keynote address

Graphics and Photography• Seam-carving

• sample paper: Avidan and Shamir, “Seam Carving for Content-Aware Image Resizing”, Siggraph 2007.

image resizing

object removal

Graphics and Photography• Delayering and

Inpainting

de-fencing; Liu et.al.

image restoration

• Sample papers: Liu et.al., “Image Defencing”, CVPR 2008; “Near-regular texture analysis and manipulation”, Siggraph2004; “A Lattice-based MRF Model for Dynamic Near-regular Texture Tracking,” PAMI 2007.

Graphics and Photographyde-fencing

texture replacement

Graphics and Photography

• sample paper: Levin et.al., “Colorization using Optimization,” ACM Transactions on Graphics, Aug 2004

greyscale photo with user-annotated colors colorized result

• Colorization

• Texture transfer and Image analogies

• sample paper: Hertzmann et.al., “Image Analogies,” Siggraph2001

Graphics and Photography

Grading Criteria

• 20% Presentations / Discussion • 40% Homework assignments • 40% Final project

Course Flickr Group

http://www.flickr.com/groups/psucompphoto/

For posting photos / resultsDiscussion forum

What you should do about itGet an accountCome to the group page and request to join We will then invite you to join the groupJoin, and start participating!