introduction to computational photography

25
Introduction to Computational Photography

Upload: ghalib

Post on 22-Jan-2016

91 views

Category:

Documents


0 download

DESCRIPTION

Introduction to Computational Photography. What is Computational Photography?. Second breakthrough by IT First : electronic image sensor (digital camera) Digital representation of “image formed by lens” Second : Re-definition of whole camera (optics, usage) - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Introduction to Computational Photography

Introduction toComputational Photography

Page 2: Introduction to Computational Photography

Computational Photography

Digital Camera

What isComputational Photography?

Second breakthrough by IT First : electronic image sensor (digital camera)

Digital representation of “image formed by lens” Second : Re-definition of whole camera (optics, usage)

Image is reconstructed by computation

Imageprocessing

Image

Image sensor

Optics

Whole part of camera is affected by computational photography

Film camera

Digital Camera

Page 3: Introduction to Computational Photography

Light field(Light space)

(X,Y,Z)

(θ,φ)

What is camera?

Camera is a machine to record the distribution of the light in a scene

How to represent the distribution of the light in the scene? 3-D coordinate of the point where the light passing through : X, Y, Z Direction of the light : θ , Φ Wavelength of the light (color of the light) : λ Time : t

The 7 parameters function P which represent the distribution of the light is called “Plenoptic function”

I = P(X,Y,Z,θ,φ,λ , t)Light source

Object

Page 4: Introduction to Computational Photography

Light field (light ray) Optics (lens) Image sensor (pixel)

(X,Y,Z)

(θ,φ)

Integration of camera

Camera integrates the light for all 7 parameters Position ( range of X, Y, Z : aperture size should not be zero) Direction ( range of θ , Φ : pixel size is not zero) Wavelength ( range of λ : No single wavelength filter) Exposure time ( range of t : shutter speed should not be too fast)

Multiple samples - θ,Φ:number of pixel, λ : RGB , t : burst shot So, what is multiple sampling for X, Y, Z?

I = P(X,Y,Z,θ,φ,λ , t)

Page 6: Introduction to Computational Photography

Use of camera array

Free-viewpoint image

Defocus generation by synthetic aperture

3-D video (Matsuyama lab, Kyoto Univ.) Synthetic aperture(Vaish@Stanford)

Page 7: Introduction to Computational Photography

Defocus control by Uncalibrated Synthetic Aperture

Natsumi Kusumoto, Shinsaku Hiura and Kosuke Sato, Uncalibrated Synthetic Aperture for Defocus Control, CVPR2009 (Jun. 2009)

Page 8: Introduction to Computational Photography

(X,Y,Z)

(θ,φ)

Reviewing “integration”

Some part of information is lost by integration Sine wave which period is just as same as the integration duration

is lost

Blur of object within an exposure time Defocus by misfocus

I = P(X,Y,Z,θ,φ,λ , t)

Asin(nt)dt0

∫ = 0

× = 0

Page 10: Introduction to Computational Photography

This

TraditionalCoded

Exposure

Image of Static Object

Deblurred Image

Deblurred Image

Slide by R. Raskar

Page 11: Introduction to Computational Photography

Coded Exposure

Temporal 1-D broadband code: Motion Deblurring

Coded Aperture

Spatial 2-D broadband mask: Focus Deblurring

Slide by R. Raskar

Page 12: Introduction to Computational Photography

Captured Blurred Photo

Slide by R. Raskar

Page 13: Introduction to Computational Photography

Refocused on Person

Slide by R. Raskar

Page 14: Introduction to Computational Photography

Coded Aperture

Levin@MIT(2007) Depth estimation by single

image (manual operation is necessary)

Page 15: Introduction to Computational Photography

Coded Aperture

Levin@MIT(2007)

Page 16: Introduction to Computational Photography

Coded Aperture

Levin@MIT(2007)

Page 18: Introduction to Computational Photography

Multi-Focus Range Sensorusing Coded Aperture

Page 19: Introduction to Computational Photography

Invariant integration Defocus : changed according to the distance

Blur : changed according to the speed of the object

Reconstruction is not easy because the estimation of the speed or distance is necessaryIs it possible to make defocus or blur invariant to the distance or speed?

Page 20: Introduction to Computational Photography

Invariant integration Defocus

Special optics : Wavefront Coding

Motion of the image sensor while exposure

Blur Reciprocal motion of the camera

CDM Optics, Inc.

Page 21: Introduction to Computational Photography

Motion of the image sensorfor invariant defocus

H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008

Page 22: Introduction to Computational Photography

H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008

Motion of the image sensorfor invariant defocus

Page 23: Introduction to Computational Photography

Deblur by reciprocal motion of the camera

A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008.

Input image Deblurred image

Page 24: Introduction to Computational Photography

A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008.

Equipment Conceptual figure forLight sources with

different speed

Deblur by reciprocal motion of the camera

Page 25: Introduction to Computational Photography

More..

Resources on www Wikipedia : computational photography http://computationalphotography.org/ http://www1.cs.columbia.edu/CAVE/projects/what_is/ http://projects.csail.mit.edu/photo/

Conferences International Conference on Computational Photography SIGGRAPH, CVPR, .. Session about computational photography