Computational Photography - ?· Computational photography • More than digital photography • Arbitrary…

Download Computational Photography - ?· Computational photography • More than digital photography • Arbitrary…

Post on 25-Aug-2018




3 download

Embed Size (px)


  • Computational Photography

    Matthias Zwicker University of Bern

    Fall 2012

  • Today Course organization

    Course overview

    Image formation

  • Course organization Instructor

    Matthias Zwicker (

    Teaching Assistant

    Daniel Donatsch (

  • Course organization Lecture

    Mondays, 14:00-16:00

    Engehaldenstrasse 8, Room 3


    Mondays, 16:00-17:00

    Engehaldenstrasse 8, Room 3

  • Class web page Class overview

  • ILIAS Use your campus account to log in

    Join course Magazin Weitere Institutionen; Weiterbildungen und Studiengnge BeNeFri Joint Master in Computer Science HS2012 2012HS: 31051 Computational Photography

    Lecture slides

    Exercise description & material

    Additional reading material


    Any questions and discussions related to class material and exercises

  • Exercises 6 assignments

    Programming projects

    Matlab Available in ExWi pool

    Exercises on paper

  • Exercises Final grade: 40% exercises, 60% final exam To qualify for final exam: need 70% of

    exercise score Late penalty

    50% of original score Exceptions for military service, illness

    Collaboration Discussion among students is encouraged Each student must write up and turn in his/her

    own solution If we detect copied material, you will need to talk

    to us and explain your material in person; if we are not satisfied, you will not get credit

  • Final exam Written, 105 minutes

    Bring two A4 sheets (4 pages) of hand written notes

    Relevant material: slides and exercises

    Wikipedia links not part of class material, but may be useful to better understand concepts discussed in class

    Date: February 2013

  • Prerequisites Familiarity with

    Linear algebra (matrix calculations, linear systems of equations, least squares problems)

    Programming experience

  • Today Course organization

    Course overview

    Image formation

  • Computational photography Topics of this class

    Role of computation, algorithms in digital photography today

    Algorithms to extend and improve capabilities of digital photography in the future

  • Photography Traditionally

    Measuring light

    Optics focuses light on sensor

    Sensor records image


    Digital Film

  • Computational photography More than digital photography

    Arbitrary computation between light measurement and final image Light measured on sensor is not final image Computation enhances and extends capabilities of

    digital photography Two types of computation

    1. Post-process after traditional imaging 2. Design of new camera devices that require

    computation to form an image Overview of recent research

  • Removing imaging artifacts Denoising & deblurring

    Blurry + Output Noisy Algorithm

  • Removing imaging artifacts High dynamic range images & tone mapping

  • Image manipulation Panoramas

  • Computational optics

    Coded aperture

    Captured image, slightly blurry everywhere

  • Computational optics

    Recovered depth

    Refocused image Sharp foreground, blurry background

  • Focus of class Fun with digital photography and

    computer programming

    Algorithms and computational techniques with potential applications in the consumer domain

    Mostly software, less hardware

    Recent research

  • What you will learn Basic understanding of photography, light,

    and color Practical experience with implementation of

    algorithms for image processing & computational photography

    Cool and creative applications of mathematical tools Fourier transforms Linear and non linear filtering Optimization techniques (least squares, iteratively

    re-weighted least squares, graph cuts) Probabilistic models

    Many applications beyond processing images!

  • Related areas, not covered Image processing for scientific applications

    Physics, biology, etc.

    Optics, lens design

    Photosensors, sensor design

    Computational imaging

    Tomography, radar, microscopy

    3D imaging

    Using photo processing tools, e.g. Photoshop

    Artistical aspects of photography

  • Syllabus 1. Introduction, image formation 2. Color & color processing 3. Dynamic range & contrast 4. Sampling, reconstruction, & the frequency domain 5. Image restoration: denoising & deblurring 6. Image manipulation using optimization 7. Gradient domain image manipulation 8. Warping & morphing 9. Panoramas 10. Automatic alignment 11. Probabilistic image models 12. Light fields 13. Capturing light transport

  • Cameras, image artifacts

    Image formation

  • Color Color perception, color spaces, color

    measurement, color processing

  • Dynamic range & contrast HDR imaging

  • Sampling, reconstruction Sampling artifacts

    Frequency domain analysis

    Spatial Domain Frequency Domain

  • Image restoration Denoising & deblurring

    Blurry input Deblurred output

    Estimated blur kernel (scaled)

  • Image manipulation using optimization Photomontage, matting, colorization

  • Gradient domain manipulation Poisson equation

  • Warping & morphing

  • Panoramas Automatic alignment, stitching

  • Probabilistic models Faces, textures

  • Beyond 2D images

    Light fields

  • Capturing light transport Dual photography

  • Today Course organization

    Course overview

    Image formation

  • Models of light

  • Question Why is there no image on a white piece of


  • Question Why is there no image on a white piece of


    Receives all light rays

    Images from all viewpoints

    Need to select light rays for specifice image, viewpoint


  • Invented by Alhazen, 10th century

    Pinhole camera

  • Limitations Small pinhole: sharper image, longer exposure

    Larger pinhole: blurrier image, shorter exposure

  • Camera model Thin lens, aperture, shutter, film

  • Lenses Gather more light

    Proportional to area of lens aperture

    Sharp image if focused

    Use refraction


    Scene point (emits or

    reflects light) Image of

    scene point

  • Lenses Pinhole Lens

    6 sec. exposure 0.01 sec exposure

  • Thin lens model

    Theoretical model for well-behaved lenses


    1. All parallel rays converge at focal length

    2. Rays through the center are not deflected

    Same perspective image as pinhole placed at center of lens

  • Thin lens model How are arbitrary rays deflected when

    passing through a thin lens?

    1. Parallel rays converge at focal length f

  • Thin lens model 2. Rays through center are not deflected

  • Thin lens model Similar triangles

  • Thin lens model More similar triangles

  • Thin lens model Thin lens formula

    All rays passing through a single point y on a plane at distance in front of the lens will pass through a single point y at distance behind the lens

  • Thin lens model Focus at infinity:

    Film at distance f

    Closest focusing distance:

    Film at infinity

    Film plane


  • Thin lens model Out of focus film plane results in spherical


    Out of focus film planes

    Spherical blur

  • Properties of real lenses Mostly undesired!


    Spherical aberration Chromatic aberration


    Barrel distortion Pincushion distortion


    Barrel & pincushion distortion

  • Question Whats the advantage of a lens with a

    short focal length? In what situation would this be useful?

    Whats the advantage of a lens with a long focal length? In what situation would this be useful?

  • Camera model Thin lens, aperture, shutter, film

  • Aperture Amount of light captured at sensor

    proportional to area of aperture


  • Aperture Blurriness of out of focus objects depends

    on aperture size

    Aperture size determines depth of field: depth range that is sharp in image


  • Depth of field

  • Circle of confusion Also called blur circle

    Calculation of radius c Lens focused at S1 Object at S2 Aperture A Focal length f sensor

    Proportional to A

  • f-number

    Definition: (focal length) / (diameter of aperture)

    Large aperture means small f-number

    Practice: f-stops increase by factors of

    f/2.0, f/2.8, f/4, f/5.6, f/8 Aperture area gets halved in each step

  • f-number Smaller f-number

    Larger aperture Capture more light Small (shallow) depth of field

    Larger f-number

    Smaller aperture Capture less light Large depth of field

  • Camera model Thin lens, aperture, shutter, film

  • Shutter speed

    Determines time the film is exposed to light

    Amount of light captured is proportional to exposure time

    Long exposure leads to motion blur

  • Reciprocity Amount of light captured stays

    same if exposure is doubled and aperture area is halved (or vice versa)

  • Reciprocity Which exposure/aperture combination?

  • Camera model Thin lens, aperture, shutter, film

  • Film Film/sensor responds roughly linearly to light

    Double the amount of light leads to double the recorded value

    Film speed: sensitivity of film to light

    Digital photography analog: sensor gain (scaling or amplification factor)

    Measured using ISO scale

    Linear: sensitivity is proportional to ISO value Double ISO value, halve the exposure time, get

    same recorded value

  • Film Trade-off: higher gain, more noise

    ISO 100 ISO 3200

  • Film Underexposure

    Not enough light, image too dark Overexposure

    Film or sensor is saturated Clipping of highlight details

    Good exposure Overexposure Underexposure

  • Conclusions Simple camera model

    Thin lens, aperture, shutter, film

    Photographs often have undesired artifacts

    Distortions, color artifacts, blur, noise, under/overexposure


    Develop algorithms to remove artifacts after image is captured

  • References Photography, by London, Upton, Stone

  • Next time Color, color processing

    Computational PhotographyTodayCourse organizationCourse organizationClass web pageILIASExercisesExercisesFinal examPrerequisitesTodayComputational photographyPhotographyComputational photographyRemoving imaging artifactsRemoving imaging artifactsImage manipulationComputational opticsComputational opticsFocus of classWhat you will learnRelated areas, not coveredSyllabusImage formationColorDynamic range & contrastSampling, reconstructionImage restorationImage manipulation using optimizationGradient domain manipulationWarping & morphingPanoramasProbabilistic modelsLight fieldsCapturing light transportTodayModels of lightQuestionQuestionPinhole cameraLimitationsCamera modelLensesLensesThin lens modelThin lens modelThin lens modelThin lens modelThin lens modelThin lens modelThin lens modelThin lens modelProperties of real lensesQuestionCamera modelApertureApertureDepth of fieldCircle of confusionf-numberf-numberCamera modelShutter speedReciprocityReciprocityCamera modelFilmFilmFilmConclusionsReferencesNext time


View more >