computational photography - ?· computational photography • more than digital photography •...

Download Computational Photography - ?· Computational photography • More than digital photography • Arbitrary…

Post on 25-Aug-2018

215 views

Category:

Documents

3 download

Embed Size (px)

TRANSCRIPT

  • Computational Photography

    Matthias Zwicker University of Bern

    Fall 2012

  • Today Course organization

    Course overview

    Image formation

  • Course organization Instructor

    Matthias Zwicker (zwicker@iam.unibe.ch)

    Teaching Assistant

    Daniel Donatsch (donatsch@iam.unibe.ch)

    mailto:zwicker@iam.unibe.chmailto:knaus@iam.unibe.ch

  • Course organization Lecture

    Mondays, 14:00-16:00

    Engehaldenstrasse 8, Room 3

    Exercises

    Mondays, 16:00-17:00

    Engehaldenstrasse 8, Room 3

  • Class web page Class overview

    http://www.cgg.unibe.ch/teaching/computational-photography

    http://www.cgg.unibe.ch/teaching/computational-photography

  • ILIAS Use your campus account to log in

    Join course Magazin Weitere Institutionen; Weiterbildungen und Studiengnge BeNeFri Joint Master in Computer Science HS2012 2012HS: 31051 Computational Photography

    Lecture slides

    Exercise description & material

    Additional reading material

    Forum

    Any questions and discussions related to class material and exercises

    https://ilias.unibe.ch/repository.php?cmd=frameset&ref_id=1https://ilias.unibe.ch/repository.php?cmd=frameset&ref_id=711https://ilias.unibe.ch/repository.php?cmd=frameset&ref_id=447636https://ilias.unibe.ch/repository.php?cmd=frameset&ref_id=447636https://ilias.unibe.ch/repository.php?cmd=frameset&ref_id=447639https://ilias.unibe.ch/repository.php?cmd=frameset&ref_id=447650

  • Exercises 6 assignments

    Programming projects

    Matlab Available in ExWi pool

    Exercises on paper

  • Exercises Final grade: 40% exercises, 60% final exam To qualify for final exam: need 70% of

    exercise score Late penalty

    50% of original score Exceptions for military service, illness

    Collaboration Discussion among students is encouraged Each student must write up and turn in his/her

    own solution If we detect copied material, you will need to talk

    to us and explain your material in person; if we are not satisfied, you will not get credit

  • Final exam Written, 105 minutes

    Bring two A4 sheets (4 pages) of hand written notes

    Relevant material: slides and exercises

    Wikipedia links not part of class material, but may be useful to better understand concepts discussed in class

    Date: February 2013

  • Prerequisites Familiarity with

    Linear algebra (matrix calculations, linear systems of equations, least squares problems)

    Programming experience

  • Today Course organization

    Course overview

    Image formation

  • Computational photography Topics of this class

    Role of computation, algorithms in digital photography today

    Algorithms to extend and improve capabilities of digital photography in the future

  • Photography Traditionally

    Measuring light

    Optics focuses light on sensor

    Sensor records image

    Sensors

    Digital Film

    http://en.wikipedia.org/wiki/Single-lens_reflex_camera http://en.wikipedia.org/wiki/Digital_single-lens_reflex_camera

    http://en.wikipedia.org/wiki/Digital_single-lens_reflex_camerahttp://en.wikipedia.org/wiki/Digital_single-lens_reflex_camerahttp://en.wikipedia.org/wiki/Digital_single-lens_reflex_camerahttp://en.wikipedia.org/wiki/Digital_single-lens_reflex_camerahttp://en.wikipedia.org/wiki/Digital_single-lens_reflex_camerahttp://en.wikipedia.org/wiki/Digital_single-lens_reflex_camerahttp://en.wikipedia.org/wiki/Digital_single-lens_reflex_camera

  • Computational photography More than digital photography

    Arbitrary computation between light measurement and final image Light measured on sensor is not final image Computation enhances and extends capabilities of

    digital photography Two types of computation

    1. Post-process after traditional imaging 2. Design of new camera devices that require

    computation to form an image Overview of recent research

    http://en.wikipedia.org/wiki/Computational_photography

    http://en.wikipedia.org/wiki/Computational_photography

  • Removing imaging artifacts Denoising & deblurring

    http://www.cs.ust.hk/~quan/publications/yuan-deblur-siggraph07.pdf

    Blurry + Output Noisy Algorithm

    http://www.cs.ust.hk/~quan/publications/yuan-deblur-siggraph07.pdf

  • Removing imaging artifacts High dynamic range images & tone mapping

  • Image manipulation Panoramas

    http://en.wikipedia.org/wiki/Image_stitching

    http://en.wikipedia.org/wiki/Image_stitching

  • Computational optics

    Coded aperture

    Captured image, slightly blurry everywhere

  • Computational optics

    Recovered depth

    Refocused image Sharp foreground, blurry background

  • Focus of class Fun with digital photography and

    computer programming

    Algorithms and computational techniques with potential applications in the consumer domain

    Mostly software, less hardware

    Recent research

  • What you will learn Basic understanding of photography, light,

    and color Practical experience with implementation of

    algorithms for image processing & computational photography

    Cool and creative applications of mathematical tools Fourier transforms Linear and non linear filtering Optimization techniques (least squares, iteratively

    re-weighted least squares, graph cuts) Probabilistic models

    Many applications beyond processing images!

  • Related areas, not covered Image processing for scientific applications

    Physics, biology, etc.

    Optics, lens design

    Photosensors, sensor design

    Computational imaging

    Tomography, radar, microscopy

    3D imaging

    Using photo processing tools, e.g. Photoshop

    Artistical aspects of photography

  • Syllabus 1. Introduction, image formation 2. Color & color processing 3. Dynamic range & contrast 4. Sampling, reconstruction, & the frequency domain 5. Image restoration: denoising & deblurring 6. Image manipulation using optimization 7. Gradient domain image manipulation 8. Warping & morphing 9. Panoramas 10. Automatic alignment 11. Probabilistic image models 12. Light fields 13. Capturing light transport

    http://www.cgg.unibe.ch/teaching/computational-photography

    http://www.cgg.unibe.ch/teaching/computational-photography

  • Cameras, image artifacts

    Image formation

  • Color Color perception, color spaces, color

    measurement, color processing

  • Dynamic range & contrast HDR imaging

    http://en.wikipedia.org/wiki/High_dynamic_range_imaging http://en.wikipedia.org/wiki/Tone_mapping

    http://en.wikipedia.org/wiki/High_dynamic_range_imaginghttp://en.wikipedia.org/wiki/Tone_mapping

  • Sampling, reconstruction Sampling artifacts

    Frequency domain analysis

    Spatial Domain Frequency Domain

  • Image restoration Denoising & deblurring

    Blurry input Deblurred output

    Estimated blur kernel (scaled) http://vision.ucsd.edu/kriegman-grp/research/psf_estimation/

    http://vision.ucsd.edu/kriegman-grp/research/psf_estimation/http://vision.ucsd.edu/kriegman-grp/research/psf_estimation/http://vision.ucsd.edu/kriegman-grp/research/psf_estimation/

  • Image manipulation using optimization Photomontage, matting, colorization

    http://grail.cs.washington.edu/projects/photomontage/

    http://www.cs.huji.ac.il/~yweiss/Colorization/

    http://grail.cs.washington.edu/projects/digital-matting/image-matting/

    http://grail.cs.washington.edu/projects/photomontage/http://www.cs.huji.ac.il/~yweiss/Colorization/http://grail.cs.washington.edu/projects/digital-matting/image-matting/

  • Gradient domain manipulation Poisson equation

    http://portal.acm.org/citation.cfm?id=882269

    http://portal.acm.org/citation.cfm?id=882269

  • Warping & morphing

  • Panoramas Automatic alignment, stitching

    http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj4/www/wwedler/

    http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj4/www/wwedler/http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj4/www/wwedler/http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj4/www/wwedler/

  • Probabilistic models Faces, textures

    http://web4.cs.ucl.ac.uk/staff/j.kautz/publications/Visio_SIG09.pdf

    http://web4.cs.ucl.ac.uk/staff/j.kautz/publications/Visio_SIG09.pdf

  • Beyond 2D images

    Light fields

    http://www-graphics.stanford.edu/papers/fourierphoto/

    http://www-graphics.stanford.edu/papers/fourierphoto/http://www-graphics.stanford.edu/papers/fourierphoto/http://www-graphics.stanford.edu/papers/fourierphoto/

  • Capturing light transport Dual photography

    http://www-graphics.stanford.edu/papers/dual_photography/

    http://www-graphics.stanford.edu/papers/dual_photography/http://www-graphics.stanford.edu/papers/dual_photography/http://www-graphics.stanford.edu/papers/dual_photography/

  • Today Course organization

    Course overview

    Image formation

  • Models of light

  • Question Why is there no image on a white piece of

    paper?

  • Question Why is there no image on a white piece of

    paper?

    Receives all light rays

    Images from all viewpoints

    Need to select light rays for specifice image, viewpoint

    How?

Recommended

View more >