15-463 (15-862): computational photography. staff prof: alexei efros (efros@cs), 4207 nsh (for...
Post on 21-Dec-2015
220 views
TRANSCRIPT
![Page 1: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/1.jpg)
15-463 (15-862): Computational Photography
![Page 2: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/2.jpg)
15-463 (15-862): Computational Photography
Staff• Prof: Alexei Efros (efros@cs), 4207 NSH (for now)• TA: Ronit Slyper (rys@cs)
Web Page• http://graphics.cs.cmu.edu/courses/15-463/
Discussion Forum:• Googlegroups ???
![Page 3: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/3.jpg)
Today
Introductions
Why Computational Photography?
Overview of the course
Administrative stuff
![Page 4: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/4.jpg)
A bit about me
Alexei (Alyosha) EfrosAssociate Professor in Robotics and CSD
also work with colleagues in Paris and Oxford
Teaching
The plan is to have fun and learn cool things, both you and me!
Social warning: I don’t see well
ResearchGraphics, Vision, Machine Learning, Visual Perception
![Page 5: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/5.jpg)
PhD Thesis on Texture and Action Synthesis
Antonio’s son cannot walk but he can fly
Smart Erase button in MS Digital Image Pro:
![Page 6: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/6.jpg)
More recent work
Derek Hoiem, Alexei Efros, Martial Hebert
![Page 7: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/7.jpg)
![Page 8: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/8.jpg)
![Page 9: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/9.jpg)
![Page 10: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/10.jpg)
![Page 11: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/11.jpg)
![Page 12: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/12.jpg)
Ronit (row-neat)
![Page 13: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/13.jpg)
What is Computational Photography?
• “computational techniques that enhance or extend photography”
• Another medium for visual expression
![Page 14: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/14.jpg)
Depicting Our World: The Beginning
Prehistoric Painting, Lascaux Cave, France~ 13,000 -- 15,000 B.C.
![Page 15: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/15.jpg)
The Empress Theodora with her court. Ravenna, St. Vitale 6th c.
Depicting Our World: Middle Ages
![Page 16: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/16.jpg)
Nuns in Procession. French ms. ca. 1300.
Depicting Our World: Middle Ages
![Page 17: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/17.jpg)
Depicting Our World: RenaissanceEast Doors (1452)North Doors (1424)
Lorenzo Ghiberti(1378-1455)
![Page 18: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/18.jpg)
Depicting Our World: Renaissance
Piero della Francesca,The Flagellation (c.1469)
![Page 19: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/19.jpg)
Depicting Our World: Toward Perfection
Jan van Eyck, The Arnolfini Marriage (c.1434)
![Page 20: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/20.jpg)
Depicting Our World: Toward Perfection
Lens Based Camera Obscura, 1568
![Page 21: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/21.jpg)
Depicting Our World: Perfection!
Still Life, Louis Jaques Mande Daguerre, 1837
![Page 22: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/22.jpg)
Depicting Our World: Realism?
![Page 23: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/23.jpg)
Paris, according to
Flickr
![Page 24: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/24.jpg)
Paris, according to GoogleStreetView
Knopp, Sivic, Pajdla, ECCV 2010
![Page 25: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/25.jpg)
Paris, according to me
![Page 26: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/26.jpg)
Depicting Our World: Ongoing Quest
Pablo Picasso David Hockney
![Page 27: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/27.jpg)
Better than realism?
David Hockney, Place Furstenberg, (1985)
![Page 28: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/28.jpg)
Alyosha EfrosPlace Furstenberg, 2009
Which one is right?
Multiple viewpoints Single viewpoint
David Hockney, Place Furstenberg, 1985
![Page 29: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/29.jpg)
Depicting Our World: Ongoing Quest
Antonio Torralba & Aude Oliva (2002)
![Page 30: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/30.jpg)
Enter Computer Graphics...
![Page 31: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/31.jpg)
GRAPHICS
Traditional Computer Graphics
3D geometry
physics
Simulation
projection
![Page 32: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/32.jpg)
State of the Art
•Amazingly real•But so sterile, lifeless, futuristic (why?)
![Page 33: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/33.jpg)
The richness of our everyday world
Photo by Svetlana Lazebnik
![Page 34: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/34.jpg)
Beauty in complexity
University Parks, Oxford
![Page 35: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/35.jpg)
Which parts are hard to model?
Photo by Svetlana Lazebnik
![Page 36: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/36.jpg)
People
From “Final Fantasy”
On the Tube, London
![Page 37: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/37.jpg)
Faces / Hair
Photo by Joaquin Rosales Gomez
From “Final Fantasy”
![Page 38: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/38.jpg)
Hyper-humans
![Page 39: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/39.jpg)
Urban Scenes
Virtual LA (SGI)
Photo of l LA
![Page 40: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/40.jpg)
Nature
River Cherwell, Oxford
![Page 41: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/41.jpg)
The Realism Spectrum
+ easy to create new worlds
+ easy to manipulate objects/viewpoint
- Very hard to look realistic
+ instantly realistic
+ easy to aquire
- very hard to manipulate objects/viewpoint
Computer Graphics PhotographyComputationalPhotography
RealismManipulationEase of capture
![Page 42: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/42.jpg)
Virtual Real World
Campanile Moviehttp://www.debevec.org/Campanile/
![Page 43: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/43.jpg)
Course Outline
![Page 44: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/44.jpg)
Programming Project 0The Vertigo Effect
![Page 45: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/45.jpg)
Programming Project 1
Images of the Russian Empire -- colorizing the Prokudin-Gorskii photo collection
![Page 46: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/46.jpg)
Programming Project 2
TBD
![Page 47: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/47.jpg)
Programming Project 3
Face warping and morphing
![Page 48: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/48.jpg)
Programming Project 4
Photo Mosaics
Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
2003 New Years Eve: http://www.panoramas.dk/fullscreen3/f1.html
![Page 49: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/49.jpg)
Programming Project 4
Automatic Mosaic Stitching
![Page 50: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/50.jpg)
Programming Project 5
Tour Into the Picture
![Page 51: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/51.jpg)
Final Project
Something cool!!!
![Page 52: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/52.jpg)
Administrative Stuff
Grading• Written and Programming Assignments (60%)• Exam (20%) Nov 16• Final Project (20%) • Class Participation: priceless
Late Policy• Five late days total, to be spent wisely• 20% off from each extra late day
Cheating• Let’s not embarrass ourselves
Hardware/Software• CMU clusters• MATLAB
![Page 53: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/53.jpg)
General CommentsPrerequisites
• Linear algebra!!!• Some computer graphics, vision, or image processing is
useful, but not required.
Emphasis on programming projects!• Building something from scratch
Graduate Version:• Need to do more on each project, plus a final paper
Misc:• No laptops, no cell phones, smartphones, etc.
![Page 54: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/54.jpg)
A Fair Warning…
Reasons not to take the class:• Lots of work• Need time to think, not just follow instructions• Not worth it if you don’t enjoy it
Reasons to take the class:• It’s your reward after 3 grueling years • You get to create pictures, unleash your
creative potential• Interested in grad school?
![Page 55: 15-463 (15-862): Computational Photography. Staff Prof: Alexei Efros (efros@cs), 4207 NSH (for now)@cs TA: Ronit Slyper (rys@cs) Web Page](https://reader036.vdocuments.net/reader036/viewer/2022062714/56649d5d5503460f94a3be0c/html5/thumbnails/55.jpg)
Cameras
Really cool
Not too expensive nowadays (<$150)
e.g. Canon A1100