spring 2019 cs543 / ece549 computer visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf ·...
TRANSCRIPT
![Page 1: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/1.jpg)
Spring 2019 CS543 / ECE549Computer Vision
Course webpage URL: http://slazebni.cs.illinois.edu/spring19/
![Page 2: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/2.jpg)
The goal of computer vision• To extract “meaning” from pixels
What we see What a computer seesSource: S. Narasimhan
![Page 3: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/3.jpg)
The goal of computer vision• To extract “meaning” from pixels
Source: “80 million tiny images” by Torralba et al.
Humans are remarkably good at this…
![Page 4: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/4.jpg)
What kind of information can be extracted from an image?
…
![Page 5: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/5.jpg)
What kind of information can be extracted from an image?
Geometric information
…
![Page 6: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/6.jpg)
What kind of information can be extracted from an image?
Geometric informationSemantic information
building
persontrashcan car car
ground
tree tree
sky
doorwindow
building
roof
chimney
Outdoor sceneCity European
…
![Page 7: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/7.jpg)
What can computer vision do today?
![Page 8: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/8.jpg)
Reconstruction: 3D from photo collections
YouTube Video
Q. Shan, R. Adams, B. Curless, Y. Furukawa, and S. Seitz, The Visual Turing Test for Scene Reconstruction, 3DV 2013
![Page 9: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/9.jpg)
Reconstruction: 4D from photo collections
YouTube Video
R. Martin-Brualla, D. Gallup, and S. Seitz, Time-Lapse Mining from Internet Photos, SIGGRAPH 2015
![Page 10: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/10.jpg)
Reconstruction: 4D from depth cameras
YouTube Video
R. Newcombe, D. Fox, and S. Seitz, DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time,
CVPR 2015
![Page 11: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/11.jpg)
Reconstruction in construction industry
reconstructinc.com Source: D. Hoiem
![Page 12: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/12.jpg)
Recognition: “Simple” patterns
![Page 13: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/13.jpg)
Recognition: Faces
![Page 14: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/14.jpg)
Concerns about face recognition
Beijing bets on facial recognition in a big drive for total surveillance – Washington Post, 1/8/2018
![Page 15: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/15.jpg)
Concerns about face recognition
https://www.newyorker.com/magazine/2018/12/17/should-we-be-worried-about-computerized-facial-recognition
![Page 16: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/16.jpg)
Recognition: General categories
• Computer Eyesight Gets a Lot More Accurate, NY Times Bits blog, August 18, 2014
• Building A Deeper Understanding of Images, Google Research Blog, September 5, 2014
![Page 18: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/18.jpg)
Object detection, instance segmentation
K. He, G. Gkioxari, P. Dollar, and R. Girshick, Mask R-CNN, ICCV 2017 (Best Paper Award)
![Page 19: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/19.jpg)
Image generation• Faces: 1024x1024 resolution, CelebA-HQ
dataset
T. Karras, T. Aila, S. Laine, and J. Lehtinen, Progressive Growing of GANs for Improved Quality, Stability, and Variation, ICLR 2018
Follow-up work
![Page 20: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/20.jpg)
Image generation• BigGAN: 512 x 512 resolution, ImageNet
A. Brock, J. Donahue, K. Simonyan, Large scale GAN training for high fidelity natural image synthesis, arXiv 2018
![Page 21: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/21.jpg)
Image generation• BigGAN: 512 x 512 resolution, ImageNet
A. Brock, J. Donahue, K. Simonyan, Large scale GAN training for high fidelity natural image synthesis, arXiv 2018
![Page 22: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/22.jpg)
Image generation• BigGAN: 512 x 512 resolution, ImageNet
A. Brock, J. Donahue, K. Simonyan, Large scale GAN training for high fidelity natural image synthesis, arXiv 2018
Easy classes Difficult classes
![Page 23: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/23.jpg)
Image generation• Image-to-image translation
P. Isola, J.-Y. Zhu, T. Zhou, A. Efros, Image-to-Image Translation with Conditional Adversarial Networks, CVPR 2017
![Page 24: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/24.jpg)
Image generation• Unpaired image-to-image translation
J.-Y. Zhu, T. Park, P. Isola, A. Efros, Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, ICCV 2017
![Page 25: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/25.jpg)
M.-Y. Liu, T. Breuel, and J. Kautz, Unsupervised Image-to-Image Translation Networks, NIPS 2017
Unsupervised image-to-image translation
![Page 26: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/26.jpg)
Unsupervised image-to-image translation
M.-Y. Liu, T. Breuel, and J. Kautz, Unsupervised Image-to-Image Translation Networks, NIPS 2017
![Page 27: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/27.jpg)
DeepFakes• Deep video portraits
• “A quiet wager has taken hold among researchers who study artificial intelligence techniques and the societal impacts of such technologies. They’re betting whether or not someone will create a so-called Deepfake video about a political candidate that receives more than 2 million views before getting debunked by the end of 2018” – IEEE Spectrum, 6/22/2018
![Page 28: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/28.jpg)
DeepFakes
https://www.newyorker.com/magazine/2018/11/12/in-the-age-of-ai-is-seeing-still-believing
![Page 29: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/29.jpg)
Origins of computer vision
L. G. Roberts, Machine Perception of Three Dimensional Solids,Ph.D. thesis, MIT Department of Electrical Engineering, 1963.
![Page 30: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/30.jpg)
Origins of computer vision
![Page 31: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/31.jpg)
Connections to other disciplines
Computer Vision
Image Processing
Machine Learning
Artificial Intelligence
Robotics
Cognitive scienceNeuroscience
Computer Graphics
![Page 32: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/32.jpg)
Growth of the field
Long list of corporate sponsors
Source
![Page 33: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/33.jpg)
Course overviewI. Early vision: Image formation and
processingII. Mid-level vision: Grouping and fittingIII. Multi-view geometryIV. RecognitionV. Additional topics
![Page 34: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/34.jpg)
I. Early visionBasic image formation and processing
Cameras and sensorsLight and color
Linear filteringEdge detection
* =
Feature extraction Optical flow
![Page 35: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/35.jpg)
II. “Mid-level vision”Fitting and grouping
Fitting: Least squaresVoting methods
Alignment
![Page 36: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/36.jpg)
III. Multi-view geometry
Structure from motion
Two-view stereoEpipolar geometry
Multi-view stereo
![Page 37: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/37.jpg)
IV. Recognition
Basic classification
Object detection
Deep learning
Segmentation
![Page 38: Spring 2019 CS543 / ECE549 Computer Visionslazebni.cs.illinois.edu/spring19/lec01_intro.pdf · Spring 2019 CS543 / ECE549 Computer Vision Course webpage URL: ... Computer Vision Image](https://reader034.vdocuments.net/reader034/viewer/2022042312/5edace353ee6fd189f66e5f1/html5/thumbnails/38.jpg)
V. Additional Topics (time permitting)
Video
3D scene understanding Images and text
Generation