ee4830 digital image processing lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · ee4830...
TRANSCRIPT
![Page 1: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/1.jpg)
1
Object Recognition
April 16, 2007
Lexing Xiexlx at ee.columbia.edu
EE4830 Digital Image ProcessingLecture 11
![Page 2: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/2.jpg)
2
Announcements
� EXP#3 extended to Wed by 10am
� Problem Set #6 assigned
� One analytical question, one practical
� Due next Monday 04/23
![Page 3: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/3.jpg)
3
Roadmap to Date
Spatial Domain processing and enhancement
Image Transform and Filtering
Morphological Processing
Image Descriptors
Image Segmentation
Applicatoions:Object recognition, Image Compression Indexing and Retrieval, Reconstruction
![Page 4: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/4.jpg)
4
Lecture Outline
Problem: object recognition from images.
� What and why
� Object recognition as pattern classification
� General object recognition systems
![Page 5: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/5.jpg)
5
What is Object Recognition?
perceptibleperceptible materialmaterial
thingthing
Courtesy of http://people.csail.mit.edu/torralba/iccv2005/
![Page 6: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/6.jpg)
6
What is Object Recognition?
Sensory data Descriptions
Apple
+
One of the fundamental problems of computer vision:
Color, texture, shape, motion, size, weight, smell, touch, sound, …
“toy”, “stuffed Pooh”, “a frontal, close-up shot of stuffed Pooh”, “ToysRus#345812”, …
![Page 7: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/7.jpg)
7
Why?
� Science
� How do we recognize objects?
� Practice
� Robot navigation
� Medical diagnosis
� Security
� Industrial inspection and automation
� Human-computer interface
� Information retrieval
� …
![Page 8: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/8.jpg)
8
Applications of Object Recognition
Some images from http://www.cs.utexas.edu/~grauman/research/research.html
![Page 9: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/9.jpg)
9
The Economists, April 7th, 2007
![Page 10: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/10.jpg)
10
Lecture Outline
� Object recognition: what and why
� Object recognition as pattern classification
� Distance-based classifiers
� Neural networks
� Bayes classifiers
� Object recognition in practice
� General object recognition systems
� Summary
![Page 11: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/11.jpg)
11
Objects as Vectors …
![Page 12: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/12.jpg)
12
Minimum Distance Classifier
![Page 13: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/13.jpg)
13
Nearest Neighbor Classifier
![Page 14: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/14.jpg)
14
Nearest Neighbor Classifier
![Page 15: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/15.jpg)
15
Support Vector Machines
� Two key ideas:� Class boundary can be linear in a higher-dimensional space, e.g.,
� The “best” separating hyperplane has the largest margin.
![Page 16: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/16.jpg)
16
Neural Networks
![Page 17: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/17.jpg)
17
Digit Recognition with Neural Net
� LeCun et al, 1992, 1998, …http://yann.lecun.com/exdb/mnist/
0
1
![Page 18: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/18.jpg)
18
Bayes Classifier
![Page 19: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/19.jpg)
19
Bayes Classifier for Gaussian Classes
![Page 20: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/20.jpg)
20
![Page 21: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/21.jpg)
21
![Page 22: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/22.jpg)
22
Homework: Classifying Faces …
� Goal: learn male/female face model from images
� Steps
� Read images
� Generate corresponding label (F/M +1/-1)
� Train classifier (SVM/NN/ …)
� Estimate performance
� Leave-one-out
� “leave-one-person-out”
� Report error rate:
1st round testing samples
nth round …
![Page 23: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/23.jpg)
23
� Commercial object recognition
� Currently a $4 billion/year industry for inspection and assembly
� Almost entirely based on template matching
� Upcoming applications
� Mobile robots, toys, user interfaces
� Location recognition
� Digital camera panoramas, 3D scene modeling
Object Recognition in Practice
This section courtesy of David Lowe, website and CVPR 2003 Tutorial
![Page 24: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/24.jpg)
24
Industrial Applications
http://www.cs.ubc.ca/spider/lowe/vision.html
![Page 25: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/25.jpg)
25
http://www.appian-tech.com/
![Page 26: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/26.jpg)
26
http://www.sportvision.com/
![Page 27: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/27.jpg)
27
http://www.dipix.com/
![Page 28: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/28.jpg)
28
Lecture Outline
� Object recognition: what and why
� Object recognition as pattern classification
� General object recognition systems
� Another view of object recognition
� Real-world challenges
� Survey of state-of the art
� Summary
![Page 29: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/29.jpg)
29
Object Recognition End-to-End
Pre-processing
Feature Extraction
Sensor(s)
Classification
Post-processing
train classifier
Pre-processing
Feature Extraction
Sensor(s)
training testing
images
segments
features
classes
� window� parts or patches� salient points
� pixel vector� descriptors
� face? car?� statue of liberty?
� spatial/temporal smoothing� context, etc.
Recognition Target
![Page 30: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/30.jpg)
30
Object Category Recognition
![Page 31: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/31.jpg)
31
Demos
� Pittpatt http://demo.pittpatt.com/
![Page 32: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/32.jpg)
32It’s not just vision…
•8pm 10 Oct 05
•London
•3pm 10 Sep 05
•downloaded
from http://...
•10am 7 Sep 05
•Australian park
•Jim, Jill nearby
•4pm 8 Sep 05
•Sydney
“two koalas
seen on nat. park trip
with Jim and Jill”
“John and
his new car”
Integrate with mobile sensor information (GPS, time, nearby object or people), calendar, schedule…Infer semantically rich meta-data labels from joint sources.
“Jill and koala on
nat. park trip”
“office parking lot”
“car to consider
purchasing”
Courtesy of Kristen Grauman http://www.cs.utexas.edu/~grauman/
![Page 33: EE4830 Digital Image Processing Lecture 11xlx/courses/ee4830-sp07/notes/lec11_notes.pdf · EE4830 Digital Image Processing Lecture 11. ... smell, touch, sound, ... Microsoft PowerPoint](https://reader031.vdocuments.net/reader031/viewer/2022022015/5b566c0e7f8b9ab7348c6ff9/html5/thumbnails/33.jpg)
33
Summary
� The object recognition problem
� Object recognition as pattern classification
� Object recognition grown up
� Readings: G&W 12.1-12.2
� Reference: Duda, Hart, Stork, “Pattern Classification”, 2nd Ed.
� Next time: Image Compression
Other acknowledgements: Dan Ellis, EE6820 Slides; Duda, Hart& Stork, Pattern Classification 2nd Ed., David Claus and Christoph F. Eick: Nearest Neighbor Editing and Condensing Techniques.