cs 1674: intro to computer...
TRANSCRIPT
![Page 1: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/1.jpg)
CS 1674: Intro to Computer Vision
Texture Representation + Image Pyramids
Prof. Adriana KovashkaUniversity of Pittsburgh
September 14, 2016
![Page 2: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/2.jpg)
Reminders/Announcements
• HW2P due tonight, 11:59pm
• HW3W, HW3P out
• Shuffle!
![Page 3: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/3.jpg)
Plan for today
• Filtering
– Representing texture
– Application to subsampling
– Image pyramids
• Detecting interesting content (start)
![Page 4: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/4.jpg)
Convolution vs. correlationConvolution
Cross-correlation
![Page 5: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/5.jpg)
Texture
What defines a texture?
Kristen Grauman
![Page 6: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/6.jpg)
Includes: more regular patterns
Kristen Grauman
![Page 7: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/7.jpg)
Includes: more random patterns
Kristen Grauman
![Page 8: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/8.jpg)
Kristen Grauman
![Page 9: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/9.jpg)
http://animals.nationalgeographic.com/Kristen Grauman
![Page 10: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/10.jpg)
Why analyze texture?
• Important for how we perceive objects
• Often indicative of a material’s properties
• Can be important appearance cue, especially if shape is similar across objects
• To represent objects, we want a feature one step above “building blocks” of filters, edges
Adapted from Kristen Grauman
![Page 11: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/11.jpg)
Texture representation
• Textures are made up of repeated local patterns, so:
– Find the patterns
• Use filters that look like patterns (spots, bars, raw patches…)
• Consider magnitude of response
– Describe their statistics within each local window
• Mean, standard deviation
• Histogram
Kristen Grauman
![Page 12: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/12.jpg)
Texture representation: example
original image
derivative filter responses, squared
statistics to summarize patterns in small
windows
mean d/dxvalue
mean d/dyvalue
Win. #1 4 10
…
Kristen Grauman
![Page 13: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/13.jpg)
Texture representation: example
original image
derivative filter responses, squared
statistics to summarize patterns in small
windows
mean d/dxvalue
mean d/dyvalue
Win. #1 4 10
Win.#2 18 7
…
Kristen Grauman
![Page 14: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/14.jpg)
Texture representation: example
original image
derivative filter responses, squared
statistics to summarize patterns in small
windows
mean d/dxvalue
mean d/dyvalue
Win. #1 4 10
Win.#2 18 7
Win.#9 20 20
…
…
Kristen Grauman
![Page 15: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/15.jpg)
Texture representation: example
statistics to summarize patterns in small
windows
mean d/dxvalue
mean d/dyvalue
Win. #1 4 10
Win.#2 18 7
Win.#9 20 20
…
…
Dimension 1 (mean d/dx value)
Dim
en
sio
n 2
(m
ean
d/d
y va
lue
)
Kristen Grauman
![Page 16: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/16.jpg)
Texture representation: example
statistics to summarize patterns in small
windows
mean d/dxvalue
mean d/dyvalue
Win. #1 4 10
Win.#2 18 7
Win.#9 20 20
…
…
Dimension 1 (mean d/dx value)
Dim
en
sio
n 2
(m
ean
d/d
y va
lue
)
Windows with small gradient in both directions
Windows with primarily vertical edges
Windows with primarily horizontal edges
Both
Kristen Grauman
![Page 17: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/17.jpg)
Texture representation: example
original image
derivative filter responses, squared
visualization of the assignment to texture
“types”
Kristen Grauman
![Page 18: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/18.jpg)
Texture representation: example
statistics to summarize patterns in small
windows
mean d/dxvalue
mean d/dyvalue
Win. #1 4 10
Win.#2 18 7
Win.#9 20 20
…
…
Dimension 1 (mean d/dx value)
Dim
en
sio
n 2
(m
ean
d/d
y va
lue
)
Far: dissimilar textures
Close: similar textures
Kristen Grauman
![Page 19: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/19.jpg)
Computing distances using texture
Dimension 1
Dim
en
sio
n 2
a
b
dim#
1
2
222
211
)(),(
)()(),(
i
ii babaD
bababaD
Kristen Grauman
![Page 20: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/20.jpg)
Texture representation: example
Dimension 1
Dim
en
sio
n 2
a
b
a
b
Distance reveals how dissimilar texture from window a is from texture in window b.
b
Kristen Grauman
![Page 21: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/21.jpg)
Filter banks
• Our previous example used two filters, and resulted in a 2-dimensional feature vector to describe texture in a window.
– x and y derivatives revealed something about local structure.
• We can generalize to apply a collection of multiple (d) filters: a “filter bank”
• Then our feature vectors will be d-dimensional.
– still can think of nearness, farness in feature space
Adapted from Kristen Grauman
![Page 22: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/22.jpg)
Filter banks
• What filters to put in the bank?
– Typically we want a combination of scales and orientations, different types of patterns.
Matlab code available for these examples: http://www.robots.ox.ac.uk/~vgg/research/texclass/filters.html
scales
orientations
“Edges” “Bars”
“Spots”
Kristen Grauman
![Page 23: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/23.jpg)
Filter bank
Kristen Grauman
![Page 24: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/24.jpg)
Multivariate Gaussian
90
09
90
016
55
510
Kristen Grauman
![Page 25: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/25.jpg)
Imag
e fr
om
htt
p:/
/ww
w.t
exas
exp
lore
r.co
m/a
ust
inca
p2
.jp
g
Kristen Grauman
![Page 26: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/26.jpg)
Showing magnitude of responses
Kristen Grauman
![Page 27: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/27.jpg)
Kristen Grauman
![Page 28: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/28.jpg)
Kristen Grauman
![Page 29: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/29.jpg)
Kristen Grauman
![Page 30: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/30.jpg)
Kristen Grauman
![Page 31: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/31.jpg)
Kristen Grauman
![Page 32: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/32.jpg)
Kristen Grauman
![Page 33: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/33.jpg)
[r1, r2, …, r38]
We can form a “feature vector” from the list of responses at each pixel; gives us a representation of the pixel, image.
Adapted from Kristen Grauman
Vectors of texture responses
![Page 34: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/34.jpg)
You try: Can you match the texture
to the response?
Mean abs responses
FiltersA
B
C
1
2
3
Derek Hoiem
![Page 35: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/35.jpg)
Representing texture by mean abs
response
Mean abs responses
Filters
Derek Hoiem
![Page 36: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/36.jpg)
Vectors of texture responses
• The frequency of image patch responses in an image tells us something about the image
• If my patches tend to respond strongly to blobs, and another image’s patches respond strongly to vertical edges, then that image and I are probably of different materials
• How to capture? • One idea: Concatenate [r1, …, r38] from the
previous slide, for all image pixels• Problems?
![Page 37: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/37.jpg)
Means or histograms of feature responses
1) Instead of concatenating, compute the mean across all image pixels to each of the filters
2) Instead of concatenating, count:– Assume all responses between 0 and 1
– How many times within the image did I see an r1 response between 0 and 0.1?
– … between 0.1 and 0.2?
– How many times did I see r2 response between 0 and 0.1?
– …
– Concatenate these counts into a histogram
• Tradeoffs?
![Page 38: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/38.jpg)
Classifying materials, “stuff”
Figure by Varma & Zisserman
Kristen Grauman
![Page 39: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/39.jpg)
1-minute break
![Page 40: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/40.jpg)
Plan for today
• Filtering
– Representing texture
– Application to subsampling
– Image pyramids
• Detecting interesting content (start)
![Page 41: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/41.jpg)
Why does a lower resolution image still make sense to us? What do we lose?
Image: http://www.flickr.com/photos/igorms/136916757/
Sampling
Derek Hoiem
![Page 42: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/42.jpg)
Throw away every other row and column to create a 1/2 size image
Subsampling by a factor of 2
Derek Hoiem
![Page 43: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/43.jpg)
Aliasing problem
• 1D example (sinewave):
Source: S. Marschner
![Page 44: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/44.jpg)
Source: S. Marschner
Aliasing problem
• 1D example (sinewave):
![Page 45: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/45.jpg)
Aliasing problem
• Sub-sampling may be dangerous….
• Characteristic errors may appear:
– “Wagon wheels rolling the wrong way in movies”
– “Striped shirts look funny on color television”
Source: D. Forsyth
![Page 46: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/46.jpg)
Sampling and aliasing
Derek Hoiem
![Page 47: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/47.jpg)
Nyquist-Shannon Sampling Theorem
• When sampling a signal at discrete intervals, the sampling frequency must be 2 fmax
• fmax = max frequency of the input signal
• This will allows to reconstruct the original perfectly from the sampled version
good
bad
v v v
Derek Hoiem
![Page 48: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/48.jpg)
Anti-aliasing
Solutions:
• Sample more often
• Get rid of all frequencies that are greater than half the new sampling frequency
– Will lose information
– But it’s better than aliasing
– Apply a smoothing filter
Derek Hoiem
![Page 49: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/49.jpg)
Algorithm for downsampling by factor of 2
1. Start with image(h, w)
2. Apply low-pass filter
im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1))
3. Sample every other pixel
im_small = im_blur(1:2:end, 1:2:end);
Low-Pass Filtered Image
Image
Gaussian
Filter SampleLow-Res Image
Derek Hoiem
![Page 50: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/50.jpg)
Forsyth and Ponce 2002
Anti-aliasing
![Page 51: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/51.jpg)
Subsampling without pre-filtering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Slide by Steve Seitz
![Page 52: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/52.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Slide by Steve Seitz
![Page 53: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/53.jpg)
Subsampling away…
http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html
Can we reconstruct the original
from the Laplacian pyramid?
h2 = f2{f0 , f1 , …, fn} = Gaussian pyramid
{h0 , h1 , …, hn} = Laplacian pyramid
f0 – l0 = h0
f1 – l1 = h1
Derek Hoiem
Why would we want to do this?
![Page 54: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/54.jpg)
Gaussian pyramid
Source: Forsyth
![Page 55: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/55.jpg)
Laplacian pyramid
Source: Forsyth
![Page 56: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/56.jpg)
Laplacian filter
Gaussianunit impulse
Laplacian of Gaussian
Source: Lazebnik
![Page 57: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/57.jpg)
Plan for today
• Filtering
– Representing texture
– Application to subsampling
– Image pyramids
• Detecting interesting content (start)
![Page 58: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/58.jpg)
An image is a set of pixels…
What we see What a computer sees
Source: S. NarasimhanAdapted from S. Narasimhan
![Page 59: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/59.jpg)
Problems with pixel representation
• Not invariant to small changes
– Translation
– Illumination
– etc.
• Some parts of an image are more important than others
• What do we want to represent?
![Page 60: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/60.jpg)
Human eye movements
Yarbus eye tracking
D. Hoiem
![Page 61: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/61.jpg)
Choosing distinctive interest(ing) points
• If you wanted to meet a friend would you saya) “Let’s meet on campus.”
b) “Let’s meet on Green street.”
c) “Let’s meet at Green and Wright.”
– Corner detection
• Or if you were in a secluded area:a) “Let’s meet in the Plains of Akbar.”
b) “Let’s meet on the side of Mt. Doom.”
c) “Let’s meet on top of Mt. Doom.”
– Blob (valley/peak) detection
D. Hoiem
![Page 62: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/62.jpg)
Choosing interest(ing) points
Where would you tell your friend to meet you?
D. Hoiem
![Page 63: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/63.jpg)
Choosing interest(ing) points
Where would you tell your friend to meet you?
D. Hoiem
![Page 64: CS 1674: Intro to Computer Visionpeople.cs.pitt.edu/~kovashka/cs1674_fa16/vision_05_texture_pyramids.pdfConvolution vs. correlation Convolution Cross-correlation. Texture What defines](https://reader033.vdocuments.net/reader033/viewer/2022060903/609f617cca84e0063226b131/html5/thumbnails/64.jpg)
Interest points
• Suppose you have to click on some point, go away and come back after I deform the image, and click on the same points again.
– Which points would you choose?
original
deformed
D. Hoiem