soen6761 multimedia computing
TRANSCRIPT
![Page 1: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/1.jpg)
Copyright © Charalambos Poullis
SOEN6761MULTIMEDIA COMPUTING
LECTURE 5IMAGE SAMPLING
![Page 2: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/2.jpg)
Copyright © Charalambos Poullis
Today’s Lecture
• Image Sampling• Slides acknowledgment: A. Farhadi, S. Seitz
• Questions
![Page 3: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/3.jpg)
Image Sampling
Moire patterns - http://www.sandlotscience.com/Moire/Circular_3_Moire.htm
![Page 4: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/4.jpg)
Image Sampling
![Page 5: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/5.jpg)
Image Scaling
This image is too big tofit on the screen. Howcan we reduce it?
How to generate a half-sized version?
![Page 6: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/6.jpg)
Image sub-sampling
Throw away every other row and column to create a 1/2 size image- called image sub-sampling
![Page 7: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/7.jpg)
Image sub-sampling
Throw away every other row and column to create a 1/2 size image- called image sub-sampling
1/4
![Page 8: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/8.jpg)
Image sub-sampling
Throw away every other row and column to create a 1/2 size image- called image sub-sampling
1/41/8
![Page 9: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/9.jpg)
Image sub-sampling
1/4 (2x zoom) 1/8 (4x zoom)
Why is this a bad effect?
1/2
![Page 10: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/10.jpg)
Image sub-sampling
Image credit: F. Durand
Why is this a bad effect?
![Page 11: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/11.jpg)
Even worse for synthetic images
![Page 12: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/12.jpg)
Down-sampling
Aliasing can arise when you sample a continuous signal or image● occurs when your sampling rate is not high enough to capture the
amount of detail in your image● Can give you the wrong signal/image—an alias● formally, the image contains structure at different scales
○ called “frequencies” in the Fourier domain● the sampling rate must be high enough to capture the highest frequency
in the image
![Page 13: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/13.jpg)
Sampling and the Nyquist rate
● To avoid aliasing:○ sampling rate ≥ 2 * max frequency in the image
■ said another way: ≥ two samples per cycle○ This minimum sampling rate is called the Nyquist rate
![Page 14: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/14.jpg)
http://www.michaelbach.de/ot/mot-wagonWheel/index.html
![Page 15: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/15.jpg)
2D example
![Page 16: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/16.jpg)
2D example
Good sampling
Bad sampling
![Page 17: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/17.jpg)
How to fix this?
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 18: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/18.jpg)
Subsampling with Gaussian pre-filtering
Gaussian 1/2Solution: filter the image, then subsample• Filter size should double for each ½ size reduction. Why?
![Page 19: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/19.jpg)
Subsampling with Gaussian pre-filtering
G 1/4
Gaussian 1/2Solution: filter the image, then subsample• Filter size should double for each ½ size reduction. Why?
![Page 20: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/20.jpg)
Subsampling with Gaussian pre-filtering
G 1/4G 1/8
Gaussian 1/2Solution: filter the image, then subsample• Filter size should double for each ½ size reduction. Why?
![Page 21: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/21.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2Solution: filter the image, then subsample• Why does this work?
![Page 22: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/22.jpg)
Compare with...
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 23: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/23.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Solution: filter the image, then subsample• Filter size should double for each ½ size reduction. Why?• How can we speed this up?
![Page 24: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/24.jpg)
Gaussian pre-filtering• Solution: filter
the image, then subsample
blur subsample …blur subsample
F0
F0 * H
F1
F1 * H
F2
![Page 25: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/25.jpg)
Gaussian pyramid
blur subsample …blur subsample
F0
F0 * H
F1
F1 * H
F2
![Page 26: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/26.jpg)
The Gaussian Pyramid
High resolution
Low resolution
![Page 27: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/27.jpg)
Gaussian Pyramid
![Page 28: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/28.jpg)
expand
expand
expand
Gaussian Pyramid Laplacian Pyramid
The Laplacian Pyramid
- =
- =- =
![Page 29: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/29.jpg)
Some times we want many resolutions
Known as a Gaussian Pyramid [Burt and Adelson, 1983]• In computer graphics, a mip map [Williams, 1983]• A precursor to wavelet transformGaussian Pyramids have all sorts of applications in computer vision• We’ll talk about these later in the course
![Page 30: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/30.jpg)
Gaussian pyramid construction
filter kernel
Repeat• Filter• SubsampleUntil minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid)
The whole pyramid is only 4/3 the size of the original image!
![Page 31: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/31.jpg)
Moire patterns in real-world images. Here are comparison images by Dave Etchells of Imaging Resource using the Canon D60 (with an antialias filter) and the Sigma SD-9 (which has no antialias filter). The bands below the fur in the image at right are the kinds of artifacts that appear in images when no antialias filter is used. Sigma chose to eliminate the filter to get more sharpness, but the resulting apparent detail may or may not reflect features in the image.
![Page 32: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/32.jpg)
More examples
![Page 33: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/33.jpg)
Up-samplingHow do we compute the values of pixels at fractional positions?
![Page 34: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/34.jpg)
Up-samplingHow do we compute the values of pixels at fractional positions?
![Page 35: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/35.jpg)
Up-samplingHow do we compute the values of pixels at fractional positions?
Bilinear sampling:
Bicubic sampling fits a higher order function using a larger area of support
![Page 36: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/36.jpg)
Up-sampling methods
![Page 37: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/37.jpg)
Up-sampling
nearest neighbour bilinear bicubic
![Page 38: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/38.jpg)
Up-sampling
nearest neighbour bilinear bicubic
![Page 39: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/39.jpg)
Copyright © Charalambos Poullis
![Page 40: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/40.jpg)
Image resamplingSo far, we considered only power-of-two subsampling
• What about arbitrary scale reduction?• How can we increase the size of the image?
Recall how a digital image is formed
• It is a discrete point-sampling of a continuous function• If we could somehow reconstruct the original function, any new
image could be generated, at any resolution and scale
1 2 3 4 5
d = 1 in this example
![Page 41: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/41.jpg)
Image resamplingSo far, we considered only power-of-two subsampling
• What about arbitrary scale reduction?• How can we increase the size of the image?
Recall how a digital image is formed
• It is a discrete point-sampling of a continuous function• If we could somehow reconstruct the original function, any new
image could be generated, at any resolution and scale
1 2 3 4 5
d = 1 in this example
![Page 42: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/42.jpg)
Image resamplingSo what to do if we don’t know
1 2 3 4 52.5
1 d = 1 in this example
• Answer: guess an approximation• Can be done in a principled way: filtering
Image reconstruction• Convert to a continuous function • Reconstruct by cross-correlation:
![Page 43: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/43.jpg)
Resampling filtersWhat does the 2D version of this hat function look like?
Simpler implementation of bilinear interpolationBetter filters give better resampled images• Bicubic is common choice• fit 3rd degree polynomial surface to pixels in neighborhood• can also be implemented by a convolution
performs linear interpolation
(tent function) performs bilinear interpolation
![Page 44: SOEN6761 MULTIMEDIA COMPUTING](https://reader031.vdocuments.net/reader031/viewer/2022013001/61cbd67d03c081274b143ff4/html5/thumbnails/44.jpg)
Copyright © Charalambos Poullis