image processing xuejin chen [email protected] ref:
TRANSCRIPT
![Page 1: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/1.jpg)
Image Processing
Xuejin [email protected]
Ref: http://fourier.eng.hmc.edu/e161/lectures/gradient/node10.html
![Page 2: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/2.jpg)
Image processing operations
Original image Increased contrast Change in hue
Posterized (quantized colors)
Blurred Rotated
![Page 3: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/3.jpg)
Point operators
• Only depends on the pixel value– Plus, potentially, some globally collected
information or parameters
• Brightness scaling • Image composition
– Matting – Blending
• Histogram equalization
![Page 4: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/4.jpg)
Linear Filter
• Smoothing– Box, Bilinear, Gaussian
![Page 5: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/5.jpg)
Linear Filter
• Smoothing– Box, Bilinear, Gaussian
• Edge – Sobel
![Page 6: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/6.jpg)
Linear Filter
• Smoothing– Box, Bilinear, Gaussian
• Edge – Sobel
• Corner
![Page 7: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/7.jpg)
Separable Filter
• Convolution of K-size kernel requires K2 operations
• Can be sped up to 2K operations by – First performing a 1D horizontal convolution – Followed by a 1D vertical convolution
TK vh
![Page 8: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/8.jpg)
Separable Filter
![Page 9: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/9.jpg)
• Fourier transformation
![Page 10: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/10.jpg)
Band-pass filters
• More sophisticated kernels – 1 Smoothing with a Gaussian filter
– 2 Taking first or second derivatives • Sobel • Laplacian • Coner
2 2
222
1( , ; )
2
x y
G x y e
![Page 11: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/11.jpg)
Band-pass filters
• Undirected second derivatives: Laplacian operator
• Laplacian of Gaussian (LoG) filter
– Five point Laplacian
2 22
2 2
f ff
x y
2 22
4 2
2( , ; ) ( , ; )
x yG x y G x y
![Page 12: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/12.jpg)
Steerable Filters
• Directional/Oriented filter– Sobel– Directional derivative
A whole family of filters can be evaluated with very little cost by first convolving the image with (Gx, Gy)
![Page 13: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/13.jpg)
Steerable Filters
• Second-order filter
For directional Gaussian derivatives, it is possible to steer any order of derivative with a relatively small number of basis functions.
![Page 14: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/14.jpg)
Steerable Filters
• Second-order filter
Original image orientation map Original image with oriented structures enhanced.
![Page 15: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/15.jpg)
Steerable Filters
• Fourth-order steerable filter
test image -bars (lines) -step edges -different orientations
average orientedenergy
dominant orientation
oriented energy as a function of angle
(Freeman and Adelson 1991)
![Page 16: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/16.jpg)
Summed Area Table (Integral Image)
• Repeatedly convolved with different box filters– different sizes at – different locations
• Precompute the summed area table (Crow1984)
![Page 17: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/17.jpg)
Summed Area Table (Integral Image)
• Compute the sum of any rectangle area easily
Recursive filtering
![Page 18: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/18.jpg)
Laplacian of Gaussian (LoG)
![Page 19: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/19.jpg)
LoG
• Discrete convolution kernel– Can be any size– Sum_elements = Zero
2 2
22 2 2 2 2
22 2 4
2( , ) ( , ) ( , )
x yx y
LoG G x y G x y G x y ex y
2 22
4 2
2( , ) ( , )
x yG x y G x y
![Page 20: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/20.jpg)
Laplacian of Gaussian (LoG)
![Page 21: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/21.jpg)
Difference of Gaussian (DoG)
• Gaussian
• DoG
2 2
21
1
1
2
2
1( , )
2
x y
G x y e
11( , ) ( , ) ( , )g x y G x y f x y 22 ( , ) ( , ) ( , )g x y G x y f x y
1 2
1 2
1 2( , ) ( , ) ( , ) ( , )
( ) ( , )
g x y g x y G f x y G f x y
G G f x y
2 2 2 2
2 21 2
1 2
1 2
2 2
2 2
1 1 1
2
x y x y
DoG G G e e
![Page 22: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/22.jpg)
Difference of Gaussian (DoG)
![Page 23: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/23.jpg)
Difference of Gaussian (DoG)
![Page 24: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/24.jpg)
LoG and DoG
![Page 25: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/25.jpg)
LoG and DoG2 2
24 2
2( , ) ( , )
x yG x y G x y
2 2
2
22
1( , )
2
x y
G x y e
2 2 2 2 2 2
2 2 22 2 2 2
22 2 23 5 4 2 2
1 2 1
2 2
x y x y x yG x y x y
e e e G
![Page 26: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/26.jpg)
LoG and DoG2 2
24 2
2( , ) ( , )
x yG x y G x y
2 2
2
22
1( , )
2
x y
G x y e
2 2 2 2 2 2
2 2 22 2 2 2
22 2 23 5 4 2 2
1 2 1
2 2
x y x y x yG x y x y
e e e G
2GG
![Page 27: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/27.jpg)
LoG and DoG
2 2
( , , ) ( , , )
( 1)
( , , ) ( , , ) ( 1) ( 1)
G G x y k G x y
k
GG x y k G x y k k G
2GG
DoG LoG
![Page 28: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/28.jpg)
Laplacian for Edge
• Zero-crossing detection
LoG DoG
![Page 29: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/29.jpg)
Pyramids
• Change resolution – Upsampling (Interpolation)– Downsmapling (Decimation)
![Page 30: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/30.jpg)
Interpolation
• Interpolation kernel h() with sampling rate r
• Bilinear
,
( , ) ( , ) ( , )k l
g i j f k l h i rk j rl
![Page 31: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/31.jpg)
Interpolation
• Interpolation kernel h() with sampling rate r
• Bicubic interpolation
,
( , ) ( , ) ( , )k l
g i j f k l h i rk j rl
![Page 32: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/32.jpg)
Bicubic Interpolation
• a specifies the derivative at x=1 • Usually a=-1, best matches the frequency
characteristics of a sinc function– A small amount of sharpening – Ringing (does not linearly interpolate straight lines
• Quadratic reproducing spline a=-0.5
![Page 33: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/33.jpg)
Bicubic Interpolation
Bilinear Cubic a=-1
Cubic a=-0.5
windowed sinc
![Page 34: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/34.jpg)
Windowed sinc function
• Best quality interpolator (Usually)– Both preserves details in the lower resolution
image and avoids aliasing
![Page 35: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/35.jpg)
Windowed sinc function
• Best quality interpolator (Usually)– Both preserves details in the lower resolution
image and avoids aliasing
• Ringing effect– Instead, repeatedly interpolate images by a small
fractional amount
![Page 36: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/36.jpg)
Decimation (Downsampling)
• Same kernel h(k,l) for both interpolation and decimation
• Avoid aliasing– Convolve the image with a low-pass filter
,
1( , ) ( , ) ( , )
k l
k lg i j f k l h i jr rr
![Page 37: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/37.jpg)
Decimation (Downsampling)
• Linear • Binomial
– Separating the high and low frequencies, – but leaves a fair amount of high-frequency detail,
which can lead to aliasing after downsampling
![Page 38: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/38.jpg)
Decimation (Downsampling)
• Linear• Binomial • Cubic
– a=-1, – a=-0.5
• Windowed sinc • QMF-9• Jpeg2000
Sample rate = 2
![Page 39: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/39.jpg)
Decimation (Downsampling)
• Cubic a=-1 – Sharpest but ringing
• QMF-9 and Jpeg2000– Wavelet analysis filters– Useful for compression – More aliasing
![Page 40: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/40.jpg)
Multi-resolution Representations
• Image pyramid– Accelerate coarse-to-fine
search algorithms– Look for objects or
patterns at different scales
– Perform multi-resolution blending operations
![Page 41: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/41.jpg)
Multi-resolution Representations
• Laplacian pyramid [Burt and Adelson’s (1983a)]
– Best known and most widely used in computer vision
![Page 42: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/42.jpg)
Laplacian Pyramid
• First: blur and subsample the original image with sample rate r=2
• Five-tap kernel
Octave pyramid
![Page 43: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/43.jpg)
Laplacian pyramid
• First: blur and subsample the original image by sample rate = 2
Gaussian pyramid:Repeated convolutions of the binomial kernel
converge to a Gaussian
![Page 44: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/44.jpg)
Laplacian Pyramid
![Page 45: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/45.jpg)
Laplacian Pyramid
• Actual computation of high-pass filter• Results in perfect reconstruction when Q=I
Laplacian image
Gaussian image
![Page 46: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/46.jpg)
Application: Image Blending
regular splice
pyramid blend
![Page 47: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/47.jpg)
Low frequency part
Medium frequency part
High frequency part
![Page 48: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/48.jpg)
![Page 49: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/49.jpg)
![Page 50: Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref:](https://reader036.vdocuments.net/reader036/viewer/2022062804/56649ede5503460f94beea55/html5/thumbnails/50.jpg)
Image Blending