image processing honza Černocký, Úpgm. 1d signal 2
TRANSCRIPT
![Page 1: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/1.jpg)
Image processing
Honza Černocký, ÚPGM
![Page 2: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/2.jpg)
1D signal
2
![Page 3: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/3.jpg)
Images
• 2D – grayscale photo
• Multiple 2D – color photo
• 3D – medical imaging, vector graphics, point-clouds (depth information, Kinect)
• Video – time is another dimension
3
![Page 4: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/4.jpg)
Image
4
K
L
![Page 5: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/5.jpg)
Mathematically
5
![Page 6: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/6.jpg)
Representation of pixels
• For storage – quantization
• 8 (normal) /16 (HDR) bits
• 0…255 etc
• For computing [0…1]
• floats
6
![Page 7: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/7.jpg)
Operations on pixels – more brightness
7
![Page 8: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/8.jpg)
Operations on pixels – less brightness
8
![Page 9: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/9.jpg)
More contrast
9
![Page 10: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/10.jpg)
Less contrast
10
![Page 11: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/11.jpg)
What about bad values ?
x[k,l] < 0 or x[k,l] > 1
•Let them be …
•Clip them:
11
![Page 12: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/12.jpg)
Use statistics of values
• Histogram
12
![Page 13: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/13.jpg)
Histogram equalization
13
![Page 14: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/14.jpg)
Thresholding
14
![Page 15: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/15.jpg)
Filtering
15
signal filter signal
Task ?
Obtain a new signal with desired properties.
![Page 16: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/16.jpg)
Filtering – reminder of 1D
16
![Page 17: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/17.jpg)
Filtering – reminder of 1D
17
![Page 18: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/18.jpg)
2D filter
18
IJ
![Page 19: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/19.jpg)
Filtering …
Edge ? Insert zeros for example …19
![Page 20: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/20.jpg)
Coefficients h[k,l]
Terminology•Coefficients•Mask•Convolution kernel …
We want them •to have some sense … •not to change the dynamics of the signal
20
![Page 21: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/21.jpg)
Wire
21
![Page 22: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/22.jpg)
Smoothing
22
![Page 23: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/23.jpg)
Vertical edge detector
23
![Page 24: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/24.jpg)
What to do with negative samples?
• Let them as they are for further computation…
• Or convert to [0…1] for visualization – for example take absolute value
24
![Page 25: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/25.jpg)
Horizontal edge detector
25
![Page 26: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/26.jpg)
Both together …
26
![Page 27: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/27.jpg)
Denoising…
• Mask 9x9, values 1/81 … zoom noise_low_pass.png
27
![Page 28: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/28.jpg)
Denoising II – median filter
• zoom noise_median.png
28
![Page 29: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/29.jpg)
Spectral analysis
• Task:Determine, what is in the signal on different frequencies
• Why ?–Visualization, –Feature extraction (think of Facebook)–Filtering (conversion to spectral domain and multiplication therein can be more efficient)–Coding (think of JPEG)
29
![Page 30: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/30.jpg)
Reminder – 1D
• What is in ?
30
![Page 31: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/31.jpg)
Reminder – 1D cosine bases
31
Problem with the phase – the signal “wave” begins not necessarily at zero …
![Page 32: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/32.jpg)
OK …
32
![Page 33: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/33.jpg)
Also OK …
33
![Page 34: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/34.jpg)
Bad bad bad
34
• How come zero when ?
![Page 35: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/35.jpg)
Complex exponentials
35
![Page 36: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/36.jpg)
36
![Page 37: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/37.jpg)
37
![Page 38: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/38.jpg)
1D ultimate result - DFT
38
![Page 39: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/39.jpg)
Now finally 2D
• Correlation = determination of similarity = projection to bases
• We have seen this, it’s boring, it’s all the time the same … YES IT IS !
39
![Page 40: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/40.jpg)
Analyzing signal = d.c.
c=4.8287e+03 c=6.8287e+03 c=8.8287e+0340
![Page 41: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/41.jpg)
Analyzzing signal – horizontal cosine
41
• Changes only in one direction
• We’ll need to visualize negative values too.
![Page 42: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/42.jpg)
Geeks: how to do this in Matlab ??42
![Page 43: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/43.jpg)
Analysis for
43
c=4.5475e-13 c=2500 c=3.4106e-13
![Page 44: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/44.jpg)
2 x fasterhorizontal cos
44
c=2.9843e-13 c=1.5064e-12 c=2.5000e+03
![Page 45: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/45.jpg)
Vertical cos
45
c= -7.7716e-14 c=2.5000e+03 c=6.9944e-13
![Page 46: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/46.jpg)
2 x fastervertical cos
46
c= 4.9960e-13 c=2.5000e+03 c=-65.4694
![Page 47: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/47.jpg)
Mix of both directions …
47
![Page 48: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/48.jpg)
Generalization
• m/K – horizontal frequency
• n/L – vertical frequency
48
![Page 49: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/49.jpg)
Reasonable ranges of frequencies ?
• m/K and n/L 0 … ½ => OK
> ½ => hm…
49
K
L
x[k,l]
X[k,l]
K/2
L/2
0
0
![Page 50: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/50.jpg)
More on frequencies …
• 1D
• 2D
50
![Page 51: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/51.jpg)
Phase – problem again
51
c=2.5000e+03 c=9.9476e-13 c=-5.6843e-14
![Page 52: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/52.jpg)
Solution – complex exponentials (again)
Reminder what is what :
•k,l indices of pixels (input)
•m,n indices of frequencies (result)
•m/K normalized vertical frequency
•n/L normalized horizontal frequency
52
![Page 53: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/53.jpg)
Imagining it I.
53
![Page 54: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/54.jpg)
Imagining it II.
54
![Page 55: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/55.jpg)
Correlation with x[n]
55
Re(c) = 2500 Im(c) = 0
|c| = 2500
![Page 56: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/56.jpg)
Correlation with x[n]
56
Re(c) = 1250 Im(c) = -2165
|c| = 2500
![Page 57: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/57.jpg)
Correlation with x[n]
57
Re(c) = -772 Im(c) = 2378
|c| = 2500
![Page 58: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/58.jpg)
Change in both directions…X[7,3]
58
Re(c) = 0 Im(c) = 0
|c| = 0 … not similar
![Page 59: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/59.jpg)
Change in both directions…X[7,3]
59
Re(c) = 2165 Im(c) = 1250
|c| = 2500 … similar
![Page 60: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/60.jpg)
2D DFT using 2 x 1D DFT
• so that
60
![Page 61: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/61.jpg)
2D DFT for a real signal
61
![Page 62: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/62.jpg)
62
![Page 63: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/63.jpg)
Something with higher frequencies
63
![Page 64: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/64.jpg)
64
![Page 65: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/65.jpg)
DFT produces K x L points !
65
K
L
x[k,l]K
L
X[k,l]
![Page 66: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/66.jpg)
The whole result for k=0…K-1, l=0…L-1
66
![Page 67: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/67.jpg)
Symmetry ?
Source: http://www-personal.umich.edu/~hyunjinp/notes/n-dft.pdf 67
K/2
0
K-1
0 L/2 L-1
![Page 68: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/68.jpg)
Re-shuffling…
• Low frequencies to the center
68
![Page 69: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/69.jpg)
Inverse 2D DFT
69
![Page 70: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/70.jpg)
Playing with frequencies
70
![Page 71: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/71.jpg)
71
![Page 72: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/72.jpg)
DCT
Why ?
•People don’t like complex exponentials
•People don’t like symmetries and “useless” values …
•For image K x L, we want K x L real values.
72
![Page 73: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/73.jpg)
1D DCT
73
+ normalization of coefficients…
![Page 74: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/74.jpg)
2D DCT bases
Source: https://en.wikipedia.org/wiki/Discrete_cosine_transform#Multidimensional_DCTs
74
![Page 75: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/75.jpg)
Filtering in DCT
75
![Page 76: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/76.jpg)
76
![Page 77: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/77.jpg)
JPEG
• Source: http://www.eetimes.com/document.asp?doc_id=1225736 • More in computer graphics courses and lab exercise. 77
![Page 78: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/78.jpg)
SUMMARY
• Image is a 2D signal
• Filtering– Convolution, analogy with 1D FIR filters– No feedback (IIR) in images
• Sweeping mask over the image, multiplying everything that is underneath with its coefficients, adding.
78
![Page 79: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/79.jpg)
SUMMARY II.
• Frequency analysis 2D– Similar to 1D – projection to bases– Image frequencies also have some sense. – Cos bases are a good exercise, but not enough.
• 2D DFT – Projection/similarity/correlation with complex
exponentials– Coefficients are complex – magnitude and angle. – Lots of symmetries in the resulting DFT matrix– Can use spectrum for filtering
79
![Page 80: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/80.jpg)
SUMMARY III.
• DCT– 2x „slower“ bases than DFT– Slightly More complicated definition– Produces real coefficients, low frequencies
(only!) at the beginning. – Use in JPEG
80
![Page 81: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/81.jpg)
TO BE DONE
• Determining frequency response of a 2D filter• How is it exactly with the symmetries of 2D
DFT ?• Why are 1D and 2D DCT exactly defined and
why this half-sample shift ?• Colors (color models, etc). • How does the face matching on Facebook
work?– Hint:
https://research.facebook.com/researchers/684639631606527/yann-lecun/
81
![Page 82: Image processing Honza Černocký, ÚPGM. 1D signal 2](https://reader035.vdocuments.net/reader035/viewer/2022062322/5697bff31a28abf838cbc8c0/html5/thumbnails/82.jpg)
The END
82