computer vision: correlation, convolution, and gradient
TRANSCRIPT
![Page 1: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/1.jpg)
Computer Vision: Correlation, Convolution, and Gradient
MENOUFIA UNIVERSITYFACULTY OF COMPUTERS AND INFORMATION
INFORMATION TECHNOLOGYCOMPUTER VISION
جامعة المنوفية
كلية الحاسبات والمعلومات
تكنولوجيا المعلومات
الرؤيا بالحاسب
جامعة المنوفية
Ahmed Fawzy Gad
[email protected] 3 October 2017
![Page 2: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/2.jpg)
Index
• Correlation• Python Implementation
• Convolution• Python Implementation
• Gradient• Python Implementation
Correlation
Convolution
Gradient Filtering
![Page 3: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/3.jpg)
CORRELATION
![Page 4: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/4.jpg)
Correlation for 2D Image
• Correlation is used to match atemplate to an image.
• Can you tell where the followingtemplate exactly located in theimage?
• Using correlation we can find theimage region that best matchesthe template.
?
![Page 5: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/5.jpg)
How 2D Correlation Works?
• Given a template, using correlation thetemplate will pass through each image partand a similarity check take place to findhow similar the template and the currentimage part being processed.
• Starting by placing the template top-leftcorner on the top-left corner of the image,a similarity measure is calculated.
![Page 6: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/6.jpg)
How 2D Correlation Works?
• Given a template, using correlation thetemplate will pass through each image partand a similarity check take place to findhow similar the template and the currentimage part being processed.
• Starting by placing the template top-leftcorner on the top-left corner of the image,a similarity measure is calculated.
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
![Page 7: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/7.jpg)
How 2D Correlation Works?
• Given a template, using correlation thetemplate will pass through each image partand a similarity check take place to findhow similar the template and the currentimage part being processed.
• Starting by placing the template top-leftcorner on the top-left corner of the image,a similarity measure is calculated.
![Page 8: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/8.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
![Page 9: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/9.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
![Page 10: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/10.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 ∗ 1 + 1 ∗ 3 − 4 ∗ 4 + 3 ∗ 2 + 2 ∗ 7 + 5 ∗ 4− 1 ∗ 6 + 8 ∗ 2 + 1 ∗ 5= 𝟒𝟒
![Page 11: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/11.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44
Did you get why template sizes are odd?
![Page 12: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/12.jpg)
Even Template Size
58 3 213 81 78
185 87 32 27 11
70 66 60 2 19
61 91 129 89 38
14 7 58 14 42
Even template sizes has no center.E.g. 2x3, 2x2, 5x6, …
𝟖𝟒
???
![Page 13: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/13.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
44
![Page 14: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/14.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 ∗ 3 + 1 ∗ 4 − 4 ∗ 3 + 3 ∗ 7 + 2 ∗ 4 + 5 ∗ 1− 1 ∗ 2 + 8 ∗ 5 + 1 ∗ 2= 𝟕𝟐
44
![Page 15: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/15.jpg)
Correlation for 2D Image
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72
Based on the current step, what is the region that best matches the template?
It is the one corresponding to the highest score in the result matrix.
![Page 16: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/16.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
44 72
![Page 17: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/17.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
44 72
This will cause the program performing correlation to fall into index out of bounds exception.
![Page 18: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/18.jpg)
Padding by Zeros
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 0
0
0
0
It doesn`t affect multiplication.
Why Zero?
![Page 19: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/19.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 0
0
0
0
2 1 -4
3 2 5
-1 8 1
![Page 20: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/20.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 0
0
0
0
2 ∗ 4 + 1 ∗ 3 − 4 ∗ 0 + 3 ∗ 4 + 2 ∗ 1 + 5 ∗ 0− 1 ∗ 5 + 8 ∗ 2 + 1 ∗ 0= 𝟑𝟔
![Page 21: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/21.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 36 0
0
0
0
2 ∗ 4 + 1 ∗ 3 − 4 ∗ 0 + 3 ∗ 4 + 2 ∗ 1 + 5 ∗ 0− 1 ∗ 5 + 8 ∗ 2 + 1 ∗ 0= 𝟑𝟔
![Page 22: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/22.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
44 72 36
![Page 23: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/23.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
2 1 -4
3 2 5
-1 8 1
44 72 36
![Page 24: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/24.jpg)
Pad by Zeros
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 36
0 0 0 0
![Page 25: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/25.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 36
0 0 0 0
2 1 -4
3 2 5
-1 8 1
![Page 26: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/26.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 36
0 0 0 0
2 ∗ 2 + 1 ∗ 5 − 4 ∗ 2 + 3 ∗ 6 + 2 ∗ 8 + 5 ∗ 9− 1 ∗ 0 + 8 ∗ 0 + 1 ∗ 0= 𝟖𝟎
![Page 27: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/27.jpg)
Continue Correlating the Template
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
Template – 3x3
44 72 36
80
Continue till end.
![Page 28: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/28.jpg)
• It depends on both the template size.
• For 3x3 template, pad two columns and tworows. One column to the left and another tothe right. One row at the top and one rowat the bottom.
• For 5x5? Pad 4 rows (two left & two right)and 5 columns (two top & two bottom).
• Generally, for N*N template pad (N-1)/2columns and rows at each side.
How much Padding Required?
1 3 4 3
2 7 4 1
6 2 5 2
13 6 8 9
0 0 0 0
0 0 0 0
0
0
0
0
0
0
0
0
0
0
0
0
How to do this Programmatically.?
![Page 29: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/29.jpg)
Implementing Correlation in Python – 3x3Template
![Page 30: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/30.jpg)
Implementing Correlation in Python – 141x141Template
![Page 31: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/31.jpg)
Implementing Correlation in Python – 141x141Template
![Page 32: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/32.jpg)
Implementing Correlation in Python – 141x141Template
![Page 33: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/33.jpg)
Implementing Correlation in Python – 141x141Template
Is this is the optimal results?
No. But WHY?
![Page 34: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/34.jpg)
Matching using Correlation
• Matching depends on justmultiplication.
• The highest results ofmultiplication is the one selectedas best match.
• But the highest results ofmultiplication not always refersto best match.
Template Result
![Page 35: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/35.jpg)
Matching usingCorrelation• Make a simple edit on the imageby adding a pure whiterectangular area.
• Use the previous template.
• Apply the algorithm.
Template
![Page 36: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/36.jpg)
Matching using Correlation
• The best matched region is thewhite region.
• The reason is that correlationdepends on multiplication.
• What gives highest multiplicationresults is the best match.
• Getting high results corresponds tomultiplying by higher numbers.
• The highest pixel value for grayimages is 255 for white.
![Page 37: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/37.jpg)
Implementing Correlation in Python – GenericCode – Squared Odd Sized Templates
![Page 38: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/38.jpg)
Implementing Correlation in Python – GenericCode – Squared Odd Sized Templates
![Page 39: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/39.jpg)
Implementing Correlation in Python – GenericCode – Squared Odd Sized Templates
![Page 40: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/40.jpg)
Implementing Correlation in Python – GenericCode – Squared Odd Sized Templates
![Page 41: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/41.jpg)
CONVOLUTION
![Page 42: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/42.jpg)
What is Convolution?
• The primary objective of mathematical convolution is combining twosignals to generate a third signal. Convolution is not limited on digitalimage processing and it is a broad term that works on signals.
Input SignalImpulse Response
Output SignalSystem
Convolution
![Page 43: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/43.jpg)
Convolution Formula for 2D Digital Signal• Convolution is applied similarly to correlation.
• Note how similar the formulas for correlation and convolution. Theonly difference is that in the signs of 𝒖 and 𝒗 variables.
• Correlation formula has positive signs for 𝒖 and 𝒗 while correlationhas negative signs for them.
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]
Is just changing signs make sense?
![Page 44: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/44.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 45: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/45.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 46: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/46.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 47: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/47.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 48: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/48.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 49: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/49.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 50: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/50.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 51: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/51.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 52: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/52.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
![Page 53: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/53.jpg)
Correlation Vs. Convolution
𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 =
𝒖=−𝒌
𝒌
𝒗=−𝒌
𝒌
𝒉 𝒖,𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
Correlation Convolution
13204
458047
09373
.2.4.1
.5.8.3
.9.5.7
.2.4.1
.5.8.3
.9.5.7
-1, 1-1, 0-1, -1
0, 10, 00, -1
1, 11, 01, -1
1, -11, 01, 1
0, -10, 00, 1
-1, -1-1, 0-1, 1
How to simplify convolution?
![Page 54: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/54.jpg)
Simplifying Convolution
.2.4.1
.5.8.3
.9.5.7
1, -11, 01, 1
0, -10, 00, 1
-1, -1-1, 0-1, 1
*13204
458047
09373
.7.5.9
.3.8.5
.1.4.213204
458047
09373= -1, 1-1, 0-1, -1
0, 10, 00, -1
1, 11, 01, -1
Just rotate the template with 180 degrees before multiplying by the image.
*Applying convolution is similar to correlation except
for flipping the template before multiplication.
![Page 55: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/55.jpg)
Two Many Multiplications & Additions
• Assume to remove noise from an image, it should be applied to twofilters (filter1 and filter2). Each filter has size of 7x7 pixels and theimage is of size 1024x1024 pixels.
• A simple approach is to apply filter1 to the image then apply filter2 tothe output of filter1.
• Too many multiplications and additions due to applying two filters onthe image. 1,024 x 1,024 x 49 multiplication and 1,000 x 1,000 x 49addition for the single filter.
1024x1024 Image
Filter1* Temp. Result Filter2* Result
![Page 56: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/56.jpg)
Convolution Associative Property
• Using the associative property of convolution, we can reduce thenumber of multiplications and additions.
• To apply two filters f1 and f2 over an image, we can merge the twofilters together then apply the convolution between the resultant newfilter and the image.
• Finally, there is only 1,024 x 1024 x 49 multiplication and addition.
(Im * 𝒇𝟏) * 𝒇𝟐 = Im * (𝒇𝟏 * 𝒇𝟐)For 𝒇𝟏 * 𝒇𝟐 = 𝒇𝟑
Result is Im * 𝒇𝟑
![Page 57: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/57.jpg)
Convolution Applications
• Convolution is used to merge signals.
• It is used to apply operations like smoothing and filtering imageswhere the primary task is selecting the appropriate filter template ormask.
• Convolution can also be used to find gradients of the image.
![Page 58: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/58.jpg)
Implementing Convolution in Python
• The implementation of convolution is identical to correlation exceptfor the new command that rotates the template.
![Page 59: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/59.jpg)
GRADIENT
![Page 60: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/60.jpg)
Image Gradients
• Image gradients can be defined as change of intensity in some direction.
• Based on the way gradients were calculated and specifically based onthe template/kernel used, gradients can be horizontal (X direction) orvertical (Y direction) in direction.
-1-1-1
000
111
Horizontal
10-1
10-1
10-1
Vertical
θ = 𝑡𝑎𝑛−1[𝐺𝑦𝐺𝑥] 𝐺 = 𝐺𝑥 + 𝐺𝑦
![Page 61: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/61.jpg)
Image Gradient Calculation Steps
• Calculate the image gradient in X direction at each pixel.
• Calculate the image gradient in Y direction at each pixel.
• Calculate the overall gradient for the pixel.
![Page 62: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/62.jpg)
Implementing Horizontal Gradient in Python
![Page 63: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/63.jpg)
Implementing Horizontal Gradient in Python
![Page 64: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/64.jpg)
Implementing Vertical Gradient in Python
![Page 65: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/65.jpg)
Implementing Vertical Gradient in Python
![Page 66: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/66.jpg)
FILTERING
![Page 67: Computer Vision: Correlation, Convolution, and Gradient](https://reader034.vdocuments.net/reader034/viewer/2022051123/5a677a067f8b9a656a8b5609/html5/thumbnails/67.jpg)
Example – Mean Filter
•Mean Filter Kernel.𝟏
𝟗
𝟏
𝟗
𝟏
𝟗𝟏
𝟗
𝟏
𝟗
𝟏
𝟗𝟏
𝟗
𝟏
𝟗
𝟏
𝟗
58 3 213 81 78
185 87 32 27 11
70 66 60 2 19
61 91 129 89 38
14 7 58 14 42
𝟏
𝟗∗ 58 +
𝟏
𝟗∗ 3 +𝟏
𝟗∗ 213 +
𝟏
𝟗∗ 185 +
𝟏
𝟗∗ 87 +
𝟏
𝟗∗ 32
+𝟏
𝟗∗ 70 +
𝟏
𝟗∗ 66 +
𝟏
𝟗∗ 60
= 85.67 ~ 86
Implement in Python (Deadline next week starting @ 7 Oct. 2017).1. Mean Smoothing Filter2. Gaussian Smoothing Filter3. Sharpening Filter4. Median Filter