last time application: edge detectionayan/courses/cse559a/pdfs/lec... · 2019-09-05 · bilateral...
Post on 01-Aug-2020
1 Views
Preview:
TRANSCRIPT
CSE 559A: Computer Vision
Fall 2020: T-R: 11:30-12:50pm @ Wrighton 300 / Zoom
Instructor: Ayan Chakrabarti (ayan@wustl.edu).Course Staff: Adith Boloor, Patrick Williams
Sep 24, 2020
http://www.cse.wustl.edu/~ayan/courses/cse559a/
1
ADMINISTRIVIAADMINISTRIVIAREMINDER: Office Hours start this Friday. Zoom link available through Canvas.
Received your problem set 0 solutions. Will be giving individual feedback Soon.Meanwhile, a few points:
Make sure your write-up is named solution.pdf, and not something else.Make sure you git add the solution.pdf file before commit & push. We need the PDF, not just the .tex.Don’t rename the .py files. Your submitted code should be added in to the original prob?.py files.Be a little careful about googling for answers to specific questions. Problem set 0 was pretty basic, so bothwere the same. But in problem set 1, don’t google: how to implement histogram equalization!
Stack overflow type web-sites can especially get problematic sometimes.Do not use OpenCV! You have to implement everything using numpy / scipy. The point is to know howcomputer vision algorithms work and are implemented.
2
LAST TIMELAST TIMETalked about point-wise operations.Began talking about efficient implementations in numpy (with parallel elementwise operations, reshapes,and broadcasting).
3
POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYImages themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul doesmatrix multiplication).But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determinedorder, as rows. Represent as:
matrix: color images
vector: grayscale images.
X
(HW) × 3
(HW) × 1
Y[n] = C X[n] ⇒ Y = ?
4
POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYImages themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul doesmatrix multiplication).But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determinedorder, as rows. Represent as:
matrix: color images
vector: grayscale images.
X
(HW) × 3
(HW) × 1
Y[n] = C X[n] ⇒ Y = X CT
# Begin with X as (H,W,3) array
Xflt = np.reshape(X,(-1,3)) # Flatten X to a (H*W, 3) matrix
Yflt = np.matmul(Xflt,C.T) # Post-multiply by C
Y = np.reshape(Yflt,X.shape) # Turn Y back to an image array
5
POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYOne last trick: what if your pointwise function is a lookup table (for grayscale images).Assume that you are trying to map all intensities in an array X, which is of type integer (uint8, etc.), and youhave a lookup table as a vector H, where you want to map an intensity f to H[f].So what you want to do is: Y[i,j] = H[X[i,j]]You can do this simply as Y = H[X]. Y will be the same shape as X (and not H).
6
CONVOLUTIONCONVOLUTION
7
CONVOLUTIONCONVOLUTION
8
CONVOLUTIONCONVOLUTION
9
CONVOLUTIONCONVOLUTION
10
CONVOLUTIONCONVOLUTION
11
CONVOLUTIONCONVOLUTION
12
CONVOLUTIONCONVOLUTION
13
CONVOLUTIONCONVOLUTION
14
CONVOLUTIONCONVOLUTION
15
CONVOLUTIONCONVOLUTION
Double summation over the support / size of the kernel
We assume is scalar vaued.
If is scalar, so is .
If is a color image, each channel convolved with independently.
To go from m to n channels in a “conv layer”: is matrix valued, and is a matrix-vector product.
Notation: Y = X ∗ k
Y[n] = k[ ] X[n − ]∑n′
n′
n′
Y[ , ] = k[ , ] X[( − ), ( − )]nx ny ∑n′x
∑n′y
n′x n′y nx n′x ny n′y
k
k[n] ∈ ℝ
X[n] Y[n]
X k
k[n] ∈ ℝn×mk[ ] X[n − ]n
′n
′
16
CONVOLUTIONCONVOLUTION
17
CONVOLUTIONCONVOLUTION
18
CONVOLUTIONCONVOLUTION
19
CONVOLUTIONCONVOLUTION
20
CONVOLUTIONCONVOLUTION
21
CONVOLUTIONCONVOLUTION
22
CONVOLUTIONCONVOLUTION
23
CONVOLUTIONCONVOLUTION
24
CONVOLUTIONCONVOLUTION
25
CONVOLUTIONCONVOLUTION
26
CONVOLUTIONCONVOLUTION
27
CONVOLUTIONCONVOLUTION
28
CONVOLUTIONCONVOLUTION
29
CONVOLUTIONCONVOLUTION
30
CONVOLUTIONCONVOLUTION
31
CONVOLUTION: PROPERTIESCONVOLUTION: PROPERTIESLet , , and denote full, valid, and same convolution (with zero padding for full and same)
Linear / Distributive: For scalars ;
If , then:
If and , ( , same size):
If and , ( , same size):
X k∗full
X k∗val
X k∗same
α, β
Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY
= X ∗Y1 k1 = X ∗Y2 k2 k1 k2 X ∗ (α + β ) = α + βk1 k2 Y1 Y2
= ∗ kY1 X1 = ∗ kY2 X2 X1 X2 (α + β ) ∗ k = α + βX1 X2 Y1 Y2
X ∗ (α + β ) [n] = (α [ ] + β [ ])X[n − ]k1 k2 ∑n′
k1 n′
k2 n′
n′
= α [ ]X[n − ] + β [ ]X[n − ]∑n′
k1 n′
n′
∑n′
k2 n′
n′
32
CONVOLUTION: PROPERTIESCONVOLUTION: PROPERTIESLet , , and denote full, valid, and same convolution (with zero padding for full and same)
Linear / Distributive: For scalars ;
If , then:
If and , ( , same size):
If and , ( , same size):
Associative
X k∗full
X k∗val
X k∗same
α, β
Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY
= X ∗Y1 k1 = X ∗Y2 k2 k1 k2 X ∗ (α + β ) = α + βk1 k2 Y1 Y2
= ∗ kY1 X1 = ∗ kY2 X2 X1 X2 (α + β ) ∗ k = α + βX1 X2 Y1 Y2
(X ) = X ( )∗fullk1 ∗
fullk2 ∗
fullk1 ∗full
k2
(X ) = X ( )∗valk1 ∗
valk2 ∗
valk1 ∗full
k2
(X ) ≠ X ( )∗samek1 ∗
samek2 ∗
samek1 ∗full
k2
(X ) [n] = [ ](X ∗ )[n − ] = [ ] [ ]X[n − − ]∗fullk1 ∗
fullk2 ∑
n′
k2 n′
k1 n′
∑n′
k2 n′
∑n″
k1 n″
n′
n″
= ( [ ] [( + ) − ])X[n − ( + )]∑+n′ n″∑n′
k2 n′k1 n
′n″
n′
n′
n″
33
CONVOLUTION: PROPERTIESCONVOLUTION: PROPERTIESLet , , and denote full, valid, and same convolution (with zero padding for full and same)
Linear / Distributive: For scalars ;
If , then:
If and , ( , same size):
If and , ( , same size):
Associative
Commutative:
X k∗full
X k∗val
X k∗same
α, β
Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY
= X ∗Y1 k1 = X ∗Y2 k2 k1 k2 X ∗ (α + β ) = α + βk1 k2 Y1 Y2
= ∗ kY1 X1 = ∗ kY2 X2 X1 X2 (α + β ) ∗ k = α + βX1 X2 Y1 Y2
(X ) = X ( )∗fullk1 ∗
fullk2 ∗
fullk1 ∗full
k2
(X ) = X ( )∗valk1 ∗
valk2 ∗
valk1 ∗full
k2
(X ) ≠ X ( )∗samek1 ∗
samek2 ∗
samek1 ∗full
k2
=k1 ∗fullk2 k2 ∗full
k1
(X ) = (X )∗fullk1 ∗
fullk2 ∗
fullk2 ∗
fullk1
(X ) = (X )∗valk1 ∗
valk2 ∗
valk2 ∗
valk1
(X ) ≠ (X )∗samek1 ∗
samek2 ∗
samek2 ∗
samek1
34
CONVOLUTIONCONVOLUTION
35
CONVOLUTIONCONVOLUTION
36
CONVOLUTIONCONVOLUTION
37
CONVOLUTIONCONVOLUTION
38
CONVOLUTIONCONVOLUTION
39
CONVOLUTIONCONVOLUTION
40
CONVOLUTIONCONVOLUTION
41
CONVOLUTIONCONVOLUTION
42
CONVOLUTIONCONVOLUTION
43
CONVOLUTIONCONVOLUTION
44
CONVOLUTIONCONVOLUTION
45
CONVOLUTIONCONVOLUTION
46
CONVOLUTIONCONVOLUTION
47
CONVOLUTIONCONVOLUTION
48
CONVOLUTIONCONVOLUTION
49
CONVOLUTIONCONVOLUTION
50
CONVOLUTIONCONVOLUTION
51
CONVOLUTIONCONVOLUTION
52
CONVOLUTIONCONVOLUTION
53
CONVOLUTIONCONVOLUTION
54
CONVOLUTIONCONVOLUTION
55
CONVOLUTIONCONVOLUTION
56
CONVOLUTIONCONVOLUTION
57
CONVOLUTIONCONVOLUTION
58
CONVOLUTIONCONVOLUTION
59
CONVOLUTIONCONVOLUTION
60
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
61
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
62
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
63
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
64
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
65
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
66
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
67
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
68
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
69
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
70
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
71
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
72
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
73
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
74
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
75
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
76
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
77
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
78
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
79
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
80
APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION
81
top related