cs-f441: selected topics from computer science (deep learning for nlp & cv) · 2019-11-06 ·...
TRANSCRIPT
CS-F441: SELECTED TOPICS FROM COMPUTER
SCIENCE (DEEP LEARNING FOR NLP & CV)
Lecture-KT-10: SIFT, HOG
Dr. Kamlesh Tiwari,Assistant Professor,
Department of Computer Science and Information Systems,BITS Pilani, Rajasthan-333031 INDIA
Nov 06, 2019 (Campus @ BITS-Pilani July-Dec 2019)
Recap: Harris OperatorUse
f =λ1λ2
λ1 + λ2=
determinant(H)
trace(H)
Do the following:1 Compute cornerness score of each point2 Find points whose surrounding window gave large corner response
(f > threshold)3 Take the points of local maxima, i.e., perform non-maximum
suppression
Rotation but Scale Invariance
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 2 / 16
Recap: Scale invariant interest points
Spatial selection: the magnitude of the Laplacian response will achievea maximum at the center of the blob, provided the scale of theLaplacian is “matched” to the scale of the blob
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 3 / 16
Recap: Scale invariant interest pointsInterest points are local extrema in both position and scale.
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 4 / 16
Recap: Laplacian and Difference of Gaussians
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 5 / 16
Building Scale Space
∂G∂σ
= σ42 G
G(x , y , kσ)−G(x , y , σ)kσ − σ
= σ42 G
G(x , y , kσ)−G(x , y , σ) = (k − 1)σ2 42 G
Difference of Gaussian at two scale is similar to Laplacian
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 6 / 16
Scale Space Octave
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 7 / 16
SIFT Steps1 Determine interest points in scale space by looking extrema2 For x = (x , y , σ)T we can approximate DoG by
D(x) = D + ∂DT
∂x x + 12xT ∂2D
∂2x x it has extrema at (̂x) = −∂2D−1
∂x2∂D−1
∂xFilter outliers where |D(x)| > Th Effect: 832 -to- 729
3 Find Hessian and its determinant, trace. For r = λ1λ2
Discard points having r > 10 potential edges. Effect: 729 -to- 5364 Get orientation of interest point. In small neighborhood; histogram
of 36 bins of 10◦ each. Use magnitude for weighting
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 8 / 16
SIFT Descriptor
1 Gradient is more robust than magnitude2 Compute relative orientation and magnitude in the 16× 16
neighbourhood (8 bins)
3 Take 4× 4 blocks and create weighted histogram (scale andorientation) 1
18 × (16/4)× (16/4) = 128 dimensional vectorSTCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 9 / 16
SIFT Parameters
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 10 / 16
SIFT Matching 2
1 Key point with minimum Euclidean distance are similar2 Ratio between best and second best match should be large (.8 is
not good)
253406: Lowe, David G, “Distinctive image features from scale-invariant keypoints”, International journal of computer vision,
60(2), pp 91–110, Springer (2004)
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 11 / 16
SIFT Parameters
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 12 / 16
HOG: Histogram of Oriented Gradients
Histograms of oriented gradients for human detection 3
3[cite 28654] Histograms of oriented gradients for human detection, Dalal, Navneet and Triggs, Bill, CVPR 2005
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 13 / 16
HOG: Histogram of Oriented Gradients
Take 64× 128 image, divide it into 16× 16 blocks of 50% overlapTotal blocks 7× 15 = 105. Block is 2× 2 cell of 8× 8 sizeQuantize orientation in 9 direction (amplitude is vote)Feature size 105× (2× 2)× 9 = 3780
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 14 / 16
HOG
Example: 80◦ has distance 15 and 5 from 70 and 90. Hence ratiois 5/20 and 15/20
More: deformable part model (as there could be occlusion and action)
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 15 / 16
Thank You!
Thank you very much for your attention4!
Queries ?
4https://www.cs.cornell.edu/courses/cs6670/2011sp/lectures/lec02 filter.pdf
STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 16 / 16