structured forests for fast edge detection [paper presentation]
TRANSCRIPT
![Page 1: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/1.jpg)
![Page 2: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/2.jpg)
Dollár, Piotr, and C. Lawrence Zitnick. "Structured forests for fast edge detection.“
Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 2013.
![Page 3: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/3.jpg)
Main
Contribution
Compute edge maps in realtime,
faster than the competing state-of-the-art
Proposed
Method
Structured Random Forests
This presentation is inspired by the talk: http://techtalks.tv/talks/structured-forest-for-fast-edge-detection/59412/
![Page 4: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/4.jpg)
Edge Definition
Source: http://upload.wikimedia.org/wikipedia/en/8/8e/EdgeDetectionMathematica.png
![Page 5: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/5.jpg)
Edge Definition
Source: http://upload.wikimedia.org/wikipedia/en/8/8e/EdgeDetectionMathematica.png
![Page 6: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/6.jpg)
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
![Page 7: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/7.jpg)
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
![Page 8: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/8.jpg)
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
![Page 9: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/9.jpg)
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
![Page 10: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/10.jpg)
Edge Detection
as
Classification Problem
![Page 11: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/11.jpg)
Edge Detection as Classification Problem
• {0, 1}
![Page 12: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/12.jpg)
Edge Detection as Classification Problem
• {0, 1}
• Binary classification ignoring the local structures of the edges
![Page 13: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/13.jpg)
Edges have Structures
![Page 14: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/14.jpg)
Clustering Sketch Tokens
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection, Joseph J. Lim et al. 2013
![Page 15: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/15.jpg)
Random Forests
![Page 16: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/16.jpg)
Random Forests
ℎ 𝑥, 𝜃 = 𝑥 𝑘1 − 𝑥 𝑘2 < 𝜏
![Page 17: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/17.jpg)
Random Forests
![Page 18: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/18.jpg)
Random Forests
![Page 19: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/19.jpg)
Random Forests
![Page 20: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/20.jpg)
Random Forests For Edge Detection
![Page 21: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/21.jpg)
Random Forests For Edge Detection
![Page 22: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/22.jpg)
Random Forests For Edge Detection
![Page 23: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/23.jpg)
Random Forests For Edge Detection
![Page 24: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/24.jpg)
Random Forests For Edge Detection
![Page 25: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/25.jpg)
Random Forests For Edge Detection
Decision:
![Page 26: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/26.jpg)
Structured Random Forests
![Page 27: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/27.jpg)
The Output Space
{0, 1} 2
{ ,….} 151
DimensionalityInput Space
![Page 28: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/28.jpg)
The Output Space
{0, 1} 2
{ ,….} 151
2256
DimensionalityInput Space
![Page 29: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/29.jpg)
Node Split
Low entropy split
![Page 30: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/30.jpg)
Training Model
Bad split
![Page 31: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/31.jpg)
Training Model
Go od split
![Page 32: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/32.jpg)
Training Model
Cluster the
structured labels
![Page 33: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/33.jpg)
Training Model
Just one difference to random forests:
cluster the output into a binary or multiclass output using distance function
![Page 34: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/34.jpg)
Clustering
𝑌: Structured space where information gain not well defined
𝐶: Discrete space where information space is good defined
𝑍: Intermediate space where similarity measurement is easy to compute
Π ∶ 𝑌 → 𝑍 , 𝑍 → 𝐶
![Page 35: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/35.jpg)
Training Model
• Computing information gain
– Labels 𝐶 are discrete, standard entropy criterions used.
• Combining predictions
– To combine 𝑦1… 𝑦𝑛 ∈ 𝑌 into a prediction:
• Compute 𝑧𝑖 = Π𝜑(𝑦𝑖) of dimension 𝑚
• Select 𝑦𝑘 , whose 𝑧𝑘 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑧𝑘 𝑖,𝑗(𝑧𝑘𝑗 − 𝑧𝑖𝑗)2
(medoid)
+ Computing medoids is fast, 𝑂(𝑛𝑚)
![Page 36: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/36.jpg)
Training Structured Forests For
Edge Detection
![Page 37: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/37.jpg)
Training Structured Forests For Edge Detection
32x32 RGB image patch
→ 7228 features
![Page 38: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/38.jpg)
Training Structured Forests For Edge Detection
32x32 RGB image patch
→ 7228 features
Π ∶ 𝑌 → 𝑍
Dimension of 𝑍 = 2562
Down-sampled to m = 256
![Page 39: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/39.jpg)
Training Structured Forests For Edge Detection
32x32 RGB image patch
→ 7228 features
Π ∶ 𝑌 → 𝑍
Dimension of 𝑍 = 2562
Down-sampled to m = 256
![Page 40: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/40.jpg)
Edge Detection with Structured Forests
32x32 RGB image patch
→ 7228 features
![Page 41: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/41.jpg)
Edge Detection with Structured Forests
32x32 RGB image patch
→ 7228 features
𝑌 is a 16x16 segmentation
mask
![Page 42: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/42.jpg)
Multi-scale Detection
![Page 43: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/43.jpg)
Multi-scale Detection
![Page 44: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/44.jpg)
Multi-scale Detection
![Page 45: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/45.jpg)
Results
• BSDS 500 image set
– Multi-scale ties or outperforms the accuracy of the state of the art.
– Single-scale improves runtime by 5x to 10x
![Page 46: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/46.jpg)
Results
• BSDS 500 image set
– Multi-scale ties or outperforms the accuracy of the state of the art.
– Single-scale improves runtime by 5x to 10x
![Page 47: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/47.jpg)
Results
• BSDS 500 image set
– Multi-scale ties or outperforms the accuracy of the state of the art.
– Single-scale improves runtime by 5x to 10x
![Page 48: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/48.jpg)
Results
• NYU image set
– Multi-scale is slightly better than the state of the art.
– Improved performance by multiple orders of magnitude
![Page 49: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/49.jpg)
Conclusions
• Realtime structured learning method for edge detection
• General purpose method for learning structured random forests
• Real time + state of the art accuracy → new applications possible
• Novel learning approach may be applicable to other problems.
![Page 50: Structured Forests for Fast Edge Detection [Paper Presentation]](https://reader034.vdocuments.net/reader034/viewer/2022042614/55a1c68f1a28ab6c758b46bf/html5/thumbnails/50.jpg)
T h a n k yo u