high level computer vision deep learning for …...high level computer vision deep learning for...
TRANSCRIPT
High Level Computer Vision
Deep Learning for Computer Vision Part 3 - Segmentation
Bernt Schiele - [email protected] Mario Fritz - [email protected]
https://www.mpi-inf.mpg.de/hlcv
High Level Computer Vision - July 15, 2o15
• From detection to segmentation ‣ Main Reading:
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs, Chen, Papandreou, Kokkins, Murphy, Yuille, ICLR’15
‣ Further interesting reading - Hypercolumns for object segmentation and fine-grained localization
Bharath Hariharan, Pablo Arbeláez, Ross Girshick, Jitendra Malik, CVPR’15 - Fully Concolutional Networks for Semantic Segmentation
John Long, Evan Shelhamer, Trevor Darelle, CVPR’15
2
High Level Computer Vision - July 15, 2o15 3
High Level Computer Vision - July 15, 2o15 4
High Level Computer Vision - July 15, 2o15 5
Sliding Window with ConvNet
Input Window
224
224
66
256C
classes
Conv Conv Conv Conv Conv Full Full
Feature Extractor Classifier
Sliding Window with ConvNet
Input Window
224 6256
Conv Conv Conv Conv Conv Full Full
Feature Extractor167
240
1
No need to compute two separate windowsJust one big input window, computed in a single pass
C classes
High Level Computer Vision - July 15, 2o15 8
High Level Computer Vision - July 15, 2o15 9
High Level Computer Vision - July 15, 2o15 10
High Level Computer Vision - July 15, 2o15 11
High Level Computer Vision - July 15, 2o15 12
High Level Computer Vision - July 15, 2o15
“Hole” algorithm
• skip subsampling ‣ in their case for VGG-net: after the last two max-pooling layers)
• for the next layer filter: sparsely sample the feature map with “input stride” 2 (or 4 respectively)
13
High Level Computer Vision - July 15, 2o15 14
High Level Computer Vision - July 15, 2o15
CRF - Conditional Random Field
• Energy function to be minimized
‣ with unary terms obtained from the CNN:
‣ and pairwise terms (Potts model)
- with
15
High Level Computer Vision - July 15, 2o15 16
High Level Computer Vision - July 15, 2o15 17
High Level Computer Vision - July 15, 2o15 18
High Level Computer Vision - July 15, 2o15 19
High Level Computer Vision - July 15, 2o15 20
High Level Computer Vision - July 15, 2o15
Another fully convolutional network for semantic segmentation (without CRF)
21
High Level Computer Vision - July 15, 2o15 22
High Level Computer Vision - July 15, 2o15 23