cvpr 2013 diversity tutorial closing remarks: what can we do with multiple diverse solutions? dhruv...
TRANSCRIPT
CVPR 2013 Diversity Tutorial
Closing Remarks:What can we do with multiple
diverse solutions?
Dhruv Batra
Virginia Tech
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 2
Example Result
Now what?
CVPR 2013 Diversity Tutorial
Your Options• Nothing
– User in the loop
• (Approximate) Min Bayes Risk– Use solutions to estimate the distribution and optimize
Bayes Risk
• Re-ranking– Pick a good solution from the list
(C) Dhruv Batra 3
Increasing Side Information
CVPR 2013 Diversity Tutorial
Interactive Segmentation• Setup
– Model: Color/Texture + Potts Grid CRF– Inference: Graph-cuts– Dataset: 50 train/val/test images
(C) Dhruv Batra 4
Image + Scribbles Diverse 2nd Best2nd Best MAPMAP
1-2 Nodes Flipped 100-500 Nodes Flipped
CVPR 2013 Diversity Tutorial
Interactive Segmentation
(C) Dhruv Batra 5
MAP M-Best-MAP Confidence DivMBest89%
90%
91%
92%
93%
94%
95%
96%
+0.05%
+1.61%
+3.62%
(Oracle) (Oracle) (Oracle)
M=6
Seg
men
tatio
n A
ccur
acy
Better
CVPR 2013 Diversity Tutorial
Your Options• Nothing
– User in the loop
• (Approximate) Min Bayes Risk– Use solutions to estimate the distribution and optimize
Bayes Risk
• Re-ranking– Pick a good solution from the list
(C) Dhruv Batra 6
CVPR 2013 Diversity Tutorial
Statistics 101• Loss
– PCP, Pascal Loss, etc
• “True” Distribution
• Expected Loss:
• Min Bayes Risk
(C) Dhruv Batra 7
CVPR 2013 Diversity Tutorial
Structured Output Problems• Min Bayes Risk
• Two Problems
• Approximate MBR:
(C) Dhruv Batra 8
IntractableIntractable
CVPR 2013 Diversity Tutorial
Semantic Segmentation• Setup
– Models: • Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]
• Second-Order Pooling [Carreira ECCV ‘12]
– Inference: • Alpha-expansion• Greedy
– Dataset: Pascal Segmentation Challenge (VOC 2012) • 20 categories + background; ~1500 train/val/test images
(C) Dhruv Batra 9
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 10
Large-Margin Re-ranking
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 11
Input MAP Best of 10-Div
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 12
PA
CA
L A
ccur
acy
Better
#Solutions / Image
1 2 3 4 5 6 7 8 9 1044%
47%
50%
53%
56%
59%
MAP[State-of-art circa 2012]
15%-gain possible
Same FeaturesSame Model
DivMBest (Oracle)
Rand (Re-rank)
MBR
CVPR 2013 Diversity Tutorial
Your Options• Nothing
– User in the loop
• (Approximate) Min Bayes Risk– Use solutions to estimate the distribution and optimize
Bayes Risk
• Re-ranking– Pick a good solution from the list
(C) Dhruv Batra 13
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 14
Large-Margin Re-ranking
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 15
Large-Margin Re-ranking
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 16
Large-Margin Re-ranking
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 17
Large-Margin Re-ranking
Discriminative Re-ranking of Diverse Segmentation
[Yadollahpour et al., CVPR13, Wednesday Poster]
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 18
PA
CA
L A
ccur
acy
Better
#Solutions / Image
1 2 3 4 5 6 7 8 9 1044%
47%
50%
53%
56%
59%
MAP[State-of-art circa 2012]
DivMBest (Oracle)
Rand (Re-rank)
DivMBest (Re-ranked) [Y.B.S., CVPR ‘13]
MBR
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 19
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 20
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 21
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 22
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 23
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 24
CVPR 2013 Diversity Tutorial
Summary• All models are wrong
• Some beliefs are useful
• Diverse Multiple Solutions– A way to get useful beliefs out.
• DivMBest + Reranking– Big impact possible on many applications!
(C) Dhruv Batra 25
CVPR 2013 Diversity Tutorial
Summary
• What does my model believe?
(C) Dhruv Batra 26
Posterior Summary
CVPR 2013 Diversity Tutorial
Thanks!
(C) Dhruv Batra 27