unsupervised learning of hierarchical spatial structures
DESCRIPTION
Unsupervised Learning of Hierarchical Spatial Structures. Devi Parikh , Larry Zitnick and Tsuhan Chen. Our visual world…. Intro Approach Results Conclusion. What is an object?. What is context?. … hierarchical spatial patterns. Goal. Intro Approach Results Conclusion. - PowerPoint PPT PresentationTRANSCRIPT
Unsupervised Learning of Hierarchical Spatial Structures
Devi Parikh, Larry Zitnick and Tsuhan Chen
2… hierarchical spatial patterns
Our visual world…
What is an object?What is context?
Intro
Approach
Results
Conclusion
3
Goal
Unsupervised!
Intro
Approach
Results
Conclusion
4
Related work
[Todorovic 2008]
[Fidler 2007] [Zhu 2008]
[Sivic 2008]
Fully unsupervised
Structure and parameters learnt
From features to multiple objects
Intro
Approach
Results
Conclusion
5
ModelRule based
c2
c4
c1
c2
c3
r1 0.9
0.1
0.60.7
0.6
Intro
Approach
Results
Conclusion
6
c2
r2
c1
c2
c3
r1 0.9
0.1
0.60.7
0.6
ModelRule based
Intro
Approach
Results
Conclusion
7
c2
r2
c1
c2
c3
r1 0.9
0.1
0.60.7
0.6
ModelHierarchical rule-based
Intro
Approach
Results
Conclusion
8
Rules R
Image-parts V
Model
Codewords C
Features F
Intro
Approach
Results
Conclusion
9
Model NotationV = {v} instantiated image-parts
rv rule corresponding to instantiated part v
Ch(rv) = {x} children of rule rv
includes instantiated children Ch(v) and un-instantiated children
Intro
Approach
Results
Conclusion
10
Model
Intro
Approach
Results
Conclusion
11
Inference
Intro
Approach
Results
Conclusion
12
Inference
Intro
Approach
Results
Conclusion
13
Inference
Intro
Approach
Results
Conclusion
14
Inference
Intro
Approach
Results
Conclusion
15
Inference
Intro
Approach
Results
Conclusion
16
Inference
Intro
Approach
Results
Conclusion
17
Inference
Intro
Approach
Results
Conclusion
18
Inference
Intro
Approach
Results
Conclusion
19
Inference
Intro
Approach
Results
Conclusion
20
Inference
Intro
Approach
Results
Conclusion
21Minimum Cost
Steiner TreeCharikar 1998
Inference
Intro
Approach
Results
Conclusion
22
Inference
Intro
Approach
Results
Conclusion
23
Generalized distance transformFelzenszwalb et al. 2001
Inference
Intro
Approach
Results
Conclusion
24
EM style
Initialize rules
Infer rules Update parameters Modify rules
Learning
Intro
Approach
Results
Conclusion
25
Initialize rules
…
Learning
Intro
Approach
Results
Conclusion
26
Inference
…
Learning
Intro
Approach
Results
Conclusion
27
Inference
…
Learning
Intro
Approach
Results
Conclusion
28
Add children
…
Learning
Intro
Approach
Results
Conclusion
29
Add children
Update parameters
Pruning children
Removing rules
…
Learning
Intro
Approach
Results
Conclusion
30
Adding rules
Randomly add rules
…
…
Learning
Intro
Approach
Results
Conclusion
31
Behavior Competition among rules Competition with root (noise)
Intro
Approach
Results
Conclusion
32
Behavior Competition among rules Competition with root (noise) Dropping children and rules Number of children Structure of DAG and tree # rules, parameters, structure learnt automatically Multiple instantiations of rules Multiple children with same appearance
Intro
Approach
Results
Conclusion
Experiment 1: Faces & MotorbikesIntro
Approach
Results
Conclusion
34
Faces and Motorbikes SIFT (200 words)
Learnt 15 L1 rules, 2 L2 rules Each L1 rule average ~7 children Each L2 rule average ~4 children
Faces & Motorbikes
Intro
Approach
Results
Conclusion
35
Example rules
Intro
Approach
Results
Conclusion
36
Patches
Intro
Approach
Results
Conclusion
37
Localization behavior
Intro
Approach
Results
Conclusion
38
Categorization behavior
Faces Motorbikes Faces Motorbikes Faces Motorbikes
occu
rren
ce
code-words first level rules second level rules
Intro
Approach
Results
Conclusion
39
Categorization behavior
Words Rules Tree
Words: 94 %
Tree: 100%
KmeansPLSASVM
Intro
Approach
Results
Conclusion
40
Edge features
Words: 55 %
Tree: 82%
Intro
Approach
Results
Conclusion
Experiment 2: Six categoriesIntro
Approach
Results
Conclusion
42
Six categories
61 L1 rules (~9 children)12 L2 rules (~3 children)
Kim 2008: 95 %
Words: 87 %
Tree: 95 %
Intro
Approach
Results
Conclusion
Experiment 3: Scene categoriesIntro
Approach
Results
Conclusion
44
Scene categories
Image Segmentation
Mean color Codeword
Intro
Approach
Results
Conclusion
45
Outdoor scenes
rule
s
images
Intro
Approach
Results
Conclusion
Experiment 4: Structured street scenesIntro
Approach
Results
Conclusion
47
Windows
Intro
Approach
Results
Conclusion
48
Object categories
Intro
Approach
Results
Conclusion
49
Object categories
Intro
Approach
Results
Conclusion
50
Object categories
Intro
Approach
Results
Conclusion
51
Parts of objects
Intro
Approach
Results
Conclusion
52
Multiple objects
Intro
Approach
Results
Conclusion
53
Street Scenes (PLSA)
Intro
Approach
Results
Conclusion
54
Dataset specific rules
irrelevant
relevantIntro
Approach
Results
Conclusion
55
Conclusion
Unsupervised learning of hierarchical spatial patterns Low level features, object parts, objects, regions in scene
Rule-based approach Learning: EM style Inference: Minimum cost Steiner tree
Features SIFT, edges, color segments
Intro
Approach
Results
Conclusion
56
Summary
I
Root
Scene
Objects
Object Parts
Features
Intro
Approach
Results
Conclusion