context-aware clustering

1

Context-Aware Clustering

Junsong Yuan and Ying WuEECS Dept., Northwestern University

2

Contextual pattern and co-occurrences

?

Spatial contexts provide useful cues for clustering

3

K-means revisit

Assumption: data samples are independent

Binary label indicator

Limitation: contextual information of spatial dependency is not considered in clustering data samples

EM Update

4

Clustering higher-level patterns

• Regularized k-means

Distortion in original feature space

Distortion in hamming spacecharactering contextual patterns

Same as traditionalK-means clustering

Regularization term due tocontextual patterns

Not a smooth term!

5

Chicken and Egg Problem

• Hamming distance in clustering contextual patterns

• Matrix form

• Cannot minimize J1 and J2 separately !

J1 is coupled with J2

6

Decoupling

Fix Update

Fix Update

7

Nested-EM solution

NestedE-step

M-stepUpdate and separately

the nested-EM algorithm can converge in finite steps.

Theorem of convergence

8

Simulation results (feature space)

9

Simulation results (spatial space)

10

11

K-m

eans

Initializatio

n

12

1st rou

nd

Fin

al Ph

rases

13

14

15

16

Multiple-feature clustering

• Dataset: handwritten numerical (‘0’-‘9’) from UCI data set– Each digit has three different types of features– Contextual pattern corresponds to compositional feature

• Different types of features serve as contexts of each other– Clustering each type of features into 10 “words”– Clustering 10 “phrases” based on a word-lexicon of size 3x10

17

Conclusion

• A context-aware clustering formulation proposed– Targets on higher-level compositional patterns in

terms of co-occurrences– Discovered contextual patterns can feed back to

improve the primitive feature clustering

• An efficient nested-EM solution which is guaranteed to converge in finite steps

• Successful applications in image pattern discovery and multiple-feature clustering– Can be applied to other general clustering problems