a general model for relational clustering bo long and zhongfei (mark) zhang computer science...
Post on 19-Dec-2015
220 views
TRANSCRIPT
![Page 1: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/1.jpg)
A General Model for Relational Clustering
Bo Long and Zhongfei (Mark) ZhangComputer Science Dept./Watson SchoolSUNY Binghamton
Xiaoyun WuYahoo! Inc.
Philip S.YuIBM Watson Research Center
![Page 2: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/2.jpg)
Multi-type Relational Data (MTRD) is Everywhere! Bibliometrics
Papers, authors, journals
Social networks People, institutions, friendship links
Biological data Genes, proteins, conditions
Corporate databases Customers, products, suppliers, shareholders
Papers
Authors
Key words
![Page 3: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/3.jpg)
Challenges for Clustering! Data objects are not identically distributed:
Heterogeneous data objects (papers, authors). Data objects are not independent
Heterogeneous data objects are related to each other.
No IID assumption
![Page 4: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/4.jpg)
Relational Data Flat Data?Paper ID word1 word2 …… author1 author2 …… ………… ………. …….
1 1 3 …… 1 0 …… ……… ……….. ……..
…… …… ……. ……. …… ……. …… ……… ………. ……..
Author ID Paper 1 Paper 2 …… ………… ………. …….
1 1 0 …… ……… ……….. ……..
…… …… ……. …… ……… ………. ……..
High dimensional and sparse data Data redundancy
Word ID Paper 1 Paper 2 …… ………… ………. …….
1 1 3 …… ……… ……….. ……..
…… …… ……. ……. ……… ………. ……..
Papers
Authors
Key words
![Page 5: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/5.jpg)
Relational Data Flat Data? No interactions of hidden structures of
different types of data objects Difficult to discover the global community
structure.
users
Web pages
queries
![Page 6: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/6.jpg)
A General Model: Collective Factorization on Related Matrices Formulate multi-type relational data as a set
of related matrices; cluster different types of objects
simultaneously by factorizing the related matrices simultaneously.
Make use of the interaction of hidden structures of different types of objects.
![Page 7: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/7.jpg)
Data Representation Represent a MTRD set as a set of related matrices:
Relation matrix, R(ij), denotes the relations between ith type of objects and jth type of objects.
Feature matrix, F(i), denotes the feature values for ith type of objects.
Users
Movies Words
Authors
Papers
f
R(12) R (12)
R (23)
F(1)
![Page 8: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/8.jpg)
Matrix Factorization
)()()( iii BCF
Exploring the hidden structure of the data matrix by its factorization:
.
Tjijiij CACR )( )()()()(
Feature basis
matrix
Cluster association
matrix
![Page 9: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/9.jpg)
Model: Collective Factorization on Related Matrices (CFRM)
2)()(
1
)()(
1
2)()()()()(
,,
||||
||)(||min)()()(
ii
mji
iib
mji
Tjijiijija
ABC
BCFw
CACRwijiji
![Page 10: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/10.jpg)
CFRM Model: Example
3
1
2
f
2)3()23()2()23()23(
2)2()12()1()12()12(
2)1()1()1()1(
||)(||
||)(||
||||
Ta
Ta
b
CACRw
CACRw
BCFw
![Page 11: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/11.jpg)
Spectral Clustering Algorithms that cluster points using
eigenvectors of matrices derived from the data
Obtain data representation in the low-dimensional space that can be easily clustered
Traditional spectral clustering focuses on homogeneous data
![Page 12: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/12.jpg)
Main Theorem:
2)()(
1
)()(
1
2)()()()()(
,,
||||
||)(||min)()()(
ii
mji
iib
mji
Tjijiijija
ABC
BCFw
CACRwijiji
mji
iTjjijTiija
mi
iTiiTiib
ICC
CCCRCtrw
CFFCtrwik
iTi
1
)()()()()()(
1
)()()()()(
)(
))()((
)()((max)()(
![Page 13: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/13.jpg)
Algorithm Derivation: Iterative Updating
pj
pjTjjTjpjpa
mjp
TpjTjjpjpja
Tpppb
p
RCCRw
RCCRw
FFwM
1
)()()()()(
)()()()()(
)()()()(
))()((
))()((
))((
))((max )()()(
)( )()(
ppTp
ICCCMCtr
ppTp
where,
![Page 14: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/14.jpg)
Spectral Relaxation
Apply real relaxation to C(p) to let it be an arbitrary orthornormal matrix.
By Ky-Fan Theorem, the optimal solution is given by the leading kp eigenvectors of M(p).
))((max )()()(
)( )()(
ppTp
ICCCMCtr
ppTp
![Page 15: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/15.jpg)
Spectral Relational Clustering (SRC)
![Page 16: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/16.jpg)
Spectral Relational Clustering: Example
Update C (1) as k1 leading eigenvectors of
Update C (2) as k2 leading eigenvectors of
Update C (3) as k3 leading eigenvectors of )23()2()2()23()23()3( )()( RCCRwM TT
a
3
1
2
TTa
TTa
RCCRw
RCCRwM
)()(
)()()23()3()3()23()23(
)12()1()1()12()12()2(
TTa RCCRwM )()( )12()2()2()12()12()1(
![Page 17: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/17.jpg)
Advantages of Spectral Relational Clustering (SRC)
Simple as traditional spectral approaches Applicable to relational data with various
structures. Adaptive low dimension embedding Efficient: O(tmn2k). For sparse data, it is
reduced to O(tmzk) where z denotes the number of non-zero elements
![Page 18: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/18.jpg)
Special case 1: k-means and spectral clustering Flat data: a special MTRD with only one feature
matrix F,
By the main theorem, k-means is equivalent to the trace maximization,
2
,||||min CBF
BC
max ( )T
k
T T
c c Itr C FF C
![Page 19: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/19.jpg)
Special case 2: Bipartite Spectral Graph Partitioning (BSGP) Bipartite graph: a special case of MTRD with one
relation matrix R,
BSGP restricts the clusters of different types of objects to have one-to-one associations, i.e., diagonal constraints on A.
2)2()1(
)(
)(||)(||min
2)2()2(
1)1()1(
T
ICC
ICCCACR
kT
kT
![Page 20: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/20.jpg)
Experiments Bi-type relational data:
Document-word data Tri-type relational data:
Category-document-word data. Comparing algorithms:
Normalized Cut (NC), Bipartite Spectral Graph Partitioning (BSGP), Mutual Reinforcement K-means (MRK) Consistent Bipartite Graph Co-partitioning
(CBGC).
![Page 21: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/21.jpg)
Experimental Results on Bi-type Relational Data
Multi2 Multi3 Multi5 Multi8 Multi100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8NMI Comparisons on Bi-type Relational Data
Norm
aliz
ed M
utu
al In
form
ation
SRC
NC
BSGP
![Page 22: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/22.jpg)
Eigenvectors of a multi2 data set
-1 0 1-1
-0.5
0NC
u2
u 1
0 0.2 0.40
0.5
1BSGP
u2
-1 0 1-1
-0.5
0SRC
u2
u 1
0 5 100
1
2
Number of iterations
Obj
ectiv
e V
alue
Convergence
![Page 23: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/23.jpg)
Experimental Results on Tri-type Relational Data
BRM TM1 TM2 TM30
0.2
0.4
0.6
0.8
1NMI Comparisons on Tri-type Relational Data
Norm
aliz
ed M
utu
al In
form
ation
SRC
MRKCBGC
![Page 24: A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc](https://reader031.vdocuments.net/reader031/viewer/2022020117/56649d395503460f94a13a4f/html5/thumbnails/24.jpg)
Summary Collective Factorization on Related Matrices– a
general model for MTRD clustering. Spectral Relational Clustering– A novel spectral
approach Simple and applicable to relational data with various
structures. Adaptive low dimension embedding Efficient
Theoretic analysis and experiments demonstrate the effectiveness and the promise of the model and of the algorithm.