unsupervised feature selection with adaptive...
TRANSCRIPT
![Page 1: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/1.jpg)
Unsupervised Feature Selection with Adaptive Structure Learning
Reporter: Songling Liu
Data Mining LabBig Data Research Center
![Page 2: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/2.jpg)
Overview
• Current feature selection methods• Motivation• Method• Optimization Algorithm• Experiments• Conclusion
![Page 3: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/3.jpg)
Current feature selection methods
![Page 4: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/4.jpg)
Current feature selection methods
Filter:Addressing this issue by selecting the top ranked features based on some scores computed independently for each feature.
feature 1 feature 2 feature 3
If in variance:1>2>3 and we choose the largest two
![Page 5: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/5.jpg)
Embedded Method:Using all features to estimate the underlying structures of data and select features which can preserve data structures.
Current feature selection methods
Common drawback--Using all features whichcould be redundant and include noisy features.
![Page 6: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/6.jpg)
Current feature selection methods
a
b
a
c
a
d = b
Suppose a dataset with 4 features a,b,c and d.
![Page 7: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/7.jpg)
Data structure estimating VS Feature selection
![Page 8: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/8.jpg)
We want to preserve the similarity after feature selectionPairwise distance prefered to be unchanged
![Page 9: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/9.jpg)
Motivation
• A unified learning framework which performs structure learning and feature selection simultaneously.• Select good features from well-estimated data structure.• Estimate data structure with good features.
![Page 10: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/10.jpg)
How to preserve pairwise global structure?
PCA? MaxVar?
However, such dense similarity becomes less discriminative for high dimension data, especially when there are many unfavorable features in the original high dimensional space.
Problem I
![Page 11: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/11.jpg)
Method
Adaptive Global Structure Learning
using the sparse reconstruction coeffcients to extract the global structure of data.
![Page 12: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/12.jpg)
Method
the selected features should preserve such globaland sparse reconstruction structure
Compared with the last equation:1) The global structure captured by S can be used to guide the search of relevant features; 2) the global structure can also be better estimated.
![Page 13: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/13.jpg)
How to preserve pairwise local structure?
LLE? Graph Laplacian?
They would be inevitably affected by the redundant and noisy features. Moreover, the iterative updating of discrete neighborhood relationship using the result of feature selection still sffuers from the lack of theoretical guarantee of its convergence.
Problem II
![Page 14: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/14.jpg)
Method
Adaptive Local Structure Learning
Learn a euclidean distance induced probabilistic neighborhood matrix
Using matrix P,graph laplacian can be characterized as
Where Dp’s element is
![Page 15: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/15.jpg)
Method
we can learn a better probabilistic neighborhood graph for local structure characterization
Using the sparse matrix to select infomative features
![Page 16: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/16.jpg)
Method
Unsupervised Feature Selection with AdaptiveStructure Learning
![Page 17: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/17.jpg)
Method
When S and P fixedIt selects those features to well respect both the global and local structure of data;
When W fixedIt learns the global and local structure of data in a transformed space
![Page 18: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/18.jpg)
Optimization Algorithm
First, when W and P are fixed,
Where
We get sparsity representation of instances,which means global structure
![Page 19: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/19.jpg)
Optimization Algorithm
Next, when W and S are fixed,
With transformation
Where
![Page 20: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/20.jpg)
Next, when S and P are fixed,
Optimization Algorithm
![Page 21: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/21.jpg)
Optimization Algorithm
![Page 22: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/22.jpg)
Experiments
![Page 23: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/23.jpg)
Experiments
![Page 24: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/24.jpg)
Experiments
![Page 25: Unsupervised Feature Selection with Adaptive …dm.uestc.edu.cn/wp-content/uploads/seminar...structures of data and select features which can preserve data structures. Current feature](https://reader034.vdocuments.net/reader034/viewer/2022042401/5f1037bd7e708231d44805bb/html5/thumbnails/25.jpg)
Q&A