Learning robust graph for clustering
Zheng Liu
College of Control Science and Engineering, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
Search for more papers by this authorCorresponding Author
Wei Jin
College of Control Science and Engineering, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
College of Control Science and Engineering, Huzhou Institute of Zhejiang University, Huzhou, China
Correspondence Wei Jin, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, 310027 Hangzhou, China.
Email: [email protected]
Search for more papers by this authorYing Mu
College of Control Science and Engineering, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
Search for more papers by this authorZheng Liu
College of Control Science and Engineering, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
Search for more papers by this authorCorresponding Author
Wei Jin
College of Control Science and Engineering, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
College of Control Science and Engineering, Huzhou Institute of Zhejiang University, Huzhou, China
Correspondence Wei Jin, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, 310027 Hangzhou, China.
Email: [email protected]
Search for more papers by this authorYing Mu
College of Control Science and Engineering, Research Center for Analytical Instrumentation, Institute of Cyber-Systems and Control, State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
Search for more papers by this authorAbstract
Graph is a popular technique to explore the structure of data. Many related algorithms directly construct graphs based on the original data. Actually, the samples collected in real life usually contain noise. Besides, some unimportant features probably exist in high-dimensional data. Therefore, this way cannot assure a high-quality graph and furthermore brings some adverse influence to the following tasks. In this paper, we incorporate robust graph learning and dimensionality reduction into a unified framework which also seamlessly integrates the clustering task. On the basis of the framework, Euclidean distance-based robust graph (EDBRG) and self-expressiveness-based robust graph (SEBRG) are presented. Both EDBRG and SEBRG contain clustering information from which the clustering results can be obtained directly. By projecting the original data into a discriminative subspace where the negative effect of redundant features and noise is removed, EDBRG and SEBRG are informative, robust, and sparse. During the whole mapping process, the main energy of data is preserved. Finally, some data sets are adopted to test the performances of EDBRG and SEBRG. Extensive experiments illustrate that the proposed methods have many advantages for the task of clustering, comparing with the state-of-the-art algorithms.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1Li W, Ke L, Meng W, Han J. An empirical study of supervised email classification in Internet of Things: practical performance and key influencing factors. Int J Intell Syst. 2022; 37(1): 287-304.
- 2Li H, Yan H, Li L, Pan Z, Liu X, Zhang Z. MHAT: an efficient model-heterogeneous aggregation training scheme for federated learning. Inf Sci. 2021; 560: 493-503.
- 3Lu S, Zhu Z, Gorriz JM, Wang SH, Zhang YD. NAGNN: classification of COVID-19 based on neighboring aware representation from deep graph neural network. Int J Intell Syst. 2022; 37(2): 1572-1598.
- 4Lian H, Xu H, Wang S, Li M, Zhu X, Liu X. Partial multiview clustering with locality graph regularization. Int J Intell Syst. 2021; 36(6): 2991-3010.
- 5Ai S, Hong S, Zheng X, Wang Y, Liu X. CSRT rumor spreading model based on complex network. Int J Intell Syst. 2021; 36(5): 1903-1913.
- 6Hossain RR, Huang Q, Huang R. Graph convolutional network-based topology embedded deep reinforcement learning for voltage stability control. IEEE Trans Power Syst. 2021; 36(5): 4848-4851.
- 7Mo K, Tang W, Li J, Yuan X. Attacking deep reinforcement learning with decoupled adversarial policy. IEEE Trans Dependable Secur Comput. 2022. doi:10.1109/TDSC.2022.3143566
- 8Chong Y, Ding Y, Yan Q, Pan S. Graph-based semi-supervised learning: a review. Neurocomputing. 2020; 408: 216-230.
- 9Lin Z, Kang Z, Zhang L, Tian L. Multi-view attributed graph clustering. IEEE Trans Knowl Data Eng. 2021. doi:10.1109/TKDE.2021.3101227
- 10Huang G, Song S, Gupta JND, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern. 2017; 44(12): 2405-2417.
- 11Elhamifar E, Vidal R. Sparse subspace clustering. In: IEEE Conference on Computer Vision & Pattern Recognition; 2009: 2790-2797.
- 12Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y. Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell. 2013; 35(1): 171-184.
- 13Kang Z, Wen L, Chen W, Xu Z. Low-rank kernel learning for graph-based clustering. Knowl-Based Syst. 2019; 163: 510-517.
- 14Wen J, Zhang B, Xu Y, Jian Y, Na H. Adaptive weighted nonnegative low-rank representation. Pattern Recognit. 2018; 81: 326-340.
- 15Wang Y, Xu Y, Leng C. Provable subspace clustering: when LRR meets SSC. IEEE Trans Inf Theory. 2019; 65(9): 5406-5432.
- 16Lv J, Kang Z, Lu X, Xu Z. Pseudo-supervised deep subspace clustering. IEEE Trans Image Process. 2021; 30: 5252-5263.
- 17Zhang J, Li C, You C, et al. Self-supervised convolutional subspace clustering network. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019: 5468-5477.
10.1109/CVPR.2019.00562 Google Scholar
- 18Wen J, Fang X, Yong X, Tian C, Fei L. Low-rank representation with adaptive graph regularization. Neural Networks. 2018; 108: 83-96.
- 19Zhuang L, Gao S, Tang J, Wang J, Yi M. Constructing a non-negative low rank and sparse graph with data-adaptive features. IEEE Trans Image Process. 2014; 24(11): 3717-3728.
- 20Yin M, Gao J, Lin Z. Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell. 2016; 38(3): 504-517.
- 21Fang X, Na H, Wu J, Yong X, Li X. Approximate low-rank projection learning for feature extraction. IEEE Trans Neural Networks Learn Syst. 2018; 29(11): 5228-5241.
- 22Zhan S, Wu J, Han N, Wen J, Fang X. Unsupervised feature extraction by low-rank and sparsity preserving embedding. Neural Networks. 2019; 109: 56-66.
- 23Xie L, Ming Y, Yin X, Liu Y, Yin G. Low-rank sparse preserving projections for dimensionality reduction. IEEE Trans Image Process. 2018; 27: 5261-5274.
- 24Fei L, Yong X, Fang X, Jian Y. Low rank representation with adaptive distance penalty for semi-supervised subspace classification. Pattern Recognit. 2017; 67: 252-262.
- 25Kang Z, Pan H, Hoi S, Xu Z. Robust graph learning from noisy data. IEEE Trans Cybern. 2020; 50(5): 1833-1843.
- 26Xiang S, Zhu Y, Shen X. Optimal exact least squares rank minimization. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2012: 480-488.
10.1145/2339530.2339609 Google Scholar
- 27Nie F, Cai G, Li X. Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours. AAAI; 2017: 2408-2414.
- 28Fang X, Xu Y, Li X, Lai Z, Wong WK. Learning a nonnegative sparse graph for linear regression. IEEE Trans Image Process. 2015; 24(6): 2760-2771.
- 29Nie F, Wang X, Huang H. Clustering and projected clustering with adaptive neighbors. In: KDD'14; 2014: 977-986.
10.1145/2623330.2623726 Google Scholar
- 30Liu Z, Huang S, Jin W, Mu Y. Broad learning system for semi-supervised learning. Neurocomputing. 2021; 444: 38-47.
- 31Candes E, Li X, Ma Y, Wright J. Robust principal component analysis? J ACM. 2011; 58(3): 11.
- 32Wen J, Fang X, Cui J, et al. Robust sparse linear discriminant analysis. IEEE Trans Circuits Syst Video Technol. 2019; 29(2): 390-403.
- 33Fan K. On a theorem of Weyl concerning eigenvalues of linear transformations. Proc Nat Acad Sci. 1950; 36(1): 31-35.
- 34Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn. 2011; 3(1): 1-122.
- 35Zou H, Hastie T, Tibshirani R. Sparse principal component analysis. J Comput Graph Stat. 2006; 15(2): 265-286.
- 36Nie F, Wang X, Jordan MI, Huang H. The Constrained Laplacian Rank Algorithm for Graph-Based Clustering. AAAI; 2016: 1969-1976.
- 37Fang X, Teng S, Lai Z. Robust latent subspace learning for image classification. IEEE Trans Neural Networks Learn Syst. 2017; 29(6): 2502-2515.
- 38Lin Q, Yang L, Zhong P, Zou H. Robust supervised multi-view feature selection with weighted shared loss and maximum margin criterion. Knowl-Based Syst. 2021; 229:107331. doi:10.1016/j.knosys.2021.107331
- 39Liu G, Yan S. Latent low-rank representation for subspace segmentation and feature extraction. In: ICCV; 2011: 6-13.
- 40Kang Z, Lin Z, Zhu X, Xu W. Structured graph learning for scalable subspace clustering: from single-view to multi-view. IEEE Trans Cybern. 2021. doi:10.1109/TCYB.2021.3061660
- 41Shi D, Zhu L, Li Y, Li J, Nie X. Robust structured graph clustering. IEEE Trans Neural Networks Learn Syst. 2020; 31(11): 4424-4436.
- 42Cuell C, Bonsai B. An assessment of climatological synoptic typing by principal component analysis and kmeans clustering. Theor Appl Climatol. 2009; 98(3-4): 361-373.