Partial multiview clustering with locality graph regularization
Huiqiang Lian
University of Chinese Academy of Sciences, Beijing, China
Search for more papers by this authorCorresponding Author
Huiying Xu
College of Mathematics and Computer Science, Zhejiang Normal University, Jinhua, China
Correspondence Huiying Xu, College of Mathematics and Computer Science, Zhejiang Normal University, 321004 Jinhua, China.
Email: [email protected]
Search for more papers by this authorSiwei Wang
College of Computer, National University of Defense Technology, Changsha, China
Search for more papers by this authorMiaomiao Li
College of Electronic Information and Electrical Engineering, Changsha University, Changsha, China
Search for more papers by this authorXinzhong Zhu
College of Mathematics and Computer Science, Zhejiang Normal University, Jinhua, China
Research Institute of Ningbo Cixing Co. Ltd, Cixi City, China
Search for more papers by this authorXinwang Liu
College of Computer, National University of Defense Technology, Changsha, China
Search for more papers by this authorHuiqiang Lian
University of Chinese Academy of Sciences, Beijing, China
Search for more papers by this authorCorresponding Author
Huiying Xu
College of Mathematics and Computer Science, Zhejiang Normal University, Jinhua, China
Correspondence Huiying Xu, College of Mathematics and Computer Science, Zhejiang Normal University, 321004 Jinhua, China.
Email: [email protected]
Search for more papers by this authorSiwei Wang
College of Computer, National University of Defense Technology, Changsha, China
Search for more papers by this authorMiaomiao Li
College of Electronic Information and Electrical Engineering, Changsha University, Changsha, China
Search for more papers by this authorXinzhong Zhu
College of Mathematics and Computer Science, Zhejiang Normal University, Jinhua, China
Research Institute of Ningbo Cixing Co. Ltd, Cixi City, China
Search for more papers by this authorXinwang Liu
College of Computer, National University of Defense Technology, Changsha, China
Search for more papers by this authorAbstract
Multiview clustering (MVC) collects complementary and abundant information, which draws much attention in machine learning and data mining community. Existing MVC methods usually hold the assumption that all the views are complete. However, multiple source data are often incomplete in real-world applications, and so on sensor failure or unfinished collection process, which gives rise to incomplete multiview clustering (IMVC). Although enormous efforts have been devoted in IMVC, there still are some urgent issues that need to be solved: (i) The locality among multiple views has not been utilized in the existing mechanism; (ii) Existing methods inappropriately force all the views to share consensus representation while ignoring specific structures. In this paper, we propose a novel method termed partial MVC with locality graph regularization to address these issues. First, followed the traditional IMVC approaches, we construct weighted semi-nonnegative matrix factorization models to handle incomplete multiview data. Then, upon the consensus representation matrix, the locality graph is constructed for regularizing the shared feature matrix. Moreover, we add the coefficient regression term to constraint the various base matrices among views. We incorporate the three aforementioned processes into a unified framework, whereas they can negotiate with each other serving for learning tasks. An effective iterative algorithm is proposed to solve the resultant optimization problem with theoretically guaranteed convergence. The comprehensive experiment results on several benchmarks demonstrate the effectiveness of the proposed method.
REFERENCES
- 1Gao J, Han J, Liu J, Wang C. Multi-view clustering via joint nonnegative matrix factorization. In: Proceedings of the 2013 SIAM International Conference on Data Mining; 2013. https://epubs.siam.org/doi/10.1137/1.9781611972832.28
- 2Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y. Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell. 2013; 35(1): 171-184.
- 3Wang S, Zhu E, Hu J, et al. Efficient multiple kernel k-means clustering with late fusion. IEEE Access. 2019; 7: 61109-61120.
- 4Wang S, Liu X, Zhu E, et al. Multi-view clustering via late fusion alignment maximization. In: IJCAI; 2019:3778-3784.
- 5Liang W, Zhou S, Xiong J, et al. Multi-view spectral clustering with high-order optimal neighborhood laplacian matrix. IEEE Trans Knowl Data Eng. 2020; 1-1. http://doi.org/10.1109/tkde.2020.3025100
- 6Ou Q, Wang S, Zhou S, Li M, Guo X, Zhu E. Anchor-based multiview subspace clustering with diversity regularization. IEEE MultiMedia. 2020; 27(4): 91-101.
- 7Liu X, Zhu X, Li M, et al. Late fusion incomplete multi-view clustering. IEEE Trans Pattern Anal Mach Intell. 2018; 41(10): 2410-2423.
- 8Liu X, Zhu X, Li M, et al. Efficient and effective incomplete multi-view clustering. Proc AAAI Conf Artif Intell. 2019; 33: 4392-4399.
- 9Liu X, Zhu X, Li M, et al. Multiple kernel k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell. 2019; 42(5): 1191-1204.
- 10Liu X, Li M, Tang C, et al. Efficient and effective regularized incomplete multi-view clustering. In: IEEE Transactions on Pattern Analysis and Machine Intelligence; 2020.
- 11Zhang P, Liu X, Xiong J, et al. Consensus one-step multi-view subspace clustering. In: IEEE Transactions on Knowledge and Data Engineering; 2020:1. https://doi.org/10.1109/TKDE.2020.3045770
- 12Meng F, Tang J, Xu Z. Deriving priority weights from intuitionistic fuzzy multiplicative preference relations. Int J Intell Syst. 2019; 34(11): 2937-2969.
- 13Fei L, Feng Y, Liu L. Evidence combination using OWA-based soft likelihood functions. Int J Intell Syst. 2019; 34(9): 2269-2290.
- 14Merigó JM, Blanco-Mesa F, Gil-Lafuente AM, Yager RR. Thirty years of the International journal of intelligent systems: a bibliometric review. Int J Intell Syst. 2017; 32(5): 526-554.
- 15Garg H, Rani D. Some results on information measures for complex intuitionistic fuzzy sets. Int J Intell Syst. 2019; 34(10): 2319-2363.
- 16Torra V. Hesitant fuzzy sets. Int J Intell Syst. 2010; 25(6): 529-539.
- 17Li SY, Jiang Y, Zhou ZH. Partial multi-view clustering. Proc AAAI Conf Artif Intell. 2014; 28: 1.
- 18Zhao H, Liu H, Fu Y. Incomplete multi-modal visual data grouping. In: IJCAI; 2016:2392-2398.
- 19Hu M, Chen S. Doubly aligned incomplete multi-view clustering. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence; 2018: 2262-2268.
- 20Hu J, Zhu E, Wang S, Wang S, Liu X, Yin J. Two-stage unsupervised video anomaly detection using low-rank based unsupervised one-class learning with ridge regression. In: 2019 International Joint Conference on Neural Networks (IJCNN); 2019:1-8.
- 21Wang S, Li M, Hu N, et al. K-means clustering with incomplete data. IEEE Access. 2019; 7: 69162-69171.
- 22Zhuge W, Hou C, Liu X, Tao H, Yi D. Simultaneous representation learning and clustering for incomplete multi-view data. In: IJCAI; 2019:4482-4488.
- 23Hu M, Chen S. One-pass incomplete multi-view clustering. Proc AAAI Conf Artif Intell. 2019; 33: 3838-3845.
- 24Xu C, Guan Z, Zhao W, Wu H, Niu Y, Ling B. Adversarial incomplete multi-view clustering. In: IJCAI; 2019:3933-3939.
- 25Zhuge W, Luo T, Tao H, Hou C, Yi D. Multi-view spectral clustering with incomplete graphs. IEEE Access. 2020; 8: 99820-99831.
- 26Deng W, Liu L, Li J, Lin Y. Auto-weighted incomplete multi-view clustering. IEEE Access. 2020; 8: 138752-138762. http://doi.org/10.1109/access.2020.2997755
- 27Ji M, Rao H, Li Z, Zhu J, Wang N. Partial multi-view clustering based on sparse embedding framework. IEEE Access. 2019; 7: 29332-29343.
- 28Wang Q, Ding Z, Tao Z, Gao Q, Fu Y. Partial multi-view clustering via consistent GAN. In: 2018 IEEE International Conference on Data Mining (ICDM); 2018:1290-1295.
- 29Shao W, He L, Philip SY. Multiple incomplete views clustering via weighted nonnegative matrix factorization with L2, 1 regularization. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases; 2015: 318-334.
- 30Gao H, Peng Y, Jian S. Incomplete multi-view clustering. In: International Conference on Intelligent Information Processing; 2016: 245-255.
- 31Zhang P, Wang S, Hu J, et al. Adaptive weighted graph fusion incomplete multi-view subspace clustering. Sensors. 2020; 20(20): 5755.
- 32Ding CH, Li T, Jordan MI. Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell. 2008; 32(1): 45-55.
- 33Ren Z, Sun Q. Simultaneous global and local graph structure preserving for multiple kernel clustering. IEEE Trans Neural Netw Learn Syst. 2020; 1-13.
- 34Wen J, Fang X, Xu Y, Tian C, Fei L. Low-rank representation with adaptive graph regularization. Neural Netw. 2018; 108: 83-96.
- 35Greene D, Cunningham P. Practical solutions to the problem of diagonal dominance in kernel document clustering. In: Proceedings of the 23rd international conference on Machine learning; 2006: 377-384.
- 36Wen J, Xu Y, Liu H. Incomplete multiview spectral clustering with adaptive graph learning. IEEE Trans Cybernet. 2020; 50(4): 1418-1429.
- 37Elhamifar E, Vidal R. Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell. 2013; 35(11): 2765-2781.
- 38Parsons L, Haque E, Liu H. Subspace clustering for high dimensional data: a review. ACM Sigkdd Explorations Newsletter. 2004; 6(1): 90-105.
- 39Liu X, Wang L, Huang GB, Zhang J, Yin J. Multiple kernel extreme learning machine. Neurocomputing. 2015; 149: 253-264.
- 40Liu X, Dou Y, Yin J, Wang L, Zhu E. Multiple kernel k-means clustering with matrix-induced regularization. In: Thirtieth AAAI Conference on Artificial Intelligence; 2016.
- 41Wang H, Yang Y, Liu B. GMC: graph-based multi-view clustering. IEEE Trans Knowl Data Eng. 2019; 32(6): 1116-1129.
- 42Cai X, Nie F, Huang H. Multi-view k-means clustering on big data. In: Twenty-Third International Joint Conference on Artificial Intelligence; 2013.
- 43Nie F, Li J, Li X, et al. Parameter-free auto-weighted multiple graph learning: a framework for multiview clustering and semi-supervised classification. In: IJCAI; 2016:1881-1887.
- 44Guo Y. Convex subspace representation learning from multi-view data. In: Twenty-Seventh AAAI Conference on Artificial Intelligence; 2013.
- 45Greene D, Cunningham P. A matrix factorization approach for integrating multiple data views. In: Joint European conference on machine learning and knowledge discovery in databases; 2009: 423-438.
- 46Wang Y, Wu L, Lin X, Gao J. Multiview spectral clustering via structured low-rank matrix factorization. IEEE Trans Neural Netw Learn Syst. 2018; 29(10): 4833-4843.
- 47Zhang Z, Liu L, Shen F, Shen HT, Shao L. Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell. 2018; 41(7): 1774-1782.
- 48Chang W, Wei C, Wang YF. Multi-view nonnegative matrix factorization for clothing image characterization. In: 2014 22nd International Conference on Pattern Recognition; 2014: 1272-1277.
- 49Yu S, Tranchevent L, Liu X, et al. Optimized data fusion for kernel k-means clustering. IEEE Trans Pattern Anal Mach Intell. 2012; 34(5): 1031-1039.
- 50Chen J, Zhao Z, Ye J, Liu H. Nonlinear adaptive distance metric learning for clustering. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2007: 123-132.
- 51Chaudhuri K, Kakade SM, Livescu K, Sridharan K. Multi-view clustering via canonical correlation analysis. In: Proceedings of the 26th Annual International Conference on Machine Learning; 2009: 129-136.
- 52Huang HC, Chuang YY, Chen CS. Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst. 2012; 20(1): 120-134.
- 53Liu X, Wang L, Yin J, Zhu E, Zhang J. An efficient approach to integrating radius information into multiple kernel learning. IEEE Trans Cybernet. 2013; 43(2): 557-569.
- 54Mallah C, Cope J, Orwell J. Plant leaf classification using probabilistic integration of shape, texture and margin features. Signal Process Pattern Recogn Appl. 2013; 5(1): 45-54.
- 55Zhu X, Liu X, Li M, et al. Localized incomplete multiple Kernel k-means. In: IJCAI; 2018:3271-3277.