Joint multilabel classification and feature selection based on deep canonical correlation analysis
Liang Dai
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorGuodong Du
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorJia Zhang
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorCorresponding Author
Candong Li
College of Traditional Chinese Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, PR China
Correspondence
Candong Li, College of Traditional Chinese Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, 350122, PR China. Email: [email protected]
Search for more papers by this authorRong Wei
The First Affiliated Hospital, Guizhou University of traditional Chinese Medicine, Guiyang, PR China
Search for more papers by this authorShaozi Li
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorLiang Dai
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorGuodong Du
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorJia Zhang
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorCorresponding Author
Candong Li
College of Traditional Chinese Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, PR China
Correspondence
Candong Li, College of Traditional Chinese Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, 350122, PR China. Email: [email protected]
Search for more papers by this authorRong Wei
The First Affiliated Hospital, Guizhou University of traditional Chinese Medicine, Guiyang, PR China
Search for more papers by this authorShaozi Li
Department of Artificial Intelligence, Xiamen University, Xiamen, PR China
Search for more papers by this authorFunding information: Collaborative Innovation Center of Chinese Oo- long Tea Industry-Collaborative Innovation Center (2011) of Fujian Province, Fujian Province 2011 Collaborative Innovation Center of TCM Health Management, the National Key Research and Development Program of China, 2018YFC0831402; the National Nature Science Foundation of China, 61806172; 61876159; U1705286
Summary
In recent years, multilabel learning has been applied to a lot of application areas and is yet a challenging task. In multilabel learning, an instance often belongs to multiple class labels simultaneously. The labels usually have correlations with others, and mining label correlations is helpful to enhance the multilabel classification performance. Aiming at increasing the accuracy of prediction, Label embedding (LE) is an important technique, and conducive to extracting label information for multilabel learning. In this paper, we present a novel multilabel learning approach via exploiting label correlations, which can be naturally extended to tackle feature selection problem. First, to obtain the discriminative features shared by all labels, the proposed algorithm learns a latent space by employing deep canonical correlation analysis. Then we exploit label correlations by enforcing predictions on similar labels to be similar, thereby improving the prediction performance. Results on several multiple datasets illustrate that the proposed algorithm has the advantages on multilabel classification and feature selection.
REFERENCES
- 1Dai L, Zhang J, Li C, Zhou C, Li S. Multi-label feature selection with application to TCM state identification. Concurr Comput Pract Exp. 2018; 31(23):e4634.
- 2Elisseeff A, Weston J. A kernel method for multi-labelled classification. Advances in Neural Information Processing Systems. Vancouver, BC: MIT Press; 2002: 681-687.
- 3Turnbull D, Barrington L, Torres D, Lanckriet G. Semantic annotation and retrieval of music and sound effects. IEEE Trans Audio Speech Lang Process. 2008; 16(2): 467-476.
- 4Zhang ML, Zhou ZH. A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng. 2013; 26(8): 1819-1837.
- 5Tsoumakas G, Katakis I, Vlahavas I. Mining multi-label data. Data Mining and Knowledge Discovery Handbook. Boston, MA: Springer; 2009: 667-685.
10.1007/978-0-387-09823-4_34 Google Scholar
- 6Read J. A pruned problem transformation method for multi-label classification. Paper presented at: Proceedings of the 2008 New Zealand Computer Science Research Student Conference (NZCSRS 2008); Vol. 143150; 2008:41.
- 7Yu K, Yu S, Tresp V. Multi-label informed latent semantic indexing. Paper presented at: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2005:258-265; ACM.
- 8Zhang ML, Zhou ZH. Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng. 2006; 18(10): 1338-1351.
- 9Zhou ZH, Zhang ML. Multi-label learning; 2017.
- 10Fürnkranz J, Hüllermeier E, Mencia EL, Brinker K. Multilabel classification via calibrated label ranking. Mach Learn. 2008; 73(2): 133-153.
- 11Bhatia K, Jain H, Kar P, Varma M, Jain P. Sparse local embeddings for extreme multi-label classification. Advances in Neural Information Processing Systems. Montreal, QC: Curran Associates, Inc; 2015: 730-738.
- 12Cao X, Zhang H, Guo X, Liu S, Meng D. Sled: semantic label embedding dictionary representation for multilabel image annotation. IEEE Trans Image Process. 2015; 24(9): 2746-2759.
- 13Yeh MC, Li YN. Multilabel deep visual-semantic embedding. IEEE Trans Pattern Anal Mach Intell. 2019. 42: 1530–1536.
- 14Huang KH, Lin HT. Cost-sensitive label embedding for multi-label classification. Mach Learn. 2017; 106(9-10): 1725-1746.
- 15Duda RO, Hart PE, Stork DG. Pattern Classification. New York, NY: Wiley; 2001.
- 16Li J, Cheng K, Wang S, et al. Feature selection: a data perspective. ACM Comput Surv (CSUR). 2018; 50(6): 94.
- 17Doquire G, Verleysen M. Feature selection for multi-label classification problems. Paper presented at: Proceedings of the International Work-Conference on Artificial Neural Networks; 2011:9-16.
- 18Cai Z, Zhu W. Multi-label feature selection via feature manifold learning and sparsity regularization. Int J Mach Learn Cybern. 2018; 9(8): 1321-1334.
- 19Du G, Ma L, Hu JS, et al. Prediction of 30-Day readmission: an improved gradient boosting decision tree approach. J Med Imag Health Inform. 2019; 9(3): 620-627.
- 20Du G, Zhang J, Luo Z, Ma F, Ma L, Li S. Joint imbalanced classification and feature selection for hospital readmissions. Knowledge-Based Systems. 2020; 200(): j.knosys.2020.106020-.https://doi.org/10.1016/j.knosys.2020.106020.
- 21Lin Y, Li J, Tan A, Zhang J. Granular matrix-based knowledge reductions of formal fuzzy contexts. Int J Mach Learn Cybern. 2020; 11(3): 643-656.
- 22Zhu P, Xu Q, Hu Q, Zhang C, Zhao H. Multi-label feature selection with missing labels. Pattern Recogn. 2018; 74: 488-502.
- 23Liu J, Lin Y, Lin M, Wu S, Zhang J. Feature selection based on quality of information. Neurocomputing. 2017; 225: 11-22.
- 24Lin Y, Hu Q, Zhang J, Wu X. Multi-label feature selection with streaming labels. Inf Sci. 2016; 372: 256-275.
- 25Lin Y, Hu Q, Liu J, Duan J. Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing. 2015; 168: 92-103.
- 26Zhang ML, Peña JM, Robles V. Feature selection for multi-label naive Bayes classification. Inf Sci. 2009; 179(19): 3218-3229.
- 27Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst. 2012; 23(11): 1738-1754.
- 28Boutell MR, Luo J, Shen X, Brown CM. Learning multi-label scene classification. Pattern Recogn. 2004; 37(9): 1757-1771.
- 29Zhang ML, Zhou ZH. ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 2007; 40(7): 2038-2048.
- 30Read J, Pfahringer B, Holmes G, Frank E. Classifier chains for multi-label classification. Mach Learn. 2011; 85(3): 333.
- 31Huang SJ, Zhou ZH. Multi-label learning by exploiting label correlations locally. Paper presented at: Proceedings of the 26th AAAI Conference on Artificial Intelligence; 2012.
- 32Yu Y, Pedrycz W, Miao D. Multi-label classification by exploiting label correlations. Exp Syst Appl. 2014; 41(6): 2989-3004.
- 33Huang J, Li G, Wang S, Xue Z, Huang Q. Multi-label classification by exploiting local positive and negative pairwise label correlation. Neurocomputing. 2017; 257: 164-174.
- 34Zhu Y, Kwok JT, Zhou ZH. Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng. 2017; 30(6): 1081-1094.
- 35Zhang J, Luo Z, Li C, Zhou C, Li S. Manifold regularized discriminative feature selection for multi-label learning. Pattern Recogn. 2019. 95: 136–150.
- 36Huang R, Jiang W, Sun G. Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recog Lett. 2018; 112: 346-352.
- 37Wang J, Yang Y, Mao J, Huang Z, Huang C, Xu W. Cnn-rnn: a unified framework for multi-label image classification. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016:2285-2294.
- 38Cui Z, Xue F, Cai X, Cao Y, Wang G, Chen J. Detection of malicious code variants based on deep learning. IEEE Trans Ind Inform. 2018; 14(7): 3187-3196.
- 39Akata Z, Perronnin F, Harchaoui Z, Schmid C. Label-embedding for attribute-based classification. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2013:819-826.
- 40Jing XY, Wu F, Li Z, Hu R, Zhang D. Multi-label dictionary learning for image annotation. IEEE Trans Image Process. 2016; 25(6): 2712-2725.
- 41Hardoon DR, Szedmak S, Shawe-Taylor J. Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 2004; 16(12): 2639-2664.
- 42Harold H. Relations between two sets of variates. Biometrika. 1936; 28(3/4): 321-377.
10.2307/2333955 Google Scholar
- 43Zhang Y, Schneider J. Multi-label output codes using canonical correlation analysis. Paper presented at: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics; 2011:873-882.
- 44Ranjan V, Rasiwasia N, Jawahar C. Multi-label cross-modal retrieval. Paper presented at: Proceedings of the IEEE International Conference on Computer Vision; 2015:4094-4102.
- 45Andrew G, Arora R, Bilmes J, Livescu K. Deep canonical correlation analysis. Paper presented at: Proceedings of the International Conference on Machine Learning; 2013; 1247–1255.
- 46Yeh C, Wu W, Ko W, Wang YF. Learning deep latent space for multi-label classification. Paper presented at: Proceedings of the 31st AAAI Conference on Artificial Intelligence; February 4-9, 2017:2838-2844; San Francisco, CA.
- 47Belkin M, Niyogi P, Sindhwani V. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res. 2006; 7(Nov): 2399-2434.
- 48Hou P, Geng X, Zhang ML. Multi-label manifold learning. Paper presented at: Proceedings of the 13th AAAI Conference on Artificial Intelligence; 2016.
- 49Hassan MU, Rehmani MH, Chen J. Differential privacy techniques for cyber physical systems: a survey. IEEE Commun Surv Tutor. 2019; 22(1): 746–789. https://doi.org/10.1109/COMST.2019.2944748.
- 50Cai X, Niu Y, Geng S, et al. An under-sampled software defect prediction method based on hybrid multi-objective cuckoo search. Concurr Comput Pract Exp. 2019; 32(5):e5478. https://doi.org/10.1002/cpe.5478.
- 51Cui Z, Cao Y, Cai X, Cai J, Chen J. Optimal LEACH protocol with modified bat algorithm for big data sensing systems in Internet of Things. J Parallel Distrib Comput. 2019; 132: 217-229.
- 52Cui Z, Sun B, Wang G, Xue Y, Chen J. A novel oriented cuckoo search algorithm to improve DV-Hop performance for cyber-physical systems. J Parallel Distrib Comput. 2017; 103: 42-52.
- 53Cui Z, Zhang J, Wang Y, et al. A pigeon-inspired optimization algorithm for many-objective optimization problems. Sci Chin Inf Sci. 2019; 62(7): 70212:1-70212:3.
- 54Wang P, Xue F, Li H, Cui Z, Xie L, Chen J. A multi-objective DV-Hop localization algorithm based on NSGA-II in Internet of Things. Mathematics. 2019; 7(2): 184.
- 55Zhang M, Wang H, Cui Z, Chen J. Hybrid multi-objective cuckoo search with dynamical local search. Memetic Comput. 2018; 10(2): 199-208.
- 56Wang GG, Cai X, Cui Z, Min G, Chen J. High performance computing for cyber physical social systems by using evolutionary multi-objective optimization algorithm. IEEE Trans Emerg Topics Comput. 2017; 8(1): 20–30 . https://doi.org/10.1109/TETC.2017.2703784.
- 57Wang P, Huang J, Cui Z, Xie L, Chen J. A Gaussian error correction multi-objective positioning model with NSGA-II. Concurr Comput Pract Exp. 2019; 32(5): e5464. https://doi.org/10.1002/cpe.5464.
- 58Wang Y, Wang P, Zhang J, et al. A novel bat algorithm with multiple strategies coupling for numerical optimization. Mathematics. 2019; 7(2): 135.
- 59Bartels RH, Stewart GW. Solution of the matrix equation AX+ XB= C [F4]. Commun ACM. 1972; 15(9): 820-826.
- 60Tsoumakas G, Katakis I, Vlahavas I. Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng. 2010; 23(7): 1079-1089.
- 61Liu J, Li Y, Weng W, Zhang J, Chen B, Wu S. Feature selection for multi-label learning with streaming label. Neurocomputing. 2020. 387: 268–278.
- 62Sun Z, Zhang J, Dai L, et al. Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing. 2019; 329: 447-456.
- 63Zhang J, Li C, Cao D, et al. Multi-label learning with label-specific features by resolving label correlations. Knowledge-Based Systems. 2018; 159: 148-157.
- 64Zhang J, Li C, Sun Z, Luo Z, Zhou C, Li S. Towards a unified multi-source-based optimization framework for multi-label learning. Appl Soft Comput. 2019; 76: 425-435.
- 65Huang J, Li G, Huang Q, Wu X. Learning label specific features for multi-label classification. Paper presented at: Proceedings of the 2015 IEEE International Conference on Data Mining; 2015:181-190; IEEE.
- 66Jian L, Li J, Shu K, Liu H. Multi-label informed feature selection. Paper presented at: Proceedings of the IJCAI; 2016:1627-1633.