M2CE: Multi-convolutional neural network ensemble approach for improved multiclass classification of skin lesion
Corresponding Author
Himanshu K. Gajera
Department of Computer Science and Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India
Correspondence
Himanshu K. Gajera, Department of Computer Science and Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India.
Email: [email protected]
Search for more papers by this authorDeepak Ranjan Nayak
Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, Rajasthan, India
Search for more papers by this authorMukesh A. Zaveri
Department of Computer Science and Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India
Search for more papers by this authorCorresponding Author
Himanshu K. Gajera
Department of Computer Science and Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India
Correspondence
Himanshu K. Gajera, Department of Computer Science and Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India.
Email: [email protected]
Search for more papers by this authorDeepak Ranjan Nayak
Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, Rajasthan, India
Search for more papers by this authorMukesh A. Zaveri
Department of Computer Science and Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India
Search for more papers by this authorAbstract
Due to inter-class homogeneity and intra-class variability, the classification of skin lesions in dermoscopy images has remained difficult. Although deep convolutional neural networks (DCNNs) have achieved satisfactory performance for binary skin cancer classification, multiclass skin lesion classification is still an open problem due to the limited training samples and class imbalance issues. To tackle these issues, in this article, we propose a multi-CNN ensemble approach dubbed for multiclass skin lesion classification. The includes three individual CNN models, each helping in extracting different high-level features from skin images and thereby yielding different prediction results. First, we design a lightweight CNN model to extract prominent features and train it from scratch, which primarily aims at avoiding the data scarcity problem. Then, we ensemble two different pre-trained CNN models with the lightweight model to improve the performance and generalization capability. The proposed ensemble approach can effectively fuse the predictions of each individual CNN model using the averaging method. The approach is validated using a benchmark data set, HAM10000, which contains skin lesion images of seven different classes. The results demonstrate that the outperforms base CNN models and state-of-the-art approaches without using any external data.
Open Research
DATA AVAILABILITY STATEMENT
The dataset is publicly available.
REFERENCES
- Abayomi-Alli, O. O., Damasevicius, R., Misra, S., Maskeliunas, R., & Abayomi-Alli, A. (2021). Malignant skin melanoma detection using image augmentation by oversampling in nonlinear lower-dimensional embedding manifold. Turkish Journal of Electrical Engineering and Computer Sciences, 29(8), 2600–2614.
10.3906/elk-2101-133 Google Scholar
- Abdar, M., Samami, M., Mahmoodabad, S. D., Doan, T., Mazoure, B., Hashemifesharaki, R., Liu, L., Khosravi, A., Acharya, U. R., Makarenkov, V., & Nahavandi, S. (2021). Uncertainty quantification in skin cancer classification using three-way decision-based bayesian deep learning. Computers in Biology and Medicine, 135, 104418.
- Adegun, A., & Viriri, S. (2021). Deep learning techniques for skin lesion analysis and melanoma cancer detection: A survey of state-of-the-art. Artificial Intelligence Review, 54(2), 811–841.
- Al-Masni, M. A., Kim, D.-H., & Kim, T.-S. (2020). Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Computer Methods and Programs in Biomedicine, 190, 105351.
- Bagchi, S., Banerjee, A., & Bathula, D. R. (2020). Learning a meta-ensemble technique for skin lesion classification and novel class detection. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp. 746–747). IEEE.
- Barata, C., Celebi, M. E., & Marques, J. S. (2018). A survey of feature extraction in dermoscopy image analysis of skin cancer. IEEE Journal of Biomedical and Health Informatics, 23(3), 1096–1109.
- Celebi, M. E., Codella, N., & Halpern, A. (2019). Dermoscopy image analysis: Overview and future directions. IEEE Journal of Biomedical and Health Informatics, 23(2), 474–478.
- Chaturvedi, S. S., Gupta, K., & Prasad, P. S. (2021). Skin lesion analyser: An efficient seven-way multi-class skin cancer classification using mobilenet. Advanced machine learning technologies and applications: Proceedings of AMLTA 2020 (pp. 165–176). Springer.
- Chen, Y., Li, D., Zhang, X., Jin, J., & Shen, Y. (2021). Computer aided diagnosis of thyroid nodules based on the devised small-datasets multi-view ensemble learning. Medical Image Analysis, 67, 101819.
- Codella, N., Cai, J., Abedini, M., Garnavi, R., Halpern, A., & Smith, J. R. (2015). Deep learning, sparse coding, and SVM for melanoma recognition in dermoscopy images. International workshop on machine learning in medical imaging (pp. 118–126). Springer.
- Demyanov, S., Chakravorty, R., Abedini, M., Halpern, A., & Garnavi, R. (2016). Classification of dermoscopy patterns using deep convolutional neural networks. 13th International symposium on biomedical imaging (ISBI) (pp. 364–368). IEEE.
- Dong, X., Yu, Z., Cao, W., Shi, Y., & Ma, Q. (2020). A survey on ensemble learning. Frontiers of Computer Science, 14(2), 241–258.
- Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118.
- Fraiwan, M., & Faouri, E. (2022). On the automatic detection and classification of skin cancer using deep transfer learning. Sensors, 22(13), 4963.
- Gajera, H. K., Nayak, D. R., & Zaveri, M. A. (2022a). A comprehensive analysis of dermoscopy images for melanoma detection via deep CNN features. Biomedical Signal Processing and Control, 79, 104186.
- Gajera, H. K., Nayak, D. R., & Zaveri, M. A. (2022b). Fusion of local and global feature representation with sparse autoencoder for improved melanoma classification. 44th Annual international conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, 5051–5054.
- Gajera, H. K., Zaveri, M. A., & Nayak, D. R. (2021). Improving the performance of melanoma detection in dermoscopy images using deep CNN features. International conference on artificial intelligence in medicine (pp. 349–354). Springer.
- Gajera, H. K., Zaveri, M. A., & Nayak, D. R. (2022). Patch-based local deep feature extraction for automated skin cancer classification. International Journal of Imaging Systems and Technology, 32(5), 1774–1788.
- Ganster, H., Pinz, P., Rohrer, R., Wildling, E., Binder, M., & Kittler, H. (2001). Automated melanoma recognition. IEEE Transactions on Medical Imaging, 20(3), 233–239.
- Gessert, N., Nielsen, M., Shaikh, M., Werner, R., & Schlaefer, A. (2020). Skin lesion classification using ensembles of multi-resolution efficientNets with meta data. MethodsX, 7, 100864.
- Gessert, N., Sentker, T., Madesta, F., Schmitz, R., Kniep, H., Baltruschat, I., Werner, R., & Schlaefer, A. (2019). Skin lesion classification using CNNs with patch-based attention and diagnosis-guided loss weighting. IEEE Transactions on Biomedical Engineering, 67(2), 495–503.
- Gong, Z., Zhong, P., Yu, Y., Hu, W., & Li, S. (2019). A cnn with multiscale convolution and diversified metric for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(6), 3599–3618.
- He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE international conference on computer vision (CVPR) (pp. 1026–1034). IEEE.
- He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778). IEEE.
- Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv [Preprint] arXiv:1704.04861.
- Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4700–4708). IEEE.
- Khan, M. A., Akram, T., Sharif, M., Shahzad, A., Aurangzeb, K., Alhussein, M., Haider, S. I., & Altamrah, A. (2018). An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification. BMC Cancer, 18(1), 1–20.
- Khan, M. A., Zhang, Y.-D., Sharif, M., & Akram, T. (2021). Pixels to classes: Intelligent learning framework for multiclass skin lesion localization and classification. Computers & Electrical Engineering, 90, 106956.
- Liu, Y.-P., Wang, Z., Li, Z., Li, J., Li, T., Chen, P., & Liang, R. (2021). Multiscale ensemble of convolutional neural networks for skin lesion classification. IET Image Processing, 15(10), 2309–2318.
- Luz, D. S., Lima, T. J., Silva, R. R., Magalhães, D. M., & Araujo, F. H. (2022). Automatic detection metastasis in breast histopathological images based on ensemble learning and color adjustment. Biomedical Signal Processing and Control, 75, 103564.
- Maqsood, S., & Damasevicius, R. (2023). Multiclass skin lesion localization and classification using deep learning based features fusion and selection framework for smart healthcare. Neural Networks, 160, 238–258.
- Melbin, K., & Raj, Y. J. V. (2021). Automated detection and classification of skin diseases using diverse features and improved gray wolf-based multiple-layer perceptron neural network. International Journal of Imaging Systems and Technology, 31(3), 1317–1333.
- Müller, D., Soto-Rey, I., & Kramer, F. (2022). An analysis on ensemble learning optimized medical image classification with deep convolutional neural networks. arXiv [Preprint] arXiv:2201.11440.
- Nachbar, F., Stolz, W., Merkle, T., Cognetta, A. B., Vogt, T., Landthaler, M., Bilek, P., Braun-Falco, O., & Plewig, G. (1994). The ABCD rule of dermatoscopy: High prospective value in the diagnosis of doubtful melanocytic skin lesions. Journal of the American Academy of Dermatology, 30(4), 551–559.
- Nayak, D. R., Dash, R., & Majhi, B. (2020). Automated diagnosis of multi-class brain abnormalities using MRI images: A deep convolutional neural network based method. Pattern Recognition Letters, 138, 385–391.
- Osowski, S., & Les, T. (2020). Deep learning ensemble for melanoma recognition. International joint conference on neural networks (IJCNN) (pp. 1–7). IEEE.
- Ozkan, I. A., & Koklu, M. (2017). Skin lesion classification using machine learning algorithms. International Journal of Intelligent Systems and Applications in Engineering, 5(4), 285–289.
10.18201/ijisae.2017534420 Google Scholar
- Pacheco, A. G., Sastry, C. S., Trappenberg, T., Oore, S., & Krohling, R. A. (2020). On out-of-distribution detection algorithms with deep neural skin cancer classifiers. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp. 732–733). IEEE.
- Pacheco, A. G. C., & Krohling, R. (2021). An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification. Journal of Biomedical and Health Informatics, 25, 3554–3563.
- Paul, A., Tang, Y.-X., Shen, T. C., & Summers, R. M. (2021). Discriminative ensemble learning for few-shot chest x-ray diagnosis. Medical Image Analysis, 68, 101911.
- Polap, D., Winnicka, A., Serwata, K., Kesik, K., & Wozniak, M. (2018). An intelligent system for monitoring skin diseases. Sensors, 18(8), 2552.
- Rahman, Z., & Ami, A. M. (2020). A transfer learning based approach for skin lesion classification from imbalanced data. 11th International conference on electrical and computer engineering (ICECE) (pp. 65–68). IEEE.
- Rahman, Z., Hossain, M. S., Islam, M. R., Hasan, M. M., & Hridhee, R. A. (2021). An approach for multiclass skin lesion classification based on ensemble learning. Informatics in Medicine Unlocked, 25, 100659.
10.1016/j.imu.2021.100659 Google Scholar
- Reisinho, J., Coimbra, M., & Renna, F. (2020). Deep convolutional neural network ensembles for multi-classification of skin lesions from dermoscopic and clinical images. 42nd Annual international conference of the engineering in medicine & biology society (EMBC) (pp. 1940–1943). IEEE.
- Ruela, M., Barata, C., Marques, J. S., & Rozeira, J. (2017). A system for the detection of melanomas in dermoscopy images using shape and symmetry features. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 5(2), 127–137.
- Sabri, M. A., Filali, Y., El Khoukhi, H., & Aarab, A. (2020). Skin cancer diagnosis using an improved ensemble machine learning model. International conference on intelligent systems and computer vision (ISCV) (pp. 1–5). IEEE.
- Sae-Lim, W., Wettayaprasit, W., & Aiyarak, P. (2019). Convolutional neural networks using mobilenet for skin lesion classification. 16th International joint conference on computer science and software engineering (JCSSE) (pp. 242–247). IEEE.
- Sagi, O., & Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1249.
- Shahin, A. H., Kamal, A., & Elattar, M. A. (2018). Deep ensemble learning for skin lesion classification from dermoscopic images. 9th Cairo international biomedical engineering conference (CIBEC) (pp. 150–153). IEEE.
- Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv [Preprint] arXiv:1409.1556.
- Sinz, C., Tschandl, P., Rosendahl, C., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J., Lallas, A., Lapins, J., Marghoob, A. A., Menzies, S. W., Paoli, J., Rabinovitz, H. S., Rinner, C., Scope, A., Soyer, H. P., … Kittler, H. (2017). Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin. Journal of the American Academy of Dermatology, 77(6), 1100–1109.
- Su, L., Liu, Y., Wang, M., & Li, A. (2021). Semi-hic: A novel semi-supervised deep learning method for histopathological image classification. Computers in Biology and Medicine, 137, 104788.
- Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9). IEEE.
- Tang, P., Liang, Q., Yan, X., Xiang, S., & Zhang, D. (2020). GP-CNN-DTEL: Global-part CNN model with data-transformed ensemble learning for skin lesion classification. IEEE Journal of Biomedical and Health Informatics, 24(10), 2870–2882.
- Thiyaneswaran, B., Anguraj, K., Kumarganesh, S., & Thangaraj, K. (2021). Early detection of melanoma images using gray level co-occurrence matrix features and machine learning techniques for effective clinical diagnosis. International Journal of Imaging Systems and Technology, 31(2), 682–694.
- Tschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R. P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., Lallas, A., Lapins, J., Longo, C., Malvehy, J., Marchetti, M. A., Marghoob, A., Menzies, S., Oakley, A., Paoli, J., … Kittler, H. (2019). Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: An open, web-based, international, diagnostic study. The Lancet Oncology, 20(7), 938–947.
- Tschandl, P., Rosendahl, C., & Kittler, H. (2018). The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1), 1–9.
- Vestergaard, M., Macaskill, P., Holt, P., & Menzies, S. (2008). Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: A meta-analysis of studies performed in a clinical setting. British Journal of Dermatology, 159(3), 669–676.
- Waweru, A. K., Ahmed, K., Miao, Y., & Kawan, P. (2020). Deep learning in skin lesion analysis towards cancer detection. 24th International conference information visualisation (IV) (pp. 740–745). IEEE.
- WHO. (2022). Radiation: Ultraviolet (UV) radiation and skin cancer. https://www.who.int/news-room/questions-and-answers/item/radiation-ultraviolet-(uv)-radiation-and-skin-cancer
- Wu, J., Hu, W., Wen, Y., Tu, W., & Liu, X. (2020). Skin lesion classification using densely connected convolutional networks with attention residual learning. Sensors, 20(24), 7080.
- Xie, F., Fan, H., Li, Y., Jiang, Z., Meng, R., & Bovik, A. (2016). Melanoma classification on dermoscopy images using a neural network ensemble model. IEEE Transactions on Medical Imaging, 36(3), 849–858.
- Yu, L., Chen, H., Dou, Q., Qin, J., & Heng, P.-A. (2016). Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Transactions on Medical Imaging, 36(4), 994–1004.
- Yu, Z., Jiang, X., Zhou, F., Qin, J., Ni, D., Chen, S., Lei, B., & Wang, T. (2018). Melanoma recognition in dermoscopy images via aggregated deep convolutional features. IEEE Transactions on Biomedical Engineering, 66(4), 1006–1016.
- Zhang, J., Xie, Y., Xia, Y., & Shen, C. (2019). Attention residual learning for skin lesion classification. IEEE Transactions on Medical Imaging, 38(9), 2092–2103.