Patch-based local deep feature extraction for automated skin cancer classification
Corresponding Author
Himanshu K. Gajera
Department of Computer Science and Engineering, S. V. National Institute of Technology, Surat, India
Correspondence
Himanshu K. Gajera, Department of Computer Science and Engineering, S. V. National Institute of Technology, Surat, India.
Email: [email protected]
Search for more papers by this authorMukesh A. Zaveri
Department of Computer Science and Engineering, S. V. National Institute of Technology, Surat, India
Search for more papers by this authorDeepak Ranjan Nayak
Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, India
Search for more papers by this authorCorresponding Author
Himanshu K. Gajera
Department of Computer Science and Engineering, S. V. National Institute of Technology, Surat, India
Correspondence
Himanshu K. Gajera, Department of Computer Science and Engineering, S. V. National Institute of Technology, Surat, India.
Email: [email protected]
Search for more papers by this authorMukesh A. Zaveri
Department of Computer Science and Engineering, S. V. National Institute of Technology, Surat, India
Search for more papers by this authorDeepak Ranjan Nayak
Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, India
Search for more papers by this authorAbstract
Skin cancer detection through dermoscopy images has remained challenging due to higher inter-class uniformity and intra-class diversity. Deep convolutional neural networks (CNNs) have recently obtained remarkable attention for automated skin cancer classification; however, most of these methods extract features from the global image of high resolution. One of the major drawbacks of these methods is the downscaling of input images to a low resolution, which leads to information loss. Moreover, the lack of a vast number of dermoscopy images is a concern. To overcome these issues and improve the classification accuracy, we propose a novel method using patch-based local deep feature extraction. In the proposed method, the features are extracted from different patches of a dermoscopy image using a pre-trained CNN model and are fused to preserve fine details. The kernel principal component analysis is then employed to select the prominent features, and a feed-forward neural network is finally used to detect skin cancer. The approach is validated using two benchmark datasets: ISIC 2016 and ISIC 2017. The experimental results show that the suggested method achieves promising results in comparison with state-of-the-art approaches. Moreover, a comparative analysis is made among contemporary pre-trained CNN models.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are openly available in ISIC Challenge at https://challenge.isic-archive.com/, reference number.53, 54
REFERENCES
- 1Vestergaard M, Macaskill P, Holt P, Menzies S. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical setting. Br J Dermatol. 2008; 159(3): 669-676.
- 2Balch CM, Gershenwald JE, Soong S-J, et al. Final version of 2009 AJCC melanoma staging and classification. J Clin Oncol. 2009; 27(36): 6199-6206.
- 3Ruela M, Barata C, Marques JS, Rozeira J. A system for the detection of melanomas in dermoscopy images using shape and symmetry features. Comput Methods Biomech Biomed Eng Imaging Vis. 2017; 5(2): 127-137.
- 4Khan MA, Akram T, Sharif M, et al. An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification. BMC Cancer. 2018; 18(1): 1-20.
- 5Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017; 42: 60-88.
- 6Nayak DR, Dash R, Majhi B. Automated diagnosis of multi-class brain abnormalities using MRI images: a deep convolutional neural network based method. Pattern Recognit Lett. 2020; 138: 385-391.
- 7Barata C, Celebi ME, Marques JS. A survey of feature extraction in dermoscopy image analysis of skin cancer. IEEE J Biomed Health Inform. 2018; 23(3): 1096-1109.
- 8Gupta A, Thakur S, Rana A. Study of melanoma detection and classification techniques. 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO). IEEE; 2020: 1345-1350.
10.1109/ICRITO48877.2020.9197820 Google Scholar
- 9Yu L, Chen H, Dou Q, Qin J, Heng P-A. Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging. 2016; 36(4): 994-1004.
- 10Nachbar F, Stolz W, Merkle T, et al. The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions. J Am Acad Dermatol. 1994; 30(4): 551-559.
- 11Ganster H, Pinz P, Rohrer R, Wildling E, Binder M, Kittler H. Automated melanoma recognition. IEEE Trans Med Imaging. 2001; 20(3): 233-239.
- 12Xie F, Fan H, Li Y, Jiang Z, Meng R, Bovik A. Melanoma classification on dermoscopy images using a neural network ensemble model. IEEE Trans Med Imaging. 2016; 36(3): 849-858.
- 13Ozkan IA, Koklu M. Skin lesion classification using machine learning algorithms. Int J Intell Syst Appl Eng. 2017; 5(4): 285-289.
10.18201/ijisae.2017534420 Google Scholar
- 14Thiyaneswaran B, Anguraj K, Kumarganesh S, Thangaraj K. Early detection of melanoma images using gray level co-occurrence matrix features and machine learning techniques for effective clinical diagnosis. Int J Imaging Syst Technol. 2021; 31(2): 682-694.
- 15Melbin K, Jacob Vetha Raj Y. Automated detection and classification of skin diseases using diverse features and improved gray wolf-based multiple-layer perceptron neural network. Int J Imaging Syst Technol. 2021; 31(3): 1317-1333.
- 16Codella N, Cai J, Abedini M, Garnavi R, Halpern A, Smith JR. Deep learning, sparse coding, and SVM for melanoma recognition in dermoscopy images. International Workshop on Machine Learning in Medical Imaging. Springer; 2015: 118-126.
10.1007/978-3-319-24888-2_15 Google Scholar
- 17Demyanov S, Chakravorty R, Abedini M, Halpern A, Garnavi R. Classification of dermoscopy patterns using deep convolutional neural networks. 13th International Symposium on Biomedical Imaging (ISBI). IEEE; 2016: 364-368.
10.1109/ISBI.2016.7493284 Google Scholar
- 18Majtner T, Yildirim-Yayilgan S, Hardeberg JY. Combining deep learning and hand-crafted features for skin lesion classification. Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE; 2016: 1-6.
10.1109/IPTA.2016.7821017 Google Scholar
- 19Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017; 542(7639): 115-118.
- 20Yu Z, Jiang X, Zhou F, et al. Melanoma recognition in dermoscopy images via aggregated deep convolutional features. IEEE Trans Biomed Eng. 2018; 66(4): 1006-1016.
- 21Hameed N, Shabut AM, Hossain MA. Multi-class skin diseases classification using deep convolutional neural network and support vector machine. 2018 12th International Conference on Software, Knowledge, Information Management & Applications (SKIMA). IEEE; 2018: 1-7.
10.1109/SKIMA.2018.8631525 Google Scholar
- 22Gessert N, Sentker T, Madesta F, et al. Skin lesion classification using CNNs with patch-based attention and diagnosis-guided loss weighting. IEEE Trans Biomed Eng. 2019; 67(2): 495-503.
- 23Jayapriya K, Jacob IJ. Hybrid fully convolutional networks-based skin lesion segmentation and melanoma detection using deep feature. Int J Imaging Syst Technol. 2020; 30(2): 348-357.
- 24Wu J, Hu W, Wen Y, Tu W, Liu X. Skin lesion classification using densely connected convolutional networks with attention residual learning. Sensors. 2020; 20(24): 7080.
- 25Zghal NS, Kallel IK. An effective approach for the diagnosis of melanoma using the sparse auto-encoder for features detection and the SVM for classification. 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP). IEEE; 2020: 1-6.
10.1109/ATSIP49331.2020.9231611 Google Scholar
- 26Zhang J, Xie Y, Xia Y, Shen C. Attention residual learning for skin lesion classification. IEEE Trans Med Imaging. 2019; 38(9): 2092-2103.
- 27Tang P, Liang Q, Yan X, Xiang S, Zhang D. GP-CNN-DTEL: global-part CNN model with data-transformed ensemble learning for skin lesion classification. IEEE J Biomed Health Inform. 2020; 24(10): 2870-2882.
- 28Chaturvedi SS, Tembhurne JV, Diwan T. A multi-class skin cancer classification using deep convolutional neural networks. Multimed Tools Appl. 2020; 79(39): 28477-28498.
- 29Rajput G, Agrawal S, Raut G, Vishvakarma SK. An accurate and noninvasive skin cancer screening based on imaging technique. Int J Imaging Syst Technol. 2021; 32: 354-368.
- 30Pacheco AGC, Krohling R. An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification. J Biomed Health Inform. 2021; 25: 3554-3563.
- 31Abdar M, Samami M, Mahmoodabad SD, et al. Uncertainty quantification in skin cancer classification using three-way decision-based bayesian deep learning. Computers in Biology and Medicine. 2021; 135: 104418.
- 32Haggenmüller S, Maron RC, Hekler A, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer. 2021; 156: 202-216.
- 33Gajera HK, Zaveri MA, Nayak DR. Improving the performance of melanoma detection in dermoscopy images using deep CNN features. International Conference on Artificial Intelligence in Medicine. Springer; 2021: 349-354.
10.1007/978-3-030-77211-6_39 Google Scholar
- 34Sharma M, Bhave A. Lesion classification using convolutional neural network. Soft Computing and Signal Processing. Springer; 2019: 357-365.
10.1007/978-981-13-3393-4_37 Google Scholar
- 35Alizadeh SM, Mahloojifar A. Automatic skin cancer detection in dermoscopy images by combining convolutional neural networks and texture features. Int J Imaging Syst Technol. 2021; 31(2): 695-707.
- 36Adegun A, Viriri S. Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif Intell Rev. 2021; 54(2): 811-841.
- 37Menegola A, Fornaciali M, Pires R, Bittencourt FV, Avila S, Valle E. Knowledge transfer for melanoma screening with deep learning. 14th International Symposium on Biomedical Imaging (ISBI). IEEE; 2017: 297-300.
10.1109/ISBI.2017.7950523 Google Scholar
- 38Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ArXiv Preprint ArXiv:1409.1556, 2014.
- 39Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2015: 1-9.
10.1109/CVPR.2015.7298594 Google Scholar
- 40He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2016: 770-778.
10.1109/CVPR.2016.90 Google Scholar
- 41Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2017: 4700-4708.
10.1109/CVPR.2017.243 Google Scholar
- 42Howard AG, Zhu M, Chen M, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. ArXiv Preprint ArXiv:1704.04861, 2017.
- 43Tan M, Le Q. Efficientnet: rethinking model scaling for convolutional neural networks. International Conference on Machine Learning. PMLR; 2019: 6105-6114.
- 44Abdar M, Fahami MA, Chakrabarti S, et al. BARF: a new direct and cross-based binary residual feature fusion with uncertainty-aware module for medical image classification. Inform Sci. 2021; 577: 353-378.
- 45Zhang X, Wang J, Wang T, Jiang R. Hierarchical feature fusion with mixed convolution attention for single image dehazing. IEEE Trans Circuits and Syst Video Technol. 2021; 32: 510-522.
- 46Liang X, Li N, Zhang Z, Xiong J, Zhou S, Xie Y. Incorporating the hybrid deformable model for improving the performance of abdominal CT segmentation via multi-scale feature fusion network. Med Image Anal. 2021; 73:102156.
- 47Chen Y, Tu Z, Kang D, et al. Joint hand-object 3d reconstruction from a single image with cross-branch feature fusion. IEEE Trans Image Process. 2021; 30: 4008-4021.
- 48Abdar M, Salari S, Qahremani S, et al. Uncertaintyfusenet: robust uncertainty-aware hierarchical feature fusion with ensemble Monte Carlo dropout for COVID-19 detection. ArXiv Preprint arXiv:2105.08590, 2021.
- 49Schölkopf B, Smola A, Müller K-R. Kernel principal component analysis. International Conference on Artificial Neural Networks. Springer; 1997: 583-588.
10.1007/BFb0020217 Google Scholar
- 50Schölkopf B. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 1998; 10(5): 1299-1319.
- 51Nayak DR, Dash R, Majhi B. Least squares SVM approach for abnormal brain detection in MRI using multiresolution analysis. 2015 International Conference on Computing, Communication and Security (ICCCS). IEEE; 2015: 1-6.
10.1109/CCCS.2015.7374140 Google Scholar
- 52Chen T, Guestrin C. Xgboost: a scalable tree boosting system. 22nd SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016: 785-794.
10.1145/2939672.2939785 Google Scholar
- 53Gutman D, Codella NC, Celebi E, et al. Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC),” ArXiv Preprint ArXiv:1605.01397, 2016.
- 54Berseth M. ISIC 2017 - skin lesion analysis towards melanoma detection. 2017.
- 55Matsunaga K, Hamada A, Minagawa A, Koga H. Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. ArXiv Preprint ArXiv:1703.03108, 2017.
- 56Díaz IG. Incorporating the knowledge of dermatologists to convolutional neural networks for the diagnosis of skin lesions. arXiv preprint arXiv:1703.01976, 2017.
- 57Menegola A, Tavares J, Fornaciali M, Li LT, Avila S, Valle E RECOD titans at ISIC challenge 2017. ArXiv Preprint ArXiv:1703.04819, 2017.
- 58Bi L, Kim J, Ahn E, Feng D. Automatic skin lesion analysis using large-scale dermoscopy images and deep residual networks. ArXiv Preprint ArXiv:1703.04197, 2017.
- 59DeVries T, Ramachandram D. Skin lesion classification using deep multi-scale convolutional neural networks. ArXiv Preprint ArXiv:1703.01402, 2017.
- 60Senousy Z, Abdelsamea M, Gaber MM, et al. MCUa: multi-level context and uncertainty aware dynamic deep ensemble for breast cancer histology image classification. IEEE Trans Biomed Eng. 2021; 69: 818-829.
- 61Liu JZ, Lin Z, Padhy S, Tran D, Bedrax-Weiss T, Lakshminarayanan B. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. ArXiv Preprint ArXiv:2006.10108, 2020.
- 62Abdar M, Pourpanah F, Hussain S, et al. A review of uncertainty quantification in deep learning: techniques, applications and challenges. Inf Fusion. 2021; 76: 243-297.