MultiResFF-Net: Multilevel Residual Block-Based Lightweight Feature Fused Network With Attention for Gastrointestinal Disease Diagnosis
Sohaib Asif
Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Center of Intelligent Diagnosis and Therapy (Taizhou) , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Taizhou , Zhejiang, 317502 , China , cas.cn
Wenling Institute of Big Data and Artificial Intelligence in Medicine , Taizhou , Zhejiang, 317502 , China
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Search for more papers by this authorYajun Ying
Department of Pathology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorTingting Qian
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Graduate School , The Second Clinical Medical College of Zhejiang Chinese Medical University , Hangzhou , Zhejiang, 310014 , China , zcmu.edu.cn
Search for more papers by this authorJun Yao
Department of Surgical Oncology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorJinjie Qu
Department of Surgical Oncology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorVicky Yang Wang
Center of Intelligent Diagnosis and Therapy (Taizhou) , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Taizhou , Zhejiang, 317502 , China , cas.cn
Wenling Institute of Big Data and Artificial Intelligence in Medicine , Taizhou , Zhejiang, 317502 , China
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Search for more papers by this authorCorresponding Author
Rongbiao Ying
Department of Surgical Oncology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorCorresponding Author
Dong Xu
Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Center of Intelligent Diagnosis and Therapy (Taizhou) , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Taizhou , Zhejiang, 317502 , China , cas.cn
Wenling Institute of Big Data and Artificial Intelligence in Medicine , Taizhou , Zhejiang, 317502 , China
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Search for more papers by this authorSohaib Asif
Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Center of Intelligent Diagnosis and Therapy (Taizhou) , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Taizhou , Zhejiang, 317502 , China , cas.cn
Wenling Institute of Big Data and Artificial Intelligence in Medicine , Taizhou , Zhejiang, 317502 , China
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Search for more papers by this authorYajun Ying
Department of Pathology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorTingting Qian
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Graduate School , The Second Clinical Medical College of Zhejiang Chinese Medical University , Hangzhou , Zhejiang, 310014 , China , zcmu.edu.cn
Search for more papers by this authorJun Yao
Department of Surgical Oncology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorJinjie Qu
Department of Surgical Oncology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorVicky Yang Wang
Center of Intelligent Diagnosis and Therapy (Taizhou) , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Taizhou , Zhejiang, 317502 , China , cas.cn
Wenling Institute of Big Data and Artificial Intelligence in Medicine , Taizhou , Zhejiang, 317502 , China
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Search for more papers by this authorCorresponding Author
Rongbiao Ying
Department of Surgical Oncology , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Search for more papers by this authorCorresponding Author
Dong Xu
Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence , Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital) , Taizhou , Zhejiang, 317502 , China
Center of Intelligent Diagnosis and Therapy (Taizhou) , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Taizhou , Zhejiang, 317502 , China , cas.cn
Wenling Institute of Big Data and Artificial Intelligence in Medicine , Taizhou , Zhejiang, 317502 , China
Department of Diagnostic Ultrasound Imaging & Interventional Therapy , Zhejiang Cancer Hospital , Hangzhou Institute of Medicine (HIM) , Chinese Academy of Sciences , Hangzhou , Zhejiang, 310022 , China , cas.cn
Search for more papers by this authorAbstract
Accurate detection of gastrointestinal (GI) diseases is crucial due to their high prevalence. Screening is often inefficient with existing methods, and the complexity of medical images challenges single-model approaches. Leveraging diverse model features can improve accuracy and simplify detection. In this study, we introduce a novel deep learning model tailored for the diagnosis of GI diseases through the analysis of endoscopy images. This innovative model, named MultiResFF-Net, employs a multilevel residual block-based feature fusion network. The key strategy involves the integration of features from truncated DenseNet121 and MobileNet architectures. This fusion not only optimizes the model’s diagnostic performance but also strategically minimizes complexity and computational demands, making MultiResFF-Net a valuable tool for efficient and accurate disease diagnosis in GI endoscopy images. A pivotal component enhancing the model’s performance is the introduction of the Modified MultiRes-Block (MMRes-Block) and the Convolutional Block Attention Module (CBAM). The MMRes-Block, a customized residual learning component, optimally handles fused features at the endpoint of both models, fostering richer feature sets without escalating parameters. Simultaneously, the CBAM ensures dynamic recalibration of feature maps, emphasizing relevant channels and spatial locations. This dual incorporation significantly reduces overfitting, augments precision, and refines the feature extraction process. Extensive evaluations on three diverse datasets—endoscopic images, GastroVision data, and histopathological images—demonstrate exceptional accuracy of 99.37%, 97.47%, and 99.80%, respectively. Notably, MultiResFF-Net achieves superior efficiency, requiring only 2.22 MFLOPS and 0.47 million parameters, outperforming state-of-the-art models in both accuracy and cost-effectiveness. These results establish MultiResFF-Net as a robust and practical diagnostic tool for GI disease detection.
Conflicts of Interest
The authors declare no conflicts of interest.
Open Research
Data Availability Statement
The data that support the findings of this study are available from the corresponding authors upon reasonable request.
References
- 1 Khan M. A., Kadry S., Alhaisoni M. et al., Computer-Aided Gastrointestinal Diseases Analysis From Wireless Capsule Endoscopy: A Framework of Best Features Selection, IEEE Access. (2020) 8, 132850–132859, https://doi.org/10.1109/access.2020.3010448.
- 2 Mirza O. M., Alsobhi A., Hasanin T., Ishak M. K., Karim F. K., and Mostafa S. M., Computer Aided Diagnosis for Gastrointestinal Cancer Classification Using Hybrid Rice Optimization with Deep Learning, IEEE Access. (2023) 11, 76321–76329, https://doi.org/10.1109/access.2023.3297441.
- 3 Ao C., Yu L., and Zou Q., Prediction of Bio-Sequence Modifications and the Associations With Diseases, Briefings in Functional Genomics. (2021) 20, 1–18, https://doi.org/10.1093/bfgp/elaa023.
- 4 Lu J., Chen Y., Shi L. et al., Cognition of Abdominal Pain and Abdominal Discomfort in Chinese Patients With Irritable Bowel Syndrome With Diarrhea, BioPsychoSocial Medicine. (2023) 17, no. 1, https://doi.org/10.1186/s13030-023-00286-1.
- 5 Gatzinsky C., Sillén U., Thornberg S., and Sjöström S., Bowel Habits in Healthy Infants and the Prevalence of Functional Constipation, Infant Colic and Infant Dyschezia, Acta Paediatrica. (2023) 112, no. 6, 1341–1350, https://doi.org/10.1111/apa.16736.
- 6 Kim S. E., Kim H. J., Koh M. et al., A Practical Approach for Small Bowel Bleeding, Clinical Endoscopy. (2023) 56, no. 3, 283–289, https://doi.org/10.5946/ce.2022.302.
- 7 Singh J., Dinkar A., Kumar N., and Kumar K., Recurrent Nausea and Vomiting With Weight Loss Associated With Hypothyroidism: Fact or Myth, Endocrine, Metabolic & Immune Disorders-Drug Targets. (2023) 23, no. 6, 867–872, https://doi.org/10.2174/1871530323666221205110210.
- 8 Mohapatra S., Kumar Pati G., Mishra M., and Swarnkar T., Gastrointestinal Abnormality Detection and Classification Using Empirical Wavelet Transform and Deep Convolutional Neural Network From Endoscopic Images, Ain Shams Engineering Journal. (2023) 14, no. 4, https://doi.org/10.1016/j.asej.2022.101942.
- 9 Mori Y., East J. E., Hassan C. et al., Benefits and Challenges in Implementation of Artificial Intelligence in Colonoscopy: World Endoscopy Organization Position Statement, Digestive Endoscopy. (2023) 35, no. 4, 422–429, https://doi.org/10.1111/den.14531.
- 10 Lima A. C. D. M., de Paiva L. F., Bráz G. et al., A Two-Stage Method for Polyp Detection in Colonoscopy Images Based on Saliency Object Extraction and Transformers, IEEE Access. (2023) 11, 76108–76119, https://doi.org/10.1109/access.2023.3297097.
- 11 Wang T., Zhang X., Zhou Y. et al., PCDAL: A Perturbation Consistency-Driven Active Learning Approach for Medical Image Segmentation and Classification, 2023, https://arxiv.org/abs/230616918.
- 12 Xiao Z., Ji D., Li F., Li Z., and Bao Z., Application of Artificial Intelligence in Early Gastric Cancer Diagnosis, Digestion. (2022) 103, no. 1, 69–75, https://doi.org/10.1159/000519601.
- 13 Sharma A., Kumar R., and Garg P., Deep Learning-Based Prediction Model for Diagnosing Gastrointestinal Diseases Using Endoscopy Images, International Journal of Medical Informatics. (2023) 177, https://doi.org/10.1016/j.ijmedinf.2023.105142.
- 14 Song Q., Zheng Y.-J., Sheng W.-G., and Yang J., Tridirectional Transfer Learning for Predicting Gastric Cancer Morbidity, IEEE Transactions on Neural Networks and Learning Systems. (2021) 32, no. 2, 561–574, https://doi.org/10.1109/tnnls.2020.2979486.
- 15 Li K., Chen C., Cao W. et al., DeAF: A Multimodal Deep Learning Framework for Disease Prediction, Computers in Biology and Medicine. (2023) 156, https://doi.org/10.1016/j.compbiomed.2023.106715.
- 16 Pang X., Zhao Z., Wu Y., Chen Y., and Liu J., Computer-Aided Diagnosis System Based on Multi-Scale Feature Fusion for Screening Large-Scale Gastrointestinal Diseases, Journal of Computational Design and Engineering. (2023) 10, no. 1, 368–381, https://doi.org/10.1093/jcde/qwac138.
- 17 Guan X., Lu N., and Zhang J., Computed Tomography-Based Deep Learning Nomogram Can Accurately Predict Lymph Node Metastasis in Gastric Cancer, Digestive Diseases and Sciences. (2023) 68, no. 4, 1473–1481, https://doi.org/10.1007/s10620-022-07640-3.
- 18
Costa C. L.,
Lima D. A.,
Zorzo Barcelos C. A., and
Travençolo B. A., Ensemble Architectures and Efficient Fusion Techniques for Convolutional Neural Networks: An Analysis on Resource Optimization Strategies, Brazilian Conference on Intelligent Systems. (2023) Springer, 107–121.
10.1007/978-3-031-45389-2_8 Google Scholar
- 19 Saldanha O. L., Muti H. S., Grabsch H. I. et al., Direct Prediction of Genetic Aberrations From Pathology Images in Gastric Cancer With Swarm Learning, Gastric Cancer. (2023) 26, no. 2, 264–274, https://doi.org/10.1007/s10120-022-01347-0.
- 20 Kiyokawa H., Abe M., Matsui T. et al., Deep Learning Analysis of Histologic Images From Intestinal Specimen Reveals Adipocyte Shrinkage and Mast Cell Infiltration to Predict Postoperative Crohn Disease, The American Journal of Pathology. (2022) 192, no. 6, 904–916, https://doi.org/10.1016/j.ajpath.2022.03.006.
- 21
Baradaran Rezaei H.,
Amjadian A.,
Sebt M. V.,
Askari R., and
Gharaei A., An Ensemble Method of the Machine Learning to Prognosticate the Gastric Cancer, Annals of Operations Research. (2023) 328, no. 1, 151–192, https://doi.org/10.1007/s10479-022-04964-1.
10.1007/s10479-022-04964-1 Google Scholar
- 22 Su Q., Wang F., Chen D., Chen G., Li C., and Wei L., Deep Convolutional Neural Networks With Ensemble Learning and Transfer Learning for Automated Detection of Gastrointestinal Diseases, Computers in Biology and Medicine. (2022) 150, https://doi.org/10.1016/j.compbiomed.2022.106054.
- 23 Hosain A. S., Islam M., Mehedi M. H. K., Kabir I. E., and Khan Z. T., Gastrointestinal Disorder Detection With a Transformer Based Approach, 2022 IEEE 13th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). (2022) IEEE, 0280–0285.
- 24 Tang S., Yu X., Cheang C. F. et al., Transformer-Based Multi-Task Learning for Classification and Segmentation of Gastrointestinal Tract Endoscopic Images, Computers in Biology and Medicine. (2023) 157, https://doi.org/10.1016/j.compbiomed.2023.106723.
- 25 Wang W., Yang X., and Tang J., Vision Transformer With Hybrid Shifted Windows for Gastrointestinal Endoscopy Image Classification, IEEE Transactions on Circuits and Systems for Video Technology. (2023) 33, no. 9, 4452–4461, https://doi.org/10.1109/tcsvt.2023.3277462.
- 26 Montalbo F. J. P., Diagnosing Gastrointestinal Diseases From Endoscopy Images Through a Multi-Fused CNN With Auxiliary Layers, Alpha Dropouts, and a Fusion Residual Block, Biomedical Signal Processing and Control. (2022) 76, https://doi.org/10.1016/j.bspc.2022.103683.
- 27 Howard A. G., Zhu M., Chen B. et al., Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications, 2017, https://arxiv.org/abs/170404861.
- 28 Huang G., Liu Z., Van Der Maaten L., and Weinberger K. Q., Densely Connected Convolutional Networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, 4700–4708.
- 29 Wang S., Celebi M. E., Zhang Y. D. et al., Advances in Data Preprocessing for Biomedical Data Fusion: An Overview of the Methods, Challenges, and Prospects, Information Fusion. (2021) 76, 376–421, https://doi.org/10.1016/j.inffus.2021.07.001.
- 30 Poudel S., Kim Y. J., Vo D. M., and Lee S.-W., Colorectal Disease Classification Using Efficiently Scaled Dilation in Convolutional Neural Network, IEEE Access. (2020) 8, 99227–99238, https://doi.org/10.1109/access.2020.2996770.
- 31 Das D., Santosh K., and Pal U., Truncated Inception Net: COVID-19 Outbreak Screening Using Chest X-Rays, Physical and Engineering Sciences in Medicine. (2020) 43, no. 3, 915–925, https://doi.org/10.1007/s13246-020-00888-x.
- 32 Ibtehaz N. and Rahman M. S., MultiResUNet: Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation, Neural Networks. (2020) 121, 74–87, https://doi.org/10.1016/j.neunet.2019.08.025.
- 33 Woo S., Park J., Lee J.-Y., and Kweon I. S., Cbam: Convolutional Block Attention Module, Proceedings of the European Conference on Computer Vision (ECCV). (2018) 3–19.
- 34 Asif S. and Ain Q.-U., A Fuzzy Minkowski Distance-Based Fusion of Convolutional Neural Networks for Gastrointestinal Disease Detection, Applied Soft Computing. (2024) 158, https://doi.org/10.1016/j.asoc.2024.111595.
- 35
Waheed Z. and
Gui J., An Optimized Ensemble Model Bfased on Cuckoo Search With Levy Flight for Automated Gastrointestinal Disease Detection, Multimedia Tools and Applications. (2024) 83, no. 42, 89695–89722, https://doi.org/10.1007/s11042-024-18937-y.
10.1007/s11042-024-18937-y Google Scholar
- 36
Selvaraju R. R.,
Cogswell M.,
Das A.,
Vedantam R.,
Parikh D., and
Batra D., Grad-Cam: Visual Explanations from Deep Networks via Gradient-Based Localization, Proceedings of the IEEE International Conference on Computer Vision, October 2017, Venice, Italy, 618–626, https://doi.org/10.1109/iccv.2017.74, 2-s2.0-85041910265.
10.1109/iccv.2017.74 Google Scholar
- 37
Ribeiro M. T.,
Singh S., and
Guestrin C., Why Should I Trust You? Explaining the Predictions of Any Classifier, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2016, San Francisco, 1135–1144, https://doi.org/10.1145/2939672.2939778, 2-s2.0-84984985889.
10.1145/2939672.2939778 Google Scholar
- 38 Lundberg S., A Unified Approach to Interpreting Model Predictions, 2017, https://arxiv.org/abs/1705.07874.
- 39
Jha D.,
Sharma V.,
Dasu N. et al., GastroVision: A Multi-Class Endoscopy Image Dataset for Computer Aided Gastrointestinal Disease Detection, Lecture Notes in Computer Science. (2023) 125–140, https://doi.org/10.1007/978-3-031-47679-2_10.
10.1007/978-3-031-47679-2_10 Google Scholar
- 40 Borkowski A. A., Bui M. M., Thomas L. B., Wilson C. P., DeLand L. A., and Mastorides S. M., Lc25000 Lung and Colon Histopathological Image Dataset, 2023.
- 41 Hamada A., Br35H: Brain Tumor Detection 2020 Version 5, 2020, https://www.kaggle.com/ahmedhamada0/brain-tumor-detection.
- 42 Yildirim K., Bozdag P. G., Talo M., Yildirim O., Karabatak M., and Acharya U. R., Deep Learning Model for Automated Kidney Stone Detection Using Coronal CT Images, Computers in Biology and Medicine. (2021) 135, https://doi.org/10.1016/j.compbiomed.2021.104569.
- 43 Alzubaidi L., Fadhel M. A., Oleiwi S. R., Al-Shamma O., and Zhang J., DFU_QUTNet: Diabetic Foot Ulcer Classification Using Novel Deep Convolutional Neural Network, Multimedia Tools and Applications. (2020) 79, no. 21-22, 15655–15677, https://doi.org/10.1007/s11042-019-07820-w, 2-s2.0-85068318206.
- 44 Sikder J., Das U. K., and Chakma R. J., Supervised Learning-Based Cancer Detection, International Journal of Advanced Computer Science and Applications. (2021) 12, no. 5, https://doi.org/10.14569/ijacsa.2021.01205101.
- 45 Hage Chehade A., Abdallah N., Marion J.-M., Oueidat M., and Chauvet P., Lung and Colon Cancer Classification Using Medical Imaging: A Feature Engineering Approach, Physical and Engineering Sciences in Medicine. (2022) 45, no. 3, 729–746, https://doi.org/10.1007/s13246-022-01139-x.
- 46 Masud M., Sikder N., Nahid A.-A., Bairagi A. K., and AlZain M. A., A Machine Learning Approach to Diagnosing Lung and Colon Cancer Using a Deep Learning-Based Classification Framework, Sensors. (2021) 21, no. 3, https://doi.org/10.3390/s21030748.
- 47 Mengash H. A., Alamgeer M., Maashi M. et al., Leveraging Marine Predators Algorithm With Deep Learning for Lung and Colon Cancer Diagnosis, Cancers. (2023) 15, no. 5, https://doi.org/10.3390/cancers15051591.
- 48 Liang M., Ren Z., Yang J., Feng W., and Li B., Identification of Colon Cancer Using Multi-Scale Feature Fusion Convolutional Neural Network Based on Shearlet Transform, IEEE Access. (2020) 8, 208969–208977, https://doi.org/10.1109/access.2020.3038764.
- 49 Zhu H., Liu W., Gao Z., and Zhang H., Explainable Classification of Benign-Malignant Pulmonary Nodules With Neural Networks and Information Bottleneck, IEEE Transactions on Neural Networks and Learning Systems. (2025) 36, no. 2, 2028–2039, https://doi.org/10.1109/tnnls.2023.3303395.
- 50 Sharma P., Balabantaray B. K., Bora K., Mallik S., Kasugai K., and Zhao Z., An Ensemble-Based Deep Convolutional Neural Network for Computer-Aided Polyps Identification from Colonoscopy, Frontiers in Genetics. (2022) 13, https://doi.org/10.3389/fgene.2022.844391.
- 51 Younas F., Usman M., and Yan W., A Deep Ensemble Learning Method for Colorectal Polyp Classification with Optimized Network Parameters, Applied Intelligence. (2022) 53, no. 2, 2410–2433, https://doi.org/10.1007/s10489-022-03689-9.
