3D multi-resolution deep learning model for diagnosis of multiple pathological types on pulmonary nodules
Yu Fu
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorPeng Xue
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorPeng Zhao
Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
Search for more papers by this authorNing Li
Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
Search for more papers by this authorZhuodong Xu
Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
Search for more papers by this authorHuizhong Ji
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorZhili Zhang
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorCorresponding Author
Wentao Cui
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Correspondence
Enqing Dong and Wentao Cui, Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264209, China.
Email: [email protected] and [email protected]
Search for more papers by this authorCorresponding Author
Enqing Dong
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Correspondence
Enqing Dong and Wentao Cui, Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264209, China.
Email: [email protected] and [email protected]
Search for more papers by this authorYu Fu
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorPeng Xue
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorPeng Zhao
Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
Search for more papers by this authorNing Li
Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
Search for more papers by this authorZhuodong Xu
Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
Search for more papers by this authorHuizhong Ji
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorZhili Zhang
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Search for more papers by this authorCorresponding Author
Wentao Cui
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Correspondence
Enqing Dong and Wentao Cui, Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264209, China.
Email: [email protected] and [email protected]
Search for more papers by this authorCorresponding Author
Enqing Dong
Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai, China
Correspondence
Enqing Dong and Wentao Cui, Department of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264209, China.
Email: [email protected] and [email protected]
Search for more papers by this authorYu Fu, Peng Xue, and Peng Zhao are co-first authors.
[Correction added on 21st August, after first online publication: Grant number added in Funder information.]
Funding information: Fundamental Research Funds for the Central Universities; Key Research and Development Project of Shandong Province, Grant/Award Number: 2019GGX101022; National Natural Science Foundation of China, Grant/Award Numbers: 62171261, 81371635, 81671848
Abstract
To accurately diagnose multiple pathological types of pulmonary nodules based on lung computed tomography (CT) images, a multi-resolution three-dimensional (3D) multi-classification deep learning model (Mr-Mc) was proposed. The Mr-Mc model was constructed by using our own constructed lung CT image dataset of pulmonary nodules with clinical pathological information (LCID-CPI), which can accurately diagnose inflammation, squamous cell carcinoma, adenocarcinoma, and other benign diseases. In order to process nodules with different sizes, a multi-resolution extraction method was proposed to extract 3D volume data with different resolutions from lung CT images. The Mr-Mc was composed of three different resolution networks, each of which has input volume data of a specific resolution. Experiments showed that the constructed Mr-Mc model can achieve an average accuracy of 0.81 on LCID-CPI. Besides, the Mr-Mc model can also achieve a high accuracy of 0.87 on the Lung Image Database Consortium and Image Database Resource Initiative dataset.
CONFLICT OF INTEREST
The authors declare no conflicts of interest.
Open Research
DATA AVAILABILITY STATEMENT
The raw/processed lung CT image dataset generated or analyzed during this study is not publicly available as the Dicom metadata containing information that could compromise patient privacy/consent and the data also form part of an ongoing study.
REFERENCES
- 1Ferlay J, Colombet M, Soerjomataram I, et al. Estimating the global cancer incidence and mortality in 2018: Globocan sources and methods. Int J Cancer. 2019; 144: 1941-1953. https://doi.org/10.3322/caac.21492
- 2Jin X, Zhang Y, Jin Q. Pulmonary nodule detection based on CT images using convolution neural network: Proceedings of the International Symposium on Computational Intelligence & Design (ISCID), Hangzhou, China; 2017, pp. 202-204. https://doi.org/10.1109/ISCID.2016.1053
10.1109/ISCID.2016.1053 Google Scholar
- 3Zhang S, Han D, Williamson JF, et al. Experimental implementation of a joint statistical image reconstruction method for proton stopping power mapping from dual-energy CT data. Med Phys. 2019; 46(1): 273-285. https://doi.org/10.1002/mp.13287
- 4Xue P, Dong E, Ji H. Lung 4D CT image registration based on high-order Markov random field. IEEE Trans Med Imaging. 2020; 39(4): 910-921. https://doi.org/10.1109/TMI.2019.2937458
- 5Dukov N, Bliznakova K, Feradov F, et al. Models of breast lesions based on three-dimensional X-ray breast images. Phys Med. 2019; 57: 80-87. https://doi.org/10.1016/j.ejmp.2018.12.012
- 6Hernando ML, Marks LB, Bentel GC, et al. Radiation-induced pulmonary toxicity: a dose-volume histogram analysis in 201 patients with lung cancer. Int J Radiat Oncol Biol Phys. 2001; 51(3): 650-659. https://doi.org/10.1016/S0360-3016(01)01685-6
- 7Green CA, Goodsitt MM, Lau JH, et al. Deformable mapping method to relate lesions in dedicated breast CT images to those in automated breast ultrasound and digital breast tomosynthesis images. Ultrasound Med Biol. 2020; 46(3): 750-765. https://doi.org/10.1016/j.ultrasmedbio.2019.10.016
- 8Han F, Wang H, Zhang G, et al. Texture feature analysis for computer-aided diagnosis on pulmonary nodules. J Digit Imaging. 2015; 28(1): 99-115. https://doi.org/10.1007/s10278-014-9718-8
- 9Hu H, Nie S. Classification of malignant-benign pulmonary nodules in lung CT images using an improved random forest: Proceedings of the 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Guilin, China; 2017, pp. 2285-2290. https://doi.org/10.1109/FSKD.2017.8393127
10.1109/FSKD.2017.8393127 Google Scholar
- 10Jacobs C, Van Rikxoort EM, Twellmann T, et al. Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images. Med Image Anal. 2014; 18(2): 374-384. https://doi.org/10.1016/j.media.2013.12.001
- 11Haralick R. Statistical and structural approaches to texture. Proc IEEE. 1979; 67(5): 786-804. https://doi.org/10.1109/PROC.1979.11328
- 12Wang L, He D. Texture classification using texture spectrum. Pattern Recognit. 1990; 23(8): 905-910. https://doi.org/10.1016/0031-3203(90)90135-8
- 13Gabor D. Theory of communication. IEEE Proc London. 1946; 93(73): 429-441. https://doi.org/10.1049/ji-1.1947.0015
10.1049/ji-1.1947.0015 Google Scholar
- 14Armato SG, McLennan G, Bidaut L, et al. The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys. 2011; 38(2): 915-931. https://doi.org/10.1118/1.3528204
- 15Cutler A, Cutler D, Stevens JR. Random forests. Mach Learn. 2004; 45(1): 157-176. https://doi.org/10.1023/A:1010933404324
- 16Rubin M, Stein O, Turko NA, et al. TOP-GAN: stain-free cancer cell classification using deep learning with a small training set. Med Image Anal. 2019; 57: 176-185. https://doi.org/10.1016/j.media.2019.06.014
- 17Fu Y, Xue P, Dong E, et al. Deep model with Siamese network for viable and necrotic tumor regions assessment in osteosarcoma. Med Phys. 2020; 47(10): 1895-1905. https://doi.org/10.1002/mp.14397
- 18Li Y, Li K, Zhang C, et al. Learning to reconstruct computed tomography (CT) images directly from Sinogram data under a variety of data acquisition conditions. IEEE Trans Med Imaging. 2019; 38(10): 2469-2481. https://doi.org/10.1109/TMI.2019.2910760
- 19Chen X, Men K, Li Y, et al. A feasibility study on an automated method to generate patient-specific dose distributions for radiotherapy using deep learning. Med Phys. 2019; 46(1): 56-64. https://doi.org/10.1002/mp.13262
- 20Setio AAA, Traverso A, De Bel T, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Med Image Anal. 2017; 42: 1-13. https://doi.org/10.1016/j.media.2017.06.015
- 21Khoshdeli M, Cong R, Parvin B. Detection of nuclei in H&E stained sections using convolutional neural networks. Paper presented at: IEEE EMBS International Conference on Biomedical & Health Informatics (BHI); 2017, pp. 105-108. https://doi.org/10.1109/BHI.2017.7897216
10.1109/BHI.2017.7897216 Google Scholar
- 22Roth H, Lu L, Liu J, et al. Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging. 2016; 35(5): 1170-1181. https://doi.org/10.1109/TMI.2015.2482920
- 23Dou Q, Chen H, Yu L, et al. Multi-level contextual 3D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng. 2017; 64(7): 1558-1567. https://doi.org/10.1109/TBME.2016.2613502
- 24Tan M, Deklerck R, Jansen B, et al. A novel computer-aided lung nodule detection system for CT images. Med Phys. 2011; 38(10): 5630-5645. https://doi.org/10.1118/1.3633941
- 25Suzuki K, Li F, Sone S, et al. Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network. IEEE Trans Med Imaging. 2005; 24(9): 1138-1150. https://doi.org/10.1109/TMI.2005.852048
- 26Liu L, Dou Q, Chen H, et al. Multi-task deep model with margin ranking loss for lung nodule analysis. IEEE Trans Med Imaging. 2020; 39(3): 718-728. https://doi.org/10.1109/TMI.2019.2934577
- 27Shen W, Zhou M, Yang F, et al. Multi-scale convolutional neural networks for lung nodule classification. Int Conf Inf Process Med Imaging. 2015; 24: 588-599. https://doi.org/10.1007/978-3-319-19992-4_46
- 28Setio A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging. 2016; 35(5): 1160-1169. https://doi.org/10.1109/TMI.2016.2536809
- 29Zhu W, Liu C, Fan W, et al. DeepLung: deep 3D dual path nets for automated pulmonary nodule detection and classification. Paper presented at: 2018 IEEE Winter Conference on Applications of Computer Vision; 2018. https://doi.org/10.1109/wacv.2018.000792018:673-681
10.1109/wacv.2018.000792018:673-681 Google Scholar
- 30Xu X, Wang C, Guo J, et al. MSCS-DeepLN: evaluating lung nodule malignancy using multi-scale cost-sensitive neural networks. Med Image Anal. 2020; 65:101772. https://doi.org/10.1016/j.media.2020.101772
- 31He K, Zhang X, Ren S, et al. Deep residual learning for image recognition: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016, pp. 770-778. https://doi.org/10.1109/CVPR.2016.90
10.1109/CVPR.2016.90 Google Scholar
- 32Huang G, Liu Z, Maaten LVD, et al. Densely connected convolutional networks. Paper presented at: 30th IEEE Conference on Computer Vision and Pattern Recognition; 2017, pp. 2261-2269. https://doi.org/10.1109/CVPR.2017.243
10.1109/CVPR.2017.243 Google Scholar
- 33Chen Y, Li J, Xiao H, et al. Dual path networks: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017, pp. 1-11. https://doi.org/10.5555/3294996.3295200
10.5555/3294996.3295200 Google Scholar
- 34He K, Zhang X, Ren S, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. Paper presented at: IEEE International Conference on Computer Vision; 2015, pp. 1026-1034. https://doi.org/10.1109/ICCV.2015.123
10.1109/ICCV.2015.123 Google Scholar
- 35Dey R, Lu Z, Hong Y. Diagnostic classification of pulmonary nodules using 3D neural networks: Proceedings of the IEEE 15th International Symposium on Biomedical Imaging (ISBI); 2018, pp. 774-778. https://doi.org/10.1109/ISBI.2018.8363687
10.1109/ISBI.2018.8363687 Google Scholar
- 36Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization. Paper presented at: IEEE Conference on Computer Vision and Pattern Recognition; 2016, pp. 2921-2929. https://doi.org/10.1109/CVPR.2016.319
10.1109/CVPR.2016.319 Google Scholar
- 37Shen W, Zhou M, Yang F, et al. Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit. 2017; 61(61): 663-673. https://doi.org/10.1016/j.patcog.2016.05.029
- 38Liu K, Kang G. Multiview convolutional neural networks for lung nodule classification. Int J Imaging Syst Technol. 2017; 27(1): 12-22. https://doi.org/10.1002/ima.22206
- 39Wei G, Ma H, Qian W, et al. Lung nodule classification using local kernel regression models with out-of-sample extension. Biomed Signal Process Control. 2018; 40: 1-9. https://doi.org/10.1016/j.bspc.2017.08.026
