Use of a convolutional neural network for classifying microvessels of superficial esophageal squamous cell carcinomas
Ryotaro Uema
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorYoshito Hayashi
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTaku Tashiro
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorHirotsugu Saiki
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorMinoru Kato
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTakahiro Amano
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorMizuki Tani
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTakeo Yoshihara
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTakanori Inoue
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorKeiichi Kimura
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorShuko Iwatani
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorAkihiko Sakatani
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorShunsuke Yoshii
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorYoshiki Tsujii
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorShinichiro Shinzaki
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorHideki Iijima
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorCorresponding Author
Tetsuo Takehara
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Correspondence
Dr Tetsuo Takehara, Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan.
Email: [email protected]
Search for more papers by this authorRyotaro Uema
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorYoshito Hayashi
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTaku Tashiro
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorHirotsugu Saiki
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorMinoru Kato
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTakahiro Amano
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorMizuki Tani
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTakeo Yoshihara
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorTakanori Inoue
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorKeiichi Kimura
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorShuko Iwatani
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorAkihiko Sakatani
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorShunsuke Yoshii
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorYoshiki Tsujii
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorShinichiro Shinzaki
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorHideki Iijima
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Search for more papers by this authorCorresponding Author
Tetsuo Takehara
Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Osaka, Japan
Correspondence
Dr Tetsuo Takehara, Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan.
Email: [email protected]
Search for more papers by this authorFinancial support: This work was supported by the Council for Science, Technology, and Innovation (CSTI), the cross-ministerial Strategic Innovation Promotion Program (SIP), “Innovative AI Hospital System” (funding agency: National Institute of Biomedical Innovation, Health and Nutrition [NIBIOHN]); the Global Center for Medical Engineering and Informatics at Osaka University; and a grant from the Japanese Foundation for Research and Promotion of Endoscopy. No author has a financial relationship relevant to this publication.
Abstract
Background and Aim
The morphological diagnosis of microvessels on the surface of superficial esophageal squamous cell carcinomas using magnifying endoscopy with narrow-band imaging is widely used in clinical practice. Nevertheless, inconsistency, even among experts, remains a problem. We constructed a convolutional neural network-based computer-aided diagnosis system to classify the microvessels of superficial esophageal squamous cell carcinomas and evaluated its diagnostic performance.
Methods
In this retrospective study, a cropped magnifying endoscopy with narrow-band images from superficial esophageal squamous cell carcinoma lesions was used as the dataset. All images were assessed by three experts, and classified into three classes, Type B1, B2, and B3, based on the Japan Esophagus Society classification. The dataset was divided into training and validation datasets. A convolutional neural network model (ResNeXt-101) was trained and tuned with the training dataset. To evaluate diagnostic accuracy, the validation dataset was assessed by the computer-aided diagnosis system and eight endoscopists.
Results
In total, 1777 and 747 cropped images (total, 393 lesions) were included in the training and validation datasets, respectively. The diagnosis system took 20.3 s to evaluate the 747 images in the validation dataset. The microvessel classification accuracy of the computer-aided diagnosis system was 84.2%, which was higher than the average of the eight endoscopists (77.8%, P < 0.001). The area under the receiver operating characteristic curves for diagnosing Type B1, B2, and B3 vessels were 0.969, 0.948, and 0.973, respectively.
Conclusions
The computer-aided diagnosis system showed remarkable performance in the classification of microvessels on superficial esophageal squamous cell carcinomas.
Supporting Information
| Filename | Description |
|---|---|
| jgh_15479-sup-0001-Data_S1.docxWord 2007 document , 44.7 KB |
Table S1. Concordance rates and kappa statistics between three experts in all the cropped images. Table S2. The diagnostic accuracy of the invasion depth of the gold standard diagnosis for ER images. Table S3. Classification accuracy of the model and each evaluator. Table S4. Diagnostic accuracy of the invasion depth of the model and each evaluator in the validation dataset. |
| jgh_15479-sup-0002-Figure_S1.tifTIFF image, 7.4 MB |
Figure S1. The process of cropping images from ME-NBI images. (a) Areas where microvessels were clearly visualized were cropped to 500 x 500 pixels or 430 x 430 pixels, which were approximately one-fourth of the endoscopic image area. (b) When two or more vessel types coexist in one image, each region was cropped separately. |
| jgh_15479-sup-0003-Figure_S2.tifTIFF image, 2 MB |
Figure S2. The contingency tables of each evaluator's diagnosis. |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- 1Fitzmaurice C, Abate D, Abbasi N et al. Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 29 cancer groups, 1990 to 2017. JAMA Oncol. 2019; 5: 1749–1768.
- 2Arnold M, Soerjomataram I, Ferlay J, Forman D. Global incidence of oesophageal cancer by histological subtype in 2012. Gut 2015; 64: 381–387.
- 3Evans JA, Early DS, Chandraskhara V et al. The role of endoscopy in the assessment and treatment of esophageal cancer. Gastrointest. Endosc. 2013; 77: 328–334.
- 4Yamashina T, Ishihara R, Nagai K et al. Long-term outcome and metastatic risk after endoscopic resection of superficial esophageal squamous cell carcinoma. Am. J. Gastroenterol. 2013; 108: 544–551.
- 5Tsujii Y, Nishida T, Nishiyama O et al. Clinical outcomes of endoscopic submucosal dissection for superficial esophageal neoplasms: a multicenter retrospective cohort study. Endoscopy 2015; 47: 775–783.
- 6Berger A, Rahmi G, Perrod G et al. Long-term follow-up after endoscopic resection for superficial esophageal squamous cell carcinoma: a multicenter Western study. Endoscopy 2019; 51: 298–306.
- 7Bollschweiler E, Baldus SE, Schröder W et al. High rate of lymph-node metastasis in submucosal esophageal squamous-cell carcinomas and adenocarcinomas. Endoscopy 2006; 38: 149–156.
- 8Goda K, Tajiri H, Ikegami M et al. Magnifying endoscopy with narrow band imaging for predicting the invasion depth of superficial esophageal squamous cell carcinoma. Dis. Esophagus 2009; 22: 453–460.
- 9Inoue H. Magnification endoscopy in the esophagus and stomach. Dig. Endosc. 2008; 13: S40–S41.
10.1046/j.1443-1661.2001.0130s1S40.x Google Scholar
- 10Kumagai Y, Inoue H, Nagai K, Kawano T, Iwai T. Magnifying endoscopy, stereoscopic microscopy, and the microvascular architecture of superficial esophageal carcinoma. Endoscopy 2002; 34: 369–375.
- 11Arima M, Tada M, Arima H. Evaluation of microvascular patterns of superficial esophageal cancers by magnifying endoscopy. Esophagus 2005; 2: 191–197.
10.1007/s10388-005-0060-6 Google Scholar
- 12Oyama T, Monma K. Workshop 1: a new classification of magnified endoscopy for superficial esophageal squamous cell carcinoma. Esophagus 2011; 8: 247–251.
- 13Oyama T, Inoue H, Arima M et al. Prediction of the invasion depth of superficial squamous cell carcinoma based on microvessel morphology: magnifying endoscopic classification of the Japan Esophageal Society. Esophagus 2017; 14: 105–112.
- 14Ebi M, Shimura T, Yamada T et al. Multicenter, prospective trial of white-light imaging alone versus white-light imaging followed by magnifying endoscopy with narrow-band imaging for the real-time imaging and diagnosis of invasion depth in superficial esophageal squamous cell carcinoma. Gastrointest. Endosc. 2015; 81: 1355–1361.
- 15Kim SJ, Kim GH, Lee MW et al. New magnifying endoscopic classification for superficial esophageal squamous cell carcinoma. World J. Gastroenterol. 2017; 23: 4416–4421.
- 16Katada C, Tanabe S, Wada T et al. Retrospective assessment of the diagnostic accuracy of the depth of invasion by narrow band imaging magnifying endoscopy in patients with superficial esophageal squamous cell carcinoma. J. Gastrointest. Cancer 2019; 50: 292–297.
- 17Horie Y, Yoshio T, Aoyama K et al. Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks. Gastrointest. Endosc. 2019; 89: 25–32.
- 18Xue DX, Zhang R, Feng H, Wang YL. CNN-SVM for microvascular morphological type recognition with data augmentation. J. Med. Biol. Eng. 2016; 36: 755–764.
- 19Zhao YY, Xue DX, Wang YL et al. Computer-assisted diagnosis of early esophageal squamous cell carcinoma using narrow-band imaging magnifying endoscopy. Endoscopy 2019; 51: 333–341.
- 20Everson M, Herrera LCGP, Li W et al. Artificial intelligence for the real-time classification of intrapapillary capillary loop patterns in the endoscopic diagnosis of early oesophageal squamous cell carcinoma: a proof-of-concept study. United Eur. Gastroenterol. J. 2019; 7: 297–306.
- 21 Japan Esophageal Society. Japanese Classification of Esophageal Cancer, 11th Edition: part I. Esophagus 2017; 14: 1–36.
- 22Xie S, Girshick R, Dollar P, Tu Z, He K. Aggregated residual transformations for deep neural networks. Cited 24 Aug 2020. Available from URL: https://openaccess.thecvf.com/content_cvpr_2017/html/Xie_Aggregated_Residual_Transformations_CVPR_2017_paper.html
- 23Liu L, Jiang H, He P, et al. On the variance of the adaptive learning rate and beyond. Cited 24 Aug 2020. Available from URL: https://arxiv.org/abs/1908.03265
- 24Chattopadhyay A, Sarkar A, Howlader P, Balasubramanian, VN. Grad-CAM++: improved visual explanations for deep convolutional networks. Cited 24 Aug 2020. Available from URL: https://arxiv.org/abs/1710.11063
- 25Nakagawa K, Ishihara R, Aoyama K et al. Classification for invasion depth of esophageal squamous cell carcinoma using a deep neural network compared with experienced endoscopists. Gastrointest. Endosc. 2019; 90: 407–414.
- 26Kuwano H, Nishimura Y, Oyama T et al. Guidelines for diagnosis and treatment of carcinoma of the Esophagus April 2012 edited by the Japan esophageal society. Esophagus 2015; 12: 1–30.
- 27Eguchi T, Nakanishi Y, Shimoda T et al. Histopathological criteria for additional treatment after endoscopic mucosal resection for esophageal cancer: analysis of 464 surgically resected cases. Mod. Pathol. 2006; 19: 475–480.
- 28Kodama M, Kakegawa T. Treatment of superficial cancer of the esophagus: a summary of responses to a questionnaire on superficial cancer of the esophagus in Japan. Surgery 1998; 123: 432–439.
- 29Shimada H, Nabeya Y, Matsubara H et al. Prediction of lymph node status in patients with superficial esophageal carcinoma: analysis of 160 surgically resected cancers. Am. J. Surg. 2006; 191: 250–254.
- 30Mizumoto T, Hiyama T, Quach DT et al. Magnifying endoscopy with narrow band imaging in estimating the invasion depth of superficial esophageal squamous cell carcinomas. Digestion 2018; 98: 249–256.
- 31Fitzmaurice C, Akinyemiju TF, Al Lami FH et al. Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 29 cancer groups, 1990 to 2016: a systematic analysis for the global burden of disease study. JAMA Oncol. 2018; 4: 1553–1568.
- 32Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. CA Cancer J. Clin. 2018; 68: 7–30.
- 33Hori M, Matsuda T, Shibata A, Katanoda K, Sobue T, Nishimoto H. Cancer incidence and incidence rates in Japan in 2009: a study of 32 population-based cancer registries for the Monitoring of Cancer Incidence in Japan (MCIJ) project. Jpn. J. Clin. Oncol. 2015; 45: 884–891.
- 34Tachimori Y, Ozawa S, Numasaki H et al. Comprehensive registry of esophageal cancer in Japan, 2011. Esophagus 2018; 15: 127–152.
- 35Shichijo S, Nomura S, Aoyama K et al. Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine 2017; 25: 106–111.
- 36Hirasawa T, Aoyama K, Tanimoto T et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 2018; 21: 653–660.
- 37Chen PJ, Lin MC, Lai MJ, Lin JC, Lu HH, Tseng VS. Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology 2018; 154: 568–575.
- 38Wang P, Xiao X, Glissen Brown JR et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat. Biomed. Eng. 2018; 2: 741–748.
- 39Leenhardt R, Vasseur P, Li C et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest. Endosc. 2019; 89: 189–194.
- 40Aoki T, Yamada A, Aoyama K et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest. Endosc. 2019; 89: 357–336.
- 41Vinsard DG, Mori Y, Misawa M et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest. Endosc. 2019; 90: 55–63.
Citing Literature
