Funding information German Research Foundation, Grant/Award Numbers: PO 563/29-1, EG 102/9-1, PO 563/30-1, BO 4700/1-1, RO 5093/1-1, DE 735/10-1; Leibniz ScienceCampus InfectoOptics

Read the full text

About

PDF

Tools

Share a link

Email
Wechat
Bluesky

Abstract

Background

A fully convolutional neural networks (FCN)-based automated image analysis algorithm to discriminate between head and neck cancer and noncancerous epithelium based on nonlinear microscopic images was developed.

Methods

Head and neck cancer sections were used for standard histopathology and co-registered with multimodal images from the same sections using the combination of coherent anti-Stokes Raman scattering, two-photon excited fluorescence, and second harmonic generation microscopy. The images analyzed with semantic segmentation using a FCN for four classes: cancer, normal epithelium, background, and other tissue types.

Results

A total of 114 images of 12 patients were analyzed. Using a patch score aggregation, the average recognition rate and an overall recognition rate or the four classes were 88.9% and 86.7%, respectively. A total of 113 seconds were needed to process a whole-slice image in the dataset.

Conclusion

Multimodal nonlinear microscopy in combination with automated image analysis using FCN seems to be a promising technique for objective differentiation between head and neck cancer and noncancerous epithelium.

Supporting Information

REFERENCES

1Green B, Cobb AR, Brennan PA, Hopper C. Optical diagnostic techniques for use in lesions of the head and neck: review of the latest developments. Br J Oral Maxillofa Surg. 2014; 52(8): 675-680.
10.1016/j.bjoms.2014.06.010
PubMed Web of Science® Google Scholar
2Singh SP, Ibrahim O, Byrne HJ, et al. Recent advances in optical diagnosis of oral cancers: review and future perspectives. Head Neck. 2016; 38(Suppl 1): E2403-E2411.
10.1002/hed.24293
PubMed Web of Science® Google Scholar
3Bocklitz T, Brautigam K, Urbanek A, et al. Novel workflow for combining Raman spectroscopy and MALDI-MSI for tissue based studies. Anal Bioanal Chem. 2015; 407(26): 7865-7873.
10.1007/s00216-015-8987-5
CAS PubMed Web of Science® Google Scholar
4Downes A, Elfick A. Raman spectroscopy and related techniques in biomedicine. Sensors (Basel). 2010; 10(3): 1871-1889.
10.3390/s100301871
PubMed Google Scholar
5Heuke S, Chernavskaia O, Bocklitz T, et al. Multimodal nonlinear microscopy of head and neck carcinoma - toward surgery assisting frozen section analysis. Head Neck. 2016; 38(10): 1545-1552.
10.1002/hed.24477
PubMed Web of Science® Google Scholar
6Bocklitz TW, Salah FS, Vogler N, et al. Pseudo-HE images derived from CARS/TPEF/SHG multimodal imaging in combination with Raman-spectroscopy as a pathological screening tool. BMC Cancer. 2016; 16: 534.
10.1186/s12885-016-2520-x
PubMed Web of Science® Google Scholar
7Meyer T, Baumgartl M, Gottschall T, et al. A compact microscope setup for multimodal nonlinear imaging in clinics and its application to disease diagnostics. Analyst. 2013; 138(14): 4048-4057.
10.1039/c3an00354j
CAS PubMed Web of Science® Google Scholar
8Meyer T, Guntinas-Lichius O, von Eggeling F, et al. Multimodal nonlinear microscopic investigations on head and neck squamous cell carcinoma: toward intraoperative imaging. Head Neck. 2013; 35(9): E280-E287.
10.1002/hed.23139
PubMed Web of Science® Google Scholar
9Heuke S, Vogler N, Meyer T, et al. Multimodal mapping of human skin. Br J Dermatol. 2013; 169(4): 794-803.
10.1111/bjd.12427
CAS PubMed Web of Science® Google Scholar
10Duan L, Marvdashti T, Lee A, Tang JY, Ellerbee AK. Automated identification of basal cell carcinoma by polarization-sensitive optical coherence tomography. Biomed Opt Exp. 2014; 5(10): 3717-3729.
10.1364/BOE.5.003717
PubMed Web of Science® Google Scholar
11Chernavskaia O, Heuke S, Vieth M, et al. Beyond endoscopic assessment in inflammatory bowel disease: real-time histology of disease activity by non-linear multimodal imaging. Sci Rep. 2016; 6: 29239.
10.1038/srep29239
CAS PubMed Web of Science® Google Scholar
12Hoesli RC, Orringer DA, McHugh JB, Spector ME. Coherent Raman scattering microscopy for evaluation of head and neck carcinoma. Otolaryngol Head Neck Surg. 2017; 157(3): 448-453.
10.1177/0194599817700388
PubMed Web of Science® Google Scholar
13Cals FL, Koljenovic S, Hardillo JA, Baatenburg de Jong RJ, Bakker Schut TC, Puppels GJ. Development and validation of Raman spectroscopic classification models to discriminate tongue squamous cell carcinoma from non-tumorous tissue. Oral Oncol. 2016; 60: 41-47.
10.1016/j.oraloncology.2016.06.012
PubMed Web of Science® Google Scholar
14Keereweer S, Van Driel PB, Snoeks TJ, et al. Optical image-guided cancer surgery: challenges and limitations. Clin Cancer Res. 2013; 19(14): 3745-3754.
10.1158/1078-0432.CCR-12-3598
PubMed Web of Science® Google Scholar
15Dittberner A, Rodner E, Ortmann W, et al. Automated analysis of confocal laser endomicroscopy images to detect head and neck cancer. Head Neck. 2016; 38(Suppl 1): E1419-E1426.
10.1002/hed.24253
PubMed Web of Science® Google Scholar
16Lukic A, Dochow S, Chernavskaia O, et al. Fiber probe for nonlinear imaging applications. J Biophotonics. 2016; 9(1-2): 138-143.
10.1002/jbio.201500010
CAS PubMed Web of Science® Google Scholar
17Regeling B, Thies B, Gerstner AO, et al. Hyperspectral imaging using flexible endoscopy for laryngeal cancer detection. Sensors (Basel). 2016; 16(8). pii: E1288. https://doi.org/10.3390/s1608128.
Google Scholar

Citing Literature

Volume41, Issue1

January 2019

Pages 116-121

Filename	Description
hed25489-sup-0001-FigureS1.docxWord 2007 document , 6.9 MB	Supplemental Figure 1 Architecture of the presented fully-convolutional neural network used in CN24 (https://www.github.com/cvjena/cn24).
hed25489-sup-0002-FigureS2.docxWord 2007 document , 316.3 KB	Supplemental Figure 2 Development of the loss and average recognition rate on the training set while training our networks with back-propagation for all test patients. The maximum number of epochs was 80. A: loss; B: overall recognition rate; C: average recognition rate.

Fully convolutional networks in multimodal nonlinear microscopy images for automated detection of head and neck carcinoma: Pilot study

Abstract

Background

Methods

Results

Conclusion

Supporting Information

REFERENCES

Citing Literature

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Fully convolutional networks in multimodal nonlinear microscopy images for automated detection of head and neck carcinoma: Pilot study

Abstract

Background

Methods

Results

Conclusion

Supporting Information

REFERENCES

Citing Literature

References

Related

Information