Address reprint requests to: G.Z., Department of Radiology, Stanford University, School of Medicine, 1201 Welch Road, Stanford, CA 94305-5488. E-mail: [email protected]Search for more papers by this author

Endre Grøvik PhD,

Endre Grøvik PhD

orcid.org/0000-0002-9925-1162

Department of Radiology, Stanford University, Stanford, California, USA

Department for Diagnostic Physics, Oslo University Hospital, Oslo, Norway

Search for more papers by this author

Darvin Yi MS,

Darvin Yi MS

Department of Biomedical Data Science, Stanford University, Stanford, California, USA

Search for more papers by this author

Michael Iv MD,

Michael Iv MD

Department of Radiology, Stanford University, Stanford, California, USA

Search for more papers by this author

Elizabeth Tong MD,

Elizabeth Tong MD

Department of Radiology, Stanford University, Stanford, California, USA

Search for more papers by this author

Daniel Rubin PhD,

Daniel Rubin PhD

Department of Biomedical Data Science, Stanford University, Stanford, California, USA

Search for more papers by this author

Greg Zaharchuk MD, PhD,

Corresponding Author

Greg Zaharchuk MD, PhD

[email protected]

Department of Radiology, Stanford University, Stanford, California, USA

First published: 02 May 2019

https://doi.org/10.1002/jmri.26766

Citations: 185

E.G and D.Y are Co-First authors.

D.R and G.Z authors contributed equally to this work.

Share a link

Email
Wechat
Bluesky

Abstract

Background

Detecting and segmenting brain metastases is a tedious and time-consuming task for many radiologists, particularly with the growing use of multisequence 3D imaging.

Purpose

To demonstrate automated detection and segmentation of brain metastases on multisequence MRI using a deep-learning approach based on a fully convolution neural network (CNN).

Study Type

Retrospective.

Population

In all, 156 patients with brain metastases from several primary cancers were included.

Field Strength

1.5T and 3T. [Correction added on May 24, 2019, after first online publication: In the preceding sentence, the first field strength listed was corrected.]

Sequence

Pretherapy MR images included pre- and postgadolinium T₁-weighted 3D fast spin echo (CUBE), postgadolinium T₁-weighted 3D axial IR-prepped FSPGR (BRAVO), and 3D CUBE fluid attenuated inversion recovery (FLAIR).

Assessment

The ground truth was established by manual delineation by two experienced neuroradiologists. CNN training/development was performed using 100 and 5 patients, respectively, with a 2.5D network based on a GoogLeNet architecture. The results were evaluated in 51 patients, equally separated into those with few (1–3), multiple (4–10), and many (>10) lesions.

Statistical Tests

Network performance was evaluated using precision, recall, Dice/F1 score, and receiver operating characteristic (ROC) curve statistics. For an optimal probability threshold, detection and segmentation performance was assessed on a per-metastasis basis. The Wilcoxon rank sum test was used to test the differences between patient subgroups.

Results

The area under the ROC curve (AUC), averaged across all patients, was 0.98 ± 0.04. The AUC in the subgroups was 0.99 ± 0.01, 0.97 ± 0.05, and 0.97 ± 0.03 for patients having 1–3, 4–10, and >10 metastases, respectively. Using an average optimal probability threshold determined by the development set, precision, recall, and Dice score were 0.79 ± 0.20, 0.53 ± 0.22, and 0.79 ± 0.12, respectively. At the same probability threshold, the network showed an average false-positive rate of 8.3/patient (no lesion-size limit) and 3.4/patient (10 mm³ lesion size limit).

Data Conclusion

A deep-learning approach using multisequence MRI can automatically detect and segment brain metastases with high accuracy.

Level of Evidence: 3

Technical Efficacy Stage: 2 J. Magn. Reson. Imaging 2020;51:175–182.

References

1Arvold ND, Lee EQ, Mehta MP, et al. Updates in the management of brain metastases. Neuro Oncol 2016; 18: 1043–1065.
10.1093/neuonc/now127
PubMed Web of Science® Google Scholar
2Nayak L, Lee EQ, Wen PY. Epidemiology of brain metastases. Curr Oncol Rep 2012; 14: 48–54.
10.1007/s11912-011-0203-y
PubMed Web of Science® Google Scholar
3Cagney DN, Martin AM, Catalano PJ, et al. Incidence and prognosis of patients with brain metastases at diagnosis of systemic malignancy: A population-based study. Neuro Oncol 2017; 19: 1511–1521.
10.1093/neuonc/nox077
PubMed Web of Science® Google Scholar
4Fabi A, Felici A, Metro G, et al. Brain metastases from solid tumors: Disease outcome according to type of treatment and therapeutic resources of the treating center. J Exp Clin Cancer Res 2011; 30: 10.
10.1186/1756-9966-30-10
PubMed Web of Science® Google Scholar
5Nussbaum ES, Djalilian HR, Cho KH, Hall WA. Brain metastases: Histology, multiplicity, surgery, and survival. Cancer 1996; 78: 1781–1788.
10.1002/(SICI)1097-0142(19961015)78:8<1781::AID-CNCR19>3.0.CO;2-U
CAS PubMed Web of Science® Google Scholar
6Lin NU, Lee EQ, Aoyama H, et al. Response assessment criteria for brain metastases: Proposal from the RANO group. Lancet Oncol 2015; 16: e270–e278.
10.1016/S1470-2045(15)70057-4
PubMed Web of Science® Google Scholar
7Goceri E, Songul C. Computer-based segmentation, change detection and quantification for lesions in multiple sclerosis. In: IEEE Int Conf Comput Sci Eng; 2017: 177–182.
Google Scholar
8Moghbel M, Mashohor S, Mahmud R, Saripan MI Bin. Review of liver segmentation and computer assisted detection/diagnosis methods in computed tomography. Artif Intell Rev 2017; 1–41.
Web of Science® Google Scholar
9Grossiord E, Talbot H, Passat N, Meignan M, Najman L. Automated 3D lymphoma lesion segmentation from PET/CT characteristics. In: IEEE 14th Int Symp Biomed Imaging; 2017: 174–178.
Google Scholar
10Akkus Z, Galimzianova A, Hoogi A, Rubin DL, Erickson BJ. Deep learning for brain MRI segmentation: State of the art and future directions. J Digit Imaging 2017; 30: 449–459.
10.1007/s10278-017-9983-4
PubMed Web of Science® Google Scholar
11Havaei M, Guizard N, Larochelle H, Jodoin P-M. Deep learning trends for focal brain pathology segmentation in MRI. Mach Learn Heal Inform 2016; 125–148.
Google Scholar
12Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 2016; 35: 1240–1251.
10.1109/TMI.2016.2538465
PubMed Web of Science® Google Scholar
13Zhao L, Jia K. Deep feature learning with discrimination mechanism for brain tumor segmentation and diagnosis. In: IEEE Int Conf Intell Inf Hiding Multimed Signal Process; 2015: 306–309.
Google Scholar
14Kamnitsas K, Ledig C, Newcombe VFJ, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 2017; 36: 61–78.
10.1016/j.media.2016.10.004
PubMed Web of Science® Google Scholar
15Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y. A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Med Image Anal 2018; 43: 98–111.
10.1016/j.media.2017.10.002
PubMed Web of Science® Google Scholar
16Liu Y, Stojadinovic S, Hrycushko B, et al. A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery. PLoS One 2017; 12: e0185844.
10.1371/journal.pone.0185844
PubMed Web of Science® Google Scholar
17Charron O, Lallement A, Jarnet D, Noblet V, Clavier J-B, Meyer P. Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network. Comput Biol Med 2018; 95: 43–54.
10.1016/j.compbiomed.2018.02.004
PubMed Web of Science® Google Scholar
18Liu Y, Stojadinovic S, Hrycushko B, et al. Automatic metastatic brain tumor segmentation for stereotactic radiosurgery applications. Phys Med Biol 2016; 61: 8440–8461.
10.1088/0031-9155/61/24/8440
PubMed Web of Science® Google Scholar
19Smith SM. Fast robust automated brain extraction. Hum Brain Mapp 2002; 17: 143–155.
10.1002/hbm.10062
PubMed Web of Science® Google Scholar
20Szegedy C, Wei Liu, Yangqing Jia, et al. Going deeper with convolutions. In: IEEE Conf Comput Vis Pattern Recognit; 2015: 1–9.
Google Scholar
21Kingma DP, Ba JA. A method for stochastic optimization. arXiv Prepr 2014:arXiv; 1412. 6980.
Google Scholar
22Isýn A, Direkoglu C, Sah M. Review of MRI-based brain tumor image segmentation using deep learning methods. Procedia Comput Sci 2016; 102: 317–324.
10.1016/j.procs.2016.09.407
Google Scholar
23Sunwoo L, Kim YJ, Choi SH, et al. Computer-aided detection of brain metastasis on 3D MR imaging: Observer performance study. PLoS One 2017; 12: e0178265.
10.1371/journal.pone.0178265
PubMed Web of Science® Google Scholar
24He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: IEEE Conf Comput Vis Pattern Recognit; 2016: 770–778.
Google Scholar
25Huang G, Liu Z, Maaten L van der, Weinberger KQ. Densely connected convolutional networks. In: IEEE Conf Comput Vis Pattern Recognit; 2017: 2261–2269.
Google Scholar
26Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Int Conf Med Image Comput Comput Interv; 2015: 234–241.
Google Scholar
27Zhao H, Shi J, Qi X, Wang X, Jia J. Pyramid scene parsing network. In: IEEE Conf Comput Vis Pattern Recognit; 2017: 6230–6239.
Google Scholar
28Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: IEEE Conf Comput Vis Pattern Recognit; 2017: 936–944.
Google Scholar
29Ramkumar A, Dolz J, Kirisli HA, et al. User interaction in semi-automatic segmentation of organs at risk: A case study in radiotherapy. J Digit Imaging 2016; 29: 264–277.
10.1007/s10278-015-9839-8
PubMed Web of Science® Google Scholar
30Park SH, Gao Y, Shi Y, Shen D. Interactive prostate segmentation using atlas-guided semi-supervised learning and adaptive feature selection. Med Phys 2014; 41: 111715.
10.1118/1.4898200
PubMed Web of Science® Google Scholar
31Subedi KS, Takahashi T, Yamano T, et al. Usefulness of double dose contrast-enhanced magnetic resonance imaging for clear delineation of gross tumor volume in stereotactic radiotherapy treatment planning of metastatic brain tumors: A dose comparison study. J Radiat Res 2013; 54: 135–139.
10.1093/jrr/rrs053
PubMed Web of Science® Google Scholar
32Yoon BC, Saad AF, Rezaii P, Wintermark M, Zaharchuk G, Iv M. Evaluation of thick-slab overlapping MIP images of contrast-enhanced 3D T1-weighted CUBE for detection of intracranial metastases: A pilot study for comparison of lesion detection, interpretation time, and sensitivity with nonoverlapping CUBE MIP, CUBE, and inversion-recovery-prepared fast-spoiled gradient recalled brain volume. Am J Neuroradiol 2018; 39: 1635–1642.
CAS PubMed Web of Science® Google Scholar
33Whitfield GA, Price P, Price GJ, Moore CJ. Automated delineation of radiotherapy volumes: Are we going in the right direction? Br J Radiol 2013; 86: 20110718.
10.1259/bjr.20110718
CAS PubMed Web of Science® Google Scholar
34Heckel F, Moltz JH, Tietjen C, Hahn HK. Sketch-based editing tools for tumour segmentation in 3D medical images. Comput Graph Forum 2013; 32: 144–157.
10.1111/cgf.12193
Web of Science® Google Scholar

Citing Literature

Volume51, Issue1

January 2020

Pages 175-182

Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI

Abstract

Background

Purpose

Study Type

Population

Field Strength

Sequence

Assessment

Statistical Tests

Results

Data Conclusion

References

Citing Literature

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI

Abstract

Background

Purpose

Study Type

Population

Field Strength

Sequence

Assessment

Statistical Tests

Results

Data Conclusion

References

Citing Literature

References

Related

Information