A generic deep learning model for reduced gadolinium dose in contrast-enhanced brain MRI
Corresponding Author
Srivathsa Pasumarthi
Subtle Medical Inc., Menlo Park, CA, USA
Correspondence
Srivathsa Pasumarthi, Subtle Medical Inc., 883 Santa Cruz Avenue, Suite 205, Menlo Park, CA 94025, USA.
Email: [email protected]
Twitter: @SubtleMedical
Search for more papers by this authorJonathan I. Tamir
Subtle Medical Inc., Menlo Park, CA, USA
Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
Search for more papers by this authorGreg Zaharchuk
Department of Radiology, Stanford University, Stanford, CA, USA
Search for more papers by this authorCorresponding Author
Srivathsa Pasumarthi
Subtle Medical Inc., Menlo Park, CA, USA
Correspondence
Srivathsa Pasumarthi, Subtle Medical Inc., 883 Santa Cruz Avenue, Suite 205, Menlo Park, CA 94025, USA.
Email: [email protected]
Twitter: @SubtleMedical
Search for more papers by this authorJonathan I. Tamir
Subtle Medical Inc., Menlo Park, CA, USA
Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
Search for more papers by this authorGreg Zaharchuk
Department of Radiology, Stanford University, Stanford, CA, USA
Search for more papers by this authorAbstract
Purpose
With rising safety concerns over the use of gadolinium-based contrast agents (GBCAs) in contrast-enhanced MRI, there is a need for dose reduction while maintaining diagnostic capability. This work proposes comprehensive technical solutions for a deep learning (DL) model that predicts contrast-enhanced images of the brain with approximately 10% of the standard dose, across different sites and scanners.
Methods
The proposed DL model consists of a set of methods that improve the model robustness and generalizability. The steps include multi-planar reconstruction, 2.5D model, enhancement-weighted L1, perceptual, and adversarial losses. The proposed model predicts contrast-enhanced images from corresponding pre-contrast and low-dose images. With IRB approval and informed consent, 640 heterogeneous patient scans (56 train, 13 validation, and 571 test) from 3 institutions consisting of 3D T1-weighted brain images were used. Quantitative metrics were computed and 50 randomly sampled test cases were evaluated by 2 board-certified radiologists. Quantitative tumor segmentation was performed on cases with abnormal enhancements. Ablation study was performed for systematic evaluation of proposed technical solutions.
Results
The average peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) between full-dose and model prediction were 
dB and 
, respectively. Radiologists found the same enhancing pattern in 45/50 (90%) cases; discrepancies were minor differences in contrast intensity and artifacts, with no effect on diagnosis. The average segmentation Dice score between full-dose and synthesized images was 
(median = 0.91).
Conclusions
We have proposed a DL model with technical solutions for low-dose contrast-enhanced brain MRI with potential generalizability under diverse clinical settings.
Supporting Information
| Filename | Description |
|---|---|
| mrm28808-sup-0001-Supplementary.pdfapplication/PDF, 1.5 MB |
TABLE S1 Individual ablation study quantitative metrics for the randomly sampled 50 cases. FIGURE S1 Comparison of the proposed DL network with the naive linear model (LM). The LM result was obtained by taking the pixel-wise difference between the pre-contrast and the low-dose images, scaling it by a factor of 10, and adding it back to the pre-contrast image. It can be seen that the LM results are noisy, and the enhancement pattern is inaccurate when compared to the DL results, which is also confirmed by the quantitative metrics with respect to the full-dose ground truth. FIGURE S2 The patch discriminator with a series of spectral normalized convolution layers + batch normalization with a final sigmoid layer to predict a 32 × 32 patch of whether the input is real or fake. FIGURE S3 Effect of the number of rotation angles in MPR. More number of angles reduced the horizontal streaks inside the tumor, while it also increased the inference time. The experiments were run on a single GeForce RTX 2080 (16 GB) GPU. |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
REFERENCES
- 1Ellingson BM, Harris RJ, Woodworth DC, et al. Baseline pretreatment contrast enhancing tumor volume including central necrosis is a prognostic factor in recurrent glioblastoma: evidence from single and multicenter trials. Neuro-Oncology. 2016; 19: 89-98.
- 2Koudriavtseva T, Pozzilli C, Gasperini C, et al. High-dose contrast-enhanced MRI in multiple sclerosis. Neuroradiology. 1996; 38(S1): S5-S9.
- 3Montagne A, Nation DA, Pa J, Sweeney MD, Toga AW, Zlokovic BV. Brain imaging of neurovascular dysfunction in Alzheimer’s disease. Acta Neuropathol. 2016; 131: 687-707.
- 4O’Connor JPB, Tofts PS, Miles KA, Parkes LM, Thompson G, Jackson A. Dynamic contrast-enhanced imaging techniques: CT and MRI. Brit J Radiol. 2011; 84(Special Issue 2): S112-S120.
- 5Restrepo CS, Tavakoli S, Marmol-Velez A. Contrast-enhanced cardiac magnetic resonance imaging. Magn Reson Imaging C. 2016; 20: 739-760.
- 6Inan N, Arslan A, Donmez M, Sarisoy HT. Diagnostic accuracy of dynamic contrast enhanced magnetic resonance imaging in characterizing lung masses. Iran J Radiol. 2016; 13.
- 7Niendorf E, Spilseth B, Wang X, Taylor A. Contrast enhanced MRI in the diagnosis of HCC. Diagnostics (Basel). 2015; 5: 383-398.
- 8Shah LM, Salzman KL. Imaging of spinal metastatic disease. Int J Surg Oncol. 2011; 2011: 1-12.
10.1155/2011/769753 Google Scholar
- 9Grobner T. Gadolinium-a specific trigger for the development of nephrogenic fibrosing dermopathy and nephrogenic systemic fibrosis? Nephrol Dial Transpl. 2006; 21: 1104-1108.
- 10McDonald JS, Hunt CH, Kolbe AB, et al. Acute adverse events following gadolinium-based contrast agent administration: a single-center retrospective study of 281,945 injections. Radiology. 2019; 292: 620-627.
- 11Kanda T, Ishii K, Kawaguchi H, Kitajima K, Takenaka D. High signal intensity in the dentate nucleus and globus pallidus on unenhanced
-weighted MR images: relationship with increasing cumulative dose of a gadolinium-based contrast material. Radiology. 2014; 270: 834-841.
- 12McDonald RJ, Levine D, Weinreb J, et al. Gadolinium retention: a research roadmap from the 2018 NIH/ACR/RSNA workshop on gadolinium chelates. Radiology. 2018; 289: 517-534.
- 13 FDA Drug Safety Communication: FDA warns that gadolinium-based contrast agents (GBCAs) are retained in the body; requires new class warnings. U.S. FDA website. https://www.fda.gov/drugs/drug-safety-and-avai lability/fda-drug-safety-communication-fda-warns-gadolinium-based-c ontrast-agents-gbcas-are-retained-body, Published December 19, 2017. Updated May 16, 2018. Accessed June 24, 2020.
- 14 Summary Safety Review—gadolinium based contrast agents—assessing the risk of gadolinium build-up in the brain and potential brain and nervous system (neurological) side effects. Government of Canada website. https://www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/safety-reviews/gadolinium-based-contrast-agents-assessing-risk-build-up-in-brain-and-potential-brain-nervous-system-side-effects.html, Published May 1, 2018. Updated May 1, 2018. Accessed June 24, 2020.
- 15 Gadolinium-based contrast agents for MRI scans: safety advisory—potential retention in the brain but no known adverse effects. Department of Health: Government of Australia website. https://www.tga.gov.au/alert/gadolinium-based-contrast-agents-mri-scans, Published July 28, 2017. Updated July 28, 2017. Accessed June 24, 2020.
- 16 EMA/625317/2017: EMA’s final opinion confirms restrictions on use of linear gadolinium agents in body scans. European Medicines Agency websiteEMA/625317/2017: EMA’s final opinion confirms restrictions on use of linear gadolinium agents in body scans. European Medicines Agency website.
- 17Rogowska J, Olkowska E, Ratajczyk W, Wolska L. Gadolinium as a new emerging contaminant of aquatic environments. Environ Toxicol Chem. 2018; 37: 1523-1534.
- 18Inoue K, Fukushi M, Furukawa A, et al. Impact on gadolinium anomaly in river waters in Tokyo related to the increased number of MRI devices in use. Mar Pollut Bull. 2020; 154: 111-148.
- 19Schieda N, Maralani PJ, Hurrell C, Tsampalieros AK, Hiremath S. Updated clinical practice guideline on use of gadolinium-based contrast agents in kidney disease issued by the canadian association of radiologists. Can Assoc Radiol J. 2019; 70: 226-232.
- 20Shankar PR, Parikh K, Davenport MS. Financial implications of revised ACR guidelines for estimated glomerular filtration rate testing before contrast-enhanced MRI. J Am Coll Radiol. 2018; 15: 250-257.
- 21Falk Delgado A, Van Westen D, Nilsson M, et al. Diagnostic value of alternative techniques to gadolinium-based contrast agents in MR neuroimaging–a comprehensive overview. Insights Imaging. 2019; 10: 84.
- 22Wesolowski JR, Kaiser A. Alternatives to GBCA: are we there yet? Top Magn Reson Imaging. 2016; 25: 171-175.
- 23McDonald RJ, Kallmes DF. Signal intensity changes at MRI following GBCA exposure: incidental finding or cause for concern? Radiology. 2020; 201256.
- 24Gondara L. Medical image denoising using convolutional denoising autoencoders. IEEE 16th International Conference on Data Mining Workshops (ICDMW). 2016.
10.1109/ICDMW.2016.0041 Google Scholar
- 25Jifara W, Jiang F, Rho S, Cheng M, Liu S. Medical image denoising using convolutional neural network: a residual learning approach. J Supercomput. 2019; 75: 704-718.
- 26Ameen M, Ahmed SA. An extensive review of medical image denoising techniques. J Med Res. 2017; 16: 23-27.
- 27Sood R, Topiwala B, Choutagunta K, Sood R, Rusu M. An application of generative adversarial networks for super resolution medical imaging. In: Proceedings of the 17th IEEE International Conference on Machine Learning Applications (ICMLA). 2018: 326-331.
- 28Pham C-H, Tor-Díez C, Meunier H, et al. Multiscale brain MRI super-resolution using deep 3D convolutional networks. Comput Med Imaging Graph. 2019; 77.
- 29Zhang S, Liang G, Pan S, Zheng L. A fast medical image super resolution method based on deep learning network. IEEE Access. 2019; 7: 12319-12327.
- 30Wolterink JM, Dinkla AM, Savenije MHF, Seevinck PR, van den Berg CAT, Išgum I. Deep MR to CT synthesis using unpaired data. Lect Notes Comput Sci. 2017; 10557: 14-23.
10.1007/978-3-319-68127-6_2 Google Scholar
- 31Wu K, Qiang Y, Song K, et al. Image synthesis in contrast MRI based on super resolution reconstruction with multi-refinement cycle-consistent generative adversarial networks. J Intell Manuf. 2019; 1: 1-14.
- 32Kleesiek J, Morshuis JN, Isensee F, et al. Can virtual contrast enhancement in brain MRI replace gadolinium? A feasibility study. Invest Radiol. 2019; 54: 653-660.
- 33Narayana PA, Coronado I, Sujit SJ, Wolinsky JS, Lublin FD, Gabr RE. Deep learning for predicting enhancing lesions in multiple sclerosis from noncontrast MRI. Radiology. 2019; 294: 398-404.
- 34Sun H, Liu X, Feng X, et al. Substituting gadolinium in brain MRI using deepcontrast. http://arxiv.org/abs/2020arXiv.2020; preprint:2001.05551.
- 35Gong E, Pauly JM, Wintermark M, Zaharchuk G. Deep learning enables reduced gadolinium dose for contrast-enhanced brain MRI. J Magn Reson Imaing. 2018; 48: 330-340.
- 36 Brain image processing tools using Deep Learning focused on speed and accuracy. GitHub: Ivan Itzkovich. https://github.com/iitzco/deepbrain, Published August 17, 2018. Updated September 24, 2018. Accessed June 24, 2020.
- 37Marstal K, Berendsen F, Staring M, Klein S. SimpleElastix: a user-friendly, multi-lingual library for medical image registration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2016; 574-582.
- 38Nyul LG, Udupa JK, Zhang X. New variants of a method of MRI scale standardization. IEEE T Med Imaging. 2000; 19: 143-150.
- 39Carré A, Klausner G, Edjlali M, et al. Standardization of brain MR images across machines and protocols: bridging the gap for MRI-based radiomics. Sci Rep. 2020; 10: 12340.
- 40Zhao H, Gallo O, Frosio I, Kautz J. Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging. 2017; 3: 47-57.
- 41Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein KH. Brain tumor segmentation and radiomics survival prediction: contribution to the BRATS 2017 challenge. Lect Notes Comput Sci. 2018; 10670: 287-297.
10.1007/978-3-319-75238-9_25 Google Scholar
- 42Vahid G, Jiaxin S, Mark B, et al. MR image reconstruction using deep learning: evaluation of network structure and loss functions. Quant Imaging Med Surg. 2019; 9: 1516-1527.
- 43Klages P, Benslimane I, Riyahi S, et al. Comparison of patch-based conditional generative adversarial neural net models with emphasis on model robustness for use in head and neck cases for MR-only planning. arXiv.2019; preprint:1902.00536.
- 44Xue Y, Farhat FG, Boukrina O, Xue Y, Farhat FG, Boukrina O. A multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI imagesA multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI images.
- 45Angermann C, Haltmeier M, Steiger R. Gizewski E. Projection-based 2.5D U-net architecture for fast volumetric segmentation. arXiv.2019; preprint:1902.00347.
- 46Adria C, Marcel C, Irina S, Marc C, Veronica V. Cascaded V-net using ROI masks for brain tumor segmentation. International MICCAI Brainlesion Workshop. 2017; 381-391.
- 47Karen S, Andrew Z. Very deep convolutional networks for large-scale image recognition. 2015. In: Proceedings of the International Conference on Learning Representations (ICLR).
- 48Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE Computer Vision and Pattern Recognition (CVPR). 2009.
- 49Johnson J, Alahi A, Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution. arXiv.2016; preprint:1603.08155.
- 50Takeru M, Toshiki K, Masanori K, Yuichi Y. Spectral normalization for generative adversarial networks. 2018. In: Proceedings of the International Conference on Learning Representations (ICLR).
- 51Jenni S, Favaro P, Jenni S, Favaro P. On stabilizing generative adversarial training with noiseOn stabilizing generative adversarial training with noise. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 12137-12145.
- 52Seitzer M, Yang G, Schlemper J, Oktay O. Adversarial and perceptual refinement for compressed sensing MRI reconstruction. arXiv 2018; 1806.11216.
- 53Myronenko A. 3D MRI brain tumor segmentation using autoencoder regularization. arXiv.2018; preprint:1810.11654.
- 54Bakas S, Reyes M, Jakab A, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv.2018; preprint:1811.02629.
- 55 Nvidia GPU Cloud (NGC). NVIDIA corporation website. https://www.nvidia.com/en-us/gpu-cloud/, Accessed June 24, 2020.
- 56Rohlfing T, Zahr NM, Sullivan EV, Pfefferbaum A. The SRI24 multichannel atlas of normal adult human brain structure. Hum Brain Mapp. 2009; 31: 798-819.
- 57Yang S, Nam Y, Kim M-O, Kim EY, Park J, Ki DH. Computer-aided detection of metastatic brain tumors using magnetic resonance black-blood imaging. Invest Radiol. 2013; 48: 113-119.
- 58Pianykh OS, Pospelova K, Kamboj NH. Modeling human perception of image quality. J Digit Imaging. 2018; 31: 768-775.
- 59Zhang Y, Chen C, Tian Z, Feng R, Cheng Y, Xu J. The diagnostic value of MRI-based texture analysis in discrimination of tumors located in posterior fossa: A preliminary study. Front Neurosci-Switz. 2019; 13: Article 1113.
- 60Zacharaki EI, Wang S, Chawla S, et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn Reson Med. 2009; 62: 1609-1618.
- 61Barron JT. A general and adaptive Robust loss function. arXiv. 2017; preprint:1701.03077.
