Chapter 5
Evaluation of Cardiac Structure Segmentation in Cine Magnetic Resonance Imaging
Alain Lalande,
Mireille Garreau,
Frédérique Frouin,
GRIC-MediEval,
Alain Lalande
Search for more papers by this authorMireille Garreau
Search for more papers by this authorFrédérique Frouin
Search for more papers by this authorGRIC-MediEval
Search for more papers by this authorAlain Lalande,
Mireille Garreau,
Frédérique Frouin,
GRIC-MediEval,
Alain Lalande
Search for more papers by this authorMireille Garreau
Search for more papers by this authorFrédérique Frouin
Search for more papers by this authorGRIC-MediEval
Search for more papers by this authorBook Editor(s):Patrick Clarysse,
Denis Friboulet,
Patrick Clarysse
Search for more papers by this authorDenis Friboulet
Search for more papers by this authorSummary
This chapter discusses the evaluation of the segmentation of the heart from images obtained from magnetic resonance imaging (MRI). It focuses on works achieved with cine MRI-based dynamic images (gradient echo-based and in particular steady-state free precession (SSFP)-based sequences). The use of cine-MRI sequences is today the reference technique for studying the cardiac function. The use of kinetic sequences facilitates obtaining multiple images covering the cardiac cycle for each slice. The chapter reviews the empirical methods of segmentation evaluation. These are divided into three groups: visual evaluation methods, supervised methods and unsupervised methods. In each of the groups, the advantages and limitations of each method have been detailed, highlighting their application to the evaluation of the segmentation of cardiac structures from MRI. The chapter concludes with a discussion on two specific workgroups: Initiative Multicentrique pour une Plateforme d’Évaluation en Imagerie Cardiaque (IMPEIC) and Medical Image Segmentation Evaluation (MEDIEVAL).
Bibliography
- ALBEROLA-LOPEZ C., MARTIN-FERNANDEZ M., RUIZ-ALZOLA J., “Comments on: a methodology for evaluation of boundary detection algorithms on medical images”, IEEE Transactions on Medical Imaging, vol. 23, no. 5, pp. 658–660, 2004.
- ALTMAN D., BLAND J., “Measurement in medicine: the analysis of method comparison studies”, Statistician, vol. 32, no. 3, pp. 307–317, 1983.
- ASPERT N., SANTA-CRUZ D., EBRAHIMI T., “ MESH: measuring errors between surfaces using the Hausdorff distance”, Proceedings of IEEE International Conference on Multimedia and Expo, vol. 1, pp. 705–708, 2002.
- BEAUCHEMIN M., THOMSON K., EDWARDS G., “On the Hausdorff distance used for the evaluation of segmentation results”, Canadian Journal of Remote Sensing, vol. 24, no. 1, pp. 3–8, 1998.
10.1080/07038992.1998.10874685 Google Scholar
- BLAND J., ALTMAN D., “Statistical methods for assessing agreement between two methods of clinical measurement”, Lancet, vol. 1, no. 8476, pp. 307–3010, 1986.
- BORSOTTI M., CAMPADELLI P., SCHETTINI R., “Quantitative evaluation of color image segmentation results”, Pattern Recognition Letters, vol. 19, pp. 741–747, 1998.
- BOWYER K., KRANENBURG C., DOUGHERTY S., “Edge detector evaluation using empirical ROC curves”, Computer Vision and Image Understanding, vol. 84, no. 1, pp. 77–103, 2001.
- BRYANT D., BOULDING D., “ Evaluation of edge operators using relative and absolute grading”, Proceedings of the IEEE PRIP, pp. 138–145, 1979.
- BUVAT I., ALAOUI J., SORET M., “Comparison of estimation methods without a gold standard”, Journal of Nuclear Medicine, vol. 47, pp. 116, 2006.
- CAUDRON J., FARES J., LEFEBVRE V. et al., “Diagnostic accuracy and variability of three semi-quantitative methods for assessing right ventricular systolic function from cardiac MRI in patients with acquired heart disease”, European Radiology, vol. 21, no. 10, pp. 2111–2120, 2011.
- CAUDRON J., FARES J., LEFEBVRE V. et al., “Cardiac MRI assessment of right ventricular function in acquired heart disease: factors of variability”, Academic Radiology, vol. 19, no. 8, pp. 991–1002, 2012.
- CERQUEIRA M., WEISSMAN N., DILSIZIAN V. et al., “Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart – a statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association”, Circulation, vol. 105, no. 4, pp. 539–542, 2002.
- CHALANA V., KIM Y., “A methodology for evaluation of boundary detection algorithms on medical images”, IEEE Transactions on Medical Imaging, vol. 16, no. 5, pp. 642–652, 1997.
- CHABRIER S., Contribution à l'évaluation de performances en segmentation d'images, Thesis, University of Orléans, 2005.
- CHENOUNE Y., DELECHELLE E., PETITE E. et al., “Cine-MR image segmentation and myocardial deformations assessment using level set methods”,Computerized Medical Imaging and Graphics, vol. 29, no. 8, pp. 607–616, 2005.
- CLARYSSE P., FROUIN F., GARREAU M. et al., “Intégration de connaissances et modélisation en imagerie médicale”, ITBM-RBM, vol. 25, no. 3, pp. 139–149, 2004.
10.1016/j.rbmret.2004.07.002 Google Scholar
- COCQUEREZ J., PHILIPP S., Analyse d'Images: filtrage et segmentation, Masson, Paris, 1995.
- COHEN J., “A coefficient of agreement for nominal scales”, Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960.
- COMMOWICK O., WARFIELD S., “Incorporating priors on expert performance parameters for segmentation validation and label fusion: a maximum a posteriori STAPLE”, Medical Image Computing and Computer Assisted Interventions, vol. 13, pp. 25–32, 2010.
- CONSTANTINIDES C., CHENOUNE Y., KACHENOURA N. et al., “Semi-automated cardiac segmentation on cine magnetic resonance images using GVF-snake deformable models”, MIDAS Journal, 2009.
10.54294/ub0ptg Google Scholar
- CONSTANTINIDES C., CHENOUNE Y., MOUSSEAUX E. et al., “ Automated heart localization for the segmentation of the ventricular cavities on cine magnetic resonance image”, Proceedings of the Computers in Cardiology, pp. 911–914, 2010.
- CORREIA P., PEREIRA F., “ Objective evaluation of relative segmentation quality”, Proceedings of the IEEE ICIP, pp. 308–311, 2000.
- COUSTY J., NAJMAN L., COUPRIE M. et al., “Segmentation of 4D cardiac MRI: automated method based on spatio-temporal watershed cuts”, Image and Vision Computing, vol. 28, pp. 1229–1243, 2010.
- DAUM D., BUESSMER J.-L., URBAN J.-P., “ Evaluation de segmentations d'images: mesures de similarité avec une référence partielle”, Proceedings of the ORASIS, 2009.
- DETMER P., BASHEIN G., BASHEIN G., “Matched filter identification of left-ventricular endocardial borders in transesophageal echocardiograms”, IEEE Transactions on Medical Imaging, vol. 9, no. 4, pp. 396–404, 1990.
- DICE L., “Measures of the amount of ecologic association between species”, Ecology, vol. 26, no. 3, pp. 297–302, 1945.
- DULCE M., MOSTBECK G., FRIESE K. et al., “Quantification of the left ventricular volumes and function with cine MR imaging: comparison of geometric models with three-dimensional data”, Radiology, vol. 188, no. 2, pp. 371–376, 1993.
- DUQUETTE A., JODOIN P.-M., BOUCHOT O. et al., “3D segmentation of abdominal aorta from CT-scan and MR Images”, Computerized Medical Imaging and Graphics, vol. 36, no. 4, pp. 294–303, 2012.
- EVERINGHAM M., MULLER H., THOMAS B., “ Algorithm evaluation by probabilistic fitness/cost analysis and application to image segmentation”, Proceedings of the ACCV, pp. 580–586, 2002.
- FIENO D., JAFFE W., SIMONETTI O. et al., “TrueFISP: assessment of accuracy for measurement of left ventricular mass in an animal model”, Journal of Magnetic Resonance Imaging, vol. 15, no. 5, pp. 526–531, 2002.
- FRANÇOIS C., FIENO D., SHORS S. et al., “Left ventricular mass: manual and automatic segmentation of true FISP and FLASH cine MR images in dogs and pigs”, Radiology, vol. 230, no. 2, pp. 389–395, 2004.
- FROUIN F., GARREAU M., BUVAT I. et al., “ Méthodologie pour comparer différentes méthodes d'extraction de biomarqueurs sans méthode de référence. Application à la segmentation du ventricule gauche en IRM cardiaque pour estimer la fraction d'éjection”, Proceedings of the RITS, 2011.
- GOUMEIDANE A., KHAMADJE M., BELAROUSSI B. et al., “ New discrepancy measures for segmentation evaluation”, Proceedings of the IEEE ICIP, pp. 411–414, 2003.
- GUDBJARTSSON H., PATZ S., “The Rician distribution of noisy MRI data”, Magnetic Resonance in Medicine, vol. 34, no. 6, pp. 910–914, 1995.
- HAMER A., TAKAYAMA M., ABRAHAM K. et al., “End-systolic volume and long-term survival after coronary artery bypass graft surgery in patients with impaired left ventricular function”, Circulation, vol. 90, no. 6, pp. 2899–2904, 1994.
- HARALICK R., SHAPIRO L., “Image segmentation techniques”, Computer Vision Graph, vol. 29, no. 1, pp. 100–132, 1985.
- HOOVER A., JEAN-BAPTISTE G., JIANG X. et al., “An experimental comparison of range image segmentation algorithms”, IEEE Transactions on Pattern Analysis, vol. 18, no. 7, pp. 673–689, 1996.
- HOPPIN J., KUPINSKI M., KASTI G. et al., “Objective comparison of quantitative imaging modalities without the use of a gold standard”, IEEE Transactions on Medical Imaging, vol. 21, no. 5, pp. 441–449, 2002.
- HUNDLEY W., BLUEMKE D., FINN J. et al., “ACCF/ACR/AHA/NASCI/SCMR 2010 expert consensus document on cardiovascular magnetic resonance: a report of the American College of Cardiology Foundation Task Force on Expert Consensus Documents”, Journal of the American College of Cardiology, vol. 55, no. 3, pp. 2614–2662, 2010.
- HUTTENLOCHER D., KLANDERMAN G., RUCKLIDGE W., “Comparing images using the Hausdorff distance”, IEEE Transactions on Pattern Analysis, vol. 15, no. 9, pp. 850–863, 1993.
- JEHAN-BESSON S., TILMANT C., DE CESARE A. et al., “ Estimation d'une forme mutuelle pour l'évaluation de la segmentation en imagerie cardiaque”,Proceedings of the GRETSI, 2011.
- KISSI A., TILMANT C., DE CESARE A. et al., “ Initiative multicentrique pour une plateforme d'évaluation en imagerie cardiaque”, Proceedings of the RITS, 2009.
- KITCHEN L., ROSENFELD A., “Scene analysis using region-based constraint filtering”, Pattern Recognition, vol. 17, no. 2, pp. 189–203, 1984.
- KUPINSKI M., HOPPIN J., KRASNOW J. et al., “Comparing cardiac ejection fraction estimation algorithms without a gold standard”, Academic Radiology, vol. 13, no. 3, pp. 329–337, 2006.
- LALANDE A., SALVE N., COMTE A. et al., “Left ventricular ejection fraction calculation from automatically selected and processed diastolic and systolic frames in short-axis cine-MRI”, Journal of Cardiovascular Magnetic Resonance, vol. 6, no. 4, pp. 817–827, 2004.
- LANDIS J., KOCH G., “The measurement of observer agreement for categorical data”, Biometrics, vol. 33, no. 1, pp. 159–174, 1977.
- LEBENBERG J., BUVAT I., LALANDE A. et al., “Non-supervised ranking of different segmentation approaches. Application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences”, IEEE Transactions of Medical Imaging, vol. 31, no. 8, pp. 1651–1660, 2012.
- LEVINE M., NAZIF A., “Dynamic measurement of computer generated image segmentations”, IEEE Transactions on Pattern Analysis, vol. 7, no. 2, pp. 155–164, 1985.
- LIU J., YANG Y.-H., “Multiresolution color image segmentation”, IEEE Transactions on Pattern Analysis, vol. 16, no. 7, pp. 698–700, 1994.
- LÖTJÖNEN J., JÄRVINEN V., CHEONG B. et al., “Evaluation of cardiac biventricular segmentation from multiaxis MRI data: a multicenter study”, Journal of Magnetic Resonance Imaging, vol. 28, no. 31, pp. 626–636, 2008.
- MAHNKEN A., MÜHLENBRUCH G., KOOS R. et al., “Automated vs. manual assessment of left ventricular function in cardiac multidetector row computed tomography: comparison with magnetic resonance imaging”, European Radiology, vol. 16, no. 7, pp. 1416–1423, 2006.
- MARTIN D., FOWLKES C., TAL D. et al., “ A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics”, Proceedings of the ICCV, pp. 416–423, 2001.
10.1109/ICCV.2001.937655 Google Scholar
- METZ C., “ROC methodology in radiologic imaging”, Investigative Radiology, vol. 21, no. 9, pp. 720–733, 1986.
- METZ C., “ROC analysis in medical imaging: a tutorial review of the literature”, Radiological Physics and Technology, vol. 1, no. 1, pp. 2–12, 2008.
- ODET C., BELAROUSSI B., BENOIT-CATTIN H., “ Scalable discrepancy measures for segmentation evaluation”, Proceedings of the IEEE ICIP, pp. 785–788, 2002.
- ORTIZ A., OLIVER G., “On the use of the overlapping area matrix for image segmentation evaluation: a survey and new performance measures”, Pattern Recognition Letters, vol. 27, no. 16, pp. 916–926, 2006.
- PATTYNAMA P., LAMB H., VAN DER VELDE E. et al., “Left ventricular measurements with cine and spin-echo MR imaging: a study of reproducibility with variance component analysis”, Radiology, vol. 187, no. 1, pp. 261–268, 1993.
- PETITJEAN C., DACHER J.-N., “A review of segmentation methods in short axis cardiac MR images”, Medical Image Analysis, vol. 15, no. 2, pp. 169–184, 2011.
- PETITJEAN C., RUAN S., GROSGEORGE D. et al., “Right ventricle segmentation in cardiac MRI: a MICCAI'12 challenge”, MIDAS Journal, 2012.
- PHILIPP-FOLIGUET S., GUIGUES L., “Evaluation de la segmentation: état de l'article, nouveaux indices et comparaison”, Traitement du Signal, vol. 23, no. 2, pp. 109–125, 2006.
- PRATT W., FAUGERAS O., GAGALOWICZ A., “Visual discrimination of stochastic texture fields”, IEEE Transactions on Systems, Man, and Cybernetics, vol. 8, no. 11, pp. 796–804, 1978.
- RADAU P., LU Y., CONNELLY G. et al., “Evaluation framework for algorithms segmenting short axis cardiac MRI”, MIDAS Journal, 2009.
10.54294/g80ruo Google Scholar
- ROSENBERGER C., Mise en oeuvre d'un système adaptatif de segmentation d'images, PhD Thesis, University of Rennes 1, 1999.
- SAHOO P., SOLTANI S., WONG A. et al., “A survey of thresholding techniques”, Computer Vision, Graphics, and Image Processing, vol. 41, pp. 233–260, 1988.
10.1016/0734-189X(88)90022-9 Google Scholar
- SCHAERER J., CASTA C., POUSIN J. et al., “A dynamic elastic model for segmentation and tracking of the heart in MR image sequences”, Medical Image Analysis, vol. 14, no. 6, pp. 738–749, 2010.
- SIEVERS B., KIRCHBERG S., BAKAN A. et al., “Impact of papillary muscles in ventricular volume and ejection fraction assessment by cardiovascular magnetic resonance”, Journal of Cardiovascular Magnetic Resonance, vol. 6, no. 1, pp. 9–16, 2004.
- SIRI F., JELICKS L., LEINWAND L. et al., “Gated magnetic resonance imaging of normal and hypertrophied murine hearts”, American Journal of Physiology, vol. 272, no. 5, pp. 2394–2402, 1997.
- SORET M., ALAOUI J., KOULIBALY P. et al., “Accuracy of partial volume effect correction in clinical molecular imaging of dopamine transporter using SPECT”, Nuclear Instruments and Methods in Physics Research Section A, vol. 571, pp. 173–176, 2007.
- SPINU C., Une approche multi-agents pour la segmentation d'images associant estimation et évaluation, Thesis, Grenoble, 1978.
- SUINESIAPUTRA A., COWAN B., FINN J. et al., “ Left ventricular segmentation challenge from cardiac MRI: a collation study”, Proceedings of the STACOM, 2011.
- VIGNON N., COCHET A., LALANDE A. et al., “ Influence of the interslice gap on the evaluation of right and left ventricular ejection fraction by cardiac magnetic resonance imaging”, Proceedings of the ESMRMB, 2008.
- VINET L., Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, Thesis, Paris IX Dauphine, 1991.
- WARFIELD S.K., ZHOU K.H., WELLS W.M., “Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation”, IEEE Transactions on Medical Imaging, vol. 23, no. 7, pp. 903–921, 2004.
- WESZKA J., ROSENFELD A., “Threshold evaluation techniques”, IEEE Transactions on Systems, Man, and Cybernetics, vol. 8, no. 8, pp. 622–629, 1978.
- WHITE H., NORRIS R., BROWN M. et al., “Left ventricular end-systolic volume as the major determinant of survival after recovery from myocardial infarction”, Circulation, vol. 76, no. 1, pp. 44–51, 1987.
- Williams G., “Comparing the joint agreement of several raters with another rater”, Biometrics, vol. 32, no. 3, pp. 619–627, 1976.
- YANG L., ALBREGTSEN F., LONNESTAD T. et al., “A supervised approach to the evaluation of image segmentation methods”, CAIP, vol. 970, pp. 759–765, 1995.
- YASNOFF W., MUI J., BACUS J., “Error measures for scene segmentation”, Pattern Recognition, vol. 9, pp. 217–231, 1977.
- ZEBOUDJ R., Filtrage, seuillage automatique, contraste et contours: du prétraitement à l'analyse d'image, PhD Thesis, University of Saint-Etienne, 1988.
- ZHANG Y., “A survey on evaluation methods for image segmentation”, Pattern Recognition, vol. 29, no. 8, pp. 1335–1346, 1996.
