Cloud-Based Evaluation of Anatomical Structure Segmentation and Landmark Detection Algorithms: VISCERAL Anatomy Benchmarks

O. Jimenez-Del-Toro, H. Muller, M. Krenn, K. Gruenberg, A. Aziz Taha, M. Winterstein, I. Eggel, A. Foncubierta-Rodriguez, O. Goksel, A. Jakab, G. Kontokotsios, G. Langs, B. Menze, T. Salas Fernandez, R. Schaer, A. Walleyo, M. -Andre Weber, Y. Dicente Cid, T. Gass, M. Heinrich, F. Jia, F. Kahl, R. Kechichian, Dominic Mai, A. Spanier, G. Vincent, C. Wang, D. Wyeth, A. Hanbury
IEEE Transactions on Medical Imaging, Jun 2016
Abstract: Variations in the shape and appearance of anatom-ical structures in medical images are often relevant radiological signs of disease. Automatic tools can help automate parts of thismanual process. A cloudbased evaluation framework is presented in this paper including results of benchmarking current state-of-the-art medical imaging algorithms foranatomical structure segmentation and landmark detection: the VISCERAL Anatomy benchmarks. The algorithms are implemented in virtual machines in the cloud where participants can only access the training data and can be run privately by thebenchmark administrators to objectively compare their performance in an unseen common test set. Overall, 120 computed tomography and magnetic resonance patient volumes were manually annotated to create a standard Gold Corpus containing a total of 1295 struc-tures and 1760 landmarks. Ten participants contributed with automatic algorithms for the organ segmentation task, and three for the landmark localization task. Different algorithms obtained the best scores in the four available imaging modalities and for subsets of anatomical structures. The annotation framework, resulting data set, evaluation setup, results and performance analysis from the three VISCERAL Anatomy benchmarks are presented in this article. Both the VISCERAL data set and Silver Corpus generated with the fusion of the participant algorithms on a larger set of non-manually-annotated medical images are available to the research community.
Publisher's link

Other associated files : 2016_visceral_anatomy_challenge.pdf [9.2MB]  

Images and movies


BibTex reference

  author       = "O.Jimenez-Del-Toro and H.Muller and M.Krenn and K.Gruenberg and A.Aziz Taha and M.Winterstein and I.Eggel and A.Foncubierta-Rodriguez and O.Goksel and A.Jakab and G.Kontokotsios and G.Langs and B.Menze and T.Salas Fernandez and R.Schaer and A.Walleyo and M.-Andre Weber and Y.Dicente Cid and T.Gass and M.Heinrich and F.Jia and F.Kahl and R.Kechichian and D.Mai and A.Spanier and G.Vincent and C.Wang and D.Wyeth and A.Hanbury",
  title        = "Cloud-Based Evaluation of Anatomical Structure Segmentation and Landmark Detection Algorithms: VISCERAL Anatomy Benchmarks",
  journal      = "IEEE Transactions on Medical Imaging",
  month        = "Jun",
  year         = "2016",
  url          = "http://lmb.informatik.uni-freiburg.de/Publications/2016/Mai16"

Other publications in the database