1. From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge
- Author
-
Jörg Franke, Keisuke Fukuta, Hao Chen, Willem Vreuls, Aoxiao Zhong, Farhad Ghazvinian Zanjani, Svitlana Zinger, Richard J. Chen, Hunter Jackson, Fabian Both, Heidi V.N. Küsters-Vandevelde, Daisuke Komura, Babak Ehteshami Bejnordi, Marcory C. R. F. van Dijk, Bram van Ginneken, Eren Halici, Ludwig Jacobsson, Vlado Ovtcharov, Quanzheng Li, Jeroen van der Laak, Peter Bult, Oscar Geessink, Melih cetin, Shaoqun Zeng, Geert Litjens, Martin Hedlund, Anders Bjorholm Dahl, Byungjae Lee, Péter Bándi, Huangjing Lin, Jeppe Thagaard, Quirine F. Manson, Meyke Hermsen, Shenghua Cheng, Kyunghyun Paeng, Maschenka Balkenhol, Video Coding & Architectures, Center for Care & Cure Technology Eindhoven, and Biomedical Diagnostics Lab
- Subjects
Whole-slide images ,Computer science ,SDG 3 – Goede gezondheid en welzijn ,computer.software_genre ,Convolutional neural network ,Metastasis ,030218 nuclear medicine & medical imaging ,0302 clinical medicine ,Breast cancer ,Biomedical imaging ,Pathology ,False positive paradox ,Lymph nodes ,Lymph node ,Grand challenge ,Women's cancers Radboud Institute for Molecular Life Sciences [Radboudumc 17] ,Radiological and Ultrasound Technology ,Histological Techniques ,Hospitals ,Women's cancers Radboud Institute for Health Sciences [Radboudumc 17] ,Computer Science Applications ,grand challenge ,medicine.anatomical_structure ,Lymphatic Metastasis ,Urological cancers Radboud Institute for Health Sciences [Radboudumc 15] ,Metric (mathematics) ,Female ,Sentinel Lymph Node ,Algorithms ,Sentinel lymph node ,lymph node metastases ,Rare cancers Radboud Institute for Health Sciences [Radboudumc 9] ,Breast Neoplasms ,Machine learning ,Set (abstract data type) ,03 medical and health sciences ,All institutes and research themes of the Radboud University Medical Center ,SDG 3 - Good Health and Well-being ,Image Interpretation, Computer-Assisted ,medicine ,Humans ,Electrical and Electronic Engineering ,Tumors ,business.industry ,whole-slide images ,Cancer ,Confusion matrix ,medicine.disease ,Test set ,Artificial intelligence ,business ,computer ,Lymph node metastases ,Software ,Kappa - Abstract
Automated detection of cancer metastases in lymph nodes has the potential to improve the assessment of prognosis for patients. To enable fair comparison between the algorithms for this purpose, we set up the CAMELYON17 challenge in conjunction with the IEEE International Symposium on Biomedical Imaging 2017 Conference in Melbourne. Over 300 participants registered on the challenge website, of which 23 teams submitted a total of 37 algorithms before the initial deadline. Participants were provided with 899 whole-slide images (WSIs) for developing their algorithms. The developed algorithms were evaluated based on the test set encompassing 100 patients and 500 WSIs. The evaluation metric used was a quadratic weighted Cohen's kappa. We discuss the algorithmic details of the 10 best pre-conference and two post-conference submissions. All these participants used convolutional neural networks in combination with pre- and postprocessing steps. Algorithms differed mostly in neural network architecture, training strategy, and pre- and postprocessing methodology. Overall, the kappa metric ranged from 0.89 to -0.13 across all submissions. The best results were obtained with pre-trained architectures such as ResNet. Confusion matrix analysis revealed that all participants struggled with reliably identifying isolated tumor cells, the smallest type of metastasis, with detection rates below 40%. Qualitative inspection of the results of the top participants showed categories of false positives, such as nerves or contamination, which could be targeted for further optimization. Last, we show that simple combinations of the top algorithms result in higher kappa metric values than any algorithm individually, with 0.93 for the best combination.
- Published
- 2019
- Full Text
- View/download PDF