1. Not without Context-A Multiple Methods Study on Evaluation and Correction of Automated Brain Tumor Segmentations by Experts.
- Author
-
Hoebel KV, Bridge CP, Kim A, Gerstner ER, Ly IK, Deng F, DeSalvo MN, Dietrich J, Huang R, Huang SY, Pomerantz SR, Vagvala S, Rosen BR, and Kalpathy-Cramer J
- Subjects
- Adult, Humans, Algorithms, Pattern Recognition, Automated methods, Tumor Burden, Magnetic Resonance Imaging methods, Image Processing, Computer-Assisted methods, Brain Neoplasms diagnostic imaging, Brain Neoplasms pathology, Glioblastoma pathology
- Abstract
Rationale and Objectives: Brain tumor segmentations are integral to the clinical management of patients with glioblastoma, the deadliest primary brain tumor in adults. The manual delineation of tumors is time-consuming and highly provider-dependent. These two problems must be addressed by introducing automated, deep-learning-based segmentation tools. This study aimed to identify criteria experts use to evaluate the quality of automatically generated segmentations and their thought processes as they correct them., Materials and Methods: Multiple methods were used to develop a detailed understanding of the complex factors that shape experts' perception of segmentation quality and their thought processes in correcting proposed segmentations. Data from a questionnaire and semistructured interview with neuro-oncologists and neuroradiologists were collected between August and December 2021 and analyzed using a combined deductive and inductive approach., Results: Brain tumors are highly complex and ambiguous segmentation targets. Therefore, physicians rely heavily on the given context related to the patient and clinical context in evaluating the quality and need to correct brain tumor segmentation. Most importantly, the intended clinical application determines the segmentation quality criteria and editing decisions. Physicians' personal beliefs and preferences about the capabilities of AI algorithms and whether questionable areas should not be included are additional criteria influencing the perception of segmentation quality and appearance of an edited segmentation., Conclusion: Our findings on experts' perceptions of segmentation quality will allow the design of improved frameworks for expert-centered evaluation of brain tumor segmentation models. In particular, the knowledge presented here can inspire the development of brain tumor-specific metrics for segmentation model training and evaluation., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Jayashree Kalpathy-Cramer reports financial support was provided by National Cancer Institute. Jayashree Kalpathy-Cramer reports a relationship with Bayer Corporation that includes funding grants. Jayashree Kalpathy-Cramer reports a relationship with Genentech Inc that includes funding grants. Jayashree Kalpathy-Cramer reports a relationship with General Electric Company that includes funding grants. Jayashree Kalpathy-Cramer reports a relationship with Siloam Vision Llc that includes consulting or advisory. Competing Interests BRR is on the advisory board for ARIA, Butterfly, Inc., DGMIF (Daegu-Gyeongbuk Medical Innovation Foundation), QMENTA, Subtle Medical, Inc., is a consultant for Broadview Ventures, Janssen Scientific, ECRI Institute, GlaxoSmithKline, Hyperfine Research, Inc., Peking University, Wolf Greenfield, Superconducting Systems, Inc., Robins Kaplin, LLC, Millennium Pharmaceuticals, GE Healthcare, Siemens, Quinn Emanuel Trial Lawyers, Samsung, Shenzhen Maternity & Child Healthcare Hospital, and is a founder of BLINKAI Technologies, Inc. JKC has received research funding (to the institution) from Genentech, GE, and Bayer and is a consultant for Siloam Vision Llc., (Copyright © 2024. Published by Elsevier Inc.)
- Published
- 2024
- Full Text
- View/download PDF