1. Assessing the communication gap between AI models and healthcare professionals: Explainability, utility and trust in AI-driven clinical decision-making
- Author
-
Oskar Wysocki, Jessica Katharine Davies, Markel Vigo, Anne Caroline Armstrong, Dónal Landers, Rebecca Lee, and André Freitas
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,Linguistics and Language ,Artificial Intelligence (cs.AI) ,Computer Science - Artificial Intelligence ,Artificial Intelligence ,Computer Science - Human-Computer Interaction ,Language and Linguistics ,Human-Computer Interaction (cs.HC) ,Machine Learning (cs.LG) - Abstract
This paper contributes with a pragmatic evaluation framework for explainable Machine Learning (ML) models for clinical decision support. The study revealed a more nuanced role for ML explanation models, when these are pragmatically embedded in the clinical context. Despite the general positive attitude of healthcare professionals (HCPs) towards explanations as a safety and trust mechanism, for a significant set of participants there were negative effects associated with confirmation bias, accentuating model over-reliance and increased effort to interact with the model. Also, contradicting one of its main intended functions, standard explanatory models showed limited ability to support a critical understanding of the limitations of the model. However, we found new significant positive effects which repositions the role of explanations within a clinical context: these include reduction of automation bias, addressing ambiguous clinical cases (cases where HCPs were not certain about their decision) and support of less experienced HCPs in the acquisition of new domain knowledge., Comment: supplementary information in the main pdf
- Published
- 2023
- Full Text
- View/download PDF