Augmenting the Accuracy of Trainee Doctors in Diagnosing Skin Neoplasms in a Real-World Setting: A Prospective Before and After Study (Preprint)

BACKGROUND Although deep neural networks have shown promising results in diagnosing skin cancer, a prospective evaluation in a real-world setting could confirm these results. OBJECTIVE The aim of this study was to evaluate whether an algorithm (http://b2019.modelderm.com) could improve the accuracy of nondermatologists in diagnosing skin neoplasms. METHODS A total of 285 cases (random series) with skin neoplasms suspected by either physicians or patients were recruited in two tertiary care centers located in South Korea. An artificial intelligence (AI) group (144 cases, mean [SD] age, 57.0 [17.7] years; 62 [43.1%] men) was diagnosed via routine examination with capturing photographs and assisted by the algorithm, whereas the control group (141 cases, mean [SD] age, 61.0 [15.3] years; 52 [36.9%] men) was diagnosed only via routine examination with a photographic review. The accuracy of the nondermatologists before and after the interventions was compared. RESULTS Among the AI group, the accuracy of the first impression (Top-1 accuracy; 58.3%) after the assistance was higher than that before the assistance (46.5%, P = 0.0081). The number of differential diagnoses of the participants increased from 1.9 ± 0.5 to 2.2 ± 0.6 after the assistance (P < 0.0001). In the control group, the difference in the Top-1 accuracy between before and after reviewing photographs was not significant (before, 46.1%; after, 51.8%; P = 0.1867) and the number of differential diagnoses was not also significantly increased (before, 2.0 ± 0.4; after, 2.1 ± 0.5; P = 0.5653). CONCLUSIONS In real-world settings, artificial intelligence augmented the diagnostic accuracy of trainee doctors. The limitation of this study is that the algorithm was tested only for Asians recruited from a single region. Additional international randomized controlled trials involving various ethnicities are required.