Management of Thyroid Nodules Seen on Ultrasound: Deep Learning May Match Radiologists Performance

BACKGROUND: Management of thyroid nodules may be inconsistent by different observers and time consuming for radiologists. An artificial intelligence system based on deep learning may improve radiology workflow for management of thyroid nodules. PURPOSE: To develop a deep learning algorithm that uses thyroid ultrasound images to decide whether a thyroid nodule should undergo a biopsy, and to compare the performance of such algorithm to radiologists following ACR TI-RADS. MATERIALS AND METHODS: In this IRB-approved, HIPAA-compliant study, 1377 thyroid nodules from 1230 patients with complete imaging data were retrospectively analyzed. Their malignancy status was determined by either fine-needle aspiration or surgical histology and used as the gold standard. A radiologist assigned ACR TI-RADS features to each nodule. We trained a multi-task deep neural network to provide biopsy recommendations for thyroid nodules based on two orthogonal ultrasound images as the input. In the training phase, the deep learning algorithm was first evaluated using 10-fold cross-validation and then validated on an independent set of consecutive 99 cases not used for model development. The sensitivity and specificity of our algorithm were compared to (1) a consensus of three ACR TI-RADS committee experts and (2) nine other radiologists, all of whom interpreted thyroid ultrasound in clinical practice. RESULTS: On the 99 test cases, the proposed deep learning algorithm achieved 87% sensitivity, the same as expert consensus, and higher than five of nine radiologists. The specificity of the deep learning algorithm was 52% which was similar to ACR TI-RADS committee expert consensus (51%) and higher than seven of nine other radiologists. The mean sensitivity and specificity for the nine radiologists was 83% and 48%, respectively. CONCLUSIONS: A deep learning algorithm for thyroid nodule biopsy recommendations performed with similar sensitivity and specificity compared to ACR TI-RADS committee expert radiologists using ACR TI-RADS guidelines.