1. Computer Vision Foundation Models in Endoscopy: Proof of Concept in Oropharyngeal Cancer.
- Author
-
Paderno A, Rau A, Bedi N, Bossi P, Mercante G, Piazza C, and Holsinger FC
- Subjects
- Humans, Endoscopy methods, Support Vector Machine, Proof of Concept Study, ROC Curve, Image Interpretation, Computer-Assisted methods, Oropharynx pathology, Narrow Band Imaging methods, Oropharyngeal Neoplasms pathology
- Abstract
Objectives: To evaluate the performance of vision transformer-derived image embeddings for distinguishing between normal and neoplastic tissues in the oropharynx and to investigate the potential of computer vision (CV) foundation models in medical imaging., Methods: Computational study using endoscopic frames with a focus on the application of a self-supervised vision transformer model (DINOv2) for tissue classification. High-definition endoscopic images were used to extract image patches that were then normalized and processed using the DINOv2 model to obtain embeddings. These embeddings served as input for a standard support vector machine (SVM) to classify the tissues as neoplastic or normal. The model's discriminative performance was validated using an 80-20 train-validation split., Results: From 38 endoscopic NBI videos, 327 image patches were analyzed. The classification results in the validation cohort demonstrated high accuracy (92%) and precision (89%), with a perfect recall (100%) and an F1-score of 94%. The receiver operating characteristic (ROC) curve yielded an area under the curve (AUC) of 0.96., Conclusion: The use of large vision model-derived embeddings effectively differentiated between neoplastic and normal oropharyngeal tissues. This study supports the feasibility of employing CV foundation models like DINOv2 in the endoscopic evaluation of mucosal lesions, potentially augmenting diagnostic precision in Otorhinolaryngology., Level of Evidence: 4 Laryngoscope, 134:4535-4541, 2024., (© 2024 The American Laryngological, Rhinological and Otological Society, Inc.)
- Published
- 2024
- Full Text
- View/download PDF