Search

Purpose: The purpose of this study was to compare diabetic retinopathy (DR) severity levels assessed from 7 standard-field stereoscopic color photographs on a 35° fundus camera to both Clarus and Optos ultrawidefield color images., Design: Cross-sectional, comparative imaging study., Participants: Participants with DR imaged at a single-center retina practice., Methods: Participants were imaged on 3 cameras at a single visit with the Topcon 35° fundus camera, Clarus, and Optos. The DR Severity Scale (DRSS) level was determined within the 7-field (7F) area of each image set using the ETDRS scale. An additional global DRSS was assigned for both Clarus and Optos images using the entire visible retina. Weighted kappa (wκ) measured the agreement between cameras., Main Outcome Measures: The primary outcome was a 3-way comparison of DRSS level within the 7F area imaged on the 3 cameras. Secondary outcomes included a comparison of the DRSS obtained with standard 7F imaging to the global DRSS of Clarus and Optos and a comparison of the global DRSS between Clarus and Optos only., Results: Ninety-seven eyes (50 participants) were evaluated. Agreement within 1-step of ETDRS levels between standard 7F imaging and Clarus 7F was 90.1% (wκ = 0.65), and with Optos 7F in 85.9%, (wκ = 0.58). Agreement within 1-step between standard 7F imaging and Clarus global was 88.9% of eyes (wκ = 0.63), and Optos global was 85.7%, (wκ = 0.54). Agreement between Clarus and Optos global DR level within 1-step was 89.1% (wκ = 0.68). Intergrader agreement for the 7F ETDRS level was 96% for standard 7F imaging, 98% for Clarus, and 95.5% for Optos., Conclusions: These findings suggest that when evaluating the 7F area on Clarus and Optos, DR severity grades are comparable to standard 7F imaging. However, it is important to understand the unique attributes and differences of each fundus camera when changing the type of system used in a clinical setting due to upgrading equipment. Additionally, if the facility has access to > 1 device, there should not be an exchange between cameras for the same patient., Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article., (© 2023 by the American Academy of Ophthalmology.)

Purpose: The curation of images using human resources is time intensive but an essential step for developing artificial intelligence (AI) algorithms. Our goal was to develop and implement an AI algorithm for image curation in a high-volume setting. We also explored AI tools that will assist in deploying a tiered approach, in which the AI model labels images and flags potential mislabels for human review., Design: Implementation of an AI algorithm., Participants: Seven-field stereoscopic images from multiple clinical trials., Methods: The 7-field stereoscopic image protocol includes 7 pairs of images from various parts of the central retina along with images of the anterior part of the eye. All images were labeled for field number by reading center graders. The model output included classification of the retinal images into 8 field numbers. Probability scores (0-1) were generated to identify misclassified images, with 1 indicating a high probability of a correct label., Main Outcome Measures: Agreement of AI prediction with grader classification of field number and the use of probability scores to identify mislabeled images., Results: The AI model was trained and validated on 17 529 images and tested on 3004 images. The pooled agreement of field numbers between grader classification and the AI model was 88.3% (kappa, 0.87). The pooled mean probability score was 0.97 (standard deviation [SD], 0.08) for images for which the graders agreed with the AI-generated labels and 0.77 (SD, 0.19) for images for which the graders disagreed with the AI-generated labels ( P  < 0.0001). Using receiver operating characteristic curves, a probability score of 0.99 was identified as a cutoff for distinguishing mislabeled images. A tiered workflow using a probability score of < 0.99 as a cutoff would include 27.6% of the 3004 images for human review and reduce the error rate from 11.7% to 1.5%., Conclusions: The implementation of AI algorithms requires measures in addition to model validation. Tools to flag potential errors in the labels generated by AI models will reduce inaccuracies, increase trust in the system, and provide data for continuous model development., (© 2022 by the American Academy of Ophthalmology.)

Purpose: To compare Early Treatment Diabetic Retinopathy Study (ETDRS) severity levels between standard 7-field imaging and ultra-widefield (UWF) imaging and to incorporate peripheral diabetic retinopathy (DR) lesions into the ETDRS grading system., Design: Cross-sectional Study., Participants: Paired images from 192 eyes (189 participants) with diabetic retinopathy were included., Methods: The ETDRS levels were determined by masked graders in 3 ways: standard 7-field imaging, UWF within the 7-field region (7-field UWF imaging), and the entire UWF image (global ETDRS imaging)., Main Outcome Measures: Percentage agreement between 7-field and UWF imaging for ETDRS levels., Results: Of the 166 paired images evaluated, exact agreement was found in 48.8% of eyes between standard 7-field and 7-field UWF ETDRS levels with a weighted κ value of 0.59 (95% confidence interval [CI], 0.5-0.68). Agreement rates varied with DR severity and were least in early DR (30.8%) and moderate nonproliferative DR (26.5%) groups. In 156 eyes with 7-field UWF ETDRS and global UWF ETDRS levels, exact agreement was found in 143 eyes (92%), with a weighted κ value of 0.9 (95% CI, 0.9-0.98). The peripheral lesions contributed to a higher DR severity in 8% and changed the eye to a proliferative DR level in 2%. Reproducibility of the 3 ETDRS evaluations was comparable with a weighted κ value of 0.57 with standard 7-field imaging, 0.65 with 7-field UWF imaging, and 0.60 with global ETDRS scale imaging., Conclusions: Moderate agreement was found in the ETDRS DR severity scale between standard 7-field and UWF imaging, indicating caution in interchanging data from the 2 methods. Both methods showed good reproducibility for clinical trial outcome of 2-step change. The global ETDRS scale provides a comprehensive score to incorporate peripheral changes into the ETDRS scale. The implications of the global scale on progression rate are yet to be determined., (© 2021 by the American Academy of Ophthalmology.)