Search

Saif Aldeen AlRyalat,1 Praveer Singh,2 Jayashree Kalpathy-Cramer,2 Malik Y Kahook2 1Department of Ophthalmology, The University of Jordan, Amman, 11942, Jordan; 2Department of Ophthalmology, University of Colorado School of Medicine, Sue Anschutz-Rodgers Eye Center, Aurora, CO, USACorrespondence: Malik Y Kahook, Department of Ophthalmology, University of Colorado School of Medicine, Sue Anschutz-Rodgers Eye Center, Aurora, CO, USA, Email Malik.Kahook@cuanschutz.eduAbstract: There has been a recent surge in the number of publications centered on the use of artificial intelligence (AI) to diagnose various systemic diseases. The Food and Drug Administration has approved several algorithms for use in clinical practice. In ophthalmology, most advances in AI relate to diabetic retinopathy, which is a disease process with agreed upon diagnostic and classification criteria. However, this is not the case for glaucoma, which is a relatively complex disease without agreed-upon diagnostic criteria. Moreover, currently available public datasets that focus on glaucoma have inconstant label quality, further complicating attempts at training AI algorithms efficiently. In this perspective paper, we discuss specific details related to developing AI models for glaucoma and suggest potential steps to overcome current limitations.Keywords: artificial intelligence, glaucoma, deep learning, optic disc, segmentation

Objective: To demonstrate and test the capabilities of the American College of Radiology (ACR) Connect and AI-LAB software platform by implementing multi-institutional artificial intelligence (AI) training and validation for breast density classification., Methods: In this proof-of-concept study, six U.S.-based hospitals installed Connect and AI-LAB. A breast density algorithm was trained and tested on retrospective mammograms. We recorded time to receive an IRB approval, to install software locally, and to complete the testing and training. We calculated the performance of the breast density algorithm at each participating hospital and compared it to the performance of a hold-out multi-institutional clinical trial testing dataset and a retrospective multi-institutional dataset. We calculated the performance of the locally fine-tuned models on the hold-out test datasets., Results: The median time to receive IRB approval was 66 days, and the median time to successfully install Connect and AI-LAB locally was 157 days. The median time to complete breast density algorithm testing and training was 216 days. The breast density algorithm performed worse at each hospital than on the hold-out test dataset, suggesting poor generalizability of the base model. The fine-tuned models had mixed performance locally and performed poorly on the test dataset., Discussion: In this study, we demonstrate the successful installation and implementation of Connect and AI-LAB software platforms at six facilities using a breast density algorithm. Our results suggest poor generalizability of an algorithm trained on a single dataset and algorithms fine-tuned at individual institutions, emphasizing the hypothetical importance of multi-institutional testing and training., (Copyright © 2024. Published by Elsevier Inc.)

Purpose: To evaluate whether providing clinicians with an artificial intelligence (AI)-based vascular severity score (VSS) improves consistency in the diagnosis of plus disease in retinopathy of prematurity (ROP)., Design: Multireader diagnostic accuracy imaging study., Participants: Eleven ROP experts, 9 of whom had been in practice for 10 years or more., Methods: RetCam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available examinations, a subset of 150 eye examinations from 110 infants were selected for grading. An AI-based VSS was assigned to each set of images using the i-ROP DL system (Siloam Vision). The clinicians were asked to diagnose plus disease for each examination and to assign an estimated VSS (range, 1-9) at baseline, and then again 1 month later with AI-based VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye examination from the Imaging and Informatics in ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis., Main Outcome Measures: Mean linearly weighted κ value for plus disease diagnosis compared with RSD. Area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPR) for labels 1 through 9 compared with RSD for plus disease., Results: Expert agreement improved significantly, from substantial (κ value, 0.69 [0.59, 0.75]) to near perfect (κ value, 0.81 [0.71, 0.86]), when AI-based VSS was integrated. Additionally, a significant improvement in plus disease discrimination was achieved as measured by mean AUC (from 0.94 [95% confidence interval (CI), 0.92-0.96] to 0.98 [95% CI, 0.96-0.99]; difference, 0.04 [95% CI, 0.01-0.06]) and AUPR (from 0.86 [95% CI, 0.81-0.90] to 0.95 [95% CI, 0.91-0.97]; difference, 0.09 [95% CI, 0.03-0.14])., Conclusions: Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling as compared with an RSD for plus disease. If implemented in practice, AI-based VSS could reduce interobserver variability and could standardize treatment for infants with ROP., Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article., (Copyright © 2024 American Academy of Ophthalmology. All rights reserved.)

Background and Objective: Biparametric magnetic resonance imaging (bpMRI), excluding dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI), is a potential replacement for multiparametric MRI (mpMRI) in diagnosing clinically significant prostate cancer (csPCa). An extensive international multireader multicase observer study was conducted to assess the noninferiority of bpMRI to mpMRI in csPCa diagnosis., Methods: An observer study was conducted with 400 mpMRI examinations from four European centers, excluding examinations with prior prostate treatment or csPCa (Gleason grade [GG] ≥2) findings. Readers assessed bpMRI and mpMRI sequentially, assigning lesion-specific Prostate Imaging Reporting and Data System (PI-RADS) scores (3-5) and a patient-level suspicion score (0-100). The noninferiority of patient-level bpMRI versus mpMRI csPCa diagnosis was evaluated using the area under the receiver operating curve (AUROC) alongside the sensitivity and specificity at PI-RADS ≥3 with a 5% margin. The secondary outcomes included insignificant prostate cancer (GG1) diagnosis, diagnostic evaluations at alternative risk thresholds, decision curve analyses (DCAs), and subgroup analyses considering reader expertise. Histopathology and ≥3 yr of follow-up were used for the reference standard., Key Findings and Limitations: Sixty-two readers (45 centers and 20 countries) participated. The prevalence of csPCa was 33% (133/400); bpMRI and mpMRI showed similar AUROC values of 0.853 (95% confidence interval [CI], 0.819-0.887) and 0.859 (95% CI, 0.826-0.893), respectively, with a noninferior difference of -0.6% (95% CI, -1.2% to 0.1%, p < 0.001). At PI-RADS ≥3, bpMRI and mpMRI had sensitivities of 88.6% (95% CI, 84.8-92.3%) and 89.4% (95% CI, 85.8-93.1%), respectively, with a noninferior difference of -0.9% (95% CI, -1.7% to 0.0%, p < 0.001), and specificities of 58.6% (95% CI, 52.3-63.1%) and 57.7% (95% CI, 52.3-63.1%), respectively, with a noninferior difference of 0.9% (95% CI, 0.0-1.8%, p < 0.001). At alternative risk thresholds, mpMRI increased sensitivity at the expense of reduced specificity. DCA demonstrated the highest net benefit for an mpMRI pathway in cancer-averse scenarios, whereas a bpMRI pathway showed greater benefit for biopsy-averse scenarios. A subgroup analysis indicated limited additional benefit of DCE MRI for nonexperts. Limitations included that biopsies were conducted based on mpMRI imaging, and reading was performed in a sequential order., Conclusions and Clinical Implications: It has been found that bpMRI is noninferior to mpMRI in csPCa diagnosis at AUROC, along with the sensitivity and specificity at PI-RADS ≥3, showing its value in individuals without prior csPCa findings and prostate treatment. Additional randomized prospective studies are required to investigate the generalizability of outcomes., (Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)

Background: Age-related macular degeneration (AMD) is a retinal disorder characterized by central retinal (macular) damage. Approximately 10% to 20% of non-exudative AMD cases progress to the exudative form, which may result in rapid deterioration of central vision. Individuals with exudative AMD (eAMD) need prompt consultation with retinal specialists to minimize the risk and extent of vision loss. Traditional methods of diagnosing ophthalmic disease rely on clinical evaluation and multiple imaging techniques, which can be resource-consuming. Tests leveraging artificial intelligence (AI) hold the promise of automatically identifying and categorizing pathological features, enabling the timely diagnosis and treatment of eAMD., Objectives: To determine the diagnostic accuracy of artificial intelligence (AI) as a triaging tool for exudative age-related macular degeneration (eAMD)., Search Methods: We searched CENTRAL, MEDLINE, Embase, three clinical trials registries, and Data Archiving and Networked Services (DANS) for gray literature. We did not restrict searches by language or publication date. The date of the last search was April 2024., Selection Criteria: Included studies compared the test performance of algorithms with that of human readers to detect eAMD on retinal images collected from people with AMD who were evaluated at eye clinics in community or academic medical centers, and who were not receiving treatment for eAMD when the images were taken. We included algorithms that were either internally or externally validated or both., Data Collection and Analysis: Pairs of review authors independently extracted data and assessed study quality using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool with revised signaling questions. For studies that reported more than one set of performance results, we extracted only one set of diagnostic accuracy data per study based on the last development stage or the optimal algorithm as indicated by the study authors. For two-class algorithms, we collected data from the 2x2 table whenever feasible. For multi-class algorithms, we first consolidated data from all classes other than eAMD before constructing the corresponding 2x2 tables. Assuming a common positivity threshold applied by the included studies, we chose random-effects, bivariate logistic models to estimate summary sensitivity and specificity as the primary performance metrics., Main Results: We identified 36 eligible studies that reported 40 sets of algorithm performance data, encompassing over 16,000 participants and 62,000 images. We included 28 studies (78%) that reported 31 algorithms with performance data in the meta-analysis. The remaining nine studies (25%) reported eight algorithms that lacked usable performance data; we reported them in the qualitative synthesis. Study characteristics and risk of bias Most studies were conducted in Asia, followed by Europe, the USA, and collaborative efforts spanning multiple countries. Most studies identified study participants from the hospital setting, while others used retinal images from public repositories; a few studies did not specify image sources. Based on four of the 36 studies reporting demographic information, the age of the study participants ranged from 62 to 82 years. The included algorithms used various retinal image types as model input, such as optical coherence tomography (OCT) images (N = 15), fundus images (N = 6), and multi-modal imaging (N = 7). The predominant core method used was deep neural networks. All studies that reported externally validated algorithms were at high risk of bias mainly due to potential selection bias from either a two-gate design or the inappropriate exclusion of potentially eligible retinal images (or participants). Findings Only three of the 40 included algorithms were externally validated (7.5%, 3/40). The summary sensitivity and specificity were 0.94 (95% confidence interval (CI) 0.90 to 0.97) and 0.99 (95% CI 0.76 to 1.00), respectively, when compared to human graders (3 studies; 27,872 images; low-certainty evidence). The prevalence of images with eAMD ranged from 0.3% to 49%. Twenty-eight algorithms were reportedly either internally validated (20%, 8/40) or tested on a development set (50%, 20/40); the pooled sensitivity and specificity were 0.93 (95% CI 0.89 to 0.96) and 0.96 (95% CI 0.94 to 0.98), respectively, when compared to human graders (28 studies; 33,409 images; low-certainty evidence). We did not identify significant sources of heterogeneity among these 28 algorithms. Although algorithms using OCT images appeared more homogeneous and had the highest summary specificity (0.97, 95% CI 0.93 to 0.98), they were not superior to algorithms using fundus images alone (0.94, 95% CI 0.89 to 0.97) or multimodal imaging (0.96, 95% CI 0.88 to 0.99; P for meta-regression = 0.239). The median prevalence of images with eAMD was 30% (interquartile range [IQR] 22% to 39%). We did not include eight studies that described nine algorithms (one study reported two sets of algorithm results) to distinguish eAMD from normal images, images of other AMD, or other non-AMD retinal lesions in the meta-analysis. Five of these algorithms were generally based on smaller datasets (range 21 to 218 participants per study) yet with a higher prevalence of eAMD images (range 33% to 66%). Relative to human graders, the reported sensitivity in these studies ranged from 0.95 and 0.97, while the specificity ranged from 0.94 to 0.99. Similarly, using small datasets (range 46 to 106), an additional four algorithms for detecting eAMD from other retinal lesions showed high sensitivity (range 0.96 to 1.00) and specificity (range 0.77 to 1.00)., Authors' Conclusions: Low- to very low-certainty evidence suggests that an algorithm-based test may correctly identify most individuals with eAMD without increasing unnecessary referrals (false positives) in either the primary or the specialty care settings. There were significant concerns for applying the review findings due to variations in the eAMD prevalence in the included studies. In addition, among the included algorithm-based tests, diagnostic accuracy estimates were at risk of bias due to study participants not reflecting real-world characteristics, inadequate model validation, and the likelihood of selective results reporting. Limited quality and quantity of externally validated algorithms highlighted the need for high-certainty evidence. This evidence will require a standardized definition for eAMD on different imaging modalities and external validation of the algorithm to assess generalizability., (Copyright © 2024 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.)

People diagnosed with cancer and their formal and informal caregivers are increasingly faced with a deluge of complex information, thanks to rapid advancements in the type and volume of diagnostic, prognostic, and treatment data. This commentary discusses the opportunities and challenges that the society faces as we integrate large volumes of data into regular cancer care., (©2024 The Authors; Published by the American Association for Cancer Research.)

A number of challenges hinder artificial intelligence (AI) models from effective clinical translation. Foremost among these challenges is the lack of generalizability, which is defined as the ability of a model to perform well on datasets that have different characteristics from the training data. We recently investigated the development of an AI pipeline on digital images of the cervix, utilizing a multi-heterogeneous dataset of 9,462 women (17,013 images) and a multi-stage model selection and optimization approach, to generate a diagnostic classifier able to classify images of the cervix into "normal", "indeterminate" and "precancer/cancer" (denoted as "precancer+") categories. In this work, we investigate the performance of this multiclass classifier on external data not utilized in training and internal validation, to assess the generalizability of the classifier when moving to new settings. We assessed both the classification performance and repeatability of our classifier model across the two axes of heterogeneity present in our dataset: image capture device and geography, utilizing both out-of-the-box inference and retraining with external data. Our results demonstrate that device-level heterogeneity affects our model performance more than geography-level heterogeneity. Classification performance of our model is strong on images from a new geography without retraining, while incremental retraining with inclusion of images from a new device progressively improves classification performance on that device up to a point of saturation. Repeatability of our model is relatively unaffected by data heterogeneity and remains strong throughout. Our work supports the need for optimized retraining approaches that address data heterogeneity (e.g., when moving to a new device) to facilitate effective use of AI models in new settings., Competing Interests: The authors have declared that no competing interests exist., (Copyright: This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.)

Clinical Relevance: Glaucoma is a complex eye condition with varied morphological and clinical presentations, making diagnosis and management challenging. The lack of a consensus definition for glaucoma or glaucomatous optic neuropathy further complicates the development of universal diagnostic tools. Developing robust artificial intelligence (AI) models for glaucoma screening is essential for early detection and treatment but faces significant obstacles. Effective deep learning algorithms require large, well-curated datasets from diverse patient populations and imaging protocols. However, creating centralized data repositories is hindered by concerns over data sharing, patient privacy, regulatory compliance, and intellectual property. Federated Learning (FL) offers a potential solution by enabling data to remain locally hosted while facilitating distributed model training across multiple sites., Methods: A comprehensive literature review was conducted on the application of Federated Learning in training AI models for glaucoma screening. Publications from 1950 to 2024 were searched using databases such as PubMed and IEEE Xplore with keywords including "glaucoma," "federated learning," "artificial intelligence," "deep learning," "machine learning," "distributed learning," "privacy-preserving," "data sharing," "medical imaging," and "ophthalmology." Articles were included if they discussed the use of FL in glaucoma-related AI tasks or addressed data sharing and privacy challenges in ophthalmic AI development., Results: FL enables collaborative model development without centralizing sensitive patient data, addressing privacy and regulatory concerns. Studies show that FL can improve model performance and generalizability by leveraging diverse datasets while maintaining data security. FL models have achieved comparable or superior accuracy to those trained on centralized data, demonstrating effectiveness in real-world clinical settings., Conclusions: Federated Learning presents a promising strategy to overcome current obstacles in developing AI models for glaucoma screening. By balancing the need for extensive, diverse training data with the imperative to protect patient privacy and comply with regulations, FL facilitates collaborative model training without compromising data security. This approach offers a pathway toward more accurate and generalizable AI solutions for glaucoma detection and management., Financial Disclosure(s): Proprietary or commercial disclosure may be found after the references in the Footnotes and Disclosures at the end of this article., (Copyright © 2024 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.)

Artificial intelligence (AI) shows potential to improve health care by leveraging data to build models that can inform clinical workflows. However, access to large quantities of diverse data is needed to develop robust generalizable models. Data sharing across institutions is not always feasible due to legal, security, and privacy concerns. Federated learning (FL) allows for multi-institutional training of AI models, obviating data sharing, albeit with different security and privacy concerns. Specifically, insights exchanged during FL can leak information about institutional data. In addition, FL can introduce issues when there is limited trust among the entities performing the compute. With the growing adoption of FL in health care, it is imperative to elucidate the potential risks. We thus summarize privacy-preserving FL literature in this work with special regard to health care. We draw attention to threats and review mitigation approaches. We anticipate this review to become a health-care researcher's guide to security and privacy in FL., Competing Interests: The authors declare no competing interests., (© 2024 The Authors.)

The correct interpretation of breast density is important in the assessment of breast cancer risk. AI has been shown capable of accurately predicting breast density, however, due to the differences in imaging characteristics across mammography systems, models built using data from one system do not generalize well to other systems. Though federated learning (FL) has emerged as a way to improve the generalizability of AI without the need to share data, the best way to preserve features from all training data during FL is an active area of research. To explore FL methodology, the breast density classification FL challenge was hosted in partnership with the American College of Radiology, Harvard Medical Schools' Mass General Brigham, University of Colorado, NVIDIA, and the National Institutes of Health National Cancer Institute. Challenge participants were able to submit docker containers capable of implementing FL on three simulated medical facilities, each containing a unique large mammography dataset. The breast density FL challenge ran from June 15 to September 5, 2022, attracting seven finalists from around the world. The winning FL submission reached a linear kappa score of 0.653 on the challenge test data and 0.413 on an external testing dataset, scoring comparably to a model trained on the same data in a central location., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier B.V. All rights reserved.)

Background: Artificial intelligence (AI) systems can potentially aid the diagnostic pathway of prostate cancer by alleviating the increasing workload, preventing overdiagnosis, and reducing the dependence on experienced radiologists. We aimed to investigate the performance of AI systems at detecting clinically significant prostate cancer on MRI in comparison with radiologists using the Prostate Imaging-Reporting and Data System version 2.1 (PI-RADS 2.1) and the standard of care in multidisciplinary routine practice at scale., Methods: In this international, paired, non-inferiority, confirmatory study, we trained and externally validated an AI system (developed within an international consortium) for detecting Gleason grade group 2 or greater cancers using a retrospective cohort of 10 207 MRI examinations from 9129 patients. Of these examinations, 9207 cases from three centres (11 sites) based in the Netherlands were used for training and tuning, and 1000 cases from four centres (12 sites) based in the Netherlands and Norway were used for testing. In parallel, we facilitated a multireader, multicase observer study with 62 radiologists (45 centres in 20 countries; median 7 [IQR 5-10] years of experience in reading prostate MRI) using PI-RADS (2.1) on 400 paired MRI examinations from the testing cohort. Primary endpoints were the sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC) of the AI system in comparison with that of all readers using PI-RADS (2.1) and in comparison with that of the historical radiology readings made during multidisciplinary routine practice (ie, the standard of care with the aid of patient history and peer consultation). Histopathology and at least 3 years (median 5 [IQR 4-6] years) of follow-up were used to establish the reference standard. The statistical analysis plan was prespecified with a primary hypothesis of non-inferiority (considering a margin of 0·05) and a secondary hypothesis of superiority towards the AI system, if non-inferiority was confirmed. This study was registered at ClinicalTrials.gov, NCT05489341., Findings: Of the 10 207 examinations included from Jan 1, 2012, through Dec 31, 2021, 2440 cases had histologically confirmed Gleason grade group 2 or greater prostate cancer. In the subset of 400 testing cases in which the AI system was compared with the radiologists participating in the reader study, the AI system showed a statistically superior and non-inferior AUROC of 0·91 (95% CI 0·87-0·94; p<0·0001), in comparison to the pool of 62 radiologists with an AUROC of 0·86 (0·83-0·89), with a lower boundary of the two-sided 95% Wald CI for the difference in AUROC of 0·02. At the mean PI-RADS 3 or greater operating point of all readers, the AI system detected 6·8% more cases with Gleason grade group 2 or greater cancers at the same specificity (57·7%, 95% CI 51·6-63·3), or 50·4% fewer false-positive results and 20·0% fewer cases with Gleason grade group 1 cancers at the same sensitivity (89·4%, 95% CI 85·3-92·9). In all 1000 testing cases where the AI system was compared with the radiology readings made during multidisciplinary practice, non-inferiority was not confirmed, as the AI system showed lower specificity (68·9% [95% CI 65·3-72·4] vs 69·0% [65·5-72·5]) at the same sensitivity (96·1%, 94·0-98·2) as the PI-RADS 3 or greater operating point. The lower boundary of the two-sided 95% Wald CI for the difference in specificity (-0·04) was greater than the non-inferiority margin (-0·05) and a p value below the significance threshold was reached (p<0·001)., Interpretation: An AI system was superior to radiologists using PI-RADS (2.1), on average, at detecting clinically significant prostate cancer and comparable to the standard of care. Such a system shows the potential to be a supportive tool within a primary diagnostic setting, with several associated benefits for patients and radiologists. Prospective validation is needed to test clinical applicability of this system., Funding: Health~Holland and EU Horizon 2020., Competing Interests: Declaration of interests NAO provides statistical consultation to Siemens Healthineers, Takeda, and Qure, and serves as a committee member of the Eastern Cooperative Oncology Group–American College of Radiology Imaging Network, the Tomosynthesis Mammographic Imaging Screening Trial, and the National Cancer Institute's Clinical Imaging Steering Committee (Bethesda, MD, USA). AB has been a consultant and advisor for Astellas and Bayer; board membership, officer, and trustee for Glactone Pharma, and LIDDS Pharma; has received lecture honoraria for Accord, Astellas, AstraZeneca, Bayer, Ipsen, Janssen, and Merck; has participated in trials run by Astellas, Ferring, and Janssen; and holds stock in Glactone Pharma, LIDDS Pharma, Noviga, and WntResearch. BvG holds stocks in and is a founder of Thirona. JKC has received research funding from GE Healthcare and Genentech and is the co-inventor of software that has been licensed to Siloam Vision. JKC has equity ownership in Siloam Vision. GS has been an advisory board member of Exact Imaging and Angiogenesis and has received lecture honorarium from Hitachi. OR has received funding for travel expenses from Philips Medical Systems. ARP has received research funding from Siemens Healthineers, holds stocks in Lucida Medical, and has received lecture honoraria for Siemens Healthineers and Bayer. HH has received research funding from Siemens Healthineers and Canon Medical Systems. GV has been a clinical advisory board member of AGFA Healthcare. VK has received lecture honoraria on prostate cancer diagnosis from the European Association of Urology and Singapore Urology Association and has received research funding from Prostate Cancer UK and the John Black Charitable Foundation. DB has received lecture honorarium from Bayer Vital and holds stocks in NVIDIA, Microsoft, and MSCI-World ETF. RvdB has been an advisory board member for Janssen; has received lecture honoraria from Amgen, Astellas, Ipsen, Janssen, and MSD; has received research support from Astellas and Janssen; and has participated in trials run by Janssen. All other authors declare no competing interests., (Copyright © 2024 Elsevier Ltd. All rights reserved, including those for text and data mining, AI training, and similar technologies.)

Despite the remarkable advances in cancer diagnosis, treatment, and management over the past decade, malignant tumors remain a major public health problem. Further progress in combating cancer may be enabled by personalizing the delivery of therapies according to the predicted response for each individual patient. The design of personalized therapies requires the integration of patient-specific information with an appropriate mathematical model of tumor response. A fundamental barrier to realizing this paradigm is the current lack of a rigorous yet practical mathematical theory of tumor initiation, development, invasion, and response to therapy. We begin this review with an overview of different approaches to modeling tumor growth and treatment, including mechanistic as well as data-driven models based on big data and artificial intelligence. We then present illustrative examples of mathematical models manifesting their utility and discuss the limitations of stand-alone mechanistic and data-driven models. We then discuss the potential of mechanistic models for not only predicting but also optimizing response to therapy on a patient-specific basis. We describe current efforts and future possibilities to integrate mechanistic and data-driven models. We conclude by proposing five fundamental challenges that must be addressed to fully realize personalized care for cancer patients driven by computational models.

The Radiological Society of North of America (RSNA) and the Medical Image Computing and Computer Assisted Intervention (MICCAI) Society have led a series of joint panels and seminars focused on the present impact and future directions of artificial intelligence (AI) in radiology. These conversations have collected viewpoints from multidisciplinary experts in radiology, medical imaging, and machine learning on the current clinical penetration of AI technology in radiology and how it is impacted by trust, reproducibility, explainability, and accountability. The collective points-both practical and philosophical-define the cultural changes for radiologists and AI scientists working together and describe the challenges ahead for AI technologies to meet broad approval. This article presents the perspectives of experts from MICCAI and RSNA on the clinical, cultural, computational, and regulatory considerations-coupled with recommended reading materials-essential to adopt AI technology successfully in radiology and, more generally, in clinical practice. The report emphasizes the importance of collaboration to improve clinical deployment, highlights the need to integrate clinical and medical imaging data, and introduces strategies to ensure smooth and incentivized integration. Keywords: Adults and Pediatrics, Computer Applications-General (Informatics), Diagnosis, Prognosis © RSNA, 2024.

Background: We recently conducted a phase 2 trial (NCT028865685) evaluating intracranial efficacy of pembrolizumab for brain metastases (BM) of diverse histologies. Our study met its primary efficacy endpoint and illustrates that pembrolizumab exerts promising activity in a select group of patients with BM. Given the importance of aberrant vasculature in mediating immunosuppression, we explored the relationship between immune checkpoint inhibitor (ICI) efficacy and vascular architecture in the hopes of identifying potential mechanisms of intracranial ICI response or resistance for BM., Methods: Using Vessel Architectural Imaging, a histologically validated quantitative metric for in vivo tumor vascular physiology, we analyzed dual-echo DSC/DCE MRI for 44 patients on trial. Tumor and peri-tumor cerebral blood volume/flow, vessel size, arterial and venous dominance, and vascular permeability were measured before and after treatment with pembrolizumab., Results: BM that progressed on ICI were characterized by a highly aberrant vasculature dominated by large-caliber vessels. In contrast, ICI-responsive BM possessed a more structurally balanced vasculature consisting of both small and large vessels, and there was a trend toward a decrease in under-perfused tissue, suggesting a reversal of the negative effects of hypoxia. In the peri-tumor region, the development of smaller blood vessels, consistent with neo-angiogenesis, was associated with tumor growth before radiographic evidence of contrast enhancement on anatomical MRI., Conclusions: This study, one of the largest functional imaging studies for BM, suggests that vascular architecture is linked with ICI efficacy. Studies identifying modulators of vascular architecture, and effects on immune activity, are warranted and may inform future combination treatments., (© The Author(s) 2023. Published by Oxford University Press on behalf of the Society for Neuro-Oncology. All rights reserved. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.)

The Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017. This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets. The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security. Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources. Overcoming these obstacles requires meticulous project management and adherence to strict timelines. The article also highlights the potential of crowdsourced annotation to progress medical imaging research. Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes. Keywords: Use of AI in Education, Artificial Intelligence © RSNA, 2024.

Purpose: Adverse clinical events cause significant morbidity in patients with GBM (GBM). We examined whether genomic alterations were associated with AE (AE) in patients with GBM., Experimental Design: We identified adults with histologically confirmed IDH-wild-type GBM with targeted next-generation sequencing (OncoPanel) at Dana Farber Cancer Institute from 2013 to 2019. Seizure at presentation, lymphopenia, thromboembolic events, pseudoprogression, and early progression (within 6 months of diagnosis) were identified as AE. The biologic function of genetic variants was categorized as loss-of-function (LoF), no change in function, or gain-of-function (GoF) using a somatic tumor mutation knowledge base (OncoKB) and consensus protein function predictions. Associations between functional genomic alterations and AE were examined using univariate logistic regressions and multivariable regressions adjusted for additional clinical predictors., Results: Our study included 470 patients diagnosed with GBM who met the study criteria. We focused on 105 genes that had sequencing data available for ≥ 90% of the patients and were altered in ≥10% of the cohort. Following false-discovery rate (FDR) correction and multivariable adjustment, the TP53, RB1, IGF1R, and DIS3 LoF alterations were associated with lower odds of seizures, while EGFR, SMARCA4, GNA11, BRD4, and TCF3 GoF and SETD2 LoF alterations were associated with higher odds of seizures. For all other AE of interest, no significant associations were found with genomic alterations following FDR correction., Conclusions: Genomic biomarkers based on functional variant analysis of a routine clinical panel may help identify AE in GBM, particularly seizures. Identifying these risk factors could improve the management of patients through better supportive care and consideration of prophylactic therapies., (©2024 American Association for Cancer Research.)

Importance: Retinopathy of prematurity (ROP) is a leading cause of blindness in children, with significant disparities in outcomes between high-income and low-income countries, due in part to insufficient access to ROP screening., Objective: To evaluate how well autonomous artificial intelligence (AI)-based ROP screening can detect more-than-mild ROP (mtmROP) and type 1 ROP., Design, Setting, and Participants: This diagnostic study evaluated the performance of an AI algorithm, trained and calibrated using 2530 examinations from 843 infants in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) study, on 2 external datasets (6245 examinations from 1545 infants in the Stanford University Network for Diagnosis of ROP [SUNDROP] and 5635 examinations from 2699 infants in the Aravind Eye Care Systems [AECS] telemedicine programs). Data were taken from 11 and 48 neonatal care units in the US and India, respectively. Data were collected from January 2012 to July 2021, and data were analyzed from July to December 2023., Exposures: An imaging processing pipeline was created using deep learning to autonomously identify mtmROP and type 1 ROP in eye examinations performed via telemedicine., Main Outcomes and Measures: The area under the receiver operating characteristics curve (AUROC) as well as sensitivity and specificity for detection of mtmROP and type 1 ROP at the eye examination and patient levels., Results: The prevalence of mtmROP and type 1 ROP were 5.9% (91 of 1545) and 1.2% (18 of 1545), respectively, in the SUNDROP dataset and 6.2% (168 of 2699) and 2.5% (68 of 2699) in the AECS dataset. Examination-level AUROCs for mtmROP and type 1 ROP were 0.896 and 0.985, respectively, in the SUNDROP dataset and 0.920 and 0.982 in the AECS dataset. At the cross-sectional examination level, mtmROP detection had high sensitivity (SUNDROP: mtmROP, 83.5%; 95% CI, 76.6-87.7; type 1 ROP, 82.2%; 95% CI, 81.2-83.1; AECS: mtmROP, 80.8%; 95% CI, 76.2-84.9; type 1 ROP, 87.8%; 95% CI, 86.8-88.7). At the patient level, all infants who developed type 1 ROP screened positive (SUNDROP: 100%; 95% CI, 81.4-100; AECS: 100%; 95% CI, 94.7-100) prior to diagnosis., Conclusions and Relevance: Where and when ROP telemedicine programs can be implemented, autonomous ROP screening may be an effective force multiplier for secondary prevention of ROP.

Rationale and Objectives: Brain tumor segmentations are integral to the clinical management of patients with glioblastoma, the deadliest primary brain tumor in adults. The manual delineation of tumors is time-consuming and highly provider-dependent. These two problems must be addressed by introducing automated, deep-learning-based segmentation tools. This study aimed to identify criteria experts use to evaluate the quality of automatically generated segmentations and their thought processes as they correct them., Materials and Methods: Multiple methods were used to develop a detailed understanding of the complex factors that shape experts' perception of segmentation quality and their thought processes in correcting proposed segmentations. Data from a questionnaire and semistructured interview with neuro-oncologists and neuroradiologists were collected between August and December 2021 and analyzed using a combined deductive and inductive approach., Results: Brain tumors are highly complex and ambiguous segmentation targets. Therefore, physicians rely heavily on the given context related to the patient and clinical context in evaluating the quality and need to correct brain tumor segmentation. Most importantly, the intended clinical application determines the segmentation quality criteria and editing decisions. Physicians' personal beliefs and preferences about the capabilities of AI algorithms and whether questionable areas should not be included are additional criteria influencing the perception of segmentation quality and appearance of an edited segmentation., Conclusion: Our findings on experts' perceptions of segmentation quality will allow the design of improved frameworks for expert-centered evaluation of brain tumor segmentation models. In particular, the knowledge presented here can inspire the development of brain tumor-specific metrics for segmentation model training and evaluation., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Jayashree Kalpathy-Cramer reports financial support was provided by National Cancer Institute. Jayashree Kalpathy-Cramer reports a relationship with Bayer Corporation that includes funding grants. Jayashree Kalpathy-Cramer reports a relationship with Genentech Inc that includes funding grants. Jayashree Kalpathy-Cramer reports a relationship with General Electric Company that includes funding grants. Jayashree Kalpathy-Cramer reports a relationship with Siloam Vision Llc that includes consulting or advisory. Competing Interests BRR is on the advisory board for ARIA, Butterfly, Inc., DGMIF (Daegu-Gyeongbuk Medical Innovation Foundation), QMENTA, Subtle Medical, Inc., is a consultant for Broadview Ventures, Janssen Scientific, ECRI Institute, GlaxoSmithKline, Hyperfine Research, Inc., Peking University, Wolf Greenfield, Superconducting Systems, Inc., Robins Kaplin, LLC, Millennium Pharmaceuticals, GE Healthcare, Siemens, Quinn Emanuel Trial Lawyers, Samsung, Shenzhen Maternity & Child Healthcare Hospital, and is a founder of BLINKAI Technologies, Inc. JKC has received research funding (to the institution) from Genentech, GE, and Bayer and is a consultant for Siloam Vision Llc., (Copyright © 2024. Published by Elsevier Inc.)

Background: The HPV-automated visual evaluation (PAVE) Study is an extensive, multinational initiative designed to advance cervical cancer prevention in resource-constrained regions. Cervical cancer disproportionally affects regions with limited access to preventive measures. PAVE aims to assess a novel screening-triage-treatment strategy integrating self-sampled HPV testing, deep-learning-based automated visual evaluation (AVE), and targeted therapies., Methods: Phase 1 efficacy involves screening up to 100,000 women aged 25-49 across nine countries, using self-collected vaginal samples for hierarchical HPV evaluation: HPV16, else HPV18/45, else HPV31/33/35/52/58, else HPV39/51/56/59/68 else negative. HPV-positive individuals undergo further evaluation, including pelvic exams, cervical imaging, and biopsies. AVE algorithms analyze images, assigning risk scores for precancer, validated against histologic high-grade precancer. Phase 1, however, does not integrate AVE results into patient management, contrasting them with local standard care.Phase 2 effectiveness focuses on deploying AVE software and HPV genotype data in real-time clinical decision-making, evaluating feasibility, acceptability, cost-effectiveness, and health communication of the PAVE strategy in practice., Results: Currently, sites have commenced fieldwork, and conclusive results are pending., Conclusions: The study aspires to validate a screen-triage-treat protocol utilizing innovative biomarkers to deliver an accurate, feasible, and cost-effective strategy for cervical cancer prevention in resource-limited areas. Should the study validate PAVE, its broader implementation could be recommended, potentially expanding cervical cancer prevention worldwide., Funding: The consortial sites are responsible for their own study costs. Research equipment and supplies, and the NCI-affiliated staff are funded by the National Cancer Institute Intramural Research Program including supplemental funding from the Cancer Cures Moonshot Initiative. No commercial support was obtained. Brian Befano was supported by NCI/ NIH under Grant T32CA09168., Competing Interests: Sd, RP, NC, FI, DE, AR, JJ, LC, KD, PH, AN, AU, JM, SA, NW, JK, MS No competing interests declared, BB is an employee of Information Management Services Inc

Novel screening and diagnostic tests based on artificial intelligence (AI) image recognition algorithms are proliferating. Some initial reports claim outstanding accuracy followed by disappointing lack of confirmation, including our own early work on cervical screening. This is a presentation of lessons learned, organized as a conceptual step-by-step approach to bridge the gap between the creation of an AI algorithm and clinical efficacy. The first fundamental principle is specifying rigorously what the algorithm is designed to identify and what the test is intended to measure (eg, screening, diagnostic, or prognostic). Second, designing the AI algorithm to minimize the most clinically important errors. For example, many equivocal cervical images cannot yet be labeled because the borderline between cases and controls is blurred. To avoid a misclassified case-control dichotomy, we have isolated the equivocal cases and formally included an intermediate, indeterminate class (severity order of classes: case>indeterminate>control). The third principle is evaluating AI algorithms like any other test, using clinical epidemiologic criteria. Repeatability of the algorithm at the borderline, for indeterminate images, has proven extremely informative. Distinguishing between internal and external validation is also essential. Linking the AI algorithm results to clinical risk estimation is the fourth principle. Absolute risk (not relative) is the critical metric for translating a test result into clinical use. Finally, generating risk-based guidelines for clinical use that match local resources and priorities is the last principle in our approach. We are particularly interested in applications to lower-resource settings to address health disparities. We note that similar principles apply to other domains of AI-based image analysis for medical diagnostic testing., (© The Author(s) 2023. Published by Oxford University Press.)

Purpose To describe the design, conduct, and results of the Breast Multiparametric MRI for prediction of neoadjuvant chemotherapy Response (BMMR2) challenge. Materials and Methods The BMMR2 computational challenge opened on May 28, 2021, and closed on December 21, 2021. The goal of the challenge was to identify image-based markers derived from multiparametric breast MRI, including diffusion-weighted imaging (DWI) and dynamic contrast-enhanced (DCE) MRI, along with clinical data for predicting pathologic complete response (pCR) following neoadjuvant treatment. Data included 573 breast MRI studies from 191 women (mean age [±SD], 48.9 years ± 10.56) in the I-SPY 2/American College of Radiology Imaging Network (ACRIN) 6698 trial (ClinicalTrials.gov: NCT01042379). The challenge cohort was split into training (60%) and test (40%) sets, with teams blinded to test set pCR outcomes. Prediction performance was evaluated by area under the receiver operating characteristic curve (AUC) and compared with the benchmark established from the ACRIN 6698 primary analysis. Results Eight teams submitted final predictions. Entries from three teams had point estimators of AUC that were higher than the benchmark performance (AUC, 0.782 [95% CI: 0.670, 0.893], with AUCs of 0.803 [95% CI: 0.702, 0.904], 0.838 [95% CI: 0.748, 0.928], and 0.840 [95% CI: 0.748, 0.932]). A variety of approaches were used, ranging from extraction of individual features to deep learning and artificial intelligence methods, incorporating DCE and DWI alone or in combination. Conclusion The BMMR2 challenge identified several models with high predictive performance, which may further expand the value of multiparametric breast MRI as an early marker of treatment response. Clinical trial registration no. NCT01042379 Keywords: MRI, Breast, Tumor Response Supplemental material is available for this article . © RSNA, 2024.

Objectives/purpose: The reproducibility and sensitivity of image-based colposcopy is low, but agreement on lesion presence and location remains to be explored. Here, we investigate the interobserver agreement on lesions on colposcopic images by evaluating and comparing marked lesions on digitized colposcopic images between colposcopists., Methods: Five colposcopists reviewed images from 268 colposcopic examinations. Cases were selected based on histologic diagnosis, i.e., normal/cervical intraepithelial neoplasia (CIN)1 ( n = 50), CIN2 ( n = 50), CIN3 ( n = 100), adenocarcinoma in situ ( n = 53), and cancer ( n = 15). We obtained digitized time-series images every 7-10 seconds from before acetic acid application to 2 minutes after application. Colposcopists were instructed to digitally annotate all areas with acetowhitening or suspect of lesions. To estimate the agreement on lesion presence and location, we assessed the proportion of images with annotations and the proportion of images with overlapping annotated area by at least 4 (4+) colposcopists, respectively., Results: We included images from 241 examinations (1 image from each) with adequate annotations. The proportion with a least 1 lesion annotated by 4+ colposcopists increased by severity of histologic diagnosis. Among the CIN3 cases, 84% had at least 1 lesion annotated by 4+ colposcopists, whereas 54% of normal/CIN1 cases had a lesion annotated. Notably, the proportion was 70% for adenocarcinoma in situ and 71% for cancer. Regarding lesion location, there was no linear association with severity of histologic diagnosis., Conclusion: Despite that 80% of the CIN2 and CIN3 cases were annotated by 4+ colposcopists, we did not find increasing agreement on lesion location with histology severity. This underlines the subjective nature of colposcopy., Competing Interests: The authors have declared they have no conflicts of interest., (Copyright © 2023 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the ASCCP.)

Purpose To present results from a literature survey on practices in deep learning segmentation algorithm evaluation and perform a study on expert quality perception of brain tumor segmentation. Materials and Methods A total of 180 articles reporting on brain tumor segmentation algorithms were surveyed for the reported quality evaluation. Additionally, ratings of segmentation quality on a four-point scale were collected from medical professionals for 60 brain tumor segmentation cases. Results Of the surveyed articles, Dice score, sensitivity, and Hausdorff distance were the most popular metrics to report segmentation performance. Notably, only 2.8% of the articles included clinical experts' evaluation of segmentation quality. The experimental results revealed a low interrater agreement (Krippendorff α, 0.34) in experts' segmentation quality perception. Furthermore, the correlations between the ratings and commonly used quantitative quality metrics were low (Kendall tau between Dice score and mean rating, 0.23; Kendall tau between Hausdorff distance and mean rating, 0.51), with large variability among the experts. Conclusion The results demonstrate that quality ratings are prone to variability due to the ambiguity of tumor boundaries and individual perceptual differences, and existing metrics do not capture the clinical perception of segmentation quality. Keywords: Brain Tumor Segmentation, Deep Learning Algorithms, Glioblastoma, Cancer, Machine Learning Clinical trial registration nos. NCT00756106 and NCT00662506 Supplemental material is available for this article. © RSNA, 2023.

Cervical cancer is a leading cause of cancer mortality, with approximately 90% of the 250,000 deaths per year occurring in low- and middle-income countries (LMIC). Secondary prevention with cervical screening involves detecting and treating precursor lesions; however, scaling screening efforts in LMIC has been hampered by infrastructure and cost constraints. Recent work has supported the development of an artificial intelligence (AI) pipeline on digital images of the cervix to achieve an accurate and reliable diagnosis of treatable precancerous lesions. In particular, WHO guidelines emphasize visual triage of women testing positive for human papillomavirus (HPV) as the primary screen, and AI could assist in this triage task. In this work, we implemented a comprehensive deep-learning model selection and optimization study on a large, collated, multi-geography, multi-institution, and multi-device dataset of 9462 women (17,013 images). We evaluated relative portability, repeatability, and classification performance. The top performing model, when combined with HPV type, achieved an area under the Receiver Operating Characteristics (ROC) curve (AUC) of 0.89 within our study population of interest, and a limited total extreme misclassification rate of 3.4%, on held-aside test sets. Our model also produced reliable and consistent predictions, achieving a strong quadratic weighted kappa (QWK) of 0.86 and a minimal %2-class disagreement (% 2-Cl. D.) of 0.69%, between image pairs across women. Our work is among the first efforts at designing a robust, repeatable, accurate and clinically translatable deep-learning model for cervical screening., (© 2023. The Author(s).)

Objective: Functional and morphologic changes in extracranial organs can occur after acute brain injury. The neuroanatomic correlates of such changes are not fully known. Herein, we tested the hypothesis that brain infarcts are associated with cardiac and systemic abnormalities (CSAs) in a regionally specific manner., Methods: We generated voxelwise p value maps of brain infarcts for poststroke plasma cardiac troponin T (cTnT) elevation, QTc prolongation, in-hospital infection, and acute stress hyperglycemia (ASH) in 1,208 acute ischemic stroke patients prospectively recruited into the Heart-Brain Interactions Study. We examined the relationship between infarct location and CSAs using a permutation-based approach and identified clusters of contiguous voxels associated with p < 0.05., Results: cTnT elevation not attributable to a known cardiac reason was detected in 5.5%, QTc prolongation in the absence of a known provoker in 21.2%, ASH in 33.9%, and poststroke infection in 13.6%. We identified significant, spatially segregated voxel clusters for each CSA. The clusters for troponin elevation and QTc prolongation mapped to the right hemisphere. There were 3 clusters for ASH, the largest of which was in the left hemisphere. We found 2 clusters for poststroke infection, one associated with pneumonia in the left and one with urinary tract infection in the right hemisphere. The relationship between infarct location and CSAs persisted after adjusting for infarct volume., Interpretation: Our results show that there are discrete regions of brain infarcts associated with CSAs. This information could be used to bootstrap toward new markers for better differentiation between neurogenic and non-neurogenic mechanisms of poststroke CSAs. ANN NEUROL 2023;94:1155-1163., (© 2023 American Neurological Association.)

Competing Interests: Disclosures of conflicts of interest: D.K. Institutional research grants from iCAD, GenMab, and Calico; patent planned, issued, or pending (US8634610B2); deputy editor of Radiology: Artificial Intelligence. J.K.C. Grants/contracts from Genentech, GE, and Bayer; iROP DL software license to Siloam; stock/stock options in Siloam Vision; deputy editor of Radiology: Artificial Intelligence.

Purpose: The Medical Imaging and Data Resource Center (MIDRC) open data commons was launched to accelerate the development of artificial intelligence (AI) algorithms to help address the COVID-19 pandemic. The purpose of this study was to quantify longitudinal representativeness of the demographic characteristics of the primary MIDRC dataset compared to the United States general population (US Census) and COVID-19 positive case counts from the Centers for Disease Control and Prevention (CDC)., Approach: The Jensen-Shannon distance (JSD), a measure of similarity of two distributions, was used to longitudinally measure the representativeness of the distribution of (1) all unique patients in the MIDRC data to the 2020 US Census and (2) all unique COVID-19 positive patients in the MIDRC data to the case counts reported by the CDC. The distributions were evaluated in the demographic categories of age at index, sex, race, ethnicity, and the combination of race and ethnicity., Results: Representativeness of the MIDRC data by ethnicity and the combination of race and ethnicity was impacted by the percentage of CDC case counts for which this was not reported. The distributions by sex and race have retained their level of representativeness over time., Conclusion: The representativeness of the open medical imaging datasets in the curated public data commons at MIDRC has evolved over time as the number of contributing institutions and overall number of subjects have grown. The use of metrics, such as the JSD support measurement of representativeness, is one step needed for fair and generalizable AI algorithm development., (© 2023 The Authors.)

Purpose: To recognize and address various sources of bias essential for algorithmic fairness and trustworthiness and to contribute to a just and equitable deployment of AI in medical imaging, there is an increasing interest in developing medical imaging-based machine learning methods, also known as medical imaging artificial intelligence (AI), for the detection, diagnosis, prognosis, and risk assessment of disease with the goal of clinical implementation. These tools are intended to help improve traditional human decision-making in medical imaging. However, biases introduced in the steps toward clinical deployment may impede their intended function, potentially exacerbating inequities. Specifically, medical imaging AI can propagate or amplify biases introduced in the many steps from model inception to deployment, resulting in a systematic difference in the treatment of different groups., Approach: Our multi-institutional team included medical physicists, medical imaging artificial intelligence/machine learning (AI/ML) researchers, experts in AI/ML bias, statisticians, physicians, and scientists from regulatory bodies. We identified sources of bias in AI/ML, mitigation strategies for these biases, and developed recommendations for best practices in medical imaging AI/ML development., Results: Five main steps along the roadmap of medical imaging AI/ML were identified: (1) data collection, (2) data preparation and annotation, (3) model development, (4) model evaluation, and (5) model deployment. Within these steps, or bias categories, we identified 29 sources of potential bias, many of which can impact multiple steps, as well as mitigation strategies., Conclusions: Our findings provide a valuable resource to researchers, clinicians, and the public at large., (© 2023 The Authors.)

Purpose: Retinopathy of prematurity (ROP) is one of the leading causes of blindness in children. Although the role of oxygen in the pathophysiology of ROP is well established, a precise understanding of the dynamic relationship between oxygen exposure ROP incidence and severity is lacking. The purpose of this study was to evaluate the correlation between time-dependent oxygen variables and the onset of ROP., Design: Retrospective cohort study., Participants: Two hundred thirty infants who were born at a single academic center and met the inclusion criteria were included. Infants are mainly born between January 2011 and October 2022., Methods: Patient data were extracted from electronic health records (EHRs), with sufficient time-dependent oxygen data. Clinical outcomes for ROP were recorded as none/mild or moderate/severe (defined as type II or worse). Mixed-effects linear models were used to compare the 2 groups in terms of dynamic oxygen variables, such as daily average and the coefficient of variation (COV) fraction of inspired oxygen (FiO 2 ). Support vector machine (SVM) and long-short-term memory (LSTM)-based multimodal models were trained with fivefold cross-validation to predict which infants would develop moderate/severe ROP. Gestational age (GA), birth weight, and time-dependent oxygen variables were used to develop predictive models., Main Outcome Measures: Model cross-validation performance was evaluated by computing the mean area under the receiver operating characteristic (AUROC) curve, precision, recall, and F1 score., Results: We found that both daily average and COV of FiO 2 were associated with more severe ROP (adjusted P  < 0.001). With fivefold cross-validation, the multimodal LSTM models had higher performance than the best static models (SVM using GA and 3 average FiO 2 features) and SVM models trained on GA alone (mean AUROC = 0.89 ± 0.04 vs. 0.86 ± 0.05 vs. 0.83 ± 0.04)., Conclusions: The development of severe ROP might not only be influenced by oxygen exposure but also by its fluctuation, which provides direction for future study of pathophysiological factors associated with severe ROP development. Additionally, we demonstrated that multimodal neural networks can be a method to extract useful information from time-series data, which may be a valuable methodology for the investigation of other diseases using EHR data., Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article., (© 2023 Published by Elsevier Inc. on behalf of the American Academy of Ophthalmology.)

The rapid progress of large language models (LLMs) driving generative artificial intelligence applications heralds the potential of opportunities in health care. We conducted a review up to April 2023 on Google Scholar, Embase, MEDLINE, and Scopus using the following terms: "large language models," "generative artificial intelligence," "ophthalmology," "ChatGPT," and "eye," based on relevance to this review. From a clinical viewpoint specific to ophthalmologists, we explore from the different stakeholders' perspectives-including patients, physicians, and policymakers-the potential LLM applications in education, research, and clinical domains specific to ophthalmology. We also highlight the foreseeable challenges of LLM implementation into clinical practice, including the concerns of accuracy, interpretability, perpetuating bias, and data security. As LLMs continue to mature, it is essential for stakeholders to jointly establish standards for best practices to safeguard patient safety., Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article., (© 2023 by the American Academy of Ophthalmology.)

Objectives: Current state-of-the-art natural language processing (NLP) techniques use transformer deep-learning architectures, which depend on large training datasets. We hypothesized that traditional NLP techniques may outperform transformers for smaller radiology report datasets., Methods: We compared the performance of BioBERT, a deep-learning-based transformer model pre-trained on biomedical text, and three traditional machine-learning models (gradient boosted tree, random forest, and logistic regression) on seven classification tasks given free-text radiology reports. Tasks included detection of appendicitis, diverticulitis, bowel obstruction, and enteritis/colitis on abdomen/pelvis CT reports, ischemic infarct on brain CT/MRI reports, and medial and lateral meniscus tears on knee MRI reports (7,204 total annotated reports). The performance of NLP models on held-out test sets was compared after training using the full training set, and 2.5%, 10%, 25%, 50%, and 75% random subsets of the training data., Results: In all tested classification tasks, BioBERT performed poorly at smaller training sample sizes compared to non-deep-learning NLP models. Specifically, BioBERT required training on approximately 1,000 reports to perform similarly or better than non-deep-learning models. At around 1,250 to 1,500 training samples, the testing performance for all models began to plateau, where additional training data yielded minimal performance gain., Conclusions: With larger sample sizes, transformer NLP models achieved superior performance in radiology report binary classification tasks. However, with smaller sizes (<1000) and more imbalanced training data, traditional NLP techniques performed better., Advances in Knowledge: Our benchmarks can help guide clinical NLP researchers in selecting machine-learning models according to their dataset characteristics., Competing Interests: Declaration of interestsMDS reports personal fees and non-financial support from 2 Minute Medicine, Inc., and patent royalties from Frequency Therapeutics for work not related to this manuscript. JKC reports grants from GE Healthcare, non-financial support from AWS, and grants from Genentech Foundation, outside the submitted work. The other authors report no relevant conflicts of interest.

Background: Contrast-induced nephropathy (CIN) is a postprocedural complication associated with increased morbidity and mortality. An important risk factor for development of CIN is renal impairment. Identification of patients at risk for acute renal failure will allow physicians to make appropriate decisions to minimize the incidence of CIN. We developed a machine learning model to stratify risk of acute renal failure that may assist in mitigating risk for CIN in patients with peripheral artery disease (PAD) undergoing endovascular interventions., Methods: We utilized the American College of Surgeons National Surgical Quality Improvement Program database to extract clinical and laboratory information associated with 14,444 patients who underwent lower extremity endovascular procedures between 2011 and 2018. Using 11,604 cases from 2011 to 2017 for training and 2840 cases from 2018 for testing, we developed a random forest model to predict risk of 30-day acute renal failure following infra-inguinal endovascular procedures., Results: Eight variables were identified as contributing optimally to model predictions, the most important being diabetes, preoperative BUN, and claudication. Using these variables, the model achieved an area under the receiver-operating characteristic (AU-ROC) curve of 0.81, accuracy of 0.83, sensitivity of 0.67, and specificity of 0.74. The model performed equally well on white and nonwhite patients (Delong p-value = 0.955) and patients age < 65 and patients age ≥ 65 (Delong p-value = 0.659)., Conclusions: We develop a model that fairly and accurately stratifies 30-day acute renal failure risk in patients undergoing lower extremity endovascular procedures for PAD. This model may assist in identifying patients who may benefit from strategies to prevent CIN., Competing Interests: Declaration of competing interest S.K. reports receiving grants or contracts from BD and AngioDynamics. He has also received consulting fees from Penumbra, Okami Medical, Boston Scientific, Medtronic, and US Vascular. D.D. reports receiving consulting fees from Medtronic. The remaining authors declare no competing interests., (Copyright © 2023 Elsevier Inc. All rights reserved.)

This dataset is composed of cervical spine CT images with annotations related to fractures; it is available at https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/., Competing Interests: Disclosures of conflicts of interest: H.M.L. No relevant relationships. E.C. No relevant relationships. T.R. No relevant relationships. F.C.K. Consultant for MD.ai and GE HealthCare; member of the Radiological Society of North America (RSNA) Machine Learning Steering Committee member and the Society for Imaging Informatics in Medicine Machine Learning Education Subcommittee (both unpaid). L.M.P. Associate editor for Radiology: Artificial Intelligence; patents planned, issued, or pending: US-20220051060-A1, “Methods for creating privacy-protecting synthetic data leveraging a constrained generative ensemble model,” and US-20220051402-A1, “Systems for automated lesion detection and related methods.” J.T. Provided expert witness deposition for Phillips, Spallas, & Angstadt in October 2022, unrelated to this article. R.L.B. Support from RSNA to author. E.G. No relevant relationships. K.W.Y. No relevant relationships. M.H. No relevant relationships. A.L.S. Author’s lab receives funding from the National Institutes of Health (NIH); member of the advisory board for the National Cancer Institute Imaging Data Commons. J.S. No relevant relationships. D.B. No relevant relationships. S.A. No relevant relationships. A.P.L. No relevant relationships. M.I.G.A. No relevant relationships. J.O.J. No relevant relationships. J.J.P. Author’s lab receives funding from the NIH, which pays this author’s salary. M.L. No relevant relationships. H.D. No relevant relationships. E.A. No relevant relationships. A.Y. No relevant relationships. Y.M. No relevant relationships. J.K.C. Grants or contracts from GE HealthCare and Genentech; technology licensed to Boston AI; consulting fees from Siloam Vision; deputy editor of Radiology: Artificial Intelligence. A.E.F. Standing director, liaison for information technology, of RSNA board of directors; member of RSNA News editorial board., (© 2023 by the Radiological Society of North America, Inc.)

Purpose: We evaluated the efficacy of bavituximab-a mAb with anti-angiogenic and immunomodulatory properties-in newly diagnosed patients with glioblastoma (GBM) who also received radiotherapy and temozolomide. Perfusion MRI and myeloid-related gene transcription and inflammatory infiltrates in pre-and post-treatment tumor specimens were studied to evaluate on-target effects (NCT03139916)., Patients and Methods: Thirty-three adults with IDH--wild-type GBM received 6 weeks of concurrent chemoradiotherapy, followed by 6 cycles of temozolomide (C1-C6). Bavituximab was given weekly, starting week 1 of chemoradiotherapy, for at least 18 weeks. The primary endpoint was proportion of patients alive at 12 months (OS-12). The null hypothesis would be rejected if OS-12 was ≥72%. Relative cerebral blood flow (rCBF) and vascular permeability (Ktrans) were calculated from perfusion MRIs. Peripheral blood mononuclear cells and tumor tissue were analyzed pre-treatment and at disease progression using RNA transcriptomics and multispectral immunofluorescence for myeloid-derived suppressor cells (MDSC) and macrophages., Results: The study met its primary endpoint with an OS-12 of 73% (95% confidence interval, 59%-90%). Decreased pre-C1 rCBF (HR, 4.63; P = 0.029) and increased pre-C1 Ktrans were associated with improved overall survival (HR, 0.09; P = 0.005). Pre-treatment overexpression of myeloid-related genes in tumor tissue was associated with longer survival. Post-treatment tumor specimens contained fewer immunosuppressive MDSCs (P = 0.01)., Conclusions: Bavituximab has activity in newly diagnosed GBM and resulted in on-target depletion of intratumoral immunosuppressive MDSCs. Elevated pre-treatment expression of myeloid-related transcripts in GBM may predict response to bavituximab., (©2023 American Association for Cancer Research.)

Purpose: Epidemiological changes in retinopathy of prematurity (ROP) depend on neonatal care, neonatal mortality, and the ability to carefully titrate and monitor oxygen. We evaluate whether an artificial intelligence (AI) algorithm for assessing ROP severity in babies can be used to evaluate changes in disease epidemiology in babies from South India over a 5-year period., Design: Retrospective cohort study., Participants: Babies (3093) screened for ROP at neonatal care units (NCUs) across the Aravind Eye Care System (AECS) in South India., Methods: Images and clinical data were collected as part of routine tele-ROP screening at the AECS in India over 2 time periods: August 2015 to October 2017 and March 2019 to December 2020. All babies in the original cohort were matched 1:3 by birthweight (BW) and gestational age (GA) with babies in the later cohort. We compared the proportion of eyes with moderate (type 2) or treatment-requiring (TR) ROP, and an AI-derived ROP vascular severity score (from retinal fundus images) at the initial tele-retinal screening exam for all babies in a district, VSS), in the 2 time periods., Main Outcome Measures: Differences in the proportions of type 2 or worse and TR-ROP cases, and VSS between time periods., Results: Among BW and GA matched babies, the proportion [95% confidence interval {CI}] of babies with type 2 or worse and TR-ROP decreased from 60.9% [53.8%-67.7%] to 17.1% [14.0%-20.5%] (P < 0.001) and 16.8% [11.9%-22.7%] to 5.1% [3.4%-7.3%] (P < 0.001), over the 2 time periods. Similarly, the median [interquartile range] VSS in the population decreased from 2.9 [1.2] to 2.4 [1.8] (P < 0.001)., Conclusions: In South India, over a 5-year period, the proportion of babies developing moderate to severe ROP has dropped significantly for babies at similar demographic risk, strongly suggesting improvements in primary prevention of ROP. These results suggest that AI-based assessment of ROP severity may be a useful epidemiologic tool to evaluate temporal changes in ROP epidemiology., Financial Disclosure(s): Proprietary or commercial disclosure may be found after the references., (Copyright © 2023 American Academy of Ophthalmology. All rights reserved.)

Near-infrared diffuse optical tomography (DOT) is a promising functional modality for breast cancer imaging; however, the clinical translation of DOT is hampered by technical limitations. Specifically, conventional finite element method (FEM)-based optical image reconstruction approaches are time-consuming and ineffective in recovering full lesion contrast. To address this, we developed a deep learning-based reconstruction model (FDU-Net) comprised of a Fully connected subnet, followed by a convolutional encoder-Decoder subnet, and a U-Net for fast, end-to-end 3D DOT image reconstruction. The FDU-Net was trained on digital phantoms that include randomly located singular spherical inclusions of various sizes and contrasts. Reconstruction performance was evaluated in 400 simulated cases with realistic noise profiles for the FDU-Net and conventional FEM approaches. Our results show that the overall quality of images reconstructed by FDU-Net is significantly improved compared to FEM-based methods and a previously proposed deep-learning network. Importantly, once trained, FDU-Net demonstrates substantially better capability to recover true inclusion contrast and location without using any inclusion information during reconstruction. The model was also generalizable to multi-focal and irregularly shaped inclusions unseen during training. Finally, FDU-Net, trained on simulated data, could successfully reconstruct a breast tumor from a real patient measurement. Overall, our deep learning-based approach demonstrates marked superiority over the conventional DOT image reconstruction methods while also offering over four orders of magnitude acceleration in computational time. Once adapted to the clinical breast imaging workflow, FDU-Net has the potential to provide real-time accurate lesion characterization by DOT to assist the clinical diagnosis and management of breast cancer.

Importance: Retinopathy of prematurity (ROP) telemedicine screening programs have been found to be effective, but they rely on widefield digital fundus imaging (WDFI) cameras, which are expensive, making them less accessible in low- to middle-income countries. Cheaper, smartphone-based fundus imaging (SBFI) systems have been described, but these have a narrower field of view (FOV) and have not been tested in a real-world, operational telemedicine setting., Objective: To assess the efficacy of SBFI systems compared with WDFI when used by technicians for ROP screening with both artificial intelligence (AI) and human graders., Design, Setting, and Participants: This prospective cross-sectional comparison study took place as a single-center ROP teleophthalmology program in India from January 2021 to April 2022. Premature infants who met normal ROP screening criteria and enrolled in the teleophthalmology screening program were included. Those who had already been treated for ROP were excluded., Exposures: All participants had WDFI images and from 1 of 2 SBFI devices, the Make-In-India (MII) Retcam or Keeler Monocular Indirect Ophthalmoscope (MIO) devices. Two masked readers evaluated zone, stage, plus, and vascular severity scores (VSS, from 1-9) in all images. Smartphone images were then stratified by patient into training (70%), validation (10%), and test (20%) data sets and used to train a ResNet18 deep learning architecture for binary classification of normal vs preplus or plus disease, which was then used for patient-level predictions of referral warranted (RW)- and treatment requiring (TR)-ROP., Main Outcome and Measures: Sensitivity and specificity of detection of RW-ROP, and TR-ROP by both human graders and an AI system and area under the receiver operating characteristic curve (AUC) of grader-assigned VSS. Sensitivity and specificity were compared between the 2 SBFI systems using Pearson χ2testing., Results: A total of 156 infants (312 eyes; mean [SD] gestational age, 33.0 [3.0] weeks; 75 [48%] female) were included with paired examinations. Sensitivity and specificity were not found to be statistically different between the 2 SBFI systems. Human graders were effective with SBFI at detecting TR-ROP with a sensitivity of 100% and specificity of 83.49%. The AUCs with grader-assigned VSS only were 0.95 (95% CI, 0.91-0.99) and 0.96 (95% CI, 0.93-0.99) for RW-ROP and TR-ROP, respectively. For the AI system, the sensitivity of detecting TR-ROP sensitivity was 100% with specificity of 58.6%, and RW-ROP sensitivity was 80.0% with specificity of 59.3%., Conclusions and Relevance: In this cross-sectional study, 2 different SBFI systems used by technicians in an ROP screening program were highly sensitive for TR-ROP. SBFI systems with AI may be a cost-effective method to improve the global capacity for ROP screening.

Cervical cancer screening and management in the U.S. has adopted a risk-based approach. However, the majority of cervical cancer cases and deaths occur in resource-limited settings, where screening and management are not widely available. We describe a conceptual model that optimizes cervical cancer screening and management in resource-limited settings by utilizing a risk-based approach. The principles of risk-based screening and management in resource limited settings include (1) ensure that the screening method effectively separates low-risk from high-risk patients; (2) directing resources to populations at the highest cancer risk; (3) screen using HPV testing via self-sampling; (4) utilize HPV genotyping to improve risk stratification and better determine who will benefit from treatment, and (5) automated visual evaluation with artificial intelligence may further improve risk stratification. Risk-based screening and management in resource limited settings can optimize prevention by focusing triage and treatment resources on the highest risk patients while minimizing interventions in lower risk patients., Competing Interests: Declaration of Competing Interest No authors have a conflict of interest to declare related to this work., (Copyright © 2023. Published by Elsevier Ltd.)

Motion artifacts can negatively impact diagnosis, patient experience, and radiology workflow especially when a patient recall is required. Detecting motion artifacts while the patient is still in the scanner could potentially improve workflow and reduce costs by enabling immediate corrective action. We demonstrate in a clinical k-space dataset that using cross-correlation between adjacent phase-encoding lines can detect motion artifacts directly from raw k-space in multi-shot multi-slice scans. We train a split-attention residual network to examine the performance in predicting motion artifact severity. The network is trained on simulated data and tested on real clinical data.

Importance: Although race is a social construct, it is associated with variations in skin and retinal pigmentation. Image-based medical artificial intelligence (AI) algorithms that use images of these organs have the potential to learn features associated with self-reported race (SRR), which increases the risk of racially biased performance in diagnostic tasks; understanding whether this information can be removed, without affecting the performance of AI algorithms, is critical in reducing the risk of racial bias in medical AI., Objective: To evaluate whether converting color fundus photographs to retinal vessel maps (RVMs) of infants screened for retinopathy of prematurity (ROP) removes the risk for racial bias., Design, Setting, and Participants: The retinal fundus images (RFIs) of neonates with parent-reported Black or White race were collected for this study. A u-net, a convolutional neural network (CNN) that provides precise segmentation for biomedical images, was used to segment the major arteries and veins in RFIs into grayscale RVMs, which were subsequently thresholded, binarized, and/or skeletonized. CNNs were trained with patients' SRR labels on color RFIs, raw RVMs, and thresholded, binarized, or skeletonized RVMs. Study data were analyzed from July 1 to September 28, 2021., Main Outcomes and Measures: Area under the precision-recall curve (AUC-PR) and area under the receiver operating characteristic curve (AUROC) at both the image and eye level for classification of SRR., Results: A total of 4095 RFIs were collected from 245 neonates with parent-reported Black (94 [38.4%]; mean [SD] age, 27.2 [2.3] weeks; 55 majority sex [58.5%]) or White (151 [61.6%]; mean [SD] age, 27.6 [2.3] weeks, 80 majority sex [53.0%]) race. CNNs inferred SRR from RFIs nearly perfectly (image-level AUC-PR, 0.999; 95% CI, 0.999-1.000; infant-level AUC-PR, 1.000; 95% CI, 0.999-1.000). Raw RVMs were nearly as informative as color RFIs (image-level AUC-PR, 0.938; 95% CI, 0.926-0.950; infant-level AUC-PR, 0.995; 95% CI, 0.992-0.998). Ultimately, CNNs were able to learn whether RFIs or RVMs were from Black or White infants regardless of whether images contained color, vessel segmentation brightness differences were nullified, or vessel segmentation widths were uniform., Conclusions and Relevance: Results of this diagnostic study suggest that it can be very challenging to remove information relevant to SRR from fundus photographs. As a result, AI algorithms trained on fundus photographs have the potential for biased performance in practice, even if based on biomarkers rather than raw images. Regardless of the methodology used for training AI, evaluating performance in relevant subpopulations is critical.

Blood-brain barrier disruption marks the onset of cerebral adrenoleukodystrophy (CALD), a devastating cerebral demyelinating disease caused by loss of ABCD1 gene function. The underlying mechanism are not well understood, but evidence suggests that microvascular dysfunction is involved. We analyzed cerebral perfusion imaging in boys with CALD treated with autologous hematopoietic stem-cells transduced with the Lenti-D lentiviral vector that contains ABCD1 cDNA as part of a single group, open-label phase 2-3 safety and efficacy study (NCT01896102) and patients treated with allogeneic hematopoietic stem cell transplantation. We found widespread and sustained normalization of white matter permeability and microvascular flow. We demonstrate that ABCD1 functional bone marrow-derived cells can engraft in the cerebral vascular and perivascular space. Inverse correlation between gene dosage and lesion growth suggests that corrected cells contribute long-term to remodeling of brain microvascular function. Further studies are needed to explore the longevity of these effects., (© 2023. The Author(s).)

Background Radiomics is the extraction of predefined mathematic features from medical images for the prediction of variables of clinical interest. While some studies report superlative accuracy of radiomic machine learning (ML) models, the published methodology is often incomplete, and the results are rarely validated in external testing data sets. Purpose To characterize the type, prevalence, and statistical impact of methodologic errors present in radiomic ML studies. Materials and Methods Radiomic ML publications were reviewed for the presence of performance-inflating methodologic flaws. Common flaws were subsequently reproduced with randomly generated features interpolated from publicly available radiomic data sets to demonstrate the precarious nature of reported findings. Results In an assessment of radiomic ML publications, the authors uncovered two general categories of data analysis errors: inconsistent partitioning and unproductive feature associations. In simulations, the authors demonstrated that inconsistent partitioning augments radiomic ML accuracy by 1.4 times from unbiased performance and that correcting for flawed methodologic results in areas under the receiver operating characteristic curve approaching a value of 0.5 (random chance). With use of randomly generated features, the authors illustrated that unproductive associations between radiomic features and gene sets can imply false causality for biologic phenomenon. Conclusion Radiomic machine learning studies may contain methodologic flaws that undermine their validity. This study provides a review template to avoid such flaws. © RSNA, 2022 Supplemental material is available for this article . See also the editorial by Jacobs in this issue.

Deep learning (DL) algorithms have shown remarkable potential in automating various tasks in medical imaging and radiologic reporting. However, models trained on low quantities of data or only using data from a single institution often are not generalizable to other institutions, which may have different patient demographics or data acquisition characteristics. Therefore, training DL algorithms using data from multiple institutions is crucial to improving the robustness and generalizability of clinically useful DL models. In the context of medical data, simply pooling data from each institution to a central location to train a model poses several issues such as increased risk to patient privacy, increased costs for data storage and transfer, and regulatory challenges. These challenges of centrally hosting data have motivated the development of distributed machine learning techniques and frameworks for collaborative learning that facilitate the training of DL models without the need to explicitly share private medical data. The authors describe several popular methods for collaborative training and review the main considerations for deploying these models. They also highlight publicly available software frameworks for federated learning and showcase several real-world examples of collaborative learning. The authors conclude by discussing some key challenges and future research directions for distributed DL. They aim to introduce clinicians to the benefits, limitations, and risks of using distributed DL for the development of medical artificial intelligence algorithms. © RSNA, 2023 Quiz questions for this article are available in the supplemental material.