Search

Background Limited statistical knowledge can slow critical engagement with and adoption of artificial intelligence (AI) tools for radiologists. Large language models (LLMs) such as OpenAI's GPT-4, and notably its Advanced Data Analysis (ADA) extension, may improve the adoption of AI in radiology. Purpose To validate GPT-4 ADA outputs when autonomously conducting analyses of varying complexity on a multisource clinical dataset. Materials and Methods In this retrospective study, unique itemized radiologic reports of bedside chest radiographs, associated demographic data, and laboratory markers of inflammation from patients in intensive care from January 2009 to December 2019 were evaluated. GPT-4 ADA, accessed between December 2023 and January 2024, was tasked with autonomously analyzing this dataset by plotting radiography usage rates, providing descriptive statistics measures, quantifying factors of pulmonary opacities, and setting up machine learning (ML) models to predict their presence. Three scientists with 6-10 years of ML experience validated the outputs by verifying the methodology, assessing coding quality, re-executing the provided code, and comparing ML models head-to-head with their human-developed counterparts (based on the area under the receiver operating characteristic curve [AUC], accuracy, sensitivity, and specificity). Statistical significance was evaluated using bootstrapping. Results A total of 43 788 radiograph reports, with their laboratory values, from University Hospital RWTH Aachen were evaluated from 43 788 patients (mean age, 66 years ± 15 [SD]; 26 804 male). While GPT-4 ADA provided largely appropriate visualizations, descriptive statistical measures, quantitative statistical associations based on logistic regression, and gradient boosting machines for the predictive task (AUC, 0.75), some statistical errors and inaccuracies were encountered. ML strategies were valid and based on consistent coding routines, resulting in valid outputs on par with human specialist-developed reference models (AUC, 0.80 [95% CI: 0.80, 0.81] vs 0.80 [95% CI: 0.80, 0.81]; P = .51) (accuracy, 79% [6910 of 8758 patients] vs 78% [6875 of 8758 patients], respectively; P = .27). Conclusion LLMs may facilitate data analysis in radiology, from basic statistics to advanced ML-based predictive modeling. © RSNA, 2024 Supplemental material is available for this article.

Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the weights of the LLM, we can deliberately inject incorrect biomedical facts. The erroneous information is then propagated in the model's output while maintaining performance on other biomedical tasks. We validate our findings in a set of 1025 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice., (© 2024. The Author(s).)

Objectives: ChatGPT-4 Vision (GPT-4V) is a state-of-the-art multimodal large language model (LLM) that may be queried using images. We aimed to evaluate the tool's diagnostic performance when autonomously assessing clinical imaging studies., Materials and Methods: A total of 206 imaging studies (i.e., radiography (n = 60), CT (n = 60), MRI (n = 60), and angiography (n = 26)) with unequivocal findings and established reference diagnoses from the radiologic practice of a large university hospital were accessed. Readings were performed uncontextualized, with only the image provided, and contextualized, with additional clinical and demographic information. Responses were assessed along multiple diagnostic dimensions and analyzed using appropriate statistical tests., Results: With its pronounced propensity to favor context over image information, the tool's diagnostic accuracy improved from 8.3% (uncontextualized) to 29.1% (contextualized, first diagnosis correct) and 63.6% (contextualized, correct diagnosis among differential diagnoses) (p ≤ 0.001, Cochran's Q test). Diagnostic accuracy declined by up to 30% when 20 images were re-read after 30 and 90 days and seemed unrelated to the tool's self-reported confidence (Spearman's ρ = 0.117 (p = 0.776)). While the described imaging findings matched the suggested diagnoses in 92.7%, indicating valid diagnostic reasoning, the tool fabricated 258 imaging findings in 412 responses and misidentified imaging modalities or anatomic regions in 65 images., Conclusion: GPT-4V, in its current form, cannot reliably interpret radiologic images. Its tendency to disregard the image, fabricate findings, and misidentify details, especially without clinical context, may misguide healthcare providers and put patients at risk., Key Points: Question Can Generative Pre-trained Transformer 4 Vision (GPT-4V) interpret radiologic images-with and without clinical context? Findings GPT-4V performed poorly, demonstrating diagnostic accuracy rates of 8% (uncontextualized), 29% (contextualized, most likely diagnosis correct), and 64% (contextualized, correct diagnosis among differential diagnoses). Clinical relevance The utility of commercial multimodal large language models, such as GPT-4V, in radiologic practice is limited. Without clinical context, diagnostic errors and fabricated findings may compromise patient safety and misguide clinical decision-making. These models must be further refined to be beneficial., (© 2024. The Author(s).)

Objectives: Large language models (LLMs) have shown potential in radiology, but their ability to aid radiologists in interpreting imaging studies remains unexplored. We investigated the effects of a state-of-the-art LLM (GPT-4) on the radiologists' diagnostic workflow., Materials and Methods: In this retrospective study, six radiologists of different experience levels read 40 selected radiographic [n = 10], CT [n = 10], MRI [n = 10], and angiographic [n = 10] studies unassisted (session one) and assisted by GPT-4 (session two). Each imaging study was presented with demographic data, the chief complaint, and associated symptoms, and diagnoses were registered using an online survey tool. The impact of Artificial Intelligence (AI) on diagnostic accuracy, confidence, user experience, input prompts, and generated responses was assessed. False information was registered. Linear mixed-effect models were used to quantify the factors (fixed: experience, modality, AI assistance; random: radiologist) influencing diagnostic accuracy and confidence., Results: When assessing if the correct diagnosis was among the top-3 differential diagnoses, diagnostic accuracy improved slightly from 181/240 (75.4%, unassisted) to 188/240 (78.3%, AI-assisted). Similar improvements were found when only the top differential diagnosis was considered. AI assistance was used in 77.5% of the readings. Three hundred nine prompts were generated, primarily involving differential diagnoses (59.1%) and imaging features of specific conditions (27.5%). Diagnostic confidence was significantly higher when readings were AI-assisted (p > 0.001). Twenty-three responses (7.4%) were classified as hallucinations, while two (0.6%) were misinterpretations., Conclusion: Integrating GPT-4 in the diagnostic process improved diagnostic accuracy slightly and diagnostic confidence significantly. Potentially harmful hallucinations and misinterpretations call for caution and highlight the need for further safeguarding measures., Clinical Relevance Statement: Using GPT-4 as a virtual assistant when reading images made six radiologists of different experience levels feel more confident and provide more accurate diagnoses; yet, GPT-4 gave factually incorrect and potentially harmful information in 7.4% of its responses., (© 2024. The Author(s).)

Background: Wound healing complications are a major challenge following the extended lateral approach in calcaneal fractures. Soft-tissue microcirculation plays an important role via the delivery of oxygen, nutrients, and the regulation of a local milieu. The aim of this clinical study was to examine the effect of intraoperative pneumatic tourniquet application on skin and subcutaneous microcirculation, and its impact on wound healing progression., Methods: Patients with calcaneal fractures were randomly assigned to two groups defined by a surgery conducted either with use or without use of a tourniquet. Blood flow (BF [AU]), tissue oxygen saturation (SO 2 [%]) and the relative amount of haemoglobin (rHb[AU]) were intraoperatively measured at two depths (2 and 8 mm) non-invasively by spectrophotometry (Micro-Lightguide O2C ® , LEA Medizintechnik, Giessen, Germany). Time points were before and after inflation of the pneumatic tourniquet and also at the end of surgery before deflation. A linear mixed model (LMM) was fitted for statistical analysis., Results: Thirty-four patients (3 women and 31 men) with 37 calcaneal fractures were included. In 22 of them, the surgery was conducted with a tourniquet and in the other 15 without its use. A significant decrease of microcirculation, characterized by decreases in blood flow (p = 0.011) and tissue oxygenation (p = 0.023) was measured in 8 mm depth after inflating the tourniquet. However, these changes did not influence the time of postoperative wound healing., Conclusion: The use of a pneumatic tourniquet reduces deep microcirculation without affecting postoperative wound healing. Trial registration The study was registered in www., Clinicaltrials: gov (NCT01264146)., (© 2024. The Author(s).)

Reliably detecting potentially misleading patterns in automated diagnostic assistance systems, such as those powered by artificial intelligence (AI), is crucial for instilling user trust and ensuring reliability. Current techniques fall short in visualizing such confounding factors. We propose DiffChest, a self-conditioned diffusion model trained on 515,704 chest radiographs from 194,956 patients across the US and Europe. DiffChest provides patient-specific explanations and visualizes confounding factors that might mislead the model. The high inter-reader agreement, with Fleiss' kappa values of 0.8 or higher, validates its capability to identify treatment-related confounders. Confounders are accurately detected with 10%-100% prevalence rates. The pretraining process optimizes the model for relevant imaging information, resulting in excellent diagnostic accuracy for 11 chest conditions, including pleural effusion and heart insufficiency. Our findings highlight the potential of diffusion models in medical image classification, providing insights into confounding factors and enhancing model robustness and reliability., Competing Interests: Declaration of interests J.N.K. declares consulting services for Owkin, France; DoMore Diagnostics, Norway; Panakeia, UK; and Scailyte, Basel, Switzerland. Furthermore, J.N.K. holds shares in Kather Consulting, Dresden, Germany and StratifAI GmbH, Dresden, Germany and has received honoraria for lectures and advisory board participation by AstraZeneca, Bayer, Eisai, MSD, BMS, Roche, Pfizer, and Fresenius. D.T. received honoraria for lectures by Bayer and holds shares in StratifAI GmbH, Germany., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: To investigate the use of the score-based diffusion model to accelerate breast MRI reconstruction., Materials and Methods: We trained a score-based model on 9549 MRI examinations of the female breast and employed it to reconstruct undersampled MRI images with undersampling factors of 2, 5, and 20. Images were evaluated by two experienced radiologists who rated the images based on their overall quality and diagnostic value on an independent test set of 100 additional MRI examinations., Results: The score-based model produces MRI images of high quality and diagnostic value. Both T1- and T2-weighted MRI images could be reconstructed to a high degree of accuracy. Two radiologists rated the images as almost indistinguishable from the original images (rating 4 or 5 on a scale of 5) in 100% (radiologist 1) and 99% (radiologist 2) of cases when the acceleration factor was 2. This fraction dropped to 88% and 70% for an acceleration factor of 5 and to 5% and 21% with an extreme acceleration factor of 20., Conclusion: Score-based models can reconstruct MRI images at high fidelity, even at comparatively high acceleration factors, but further work on a larger scale of images is needed to ensure that diagnostic quality holds., Clinical Relevance Statement: The number of MRI examinations of the breast is expected to rise with MRI screening recommended for women with dense breasts. Accelerated image acquisition methods can help in making this examination more accessible., Key Points: Accelerating breast MRI reconstruction remains a significant challenge in clinical settings. Score-based diffusion models can achieve near-perfect reconstruction for moderate undersampling factors. Faster breast MRI scans with maintained image quality could revolutionize clinic workflows and patient experience., (© 2024. The Author(s).)

Background The level of background parenchymal enhancement (BPE) at breast MRI provides predictive and prognostic information and can have diagnostic implications. However, there is a lack of standardization regarding BPE assessment. Purpose To investigate how well results of quantitative BPE assessment methods correlate among themselves and with assessments made by radiologists experienced in breast MRI. Materials and Methods In this pseudoprospective analysis of 5773 breast MRI examinations from 3207 patients (mean age, 60 years ± 10 [SD]), the level of BPE was prospectively categorized according to the Breast Imaging Reporting and Data System by radiologists experienced in breast MRI. For automated extraction of BPE, fibroglandular tissue (FGT) was segmented in an automated pipeline. Four different published methods for automated quantitative BPE extractions were used: two methods (A and B) based on enhancement intensity and two methods (C and D) based on the volume of enhanced FGT. The results from all methods were correlated, and agreement was investigated in comparison with the respective radiologist-based categorization. For surrogate validation of BPE assessment, how accurately the methods distinguished premenopausal women with ( n = 50) versus without ( n = 896) antihormonal treatment was determined. Results Intensity-based methods (A and B) exhibited a correlation with radiologist-based categorization of 0.56 ± 0.01 and 0.55 ± 0.01, respectively, and volume-based methods (C and D) had a correlation of 0.52 ± 0.01 and 0.50 ± 0.01 ( P < .001). There were notable correlation differences ( P < .001) between the BPE determined with the four methods. Among the four quantitation methods, method D offered the highest accuracy for distinguishing women with versus without antihormonal therapy ( P = .01). Conclusion Results of different methods for quantitative BPE assessment agree only moderately among themselves or with visual categories reported by experienced radiologists; intensity-based methods correlate more closely with radiologists' ratings than volume-based methods. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Mann in this issue.

Background: Quantitative techniques such as T2 and T1ρ mapping allow evaluating the cartilage and meniscus. We evaluated multi-interleaved X-prepared turbo-spin echo with intuitive relaxometry (MIXTURE) sequences with turbo spin-echo (TSE) contrast and additional parameter maps versus reference TSE sequences in an in situ model of human cartilage defects., Methods: Standardized cartilage defects of 8, 5, and 3 mm in diameter were created in the lateral femora of ten human cadaveric knee specimens (81 ± 10 years old; nine males, one female). MIXTURE sequences providing proton density-weighted fat-saturated images and T2 maps or T1-weighted images and T1ρ maps as well as the corresponding two- and three-dimensional TSE reference sequences were acquired before and after defect creation (3-T scanner; knee coil). Defect delineability, bone texture, and cartilage relaxation times were quantified. Appropriate parametric or non-parametric tests were used., Results: Overall, defect delineability and texture features were not significantly different between the MIXTURE and reference sequences (p ≤ 0.47). After defect creation, relaxation times significantly increased in the central femur (T2 pre = 51 ± 4 ms [mean ± standard deviation] versus T2 post = 56 ± 4 ms; p = 0.002) and all regions combined (T1ρ pre = 40 ± 4 ms versus T1ρ post = 43 ± 4 ms; p = 0.004)., Conclusions: MIXTURE permitted time-efficient simultaneous morphologic and quantitative joint assessment based on clinical image contrasts. While providing T2 or T1ρ maps in clinically feasible scan time, morphologic image features, i.e., cartilage defects and bone texture, were comparable between MIXTURE and reference sequences., Relevance Statement: Equally time-efficient and versatile, the MIXTURE sequence platform combines morphologic imaging using familiar contrasts, excellent image correspondence versus corresponding reference sequences and quantitative mapping information, thereby increasing the diagnostic value beyond mere morphology., Key Points: • Combined morphologic and quantitative MIXTURE sequences are based on three-dimensional TSE contrasts. • MIXTURE sequences were studied in an in situ human cartilage defect model. • Morphologic image features, i.e., defect delineabilty and bone texture, were investigated. • Morphologic image features were similar between MIXTURE and reference sequences. • MIXTURE allowed time-efficient simultaneous morphologic and quantitative knee joint assessment., (© 2024. The Author(s).)

Quantitative MRI techniques such as T2 and T1ρ mapping are beneficial in evaluating knee joint pathologies; however, long acquisition times limit their clinical adoption. MIXTURE (Multi-Interleaved X-prepared Turbo Spin-Echo with IntUitive RElaxometry) provides a versatile turbo spin-echo (TSE) platform for simultaneous morphologic and quantitative joint imaging. Two MIXTURE sequences were designed along clinical requirements: "MIX1", combining proton density (PD)-weighted fat-saturated (FS) images and T2 mapping (acquisition time: 4:59 min), and "MIX2", combining T1-weighted images and T1ρ mapping (6:38 min). MIXTURE sequences and their reference 2D and 3D TSE counterparts were acquired from ten human cadaveric knee joints at 3.0 T. Contrast, contrast-to-noise ratios, and coefficients of variation were comparatively evaluated using parametric tests. Clinical radiologists ( n = 3) assessed diagnostic quality as a function of sequence and anatomic structure using five-point Likert scales and ordinal regression, with a significance level of α = 0.01. MIX1 and MIX2 had at least equal diagnostic quality compared to reference sequences of the same image weighting. Contrast, contrast-to-noise ratios, and coefficients of variation were largely similar for the PD-weighted FS and T1-weighted images. In clinically feasible scan times, MIXTURE sequences yield morphologic, TSE-based images of diagnostic quality and quantitative parameter maps with additional insights on soft tissue composition and ultrastructure.

Background: To compare denoising diffusion probabilistic models (DDPM) and generative adversarial networks (GAN) for recovering contrast-enhanced breast magnetic resonance imaging (MRI) subtraction images from virtual low-dose subtraction images., Methods: Retrospective, ethically approved study. DDPM- and GAN-reconstructed single-slice subtraction images of 50 breasts with enhancing lesions were compared to original ones at three dose levels (25%, 10%, 5%) using quantitative measures and radiologic evaluations. Two radiologists stated their preference based on the reconstruction quality and scored the lesion conspicuity as compared to the original, blinded to the model. Fifty lesion-free maximum intensity projections were evaluated for the presence of false-positives. Results were compared between models and dose levels, using generalized linear mixed models., Results: At 5% dose, both radiologists preferred the GAN-generated images, whereas at 25% dose, both radiologists preferred the DDPM-generated images. Median lesion conspicuity scores did not differ between GAN and DDPM at 25% dose (5 versus 5, p = 1.000) and 10% dose (4 versus 4, p = 1.000). At 5% dose, both readers assigned higher conspicuity to the GAN than to the DDPM (3 versus 2, p = 0.007). In the lesion-free examinations, DDPM and GAN showed no differences in the false-positive rate at 5% (15% versus 22%), 10% (10% versus 6%), and 25% (6% versus 4%) (p = 1.000)., Conclusions: Both GAN and DDPM yielded promising results in low-dose image reconstruction. However, neither of them showed superior results over the other model for all dose levels and evaluation metrics. Further development is needed to counteract false-positives., Relevance Statement: For MRI-based breast cancer screening, reducing the contrast agent dose is desirable. Diffusion probabilistic models and generative adversarial networks were capable of retrospectively enhancing the signal of low-dose images. Hence, they may supplement imaging with reduced doses in the future., Key Points: • Deep learning may help recover signal in low-dose contrast-enhanced breast MRI. • Two models (DDPM and GAN) were trained at different dose levels. • Radiologists preferred DDPM at 25%, and GAN images at 5% dose. • Lesion conspicuity between DDPM and GAN was similar, except at 5% dose. • GAN and DDPM yield promising results in low-dose image reconstruction., (© 2024. The Author(s).)

High-quality magnetic resonance (MR) imaging is essential for the precise assessment of the knee joint and plays a key role in the diagnostics, treatment and prognosis. Intact cartilage tissue is characterized by a smooth surface, uniform tissue thickness and an organized zonal structure, which are manifested as depth-dependent signal intensity variations. Cartilage pathologies are identifiable through alterations in signal intensity and morphology and should be communicated based on a precise terminology. Cartilage pathologies can show hyperintense and hypointense signal alterations. Cartilage defects are assessed based on their depth and should be described in terms of their location and extent. The following symptom constellations are of overarching clinical relevance in image reading and interpretation: symptom constellations associated with rapidly progressive forms of joint degeneration and unfavorable prognosis, accompanying symptom constellations mostly in connection with destabilizing meniscal lesions and subchondral insufficiency fractures (accelerated osteoarthritis) as well as symptoms beyond the "typical" degeneration, especially when a discrepancy is observed between (minor) structural changes and (major) synovitis and effusion (inflammatory arthropathy)., (© 2023. The Author(s), under exclusive licence to Springer Medizin Verlag GmbH, ein Teil von Springer Nature.)

Magnetic resonance imaging (MRI) is the clinical method of choice for cartilage imaging in the context of degenerative and nondegenerative joint diseases. The MRI-based definitions of osteoarthritis rely on the detection of osteophytes, cartilage pathologies, bone marrow edema and meniscal lesions but currently a scientific consensus is lacking. In the clinical routine proton density-weighted, fat-suppressed 2D turbo spin echo sequences with echo times of 30-40 ms are predominantly used, which are sufficiently sensitive and specific for the assessment of cartilage. The additionally acquired T1-weighted sequences are primarily used for evaluating other intra-articular and periarticular structures. Diagnostically relevant artifacts include magic angle and chemical shift artifacts, which can lead to artificial signal enhancement in cartilage or incorrect representations of the subchondral lamina and its thickness. Although scientifically validated, high-resolution 3D gradient echo sequences (for cartilage segmentation) and compositional MR sequences (for quantification of physical tissue parameters) are currently reserved for scientific research questions. The future integration of artificial intelligence techniques in areas such as image reconstruction (to reduce scan times while maintaining image quality), image analysis (for automated identification of cartilage defects), and image postprocessing (for automated segmentation of cartilage in terms of volume and thickness) will significantly improve the diagnostic workflow and advance the field further., (© 2023. The Author(s), under exclusive licence to Springer Medizin Verlag GmbH, ein Teil von Springer Nature.)

Objective: Clinical-standard MRI is the imaging modality of choice for the wrist, yet limited to static evaluation, thereby potentially missing dynamic instability patterns. We aimed to investigate the clinical benefit of (dynamic) real-time MRI, complemented by automatic analysis, in patients with complete or partial scapholunate ligament (SLL) tears., Material and Methods: Both wrists of ten patients with unilateral SLL tears (six partial, four complete tears) as diagnosed by clinical-standard MRI were imaged during continuous active radioulnar motion using a 1.5-T MRI scanner in combination with a custom-made motion device. Following automatic segmentation of the wrist, the scapholunate and lunotriquetral joint widths were analyzed across the entire range of motion (ROM). Mixed-effects model analysis of variance (ANOVA) followed by Tukey's posthoc test and two-way ANOVA were used for statistical analysis., Results: With the increasing extent of SLL tear, the scapholunate joint widths in injured wrists were significantly larger over the entire ROM compared to those of the contralateral healthy wrists (p<0.001). Differences between partial and complete tears were most pronounced at 5°-15° ulnar abduction (p<0.001). Motion patterns and trajectories were altered. Complete SLL deficiency resulted in complex alterations of the lunotriquetral joint widths., Conclusion: Real-time MRI may improve the functional diagnosis of SLL insufficiency and aid therapeutic decision-making by revealing dynamic forms of dissociative instability within the proximal carpus. Static MRI best differentiates SLL-injured wrists at 5°-15° of ulnar abduction., (© 2023. The Author(s).)

Background: Artificial intelligence (AI) models are increasingly used in the medical domain. However, as medical data is highly sensitive, special precautions to ensure its protection are required. The gold standard for privacy preservation is the introduction of differential privacy (DP) to model training. Prior work indicates that DP has negative implications on model accuracy and fairness, which are unacceptable in medicine and represent a main barrier to the widespread use of privacy-preserving techniques. In this work, we evaluated the effect of privacy-preserving training of AI models regarding accuracy and fairness compared to non-private training., Methods: We used two datasets: (1) A large dataset (N = 193,311) of high quality clinical chest radiographs, and (2) a dataset (N = 1625) of 3D abdominal computed tomography (CT) images, with the task of classifying the presence of pancreatic ductal adenocarcinoma (PDAC). Both were retrospectively collected and manually labeled by experienced radiologists. We then compared non-private deep convolutional neural networks (CNNs) and privacy-preserving (DP) models with respect to privacy-utility trade-offs measured as area under the receiver operating characteristic curve (AUROC), and privacy-fairness trade-offs, measured as Pearson's r or Statistical Parity Difference., Results: We find that, while the privacy-preserving training yields lower accuracy, it largely does not amplify discrimination against age, sex or co-morbidity. However, we find an indication that difficult diagnoses and subgroups suffer stronger performance hits in private training., Conclusions: Our study shows that - under the challenging realistic circumstances of a real-life clinical dataset - the privacy-preserving training of diagnostic deep learning models is possible with excellent diagnostic accuracy and fairness., (© 2024. The Author(s).)

Deep learning applied to whole-slide histopathology images (WSIs) has the potential to enhance precision oncology and alleviate the workload of experts. However, developing these models necessitates large amounts of data with ground truth labels, which can be both time-consuming and expensive to obtain. Pathology reports are typically unstructured or poorly structured texts, and efforts to implement structured reporting templates have been unsuccessful, as these efforts lead to perceived extra workload. In this study, we hypothesised that large language models (LLMs), such as the generative pre-trained transformer 4 (GPT-4), can extract structured data from unstructured plain language reports using a zero-shot approach without requiring any re-training. We tested this hypothesis by utilising GPT-4 to extract information from histopathological reports, focusing on two extensive sets of pathology reports for colorectal cancer and glioblastoma. We found a high concordance between LLM-generated structured data and human-generated structured data. Consequently, LLMs could potentially be employed routinely to extract ground truth data for machine learning from unstructured pathology reports in the future. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland., (© 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.)

Increased attention has been given to MRI in radiation-free screening for malignant nodules in recent years. Our objective was to compare the performance of human readers and radiomic feature analysis based on stand-alone and complementary CT and MRI imaging in classifying pulmonary nodules. This single-center study comprises patients with CT findings of pulmonary nodules who underwent additional lung MRI and whose nodules were classified as benign/malignant by resection. For radiomic features analysis, 2D segmentation was performed for each lung nodule on axial CT, T2-weighted (T2w), and diffusion (DWI) images. The 105 extracted features were reduced by iterative backward selection. The performance of radiomics and human readers was compared by calculating accuracy with Clopper-Pearson confidence intervals. Fifty patients (mean age 63 +/- 10 years) with 66 pulmonary nodules (40 malignant) were evaluated. ACC values for radiomic features analysis vs. radiologists based on CT alone (0.68; 95%CI: 0.56, 0.79 vs. 0.59; 95%CI: 0.46, 0.71), T2w alone (0.65; 95%CI: 0.52, 0.77 vs. 0.68; 95%CI: 0.54, 0.78), DWI alone (0.61; 95%CI:0.48, 0.72 vs. 0.73; 95%CI: 0.60, 0.83), combined T2w/DWI (0.73; 95%CI: 0.60, 0.83 vs. 0.70; 95%CI: 0.57, 0.80), and combined CT/T2w/DWI (0.83; 95%CI: 0.72, 0.91 vs. 0.64; 95%CI: 0.51, 0.75) were calculated. This study is the first to show that by combining quantitative image information from CT, T2w, and DWI datasets, pulmonary nodule assessment through radiomics analysis is superior to using one modality alone, even exceeding human readers' performance.

A knowledge gap persists between machine learning (ML) developers (e.g., data scientists) and practitioners (e.g., clinicians), hampering the full utilization of ML for clinical data analysis. We investigated the potential of the ChatGPT Advanced Data Analysis (ADA), an extension of GPT-4, to bridge this gap and perform ML analyses efficiently. Real-world clinical datasets and study details from large trials across various medical specialties were presented to ChatGPT ADA without specific guidance. ChatGPT ADA autonomously developed state-of-the-art ML models based on the original study's training data to predict clinical outcomes such as cancer development, cancer progression, disease complications, or biomarkers such as pathogenic gene sequences. Following the re-implementation and optimization of the published models, the head-to-head comparison of the ChatGPT ADA-crafted ML models and their respective manually crafted counterparts revealed no significant differences in traditional performance metrics (p ≥ 0.072). Strikingly, the ChatGPT ADA-crafted ML models often outperformed their counterparts. In conclusion, ChatGPT ADA offers a promising avenue to democratize ML in medicine by simplifying complex data analyses, yet should enhance, not replace, specialized training and resources, to promote broader applications in medical research and practice., (© 2024. The Author(s).)

Background: Pretraining labeled datasets, like ImageNet, have become a technical standard in advanced medical image analysis. However, the emergence of self-supervised learning (SSL), which leverages unlabeled data to learn robust features, presents an opportunity to bypass the intensive labeling process. In this study, we explored if SSL for pretraining on non-medical images can be applied to chest radiographs and how it compares to supervised pretraining on non-medical images and on medical images., Methods: We utilized a vision transformer and initialized its weights based on the following: (i) SSL pretraining on non-medical images (DINOv2), (ii) supervised learning (SL) pretraining on non-medical images (ImageNet dataset), and (iii) SL pretraining on chest radiographs from the MIMIC-CXR database, the largest labeled public dataset of chest radiographs to date. We tested our approach on over 800,000 chest radiographs from 6 large global datasets, diagnosing more than 20 different imaging findings. Performance was quantified using the area under the receiver operating characteristic curve and evaluated for statistical significance using bootstrapping., Results: SSL pretraining on non-medical images not only outperformed ImageNet-based pretraining (p < 0.001 for all datasets) but, in certain cases, also exceeded SL on the MIMIC-CXR dataset. Our findings suggest that selecting the right pretraining strategy, especially with SSL, can be pivotal for improving diagnostic accuracy of artificial intelligence in medical imaging., Conclusions: By demonstrating the promise of SSL in chest radiograph analysis, we underline a transformative shift towards more efficient and accurate AI models in medical imaging., Relevance Statement: Self-supervised learning highlights a paradigm shift towards the enhancement of AI-driven accuracy and efficiency in medical imaging. Given its promise, the broader application of self-supervised learning in medical imaging calls for deeper exploration, particularly in contexts where comprehensive annotated datasets are limited., (© 2024. The Author(s).)

Artificial intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively training AI models while avoiding data transfer. However, in these decentralized methods, weight updates are still transferred to the aggregation server for merging the models. This leaves the possibility for a breach of data privacy, for example by model inversion or membership inference attacks by untrusted servers. Somewhat-homomorphically-encrypted federated learning (SHEFL) is a solution to this problem because only encrypted weights are transferred, and model updates are performed in the encrypted space. Here, we demonstrate the first successful implementation of SHEFL in a range of clinically relevant tasks in cancer image analysis on multicentric datasets in radiology and histopathology. We show that SHEFL enables the training of AI models which outperform locally trained models and perform on par with models which are centrally trained. In the future, SHEFL can enable multiple institutions to co-train AI models without forsaking data governance and without ever transmitting any decryptable data to untrusted servers., Competing Interests: Declaration of Competing Interest The Authors declare no competing financial or non-financial interests. For transparency, we provide the following information: JNK declares consulting services for Owkin, France, DoMore Diagnostics, Norway, Panakeia, UK, Scailyte, Switzerland, Cancilico, Germany, Mindpeak, Germany, and Histofy, UK; furthermore he holds shares in StratifAI GmbH, Germany, and has received honoraria for lectures by AstraZeneca, Bayer, Eisai, MSD, BMS, Roche, Pfizer and Fresenius. DT holds shares in StraifAI GmbH, Germany and received honoraria for lectures by Bayer. PQ and NW declare research funding from Roche and PQ consulting and speaker services for Roche. MST has recently received honoraria for advisory work in relation to the following companies: Incyte, MindPeak, MSD, BMS and Sonrai; these are all unrelated to this work. No other potential conflicts of interest are reported by any of the authors. The authors received advice from NVIDIA when performing this study, but NVIDIA did not have any role in study design, conducting the experiments, interpretation of the results or decision to submit for publication., (Copyright © 2023 The Authors. Published by Elsevier B.V. All rights reserved.)

Developing robust artificial intelligence (AI) models that generalize well to unseen datasets is challenging and usually requires large and variable datasets, preferably from multiple institutions. In federated learning (FL), a model is trained collaboratively at numerous sites that hold local datasets without exchanging them. So far, the impact of training strategy, i.e., local versus collaborative, on the diagnostic on-domain and off-domain performance of AI models interpreting chest radiographs has not been assessed. Consequently, using 610,000 chest radiographs from five institutions across the globe, we assessed diagnostic performance as a function of training strategy (i.e., local vs. collaborative), network architecture (i.e., convolutional vs. transformer-based), single versus cross-institutional performance (i.e., on-domain vs. off-domain), imaging finding (i.e., cardiomegaly, pleural effusion, pneumonia, atelectasis, consolidation, pneumothorax, and no abnormality), dataset size (i.e., from n = 18,000 to 213,921 radiographs), and dataset diversity. Large datasets not only showed minimal performance gains with FL but, in some instances, even exhibited decreases. In contrast, smaller datasets revealed marked improvements. Thus, on-domain performance was mainly driven by training data size. However, off-domain performance leaned more on training diversity. When trained collaboratively across diverse external institutions, AI models consistently surpassed models trained locally for off-domain tasks, emphasizing FL's potential in leveraging data diversity. In conclusion, FL can bolster diagnostic privacy, reproducibility, and off-domain reliability of AI models and, potentially, optimize healthcare outcomes., (© 2023. The Author(s).)