Search

The Radiological Society of North of America (RSNA) and the Medical Image Computing and Computer Assisted Intervention (MICCAI) Society have led a series of joint panels and seminars focused on the present impact and future directions of artificial intelligence (AI) in radiology. These conversations have collected viewpoints from multidisciplinary experts in radiology, medical imaging, and machine learning on the current clinical penetration of AI technology in radiology and how it is impacted by trust, reproducibility, explainability, and accountability. The collective points-both practical and philosophical-define the cultural changes for radiologists and AI scientists working together and describe the challenges ahead for AI technologies to meet broad approval. This article presents the perspectives of experts from MICCAI and RSNA on the clinical, cultural, computational, and regulatory considerations-coupled with recommended reading materials-essential to adopt AI technology successfully in radiology and, more generally, in clinical practice. The report emphasizes the importance of collaboration to improve clinical deployment, highlights the need to integrate clinical and medical imaging data, and introduces strategies to ensure smooth and incentivized integration. Keywords: Adults and Pediatrics, Computer Applications-General (Informatics), Diagnosis, Prognosis © RSNA, 2024.

The Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017. This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets. The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security. Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources. Overcoming these obstacles requires meticulous project management and adherence to strict timelines. The article also highlights the potential of crowdsourced annotation to progress medical imaging research. Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes. Keywords: Use of AI in Education, Artificial Intelligence © RSNA, 2024.

Purpose To present results from a literature survey on practices in deep learning segmentation algorithm evaluation and perform a study on expert quality perception of brain tumor segmentation. Materials and Methods A total of 180 articles reporting on brain tumor segmentation algorithms were surveyed for the reported quality evaluation. Additionally, ratings of segmentation quality on a four-point scale were collected from medical professionals for 60 brain tumor segmentation cases. Results Of the surveyed articles, Dice score, sensitivity, and Hausdorff distance were the most popular metrics to report segmentation performance. Notably, only 2.8% of the articles included clinical experts' evaluation of segmentation quality. The experimental results revealed a low interrater agreement (Krippendorff α, 0.34) in experts' segmentation quality perception. Furthermore, the correlations between the ratings and commonly used quantitative quality metrics were low (Kendall tau between Dice score and mean rating, 0.23; Kendall tau between Hausdorff distance and mean rating, 0.51), with large variability among the experts. Conclusion The results demonstrate that quality ratings are prone to variability due to the ambiguity of tumor boundaries and individual perceptual differences, and existing metrics do not capture the clinical perception of segmentation quality. Keywords: Brain Tumor Segmentation, Deep Learning Algorithms, Glioblastoma, Cancer, Machine Learning Clinical trial registration nos. NCT00756106 and NCT00662506 Supplemental material is available for this article. © RSNA, 2023.

Competing Interests: Disclosures of conflicts of interest: D.K. Institutional research grants from iCAD, GenMab, and Calico; patent planned, issued, or pending (US8634610B2); deputy editor of Radiology: Artificial Intelligence. J.K.C. Grants/contracts from Genentech, GE, and Bayer; iROP DL software license to Siloam; stock/stock options in Siloam Vision; deputy editor of Radiology: Artificial Intelligence.

This dataset is composed of cervical spine CT images with annotations related to fractures; it is available at https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/., Competing Interests: Disclosures of conflicts of interest: H.M.L. No relevant relationships. E.C. No relevant relationships. T.R. No relevant relationships. F.C.K. Consultant for MD.ai and GE HealthCare; member of the Radiological Society of North America (RSNA) Machine Learning Steering Committee member and the Society for Imaging Informatics in Medicine Machine Learning Education Subcommittee (both unpaid). L.M.P. Associate editor for Radiology: Artificial Intelligence; patents planned, issued, or pending: US-20220051060-A1, “Methods for creating privacy-protecting synthetic data leveraging a constrained generative ensemble model,” and US-20220051402-A1, “Systems for automated lesion detection and related methods.” J.T. Provided expert witness deposition for Phillips, Spallas, & Angstadt in October 2022, unrelated to this article. R.L.B. Support from RSNA to author. E.G. No relevant relationships. K.W.Y. No relevant relationships. M.H. No relevant relationships. A.L.S. Author’s lab receives funding from the National Institutes of Health (NIH); member of the advisory board for the National Cancer Institute Imaging Data Commons. J.S. No relevant relationships. D.B. No relevant relationships. S.A. No relevant relationships. A.P.L. No relevant relationships. M.I.G.A. No relevant relationships. J.O.J. No relevant relationships. J.J.P. Author’s lab receives funding from the NIH, which pays this author’s salary. M.L. No relevant relationships. H.D. No relevant relationships. E.A. No relevant relationships. A.Y. No relevant relationships. Y.M. No relevant relationships. J.K.C. Grants or contracts from GE HealthCare and Genentech; technology licensed to Boston AI; consulting fees from Siloam Vision; deputy editor of Radiology: Artificial Intelligence. A.E.F. Standing director, liaison for information technology, of RSNA board of directors; member of RSNA News editorial board., (© 2023 by the Radiological Society of North America, Inc.)

Body composition on chest CT scans encompasses a set of important imaging biomarkers. This study developed and validated a fully automated analysis pipeline for multi-vertebral level assessment of muscle and adipose tissue on routine chest CT scans. This study retrospectively trained two convolutional neural networks on 629 chest CT scans from 629 patients (55% women; mean age, 67 years ± 10 [standard deviation]) obtained between 2014 and 2017 prior to lobectomy for primary lung cancer at three institutions. A slice-selection network was developed to identify an axial image at the level of the fifth, eighth, and 10th thoracic vertebral bodies. A segmentation network (U-Net) was trained to segment muscle and adipose tissue on an axial image. Radiologist-guided manual-level selection and segmentation generated ground truth. The authors then assessed the predictive performance of their approach for cross-sectional area (CSA) (in centimeters squared) and attenuation (in Hounsfield units) on an independent test set. For the pipeline, median absolute error and intraclass correlation coefficients for both tissues were 3.6% (interquartile range, 1.3%-7.0%) and 0.959-0.998 for the CSA and 1.0 HU (interquartile range, 0.0-2.0 HU) and 0.95-0.99 for median attenuation. This study demonstrates accurate and reliable fully automated multi-vertebral level quantification and characterization of muscle and adipose tissue on routine chest CT scans. Keywords : Skeletal Muscle, Adipose Tissue, CT, Chest, Body Composition Analysis, Convolutional Neural Network (CNN), Supervised Learning Supplemental material is available for this article. © RSNA, 2022., Competing Interests: Disclosures of conflicts of interest: C.P.B. Support from the MGH and BWH Center for Clinical Data Science (CCDS) for travel to conferences. The CCDS in turn receives support from GE Healthcare, Nuance Communications, Nvidia, Diagnóstico da América S.A., and Fujifilm Sonosite; US Patent applications pending for: Computed Tomography Medical Imaging Intracranial Hemorrhage Model (US Patent Application 16/587,828). In collaboration with GE Healthcare and Medical Imaging Stroke Model (US Patent Application 16/588,080). In collaboration with GE Healthcare. T.D.B. No relevant relationships. M.M.W. No relevant relationships. J.P.M. No relevant relationships. K.M. Former trainee editorial board member for Radiology: Artificial Intelligence. C.J. No relevant relationships. J.H.C. Editorial board member of Radiology: Cardiothoracic Imaging. J.K.C. Deputy editor of Radiology: Artificial Intelligence. K.P.A. A Mobile Health Diagnostic Device for HIV Self-Testing NIH 1R61 AI140489-01A1 PI: Shafiee, Andriole, Co-Investigator, Study goals are to develop a hand-held device for HIV self-testing using artificial intelligence algorithms for data analysis. 8/2019-7/2022; associate editor of Radiology: Artificial Intelligence. F.J.F. American Roentgen Ray Society scholarship grant (related to this work); grant from William M. Wood Foundation (not related to this work); grant form Society of Interventional Oncology (unrelated to this work); research support from Boston Scientific (unrelated to this work); patents related to body composition analysis., (2022 by the Radiological Society of North America, Inc.)

Purpose: To evaluate the trustworthiness of saliency maps for abnormality localization in medical imaging., Materials and Methods: Using two large publicly available radiology datasets (Society for Imaging Informatics in Medicine-American College of Radiology Pneumothorax Segmentation dataset and Radiological Society of North America Pneumonia Detection Challenge dataset), the performance of eight commonly used saliency map techniques were quantified in regard to (a) localization utility (segmentation and detection), (b) sensitivity to model weight randomization, (c) repeatability, and (d) reproducibility. Their performances versus baseline methods and localization network architectures were compared, using area under the precision-recall curve (AUPRC) and structural similarity index measure (SSIM) as metrics., Results: All eight saliency map techniques failed at least one of the criteria and were inferior in performance compared with localization networks. For pneumothorax segmentation, the AUPRC ranged from 0.024 to 0.224, while a U-Net achieved a significantly superior AUPRC of 0.404 ( P < .005). For pneumonia detection, the AUPRC ranged from 0.160 to 0.519, while a RetinaNet achieved a significantly superior AUPRC of 0.596 ( P <.005). Five and two saliency methods (of eight) failed the model randomization test on the segmentation and detection datasets, respectively, suggesting that these methods are not sensitive to changes in model parameters. The repeatability and reproducibility of the majority of the saliency methods were worse than localization networks for both the segmentation and detection datasets., Conclusion: The use of saliency maps in the high-risk domain of medical imaging warrants additional scrutiny and recommend that detection or segmentation models be used if localization is the desired output of the network. Keywords: Technology Assessment, Technical Aspects, Feature Detection, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2021., Competing Interests: Disclosures of Conflicts of Interest: N.A. No relevant relationships. N.G. No relevant relationships. P.S. No relevant relationships. K.C. Research support through training grant from National Institute of Biomedical Imaging and Bioengineering of National Institutes of Health (NIH) (award no. 5T32EB1680) and by National Cancer Institute of NIH (award no. F30CA239407). M.A. No relevant relationships. B.C. No relevant relationships. K.H. No relevant relationships. S.G. No relevant relationships. J.P. No relevant relationships. M.G. No relevant relationships. J.A. No relevant relationships. M.D.L. Member of Radiology: Artificial Intelligence trainee editorial board; Radiological Society of North America Research and Education Foundation Presidents Circle grant. J.K.C. Institution has grant/grants pending from NIH, GE, Genentech, National Science Foundation; deputy editor for Radiology: Artificial Intelligence; payment from IBM for lectures including service on speakers bureaus; royalties from Boston AI., (2021 by the Radiological Society of North America, Inc.)

On October 5, 2020, the Medical Image Computing and Computer Assisted Intervention Society (MICCAI) 2020 conference hosted a virtual panel discussion with members of the Machine Learning Steering Subcommittee of the Radiological Society of North America. The MICCAI Society brings together scientists, engineers, physicians, educators, and students from around the world. Both societies share a vision to develop radiologic and medical imaging techniques through advanced quantitative imaging biomarkers and artificial intelligence. The panel elaborated on how collaborations between radiologists and machine learning scientists facilitate the creation and clinical success of imaging technology for radiology. This report presents structured highlights of the moderated dialogue at the panel. Keywords: Back-Propagation, Artificial Neural Network Algorithms, Machine Learning Algorithms © RSNA, 2021., Competing Interests: Disclosures of Conflicts of Interest: J.M. institution received grant from GE Healthcare; author receives royalties from GE Healthcare for licensing of pneumothorax detection algorithm; author paid for development of educational presentations from UCSF Postgraduate Medical Education; author’s spouse has been employed by AbbVie and Annexon Biosciences; author is associate editor of Radiology: Artificial Intelligence. J.K.C. institution received grants from GE, NIH, NSF, and Genentech; author received travel accommodations from IBM; author is deputy editor of Radiology: Artificial Intelligence. A.F. disclosed no relevant relationships. M.G.L. author is consultant to the National Institutes of Health for grant review; author is co-founder of PediaMetrix; author’s institution received grants from National Institutes of Health, National Science Foundation, Department of Defense, and Philips Healthcare; author has stock/stock options in PediaMetrix., (2021 by the Radiological Society of North America, Inc.)

Supplemental material is available for this article., Competing Interests: Disclosures of Conflicts of Interest: E.C. disclosed no relevant relationships. F.C.K. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: consultant for MD.ai; employed by DASA as Head of AI. Other relationships: disclosed no relevant relationships. S.B.H. disclosed no relevant relationships. C.C.W. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: institution received grant from NIBIB and has MIDRC grant pending; author receives royalties from Elsevier. Other relationships: disclosed no relevant relationships. M.P.L. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: board member of Nines Radiology, SegMed, and BunkerHill; author has stock/stock options in Nines Radiology, SegMed, and BunkerHill. Other relationships: disclosed no relevant relationships. L.M.P. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: editorial board member of Radiology: Artificial Intelligence. Other relationships: disclosed no relevant relationships J.K.C. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: institution has grants pending with GE and Genentech; author receives support for travel/accommodation/meeting expenses from IBM; deputy editor of Radiology: Artificial Intelligence. Other relationships: disclosed no relevant relationships. R.L.B. Activities related to the present article: author paid consulting fee or honorarium from RSNA. Activities not related to the present article: employed by Stanford University. Other relationships: disclosed no relevant relationships. G.S. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: board member of MD.ai (no compensation); consultant for MD.ai (no compensation); stock/stock options in MD.ai; editorial board member of Radiology: Artificial Intelligence. Other relationships: disclosed no relevant relationships. A.S. Activities related to the present article: employee of MD.ai. Activities not related to the present article: employed by MD.ai. Other relationships: disclosed no relevant relationships. S.S.H. disclosed no relevant relationships. E.A. disclosed no relevant relationships. M.L. disclosed no relevant relationships. P.K. disclosed no relevant relationships. K.A.M. disclosed no relevant relationships. D.C.N.R. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: speaker for Bayer de Mexico (author gave one lecture for the International Day of Radiology and received speaker training from Bayer). Other relationships: disclosed no relevant relationships. J.W.S. disclosed no relevant relationships. P. Germaine disclosed no relevant relationships. E.C.L. disclosed no relevant relationships. T.A. disclosed no relevant relationships. P. Gupta disclosed no relevant relationships. M.J. disclosed no relevant relationships. F.U.K. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: editorial board member of Radiology: Cardiothoracic Imaging; employed at UT Southwestern Medical Center. Other relationships: disclosed no relevant relationships. C.T.L. disclosed no relevant relationships. S.S. disclosed no relevant relationships. J.W.R. disclosed no relevant relationships. C.C.B. disclosed no relevant relationships. J.M. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: institution has grant from GE Healthcare; editorial board member of Radiology: Artificial Intelligence. Other relationships: disclosed no relevant relationships., (2021 by the Radiological Society of North America, Inc.)

Purpose: To determine the influence of preprocessing on the repeatability and redundancy of radiomics features extracted using a popular open-source radiomics software package in a scan-rescan glioblastoma MRI study., Materials and Methods: In this study, a secondary analysis of T2-weighted fluid-attenuated inversion recovery (FLAIR) and T1-weighted postcontrast images from 48 patients (mean age, 56 years [range, 22-77 years]) diagnosed with glioblastoma were included from two prospective studies (ClinicalTrials.gov NCT00662506 [2009-2011] and NCT00756106 [2008-2011]). All patients underwent two baseline scans 2-6 days apart using identical imaging protocols on 3-T MRI systems. No treatment occurred between scan and rescan, and tumors were essentially unchanged visually. Radiomic features were extracted by using PyRadiomics ( https://pyradiomics.readthedocs.io/ ) under varying conditions, including normalization strategies and intensity quantization. Subsequently, intraclass correlation coefficients were determined between feature values of the scan and rescan., Results: Shape features showed a higher repeatability than intensity (adjusted P < .001) and texture features (adjusted P < .001) for both T2-weighted FLAIR and T1-weighted postcontrast images. Normalization improved the overlap between the region of interest intensity histograms of scan and rescan (adjusted P < .001 for both T2-weighted FLAIR and T1-weighted postcontrast images), except in scans where brain extraction fails. As such, normalization significantly improves the repeatability of intensity features from T2-weighted FLAIR scans (adjusted P = .003 [ z score normalization] and adjusted P = .002 [histogram matching]). The use of a relative intensity binning strategy as opposed to default absolute intensity binning reduces correlation between gray-level co-occurrence matrix features after normalization., Conclusion: Both normalization and intensity quantization have an effect on the level of repeatability and redundancy of features, emphasizing the importance of both accurate reporting of methodology in radiomics articles and understanding the limitations of choices made in pipeline design. Supplemental material is available for this article. © RSNA, 2020See also the commentary by Tiwari and Verma in this issue., Competing Interests: Disclosures of Conflicts of Interest: K.V.H. disclosed no relevant relationships. J.B.P. disclosed no relevant relationships. A.L.B. disclosed no relevant relationships. K.C. disclosed no relevant relationships. P.S. disclosed no relevant relationships. J.M.B. disclosed no relevant relationships. M.C.P. disclosed no relevant relationships. T.T.B. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: received research support from Champions Biotechnology, AstraZeneca, Pfizer, and Millennium; is on the advisory board for UpToDate; is a consultant for Genomicare, Merck, NXDC, Amgen, Roche, Oxigene, Foundation Medicine, and Proximagen; provided CME lectures or material for UpToDate, Research to Practice, Oakstone Medical Publishing, and Imedex. Other relationships: disclosed no relevant relationships. E.R.G. disclosed no relevant relationships. B.R.R. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: is on the advisory board for ARIA, Butterfly, DGMIF (Daegu-Gyeongbuk Medical Innovation Foundation), QMENTA, and Subtle Medical; is a consultant for Broadview Ventures, Janssen Scientific, ECRI Institute, GlaxoSmithKline, Hyperfine Research, Peking University, Wolf Greenfield, Superconducting Systems, Robins Kaplin, Millennium Pharmaceuticals, GE Healthcare, Siemens, Quinn Emanuel Trial Lawyers, Samsung, and Shenzhen Maternity and Child Health Care Hospital; is a founder of BLINKAI Technologies. Other relationships: disclosed no relevant relationships. J.K.C. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: is a consultant and advisory board member for Infotech, Soft. Other relationships: disclosed no relevant relationships., (2020 by the Radiological Society of North America, Inc.)

Purpose: To develop an automated measure of COVID-19 pulmonary disease severity on chest radiographs (CXRs), for longitudinal disease tracking and outcome prediction., Materials and Methods: A convolutional Siamese neural network-based algorithm was trained to output a measure of pulmonary disease severity on CXRs (pulmonary x-ray severity (PXS) score), using weakly-supervised pretraining on ∼160,000 anterior-posterior images from CheXpert and transfer learning on 314 frontal CXRs from COVID-19 patients. The algorithm was evaluated on internal and external test sets from different hospitals (154 and 113 CXRs respectively). PXS scores were correlated with radiographic severity scores independently assigned by two thoracic radiologists and one in-training radiologist (Pearson r). For 92 internal test set patients with follow-up CXRs, PXS score change was compared to radiologist assessments of change (Spearman ρ). The association between PXS score and subsequent intubation or death was assessed. Bootstrap 95% confidence intervals (CI) were calculated., Results: PXS scores correlated with radiographic pulmonary disease severity scores assigned to CXRs in the internal and external test sets (r=0.86 (95%CI 0.80-0.90) and r=0.86 (95%CI 0.79-0.90) respectively). The direction of change in PXS score in follow-up CXRs agreed with radiologist assessment (ρ=0.74 (95%CI 0.63-0.81)). In patients not intubated on the admission CXR, the PXS score predicted subsequent intubation or death within three days of hospital admission (area under the receiver operating characteristic curve=0.80 (95%CI 0.75-0.85))., Conclusion: A Siamese neural network-based severity score automatically measures radiographic COVID-19 pulmonary disease severity, which can be used to track disease change and predict subsequent intubation or death., (2020 by the Radiological Society of North America, Inc.)

This dataset is composed of annotations of the five hemorrhage subtypes (subarachnoid, intraventricular, subdural, epidural, and intraparenchymal hemorrhage) typically encountered at brain CT., Competing Interests: Disclosures of Conflicts of Interest: A.E.F. disclosed no relevant relationships. L.M.P. Activities related to the present article: institution receives time-limited loaner access by OSU AI Lab to NVIDIA GPUs via Master Research Agreement between The Ohio State University and NVIDIA, no money transfer; time-limited access by OSU AI Lab to WIP postprocessing software via Master Research Agreement between OSU and Siemens Healthineers. No money transfer; unrestrictive support of OSU AI Lab by DeBartolo Family Funds. Activities not related to the present article: disclosed no relevant relationships. Other relationships: disclosed no relevant relationships. G.S. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: board member and shareholder of MD.ai but no money paid to author or MD.ai. Other relationships: disclosed no relevant relationships. S.S.H. disclosed no relevant relationships. J.K. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author is consultant for INFOTECH, Soft; institution receives grants from Genentech and GE. Other relationships: disclosed no relevant relationships. R.B. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: previously employed by Roam Analytics (employment ended prior to working on this study); stockholder in Roam Analytics. Other relationships: disclosed no relevant relationships. J.T.M. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: consultant for Siemens (PAMA clinical decision support related); institution receives grants from GE (past and pending grants related to PTX detection); author may receive future royalties from GE related to PTX AI detector licensed to GE. Other relationships: disclosed no relevant relationships. A.S. Activities related to the present article: employee and shareholder in MD.ai. Activities not related to the present article: employee and shareholder in MD.ai. Other relationships: disclosed no relevant relationships. F.C.K. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: consultant for MD.ai; employed by Diagnósticos da América (DASA). Other relationships: disclosed no relevant relationships. M.P.L. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: consultant for and stockholder in Nines, SegMed, and Bunker Hill. Other relationships: disclosed no relevant relationships. G.C. disclosed no relevant relationships. L. Cala disclosed no relevant relationships. L. Coelho disclosed no relevant relationships. M.M. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: consultant for Cerebrotech Medical Systems. Other relationships: disclosed no relevant relationships. F.M. disclosed no relevant relationships. E.M. disclosed no relevant relationships. I.I. disclosed no relevant relationships. V.Z. disclosed no relevant relationships. O.M. disclosed no relevant relationships. C.L. disclosed no relevant relationships. L.S. disclosed no relevant relationships. D.J. disclosed no relevant relationships. A.A. disclosed no relevant relationships. R.K.L. disclosed no relevant relationships. J.N. disclosed no relevant relationships., (2020 by the Radiological Society of North America, Inc.)

Purpose: To investigate improvements in performance for automatic bone age estimation that can be gained through model ensembling., Materials and Methods: A total of 48 submissions from the 2017 RSNA Pediatric Bone Age Machine Learning Challenge were used. Participants were provided with 12 611 pediatric hand radiographs with bone ages determined by a pediatric radiologist to develop models for bone age determination. The final results were determined using a test set of 200 radiographs labeled with the weighted average of six ratings. The mean pairwise model correlation and performance of all possible model combinations for ensembles of up to 10 models using the mean absolute deviation (MAD) were evaluated. A bootstrap analysis using the 200 test radiographs was conducted to estimate the true generalization MAD., Results: The estimated generalization MAD of a single model was 4.55 months. The best-performing ensemble consisted of four models with an MAD of 3.79 months. The mean pairwise correlation of models within this ensemble was 0.47. In comparison, the lowest achievable MAD by combining the highest-ranking models based on individual scores was 3.93 months using eight models with a mean pairwise model correlation of 0.67., Conclusion: Combining less-correlated, high-performing models resulted in better performance than naively combining the top-performing models. Machine learning competitions within radiology should be encouraged to spur development of heterogeneous models whose predictions can be combined to achieve optimal performance.© RSNA, 2019 Supplemental material is available for this article. See also the commentary by Siegel in this issue., (2019 by the Radiological Society of North America, Inc.)

Purpose: To describe an unsupervised three-dimensional cardiac motion estimation network (CarMEN) for deformable motion estimation from two-dimensional cine MR images., Materials and Methods: A function was implemented using CarMEN, a convolutional neural network that takes two three-dimensional input volumes and outputs a motion field. A smoothness constraint was imposed on the field by regularizing the Frobenius norm of its Jacobian matrix. CarMEN was trained and tested with data from 150 cardiac patients who underwent MRI examinations and was validated on synthetic ( n = 100) and pediatric ( n = 33) datasets. CarMEN was compared to five state-of-the-art nonrigid body registration methods by using several performance metrics, including Dice similarity coefficient (DSC) and end-point error., Results: On the synthetic dataset, CarMEN achieved a median DSC of 0.85, which was higher than all five methods (minimum-maximum median [or MMM], 0.67-0.84; P < .001), and a median end-point error of 1.7, which was lower than (MMM, 2.1-2.7; P < .001) or similar to (MMM, 1.6-1.7; P > .05) all other techniques. On the real datasets, CarMEN achieved a median DSC of 0.73 for Automated Cardiac Diagnosis Challenge data, which was higher than (MMM, 0.33; P < .0001) or similar to (MMM, 0.72-0.75; P > .05) all other methods, and a median DSC of 0.77 for pediatric data, which was higher than (MMM, 0.71-0.76; P < .0001) or similar to (MMM, 0.77-0.78; P > .05) all other methods. All P values were derived from pairwise testing. For all other metrics, CarMEN achieved better accuracy on all datasets than all other techniques except for one, which had the worst motion estimation accuracy., Conclusion: The proposed deep learning-based approach for three-dimensional cardiac motion estimation allowed the derivation of a motion model that balances motion characterization and image registration accuracy and achieved motion estimation accuracy comparable to or better than that of several state-of-the-art image registration algorithms.© RSNA, 2019 Supplemental material is available for this article., (2019 by the Radiological Society of North America, Inc.)

In recent years, there has been enormous interest in applying artificial intelligence (AI) to radiology. Although some of this interest may have been driven by exaggerated expectations that the technology can outperform radiologists in some tasks, there is a growing body of evidence that illustrates its limitations in medical imaging. The true potential of the technique probably lies somewhere in the middle, and AI will ultimately play a key role in medical imaging in the future. The limitless power of computers makes AI an ideal candidate to provide the standardization, consistency, and dependability needed to support radiologists in their mission to provide excellent patient care. However, important roadblocks currently limit the expansion of this field in medical imaging. This article reviews some of the challenges and potential solutions to advance the field forward, with focus on the experience gained by hosting image-based competitions., (2019 by the Radiological Society of North America, Inc.)