Search

Objective: To demonstrate and test the capabilities of the American College of Radiology (ACR) Connect and AI-LAB software platform by implementing multi-institutional artificial intelligence (AI) training and validation for breast density classification., Methods: In this proof-of-concept study, six U.S.-based hospitals installed Connect and AI-LAB. A breast density algorithm was trained and tested on retrospective mammograms. We recorded time to receive an IRB approval, to install software locally, and to complete the testing and training. We calculated the performance of the breast density algorithm at each participating hospital and compared it to the performance of a hold-out multi-institutional clinical trial testing dataset and a retrospective multi-institutional dataset. We calculated the performance of the locally fine-tuned models on the hold-out test datasets., Results: The median time to receive IRB approval was 66 days, and the median time to successfully install Connect and AI-LAB locally was 157 days. The median time to complete breast density algorithm testing and training was 216 days. The breast density algorithm performed worse at each hospital than on the hold-out test dataset, suggesting poor generalizability of the base model. The fine-tuned models had mixed performance locally and performed poorly on the test dataset., Discussion: In this study, we demonstrate the successful installation and implementation of Connect and AI-LAB software platforms at six facilities using a breast density algorithm. Our results suggest poor generalizability of an algorithm trained on a single dataset and algorithms fine-tuned at individual institutions, emphasizing the hypothetical importance of multi-institutional testing and training., (Copyright © 2024. Published by Elsevier Inc.)

Purpose: To evaluate whether providing clinicians with an artificial intelligence (AI)-based vascular severity score (VSS) improves consistency in the diagnosis of plus disease in retinopathy of prematurity (ROP)., Design: Multireader diagnostic accuracy imaging study., Participants: Eleven ROP experts, 9 of whom had been in practice for 10 years or more., Methods: RetCam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available examinations, a subset of 150 eye examinations from 110 infants were selected for grading. An AI-based VSS was assigned to each set of images using the i-ROP DL system (Siloam Vision). The clinicians were asked to diagnose plus disease for each examination and to assign an estimated VSS (range, 1-9) at baseline, and then again 1 month later with AI-based VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye examination from the Imaging and Informatics in ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis., Main Outcome Measures: Mean linearly weighted κ value for plus disease diagnosis compared with RSD. Area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPR) for labels 1 through 9 compared with RSD for plus disease., Results: Expert agreement improved significantly, from substantial (κ value, 0.69 [0.59, 0.75]) to near perfect (κ value, 0.81 [0.71, 0.86]), when AI-based VSS was integrated. Additionally, a significant improvement in plus disease discrimination was achieved as measured by mean AUC (from 0.94 [95% confidence interval (CI), 0.92-0.96] to 0.98 [95% CI, 0.96-0.99]; difference, 0.04 [95% CI, 0.01-0.06]) and AUPR (from 0.86 [95% CI, 0.81-0.90] to 0.95 [95% CI, 0.91-0.97]; difference, 0.09 [95% CI, 0.03-0.14])., Conclusions: Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling as compared with an RSD for plus disease. If implemented in practice, AI-based VSS could reduce interobserver variability and could standardize treatment for infants with ROP., Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article., (Copyright © 2024 American Academy of Ophthalmology. All rights reserved.)

The correct interpretation of breast density is important in the assessment of breast cancer risk. AI has been shown capable of accurately predicting breast density, however, due to the differences in imaging characteristics across mammography systems, models built using data from one system do not generalize well to other systems. Though federated learning (FL) has emerged as a way to improve the generalizability of AI without the need to share data, the best way to preserve features from all training data during FL is an active area of research. To explore FL methodology, the breast density classification FL challenge was hosted in partnership with the American College of Radiology, Harvard Medical Schools' Mass General Brigham, University of Colorado, NVIDIA, and the National Institutes of Health National Cancer Institute. Challenge participants were able to submit docker containers capable of implementing FL on three simulated medical facilities, each containing a unique large mammography dataset. The breast density FL challenge ran from June 15 to September 5, 2022, attracting seven finalists from around the world. The winning FL submission reached a linear kappa score of 0.653 on the challenge test data and 0.413 on an external testing dataset, scoring comparably to a model trained on the same data in a central location., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier B.V. All rights reserved.)

Background: Contrast-induced nephropathy (CIN) is a postprocedural complication associated with increased morbidity and mortality. An important risk factor for development of CIN is renal impairment. Identification of patients at risk for acute renal failure will allow physicians to make appropriate decisions to minimize the incidence of CIN. We developed a machine learning model to stratify risk of acute renal failure that may assist in mitigating risk for CIN in patients with peripheral artery disease (PAD) undergoing endovascular interventions., Methods: We utilized the American College of Surgeons National Surgical Quality Improvement Program database to extract clinical and laboratory information associated with 14,444 patients who underwent lower extremity endovascular procedures between 2011 and 2018. Using 11,604 cases from 2011 to 2017 for training and 2840 cases from 2018 for testing, we developed a random forest model to predict risk of 30-day acute renal failure following infra-inguinal endovascular procedures., Results: Eight variables were identified as contributing optimally to model predictions, the most important being diabetes, preoperative BUN, and claudication. Using these variables, the model achieved an area under the receiver-operating characteristic (AU-ROC) curve of 0.81, accuracy of 0.83, sensitivity of 0.67, and specificity of 0.74. The model performed equally well on white and nonwhite patients (Delong p-value = 0.955) and patients age < 65 and patients age ≥ 65 (Delong p-value = 0.659)., Conclusions: We develop a model that fairly and accurately stratifies 30-day acute renal failure risk in patients undergoing lower extremity endovascular procedures for PAD. This model may assist in identifying patients who may benefit from strategies to prevent CIN., Competing Interests: Declaration of competing interest S.K. reports receiving grants or contracts from BD and AngioDynamics. He has also received consulting fees from Penumbra, Okami Medical, Boston Scientific, Medtronic, and US Vascular. D.D. reports receiving consulting fees from Medtronic. The remaining authors declare no competing interests., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Purpose: Epidemiological changes in retinopathy of prematurity (ROP) depend on neonatal care, neonatal mortality, and the ability to carefully titrate and monitor oxygen. We evaluate whether an artificial intelligence (AI) algorithm for assessing ROP severity in babies can be used to evaluate changes in disease epidemiology in babies from South India over a 5-year period., Design: Retrospective cohort study., Participants: Babies (3093) screened for ROP at neonatal care units (NCUs) across the Aravind Eye Care System (AECS) in South India., Methods: Images and clinical data were collected as part of routine tele-ROP screening at the AECS in India over 2 time periods: August 2015 to October 2017 and March 2019 to December 2020. All babies in the original cohort were matched 1:3 by birthweight (BW) and gestational age (GA) with babies in the later cohort. We compared the proportion of eyes with moderate (type 2) or treatment-requiring (TR) ROP, and an AI-derived ROP vascular severity score (from retinal fundus images) at the initial tele-retinal screening exam for all babies in a district, VSS), in the 2 time periods., Main Outcome Measures: Differences in the proportions of type 2 or worse and TR-ROP cases, and VSS between time periods., Results: Among BW and GA matched babies, the proportion [95% confidence interval {CI}] of babies with type 2 or worse and TR-ROP decreased from 60.9% [53.8%-67.7%] to 17.1% [14.0%-20.5%] (P < 0.001) and 16.8% [11.9%-22.7%] to 5.1% [3.4%-7.3%] (P < 0.001), over the 2 time periods. Similarly, the median [interquartile range] VSS in the population decreased from 2.9 [1.2] to 2.4 [1.8] (P < 0.001)., Conclusions: In South India, over a 5-year period, the proportion of babies developing moderate to severe ROP has dropped significantly for babies at similar demographic risk, strongly suggesting improvements in primary prevention of ROP. These results suggest that AI-based assessment of ROP severity may be a useful epidemiologic tool to evaluate temporal changes in ROP epidemiology., Financial Disclosure(s): Proprietary or commercial disclosure may be found after the references., (Copyright © 2023 American Academy of Ophthalmology. All rights reserved.)

Cervical cancer screening and management in the U.S. has adopted a risk-based approach. However, the majority of cervical cancer cases and deaths occur in resource-limited settings, where screening and management are not widely available. We describe a conceptual model that optimizes cervical cancer screening and management in resource-limited settings by utilizing a risk-based approach. The principles of risk-based screening and management in resource limited settings include (1) ensure that the screening method effectively separates low-risk from high-risk patients; (2) directing resources to populations at the highest cancer risk; (3) screen using HPV testing via self-sampling; (4) utilize HPV genotyping to improve risk stratification and better determine who will benefit from treatment, and (5) automated visual evaluation with artificial intelligence may further improve risk stratification. Risk-based screening and management in resource limited settings can optimize prevention by focusing triage and treatment resources on the highest risk patients while minimizing interventions in lower risk patients., Competing Interests: Declaration of Competing Interest No authors have a conflict of interest to declare related to this work., (Copyright © 2023. Published by Elsevier Ltd.)

Purpose: To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), committee., Design: Validation study of an AI-based ROP vascular severity score., Participants: A total of 34 ROP experts from the ICROP3 committee., Methods: Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, preplus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labeled with a vascular severity score from the Imaging and Informatics in ROP deep learning system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage., Main Outcome Measures: Weighted kappa and Pearson correlation coefficients (CCs) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe., Results: The mean weighted kappa and CC for all interobserver pairs for plus disease image comparison were 0.67 and 0.88, respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, P < 0.001) and the ophthalmoscopic diagnosis of stage (P < 0.001 by analysis of variance) among all experts., Conclusions: The ROP vascular severity score correlates well with the International Classification of Retinopathy of Prematurity committee member's labels for plus disease and stage, which had significant intergrader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies., (Copyright © 2022. Published by Elsevier Inc.)

Purpose: Deploying external artificial intelligence (AI) models locally can be logistically challenging. We aimed to use the ACR AI-LAB software platform for local testing of a chest radiograph (CXR) algorithm for COVID-19 lung disease severity assessment., Methods: An externally developed deep learning model for COVID-19 radiographic lung disease severity assessment was loaded into the AI-LAB platform at an independent academic medical center, which was separate from the institution in which the model was trained. The data set consisted of CXR images from 141 patients with reverse transcription-polymerase chain reaction-confirmed COVID-19, which were routed to AI-LAB for model inference. The model calculated a Pulmonary X-ray Severity (PXS) score for each image. This score was correlated with the average of a radiologist-based assessment of severity, the modified Radiographic Assessment of Lung Edema score, independently interpreted by three radiologists. The associations between the PXS score and patient admission and intubation or death were assessed., Results: The PXS score deployed in AI-LAB correlated with the radiologist-determined modified Radiographic Assessment of Lung Edema score (r = 0.80). PXS score was significantly higher in patients who were admitted (4.0 versus 1.3, P < .001) or intubated or died within 3 days (5.5 versus 3.3, P = .001)., Conclusions: AI-LAB was successfully used to test an external COVID-19 CXR AI algorithm on local data with relative ease, showing generalizability of the PXS score model. For AI models to scale and be clinically useful, software tools that facilitate the local testing process, like the freely available AI-LAB, will be important to cross the AI implementation gap in health care systems., (Copyright © 2022 American College of Radiology. Published by Elsevier Inc. All rights reserved.)

Purpose: To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale., Design: Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity., Participants: Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9., Methods: A quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement., Main Outcome Measures: Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale., Results: For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale., Conclusions: A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis., (Published by Elsevier Inc.)

Objective: We developed deep learning algorithms to automatically assess BI-RADS breast density., Methods: Using a large multi-institution patient cohort of 108,230 digital screening mammograms from the Digital Mammographic Imaging Screening Trial, we investigated the effect of data, model, and training parameters on overall model performance and provided crowdsourcing evaluation from the attendees of the ACR 2019 Annual Meeting., Results: Our best-performing algorithm achieved good agreement with radiologists who were qualified interpreters of mammograms, with a four-class κ of 0.667. When training was performed with randomly sampled images from the data set versus sampling equal number of images from each density category, the model predictions were biased away from the low-prevalence categories such as extremely dense breasts. The net result was an increase in sensitivity and a decrease in specificity for predicting dense breasts for equal class compared with random sampling. We also found that the performance of the model degrades when we evaluate on digital mammography data formats that differ from the one that we trained on, emphasizing the importance of multi-institutional training sets. Lastly, we showed that crowdsourced annotations, including those from attendees who routinely read mammograms, had higher agreement with our algorithm than with the original interpreting radiologists., Conclusion: We demonstrated the possible parameters that can influence the performance of the model and how crowdsourcing can be used for evaluation. This study was performed in tandem with the development of the ACR AI-LAB, a platform for democratizing artificial intelligence., (Copyright © 2020. Published by Elsevier Inc.)

Purpose: Aggressive posterior retinopathy of prematurity (AP-ROP) is a vision-threatening disease with a significant rate of progression to retinal detachment. The purpose of this study was to characterize AP-ROP quantitatively by demographics, rate of disease progression, and a deep learning-based vascular severity score., Design: Retrospective analysis., Participants: The Imaging and Informatics in ROP cohort from 8 North American centers, consisting of 947 patients and 5945 clinical eye examinations with fundus images, was used. Pretreatment eyes were categorized by disease severity: none, mild, type 2 or pre-plus, treatment-requiring (TR) without AP-ROP, TR with AP-ROP. Analyses compared TR with AP-ROP and TR without AP-ROP to investigate differences between AP-ROP and other TR disease., Methods: A reference standard diagnosis was generated for each eye examination using previously published methods combining 3 independent image-based gradings and 1 ophthalmoscopic grading. All fundus images were analyzed using a previously published deep learning system and were assigned a score from 1 through 9., Main Outcome Measures: Birth weight, gestational age, postmenstrual age, and vascular severity score., Results: Infants who demonstrated AP-ROP were more premature by birth weight (617 g vs. 679 g; P = 0.01) and gestational age (24.3 weeks vs. 25.0 weeks; P < 0.01) and reached peak severity at an earlier postmenstrual age (34.7 weeks vs. 36.9 weeks; P < 0.001) compared with infants with TR without AP-ROP. The mean vascular severity score was greatest in TR with AP-ROP infants compared with TR without AP-ROP infants (8.79 vs. 7.19; P < 0.001). Analyzing the severity score over time, the rate of progression was fastest in infants with AP-ROP (P < 0.002 at 30-32 weeks)., Conclusions: Premature infants in North America with AP-ROP are born younger and demonstrate disease earlier than infants with less severe ROP. Disease severity is quantifiable with a deep learning-based score, which correlates with clinically identified categories of disease, including AP-ROP. The rate of progression to peak disease is greatest in eyes that demonstrate AP-ROP compared with other treatment-requiring eyes. Analysis of quantitative characteristics of AP-ROP may help improve diagnosis and treatment of an aggressive, vision-threatening form of ROP., (Copyright © 2020 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.)

Background: Machine learning (ML) has been increasingly used in medicine and neurosurgery. We sought to determine whether ML models can distinguish ruptured from unruptured aneurysms and identify features associated with rupture., Methods: We performed a retrospective review of patients with intracranial aneurysms detected on vascular imaging at our institution between 2002 and 2018. The dataset was used to train 3 ML models (random forest, linear support vector machine [SVM], and radial basis function kernel SVM). Relative contributions of individual predictors were derived from the linear SVM model., Results: Complete data were available for 845 aneurysms in 615 patients. Ruptured aneurysms (n = 309, 37%) were larger (mean 6.51 mm vs. 5.73 mm; P = 0.02) and more likely to be in the posterior circulation (20% vs. 11%; P < 0.001) than unruptured aneurysms. Area under the receiver operating curve was 0.77 for the linear SVM, 0.78 for the radial basis function kernel SVM models, and 0.81 for the random forest model. Aneurysm location and size were the 2 features that contributed most significantly to the model. Posterior communicating artery, anterior communicating artery, and posterior inferior cerebellar artery locations were most highly associated with rupture, whereas paraclinoid and middle cerebral artery locations had the strongest association with unruptured status., Conclusions: ML models are capable of accurately distinguishing ruptured from unruptured aneurysms and identifying features associated with rupture. Consistent with prior studies, location and size show the strongest association with aneurysm rupture., (Copyright © 2019. Published by Elsevier Inc.)

Purpose: To determine expert agreement on relative retinopathy of prematurity (ROP) disease severity and whether computer-based image analysis can model relative disease severity, and to propose consideration of a more continuous severity score for ROP., Design: We developed 2 databases of clinical images of varying disease severity (100 images and 34 images) as part of the Imaging and Informatics in ROP (i-ROP) cohort study and recruited expert physician, nonexpert physician, and nonphysician graders to classify and perform pairwise comparisons on both databases., Participants: Six participating expert ROP clinician-scientists, each with a minimum of 10 years of clinical ROP experience and 5 ROP publications, and 5 image graders (3 physicians and 2 nonphysician graders) who analyzed images that were obtained during routine ROP screening in neonatal intensive care units., Methods: Images in both databases were ranked by average disease classification (classification ranking), by pairwise comparison using the Elo rating method (comparison ranking), and by correlation with the i-ROP computer-based image analysis system., Main Outcome Measures: Interexpert agreement (weighted κ statistic) compared with the correlation coefficient (CC) between experts on pairwise comparisons and correlation between expert rankings and computer-based image analysis modeling., Results: There was variable interexpert agreement on diagnostic classification of disease (plus, preplus, or normal) among the 6 experts (mean weighted κ, 0.27; range, 0.06-0.63), but good correlation between experts on comparison ranking of disease severity (mean CC, 0.84; range, 0.74-0.93) on the set of 34 images. Comparison ranking provided a severity ranking that was in good agreement with ranking obtained by classification ranking (CC, 0.92). Comparison ranking on the larger dataset by both expert and nonexpert graders demonstrated good correlation (mean CC, 0.97; range, 0.95-0.98). The i-ROP system was able to model this continuous severity with good correlation (CC, 0.86)., Conclusions: Experts diagnose plus disease on a continuum, with poor absolute agreement on classification but good relative agreement on disease severity. These results suggest that the use of pairwise rankings and a continuous severity score, such as that provided by the i-ROP system, may improve agreement on disease severity in the future., (Copyright © 2016 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.)

Purpose: To identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP)., Design: We developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD., Participants: Eight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units., Methods: Expert classification of images of plus disease in ROP., Main Outcome Measures: Interexpert agreement (weighted κ statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement)., Results: There was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted κ, 0-0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R 2  = 0.82; and dataset B: P < 0.05 and adjusted R 2  = 0.6615)., Conclusions: There is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future., (Copyright © 2016 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.)

Purpose: A number of studies have previously assessed the role of teaching interventions to improve organ-at-risk (OAR) delineation. We present a preliminary study demonstrating the benefit of a combined atlas and real time software-based feedback intervention to aid in contouring of OARs in the head and neck., Methods and Materials: The study consisted of a baseline evaluation, a real-time feedback intervention, atlas presentation, and a follow-up evaluation. At baseline evaluation, 8 resident observers contoured 26 OARs on a computed tomography scan without intervention or aid. They then received feedback comparing their contours both statistically and graphically to a set of atlas-based expert contours. Additionally, they received access to an atlas to contour these structures. The resident observers were then asked to contour the same 26 OARs on a separate computed tomography scan with atlas access. In addition, 6 experts (5 radiation oncologists specializing in the head and neck, and 1 neuroradiologist) contoured the 26 OARs on both scans. A simultaneous truth and performance level estimation (STAPLE) composite of the expert contours was used as a gold-standard set for analysis of OAR contouring., Results: Of the 8 resident observers who initially participated in the study, 7 completed both phases of the study. Dice similarity coefficients were calculated for each user-drawn structure relative to the expert STAPLE composite for each structure. Mean dice similarity coefficients across all structures increased between phase 1 and phase 2 for each resident observer, demonstrating a statistically significant improvement in overall OAR-contouring ability (P < .01). Additionally, intervention improved contouring in 16/26 delineated organs-at-risk across resident observers at a statistically significant level (P ≤ .05) including all otic structures and suprahyoid lymph node levels of the head and neck., Conclusions: Our data suggest that a combined atlas and real-time feedback-based educational intervention detectably improves contouring of OARs in the head and neck., (Copyright © 2013 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.)

Quantitative analysis has tremendous but mostly unrealized potential in healthcare to support objective and accurate interpretation of the clinical imaging. In 2008, the National Cancer Institute began building the Quantitative Imaging Network (QIN) initiative with the goal of advancing quantitative imaging in the context of personalized therapy and evaluation of treatment response. Computerized analysis is an important component contributing to reproducibility and efficiency of the quantitative imaging techniques. The success of quantitative imaging is contingent on robust analysis methods and software tools to bring these methods from bench to bedside. 3D Slicer is a free open-source software application for medical image computing. As a clinical research tool, 3D Slicer is similar to a radiology workstation that supports versatile visualizations but also provides advanced functionality such as automated segmentation and registration for a variety of application domains. Unlike a typical radiology workstation, 3D Slicer is free and is not tied to specific hardware. As a programming platform, 3D Slicer facilitates translation and evaluation of the new quantitative methods by allowing the biomedical researcher to focus on the implementation of the algorithm and providing abstractions for the common tasks of data communication, visualization and user interface development. Compared to other tools that provide aspects of this functionality, 3D Slicer is fully open source and can be readily extended and redistributed. In addition, 3D Slicer is designed to facilitate the development of new functionality in the form of 3D Slicer extensions. In this paper, we present an overview of 3D Slicer as a platform for prototyping, development and evaluation of image analysis tools for clinical research applications. To illustrate the utility of the platform in the scope of QIN, we discuss several use cases of 3D Slicer by the existing QIN teams, and we elaborate on the future directions that can further facilitate development and validation of imaging biomarkers using 3D Slicer., (Copyright © 2012 Elsevier Inc. All rights reserved.)