Search

Abstract Purpose To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. Methods We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. Result In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. Conclusion In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. Critical relevance statement A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. Key points • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ). Graphical Abstract

Objectives: To develop a mutation-based radiomics signature to predict response to imatinib in Gastrointestinal Stromal Tumors (GISTs). Methods: Eighty-two patients with GIST were enrolled in this retrospective study, including 52 patients from one center that were used to develop the model, and 30 patients from a second center to validate it. Reference standard was the mutational status of tyrosine-protein kinase (KIT) and platelet-derived growth factor α (PDGFRA). Patients were dichotomized in imatinib sensitive (group 0 - mutation in KIT or PDGFRA, different from exon 18-D842V), and imatinib non-responsive (group 1 - PDGFRA exon 18-D842V mutation or absence of mutation in KIT/PDGFRA). Initially, 107 texture features were extracted from the tumor masks of baseline computed tomography scans. Different machine learning methods were then implemented to select the best combination of features for the development of the radiomics signature. Results: The best performance was obtained with the 5 features selected by the ANOVA model and the Bayes classifier, using a threshold of 0.36. With this setting the radiomics signature had an accuracy and precision for sensitive patients of 82 % (95 % CI:60–95) and 90 % (95 % CI:73–97), respectively. Conversely, a precision of 80 % (95 % CI:34–97) was obtained in non-responsive patients using a threshold of 0.9. Indeed, with the latter setting 4 patients out of 5 were correctly predicted as non-responders. Conclusions: The results are a first step towards using radiomics to improve the management of patients with GIST, especially when tumor tissue is unavailable for molecular analysis or when molecular profiling is inconclusive.

Goal: Artificial intelligence applied to medical image analysis has been extensively used to develop non-invasive diagnostic and prognostic signatures. However, these imaging biomarkers should be largely validated on multi-center datasets to prove their robustness before they can be introduced into clinical practice. The main challenge is represented by the great and unavoidable image variability which is usually addressed using different pre-processing techniques including spatial, intensity and feature normalization. The purpose of this study is to systematically summarize normalization methods and to evaluate their correlation with the radiomics model performances through meta-analyses. This review is carried out according to the PRISMA statement: 4777 papers were collected, but only 74 were included. Two meta-analyses were carried out according to two clinical aims: characterization and prediction of response. Findings of this review demonstrated that there are some commonly used normalization approaches, but not a commonly agreed pipeline that can allow to improve performance and to bridge the gap between bench and bedside.

Abstract Background Pathological complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer (LARC) is achieved in 15–30% of cases. Our aim was to implement and externally validate a magnetic resonance imaging (MRI)-based radiomics pipeline to predict response to treatment and to investigate the impact of manual and automatic segmentations on the radiomics models. Methods Ninety-five patients with stage II/III LARC who underwent multiparametric MRI before chemoradiotherapy and surgical treatment were enrolled from three institutions. Patients were classified as responders if tumour regression grade was 1 or 2 and nonresponders otherwise. Sixty-seven patients composed the construction dataset, while 28 the external validation. Tumour volumes were manually and automatically segmented using a U-net algorithm. Three approaches for feature selection were tested and combined with four machine learning classifiers. Results Using manual segmentation, the best result reached an accuracy of 68% on the validation set, with sensitivity 60%, specificity 77%, negative predictive value (NPV) 63%, and positive predictive value (PPV) 75%. The automatic segmentation achieved an accuracy of 75% on the validation set, with sensitivity 80%, specificity 69%, and both NPV and PPV 75%. Sensitivity and NPV on the validation set were significantly higher (p = 0.047) for the automatic versus manual segmentation. Conclusion Our study showed that radiomics models can pave the way to help clinicians in the prediction of tumour response to chemoradiotherapy of LARC and to personalise per-patient treatment. The results from the external validation dataset are promising for further research into radiomics approaches using both manual and automatic segmentations.

High-resolution intraoperative PET/CT specimen imaging, coupled with prostate-specific membrane antigen (PSMA) molecular targeting, holds great potential for the rapid ex vivo identification of disease localizations in high-risk prostate cancer patients undergoing surgery. However, the accurate analysis of radiotracer uptake would require time-consuming manual volumetric segmentation of 3D images. The aim of this study was to test the feasibility of using machine learning to perform automatic nodal segmentation of intraoperative 68Ga-PSMA-11 PET/CT specimen images. Six (n = 6) lymph-nodal specimens were imaged in the operating room after an e.v. injection of 2.1 MBq/kg of 68Ga-PSMA-11. A machine learning-based approach for automatic lymph-nodal segmentation was developed using only open-source Python libraries (Scikit-learn, SciPy, Scikit-image). The implementation of a k-means clustering algorithm (n = 3 clusters) allowed to identify lymph-nodal structures by leveraging differences in tissue density. Refinement of the segmentation masks was performed using morphological operations and 2D/3D-features filtering. Compared to manual segmentation (ITK-SNAP v4.0.1), the automatic segmentation model showed promising results in terms of weighted average precision (97–99%), recall (68–81%), Dice coefficient (80–88%) and Jaccard index (67–79%). Finally, the ML-based segmentation masks allowed to automatically compute semi-quantitative PET metrics (i.e., SUVmax), thus holding promise for facilitating the semi-quantitative analysis of PET/CT images in the operating room.

Abstract Background Radiomics is expected to improve the management of metastatic colorectal cancer (CRC). We aimed at evaluating the impact of liver lesion contouring as a source of variability on radiomic features (RFs). Methods After Ethics Committee approval, 70 liver metastases in 17 CRC patients were segmented on contrast-enhanced computed tomography scans by two residents and checked by experienced radiologists. RFs from grey level co-occurrence and run length matrices were extracted from three-dimensional (3D) regions of interest (ROIs) and the largest two-dimensional (2D) ROIs. Inter-reader variability was evaluated with Dice coefficient and Hausdorff distance, whilst its impact on RFs was assessed using mean relative change (MRC) and intraclass correlation coefficient (ICC). For the main lesion of each patient, one reader also segmented a circular ROI on the same image used for the 2D ROI. Results The best inter-reader contouring agreement was observed for 2D ROIs according to both Dice coefficient (median 0.85, interquartile range 0.78–0.89) and Hausdorff distance (0.21 mm, 0.14–0.31 mm). Comparing RF values, MRC ranged 0–752% for 2D and 0–1567% for 3D. For 24/32 RFs (75%), MRC was lower for 2D than for 3D. An ICC > 0.90 was observed for more RFs for 2D (53%) than for 3D (34%). Only 2/32 RFs (6%) showed a variability between 2D and circular ROIs higher than inter-reader variability. Conclusions A 2D contouring approach may help mitigate overall inter-reader variability, albeit stable RFs can be extracted from both 3D and 2D segmentations of CRC liver metastases.

In the last years, the widespread use of the prostate-specific antigen (PSA) blood examination to triage patients who will enter the diagnostic/therapeutic path for prostate cancer (PCa) has almost halved PCa-specific mortality. As a counterpart, millions of men with clinically insignificant cancer not destined to cause death are treated, with no beneficial impact on overall survival. Therefore, there is a compelling need to develop tools that can help in stratifying patients according to their risk, to support physicians in the selection of the most appropriate treatment option for each individual patient. The aim of this study was to develop and validate on multivendor data a fully automated computer-aided diagnosis (CAD) system to detect and characterize PCas according to their aggressiveness. We propose a CAD system based on artificial intelligence algorithms that a) registers all images coming from different MRI sequences, b) provides candidates suspicious to be tumor, and c) provides an aggressiveness score of each candidate based on the results of a support vector machine classifier fed with radiomics features. The dataset was composed of 131 patients (149 tumors) from two different institutions that were divided in a training set, a narrow validation set, and an external validation set. The algorithm reached an area under the receiver operating characteristic (ROC) curve in distinguishing between low and high aggressive tumors of 0.96 and 0.81 on the training and validation sets, respectively. Moreover, when the output of the classifier was divided into three classes of risk, i.e., indolent, indeterminate, and aggressive, our method did not classify any aggressive tumor as indolent, meaning that, according to our score, all aggressive tumors would undergo treatment or further investigations. Our CAD performance is superior to that of previous studies and overcomes some of their limitations, such as the need to perform manual segmentation of the tumor or the fact that analysis is limited to single-center datasets. The results of this study are promising and could pave the way to a prediction tool for personalized decision making in patients harboring PCa.

Abstract Background Preoperative evaluation of nipple-areola complex (NAC) tumour involvement is crucial to select patients candidates for nipple-sparing mastectomy. Our aim was to validate a previously developed automated method able to compute the three-dimensional (3D) tumour-to-NAC distance (the most predictive parameter of nipple involvement), using magnetic resonance imaging (MRI) datasets acquired with a scanner and protocol different from those of the development phase. Methods We performed a retrospective analysis of 77 patients submitted to total mastectomy and preoperatively studied with MRI. The new method consisted of automated segmentation of both NAC and tumour and subsequent computation of the 3D distance between them; standard manual two-dimensional segmentation was independently performed. Paraffin-embedded section examination of the removed NAC was performed to identify the neoplastic involvement. The ability of both methods to discriminate between patients with and without NAC involvement was compared using receiver operating characteristic (ROC) analysis. Results The 3D tumour-to-NAC distance was correctly computed for 72/77 patients (93.5%); tumour and NAC segmentation method failed in two and three cases, respectively. The diagnostic performance of the 3D automated method at best cut-off values was consistently better than that of the 2D manual method (sensitivity 78.3%, specificity 71.4%, positive predictive value 87.5%, negative predictive value 56.3%, and AUC 0.77 versus 73.9%, 61.2%, 47.2%, 83.3%, and 0.72, respectively), even if the difference did not reach statistical significance (p = 0.431). Conclusions The introduction of the 3D automated method in a clinical setting could improve the diagnostic performance in the preoperative assessment of NAC tumour involvement.

Magnetic resonance imaging (MRI) has a growing role in the clinical workup of prostate cancer. However, manual three-dimensional (3D) segmentation of the prostate is a laborious and time-consuming task. In this scenario, the use of automated algorithms for prostate segmentation allows us to bypass the huge workload of physicians. In this work, we propose a fully automated hybrid approach for prostate gland segmentation in MR images using an initial segmentation of prostate volumes using a custom-made 3D deep network (VNet-T2), followed by refinement using an Active Shape Model (ASM). While the deep network focuses on three-dimensional spatial coherence of the shape, the ASM relies on local image information and this joint effort allows for improved segmentation of the organ contours. Our method is developed and tested on a dataset composed of T2-weighted (T2w) MRI prostatic volumes of 60 male patients. In the test set, the proposed method shows excellent segmentation performance, achieving a mean dice score and Hausdorff distance of 0.851 and 7.55 mm, respectively. In the future, this algorithm could serve as an enabling technology for the development of computer-aided systems for prostate cancer characterization in MR imaging.

Recently, Computer Aided Diagnosis (CAD) systems have been proposed to help radiologists in detecting and characterizing Prostate Cancer (PCa). However, few studies evaluated the performances of these systems in a clinical setting, especially when used by non-experienced readers. The main aim of this study is to assess the diagnostic performance of non-experienced readers when reporting assisted by the likelihood map generated by a CAD system, and to compare the results with the unassisted interpretation. Three resident radiologists were asked to review multiparametric-MRI of patients with and without PCa, both unassisted and assisted by a CAD system. In both reading sessions, residents recorded all positive cases, and sensitivity, specificity, negative and positive predictive values were computed and compared. The dataset comprised 90 patients (45 with at least one clinically significant biopsy-confirmed PCa). Sensitivity significantly increased in the CAD assisted mode for patients with at least one clinically significant lesion (GS > 6) (68.7% vs. 78.1%, p = 0.018). Overall specificity was not statistically different between unassisted and assisted sessions (94.8% vs. 89.6, p = 0.072). The use of the CAD system significantly increases the per-patient sensitivity of inexperienced readers in the detection of clinically significant PCa, without negatively affecting specificity, while significantly reducing overall reporting time.

While cross-sectional imaging has seen continuous progress and plays an undiscussed pivotal role in the diagnostic management and treatment planning of patients with rectal cancer, a largely unmet need remains for improved staging accuracy, assessment of treatment response and prediction of individual patient outcome. Moreover, the increasing availability of target therapies has called for developing reliable diagnostic tools for identifying potential responders and optimizing overall treatment strategy on a personalized basis. Radiomics has emerged as a promising, still fully evolving research topic, which could harness the power of modern computer technology to generate quantitative information from imaging datasets based on advanced data-driven biomathematical models, potentially providing an added value to conventional imaging for improved patient management. The present study aimed to illustrate the contribution that current radiomics methods applied to magnetic resonance imaging can offer to managing patients with rectal cancer.