Search

Objectives: To investigate the intra- and inter-rater reliability of the total radiomics quality score (RQS) and the reproducibility of individual RQS items' score in a large multireader study., Methods: Nine raters with different backgrounds were randomly assigned to three groups based on their proficiency with RQS utilization: Groups 1 and 2 represented the inter-rater reliability groups with or without prior training in RQS, respectively; group 3 represented the intra-rater reliability group. Thirty-three original research papers on radiomics were evaluated by raters of groups 1 and 2. Of the 33 papers, 17 were evaluated twice with an interval of 1 month by raters of group 3. Intraclass coefficient (ICC) for continuous variables, and Fleiss' and Cohen's kappa (k) statistics for categorical variables were used., Results: The inter-rater reliability was poor to moderate for total RQS (ICC 0.30-055, p < 0.001) and very low to good for item's reproducibility (k - 0.12 to 0.75) within groups 1 and 2 for both inexperienced and experienced raters. The intra-rater reliability for total RQS was moderate for the less experienced rater (ICC 0.522, p = 0.009), whereas experienced raters showed excellent intra-rater reliability (ICC 0.91-0.99, p < 0.001) between the first and second read. Intra-rater reliability on RQS items' score reproducibility was higher and most of the items had moderate to good intra-rater reliability (k - 0.40 to 1)., Conclusions: Reproducibility of the total RQS and the score of individual RQS items is low. There is a need for a robust and reproducible assessment method to assess the quality of radiomics research., Clinical Relevance Statement: There is a need for reproducible scoring systems to improve quality of radiomics research and consecutively close the translational gap between research and clinical implementation., Key Points: • Radiomics quality score has been widely used for the evaluation of radiomics studies. • Although the intra-rater reliability was moderate to excellent, intra- and inter-rater reliability of total score and point-by-point scores were low with radiomics quality score. • A robust, easy-to-use scoring system is needed for the evaluation of radiomics research., (© 2023. The Author(s).)

Objective: Our study objective was to explore the additional value of dual-energy CT (DECT) material decomposition for squamous cell carcinoma of the head and neck (SCCHN) survival prognostication., Methods: A group of 50 SCCHN patients (male, 37; female, 13; mean age, 63.6 ± 10.82 years) with baseline head and neck DECT between September 2014 and August 2020 were retrospectively included. Primary tumors were segmented, radiomics features were extracted, and DECT material decomposition was performed. We used independent train and validation datasets with cross-validation and 100 independent iterations to identify prognostic signatures applying elastic net (EN) and random survival forest (RSF). Features were ranked and intercorrelated according to their prognostic importance. We benchmarked the models against clinical parameters. Intraclass correlation coefficients were used to analyze the interreader variation., Results: The exclusively radiomics-trained models achieved similar ( P = 0.947) prognostic performance of area under the curve (AUC) = 0.784 (95% confidence interval [CI], 0.775-0.812) (EN) and AUC = 0.785 (95% CI, 0.759-0.812) (RSF). The additional application of DECT material decomposition did not improve the model's performance (EN, P = 0.594; RSF, P = 0.198). In the clinical benchmark, the top averaged AUC value of 0.643 (95% CI, 0.611-0.675) was inferior to the quantitative imaging-biomarker models ( P < 0.001). A combined imaging and clinical model did not improve the imaging-based models ( P > 0.101). Shape features revealed high prognostic importance., Conclusions: Radiomics AI applications may be used for SCCHN survival prognostication, but the spectral information of DECT material decomposition did not improve the model's performance in our preliminary investigation., Competing Interests: The authors declare no conflict of interest., (Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.)

Objectives: To investigate the model-, code-, and data-sharing practices in the current radiomics research landscape and to introduce a radiomics research database., Methods: A total of 1254 articles published between January 1, 2021, and December 31, 2022, in leading radiology journals (European Radiology, European Journal of Radiology, Radiology, Radiology: Artificial Intelligence, Radiology: Cardiothoracic Imaging, Radiology: Imaging Cancer) were retrospectively screened, and 257 original research articles were included in this study. The categorical variables were compared using Fisher's exact tests or chi-square test and numerical variables using Student's t test with relation to the year of publication., Results: Half of the articles (128 of 257) shared the model by either including the final model formula or reporting the coefficients of selected radiomics features. A total of 73 (28%) models were validated on an external independent dataset. Only 16 (6%) articles shared the data or used publicly available open datasets. Similarly, only 20 (7%) of the articles shared the code. A total of 7 (3%) articles both shared code and data. All collected data in this study is presented in a radiomics research database (RadBase) and could be accessed at https://github.com/EuSoMII/RadBase ., Conclusion: According to the results of this study, the majority of published radiomics models were not technically reproducible since they shared neither model nor code and data. There is still room for improvement in carrying out reproducible and open research in the field of radiomics., Clinical Relevance Statement: To date, the reproducibility of radiomics research and open science practices within the radiomics research community are still very low. Ensuring reproducible radiomics research with model-, code-, and data-sharing practices will facilitate faster clinical translation., Key Points: • There is a discrepancy between the number of published radiomics papers and the clinical implementation of these published radiomics models. • The main obstacle to clinical implementation is the lack of model-, code-, and data-sharing practices. • In order to translate radiomics research into clinical practice, the radiomics research community should adopt open science practices., (© 2023. The Author(s).)

Be sure your imaging data is heterogeneous with distributional similarity among data partitions. The dataset must be split appropriately before performing any action on the imaging data set, such as image and radiomics feature preprocessing and feature dimensionality reduction [[13]]. Several algorithms perform better when feature values are in a common scale (i.e., feature scaling), which must be fitted exclusively to the training data [[18]]. Feature selection or data transformations should also be applied to reduce the dimensionality of the data. [Extracted from the article]

Background: Treatment plans for squamous cell carcinoma of the head and neck (SCCHN) are individually decided in tumor board meetings but some treatment decision-steps lack objective prognostic estimates. Our purpose was to explore the potential of radiomics for SCCHN therapy-specific survival prognostication and to increase the models' interpretability by ranking the features based on their predictive importance. Methods: We included 157 SCCHN patients (male, 119; female, 38; mean age, 64.39 ± 10.71 years) with baseline head and neck CT between 09/2014 and 08/2020 in this retrospective study. Patients were stratified according to their treatment. Using independent training and test datasets with cross-validation and 100 iterations, we identified, ranked and inter-correlated prognostic signatures using elastic net (EN) and random survival forest (RSF). We benchmarked the models against clinical parameters. Inter-reader variation was analyzed using intraclass-correlation coefficients (ICC). Results: EN and RSF achieved top prognostication performances of AUC = 0.795 (95% CI 0.767–0.822) and AUC = 0.811 (95% CI 0.782–0.839). RSF prognostication slightly outperformed the EN for the complete (ΔAUC 0.035, p = 0.002) and radiochemotherapy (ΔAUC 0.092, p < 0.001) cohort. RSF was superior to most clinical benchmarking (p ≤ 0.006). The inter-reader correlation was moderate or high for all features classes (ICC ≥ 0.77 (± 0.19)). Shape features had the highest prognostic importance, followed by texture features. Conclusions: EN and RSF built on radiomics features may be used for survival prognostication. The prognostically leading features may vary between treatment subgroups. This warrants further validation to potentially aid clinical treatment decision making in the future. [ABSTRACT FROM AUTHOR]

Objectives: To systematically review current research applications of radiomics in patients with cholangiocarcinoma and to assess the quality of CT and MRI radiomics studies. Methods: A systematic search was conducted on PubMed/Medline, Web of Science, and Scopus databases to identify original studies assessing radiomics of cholangiocarcinoma on CT and/or MRI. Three readers with different experience levels independently assessed quality of the studies using the radiomics quality score (RQS). Subgroup analyses were performed according to journal type, year of publication, quartile and impact factor (from the Journal Citation Report database), type of cholangiocarcinoma, imaging modality, and number of patients. Results: A total of 38 original studies including 6242 patients (median 134 patients) were selected. The median RQS was 9 (corresponding to 25.0% of the total RQS; IQR 1–13) for reader 1, 8 (22.2%, IQR 3–12) for reader 2, and 10 (27.8%; IQR 5–14) for reader 3. The inter-reader agreement was good with an ICC of 0.75 (95% CI 0.62–0.85) for the total RQS. All studies were retrospective and none of them had phantom assessment, imaging at multiple time points, nor performed cost-effectiveness analysis. The RQS was significantly higher in studies published in journals with impact factor > 4 (median 11 vs. 4, p = 0.048 for reader 1) and including more than 100 patients (median 11.5 vs. 0.5, p < 0.001 for reader 1). Conclusions: Quality of radiomics studies on cholangiocarcinoma is insufficient based on the radiomics quality score. Future research should consider prospective studies with a standardized methodology, validation in multi-institutional external cohorts, and open science data. Key points: The quality of current radiomics studies on cholangiocarcinoma is insufficient, with a median radiomics quality score of 8–10, corresponding to 22–28% of the ideal quality score. None of the current studies conducted phantom assessment, imaging at multiple time points, prospective registration in a trial database, nor cost-effectiveness analysis. The inter-reader agreement of the radiomics quality score is good (ICC of 0.75; 95% CI 0.62–0.85) among readers with different levels of experience. [ABSTRACT FROM AUTHOR]

Simple Summary: Portal vein infiltration (PVI) is a complication of HCC with critical impact on further patient management as systemic therapies are recommended once PVI is diagnosed. In our study, we matched 44 patients with HCC who developed PVI in the course of disease with no CT-detectable PVI at initial diagnosis to the same number of patients who never developed PVI during follow-up, but showed the same conventional tumor traits (size and number of lesions, growth type, contrast enhancement pattern, etc.). Using LASSO regression, radiomics feature analysis showed a sensitivity and specificity of 0.78 to detect the occurrence of PVI in the validation set. Therefore, an additional radiomics evaluation at initial diagnosis could help to identify patients benefiting from a closer surveillance. Portal vein infiltration (PVI) is a typical complication of HCC. Once diagnosed, it leads to classification as BCLC C with an enormous impact on patient management, as systemic therapies are henceforth recommended. Our aim was to investigate whether radiomics analysis using imaging at initial diagnosis can predict the occurrence of PVI in the course of disease. Between 2008 and 2018, we retrospectively identified 44 patients with HCC and an in-house, multiphase CT scan at initial diagnosis who presented without CT-detectable PVI but developed it in the course of disease. Accounting for size and number of lesions, growth type, arterial enhancement pattern, Child–Pugh stage, AFP levels, and subsequent therapy, we matched 44 patients with HCC who did not develop PVI to those developing PVI in the course of disease (follow-up ended December 2021). After segmentation of the tumor at initial diagnosis and texture analysis, we used LASSO regression to find radiomics features suitable for PVI detection in this matched set. Using an 80:20 split between training and holdout validation dataset, 17 radiomics features remained in the fitted model. Applying the model to the holdout validation dataset, sensitivity to detect occurrence of PVI was 0.78 and specificity was 0.78. Radiomics feature extraction had the ability to detect aggressive HCC morphology likely to result in future PVI. An additional radiomics evaluation at initial diagnosis might be a useful tool to identify patients with HCC at risk for PVI during follow-up benefiting from a closer surveillance. [ABSTRACT FROM AUTHOR]

Our purpose was to analyze the robustness and reproducibility of magnetic resonance imaging (MRI) radiomic features. We constructed a multi-object fruit phantom to perform MRI acquisition as scan-rescan using a 3 Tesla MRI scanner. We applied T2-weighted (T2w) half-Fourier acquisition single-shot turbo spin-echo (HASTE), T2w turbo spin-echo (TSE), T2w fluid-attenuated inversion recovery (FLAIR), T2 map and T1-weighted (T1w) TSE. Images were resampled to isotropic voxels. Fruits were segmented. The workflow was repeated by a second reader and the first reader after a pause of one month. We applied PyRadiomics to extract 107 radiomic features per fruit and sequence from seven feature classes. We calculated concordance correlation coefficients (CCC) and dynamic range (DR) to obtain measurements of feature robustness. Intraclass correlation coefficient (ICC) was calculated to assess intra- and inter-observer reproducibility. We calculated Gini scores to test the pairwise discriminative power specific for the features and MRI sequences. We depict Bland Altmann plots of features with top discriminative power (Mann–Whitney U test). Shape features were the most robust feature class. T2 map was the most robust imaging technique (robust features (rf), n = 84). HASTE sequence led to the least amount of rf (n = 20). Intra-observer ICC was excellent (≥ 0.75) for nearly all features (max–min; 99.1–97.2%). Deterioration of ICC values was seen in the inter-observer analyses (max–min; 88.7–81.1%). Complete robustness across all sequences was found for 8 features. Shape features and T2 map yielded the highest pairwise discriminative performance. Radiomics validity depends on the MRI sequence and feature class. T2 map seems to be the most promising imaging technique with the highest feature robustness, high intra-/inter-observer reproducibility and most promising discriminative power. [ABSTRACT FROM AUTHOR]

Key Points: • Radiomics might help predict survival of patients with lower-grade gliomas. • Several different models using different radiomics features have been proposed with only little overlap in included features. • Prospective trials and validation studies are needed to establish which models offer clinical benefit and which do not. [ABSTRACT FROM AUTHOR]

Simple Summary: Virtual monoenergetic images from dual-energy CT are incrementally used in routine clinical practice. Thus, radiomic analysis will be more often performed on these images in the future. This study characterized the test–retest repeatability and reproducibility of radiomic features from virtual monoenergetic images and their impact on machine-learning-based lesion classification. The results of this study provide a basis to improve radiomic analyses and identify the role of feature stability in classification tasks when using virtual monoenergetic imaging with different scan or reconstruction parameters in multicenter clinical studies. The purpose of this study was to (i) evaluate the test–retest repeatability and reproducibility of radiomic features in virtual monoenergetic images (VMI) from dual-energy CT (DECT) depending on VMI energy (40, 50, 75, 120, 190 keV), radiation dose (5 and 15 mGy), and DECT approach (dual-source and split-filter DECT) in a phantom (ex vivo), and (ii) to assess the impact of VMI energy and feature repeatability on machine-learning-based classification in vivo in 72 patients with 72 hypodense liver lesions. Feature repeatability and reproducibility were determined by concordance–correlation–coefficient (CCC) and dynamic range (DR) ≥0.9. Test–retest repeatability was high within the same VMI energies and scan conditions (percentage of repeatable features ranging from 74% for SFDE mode at 40 keV and 15 mGy to 86% for DSDE at 190 keV and 15 mGy), while reproducibility varied substantially across different VMI energies and DECTs (percentage of reproducible features ranging from 32.8% for SFDE at 5 mGy comparing 40 with 190 keV to 99.2% for DSDE at 15 mGy comparing 40 with 50 keV). No major differences were observed between the two radiation doses (<10%) in all pair-wise comparisons. In vivo, machine learning classification using penalized regression and random forests resulted in the best discrimination of hemangiomas and metastases at low-energy VMI (40 keV), and for cysts at high-energy VMI (120 keV). Feature selection based on feature repeatability did not improve classification performance. Our results demonstrate the high repeatability of radiomics features when keeping scan and reconstruction conditions constant. Reproducibility diminished when using different VMI energies or DECT approaches. The choice of optimal VMI energy improved lesion classification in vivo and should hence be adapted to the specific task. [ABSTRACT FROM AUTHOR]