Search

Background: Intended use statements (IUSs) are mandatory to obtain regulatory clearance for artificial intelligence (AI)-based medical devices in the European Union. In order to guide the safe use of AI-based medical devices, IUSs need to contain comprehensive and understandable information. This study analyzes the IUSs of CE-marked AI products listed on AIforRadiology.com for ambiguity and completeness., Methods: We retrieved 157 IUSs of CE-marked AI products listed on AIforRadiology.com in September 2022. Duplicate products (n = 1), discontinued products (n = 3), and duplicate statements (n = 14) were excluded. The resulting IUSs were assessed for the presence of 6 items: medical indication, part of the body, patient population, user profile, use environment, and operating principle. Disclaimers, defined as contra-indications or warnings in the IUS, were identified and compared with claims., Results: Of 139 AI products, the majority (n = 78) of IUSs mentioned 3 or less items. IUSs of only 7 products mentioned all 6 items. The intended body part (n = 115) and the operating principle (n = 116) were the most frequently mentioned components, while the intended use environment (n = 24) and intended patient population (n = 29) were mentioned less frequently. Fifty-six statements contained disclaimers that conflicted with the claims in 13 cases., Conclusion: The majority of IUSs of CE-marked AI-based medical devices lack substantial information and, in few cases, contradict the claims of the product., Critical Relevance Statement: To ensure correct usage and to avoid off-label use or foreseeable misuse of AI-based medical devices in radiology, manufacturers are encouraged to provide more comprehensive and less ambiguous intended use statements., Key Points: • Radiologists must know AI products' intended use to avoid off-label use or misuse. • Ninety-five percent (n = 132/139) of the intended use statements analyzed were incomplete. • Nine percent (n = 13) of the intended use statements held disclaimers contradicting the claim of the AI product. • Manufacturers and regulatory bodies must ensure that intended use statements are comprehensive., (© 2024. The Author(s).)

Objectives: To map the clinical use of CE-marked artificial intelligence (AI)-based software in radiology departments in the Netherlands (n = 69) between 2020 and 2022., Materials and Methods: Our AI network (one radiologist or AI representative per Dutch hospital organization) received a questionnaire each spring from 2020 to 2022 about AI product usage, financing, and obstacles to adoption. Products that were not listed on www.AIforRadiology.com by July 2022 were excluded from the analysis., Results: The number of respondents was 43 in 2020, 36 in 2021, and 33 in 2022. The number of departments using AI has been growing steadily (2020: 14, 2021: 19, 2022: 23). The diversity (2020: 7, 2021: 18, 2022: 34) and the number of total implementations (2020: 19, 2021: 38, 2022: 68) has rapidly increased. Seven implementations were discontinued in 2022. Four hospital organizations said to use an AI platform or marketplace for the deployment of AI solutions. AI is mostly used to support chest CT (17), neuro CT (17), and musculoskeletal radiograph (12) analysis. The budget for AI was reserved in 13 of the responding centers in both 2021 and 2022. The most important obstacles to the adoption of AI remained costs and IT integration. Of the respondents, 28% stated that the implemented AI products realized health improvement and 32% assumed both health improvement and cost savings., Conclusion: The adoption of AI products in radiology departments in the Netherlands is showing common signs of a developing market. The major obstacles to reaching widespread adoption are a lack of financial resources and IT integration difficulties., Clinical Relevance Statement: The clinical impact of AI starts with its adoption in daily clinical practice. Increased transparency around AI products being adopted, implementation obstacles, and impact may inspire increased collaboration and improved decision-making around the implementation and financing of AI products., Key Points: • The adoption of artificial intelligence products for radiology has steadily increased since 2020 to at least a third of the centers using AI in clinical practice in the Netherlands in 2022. • The main areas in which artificial intelligence products are used are lung nodule detection on CT, aided stroke diagnosis, and bone age prediction. • The majority of respondents experienced added value (decreased costs and/or improved outcomes) from using artificial intelligence-based software; however, major obstacles to adoption remain the costs and IT-related difficulties., (© 2023. The Author(s).)

Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand ( n = 95) and chest ( n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard ( r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI: 0.58, 0.69] and 0.57 years [95% CI: 0.52, 0.61]) and readers (0.68 years [95% CI: 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI: 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article . See also the editorial by Omoumi and Richiardi in this issue.

Purpose: Few studies have evaluated real-world performance of radiological AI-tools in clinical practice. Over one-year, we prospectively evaluated the use of AI software to support the detection of intracranial large vessel occlusions (LVO) on CT angiography (CTA)., Method: Quantitative measures (user log-in attempts, AI standalone performance) and qualitative data (user surveys) were reviewed by a key-user group at three timepoints. A total of 491 CTA studies of 460 patients were included for analysis., Results: The overall accuracy of the AI-tool for LVO detection and localization was 87.6%, sensitivity 69.1% and specificity 91.2%. Out of 81 LVOs, 31 of 34 (91%) M1 occlusions were detected correctly, 19 of 38 (50%) M2 occlusions, and 6 of 9 (67%) ICA occlusions. The product was considered user-friendly. The diagnostic confidence of the users for LVO detection remained the same over the year. The last measured net promotor score was -56%. The use of the AI-tool fluctuated over the year with a declining trend., Conclusions: Our pragmatic approach of evaluating the AI-tool used in clinical practice, helped us to monitor the usage, to estimate the perceived added value by the users of the AI-tool, and to make an informed decision about the continuation of the use of the AI-tool., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2023 The Authors.)

The Automation Platform (AP) is a software platform to support the workflow of radiologists and includes a stroke CT package with integrated artificial intelligence (AI) based tools. The aim of this study was to evaluate the diagnostic performance of the AP for the detection of intracranial large vessel occlusions (LVO) on conventional CT angiography (CTA), and the duration of CT processing in a cohort of acute stroke patients. The diagnostic performance for intracranial LVO detection on CTA by the AP was evaluated in a retrospective cohort of 100 acute stroke patients and compared to the diagnostic performance of five radiologists with different levels of experience. The reference standard was set by an independent neuroradiologist, with access to the readings of the different radiologists, clinical data, and follow-up. The data processing time of the AP for ICH detection on non-contrast CT, LVO detection on CTA, and the processing of CTP maps was assessed in a subset 60 patients of the retrospective cohort. This was compared to 13 radiologists, who were prospectively timed for the processing and reading of 21 stroke CTs. The AP showed shorter processing time of CTA (mean 60 versus 395 s) and CTP (mean 196 versus 243-349 s) as compared to radiologists, but showed lower sensitivity for LVO detection (sensitivity 77% of the AP vs mean sensitivity 87% of radiologists). If the AP would have been used as a stand-alone system, 1 ICA occlusion, 2 M1 occlusions and 8 M2 occlusions would have been missed, which would be eligible for mechanical thrombectomy. In conclusion, the AP showed shorter processing time of CTA and CTP as compared with radiologists, which illustrates the potential of the AP to speed-up the diagnostic work-up. However, its performance for LVO detection was lower as compared with radiologists, especially for M2 vessel occlusions., (© 2023. The Author(s).)

Since the introduction of artificial intelligence (AI) in radiology, the promise has been that it will improve health care and reduce costs. Has AI been able to fulfill that promise? We describe six clinical objectives that can be supported by AI: a more efficient workflow, shortened reading time, a reduction of dose and contrast agents, earlier detection of disease, improved diagnostic accuracy and more personalized diagnostics. We provide examples of use cases including the available scientific evidence for its impact based on a hierarchical model of efficacy. We conclude that the market is still maturing and little is known about the contribution of AI to clinical practice. More real-world monitoring of AI in clinical practice is expected to aid in determining the value of AI and making informed decisions on development, procurement and reimbursement., (© 2021. The Author(s).)

Artificial intelligence (AI) applications for chest radiography and chest CT are among the most developed applications in radiology. More than 40 certified AI products are available for chest radiography or chest CT. These AI products cover a wide range of abnormalities, including pneumonia, pneumothorax and lung cancer. Most applications are aimed at detecting disease, complemented by products that characterize or quantify tissue. At present, none of the thoracic AI products is specifically designed for the pediatric population. However, some products developed to detect tuberculosis in adults are also applicable to children. Software is under development to detect early changes of cystic fibrosis on chest CT, which could be an interesting application for pediatric radiology. In this review, we give an overview of current AI products in thoracic radiology and cover recent literature about AI in chest radiography, with a focus on pediatric radiology. We also discuss possible pediatric applications., (© 2021. The Author(s).)

Background: Limited evidence is available on the clinical impact of artificial intelligence (AI) in radiology. Early health technology assessment (HTA) is a methodology to assess the potential value of an innovation at an early stage. We use early HTA to evaluate the potential value of AI software in radiology. As a use-case, we evaluate the cost-effectiveness of AI software aiding the detection of intracranial large vessel occlusions (LVO) in stroke in comparison to standard care. We used a Markov based model from a societal perspective of the United Kingdom predominantly using stroke registry data complemented with pooled outcome data from large, randomized trials. Different scenarios were explored by varying missed diagnoses of LVOs, AI costs and AI performance. Other input parameters were varied to demonstrate model robustness. Results were reported in expected incremental costs (IC) and effects (IE) expressed in quality adjusted life years (QALYs)., Results: Applying the base case assumptions (6% missed diagnoses of LVOs by clinicians, $40 per AI analysis, 50% reduction of missed LVOs by AI), resulted in cost-savings and incremental QALYs over the projected lifetime (IC: - $156, - 0.23%; IE: + 0.01 QALYs, + 0.07%) per suspected ischemic stroke patient. For each yearly cohort of patients in the UK this translates to a total cost saving of $11 million., Conclusions: AI tools for LVO detection in emergency care have the potential to improve healthcare outcomes and save costs. We demonstrate how early HTA may be applied for the evaluation of clinically applied AI software for radiology., (© 2021. The Author(s).)

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2021 The Authors. Published by Elsevier B.V. All rights reserved.)

Objectives: Map the current landscape of commercially available artificial intelligence (AI) software for radiology and review the availability of their scientific evidence., Methods: We created an online overview of CE-marked AI software products for clinical radiology based on vendor-supplied product specifications ( www.aiforradiology.com ). Characteristics such as modality, subspeciality, main task, regulatory information, deployment, and pricing model were retrieved. We conducted an extensive literature search on the available scientific evidence of these products. Articles were classified according to a hierarchical model of efficacy., Results: The overview included 100 CE-marked AI products from 54 different vendors. For 64/100 products, there was no peer-reviewed evidence of its efficacy. We observed a large heterogeneity in deployment methods, pricing models, and regulatory classes. The evidence of the remaining 36/100 products comprised 237 papers that predominantly (65%) focused on diagnostic accuracy (efficacy level 2). From the 100 products, 18 had evidence that regarded level 3 or higher, validating the (potential) impact on diagnostic thinking, patient outcome, or costs. Half of the available evidence (116/237) were independent and not (co-)funded or (co-)authored by the vendor., Conclusions: Even though the commercial supply of AI software in radiology already holds 100 CE-marked products, we conclude that the sector is still in its infancy. For 64/100 products, peer-reviewed evidence on its efficacy is lacking. Only 18/100 AI products have demonstrated (potential) clinical impact., Key Points: • Artificial intelligence in radiology is still in its infancy even though already 100 CE-marked AI products are commercially available. • Only 36 out of 100 products have peer-reviewed evidence of which most studies demonstrate lower levels of efficacy. • There is a wide variety in deployment strategies, pricing models, and CE marking class of AI products for radiology.

Due to the upfront role of magnetic resonance imaging (MRI) for prostate cancer (PCa) diagnosis, a multitude of artificial intelligence (AI) applications have been suggested to aid in the diagnosis and detection of PCa. In this review, we provide an overview of the current field, including studies between 2018 and February 2021, describing AI algorithms for (1) lesion classification and (2) lesion detection for PCa. Our evaluation of 59 included studies showed that most research has been conducted for the task of PCa lesion classification (66%) followed by PCa lesion detection (34%). Studies showed large heterogeneity in cohort sizes, ranging between 18 to 499 patients (median = 162) combined with different approaches for performance validation. Furthermore, 85% of the studies reported on the stand-alone diagnostic accuracy, whereas 15% demonstrated the impact of AI on diagnostic thinking efficacy, indicating limited proof for the clinical utility of PCa AI applications. In order to introduce AI within the clinical workflow of PCa assessment, robustness and generalizability of AI applications need to be further validated utilizing external validation and clinical workflow experiments.

The human electroencephalogram (EEG) of sleep undergoes profound changes with age. These changes can be conceptualized as "brain age (BA)," which can be compared to chronological age to reflect the degree of deviation from normal aging. Here, we develop an interpretable machine learning model to predict BA based on 2 large sleep EEG data sets: the Massachusetts General Hospital (MGH) sleep lab data set (N = 2532; ages 18-80); and the Sleep Heart Health Study (SHHS, N = 1974; ages 40-80). The model obtains a mean absolute deviation of 7.6 years between BA and chronological age (CA) in healthy participants in the MGH data set. As validation, a subset of SHHS containing longitudinal EEGs 5.2 years apart shows an average of 5.4 years increase in BA. Participants with significant neurological or psychiatric disease exhibit a mean excess BA, or "brain age index" (BAI = BA-CA) of 4 years relative to healthy controls. Participants with hypertension and diabetes have a mean excess BA of 3.5 years. The findings raise the prospect of using the sleep EEG as a potential biomarker for healthy brain aging., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Objectives: Electroencephalography (EEG) is a central part of the medical evaluation for patients with neurological disorders. Training an algorithm to label the EEG normal vs abnormal seems challenging, because of EEG heterogeneity and dependence of contextual factors, including age and sleep stage. Our objectives were to validate prior work on an independent data set suggesting that deep learning methods can discriminate between normal vs abnormal EEGs, to understand whether age and sleep stage information can improve discrimination, and to understand what factors lead to errors., Methods: We train a deep convolutional neural network on a heterogeneous set of 8522 routine EEGs from the Massachusetts General Hospital. We explore several strategies for optimizing model performance, including accounting for age and sleep stage., Results: The area under the receiver operating characteristic curve (AUC) on an independent test set (n = 851) is 0.917 marginally improved by including age (AUC = 0.924), and both age and sleep stages (AUC = 0.925), though not statistically significant., Conclusions: The model architecture generalizes well to an independent dataset. Adding age and sleep stage to the model does not significantly improve performance., Significance: Insights learned from misclassified examples, and minimal improvement by adding sleep stage and age suggest fruitful directions for further research., (Copyright © 2018 International Federation of Clinical Neurophysiology. Published by Elsevier B.V. All rights reserved.)

To test the spectrum hypothesis--postulating that clinical and non-clinical samples are primarily differentiated by mean-level differences--, this study evaluates differences in parent-rated temperament, personality and maladjustment among a low-symptom (N = 81), a high-symptom (N = 94) ASD-group, and a comparison group (N = 500). These classic spectrum hypothesis tests are extended by adding tests for similarity in variances, reliabilities and patterns of covariation between relevant variables. Children with ASD exhibit more extreme means, except for dominance. The low- and high-symptom ASD-groups are primarily differentiated by mean sociability and internal distress. Striking similarities in reliability and pattern of covariation of variables suggest that comparable processes link traits to maladaptation in low- and high-symptom children with ASD and in children with and without autism.

International comparisons were conducted of preschool children's behavioral and emotional problems as reported on the Child Behavior Checklist for Ages 1½-5 by parents in 24 societies (N = 19,850). Item ratings were aggregated into scores on syndromes; Diagnostic and Statistical Manual of Mental Disorders-oriented scales; a Stress Problems scale; and Internalizing, Externalizing, and Total Problems scales. Effect sizes for scale score differences among the 24 societies ranged from small to medium (3-12%). Although societies differed greatly in language, culture, and other characteristics, Total Problems scores for 18 of the 24 societies were within 7.1 points of the omnicultural mean of 33.3 (on a scale of 0-198). Gender and age differences, as well as gender and age interactions with society, were all very small (effect sizes < 1%). Across all pairs of societies, correlations between mean item ratings averaged .78, and correlations between internal consistency alphas for the scales averaged .92, indicating that the rank orders of mean item ratings and internal consistencies of scales were very similar across diverse societies.

Objective: To test the fit of a seven-syndrome model to ratings of preschoolers' problems by parents in very diverse societies., Method: Parents of 19,106 children 18 to 71 months of age from 23 societies in Asia, Australasia, Europe, the Middle East, and South America completed the Child Behavior Checklist for Ages 1.5-5 (CBCL/1.5-5). Confirmatory factor analyses were used to test the seven-syndrome model separately for each society., Results: The primary model fit index, the root mean square error of approximation (RMSEA), indicated acceptable to good fit for each society. Although a six-syndrome model combining the Emotionally Reactive and Anxious/Depressed syndromes also fit the data for nine societies, it fit less well than the seven-syndrome model for seven of the nine societies. Other fit indices yielded less consistent results than the RMSEA., Conclusions: The seven-syndrome model provides one way to capture patterns of children's problems that are manifested in ratings by parents from many societies. Clinicians working with preschoolers from these societies can thus assess and describe parents' ratings of behavioral, emotional, and social problems in terms of the seven syndromes. The results illustrate possibilities for culture-general taxonomic constructs of preschool psychopathology. Problems not captured by the CBCL/1.5-5 may form additional syndromes, and other syndrome models may also fit the data., (Copyright © 2010 American Academy of Child and Adolescent Psychiatry. Published by Elsevier Inc. All rights reserved.)

The lack of empirical research relating temperament models and personality hinders conceptual integration and holds back research linking childhood traits to problem behavior or maladjustment. This study evaluates, within a sample of 443 preschoolers, the relationships between children's maladaptation and traits measured by three temperament models (Thomas and Chess, Buss and Plomin, and Rothbart), and a Five-Factor based personality model. Adequate reliabilities and expected factor structures are demonstrated for most scales. A joint principal component analysis combining 28 temperament and 18 personality scales indicates a six-factor model, distinguishing Sociability, Activity, Conscientiousness, Disagreeableness, Emotionality, and Sensitivity. Regression analyses reveal that although single temperament and personality scales explain from 23% to 37% of problem behavior variance, the six components explain from 41% to 49% and provide a clearer differentiation among CBCL-problem scales. This age-specific taxonomy refines and corroborates conclusions based on narrative reviews and furnishes a more balanced view of trait-maladjustment relationships.

This study examines 5 types of personality continuity--structural, mean-level, individual-level, differential, and ipsative--in a representative population (N=498) and a twin and sibling sample (N=548) of children and adolescents. Parents described their children on 2 successive occasions with a 36-month interval using the Hierarchical Personality Inventory for Children (I. Mervielde & F. De Fruyt, 1999). There was evidence for structural continuity in the 2 samples, and personality was shown to be largely differentially stable. A large percentage had a stable trait profile indicative of ipsative stability, and mean-level personality changes were generally small in magnitude. Continuity findings were explained mainly by genetic and nonshared environmental factors., (((c) 2006 APA, all rights reserved).)

Parenting x Child Personality interactions in predicting child externalizing and internalizing behavior were investigated in a variable-centered study and a person-centered study. The variable-centered study used data from a 3-year longitudinal study of 600 children 7 to 15 years old at Time 1 and 512 children 10 to 18 years old at Time 2. Parents rated child personality (five factor model), negative control, positive parenting, and child problem behavior, whereas children rated parental behavior. Hierarchical moderated regression analyses showed significant Parenting x Child Personality (benevolence and conscientiousness) interactions, principally for externalizing behavior. The interactions were largely replicable across informants and across time. The person-centered study, which classified participants into 3 types, showed that negative parental control was more related to externalizing behavior for undercontrollers than for resilients. Negative parental control enhanced internalizing behavior for overcontrollers., ((c) 2004 APA, all rights reserved)