Search

Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy.We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use.64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed.We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice.Melanoma Research Alliance and La Marató de TV3.

Iron is essential to the virulence of Aspergillus species, and restricting iron availability is a critical mechanism of antimicrobial host defense. Macrophages recruited to the site of infection are at the crux of this process, employing multiple intersecting mechanisms to orchestrate iron sequestration from pathogens. To gain an integrated understanding of how this is achieved in aspergillosis, we generated a transcriptomic time series of the response of human monocyte-derived macrophages to Aspergillus and used this and the available literature to construct a mechanistic computational model of iron handling of macrophages during this infection. We found an overwhelming macrophage response beginning 2 to 4 h after exposure to the fungus, which included upregulated transcription of iron import proteins transferrin receptor-1, divalent metal transporter-1, and ZIP family transporters, and downregulated transcription of the iron exporter ferroportin. The computational model, based on a discrete dynamical systems framework, consisted of 21 3-state nodes, and was validated with additional experimental data that were not used in model generation. The model accurately captures the steady state and the trajectories of most of the quantitatively measured nodes. In the experimental data, we surprisingly found that transferrin receptor-1 upregulation preceded the induction of inflammatory cytokines, a feature that deviated from model predictions. Model simulations suggested that direct induction of transferrin receptor-1 (TfR1) after fungal recognition, independent of the iron regulatory protein-labile iron pool (IRP-LIP) system, explains this finding. We anticipate that this model will contribute to a quantitative understanding of iron regulation as a fundamental host defense mechanism during aspergillosis.

BACKGROUND: Computer vision has promise in image-based cutaneous melanoma diagnosis but clinical utility is uncertain. OBJECTIVE: To determine if computer algorithms from an international melanoma detection challenge can improve dermatologist melanoma diagnostic accuracy. METHODS: Cross-sectional study using 150 dermoscopy images (50 melanomas, 50 nevi, 50 seborrheic keratoses) from the test dataset of a melanoma detection challenge, along with algorithm results from twenty-three teams. Eight dermatologists and nine dermatology residents classified dermoscopic lesion images in an online reader study and provided their confidence level. RESULTS: The top-ranked computer algorithm had a ROC area of 0.87, which was higher than the dermatologists (0.74) and the residents (0.66) (p

Significance The digital twin paradigm holds great promise for medicine, even though many technical and scientific challenges remain to be overcome, most importantly the efficient integration of many heterogeneous component models. This is an unsolved problem even in industry. It has long been understood that such models need to be built in a modular fashion, connecting together component models of individual biological processes. In conventional implementations, however, the dependency structure of the modules reflects the dependencies among these processes, making it all but impossible to modify or expand the digital twin without breaking it. This paper presents a fundamental reorganization of the internal digital twin architecture into a hub-and-spokes design that completely circumvents this dependency problem., This paper presents a modular software design for the construction of computational modeling technology that will help implement precision medicine. In analogy to a common industrial strategy used for preventive maintenance of engineered products, medical digital twins are computational models of disease processes calibrated to individual patients using multiple heterogeneous data streams. They have the potential to help improve diagnosis, prognosis, and personalized treatment for a wide range of medical conditions. Their large-scale development relies on both mechanistic and data-driven techniques and requires the integration and ongoing update of multiple component models developed across many different laboratories. Distributed model building and integration requires an open-source modular software platform for the integration and simulation of models that is scalable and supports a decentralized, community-based model building process. This paper presents such a platform, including a case study in an animal model of a respiratory fungal infection.

BACKGROUND: Whether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions. METHODS: For this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10 015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen’s disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms. FINDINGS: Between Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p

This paper presents a robust segmentation method based on multi-scale classification to identify the lesion boundary in dermoscopic images. Our proposed method leverages a collection of classifiers which are trained at various resolutions to categorize each pixel as "lesion" or "surrounding skin". In detection phase, trained classifiers are applied on new images. The classifier outputs are fused at pixel level to build probability maps which represent lesion saliency maps. In the next step, Otsu thresholding is applied to convert the saliency maps to binary masks, which determine the border of the lesions. We compared our proposed method with existing lesion segmentation methods proposed in the literature using two dermoscopy data sets (International Skin Imaging Collaboration and Pedro Hispano Hospital) which demonstrates the superiority of our method with Dice Coefficient of 0.91 and accuracy of 94%.