Search

The increasing availability of biomedical data creates valuable resources for developing new deep learning algorithms to support experts, especially in domains where collecting large volumes of annotated data is not trivial. Biomedical data include several modalities containing complementary information, such as medical images and reports: images are often large and encode low-level information, while reports include a summarized high-level description of the findings identified within data and often only concerning a small part of the image. However, only a few methods allow to effectively link the visual content of images with the textual content of reports, preventing medical specialists from properly benefitting from the recent opportunities offered by deep learning models. This paper introduces a multimodal architecture creating a robust biomedical data representation encoding fine-grained text representations within image embeddings. The architecture aims to tackle data scarcity (combining supervised and self-supervised learning) and to create multimodal biomedical ontologies. The architecture is trained on over 6,000 colon whole slide Images (WSI), paired with the corresponding report, collected from two digital pathology workflows. The evaluation of the multimodal architecture involves three tasks: WSI classification (on data from pathology workflow and from public repositories), multimodal data retrieval, and linking between textual and visual concepts. Noticeably, the latter two tasks are available by architectural design without further training, showing that the multimodal architecture that can be adopted as a backbone to solve peculiar tasks. The multimodal data representation outperforms the unimodal one on the classification of colon WSIs and allows to halve the data needed to reach accurate performance, reducing the computational power required and thus the carbon footprint. The combination of images and reports exploiting self-supervised algorithms allows to mine databases without needing new annotations provided by experts, extracting new information. In particular, the multimodal visual ontology, linking semantic concepts to images, may pave the way to advancements in medicine and biomedical analysis domains, not limited to histopathology., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Author(s). Published by Elsevier B.V. All rights reserved.)

In the fully digital Caltagirone pathology laboratory, a reverse shift from a digital to a manual workflow occurred due to a server outage in September 2023. Here, insights gained from this unplanned transition are explored. Surveying the affected pathologists and technicians revealed unanimous preferences for the time-saving and error-reducing capabilities of the digital methodology. Conversely, the return to manual methods highlighted increased dissatisfaction and reduced efficiency, emphasising the superiority of digital workflows. This case study underscores that transition challenges are not inherent to digital workflows but to transitioning itself, advocating for the adoption of digital technologies in all pathology practices., Competing Interests: Competing interests: AC, FM and FFraggetta report ad hoc advisory board membership with Roche Diagnostics Italia unrelated to the current work. FFraggetta is one of the inventors of 'Sample imaging and imagery archiving for imagery comparison Merlo, P.T. et al. US patent 16/688/613 2020'. The remaining authors have no conflicts of interest to declare., (© Author(s) (or their employer(s)) 2024. No commercial re-use. See rights and permissions. Published by BMJ.)

Molecular biomarker testing is increasingly becoming standard of care for advanced non-small cell lung cancer (NSCLC). Tissue and liquid biopsy-based next-generation sequencing (NGS) is now highly recommended and has become an integral part of the routine management of advanced NSCLC patients. This highly sensitive approach can simultaneously and efficiently detect multiple biomarkers even in scant samples. However full optimization of NGS in clinical practice requires accurate reporting and interpretation of NGS findings. Indeed, as the number of NSCLC biomarkers continues to grow, clinical reporting of NGS data is becoming increasingly complex. In this scenario, achieving standardization, simplification, and improved readability of NGS reports is key to ensuring timely and appropriate treatment decisions. In an effort to address the complexity and lengthy reporting of NGS mutation results, an Italian group of 14 healthcare professionals involved in NSCLC management convened in 2023 to address the content, structure, and ease-of-use of NGS reporting practices and proposed a standard report template for clinical use This article presents the key discussion points addressed by the Italian working group and describes the essential elements of the report template., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Umberto Malapelle has received personal fees (as consultant and/or speaker bureau) from Boehringer Ingelheim, Roche, MSD, Amgen, Thermo Fisher Scientific, Eli Lilly, Diaceutics, GSK, Merck and AstraZeneca, Janssen, Diatech, Novartis and Hedera unrelated to the current work. Fabio Pagni received personal fees (as consultant and/or speaker bureau) from, MSD, GSK, Merck and AstraZeneca, unrelated to the current work. Elena Guerini Rocco has relevant relationship (advisory fees, honoraria, travel accommodation and expenses, grants and non-financial support) with AstraZeneca, Exact Sciences, GlaxoSmithKline (GSK), Novartis, Roche, Thermo Fisher Scientific unrelated to the current work. Giulia Pasello has received personal fees (as consultant and/or speaker bureau) from Amgen; Astrazeneca; BMS; Lilly; MSD; Roche; Jansenn Research support: Astrazeneca; Roche unrelated to the current work. Giuseppe Perrone has received personal fees (as consultant and/or speaker bureau) from AstraZeneca, Amgen, Boehringer Ingelheim, Daiichi-Sankyo, Diatech Pharmacogenomics, Eli Lilly, Incyte, GSK, Janssen, Merck Serono, MSD, Novartis, Roche unrelated to the current work. Francesco Pepe has received personal fees (as consultant and/or speaker bureau) from Menarini Stemline unrelated to the current work. Giancarlo Pruneri has received personal fees (as consultant and/or speaker bureau) from Lilly, Roche Foundation One, Bayer, Novartis unrelated to the current work. Giancarlo Troncone reports personal fees (as speaker bureau or advisor) from Roche, MSD, Pfizer and Bayer, unrelated to the current work. Silvia Novello reports personal fees (as speaker bureau or advisor) from Eli Lilly, MSD, Roche, BMS, Takeda, Pfizer, Astra Zeneca and Boehringer Ingelheim, unrelated to the current work. No relevant conflict of interest declared by Alessandro Delle Donne, Filippo Fraggetta, Simona Vatrano, Sandro Pignata, Carmine Pinto, Antonio Russo, Hector J Soto Parra, Stefania Vallone, Antonio Marchetti related to the current work. No relevant conflict of interest declared by Alessandro Delle Donne, Filippo Fraggetta, Simona Vatrano, Sandro Pignata, Carmine Pinto, Antonio Russo, Hector J Soto Parra, Stefania Vallone, Antonio Marchetti related to the current work., (Copyright © 2023 The Authors. Published by Elsevier B.V. All rights reserved.)

Introduction: Cervical cancer is the fourth most common cancer in women, and its prevention is based on vaccination and screening. Screening consists of molecular human papillomavirus (HPV) testing and cytologic analysis of cervical smears, which require expensive equipment and the interaction of numerous professionals such as biologists, cytologists, laboratory technicians, and pathologists., Materials and Methods: We centralize the cervical samples from more than 51 clinics in 1 main laboratory, where automated HPV testing is performed. HPV-positive cases are collected and used to prepare a liquid-based cytology slide, which is stained and immediately scanned. The resulting whole-slide images (WSIs) are immediately available in a remote laboratory where they are examined by experienced cytologists using virtual microscopy. This setup was validated by making each of the 3 readers independently diagnose 506 specimens in random order, using both conventional light microscopy (CLM) and WSIs, with a minimum wash-out period of 3 weeks and with a final discussion for all cases., Results: Intraobserver agreement among CLM and WSI ranged from 0.71 to 0.79, and interobserver agreement for the 3 readers compared with the consensus diagnosis was similar for the 2 modes of assessment. Readers subjectively felt confident in their WSI diagnosis for inadequate and negative cases, but less so in other cases. The perceived difficulty was slightly higher in WSI readings., Conclusions: Interobserver agreement in cervicovaginal cytology is moderate and does not vary if the slides are examined conventionally or digitally. Despite higher reported subjective difficulty and lower confidence in the WSI diagnosis, we did not observe a deterioration in diagnostic performance using WSI compared with CLM., (Copyright © 2023 American Society of Cytopathology. Published by Elsevier Inc. All rights reserved.)

In colorectal cancer (CRC), artificial intelligence (AI) can alleviate the laborious task of characterization and reporting on resected biopsies, including polyps, the numbers of which are increasing as a result of CRC population screening programs ongoing in many countries all around the globe. Here, we present an approach to address two major challenges in the automated assessment of CRC histopathology whole-slide images. We present an AI-based method to segment multiple ([Formula: see text]) tissue compartments in the H &E-stained whole-slide image, which provides a different, more perceptible picture of tissue morphology and composition. We test and compare a panel of state-of-the-art loss functions available for segmentation models, and provide indications about their use in histopathology image segmentation, based on the analysis of (a) a multi-centric cohort of CRC cases from five medical centers in the Netherlands and Germany, and (b) two publicly available datasets on segmentation in CRC. We used the best performing AI model as the basis for a computer-aided diagnosis system that classifies colon biopsies into four main categories that are relevant pathologically. We report the performance of this system on an independent cohort of more than 1000 patients. The results show that with a good segmentation network as a base, a tool can be developed which can support pathologists in the risk stratification of colorectal cancer patients, among other possible uses. We have made the segmentation model available for research use on https://grand-challenge.org/algorithms/colon-tissue-segmentation/ ., (© 2023. The Author(s).)

Computational pathology targets the automatic analysis of Whole Slide Images (WSI). WSIs are high-resolution digitized histopathology images, stained with chemical reagents to highlight specific tissue structures and scanned via whole slide scanners. The application of different parameters during WSI acquisition may lead to stain color heterogeneity, especially considering samples collected from several medical centers. Dealing with stain color heterogeneity often limits the robustness of methods developed to analyze WSIs, in particular Convolutional Neural Networks (CNN), the state-of-the-art algorithm for most computational pathology tasks. Stain color heterogeneity is still an unsolved problem, although several methods have been developed to alleviate it, such as Hue-Saturation-Contrast (HSC) color augmentation and stain augmentation methods. The goal of this paper is to present Data-Driven Color Augmentation (DDCA), a method to improve the efficiency of color augmentation methods by increasing the reliability of the samples used for training computational pathology models. During CNN training, a database including over 2 million H&E color variations collected from private and public datasets is used as a reference to discard augmented data with color distributions that do not correspond to realistic data. DDCA is applied to HSC color augmentation, stain augmentation and H&E-adversarial networks in colon and prostate cancer classification tasks. DDCA is then compared with 11 state-of-the-art baseline methods to handle color heterogeneity, showing that it can substantially improve classification performance on unseen data including heterogeneous color variations., Competing Interests: The authors declare that there are no competing interests., (© 2022 The Authors.)

Exa-scale volumes of medical data have been produced for decades. In most cases, the diagnosis is reported in free text, encoding medical knowledge that is still largely unexploited. In order to allow decoding medical knowledge included in reports, we propose an unsupervised knowledge extraction system combining a rule-based expert system with pre-trained Machine Learning (ML) models, namely the Semantic Knowledge Extractor Tool (SKET). Combining rule-based techniques and pre-trained ML models provides high accuracy results for knowledge extraction. This work demonstrates the viability of unsupervised Natural Language Processing (NLP) techniques to extract critical information from cancer reports, opening opportunities such as data mining for knowledge extraction purposes, precision medicine applications, structured report creation, and multimodal learning. SKET is a practical and unsupervised approach to extracting knowledge from pathology reports, which opens up unprecedented opportunities to exploit textual and multimodal medical information in clinical practice. We also propose SKET eXplained (SKET X), a web-based system providing visual explanations about the algorithmic decisions taken by SKET. SKET X is designed/developed to support pathologists and domain experts in understanding SKET predictions, possibly driving further improvements to the system., Competing Interests: The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:Filippo Fraggetta is an author of this work and a member of the editorial board., (© 2022 The Authors.)

The digitalization of clinical workflows and the increasing performance of deep learning algorithms are paving the way towards new methods for tackling cancer diagnosis. However, the availability of medical specialists to annotate digitized images and free-text diagnostic reports does not scale with the need for large datasets required to train robust computer-aided diagnosis methods that can target the high variability of clinical cases and data produced. This work proposes and evaluates an approach to eliminate the need for manual annotations to train computer-aided diagnosis tools in digital pathology. The approach includes two components, to automatically extract semantically meaningful concepts from diagnostic reports and use them as weak labels to train convolutional neural networks (CNNs) for histopathology diagnosis. The approach is trained (through 10-fold cross-validation) on 3'769 clinical images and reports, provided by two hospitals and tested on over 11'000 images from private and publicly available datasets. The CNN, trained with automatically generated labels, is compared with the same architecture trained with manual labels. Results show that combining text analysis and end-to-end deep neural networks allows building computer-aided diagnosis tools that reach solid performance (micro-accuracy = 0.908 at image-level) based only on existing clinical data without the need for manual annotations., (© 2022. The Author(s).)