Search

Key points Computer-aided diagnostics has potential to predict the nature of ovarian tumors. Literature shows heterogeneous sensitivity and specificity of machine learning on ultrasound images, CT-scan images and MRI-scan images. More prospective studies on other computer-aided techniques and imaging modalities should be performed with an external validation set.

Abstract Artificial intelligence (AI) is entering into daily life and has the potential to play a significant role in healthcare. Aim was to investigate the perspectives (knowledge, experience, and opinion) on AI in healthcare among patients with gastrointestinal (GI) disorders, gastroenterologists, and GI-fellows. In this prospective questionnaire study 377 GI-patients, 35 gastroenterologists, and 45 GI-fellows participated. Of GI-patients, 62.5% reported to be familiar with AI and 25.0% of GI-physicians had work-related experience with AI. GI-patients preferred their physicians to use AI (mean 3.9) and GI-physicians were willing to use AI (mean 4.4, on 5-point Likert-scale). More GI-physicians believed in an increase in quality of care (81.3%) than GI-patients (64.9%, χ2(2) = 8.2, p = 0.017). GI-fellows expected AI implementation within 6.0 years, gastroenterologists within 4.2 years (t(76) = − 2.6, p = 0.011), and GI-patients within 6.1 years (t(193) = − 2.0, p = 0.047). GI-patients and GI-physicians agreed on the most important advantages of AI in healthcare: improving quality of care, time saving, and faster diagnostics and shorter waiting times. The most important disadvantage for GI-patients was the potential loss of personal contact, for GI-physicians this was insufficiently developed IT infrastructures. GI-patients and GI-physicians hold positive perspectives towards AI in healthcare. Patients were significantly more reserved compared to GI-fellows and GI-fellows were more reserved compared to gastroenterologists.

The preoperative prediction of resectability pancreatic ductal adenocarcinoma (PDAC) is challenging. This retrospective single-center study examined tumor and vessel radiomics to predict the resectability of PDAC in chemo-naïve patients. The tumor and adjacent arteries and veins were segmented in the portal-venous phase of contrast-enhanced CT scans, and radiomic features were extracted. Features were selected via stability and collinearity testing, and least absolute shrinkage and selection operator application (LASSO). Three models, using tumor features, vessel features, and a combination of both, were trained with the training set (N = 86) to predict resectability. The results were validated with the test set (N = 15) and compared to the multidisciplinary team’s (MDT) performance. The vessel-features-only model performed best, with an AUC of 0.92 and sensitivity and specificity of 97% and 73%, respectively. Test set validation showed a sensitivity and specificity of 100% and 88%, respectively. The combined model was as good as the vessel model (AUC = 0.91), whereas the tumor model showed poor performance (AUC = 0.76). The MDT’s prediction reached a sensitivity and specificity of 97% and 84% for the training set and 88% and 100% for the test set, respectively. Our clinician-independent vessel-based radiomics model can aid in predicting resectability and shows performance comparable to that of the MDT. With these encouraging results, improved, automated, and generalizable models can be developed that reduce workload and can be applied in non-expert hospitals.

Encoder-decoder networks have become the standard solution for a variety of segmentation tasks. Many of these approaches use a symmetrical design where both the encoder as well as the decoder are approximately of the same computational complexity. However, symmetrical properties of encoder-decoder networks are not necessarily optimal. This work proposes an elegant and generic method to reduce the decoder complexity in encoder-decoder networks by scaling the number of feature channels in the decoder. The popular network U-Net is used as an example for how to adapt existing models with symmetrical properties. The effect of the decoder size is investigated on three data sets with varying complexity, namely, the ISIC, Cityscapes, and SUN RDB-D data sets. We show that a reduction in decoder channels shows no statistically differing results while at the same time providing a decoder requiring up to 99% fewer FLOPs, (± 90% fewer FLOPs is attainable for all investigated problems). In addition, results show that the number of parameters in the decoder of two models which already have smaller decoders can be further optimised depending on the problem. The proposed solution is a simple method and can easily be implemented in other encoder-decoder models. Empirical results also show that a reduction in parameters may even lead to improved performance, which is likely due to fewer parameters, reducing overfitting effects.

Background and study aims Colonoscopy is considered the gold standard for decreasing colorectal cancer incidence and mortality. Optical diagnosis of colorectal polyps (CRPs) is an ongoing challenge in clinical colonoscopy and its accuracy among endoscopists varies widely. Computer-aided diagnosis (CAD) for CRP characterization may help to improve this accuracy. In this study, we investigated the diagnostic accuracy of a novel algorithm for polyp malignancy classification by exploiting the complementary information revealed by three specific modalities. Methods We developed a CAD algorithm for CRP characterization based on high-definition, non-magnified white light (HDWL), Blue light imaging (BLI) and linked color imaging (LCI) still images from routine exams. All CRPs were collected prospectively and classified into benign or premalignant using histopathology as gold standard. Images and data were used to train the CAD algorithm using triplet network architecture. Our training dataset was validated using a threefold cross validation. Results In total 609 colonoscopy images of 203 CRPs of 154 consecutive patients were collected. A total of 174 CRPs were found to be premalignant and 29 were benign. Combining the triplet network features with all three image enhancement modalities resulted in an accuracy of 90.6 %, 89.7 % sensitivity, 96.6 % specificity, a positive predictive value of 99.4 %, and a negative predictive value of 60.9 % for CRP malignancy classification. The classification time for our CAD algorithm was approximately 90 ms per image. Conclusions Our novel approach and algorithm for CRP classification differentiates accurately between benign and premalignant polyps in non-magnified endoscopic images. This is the first algorithm combining three optical modalities (HDWL/BLI/LCI) exploiting the triplet network approach.

Biomedical image analysis challenges have increased in the last ten years, but common practices have not been established yet. Here the authors analyze 150 recent challenges and demonstrate that outcome varies based on the metrics used and that limited information reporting hampers reproducibility.

The primary treatment for malignant brain tumors is surgical resection. While gross total resection improves the prognosis, a supratotal resection may result in neurological deficits. On the other hand, accurate intraoperative identification of the tumor boundaries may be very difficult, resulting in subtotal resections. Histological examination of biopsies can be used repeatedly to help achieve gross total resection but this is not practically feasible due to the turn-around time of the tissue analysis. Therefore, intraoperative techniques to recognize tissue types are investigated to expedite the clinical workflow for tumor resection and improve outcome by aiding in the identification and removal of the malignant lesion. Hyperspectral imaging (HSI) is an optical imaging technique with the power of extracting additional information from the imaged tissue. Because HSI images cannot be visually assessed by human observers, we instead exploit artificial intelligence techniques and leverage a Convolutional Neural Network (CNN) to investigate the potential of HSI in twelve in vivo specimens. The proposed framework consists of a 3D–2D hybrid CNN-based approach to create a joint extraction of spectral and spatial information from hyperspectral images. A comparison study was conducted exploiting a 2D CNN, a 1D DNN and two conventional classification methods (SVM, and the SVM classifier combined with the 3D–2D hybrid CNN) to validate the proposed network. An overall accuracy of 80% was found when tumor, healthy tissue and blood vessels were classified, clearly outperforming the state-of-the-art approaches. These results can serve as a basis for brain tumor classification using HSI, and may open future avenues for image-guided neurosurgical applications.

Colorectal polyps are critical indicators of colorectal cancer (CRC). Blue Laser Imaging and Linked Color Imaging are two modalities that allow improved visualization of the colon. In conjunction with the Blue Laser Imaging (BLI) Adenoma Serrated International Classification (BASIC) classification, endoscopists are capable of distinguishing benign and pre-malignant polyps. Despite these advancements, this classification still prevails a high misclassification rate for pre-malignant colorectal polyps. This work proposes a computer aided diagnosis (CADx) system that exploits the additional information contained in two novel imaging modalities, enabling more informative decision-making during colonoscopy. We train and benchmark six commonly used CNN architectures and compare the results with 19 endoscopists that employed the standard clinical classification model (BASIC). The proposed CADx system for classifying colorectal polyps achieves an area under the curve (AUC) of 0.97. Furthermore, we incorporate visual explanatory information together with a probability score, jointly computed from White Light, Blue Laser Imaging, and Linked Color Imaging. Our CADx system for automatic polyp malignancy classification facilitates future advances towards patient safety and may reduce time-consuming and costly histology assessment.

Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg (n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg (n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video.

In spinal surgery, surgical navigation is an essential tool for safe intervention, including the placement of pedicle screws without injury to nerves and blood vessels. Commercially available systems typically rely on the tracking of a dynamic reference frame attached to the spine of the patient. However, the reference frame can be dislodged or obscured during the surgical procedure, resulting in loss of navigation. Hyperspectral imaging (HSI) captures a large number of spectral information bands across the electromagnetic spectrum, providing image information unseen by the human eye. We aim to exploit HSI to detect skin features in a novel methodology to track patient position in navigated spinal surgery. In our approach, we adopt two local feature detection methods, namely a conventional handcrafted local feature and a deep learning-based feature detection method, which are compared to estimate the feature displacement between different frames due to motion. To demonstrate the ability of the system in tracking skin features, we acquire hyperspectral images of the skin of 17 healthy volunteers. Deep-learned skin features are detected and localized with an average error of only 0.25 mm, outperforming the handcrafted local features with respect to the ground truth based on the use of optical markers.

Surgical navigation systems are increasingly used for complex spine procedures to avoid neurovascular injuries and minimize the risk for reoperations. Accurate patient tracking is one of the prerequisites for optimal motion compensation and navigation. Most current optical tracking systems use dynamic reference frames (DRFs) attached to the spine, for patient movement tracking. However, the spine itself is subject to intrinsic movements which can impact the accuracy of the navigation system. In this study, we aimed to detect the actual patient spine features in different image views captured by optical cameras, in an augmented reality surgical navigation (ARSN) system. Using optical images from open spinal surgery cases, acquired by two gray-scale cameras, spinal landmarks were identified and matched in different camera views. A computer vision framework was created for preprocessing of the spine images, detecting and matching local invariant image regions. We compared four feature detection algorithms, Speeded Up Robust Feature (SURF), Maximal Stable Extremal Region (MSER), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB) to elucidate the best approach. The framework was validated in 23 patients and the 3D triangulation error of the matched features was < 0.5 mm. Thus, the findings indicate that spine feature detection can be used for accurate tracking in navigated surgery.

Endoscopic diagnosis of early neoplasia in Barrett’s Esophagus is generally a two-step process of primary detection in overview, followed by detailed inspection of any visible abnormalities using Narrow Band Imaging (NBI). However, endoscopists struggle with evaluating NBI-zoom imagery of subtle abnormalities. In this work, we propose the first results of a deep learning system for the characterization of NBI-zoom imagery of Barrett’s Esophagus with an accuracy, sensitivity, and specificity of 83.6%, 83.1%, and 84.0%, respectively. We also show that endoscopy-driven pretraining outperforms two models, one without pretraining as well as a model with ImageNet initialization. The final model outperforms absence of pretraining by approximately 10% and the performance is 2% higher in terms of accuracy compared to ImageNet pretraining. Furthermore, the practical deployment of our model is not hampered by ImageNet licensing, thereby paving the way for clinical application.

In the original version of this Article the values in the rightmost column of Table 1 were inadvertently shifted relative to the other columns. This has now been corrected in the PDF and HTML versions of the Article.

Barrett’s esopaghagus (BE) is a known precursor of esophageal adenocarcinoma (EAC). Patients with BE undergo regular surveillance to early detect stages of EAC. Volumetric laser endomicroscopy (VLE) is a novel technology incorporating a second-generation form of optical coherence tomography and is capable of imaging the inner tissue layers of the esophagus over a 6 cm length scan. However, interpretation of full VLE scans is still a challenge for human observers. In this work, we train an ensemble of deep convolutional neural networks to detect neoplasia in 45 BE patients, using a dataset of images acquired with VLE in a multi-center study. We achieve an area under the receiver operating characteristic curve (AUC) of 0.96 on the unseen test dataset and we compare our results with previous work done with VLE analysis, where only AUC of 0.90 was achieved via cross-validation on 18 BE patients. Our method for detecting neoplasia in BE patients facilitates future advances on patient treatment and provides clinicians with new assisting solutions to process and better understand VLE data.