Search

Backed by more powerful computational resources and optimized training routines, deep learning models have proven unprecedented performance and several benefits to extract information from chest X-ray data. This is one of the most common imaging exams, whose increasing demand is reflected in the aggravated radiologists’ workload. Consequently, healthcare would benefit from computer-aided diagnosis systems to prioritize certain exams and further identify possible pathologies. Pioneering work in chest X-ray analysis has focused on the identification of specific diseases, but to the best of the authors' knowledge no paper has specifically reviewed relevant work on abnormality detection and multi-label thoracic pathology classification. This paper focuses on those issues, selecting the leading chest X-ray based deep learning strategies for comparison. In addition, the paper discloses the current annotated public chest X-ray databases, covering the common thorax diseases.

Early diagnosis of lung cancer via computed tomography can significantly reduce the morbidity and mortality rates associated with the pathology. However, searching lung nodules is a high complexity task, which affects the success of screening programs. Whilst computer-aided detection systems can be used as second observers, they may bias radiologists and introduce significant time overheads. With this in mind, this study assesses the potential of using gaze information for integrating automatic detection systems in the clinical practice. For that purpose, 4 radiologists were asked to annotate 20 scans from a public dataset while being monitored by an eye tracker device, and an automatic lung nodule detection system was developed. Our results show that radiologists follow a similar search routine and tend to have lower fixation periods in regions where finding errors occur. The overall detection sensitivity of the specialists was $\mathbf {0.67\pm 0.07}$ , whereas the system achieved 0.69. Combining the annotations of one radiologist with the automatic system significantly improves the detection performance to similar levels of two annotators. Filtering automatic detection candidates only for low fixation regions still significantly improves the detection sensitivity without increasing the number of false-positives.

Proliferative diabetic retinopathy (PDR) is an advanced diabetic retinopathy stage, characterized by neovascularization, which leads to ocular complications and severe vision loss. However, the available DR-labeled retinal image datasets have a small representation of images of the severest DR grades, and thus there is lack of PDR cases for training DR grading models. Additionally, the criteria for labelling these images in the publicly available datasets is not always clear, with some images which do not show typical PDR lesions being labeled as PDR due to the presence of photo-coagulation treatment and laser marks. This problem, together with the datasets' high class imbalance, leads to a limited variability of the samples, which the typical data augmentation and class balancing cannot fully mitigate. We propose a heuristic-based data augmentation scheme based on the synthesis of neovessel (NV)-like structures that compensates for the lack of PDR cases in DR-labeled datasets. The proposed neovessel generation algorithm relies on the general knowledge of common location and shape of these structures. NVs are generated and introduced in pre-existent retinal images which can then be used for enlarging deep neural networks' training sets. The data augmentation scheme was tested on multiple datasets, and allows to improve the model's capacity to detect NVs.

The worldwide pandemic caused by the new coronavirus (COVID-19) has encouraged the development of multiple computer-aided diagnosis systems to automate daily clinical tasks, such as abnormality detection and classification. Among these tasks, the segmentation of COVID lesions is of high interest to the scientific community, enabling further lesion characterization. Automating the segmentation process can be a useful strategy to provide a fast and accurate second opinion to the physicians, and thus increase the reliability of the diagnosis and disease stratification. The current work explores a CNN-based approach to segment multiple COVID lesions. It includes the implementation of a U-Net structure with a ResNet34 encoder able to deal with the highly imbalanced nature of the problem, as well as the great variability of the COVID lesions, namely in terms of size, shape, and quantity. This approach yields a Dice score of 64.1%, when evaluated on the publicly available COVID-19-20 Lung CT Lesion Segmentation GrandChallenge data set.

We propose iW-Net, a deep learning model that allows for both automatic and interactive segmentation of lung nodules in computed tomography images. iW-Net is composed of two blocks: the first one provides an automatic segmentation and the second one allows to correct it by analyzing 2 points introduced by the user in the nodule's boundary. For this purpose, a physics inspired weight map that takes the user input into account is proposed, which is used both as a feature map and in the system's loss function. Our approach is extensively evaluated on the public LIDC-IDRI dataset, where we achieve a state-of-the-art performance of 0.55 intersection over union vs the 0.59 inter-observer agreement. Also, we show that iW-Net allows to correct the segmentation of small nodules, essential for proper patient referral decision, as well as improve the segmentation of the challenging non-solid nodules and thus may be an important tool for increasing the early diagnosis of lung cancer., Comment: Pre-print submitted to IEEE Transactions on Biomedical Engineering

Diabetic retinopathy (DR) is a diabetes complication, which in extreme situations may lead to blindness. Since the first stages are often asymptomatic, regular eye examinations are required for an early diagnosis. As microaneurysms (MAs) are one of the first signs of DR, several automated methods have been proposed for their detection in order to reduce the ophthalmologists' workload. Although local convergence filters (LCFs) have already been applied for feature extraction, their potential as MA enhancement operators was not explored yet. In this work, we propose a sliding band filter for MA enhancement aiming at obtaining a set of initial MA candidates. Then, a combination of the filter responses with color, contrast and shape information is used by an ensemble of classifiers for final candidate classification. Finally, for each eye fundus image, a score is computed from the confidence values assigned to the MAs detected in the image. The performance of the proposed methodology was evaluated in four datasets. At the lesion level, sensitivities of 64% and 81% were achieved for an average of 8 false positives per image (FPIs) in e-ophtha MA and SCREEN-DR, respectively. In the last dataset, an AUC of 0.83 was also obtained for DR detection.

Currently, lung cancer is the most lethal in the world. In order to make screening and follow-up a little more systematic, guidelines have been proposed. Therefore, this study aimed to create a diagnostic support approach by providing a patient label based on the LUNG-RADS™ guidelines. The only input required by the system is the nodule centroid to take the region of interest for the input of the classification system. With this in mind, two deep learning networks were evaluated: a Wide Residual Network and a DenseNet. Taking into account the annotation uncertainty we proposed to use sample weights that are introduced in the loss function, allowing nodules with a high agreement in the annotation process to take a greater impact on the training error than its counterpart. The best result was achieved with the Wide Residual Network with sample weights achieving a nodule-wise LUNG-RADS™ labelling accuracy of $0.735\pm 0.003$ .

Active Learning methods create an optimized labeled training set from unlabeled data. We introduce a novel Online Active Deep Learning method for Medical Image Analysis. We extend our MedAL active learning framework to present new results in this paper. Our novel sampling method queries the unlabeled examples that maximize the average distance to all training set examples. Our online method enhances performance of its underlying baseline deep network. These novelties contribute significant performance improvements, including improving the model's underlying deep network accuracy by 6.30%, using only 25% of the labeled dataset to achieve baseline accuracy, reducing backpropagated images during training by as much as 67%, and demonstrating robustness to class imbalance in binary and multi-class tasks., Comment: Code: https://github.com/adgaudio/o-medal ; Accepted and published by Wiley Journal of Pattern Recognition and Knowledge Discovery ; Journal URL: https://doi.org/10.1002/widm.1353

Several systemic diseases affect the retinal blood vessels, and thus, their assessment allows an accurate clinical diagnosis. This assessment entails the estimation of the arteriolar-to-venular ratio (AVR), a predictive biomarker of cerebral atrophy and cardiovascular events in adults. In this context, different automatic and semiautomatic image-based approaches for artery/vein (A/V) classification and AVR estimation have been proposed in the literature, to the point of having become a hot research topic in the last decades. Most of these approaches use a wide variety of image properties, often redundant and/or irrelevant, requiring a training process that limits their generalization ability when applied to other datasets. This paper presents a new automatic method for A/V classification that just uses the local contrast between blood vessels and their surrounding background, computes a graph that represents the vascular structure, and applies a multilevel thresholding to obtain a preliminary classification. Next, a novel graph propagation approach was developed to obtain the final A/V classification and to compute the AVR. Our approach has been tested on two public datasets (INSPIRE and DRIVE), obtaining high classification accuracy rates, especially in the main vessels, and AVR ratios very similar to those provided by human experts. Therefore, our fully automatic method provides the reliable results without any training step, which makes it suitable for use with different retinal image datasets and as part of any clinical routine.

Diabetic macular edema is a leading cause of visual loss for patients with diabetes. While diagnosis can only be performed by optical coherence tomography, diabetic macular edema risk assessment is often performed in eye fundus images in screening scenarios through the detection of hard exudates. Such screening scenarios are often associated with large amounts of data, high costs and high burden on specialists, motivating then the development of methodologies for automatic diabetic macular edema risk prediction. Nevertheless, significant dataset domain bias, due to different acquisition equipment, protocols and/or different populations can have significantly detrimental impact on the performance of automatic methods when transitioning to a new dataset, center or scenario. As such, in this study, a method based on residual neural networks is proposed for the classification of diabetic macular edema risk. This method is then validated across multiple public datasets, simulating the deployment in a multi-center setting and thereby studying the method’s generalization capability and existing dataset domain bias. Furthermore, the method is tested on a private dataset which more closely represents a realistic screening scenario. An average area under the curve across all public datasets of 0.891 ± 0.013 was obtained with a ResNet50 architecture trained on a limited amount of images from a single public dataset (IDRiD). It is also shown that screening scenarios are significantly more challenging and that training across multiple datasets leads to an improvement of performance (area under the curve of 0.911 ± 0.009).

The optic disc (OD) and the fovea are relevant landmarks in fundus images. Their localization and segmentation can facilitate the detection of some retinal lesions and the assessment of their importance to the severity and progression of several eye disorders. Distinct methodologies have been developed for detecting these structures, mainly based on color and vascular information. The methodology herein described combines the entropy of the vessel directions with the image intensities for finding the OD center and uses a sliding band filter for segmenting the OD. The fovea center corresponds to the darkest point inside a region defined from the OD position and radius. Both the Messidor and the IDRiD datasets are used for evaluating the performance of the developed methods. In the first one, a success rate of 99.56% and 100.00% are achieved for OD and fovea localization. Regarding the OD segmentation, the mean Jaccard index and Dice’s coefficient obtained are 0.87 and 0.94, respectively. The proposed methods are also amongst the top-3 performing solutions submitted to the IDRiD online challenge.

We propose a pixel color amplification theory and family of enhancement methods to facilitate segmentation tasks on retinal images. Our novel re-interpretation of the image distortion model underlying dehazing theory shows how three existing priors commonly used by the dehazing community and a novel fourth prior are related. We utilize the theory to develop a family of enhancement methods for retinal images, including novel methods for whole image brightening and darkening. We show a novel derivation of the Unsharp Masking algorithm. We evaluate the enhancement methods as a pre-processing step to a challenging multi-task segmentation problem and show large increases in performance on all tasks, with Dice score increases over a no-enhancement baseline by as much as 0.491. We provide evidence that our enhancement preprocessing is useful for unbalanced and difficult data. We show that the enhancements can perform class balancing by composing them together.

A new approach for robust edge detection on B-mode ultrasound images of the carotid artery is proposed in this paper. The proposed method uses anisotropic Gaussian derivative filters along with non-maximum suppression over the overall artery wall orientation in local regions. The anisotropic filters allow using a wider integration scale along the edges while preserving the edge location precision. They also perform edge continuation, resulting in the connection of isolated edge points along linear segments, which is a valuable feature for the segmentation of the artery wall layers. However, this usually results in false edges being detected near convex contours and isolated points. The use of non-maximum suppression over pooled local orientations is proposed to solve this issue. Experimental results are provided to demonstrate that the proposed edge detector outperforms other common methods in the detection of the lumen-intima and media-adventia layer interfaces of the carotid vessel walls. Additionally, the resulting edges are more continuous and precisely located.

Diabetic retinopathy (DR) detection is a critical retinal image analysis task in the context of early blindness prevention. Unfortunately, in order to train a model to accurately detect DR based on the presence of different retinal lesions, typically a dataset with medical expert's annotations at the pixel level is needed. In this paper, a new methodology based on the multiple instance learning (MIL) framework is developed in order to overcome this necessity by leveraging the implicit information present on annotations made at the image level. Contrary to previous MIL-based DR detection systems, the main contribution of the proposed technique is the joint optimization of the instance encoding and the image classification stages. In this way, more useful mid-level representations of pathological images can be obtained. The explainability of the model decisions is further enhanced by means of a new loss function enforcing appropriate instance and mid-level representations. The proposed technique achieves comparable or better results than other recently proposed methods, with 90% area under the receiver operating characteristic curve (AUC) on Messidor, 93% AUC on DR1, and 96% AUC on DR2, while improving the interpretability of the produced decisions.

Lung cancer is the type of cancer with highest mortality worldwide. Low-dose computerized tomography is the main tool used for lung cancer screening in clinical practice, allowing the visualization of lung nodules and the assessment of their malignancy. However, this evaluation is a complex task and subject to inter-observer variability, which has fueled the need for computer-aided diagnosis systems for lung nodule malignancy classification. While promising results have been obtained with automatic methods, it is often not straightforward to determine which features a given model is basing its decisions on and this lack of explainability can be a significant stumbling block in guaranteeing the adoption of automatic systems in clinical scenarios. Though visual malignancy assessment has a subjective component, radiologists strongly base their decision on nodule features such as nodule spiculation and texture, and a malignancy classification model should thus follow the same rationale. As such, this study focuses on the characterization of lung nodules as a means for the classification of nodules in terms of malignancy. For this purpose, different model architectures for nodule characterization are proposed and compared, with the final goal of malignancy classification. It is shown that models that combine direct malignancy prediction with specific branches for nodule characterization have a better performance than the remaining models, achieving an Area Under the Curve of 0.783. The most relevant features for malignancy classification according to the model were lobulation, spiculation and texture, which is found to be in line with current clinical practice.

Ovarian cancer is one of the most commonly occurring cancer in women. Transvaginal ultrasound is used as a screening test to detect the presence of tumors but, for specific types of ovarian tumors, malignancy can only be asserted through surgery. An automatic method to perform the detection and malignancy assessment of these tumours is thus necessary to prevent unnecessary oophorectomies.This work explores the U-Net’s architecture and investigates the selection of different hyperparameters for the ovary and the ovarian follicles segmentation. The effect of applying different post-processing methods on beam-formed radio-frequency (BRF) data is also investigated.Results show that models trained only with BRF data have the worst performance. On the other hand, the combination of B-mode with BRF data performs better for ovary segmentation. As for the hyperparameter study, results show that the U-Net with 4 levels is the architecture with the worst performance. This shows that to achieve better performance in the segmentation of ovarian structures, it is important to select an architecture that takes into account the spatial context of the regions of interest. It is also possible to conclude that the method used to analyse BRF data should be designed to take advantage of the fine-resolution of BRF data.

Lung cancer is the deadliest type of cancer worldwide and late detection is one of the major factors for the low survival rate of patients. Low dose computed tomography has been suggested as a potential early screening tool but manual screening is costly, time-consuming and prone to interobserver variability. This has fueled the development of automatic methods for the detection, segmentation and characterisation of pulmonary nodules but its application to the clinical routine is challenging. In this study, a platform for the development, deployment and testing of pulmonary nodule computer-aided strategies is presented: LNDetector. LNDetector integrates image exploration and nodule annotation tools as well as advanced nodule detection, segmentation and classification methods and gaze characterisation. Different processing modules can easily be implemented or replaced to test their efficiency in clinical environments and the use of gaze analysis allows for the development of collaborative strategies. The potential use of this platform is shown through a combination of visual search, gaze characterisation and automatic nodule detection tools for an efficient and collaborative computer-aided strategy for pulmonary nodule screening.

Diabetic Retinopathy (DR) is one of the leading causes of preventable blindness in the developed world. With the increasing number of diabetic patients there is a growing need of an automated system for DR detection. We propose Eye WeS, a method that not only detects DR in eye fundus images but also pinpoints the regions of the image that contain lesions, while being trained with image labels only. We show that it is possible to convert any pre-trained convolutional neural network into a weakly-supervised model while increasing their performance and efficiency. EyeWeS improved the results of Inception V3 from 94.9% Area Under the Receiver Operating Curve (AUC) to 95.8% AUC while maintaining only approximately 5% of the Inception V3's number of parameters. The same model is able to achieve 97.1% AUC in a cross-dataset experiment.

Visual exploration of the larynx represents a relevant technique for the early diagnosis of laryngeal disorders. However, visualizing an endoscopy for finding abnormalities is a time-consuming process, and for this reason much research has been dedicated to the automatic analysis of endoscopic video data. In this work we address the particular task of discriminating among informative laryngoscopic frames and those that carry insufficient diagnostic information. In the latter case, the goal is also to determine the reason for this lack of information. To this end, we analyze the possibility of training three different state-of-the-art Convolutional Neural Networks, but initializing their weights from configurations that have been previously optimized for solving natural image classification problems. Our findings show that the simplest of these three architectures not only is the most accurate (outperforming previously proposed techniques), but also the fastest and most efficient, with the lowest inference time and minimal memory requirements, enabling real-time application and deployment in portable devices.

The automatic differentiation of retinal vessels into arteries and veins (A/V) is a highly relevant task within the field of retinal image analysis. However, due to limitations of retinal image acquisition devices, specialists can find it impossible to label certain vessels in eye fundus images. In this paper, we introduce a method that takes into account such uncertainty by design. For this, we formulate the A/V classification task as a four-class segmentation problem, and a Convolutional Neural Network is trained to classify pixels into background, A/V, or uncertain classes. The resulting technique can directly provide pixelwise uncertainty estimates. In addition, instead of depending on a previously available vessel segmentation, the method automatically segments the vessel tree. Experimental results show a performance comparable or superior to several recent A/V classification approaches. In addition, the proposed technique also attains state-of-the-art performance when evaluated for the task of vessel segmentation, generalizing to data that was not used during training, even with considerable differences in terms of appearance and resolution.

Lung volume segmentation is a key step in the design of Computer-Aided Diagnosis systems for automated lung pathology analysis. However, isolating the lung from CT volumes can be a challenging process due to considerable deformations and the potential presence of pathologies. Convolutional Neural Networks (CNN) are effective tools for modeling the spatial relationship between lung voxels. Unfortunately, they typically require large quantities of annotated data, and manually delineating the lung from volumetric CT scans can be a cumbersome process. We propose to train a 3D CNN to solve this task based on semi-automatically generated annotations. For this, we introduce an extension of the well-known V-Net architecture that can handle higher-dimensional input data. Even if the training set labels are noisy and contain errors, our experiments show that it is possible to learn to accurately segment the lung relying on them. Numerical comparisons on an external test set containing lung segmentations provided by a medical expert demonstrate that the proposed model generalizes well to new data, reaching an average 98.7% Dice coefficient. The proposed approach results in a superior performance with respect to the standard V-Net model, particularly on the lung boundary.

This study describes a novel dataset with retinal image quality annotation, defined by three different retinal experts, and presents an inter-observer analysis for quality assessment that can be used as gold-standard for future studies. A state-of-the-art algorithm for retinal image quality assessment is also analysed and compared against the specialists performance. Results show that, for 71% of the images present in the dataset, the three experts agree on the given image quality label. The results obtained for accuracy, specificity and sensitivity when comparing one expert against another were in the ranges [83.0 − 85.2]%, [72.7 − 92.9]% and [80.0 − 94.7]%, respectively. The evaluated automatic quality assessment method, despite not being trained on the novel dataset, presents a performance which is within inter-observer variability.

Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is challenging due to the scarcity of medical professionals able to screen a growing global diabetic population at risk for DR. Computer-aided disease diagnosis in retinal image analysis could provide a sustainable approach for such large-scale screening effort. The recent scientific advances in computing capacity and machine learning approaches provide an avenue for biomedical scientists to reach this goal. Aiming to advance the state-of-the-art in automatic DR diagnosis, a grand challenge on "Diabetic Retinopathy - Segmentation and Grading" was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2018). In this paper, we report the set-up and results of this challenge that is primarily based on Indian Diabetic Retinopathy Image Dataset (IDRiD). There were three principal sub-challenges: lesion segmentation, disease severity grading, and localization of retinal landmarks and segmentation. These multiple tasks in this challenge allow to test the generalizability of algorithms, and this is what makes it different from existing ones. It received a positive response from the scientific community with 148 submissions from 495 registrations effectively entered in this challenge. This paper outlines the challenge, its organization, the dataset used, evaluation methods and results of top-performing participating solutions. The top-performing approaches utilized a blend of clinical information, data augmentation, and an ensemble of models. These findings have the potential to enable new developments in retinal image analysis and image-based DR screening in particular.

Our goal in this chapter is to describe the recent application of deep learning and artificial intelligence (AI) techniques to retinal image analysis. Automatic retinal image analysis (ARIA) is a complex task that has significant applications for diagnostic purposes for a host of retinal, neurological, and vascular diseases. A number of approaches for the automatic analysis of the retinal images have been studied for the past two decades but the recent success of deep learning (DL) for a range of computer vision and image analysis tasks has now permeated medical imaging and ARIA. Since 2016, major improvements were reported using DL discriminative methods (deep convolutional neural networks or autoencoder convolutional networks), and generative methods, in combination with other image analysis methods, that have demonstrated the ability of algorithms to perform on par with ophthalmologists and retinal specialists, for tasks such as automated classification, diagnostics, and segmentation. We review these recent developments in this chapter.

Diabetic Retinopathy is the leading cause of blindness in the working-age population of the world. The main aim of this paper is to improve the accuracy of Diabetic Retinopathy detection by implementing a shadow removal and color correction step as a preprocessing stage from eye fundus images. For this, we rely on recent findings indicating that application of image dehazing on the inverted intensity domain amounts to illumination compensation. Inspired by this work, we propose a Shadow Removal Layer that allows us to learn the pre-processing function for a particular task. We show that learning the pre-processing function improves the performance of the network on the Diabetic Retinopathy detection task.

Lung cancer is one of the most common causes of death in the world. The early detection of lung nodules allows an appropriate follow-up, timely treatment and potentially can avoid greater damage in the patient health. The texture is one of the nodule characteristics that is correlated with the malignancy. We developed convolutional neural network architectures to classify automatically the texture of nodules into the non-solid, part-solid and solid classes. The different architectures were tested to determine if the context, the number of slices considered as input and the relation between slices influence on the texture classification performance. The architecture that obtained better performance took into account different scales, different rotations and the context of the nodule, obtaining an accuracy of 0.833 ± 0.041.

International audience; Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design efficient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparo-scopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected,-H. Conze et al. / Medical Image Analysis 52 (2019) 24-41 25 the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the differential analysis of these solutions are discussed. We expect that they will guide the design of efficient surgery monitoring tools in the near future.

The segmentation and characterization of the ovarian structures are important tasks in gynecological and reproductive medicine. Ultrasound imaging is typically used for the medical diagnosis within this field but the understanding of the images can be difficult due to their characteristics. Furthermore, the complexity of ultrasound data may lead to a heavy image processing, which makes the application of classical methods of computer vision difficult. This work presents the first supervised fully convolutional neural network (fCNN) for the automatic segmentation of ovarian structures in B-mode ultrasound images. Due to the small dataset available, only 57 images were used for training. In order to overcome this limitation, several regularization techniques were used and are discussed in this paper. The experiments show the ability of the fCNN to learn features to distinguish ovarian structures, achieving a Dice similarity coefficient (DSC) of 0.855 for the segmentation of the stroma and a DSC of 0.955 for the follicles. When compared with a semi-automatic commercial application for follicle segmentation, the proposed fCNN achieved an average improvement of 19%.

Ovarian cancer is one of the pathologies with the worst prognostic in adult women and it has a very difficult early diagnosis. Clinical evaluation of gynaecological ultrasound images is performed visually, and it is dependent on the experience of the medical doctor. Besides the dependency on the specialists, the malignancy of specific types of ovarian tumors cannot be asserted until their surgical removal. This work explores the use of ultrasound data for the segmentation of the ovary and the ovarian follicles, using two different convolutional neural networks, a fully connected residual network and a U-Net, with a binary and multi-class approach. Five different types of ultrasound data (from beam-formed radio-frequency to brightness mode) were used as input. The best performance was obtained using B-mode, for both ovary and follicles segmentation. No significant differences were found between the two convolutional neural networks. The use of the multi-class approach was beneficial as it provided the model information on the spatial relation between follicles and the ovary. This study demonstrates the suitability of combining convolutional neural networks with beam-formed radio-frequency data and with brightness mode data for segmentation of ovarian structures. Future steps involve the processing of pathological data and investigation of biomarkers of pathological ovaries.

This paper proposes a new methodology to automatically segment and measure the volume of pulmonary nodules in lung computed tomography (CT) scans. Estimating the malignancy likelihood of a pulmonary nodule based on lesion characteristics motivated the development of an unsupervised pulmonary nodule segmentation and volume measurement as a preliminary stage for pulmonary nodule characterization. The idea is to optimally cluster a set of feature vectors composed by intensity and shape-related features in a given feature data space extracted from a pre-detected nodule. For that purpose, a metaheuristic search based on evolutionary computation is used for clustering the corresponding feature vectors. The proposed method is simple, unsupervised and is able to segment different types of nodules in terms of location and texture without the need for any manual annotation. We validate the proposed segmentation and volume measurement on the Lung Image Database Consortium and Image Database Resource Initiative – LIDC-IDRI dataset. The first dataset is a group of 705 solid and sub-solid (assessed as part-solid and non-solid) nodules located in different regions of the lungs, and the second, more challenging, is a group of 59 sub-solid nodules. The average Dice scores of 82.35% and 71.05% for the two datasets show the good performance of the segmentation proposal. Comparisons with previous state-of-the-art techniques also show acceptable and comparable segmentation results. The volumes of the segmented nodules are measured via ellipsoid approximation. The correlation and statistical significance between the measured volumes of the segmented nodules and the ground-truth are obtained by Pearson correlation coefficient value, obtaining an R-value ≥ 92.16% with a significance level of 5%.

Deep learning models have been successfully used in medical image analysis problems but they require a large amount of labeled images to obtain good performance. However, such large labeled datasets are costly to acquire. Active learning techniques can be used to minimize the number of required training labels while maximizing the model's performance. In this work, we propose a novel sampling method that queries the unlabeled examples that maximize the average distance to all training set examples in a learned feature space. We then extend our sampling method to define a better initial training set, without the need for a trained model, by using Oriented FAST and Rotated BRIEF (ORB) feature descriptors. We validate MedAL on 3 medical image datasets and show that our method is robust to different dataset properties. MedAL is also efficient, achieving 80% accuracy on the task of Diabetic Retinopathy detection using only 425 labeled images, corresponding to a 32% reduction in the number of required labeled examples compared to the standard uncertainty sampling technique, and a 40% reduction compared to random sampling.

In the design of computer-aided diagnosis systems for lung cancer diagnosis, an appropriate and accurate segmentation of the pulmonary nodules in computerized tomography (CT) is one of the most relevant and difficult tasks. An accurate segmentation is crucial for the posterior measurement of nodule characteristics and for lung cancer diagnosis. This paper proposes different approaches that use Hessian-based strategies for lung nodule segmentation in chest CT scans. We propose a multiscale segmentation process that uses the central medialness adaptive principle, a Hessian-based strategy that was originally formulated for tubular extraction but it also provides good segmentation results in blob-like structures as is the case of lung nodules. We compared this proposal with a well established Hessian-based strategy that calculates the Shape Index (SI) and Curvedness (CV). We adapted the SI and CV approach for multiscale nodule segmentation. Moreover, we propose the combination of both strategies by combining the results, in order to take benefit of the advantages of both strategies. Different cases with pulmonary nodules from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) database were taken and used to analyze and validate the approaches. The chest CT images present a large variability in nodule characteristics and image conditions. Our proposals provide an accurate lung nodule segmentation, similar to radiologists performance. Our Hessian-based approaches were validated with 569 solid and mostly solid nodules demonstrating that these novel strategies have good results when compared with the radiologists segmentations, providing accurate pulmonary nodule volumes for posterior characterization and appropriate diagnosis.

Image dehazing tries to solve an undesired loss of visibility in outdoor images due to the presence of fog. Recently, machine-learning techniques have shown great dehazing ability. However, in order to be trained, they require training sets with pairs of foggy images and their clean counterparts, or a depth-map. In this paper, we propose to learn the appearance of fog from weakly-labeled data. Specifically, we only require a single label per-image stating if it contains fog or not. Based on the Multiple-Instance Learning framework, we propose a model that can learn from image-level labels to predict if an image contains haze reasoning at a local level. Fog detection performance of the proposed method compares favorably with two popular techniques, and the attention maps generated by the model demonstrate that it effectively learns to disregard sky regions as indicative of the presence of fog, a common pitfall of current image dehazing techniques.

The segmentation and characterization of the lung lobes are important tasks for Computer Aided Diagnosis (CAD) systems related to pulmonary disease. The detection of the fissures that divide the lung lobes is non-trivial when using classical methods that rely on anatomical information like the localization of the airways and vessels. This work presents a fully automatic and supervised approach to the problem of the segmentation of the five pulmonary lobes from a chest Computer Tomography (CT) scan using a Fully RegularizedV-Net (FRV- Net), a 3D Fully Convolutional Neural Network trained end-to- end. Our network was trained and tested in a custom dataset that we make publicly available. It can correctly separate the lobes even in cases when the fissure is not well delineated, achieving 0.93 in per-lobe Dice Coefficient and 0.85 in the inter-lobar Dice Coefficient in the test set. Both quantitative and qualitative results show that the proposed method can learn to produce correct lobe segmentations even when trained on a reduced dataset.

Background Changes in the retinal vessel caliber are associated with a variety of major diseases, namely diabetes, hypertension and atherosclerosis. The clinical assessment of these changes in fundus images is tiresome and prone to errors and thus automatic methods are desirable for objective and precise caliber measurement. However, the variability of blood vessel appearance, image quality and resolution make the development of these tools a non-trivial task. Metholodogy A method for the estimation of vessel caliber in eye fundus images via vessel cross-sectional intensity profile model fitting is herein proposed. First, the vessel centerlines are determined and individual segments are extracted and smoothed by spline approximation. Then, the corresponding cross-sectional intensity profiles are determined, post-processed and ultimately fitted by newly proposed parametric models. These models are based on Difference-of-Gaussians (DoG) curves modified through a multiplying line with varying inclination. With this, the proposed models can describe profile asymmetry, allowing a good adjustment to the most difficult profiles, namely those showing central light reflex. Finally, the parameters of the best-fit model are used to determine the vessel width using ensembles of bagged regression trees with random feature selection. Results and conclusions The performance of our approach is evaluated on the REVIEW public dataset by comparing the vessel cross-sectional profile fitting of the proposed modified DoG models with 7 and 8 parameters against a Hermite model with 6 parameters. Results on different goodness of fitness metrics indicate that our models are constantly better at fitting the vessel profiles. Furthermore, our width measurement algorithm achieves a precision close to the observers, outperforming state-of-the art methods, and retrieving the highest precision when evaluated using cross-validation. This high performance supports the robustness of the algorithm and validates its use in retinal vessel width measurement and possible integration in a system for retinal vasculature assessment.

In medical image analysis applications, the availability of the large amounts of annotated data is becoming increasingly critical. However, annotated medical data is often scarce and costly to obtain. In this paper, we address the problem of synthesizing retinal color images by applying recent techniques based on adversarial learning. In this setting, a generative model is trained to maximize a loss function provided by a second model attempting to classify its output into real or synthetic. In particular, we propose to implement an adversarial autoencoder for the task of retinal vessel network synthesis. We use the generated vessel trees as an intermediate stage for the generation of color retinal images, which is accomplished with a generative adversarial network. Both models require the optimization of almost everywhere differentiable loss functions, which allows us to train them jointly. The resulting model offers an end-to-end retinal image synthesis system capable of generating as many retinal images as the user requires, with their corresponding vessel networks, by sampling from a simple probability distribution that we impose to the associated latent space. We show that the learned latent space contains a well-defined semantic structure, implying that we can perform calculations in the space of retinal images, e.g., smoothly interpolating new data points between two retinal images. Visual and quantitative results demonstrate that the synthesized images are substantially different from those in the training set, while being also anatomically consistent and displaying a reasonable visual quality.

The classification of retinal vessels into arteries and veins in eye fundus images is a relevant task for the automatic assessment of vascular changes. This paper presents a new approach to solve this problem by means of a Fully-Connected Convolutional Neural Network that is specifically adapted for artery/vein classification. For this, a loss function that focuses only on pixels belonging to the retinal vessel tree is built. The relevance of providing the model with different chromatic components of the source images is also analyzed. The performance of the proposed method is evaluated on the RITE dataset of retinal images, achieving promising results, with an accuracy of \(96\%\) on large caliber vessels, and an overall accuracy of \(84\%\).

We propose UOLO, a novel framework for the simultaneous detection and segmentation of structures of interest in medical images. UOLO consists of an object segmentation module which intermediate abstract representations are processed and used as input for object detection. The resulting system is optimized simultaneously for detecting a class of objects and segmenting an optionally different class of structures. UOLO is trained on a set of bounding boxes enclosing the objects to detect, as well as pixel-wise segmentation information, when available. A new loss function is devised, taking into account whether a reference segmentation is accessible for each training image, in order to suitably backpropagate the error. We validate UOLO on the task of simultaneous optic disc (OD) detection, fovea detection, and OD segmentation from retinal images, achieving state-of-the-art performance on public datasets.

Due to inevitable differences between the data used for training modern CAD systems and the data encountered when they are deployed in clinical scenarios, the ability to automatically assess the quality of predictions when no expert annotation is available can be critical. In this paper, we propose a new method for quality assessment of retinal vessel tree segmentations in the absence of a reference ground-truth. For this, we artificially degrade expert-annotated vessel map segmentations and then train a CNN to predict the similarity between the degraded images and their corresponding ground-truths. This similarity can be interpreted as a proxy to the quality of a segmentation. The proposed model can produce a visually meaningful quality score, effectively predicting the quality of a vessel tree segmentation in the absence of a manually segmented reference. We further demonstrate the usefulness of our approach by applying it to automatically find a threshold for soft probabilistic segmentations on a per-image basis. For an independent state-of-the-art unsupervised vessel segmentation technique, the thresholds selected by our approach lead to statistically significant improvements in F1-score \((+2.67\%)\) and Matthews Correlation Coefficient (+\(3.11\%\)) over the thresholds derived from ROC analysis on the training set. The score is also shown to correlate strongly with F1 and MCC when a reference is available.

The creation of retinal mosaics from sets of fundus photographs can significantly reduce the time spent on the diabetic retinopathy (DR) screening, because through mosaic analysis the ophthalmologists can examine several portions of the eye at a single glance and, consequently, detect and grade DR more easily. Like most of the methods described in the literature, this methodology includes two main steps: image registration and image blending. In the registration step, relevant keypoints are detected on all images, the transformation matrices are estimated based on the correspondences between those keypoints and the images are reprojected into the same coordinate system. However, the main contributions of this work are in the blending step. In order to combine the overlapping images, a color compensation is applied to those images and a distance-based map of weights is computed for each one. The methodology is applied to two different datasets and the mosaics obtained for one of them are visually compared with the results of two state-of-the-art methods. The mosaics obtained with our method present good quality and they can be used for DR grading.

This paper introduces a novel strategy for the task of simultaneously locating two key anatomical landmarks in retinal images of the eye fundus, namely the optic disc and the fovea. For that, instead of attempting to classify each pixel as belonging to the background, the optic disc, or the fovea center, which would lead to a highly class-imbalanced setting, the problem is reformulated as a pixelwise regression task. The regressed quantity consists of the distance from the closest landmark of interest. A Fully-Convolutional Deep Neural Network is optimized to predict this distance for each image location, implicitly casting the problem into a per-pixel Multi-Task Learning approach by which a globally consistent distribution of distances across the entire image can be learned. Once trained, the two minimal distances predicted by the model are selected as the locations of the optic disc and the fovea. The joint learning of every pixel position relative to the optic disc and the fovea favors an automatic understanding of the overall anatomical distribution. This results in an effective technique that can detect both locations simultaneously, as opposed to previous methods that handle both tasks separately. Comprehensive experimental results on a large public dataset validate the proposed approach.

Given a retinal image, can we automatically determine whether it is of high quality (suitable for medical diagnosis)? Can we also explain our decision, pinpointing the region or regions that led to our decision? Images from human retinas are vital for the diagnosis of multiple health issues, like hypertension, diabetes, and Alzheimer’s; low quality images may force the patient to come back again for a second scanning, wasting time and possibly delaying treatment. However, existing retinal image quality assessment methods are either black boxes without explanations of the results or depend heavily on feature engineering or on complex and error-prone anatomical structures’ segmentation. Therefore, we propose EyeQual, that solves exactly this problem. EyeQual is novel, fast for inference, accurate and explainable, pinpointing low-quality regions on the image. We evaluated EyeQual on two real datasets where it achieved 100% accuracy taking just 36 milliseconds for each image.

The automatic assessment of visual quality on images of the eye fundus is an important task in retinal image analysis. A novel quality assessment technique is proposed in this paper. We propose to compute Mean-Subtracted Contrast-Normalized (MSCN) coefficients on local spatial neighborhoods of a given image and analyze their distribution. It is known that for natural images, such distribution behaves normally, while distortions of different kinds perturb this regularity. The combination of MSCN coefficients with a simple measure of local contrast allows us to design a simple but effective retinal image quality assessment algorithm that successfully discriminates between good and low-quality images, while delivering a meaningful quality score. The proposed technique is validated on a recent database of quality-labeled retinal images, obtaining results aligned with state-of-the-art approaches at a low computational cost.

Image quality assessment has been a topic of intense research over the last decades. Although its application to other disciplines is growing tremendously, its use in retinal imaging is still immature and some fundamental challenges remain unsolved. Thus, we present a research methodology for the objective assessment of the quality in retinal images. The methodology can be used as a preliminary step in any computer-aided system, and is composed of four main steps: the location of the region-of-interest, the extraction of relevant image properties and their analysis by feature selection, and the final binary classification into two classes (good and poor quality). The experimental results demonstrate the adequacy of the proposed methodology in this context, being able to objectively assess the quality of retinal images with an accuracy over 99%.

A method for the detection of juxta-pleural lung nodules with radius ≤ 5mm in chest computed tomography images is proposed. The lung volume is segmented using region-growing and refined with morphological operations and active contours to include juxta-pleural nodules. Nodule candidates are searched slice-wise inside the lung volume segmentation. Solid nodules are detected by selecting an appropriate threshold inside a representative sliding window. Sub-solid and non-solid nodules are enhanced with a multiscale Laplacian-of-Gaussian filtering prior to their detection. Obvious non-nodule candidates, namely small blood vessels, are discarded using fixed rules. Then, a support vector machine with radial basis function is trained with the remaining candidates to further reduce the number of false positives (FPs). The final system sensitivity is 57.4% with 4 FPs/scan.The performance is similar or better than state-of-the-art methods, especially when considering the high number and small radius of the studied juxta-pleural nodules.

Retinal vessel caliber changes are associated with several major diseases, such as diabetes and hypertension. These caliber changes can be evaluated using eye fundus images. However, the clinical assessment is tiresome and prone to errors, motivating the development of automatic methods. An automatic method based on vessel crosssection intensity profile model fitting for the estimation of vessel caliber in retinal images is herein proposed. First, vessels are segmented from the image, vessel centerlines are detected and individual segments are extracted and smoothed. Intensity profiles are extracted perpendicularly to the vessel, and the profile lengths are determined. Then, model fitting is applied to the smoothed profiles. A novel parametric model (DoG-L7) is used, consisting on a Difference-of-Gaussians multiplied by a line which is able to describe profile asymmetry. Finally, the parameters of the best-fit model are used for determining the vessel width through regression using ensembles of bagged regression trees with random sampling of the predictors (random forests). The method is evaluated on the REVIEW public dataset. A precision close to the observers is achieved, outperforming other state-of-the-art methods. The method is robust and reliable for width estimation in images with pathologies and artifacts, with performance independent of the range of diameters.

Microcalcifications are early indicators of breast cancer that appear on mammograms as small bright regions within the breast tissue. To assist screening radiologists in reading mammograms, supervised learning techniques have been found successful to detect microcalcifications automatically. Among them, Convolutional Neural Networks (CNNs) can automatically learn and extract low-level features that capture contrast and spatial information, and use these features to build robust classifiers. Therefore, spatial enhancement that enhances local contrast based on spatial context is expected to positively influence the learning task of the CNN and, as a result, its classification performance. In this work, we propose a novel spatial enhancement technique for microcalcifications based on the removal of haze, an apparently unrelated phenomenon that causes image degradation due to atmospheric absorption and scattering. We tested the influence of dehazing of digital mammograms on the microcalcification detection performance of two CNNs inspired by the popular AlexNet and VGGnet. Experiments were performed on 1, 066 mammograms acquired with GE Senographe systems. Statistically significantly better microcalcification detection performance was obtained when dehazing was used as preprocessing. Results of dehazing were superior also to those obtained with Contrast Limited Adaptive Histogram Equalization (CLAHE).

Retinal vessel segmentation is a fundamental and well-studied problem in the retinal image analysis field. The standard images in this context are color photographs acquired with standard fundus cameras. Several vessel segmentation techniques have been proposed in the literature that perform successfully on this class of images. However, for other retinal imaging modalities, blood vessel extraction has not been thoroughly explored. In this paper, we propose a vessel segmentation technique for Scanning Laser Opthalmoscopy (SLO) retinal images. Our method adapts a Deep Neural Network (DNN) architecture initially devised for segmentation of biological images (U-Net), to perform the task of vessel segmentation. The model was trained on a recent public dataset of SLO images. Results show that our approach efficiently segments the vessel network, achieving a performance that outperforms the current state-of-the-art on this particular class of images.

Assessment of retinal vessels is fundamental for the diagnosis of many disorders such as heart diseases, diabetes and hypertension. The imaging of retina using advanced fundus camera has become a standard in computer-assisted diagnosis of opthalmic disorders. Modern cameras produce high quality color digital images, but during the acquisition process the light reflected by the retinal surface generates a luminosity and contrast variation. Irregular illumination can introduce severe distortions in the resulting images, decreasing the visibility of anatomical structures and consequently demoting the performance of the automated segmentation of these structures. In this paper, a novel approach for illumination correction of color fundus images is proposed and applied as preprocessing step for retinal vessel segmentation. Our method builds on the connection between two different phenomena, shadows and haze, and works by removing the haze from the image in the inverted intensity domain. This is shown to be equivalent to correct the nonuniform illumination in the original intensity domain. We tested the proposed method as preprocessing stage of two vessel segmentation methods, one unsupervised based on mathematical morphology, and one supervised based on deep learning Convolutional Neural Networks (CNN). Experiments were performed on the publicly available retinal image database DRIVE. Statistically significantly better vessel segmentation performance was achieved in both test cases when illumination correction was applied.