Search

This study aims to investigate retinal and choroidal vascular reactivity to carbogen in central serous chorioretinopathy (CSC) patients.An experimental pilot study including 68 eyes from 20 CSC patients and 14 age and sex-matched controls was performed. The participants inhaled carbogen (5% CONo significant differences were detected in baseline hemodynamic parameters between both groups. A significant positive correlation was found between the participants' age and arterial diameter variation (p 0.001, r = 0.447), meaning that younger participants presented a more vasoconstrictive response (negative variation) than older ones. No significant differences were detected in the vasoreactive response between CSC and controls for both arterial and venous vessels (p = 0.63 and p = 0.85, respectively). Although the vascular reactivity was not related to the activity of CSC, it was related to the time of disease, for both the arterial (p = 0.02, r = 0.381) and venous (p = 0.001, r = 0.530) beds. SFCT and MCCT were highly correlated (r = 0.830, p 0.001). Both SFCT and MCCT significantly increased in CSC patients (p 0.001 and p 0.001) but not in controls (p = 0.059 and 0.247). A significant negative correlation between CSC patients' age and MCCT variation (r = - 0.340, p = 0.049) was detected. In CSC patients, the choroidal thickness variation was not related to the activity state, time of disease, or previous photodynamic treatment.Vasoreactivity to carbogen was similar in the retinal vessels but significantly higher in the choroidal vessels of CSC patients when compared to controls, strengthening the hypothesis of a choroidal regulation dysfunction in this pathology.

Is there an initialization for deep networks that requires no learning? ExplainFix adopts two design principles: the "fixed filters" principle that all spatial filter weights of convolutional neural networks can be fixed at initialization and never learned, and the "nimbleness" principle that only few network parameters suffice. We contribute (a) visual model-based explanations, (b) speed and accuracy gains, and (c) novel tools for deep convolutional neural networks. ExplainFix gives key insights that spatially fixed networks should have a steered initialization, that spatial convolution layers tend to prioritize low frequencies, and that most network parameters are not necessary in spatially fixed models. ExplainFix models have up to 100x fewer spatial filter kernels than fully learned models and matching or improved accuracy. Our extensive empirical analysis confirms that ExplainFix guarantees nimbler models (train up to 17\% faster with channel pruning), matching or improved predictive performance (spanning 13 distinct baseline models, four architectures and two medical image datasets), improved robustness to larger learning rate, and robustness to varying model size. We are first to demonstrate that all spatial filters in state-of-the-art convolutional deep networks can be fixed at initialization, not learned., Comment: Recently Published in Wiley WIREs Journal of Data Mining and Knowledge Discovery. This version has minor formatting differences and includes the supplementary appendix with the main document. Source code: https://github.com/adgaudio/ExplainFix/

Backed by more powerful computational resources and optimized training routines, deep learning models have proven unprecedented performance and several benefits to extract information from chest X-ray data. This is one of the most common imaging exams, whose increasing demand is reflected in the aggravated radiologists’ workload. Consequently, healthcare would benefit from computer-aided diagnosis systems to prioritize certain exams and further identify possible pathologies. Pioneering work in chest X-ray analysis has focused on the identification of specific diseases, but to the best of the authors' knowledge no paper has specifically reviewed relevant work on abnormality detection and multi-label thoracic pathology classification. This paper focuses on those issues, selecting the leading chest X-ray based deep learning strategies for comparison. In addition, the paper discloses the current annotated public chest X-ray databases, covering the common thorax diseases.

Aims: Late diagnoses of patients affected by pulmonary hypertension (PH) have a poor outcome. This observation has led to a call for earlier, non-invasive PH detection. Cardiac auscultation can represent a non-invasive, cost-effective alternative to both right heart catheterization and doppler analysis to measure the pulmonary artery pressure (PAP), due to the physiological relations that links changes in the PAP with the morphology of heart sounds. Methods: We propose the use of deep neural network applied to digital heart sound recordings to detect hypertensive patients. Post-hoc explanations are used to identify which features of the heart sounds are mostly informative of the presence of PH. Results: The proposed approach was tested over a dataset of 42 subjects (29 with PH and 13 without PH), with reference PAP measurements obtained via right heart catheterization. The proposed approach achieves .95 ROC AUC, compared with .78 of an adaptation of the Gaussian mixture model (GMM) method proposed in (Kaddoura et al. 2016). Classification outputs are shown to be fundamentally based on specific regions of the heart sounds. Conclusions: The results obtained with the proposed method show that deep learning approaches applied to heart sound achieve the state-of-the-art. Aims: Late diagnoses of patients affected by pulmonary hypertension (PH) have a poor outcome. This observation has led to a call for earlier, non-invasive PH detection. Cardiac auscultation can represent a non-invasive, cost-effective alternative to both right heart catheterization and doppler analysis to measure the pulmonary artery pressure (PAP), due to the physiological relations that links changes in the PAP with the morphology of heart sounds. Methods: We propose the use of deep neural network applied to digital heart sound recordings to detect hypertensive patients. Post-hoc explanations are used to identify which features of the heart sounds are mostly informative of the presence of PH. Results: The proposed approach was tested over a dataset of 42 subjects (29 with PH and 13 without PH), with reference PAP measurements obtained via right heart catheterization. The proposed approach achieves .95 ROC AUC, compared with .78 of an adaptation of the Gaussian mixture model (GMM) method proposed in (Kaddoura et al. 2016). Classification outputs are shown to be fundamentally based on specific regions of the heart sounds. Conclusions: The results obtained with the proposed method show that deep learning approaches applied to heart sound achieve the state-of-the-art.

The coronavirus disease 2019 (COVID-19) pandemic has impacted healthcare systems across the world. Chest radiography (CXR) can be used as a complementary method for diagnosing/following COVID-19 patients. However, experience level and workload of technicians and radiologists may affect the decision process. Recent studies suggest that deep learning can be used to assess CXRs, providing an important second opinion for radiologists and technicians in the decision process, and super-human performance in detection of COVID-19 has been reported in multiple studies. In this study, the clinical applicability of deep learning systems for COVID-19 screening was assessed by testing the performance of deep learning systems for the detection of COVID-19. Specifically, four datasets were used: (1) a collection of multiple public datasets (284.793 CXRs); (2) BIMCV dataset (16.631 CXRs); (3) COVIDGR (852 CXRs) and 4) a private dataset (6.361 CXRs). All datasets were collected retrospectively and consist of only frontal CXR views. A ResNet-18 was trained on each of the datasets for the detection of COVID-19. It is shown that a high dataset bias was present, leading to high performance in intradataset train-test scenarios (area under the curve 0.55–0.84 on the collection of public datasets). Significantly lower performances were obtained in interdataset train-test scenarios however (area under the curve > 0.98). A subset of the data was then assessed by radiologists for comparison to the automatic systems. Finetuning with radiologist annotations significantly increased performance across datasets (area under the curve 0.61–0.88) and improved the attention on clinical findings in positive COVID-19 CXRs. Nevertheless, tests on CXRs from different hospital services indicate that the screening performance of CXR and automatic systems is limited (area under the curve < 0.6 on emergency service CXRs). However, COVID-19 manifestations can be accurately detected when present, motivating the use of these tools for evaluating disease progression on mild to severe COVID-19 patients.

Objectives This study evaluated the usefulness of artificial intelligence (AI) algorithms as tools in improving the accuracy of histologic classification of breast tissue. Methods Overall, 100 microscopic photographs (test A) and 152 regions of interest in whole-slide images (test B) of breast tissue were classified into 4 classes: normal, benign, carcinoma in situ (CIS), and invasive carcinoma. The accuracy of 4 pathologists and 3 pathology residents were evaluated without and with the assistance of algorithms. Results In test A, algorithm A had accuracy of 0.87, with the lowest accuracy in the benign class (0.72). The observers had average accuracy of 0.80, and most clinically relevant discordances occurred in distinguishing benign from CIS (7.1% of classifications). With the assistance of algorithm A, the observers significantly increased their average accuracy to 0.88. In test B, algorithm B had accuracy of 0.49, with the lowest accuracy in the CIS class (0.06). The observers had average accuracy of 0.86, and most clinically relevant discordances occurred in distinguishing benign from CIS (6.3% of classifications). With the assistance of algorithm B, the observers maintained their average accuracy. Conclusions AI tools can increase the classification accuracy of pathologists in the setting of breast lesions.

Early diagnosis of lung cancer via computed tomography can significantly reduce the morbidity and mortality rates associated with the pathology. However, searching lung nodules is a high complexity task, which affects the success of screening programs. Whilst computer-aided detection systems can be used as second observers, they may bias radiologists and introduce significant time overheads. With this in mind, this study assesses the potential of using gaze information for integrating automatic detection systems in the clinical practice. For that purpose, 4 radiologists were asked to annotate 20 scans from a public dataset while being monitored by an eye tracker device, and an automatic lung nodule detection system was developed. Our results show that radiologists follow a similar search routine and tend to have lower fixation periods in regions where finding errors occur. The overall detection sensitivity of the specialists was $\mathbf {0.67\pm 0.07}$ , whereas the system achieved 0.69. Combining the annotations of one radiologist with the automatic system significantly improves the detection performance to similar levels of two annotators. Filtering automatic detection candidates only for low fixation regions still significantly improves the detection sensitivity without increasing the number of false-positives.

Proliferative diabetic retinopathy (PDR) is an advanced diabetic retinopathy stage, characterized by neovascularization, which leads to ocular complications and severe vision loss. However, the available DR-labeled retinal image datasets have a small representation of images of the severest DR grades, and thus there is lack of PDR cases for training DR grading models. Additionally, the criteria for labelling these images in the publicly available datasets is not always clear, with some images which do not show typical PDR lesions being labeled as PDR due to the presence of photo-coagulation treatment and laser marks. This problem, together with the datasets' high class imbalance, leads to a limited variability of the samples, which the typical data augmentation and class balancing cannot fully mitigate. We propose a heuristic-based data augmentation scheme based on the synthesis of neovessel (NV)-like structures that compensates for the lack of PDR cases in DR-labeled datasets. The proposed neovessel generation algorithm relies on the general knowledge of common location and shape of these structures. NVs are generated and introduced in pre-existent retinal images which can then be used for enlarging deep neural networks' training sets. The data augmentation scheme was tested on multiple datasets, and allows to improve the model's capacity to detect NVs.

The worldwide pandemic caused by the new coronavirus (COVID-19) has encouraged the development of multiple computer-aided diagnosis systems to automate daily clinical tasks, such as abnormality detection and classification. Among these tasks, the segmentation of COVID lesions is of high interest to the scientific community, enabling further lesion characterization. Automating the segmentation process can be a useful strategy to provide a fast and accurate second opinion to the physicians, and thus increase the reliability of the diagnosis and disease stratification. The current work explores a CNN-based approach to segment multiple COVID lesions. It includes the implementation of a U-Net structure with a ResNet34 encoder able to deal with the highly imbalanced nature of the problem, as well as the great variability of the COVID lesions, namely in terms of size, shape, and quantity. This approach yields a Dice score of 64.1%, when evaluated on the publicly available COVID-19-20 Lung CT Lesion Segmentation GrandChallenge data set.

Breast cancer is the most common invasive cancer in women, affecting more than 10% of women worldwide. Microscopic analysis of a biopsy remains one of the most important methods to diagnose the type of breast cancer. This requires specialized analysis by pathologists, in a task that i) is highly time- and cost-consuming and ii) often leads to nonconsensual results. The relevance and potential of automatic classification algorithms using hematoxylin-eosin stained histopathological images has already been demonstrated, but the reported results are still sub-optimal for clinical use. With the goal of advancing the state-of-the-art in automatic classification, the Grand Challenge on BreAst Cancer Histology images (BACH) was organized in conjunction with the 15th International Conference on Image Analysis and Recognition (ICIAR 2018). A large annotated dataset, composed of both microscopy and whole-slide images, was specifically compiled and made publicly available for the BACH challenge. Following a positive response from the scientific community, a total of 64 submissions, out of 677 registrations, effectively entered the competition. From the submitted algorithms it was possible to push forward the state-of-the-art in terms of accuracy (87%) in automatic classification of breast cancer with histopathological images. Convolutional neuronal networks were the most successful methodology in the BACH challenge. Detailed analysis of the collective results allowed the identification of remaining challenges in the field and recommendations for future developments. The BACH dataset remains publically available as to promote further improvements to the field of automatic classification in digital pathology., Accepted for publication at Medical Image Analysis (Elsevier). Publication licensed under the Creative Commons CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/

Lung cancer is the deadliest type of cancer worldwide and late detection is the major factor for the low survival rate of patients. Low dose computed tomography has been suggested as a potential screening tool but manual screening is costly and time-consuming. This has fuelled the development of automatic methods for the detection, segmentation and characterisation of pulmonary nodules. In spite of promising results, the application of automatic methods to clinical routine is not straightforward and only a limited number of studies have addressed the problem in a holistic way. With the goal of advancing the state of the art, the Lung Nodule Database (LNDb) Challenge on automatic lung cancer patient management was organized. The LNDb Challenge addressed lung nodule detection, segmentation and characterization as well as prediction of patient follow-up according to the 2017 Fleischner society pulmonary nodule guidelines. 294 CT scans were thus collected retrospectively at the Centro Hospitalar e Universitrio de So Joo in Porto, Portugal and each CT was annotated by at least one radiologist. Annotations comprised nodule centroids, segmentations and subjective characterization. 58 CTs and the corresponding annotations were withheld as a separate test set. A total of 947 users registered for the challenge and 11 successful submissions for at least one of the sub-challenges were received. For patient follow-up prediction, a maximum quadratic weighted Cohen's kappa of 0.580 was obtained. In terms of nodule detection, a sensitivity below 0.4 (and 0.7) at 1 false positive per scan was obtained for nodules identified by at least one (and two) radiologist(s). For nodule segmentation, a maximum Jaccard score of 0.567 was obtained, surpassing the interobserver variability. In terms of nodule texture characterization, a maximum quadratic weighted Cohen's kappa of 0.733 was obtained, with part solid nodules being particularly challenging to classify correctly. Detailed analysis of the proposed methods and the differences in performance allow to identify the major challenges remaining and future directions - data collection, augmentation/generation and evaluation of under-represented classes, the incorporation of scan-level information for better decision-making and the development of tools and challenges with clinical-oriented goals. The LNDb Challenge and associated data remain publicly available so that future methods can be tested and benchmarked, promoting the development of new algorithms in lung cancer medical image analysis and patient follow-up recommendation.

Diabetic retinopathy (DR) is a diabetes complication, which in extreme situations may lead to blindness. Since the first stages are often asymptomatic, regular eye examinations are required for an early diagnosis. As microaneurysms (MAs) are one of the first signs of DR, several automated methods have been proposed for their detection in order to reduce the ophthalmologists' workload. Although local convergence filters (LCFs) have already been applied for feature extraction, their potential as MA enhancement operators was not explored yet. In this work, we propose a sliding band filter for MA enhancement aiming at obtaining a set of initial MA candidates. Then, a combination of the filter responses with color, contrast and shape information is used by an ensemble of classifiers for final candidate classification. Finally, for each eye fundus image, a score is computed from the confidence values assigned to the MAs detected in the image. The performance of the proposed methodology was evaluated in four datasets. At the lesion level, sensitivities of 64% and 81% were achieved for an average of 8 false positives per image (FPIs) in e-ophtha MA and SCREEN-DR, respectively. In the last dataset, an AUC of 0.83 was also obtained for DR detection.

Currently, lung cancer is the most lethal in the world. In order to make screening and follow-up a little more systematic, guidelines have been proposed. Therefore, this study aimed to create a diagnostic support approach by providing a patient label based on the LUNG-RADS™ guidelines. The only input required by the system is the nodule centroid to take the region of interest for the input of the classification system. With this in mind, two deep learning networks were evaluated: a Wide Residual Network and a DenseNet. Taking into account the annotation uncertainty we proposed to use sample weights that are introduced in the loss function, allowing nodules with a high agreement in the annotation process to take a greater impact on the training error than its counterpart. The best result was achieved with the Wide Residual Network with sample weights achieving a nodule-wise LUNG-RADS™ labelling accuracy of $0.735\pm 0.003$ .

Active Learning methods create an optimized labeled training set from unlabeled data. We introduce a novel Online Active Deep Learning method for Medical Image Analysis. We extend our MedAL active learning framework to present new results in this paper. Our novel sampling method queries the unlabeled examples that maximize the average distance to all training set examples. Our online method enhances performance of its underlying baseline deep network. These novelties contribute significant performance improvements, including improving the model's underlying deep network accuracy by 6.30%, using only 25% of the labeled dataset to achieve baseline accuracy, reducing backpropagated images during training by as much as 67%, and demonstrating robustness to class imbalance in binary and multi-class tasks., Comment: Code: https://github.com/adgaudio/o-medal ; Accepted and published by Wiley Journal of Pattern Recognition and Knowledge Discovery ; Journal URL: https://doi.org/10.1002/widm.1353

Diabetic macular edema is a leading cause of visual loss for patients with diabetes. While diagnosis can only be performed by optical coherence tomography, diabetic macular edema risk assessment is often performed in eye fundus images in screening scenarios through the detection of hard exudates. Such screening scenarios are often associated with large amounts of data, high costs and high burden on specialists, motivating then the development of methodologies for automatic diabetic macular edema risk prediction. Nevertheless, significant dataset domain bias, due to different acquisition equipment, protocols and/or different populations can have significantly detrimental impact on the performance of automatic methods when transitioning to a new dataset, center or scenario. As such, in this study, a method based on residual neural networks is proposed for the classification of diabetic macular edema risk. This method is then validated across multiple public datasets, simulating the deployment in a multi-center setting and thereby studying the method’s generalization capability and existing dataset domain bias. Furthermore, the method is tested on a private dataset which more closely represents a realistic screening scenario. An average area under the curve across all public datasets of 0.891 ± 0.013 was obtained with a ResNet50 architecture trained on a limited amount of images from a single public dataset (IDRiD). It is also shown that screening scenarios are significantly more challenging and that training across multiple datasets leads to an improvement of performance (area under the curve of 0.911 ± 0.009).

The optic disc (OD) and the fovea are relevant landmarks in fundus images. Their localization and segmentation can facilitate the detection of some retinal lesions and the assessment of their importance to the severity and progression of several eye disorders. Distinct methodologies have been developed for detecting these structures, mainly based on color and vascular information. The methodology herein described combines the entropy of the vessel directions with the image intensities for finding the OD center and uses a sliding band filter for segmenting the OD. The fovea center corresponds to the darkest point inside a region defined from the OD position and radius. Both the Messidor and the IDRiD datasets are used for evaluating the performance of the developed methods. In the first one, a success rate of 99.56% and 100.00% are achieved for OD and fovea localization. Regarding the OD segmentation, the mean Jaccard index and Dice’s coefficient obtained are 0.87 and 0.94, respectively. The proposed methods are also amongst the top-3 performing solutions submitted to the IDRiD online challenge.

We propose a pixel color amplification theory and family of enhancement methods to facilitate segmentation tasks on retinal images. Our novel re-interpretation of the image distortion model underlying dehazing theory shows how three existing priors commonly used by the dehazing community and a novel fourth prior are related. We utilize the theory to develop a family of enhancement methods for retinal images, including novel methods for whole image brightening and darkening. We show a novel derivation of the Unsharp Masking algorithm. We evaluate the enhancement methods as a pre-processing step to a challenging multi-task segmentation problem and show large increases in performance on all tasks, with Dice score increases over a no-enhancement baseline by as much as 0.491. We provide evidence that our enhancement preprocessing is useful for unbalanced and difficult data. We show that the enhancements can perform class balancing by composing them together.

Ovarian cancer is one of the most commonly occurring cancer in women. Transvaginal ultrasound is used as a screening test to detect the presence of tumors but, for specific types of ovarian tumors, malignancy can only be asserted through surgery. An automatic method to perform the detection and malignancy assessment of these tumours is thus necessary to prevent unnecessary oophorectomies.This work explores the U-Net’s architecture and investigates the selection of different hyperparameters for the ovary and the ovarian follicles segmentation. The effect of applying different post-processing methods on beam-formed radio-frequency (BRF) data is also investigated.Results show that models trained only with BRF data have the worst performance. On the other hand, the combination of B-mode with BRF data performs better for ovary segmentation. As for the hyperparameter study, results show that the U-Net with 4 levels is the architecture with the worst performance. This shows that to achieve better performance in the segmentation of ovarian structures, it is important to select an architecture that takes into account the spatial context of the regions of interest. It is also possible to conclude that the method used to analyse BRF data should be designed to take advantage of the fine-resolution of BRF data.

A new approach for robust edge detection on B-mode ultrasound images of the carotid artery is proposed in this paper. The proposed method uses anisotropic Gaussian derivative filters along with non-maximum suppression over the overall artery wall orientation in local regions. The anisotropic filters allow using a wider integration scale along the edges while preserving the edge location precision. They also perform edge continuation, resulting in the connection of isolated edge points along linear segments, which is a valuable feature for the segmentation of the artery wall layers. However, this usually results in false edges being detected near convex contours and isolated points. The use of non-maximum suppression over pooled local orientations is proposed to solve this issue. Experimental results are provided to demonstrate that the proposed edge detector outperforms other common methods in the detection of the lumen-intima and media-adventia layer interfaces of the carotid vessel walls. Additionally, the resulting edges are more continuous and precisely located.

Diabetic retinopathy (DR) detection is a critical retinal image analysis task in the context of early blindness prevention. Unfortunately, in order to train a model to accurately detect DR based on the presence of different retinal lesions, typically a dataset with medical expert's annotations at the pixel level is needed. In this paper, a new methodology based on the multiple instance learning (MIL) framework is developed in order to overcome this necessity by leveraging the implicit information present on annotations made at the image level. Contrary to previous MIL-based DR detection systems, the main contribution of the proposed technique is the joint optimization of the instance encoding and the image classification stages. In this way, more useful mid-level representations of pathological images can be obtained. The explainability of the model decisions is further enhanced by means of a new loss function enforcing appropriate instance and mid-level representations. The proposed technique achieves comparable or better results than other recently proposed methods, with 90% area under the receiver operating characteristic curve (AUC) on Messidor, 93% AUC on DR1, and 96% AUC on DR2, while improving the interpretability of the produced decisions.

Lung cancer is the type of cancer with highest mortality worldwide. Low-dose computerized tomography is the main tool used for lung cancer screening in clinical practice, allowing the visualization of lung nodules and the assessment of their malignancy. However, this evaluation is a complex task and subject to inter-observer variability, which has fueled the need for computer-aided diagnosis systems for lung nodule malignancy classification. While promising results have been obtained with automatic methods, it is often not straightforward to determine which features a given model is basing its decisions on and this lack of explainability can be a significant stumbling block in guaranteeing the adoption of automatic systems in clinical scenarios. Though visual malignancy assessment has a subjective component, radiologists strongly base their decision on nodule features such as nodule spiculation and texture, and a malignancy classification model should thus follow the same rationale. As such, this study focuses on the characterization of lung nodules as a means for the classification of nodules in terms of malignancy. For this purpose, different model architectures for nodule characterization are proposed and compared, with the final goal of malignancy classification. It is shown that models that combine direct malignancy prediction with specific branches for nodule characterization have a better performance than the remaining models, achieving an Area Under the Curve of 0.783. The most relevant features for malignancy classification according to the model were lobulation, spiculation and texture, which is found to be in line with current clinical practice.

In the design of computer-aided diagnosis systems for lung cancer diagnosis, an appropriate and accurate segmentation of the pulmonary nodules in computerized tomography (CT) is one of the most relevant and difficult tasks. An accurate segmentation is crucial for the posterior measurement of nodule characteristics and for lung cancer diagnosis. This paper proposes different approaches that use Hessian-based strategies for lung nodule segmentation in chest CT scans. We propose a multiscale segmentation process that uses the central medialness adaptive principle, a Hessian-based strategy that was originally formulated for tubular extraction but it also provides good segmentation results in blob-like structures as is the case of lung nodules. We compared this proposal with a well established Hessian-based strategy that calculates the Shape Index (SI) and Curvedness (CV). We adapted the SI and CV approach for multiscale nodule segmentation. Moreover, we propose the combination of both strategies by combining the results, in order to take benefit of the advantages of both strategies. Different cases with pulmonary nodules from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) database were taken and used to analyze and validate the approaches. The chest CT images present a large variability in nodule characteristics and image conditions. Our proposals provide an accurate lung nodule segmentation, similar to radiologists performance. Our Hessian-based approaches were validated with 569 solid and mostly solid nodules demonstrating that these novel strategies have good results when compared with the radiologists segmentations, providing accurate pulmonary nodule volumes for posterior characterization and appropriate diagnosis.

Lung cancer is the deadliest type of cancer worldwide and late detection is one of the major factors for the low survival rate of patients. Low dose computed tomography has been suggested as a potential early screening tool but manual screening is costly, time-consuming and prone to interobserver variability. This has fueled the development of automatic methods for the detection, segmentation and characterisation of pulmonary nodules but its application to the clinical routine is challenging. In this study, a platform for the development, deployment and testing of pulmonary nodule computer-aided strategies is presented: LNDetector. LNDetector integrates image exploration and nodule annotation tools as well as advanced nodule detection, segmentation and classification methods and gaze characterisation. Different processing modules can easily be implemented or replaced to test their efficiency in clinical environments and the use of gaze analysis allows for the development of collaborative strategies. The potential use of this platform is shown through a combination of visual search, gaze characterisation and automatic nodule detection tools for an efficient and collaborative computer-aided strategy for pulmonary nodule screening.

The lung cancer diagnosis is based on the search of lung nodules. Besides its characterization, it is also common to register the anatomical position of these findings. Even though computed-aided diagnosis systems tend to help in these tasks, there is still lacking a complete system that can qualitatively label the nodules in lung regions. In this way, this paper proposes an automatic lung reference model to facilitate the report of nodules between computed-aided diagnosis systems and the radiologist, and among radiologists. The model was applied to 115 computed tomography scans with manually and automatically segmented lobes, and the obtained sectors’ variability was evaluated. As the sectors average variability within lobes is less or equal to 0.14, the model can be a good way to promote the report of lung nodules.

Diabetic Retinopathy (DR) is one of the leading causes of preventable blindness in the developed world. With the increasing number of diabetic patients there is a growing need of an automated system for DR detection. We propose Eye WeS, a method that not only detects DR in eye fundus images but also pinpoints the regions of the image that contain lesions, while being trained with image labels only. We show that it is possible to convert any pre-trained convolutional neural network into a weakly-supervised model while increasing their performance and efficiency. EyeWeS improved the results of Inception V3 from 94.9% Area Under the Receiver Operating Curve (AUC) to 95.8% AUC while maintaining only approximately 5% of the Inception V3's number of parameters. The same model is able to achieve 97.1% AUC in a cross-dataset experiment.

Visual exploration of the larynx represents a relevant technique for the early diagnosis of laryngeal disorders. However, visualizing an endoscopy for finding abnormalities is a time-consuming process, and for this reason much research has been dedicated to the automatic analysis of endoscopic video data. In this work we address the particular task of discriminating among informative laryngoscopic frames and those that carry insufficient diagnostic information. In the latter case, the goal is also to determine the reason for this lack of information. To this end, we analyze the possibility of training three different state-of-the-art Convolutional Neural Networks, but initializing their weights from configurations that have been previously optimized for solving natural image classification problems. Our findings show that the simplest of these three architectures not only is the most accurate (outperforming previously proposed techniques), but also the fastest and most efficient, with the lowest inference time and minimal memory requirements, enabling real-time application and deployment in portable devices.

The automatic differentiation of retinal vessels into arteries and veins (A/V) is a highly relevant task within the field of retinal image analysis. However, due to limitations of retinal image acquisition devices, specialists can find it impossible to label certain vessels in eye fundus images. In this paper, we introduce a method that takes into account such uncertainty by design. For this, we formulate the A/V classification task as a four-class segmentation problem, and a Convolutional Neural Network is trained to classify pixels into background, A/V, or uncertain classes. The resulting technique can directly provide pixelwise uncertainty estimates. In addition, instead of depending on a previously available vessel segmentation, the method automatically segments the vessel tree. Experimental results show a performance comparable or superior to several recent A/V classification approaches. In addition, the proposed technique also attains state-of-the-art performance when evaluated for the task of vessel segmentation, generalizing to data that was not used during training, even with considerable differences in terms of appearance and resolution.

Lung volume segmentation is a key step in the design of Computer-Aided Diagnosis systems for automated lung pathology analysis. However, isolating the lung from CT volumes can be a challenging process due to considerable deformations and the potential presence of pathologies. Convolutional Neural Networks (CNN) are effective tools for modeling the spatial relationship between lung voxels. Unfortunately, they typically require large quantities of annotated data, and manually delineating the lung from volumetric CT scans can be a cumbersome process. We propose to train a 3D CNN to solve this task based on semi-automatically generated annotations. For this, we introduce an extension of the well-known V-Net architecture that can handle higher-dimensional input data. Even if the training set labels are noisy and contain errors, our experiments show that it is possible to learn to accurately segment the lung relying on them. Numerical comparisons on an external test set containing lung segmentations provided by a medical expert demonstrate that the proposed model generalizes well to new data, reaching an average 98.7% Dice coefficient. The proposed approach results in a superior performance with respect to the standard V-Net model, particularly on the lung boundary.

This study describes a novel dataset with retinal image quality annotation, defined by three different retinal experts, and presents an inter-observer analysis for quality assessment that can be used as gold-standard for future studies. A state-of-the-art algorithm for retinal image quality assessment is also analysed and compared against the specialists performance. Results show that, for 71% of the images present in the dataset, the three experts agree on the given image quality label. The results obtained for accuracy, specificity and sensitivity when comparing one expert against another were in the ranges [83.0 − 85.2]%, [72.7 − 92.9]% and [80.0 − 94.7]%, respectively. The evaluated automatic quality assessment method, despite not being trained on the novel dataset, presents a performance which is within inter-observer variability.

Lung cancer has an increasing preponderance in worldwide mortality, demanding for the development of efficient screening methods. With this in mind, a binary classification method using Lung-RADSTM guidelines to warn changes in the screening management is proposed. First, having into account the lack of public datasets for this task, the lung nodules in the LIDC-IDRI dataset were re-annotated to include a Lung-RADSTM-based referral label. Then, a wide residual network is used for automatically assessing lung nodules in 3D chest computed tomography exams. Unlike the standard malignancy prediction approaches, the proposed method avoids the need to segment and characterize lung nodules, and instead directly defines if a patient should be submitted for further lung cancer tests. The system achieves a nodule-wise accuracy of 0.87±0.02.

Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is challenging due to the scarcity of medical professionals able to screen a growing global diabetic population at risk for DR. Computer-aided disease diagnosis in retinal image analysis could provide a sustainable approach for such large-scale screening effort. The recent scientific advances in computing capacity and machine learning approaches provide an avenue for biomedical scientists to reach this goal. Aiming to advance the state-of-the-art in automatic DR diagnosis, a grand challenge on "Diabetic Retinopathy - Segmentation and Grading" was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2018). In this paper, we report the set-up and results of this challenge that is primarily based on Indian Diabetic Retinopathy Image Dataset (IDRiD). There were three principal sub-challenges: lesion segmentation, disease severity grading, and localization of retinal landmarks and segmentation. These multiple tasks in this challenge allow to test the generalizability of algorithms, and this is what makes it different from existing ones. It received a positive response from the scientific community with 148 submissions from 495 registrations effectively entered in this challenge. This paper outlines the challenge, its organization, the dataset used, evaluation methods and results of top-performing participating solutions. The top-performing approaches utilized a blend of clinical information, data augmentation, and an ensemble of models. These findings have the potential to enable new developments in retinal image analysis and image-based DR screening in particular.

Our goal in this chapter is to describe the recent application of deep learning and artificial intelligence (AI) techniques to retinal image analysis. Automatic retinal image analysis (ARIA) is a complex task that has significant applications for diagnostic purposes for a host of retinal, neurological, and vascular diseases. A number of approaches for the automatic analysis of the retinal images have been studied for the past two decades but the recent success of deep learning (DL) for a range of computer vision and image analysis tasks has now permeated medical imaging and ARIA. Since 2016, major improvements were reported using DL discriminative methods (deep convolutional neural networks or autoencoder convolutional networks), and generative methods, in combination with other image analysis methods, that have demonstrated the ability of algorithms to perform on par with ophthalmologists and retinal specialists, for tasks such as automated classification, diagnostics, and segmentation. We review these recent developments in this chapter.

Diabetic Retinopathy is the leading cause of blindness in the working-age population of the world. The main aim of this paper is to improve the accuracy of Diabetic Retinopathy detection by implementing a shadow removal and color correction step as a preprocessing stage from eye fundus images. For this, we rely on recent findings indicating that application of image dehazing on the inverted intensity domain amounts to illumination compensation. Inspired by this work, we propose a Shadow Removal Layer that allows us to learn the pre-processing function for a particular task. We show that learning the pre-processing function improves the performance of the network on the Diabetic Retinopathy detection task.

Lung cancer is one of the most common causes of death in the world. The early detection of lung nodules allows an appropriate follow-up, timely treatment and potentially can avoid greater damage in the patient health. The texture is one of the nodule characteristics that is correlated with the malignancy. We developed convolutional neural network architectures to classify automatically the texture of nodules into the non-solid, part-solid and solid classes. The different architectures were tested to determine if the context, the number of slices considered as input and the relation between slices influence on the texture classification performance. The architecture that obtained better performance took into account different scales, different rotations and the context of the nodule, obtaining an accuracy of 0.833 ± 0.041.

The segmentation and characterization of the ovarian structures are important tasks in gynecological and reproductive medicine. Ultrasound imaging is typically used for the medical diagnosis within this field but the understanding of the images can be difficult due to their characteristics. Furthermore, the complexity of ultrasound data may lead to a heavy image processing, which makes the application of classical methods of computer vision difficult. This work presents the first supervised fully convolutional neural network (fCNN) for the automatic segmentation of ovarian structures in B-mode ultrasound images. Due to the small dataset available, only 57 images were used for training. In order to overcome this limitation, several regularization techniques were used and are discussed in this paper. The experiments show the ability of the fCNN to learn features to distinguish ovarian structures, achieving a Dice similarity coefficient (DSC) of 0.855 for the segmentation of the stroma and a DSC of 0.955 for the follicles. When compared with a semi-automatic commercial application for follicle segmentation, the proposed fCNN achieved an average improvement of 19%.

Ovarian cancer is one of the pathologies with the worst prognostic in adult women and it has a very difficult early diagnosis. Clinical evaluation of gynaecological ultrasound images is performed visually, and it is dependent on the experience of the medical doctor. Besides the dependency on the specialists, the malignancy of specific types of ovarian tumors cannot be asserted until their surgical removal. This work explores the use of ultrasound data for the segmentation of the ovary and the ovarian follicles, using two different convolutional neural networks, a fully connected residual network and a U-Net, with a binary and multi-class approach. Five different types of ultrasound data (from beam-formed radio-frequency to brightness mode) were used as input. The best performance was obtained using B-mode, for both ovary and follicles segmentation. No significant differences were found between the two convolutional neural networks. The use of the multi-class approach was beneficial as it provided the model information on the spatial relation between follicles and the ovary. This study demonstrates the suitability of combining convolutional neural networks with beam-formed radio-frequency data and with brightness mode data for segmentation of ovarian structures. Future steps involve the processing of pathological data and investigation of biomarkers of pathological ovaries.

Deep learning models have been successfully used in medical image analysis problems but they require a large amount of labeled images to obtain good performance. However, such large labeled datasets are costly to acquire. Active learning techniques can be used to minimize the number of required training labels while maximizing the model's performance. In this work, we propose a novel sampling method that queries the unlabeled examples that maximize the average distance to all training set examples in a learned feature space. We then extend our sampling method to define a better initial training set, without the need for a trained model, by using Oriented FAST and Rotated BRIEF (ORB) feature descriptors. We validate MedAL on 3 medical image datasets and show that our method is robust to different dataset properties. MedAL is also efficient, achieving 80% accuracy on the task of Diabetic Retinopathy detection using only 425 labeled images, corresponding to a 32% reduction in the number of required labeled examples compared to the standard uncertainty sampling technique, and a 40% reduction compared to random sampling.

Lung cancer diagnosis is made by radiologists through nodule search in chest Computed Tomography (CT) scans. This task is known to be difficult and prone to errors that can lead to late diagnosis. Although Computer-Aided Diagnostic (CAD) systems are promising tools to be used in clinical practice, experienced radiologists continue to perform better diagnosis than CADs. This paper proposes a methodology for characterizing the radiologist's gaze during nodules search in chest CT scans. The main goals are to identify regions that attract the radiologists' attention, which can then be used for improving a lung CAD system, and to create a tool to assist radiologists during the search task. For that purpose, the methodology processes the radiologists' gaze and their mouse coordinates during the nodule search. The resulting data is then processed to obtain a 3D gaze path from which relevant attention studies can be derived. To better convey the found information, a reference model of the lung that eases the communication of the location of relevant anatomical/pathological findings is also proposed. The methodology is tested on a set of 24 real-practice gazes, recorded via an Eye tracker, from 3 radiologists.

Image dehazing tries to solve an undesired loss of visibility in outdoor images due to the presence of fog. Recently, machine-learning techniques have shown great dehazing ability. However, in order to be trained, they require training sets with pairs of foggy images and their clean counterparts, or a depth-map. In this paper, we propose to learn the appearance of fog from weakly-labeled data. Specifically, we only require a single label per-image stating if it contains fog or not. Based on the Multiple-Instance Learning framework, we propose a model that can learn from image-level labels to predict if an image contains haze reasoning at a local level. Fog detection performance of the proposed method compares favorably with two popular techniques, and the attention maps generated by the model demonstrate that it effectively learns to disregard sky regions as indicative of the presence of fog, a common pitfall of current image dehazing techniques.

The segmentation and characterization of the lung lobes are important tasks for Computer Aided Diagnosis (CAD) systems related to pulmonary disease. The detection of the fissures that divide the lung lobes is non-trivial when using classical methods that rely on anatomical information like the localization of the airways and vessels. This work presents a fully automatic and supervised approach to the problem of the segmentation of the five pulmonary lobes from a chest Computer Tomography (CT) scan using a Fully RegularizedV-Net (FRV- Net), a 3D Fully Convolutional Neural Network trained end-to- end. Our network was trained and tested in a custom dataset that we make publicly available. It can correctly separate the lobes even in cases when the fissure is not well delineated, achieving 0.93 in per-lobe Dice Coefficient and 0.85 in the inter-lobar Dice Coefficient in the test set. Both quantitative and qualitative results show that the proposed method can learn to produce correct lobe segmentations even when trained on a reduced dataset.

The classification of retinal vessels into arteries and veins in eye fundus images is a relevant task for the automatic assessment of vascular changes. This paper presents a new approach to solve this problem by means of a Fully-Connected Convolutional Neural Network that is specifically adapted for artery/vein classification. For this, a loss function that focuses only on pixels belonging to the retinal vessel tree is built. The relevance of providing the model with different chromatic components of the source images is also analyzed. The performance of the proposed method is evaluated on the RITE dataset of retinal images, achieving promising results, with an accuracy of \(96\%\) on large caliber vessels, and an overall accuracy of \(84\%\).

We propose UOLO, a novel framework for the simultaneous detection and segmentation of structures of interest in medical images. UOLO consists of an object segmentation module which intermediate abstract representations are processed and used as input for object detection. The resulting system is optimized simultaneously for detecting a class of objects and segmenting an optionally different class of structures. UOLO is trained on a set of bounding boxes enclosing the objects to detect, as well as pixel-wise segmentation information, when available. A new loss function is devised, taking into account whether a reference segmentation is accessible for each training image, in order to suitably backpropagate the error. We validate UOLO on the task of simultaneous optic disc (OD) detection, fovea detection, and OD segmentation from retinal images, achieving state-of-the-art performance on public datasets.