Search

Abstract Alzheimer’s disease (AD) is a global healthcare challenge lacking a simple and affordable detection method. We propose a novel deep learning framework, Eye-AD, to detect Early-onset Alzheimer’s Disease (EOAD) and Mild Cognitive Impairment (MCI) using OCTA images of retinal microvasculature and choriocapillaris. Eye-AD employs a multilevel graph representation to analyze intra- and inter-instance relationships in retinal layers. Using 5751 OCTA images from 1671 participants in a multi-center study, our model demonstrated superior performance in EOAD (internal data: AUC = 0.9355, external data: AUC = 0.9007) and MCI detection (internal data: AUC = 0.8630, external data: AUC = 0.8037). Furthermore, we explored the associations between retinal structural biomarkers in OCTA images and EOAD/MCI, and the results align well with the conclusions drawn from our deep learning interpretability analysis. Our findings provide further evidence that retinal OCTA imaging, coupled with artificial intelligence, will serve as a rapid, noninvasive, and affordable dementia detection.

Abstract Background Increased intracranial pressure (ICP) in patients with idiopathic intracranial hypertension (IIH) affects the retinal microvasculature, which can be imaged and quantified by optical coherence tomography angiography (OCTA). We aimed to identify the mediating factor between ICP and OCTA parameters association in IIH patients. Methods IIH patients with active intracranial hypertension were enrolled. OCTA imaging was performed after ICP measurement. We quantified the branching complexity of the retinal arterioles and venules from the superficial vascular complex of the OCTA image. Eyes of IIH patients were stratified into eyes with papilledema (IIH-P) and eyes without papilledema (IIH-WP). All participants underwent visual acuity (VA) examination. Results One hundred and thirty-eight eyes from 70 IIH patients and 146 eyes from 73 controls were included. Compared to the control group, IIH patients and IIH-P had reduced arteriole complexity and increased venule complexity (p

Background: The convergence of smartphone technology and artificial intelligence (AI) has revolutionized the landscape of ophthalmic care, offering unprecedented opportunities for diagnosis, monitoring, and management of ocular conditions. Nevertheless, there is a lack of systematic studies on discussing the integration of smartphone and AI in this field. Main text: This review includes 52 studies, and explores the integration of smartphones and AI in ophthalmology, delineating its collective impact on screening methodologies, disease detection, telemedicine initiatives, and patient management. The collective findings from the curated studies indicate promising performance of the smartphone-based AI screening for various ocular diseases which encompass major retinal diseases, glaucoma, cataract, visual impairment in children and ocular surface diseases. Moreover, the utilization of smartphone-based imaging modalities, coupled with AI algorithms, is able to provide timely, efficient and cost-effective screening for ocular pathologies. This modality can also facilitate patient self-monitoring, remote patient monitoring and enhancing accessibility to eye care services, particularly in underserved regions. Challenges involving data privacy, algorithm validation, regulatory frameworks and issues of trust are still need to be addressed. Furthermore, evaluation on real-world implementation is imperative as well, and real-world prospective studies are currently lacking. Conclusions: Smartphone ocular imaging merged with AI enables earlier, precise diagnoses, personalized treatments, and enhanced service accessibility in eye care. Collaboration is crucial to navigate ethical and data security challenges while responsibly leveraging these innovations, promising a potential revolution in care access and global eye health equity.

PurposeThis study aimed to evaluate the optical coherence tomography angiography (OCTA) changes in subzones of peripapillary atrophy (PPA) among type 2 diabetic patients (T2DM) with or without diabetic retinopathy (DR) using well-designed deep learning models.MethodsA multi-task joint deep-learning model was trained and validated on 2,820 images to automate the determination and quantification of the microstructure and corresponding microcirculation of beta zone and gamma zone PPA. This model was then applied in the cross-sectional study encompassing 44 eyes affected by non-proliferative diabetic retinopathy (NPDR) and 46 eyes without DR (NDR). OCTA was utilized to image the peripapillary area in four layers: superficial capillary plexus (SCP), deep capillary plexus (DCP), choroidal capillary (CC) and middle-to-large choroidal vessel (MLCV).ResultsThe patients in both groups were matched for age, sex, BMI, and axial length. The width and area of the gamma zone were significantly smaller in NPDR group compared to the NDR group. Multiple linear regression analysis revealed a negative association between the diagnosis of DR and the width and area of the gamma zone. The gamma zone exhibited higher SCP, DCP and MLCV density than the beta zone, while the beta zone showed higher CC density than the gamma zone. In comparison to the NDR group, the MLCV density of gamma zone was significantly lower in NPDR group, and this density was positively correlated with the width and area of the gamma zone.DiscussionDR-induced peripapillary vascular changes primarily occur in gamma zone PPA. After eliminating the influence of axial length, our study demonstrated a negative correlation between DR and the gamma zone PPA. Longitudinal studies are required to further elucidate the role of the gamma zone in the development and progression of DR.

BackgroundUltra-wide-field (UWF) fundus photography represents an emerging retinal imaging technique offering a broader field of view, thus enhancing its utility in screening and diagnosing various eye diseases, notably diabetic retinopathy (DR). However, the application of computer-aided diagnosis for DR using UWF images confronts two major challenges. The first challenge arises from the limited availability of labeled UWF data, making it daunting to train diagnostic models due to the high cost associated with manual annotation of medical images. Secondly, existing models' performance requires enhancement due to the absence of prior knowledge to guide the learning process.PurposeBy leveraging extensively annotated datasets within the field, which encompass large-scale, high-quality color fundus image datasets annotated at either image-level or pixel-level, our objective is to transfer knowledge from these datasets to our target domain through unsupervised domain adaptation.MethodsOur approach presents a robust model for assessing the severity of diabetic retinopathy (DR) by leveraging unsupervised lesion-aware domain adaptation in ultra-wide-field (UWF) images. Furthermore, to harness the wealth of detailed annotations in publicly available color fundus image datasets, we integrate an adversarial lesion map generator. This generator supplements the grading model by incorporating auxiliary lesion information, drawing inspiration from the clinical methodology of evaluating DR severity by identifying and quantifying associated lesions.ResultsWe conducted both quantitative and qualitative evaluations of our proposed method. In particular, among the six representative DR grading methods, our approach achieved an accuracy (ACC) of 68.18% and a precision (pre) of 67.43%. Additionally, we conducted extensive experiments in ablation studies to validate the effectiveness of each component of our proposed method.ConclusionIn conclusion, our method not only improves the accuracy of DR grading, but also enhances the interpretability of the results, providing clinicians with a reliable DR grading scheme.

PurposeFast and automated reconstruction of retinal hyperreflective foci (HRF) is of great importance for many eye-related disease understanding. In this paper, we introduced a new automated framework, driven by recent advances in deep learning to automatically extract 12 three-dimensional parameters from the segmented hyperreflective foci in optical coherence tomography (OCT).MethodsUnlike traditional convolutional neural networks, which struggle with long-range feature correlations, we introduce a spatial and channel attention module within the bottleneck layer, integrated into the nnU-Net architecture. Spatial Attention Block aggregates features across spatial locations to capture related features, while Channel Attention Block heightens channel feature contrasts. The proposed model was trained and tested on 162 retinal OCT volumes of patients with diabetic macular edema (DME), yielding robust segmentation outcomes. We further investigate HRF’s potential as a biomarker of DME.ResultsResults unveil notable discrepancies in the amount and volume of HRF subtypes. In the whole retinal layer (WR), the mean distance from HRF to the retinal pigmented epithelium was significantly reduced after treatment. In WR, the improvement in central macular thickness resulting from intravitreal injection treatment was positively correlated with the mean distance from HRF subtypes to the fovea.ConclusionOur study demonstrates the applicability of OCT for automated quantification of retinal HRF in DME patients, offering an objective, quantitative approach for clinical and research applications.

Early computer-aided early diagnosis (CAD) based on retinal imaging is critical to the timely management and treatment planning of retina-related diseases. However, the inherent characteristics of retinal images and the complexity of their pathological patterns, such as low image contrast and different lesion sizes, restrict the performance of CAD systems. Recently, ultra-wide-field (UWF) retinal images have become a useful tool for disease detection due to the capability of capturing much broader view of retina (i.e., up to 200°), in comparison with the most commonly used retinal fundus images (45°). In this paper, we propose an attention-based multi-branch network for the diseases classification of four different subject groups. The proposed method consists of a multi-scale feature fusion module and a dual attention module. Specifically, small-scale lesions are identified using the features extracted from the multi-scale feature fusion module. To better explore the obtained features, the dual attention module with a global attention graph is incorporated to enable the network to recognize the salient objects of interest. Comprehensive validations on both private and public datasets were carried out to verify the effectiveness of the proposed model.

Abstract Background Surgical video phase recognition is an essential technique in computer-assisted surgical systems for monitoring surgical procedures, which can assist surgeons in standardizing procedures and enhancing postsurgical assessment and indexing. However, the high similarity between the phases and temporal variations of cataract videos still poses the greatest challenge for video phase recognition. Methods In this paper, we introduce a global–local multi-stage temporal convolutional network (GL-MSTCN) to explore the subtle differences between high similarity surgical phases and mitigate the temporal variations of surgical videos. The presented work consists of a triple-stream network (i.e., pupil stream, instrument stream, and video frame stream) and a multi-stage temporal convolutional network. The triple-stream network first detects the pupil and surgical instruments regions in the frame separately and then obtains the fine-grained semantic features of the video frames. The proposed multi-stage temporal convolutional network improves the surgical phase recognition performance by capturing longer time series features through dilated convolutional layers with varying receptive fields. Results Our method is thoroughly validated on the CSVideo dataset with 32 cataract surgery videos and the public Cataract101 dataset with 101 cataract surgery videos, outperforming state-of-the-art approaches with 95.8% and 96.5% accuracy, respectively. Conclusions The experimental results show that the use of global and local feature information can effectively enhance the model to explore fine-grained features and mitigate temporal and spatial variations, thus improving the surgical phase recognition performance of the proposed GL-MSTCN.

BackgroundThe automatic analysis of medical images has the potential improve diagnostic accuracy while reducing the strain on clinicians. Current methods analyzing 3D-like imaging data, such as computerized tomography imaging, often treat each image slice as individual slices. This may not be able to appropriately model the relationship between slices.MethodsOur proposed method utilizes a mixed-effects model within the deep learning framework to model the relationship between slices. We externally validated this method on a data set taken from a different country and compared our results against other proposed methods. We evaluated the discrimination, calibration, and clinical usefulness of our model using a range of measures. Finally, we carried out a sensitivity analysis to demonstrate our methods robustness to noise and missing data.ResultsIn the external geographic validation set our model showed excellent performance with an AUROC of 0.930 (95%CI: 0.914, 0.947), with a sensitivity and specificity, PPV, and NPV of 0.778 (0.720, 0.828), 0.882 (0.853, 0.908), 0.744 (0.686, 0.797), and 0.900 (0.872, 0.924) at the 0.5 probability cut-off point. Our model also maintained good calibration in the external validation dataset, while other methods showed poor calibration.ConclusionDeep learning can reduce stress on healthcare systems by automatically screening CT imaging for COVID-19. Our method showed improved generalizability in external validation compared to previous published methods. However, deep learning models must be robustly assessed using various performance measures and externally validated in each setting. In addition, best practice guidelines for developing and reporting predictive models are vital for the safe adoption of such models.

Purpose: To develop a visual function-based deep learning system (DLS) using fundus images to screen for visually impaired cataracts.Materials and methods: A total of 8,395 fundus images (5,245 subjects) with corresponding visual function parameters collected from three clinical centers were used to develop and evaluate a DLS for classifying non-cataracts, mild cataracts, and visually impaired cataracts. Three deep learning algorithms (DenseNet121, Inception V3, and ResNet50) were leveraged to train models to obtain the best one for the system. The performance of the system was evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity.Results: The AUC of the best algorithm (DenseNet121) on the internal test dataset and the two external test datasets were 0.998 (95% CI, 0.996–0.999) to 0.999 (95% CI, 0.998–1.000),0.938 (95% CI, 0.924–0.951) to 0.966 (95% CI, 0.946–0.983) and 0.937 (95% CI, 0.918–0.953) to 0.977 (95% CI, 0.962–0.989), respectively. In the comparison between the system and cataract specialists, better performance was observed in the system for detecting visually impaired cataracts (p < 0.05).Conclusion: Our study shows the potential of a function-focused screening tool to identify visually impaired cataracts from fundus images, enabling timely patient referral to tertiary eye hospitals.

IntroductionNeuromyelitis optica spectrum disorders (NMOSD) are autoimmune central nervous system diseases characterized by the immune system's abnormal attack on glial cells and neurons. Optic neuritis (ON) is one of the indicators of NMOSD, often starting unilaterally and potentially affecting both eyes later in the disease progression, leading to visual impairment. Optical coherence tomography angiography (OCTA) has the potential to aid in the early diagnosis of NMOSD by examining ophthalmic imaging and may offer a window for disease prevention.MethodsIn this study, we collected OCTA images from 22 NMOSD patients (44 images) and 25 healthy individuals (50 images) to investigate retinal microvascular changes in NMOSD. We employed effective retinal microvascular segmentation and foveal avascular zone (FAZ) segmentation techniques to extract key OCTA structures for biomarker analysis. A total of 12 microvascular features were extracted using specifically designed methods based on the segmentation results. The OCTA images of NMOSD patients were classified into two groups: optic neuritis (ON) and non-optic neuritis (non-ON). Each group was compared separately with a healthy control (HC) group.ResultsStatistical analysis revealed that the non-ON group displayed shape changes in the deep layer of the retina, specifically in the FAZ. However, there were no significant microvascular differences between the non-ON group and the HC group. In contrast, the ON group exhibited microvascular degeneration in both superficial and deep retinal layers. Sub-regional analysis revealed that pathological variations predominantly occurred on the side affected by ON, particularly within the internal ring near the FAZ.DiscussionThe findings of this study highlight the potential of OCTA in evaluating retinal microvascular changes associated with NMOSD. The shape alterations observed in the FAZ of the non-ON group suggest localized vascular abnormalities. In the ON group, microvascular degeneration in both superficial and deep retinal layers indicates more extensive vascular damage. Sub-regional analysis further emphasizes the impact of optic neuritis on pathological variations, particularly near the FAZ's internal ring.ConclusionThis study provides insights into the retinal microvascular changes associated with NMOSD using OCTA imaging. The identified biomarkers and observed alterations may contribute to the early diagnosis and monitoring of NMOSD, potentially offering a time window for intervention and prevention of disease progression.

Background: Ultra-Wide-Field (UWF) fundus imaging is an essential diagnostic tool for identifying ophthalmologic diseases, as it captures detailed retinal structures within a wider field of view (FOV). However, the presence of eyelashes along the edge of the eyelids can cast shadows and obscure the view of fundus imaging, which hinders reliable interpretation and subsequent screening of fundus diseases. Despite its limitations, there are currently no effective methods or datasets available for removing eyelash artifacts from UWF fundus images. This research aims to develop an effective approach for eyelash artifact removal and thus improve the visual quality of UWF fundus images for accurate analysis and diagnosis.Methods: To address this issue, we first constructed two UWF fundus datasets: the paired synthetic eyelashes (PSE) dataset and the unpaired real eyelashes (uPRE) dataset. Then we proposed a deep learning architecture called Joint Conditional Generative Adversarial Networks (JcGAN) to remove eyelash artifacts from UWF fundus images. JcGAN employs a shared generator with two discriminators for joint learning of both real and synthetic eyelash artifacts. Furthermore, we designed a background refinement module that refines background information and is trained with the generator in an end-to-end manner.Results: Experimental results on both PSE and uPRE datasets demonstrate the superiority of the proposed JcGAN over several state-of-the-art deep learning approaches. Compared with the best existing method, JcGAN improves PSNR and SSIM by 4.82% and 0.23%, respectively. In addition, we also verified that eyelash artifact removal via JcGAN could significantly improve vessel segmentation performance in UWF fundus images. Assessment via vessel segmentation illustrates that the sensitivity, Dice coefficient and area under curve (AUC) of ResU-Net have respectively increased by 3.64%, 1.54%, and 1.43% after eyelash artifact removal using JcGAN.Conclusion: The proposed JcGAN effectively removes eyelash artifacts in UWF images, resulting in improved visibility of retinal vessels. Our method can facilitate better processing and analysis of retinal vessels and has the potential to improve diagnostic outcomes.

Left atrial appendage (LAA) is a leading cause of atrial fibrillation and thrombosis in cardiovascular disease. Clinicians can rely on LAA occlusion (LAAO) to effectively prevent and treat ischaemic strokes attributed to the LAA. The correct selection of the LAAO is one of the most critical stages in the successful surgical process, which relies on the quantification of the anatomical structure of the LAA for successful intervention in LAAO. In this paper, we propose an adversarial-based latent space alignment framework for LAA segmentation in transesophageal echocardiography (TEE) images by introducing prior knowledge from the label. The proposed method consists of an LAA segmentation network, a label reconstruction network, and a latent space alignment loss. To be specific, we first employ ConvNeXt as the backbone of the segmentation and reconstruction network to enhance the feature extraction capability of the encoder. The label reconstruction network then encodes the prior shape features from the LAA labels to the latent space. The latent space alignment loss consists of the adversarial-based alignment and the contrast learning losses. It can motivate the segmentation network to learn the prior shape features of the labels, thus improving the accuracy of LAA edge segmentation. The proposed method was evaluated on a TEE dataset including 1,783 images and the experimental results showed that the proposed method outperformed other state-of-the-art LAA segmentation methods with Dice coefficient, AUC, ACC, G-mean, and Kappa of 0.831, 0.917, 0.989, 0.911, and 0.825, respectively.

Machine learning has been increasingly utilized in the field of protein engineering, and research directed at predicting the effects of protein mutations has attracted increasing attention. Among them, so far, the best results have been achieved by related methods based on protein language models, which are trained on a large number of unlabeled protein sequences to capture the generally hidden evolutionary rules in protein sequences, and are therefore able to predict their fitness from protein sequences. Although numerous similar models and methods have been successfully employed in practical protein engineering processes, the majority of the studies have been limited to how to construct more complex language models to capture richer protein sequence feature information and utilize this feature information for unsupervised protein fitness prediction. There remains considerable untapped potential in these developed models, such as whether the prediction performance can be further improved by integrating different models to further improve the accuracy of prediction. Furthermore, how to utilize large-scale models for prediction methods of mutational effects on quantifiable properties of proteins due to the nonlinear relationship between protein fitness and the quantification of specific functionalities has yet to be explored thoroughly. In this study, we propose an ensemble learning approach for predicting mutational effects of proteins integrating protein sequence features extracted from multiple large protein language models, as well as evolutionarily coupled features extracted in homologous sequences, while comparing the differences between linear regression and deep learning models in mapping these features to quantifiable functional changes. We tested our approach on a dataset of 17 protein deep mutation scans and indicated that the integrated approach together with linear regression enables the models to have higher prediction accuracy and generalization. Moreover, we further illustrated the reliability of the integrated approach by exploring the differences in the predictive performance of the models across species and protein sequence lengths, as well as by visualizing clustering of ensemble and non-ensemble features.

IntroductionOptical Coherence Tomography Angiography (OCTA) is a new non-invasive imaging modality that gains increasing popularity for the observation of the microvasculatures in the retina and the conjunctiva, assisting clinical diagnosis and treatment planning. However, poor imaging quality, such as stripe artifacts and low contrast, is common in the acquired OCTA and in particular Anterior Segment OCTA (AS-OCTA) due to eye microtremor and poor illumination conditions. These issues lead to incomplete vasculature maps that in turn makes it hard to make accurate interpretation and subsequent diagnosis.MethodsIn this work, we propose a two-stage framework that comprises a de-striping stage and a re-enhancing stage, with aims to remove stripe noise and to enhance blood vessel structure from the background. We introduce a new de-striping objective function in a Stripe Removal Net (SR-Net) to suppress the stripe noise in the original image. The vasculatures in acquired AS-OCTA images usually exhibit poor contrast, so we use a Perceptual Structure Generative Adversarial Network (PS-GAN) to enhance the de-striped AS-OCTA image in the re-enhancing stage, which combined cyclic perceptual loss with structure loss to achieve further image quality improvement.Results and discussionTo evaluate the effectiveness of the proposed method, we apply the proposed framework to two synthetic OCTA datasets and a real AS-OCTA dataset. Our results show that the proposed framework yields a promising enhancement performance, which enables both conventional and deep learning-based vessel segmentation methods to produce improved results after enhancement of both retina and AS-OCTA modalities.

Purpose: Recent reports suggest retinal microvasculature mirror cerebral microcirculation. Using optical coherence tomography angiography (OCTA), we investigated the retinal microvasculature differences between ischemic stroke patients with large artery atherosclerosis (LAA) and small artery disease (SAD).Methods: All patients underwent MR imaging and were classified as SAD and LAA; LAA was subdivided into anterior LAS and posterior LAS depending on the location. Swept-source OCTA (SS-OCTA) was used to image and segment the retina into the superficial vascular complex (SVC) and deep vascular complex (DVC) in a 6 × 6 mm area around the fovea. A deep learning algorithm was used to assess the vessel area density (VAD, %) in the retinal microvasculature.Results: Fifty-eight (mean age = 60.26 ± 10.88 years; 81.03% males) were LAA while 64 (mean age = 55.58 ± 10.34 years; 85.94% males) were SAD. LAS patients had significantly reduced VAD in the DVC (P = 0.022) compared to SAD patients; the VAD in the SVC did not show any significant difference between the two groups (P = 0.580). Anterior LAA ischemic stroke showed significantly lower VAD (P = 0.002) in the SVC compared with posterior LAS patients. There was no significant difference in the DVC between the two groups (P = 0.376).Conclusions: We found LAA patients had significantly reduced DVC density compared with SAD; we also showed anterior LAA patients had significantly reduced SVC density compared with posterior LAA. These findings suggest retinal imaging has the potential to be used to detect microvasculature changes in subtypes of ischemic stroke.

Morphological changes of the choroid have been proved to be associated with the occurrence and pathological mechanism of many ophthalmic diseases. Optical Coherence Tomography (OCT) is a non-invasive technique for imaging of ocular biological tissues, that can reveal the structure of the retinal and choroidal layers in micron-scale resolution. However, unlike the retinal layer, the interface between the choroidal layer and the sclera is ambiguous in OCT, which makes it difficult for ophthalmologists to identify with certainty. In this paper, we propose a novel boundary-enhanced encoder-decoder architecture for choroid segmentation in retinal OCT images, in which a Boundary Enhancement Module (BEM) forms the backbone of each encoder-decoder layer. The BEM consists of three parallel branches: 1) a Feature Extraction Branch (FEB) to obtain feature maps with different receptive fields; 2) a Channel Enhancement Branch (CEB) to extract the boundary information of different channels; and 3) a Boundary Activation Branch (BAB) to enhance the boundary information via a novel activation function. In addition, in order to incorporate expert knowledge into the segmentation network, soft key point maps are generated on the choroidal boundary, and are combined with the predicted images to facilitate precise choroidal boundary segmentation. In order to validate the effectiveness and superiority of the proposed method, both qualitative and quantitative evaluations are employed on three retinal OCT datasets for choroid segmentation. The experimental results demonstrate that the proposed method yields better choroid segmentation performance than other deep learning approaches. Moreover, both 2D and 3D features are extracted for statistical analysis from normal and highly myopic subjects based on the choroid segmentation results, which is helpful in revealing the pathology of high myopia. Code is available at https://github.com/iMED-Lab/Choroid-segmentation.

Abstract Background Chest X-rays are the most commonly available and affordable radiological examination for screening thoracic diseases. According to the domain knowledge of screening chest X-rays, the pathological information usually lay on the lung and heart regions. However, it is costly to acquire region-level annotation in practice, and model training mainly relies on image-level class labels in a weakly supervised manner, which is highly challenging for computer-aided chest X-ray screening. To address this issue, some methods have been proposed recently to identify local regions containing pathological information, which is vital for thoracic disease classification. Inspired by this, we propose a novel deep learning framework to explore discriminative information from lung and heart regions. Result We design a feature extractor equipped with a multi-scale attention module to learn global attention maps from global images. To exploit disease-specific cues effectively, we locate lung and heart regions containing pathological information by a well-trained pixel-wise segmentation model to generate binarization masks. By introducing element-wise logical AND operator on the learned global attention maps and the binarization masks, we obtain local attention maps in which pixels are are 1 for lung and heart region and 0 for other regions. By zeroing features of non-lung and heart regions in attention maps, we can effectively exploit their disease-specific cues in lung and heart regions. Compared to existing methods fusing global and local features, we adopt feature weighting to avoid weakening visual cues unique to lung and heart regions. Our method with pixel-wise segmentation can help overcome the deviation of locating local regions. Evaluated by the benchmark split on the publicly available chest X-ray14 dataset, the comprehensive experiments show that our method achieves superior performance compared to the state-of-the-art methods. Conclusion We propose a novel deep framework for the multi-label classification of thoracic diseases in chest X-ray images. The proposed network aims to effectively exploit pathological regions containing the main cues for chest X-ray screening. Our proposed network has been used in clinic screening to assist the radiologists. Chest X-ray accounts for a significant proportion of radiological examinations. It is valuable to explore more methods for improving performance.

BackgroundThe retina and brain share a similar embryologic origin, blood barriers, and microvasculature features. Thus, retinal imaging has been of interest in the aging population to help in the early detection of brain disorders. Imaging evaluation of brain frailty, including brain atrophy and markers of cerebral small vessel disease (CSVD), could reflect brain health in normal aging, but is costly and time-consuming. In this study, we aimed to evaluate the retinal microvasculature and its association with radiological indicators of brain frailty in normal aging adults.MethodsSwept-source optical coherence tomography angiography (SS-OCTA) and 3T-MRI brain scanning were performed on normal aging adults (aged ≥ 50 years). Using a deep learning algorithm, microvascular tortuosity (VT) and fractal dimension parameter (Dbox) were used to evaluate the superficial vascular complex (SVC) and deep vascular complex (DVC) of the retina. MRI markers of brain frailty include brain volumetric measures and CSVD markers that were assessed.ResultsOf the 139 normal aging individuals included, the mean age was 59.43 ± 7.31 years, and 64.0% (n = 89) of the participants were females. After adjustment of age, sex, and vascular risk factors, Dbox in the DVC showed a significant association with the presence of lacunes (β = 0.58, p = 0.007), while VT in the SVC significantly correlated with the score of cerebral deep white matter hyperintensity (β = 0.31, p = 0.027). No correlations were found between brain volumes and retinal microvasculature changes (P > 0.05).ConclusionOur report suggests that imaging of the retinal microvasculature may give clues to brain frailty in the aging population.

Dementia is an incurable neurodegenerative disease primarily affecting the older population, for which the World Health Organisation has set to promoting early diagnosis and timely management as one of the primary goals for dementia care. While a range of popular machine learning algorithms and their variants have been applied for dementia diagnosis, fuzzy systems, which have been known effective in dealing with uncertainty and offer to explicitly reason how a diagnosis can be inferred, sporadically appear in recent literature. Given the advantages of a fuzzy rule-based model, which could potentially result in a clinical decision support system that offers understandable rules and a transparent inference process to support dementia diagnosis, this paper proposes a novel fuzzy inference system by adapting the concept of dominant sets that arise from the study of graph theory. A peeling-off strategy is used to iteratively extract from the constructed edge-weighted graph a collection of dominant sets. Each dominant set is further converted into a parameterized fuzzy rule, which is finally optimized in a supervised adaptive network-based fuzzy inference framework. An illustrative example is provided that demonstrates the interpretable rules and the transparent reasoning process of reaching a decision. Further systematic experiments conducted on data from the Open Access Series of Imaging Studies (OASIS) repository, also validate its superior performance over alternative methods.

The tortuosity assessment of vessels and nerve fibres in ophthalmic images has drawn substantial attention, for its potentials in assisting various medical diagnoses. Numerous morphological tortuosity measures have been leveraged to quantify tortuosity from various perspectives, which warrants the simultaneous use of multiple measures with an aim to produce a robust and accurate assessment. This paper proposes an approach for the automated assessment of curvilinear structures' tortuosity. Starting with the generation of clusters of tortuosity density for each individual measure, labelled fuzzy sets are then extracted that enhances the readability of subsequent operations. Finally, results from multiple measures are aggregated by a nearest neighbour guided approach where weights are generated in a data-driven manner to explain the derived aggregations. Experimental results on both ophthalmic vessel and nerve images demonstrate the superior performance of the proposed method over those where measurements are used independently or aggregated using conventional averaging operators.

ObjectiveTo develop an accurate and rapid computed tomography (CT)-based interpretable AI system for the diagnosis of lung diseases.BackgroundMost existing AI systems only focus on viral pneumonia (e.g., COVID-19), specifically, ignoring other similar lung diseases: e.g., bacterial pneumonia (BP), which should also be detected during CT screening. In this paper, we propose a unified sequence-based pneumonia classification network, called SLP-Net, which utilizes consecutiveness information for the differential diagnosis of viral pneumonia (VP), BP, and normal control cases from chest CT volumes.MethodsConsidering consecutive images of a CT volume as a time sequence input, compared with previous 2D slice-based or 3D volume-based methods, our SLP-Net can effectively use the spatial information and does not need a large amount of training data to avoid overfitting. Specifically, sequential convolutional neural networks (CNNs) with multi-scale receptive fields are first utilized to extract a set of higher-level representations, which are then fed into a convolutional long short-term memory (ConvLSTM) module to construct axial dimensional feature maps. A novel adaptive-weighted cross-entropy loss (ACE) is introduced to optimize the output of the SLP-Net with a view to ensuring that as many valid features from the previous images as possible are encoded into the later CT image. In addition, we employ sequence attention maps for auxiliary classification to enhance the confidence level of the results and produce a case-level prediction.ResultsFor evaluation, we constructed a dataset of 258 chest CT volumes with 153 VP, 42 BP, and 63 normal control cases, for a total of 43,421 slices. We implemented a comprehensive comparison between our SLP-Net and several state-of-the-art methods across the dataset. Our proposed method obtained significant performance without a large amount of data, outperformed other slice-based and volume-based approaches. The superior evaluation performance achieved in the classification experiments demonstrated the ability of our model in the differential diagnosis of VP, BP and normal cases.

Trigeminal neuralgia caused by paroxysmal and severe pain in the distribution of the trigeminal nerve is a rare chronic pain disorder. It is generally accepted that compression of the trigeminal root entry zone by vascular structures is the major cause of primary trigeminal neuralgia, and vascular decompression is the prior choice in neurosurgical treatment. Therefore, accurate preoperative modeling/segmentation/visualization of trigeminal nerve and its surrounding cerebrovascular is important to surgical planning. In this paper, we propose an automated method to segment trigeminal nerve and its surrounding cerebrovascular in the root entry zone, and to further reconstruct and visual these anatomical structures in three-dimensional (3D) Magnetic Resonance Angiography (MRA). The proposed method contains a two-stage neural network. Firstly, a preliminary confidence map of different anatomical structures is produced by a coarse segmentation stage. Secondly, a refinement segmentation stage is proposed to refine and optimize the coarse segmentation map. To model the spatial and morphological relationship between trigeminal nerve and cerebrovascular structures, the proposed network detects the trigeminal nerve, cerebrovasculature, and brainstem simultaneously. The method has been evaluated on a dataset including 50 MRA volumes, and the experimental results show the state-of-the-art performance of the proposed method with an average Dice similarity coefficient, Hausdorff distance, and average surface distance error of 0.8645, 0.2414, and 0.4296 on multi-tissue segmentation, respectively.

Objective To evaluate the accuracy of convolutional neural networks technique (CNN) in detecting keratoconus using colour-coded corneal maps obtained by a Scheimpflug camera.Design Multicentre retrospective study.Methods and analysis We included the images of keratoconic and healthy volunteers’ eyes provided by three centres: Royal Liverpool University Hospital (Liverpool, UK), Sedaghat Eye Clinic (Mashhad, Iran) and The New Zealand National Eye Center (New Zealand). Corneal tomography scans were used to train and test CNN models, which included healthy controls. Keratoconic scans were classified according to the Amsler-Krumeich classification. Keratoconic scans from Iran were used as an independent testing set. Four maps were considered for each scan: axial map, anterior and posterior elevation map, and pachymetry map.Results A CNN model detected keratoconus versus health eyes with an accuracy of 0.9785 on the testing set, considering all four maps concatenated. Considering each map independently, the accuracy was 0.9283 for axial map, 0.9642 for thickness map, 0.9642 for the front elevation map and 0.9749 for the back elevation map. The accuracy of models in recognising between healthy controls and stage 1 was 0.90, between stages 1 and 2 was 0.9032, and between stages 2 and 3 was 0.8537 using the concatenated map.Conclusion CNN provides excellent detection performance for keratoconus and accurately grades different severities of disease using the colour-coded maps obtained by the Scheimpflug camera. CNN has the potential to be further developed, validated and adopted for screening and management of keratoconus.

Purpose: To investigate the thickness changes of outer retinal layers in subjects with white matter hyperintensities (WMH) and Parkinson's Disease (PD).Methods: 56 eyes from 31 patients with WMH, 11 eyes from 6 PD patients, and 58 eyes from 32 healthy controls (HC) were enrolled in this study. A macular-centered scan was conducted on each participant using a spectral-domain optical coherence tomography (SD-OCT) device. After speckle noise reduction, a state-of-the-art deep learning method (i.e., a context encoder network) was employed to segment the outer retinal layers from OCT B-scans. Thickness quantification of the outer retinal layers was conducted on the basis of the segmentation results.Results: WMH patients had significantly thinner Henle fiber layers, outer nuclear layers (HFL+ONL) and photoreceptor outer segments (OS) than HC (p = 0.031, and p = 0.005), while PD patients showed a significant increase of mean thickness in the interdigitation zone and the retinal pigment epithelium/Bruch complex (IZ+RPE) (19.619 ± 4.626) compared to HC (17.434 ± 1.664). There were no significant differences in the thickness of the outer plexiform layer (OPL), the myoid and ellipsoid zone (MEZ), and the IZ+RPE layer between WMH and HC subjects. Similarly, there were also no obvious differences in the thickness of the OPL, HFL+ONL, MEZ and the OS layer between PD and HC subjects.Conclusion: Thickness changes in HFL+ONL, OS, and IZ+RPE layers may correlate with brain-related diseases such as WMH and PD. Further longitudinal study is needed to confirm HFL+ONL/OS/IZ+RPE layer thickness as potential biomarkers for detecting certain brain-related diseases.

Purpose: We examined the macular microvascular changes of the macula in neuromyelitis optica spectrum disorder (NMOSD) patients and its association with their disability and other clinical variables.Methods: Thirty-four NMOSD (13 patients without optic neuritis, NMOSD-NON, and 21 patients with a history of optic neuritis, NMOSD-ON) and 44 healthy controls (HCs) were included in the study. Optical coherence tomographic angiography (OCTA) was used to image the superficial (SCP), deep (DCP), and whole capillary plexus (WCP) in a 2.5-mm-diameter concentric circle [excluding the foveal avascular zone (FAZ)]. An algorithm (Dbox) was used to quantify the complexity of the three capillary layers by fractal analysis. We also evaluated the expanded disability scale status (EDSS).Results: Dbox values were significantly reduced in SCP (p < 0.001), DCP (p < 0.001), and WCP (p = 0.003) of NMOSD when compared with HCs. Dbox values were significantly reduced in NMOSD eyes with optic neuritis when compared with healthy controls (p < 0.001) and eyes without optic neuritis (p = 0.004) in the SCP. In the DCP, eyes with optic neuritis showed significantly reduced Dbox values when compared with eyes without optic neuritis (p = 0.016) and healthy controls (p < 0.001); eyes without optic neuritis showed significantly reduced Dbox values (p = 0.007) in the DCP when compared with healthy controls. A significant negative correlation (Rho = −0.475, p = 0.005) was shown between the superficial macula Dbox values and the EDSS in NMOSD patients. Additionally, a negative correlation (Rho = −0.715, p = 0.006) was seen in the superficial Dbox values in [e]eyes without optic neuritis and EDSS.Conclusions: Macular microvascular damage in the superficial plexus is associated with disability in NMOSD. Macular microvascular alterations arise independently of the occurrence of ON in NMOSD.

The tortuosity changes of curvilinear anatomical organs such as nerve fibers or vessels have a close relationship with a number of diseases. Therefore, the automatic estimation and representation of the tortuosity is desired in medical image for such organs. In this paper, an automated framework for tortuosity estimation is proposed for corneal nerve and retinal vessel images. First, the weighted local phase tensor-based enhancement method is employed and the curvilinear structure is extracted from raw image. For each curvilinear structure with a different position and orientation, the curvature is measured by the exponential curvature estimation in the 3D space. Then, the tortuosity of an image is calculated as the weighted average of all the curvilinear structures. Our proposed framework has been evaluated on two corneal nerve fiber datasets and one retinal vessel dataset. Experiments on three curvilinear organ datasets demonstrate that our proposed tortuosity estimation method achieves a promising performance compared with other state-of-the-art methods in terms of accuracy and generality. In our nerve fiber dataset, the method achieved overall accuray of 0.820, and 0.734, 0.881 for sensitivity and specificity, respectively. The proposed method also achieved Spearman correlation scores 0.945 and 0.868 correlated with tortuosity grading ground truth for arteries and veins in the retinal vessel dataset. Furthermore, the manual labeled 403 corneal nerve fiber images with different levels of tortuosity, and all of them are also released for public access for further research.

Objective. Deep brain stimulation (DBS) provides dramatic tremor relief in patients with severe essential tremor (ET). Typically, the VIM nucleus is the most effective brain area to target for high-frequency electrical stimulation in these patients. Correlation analysis between electrical local field potential (LFP) recordings from the thalamic DBS leads and electrical muscle activity from the contralateral tremulous limb has become an attractive practical tool to interpret the LFPs and their association with the tremulous clinical manifestations. Although functional connectivity analysis between brain electrical recordings and electromyographic (EMG) signals from the tremor has been of interest to an increasing number of engineering researchers, there is no well-accepted tailored framework to consistently characterise the association between thalamic electrical recordings and the tremorogenic EMG activity. Methods. This paper proposes a novel framework to address this challenge, including an estimation of the interaction strength using wavelet cross-spectrum and phase lag index while demonstrating the statistical significance of the findings. Results. Consistent results were estimated for single and multiple trials of consecutive or partially overlapping epochs of data. The latter approach reveals a substantial increase on the range of statistically significant dynamic low-frequency interrelationships while decreasing the dynamic range of high-frequency interactions. Conclusion. Results from both simulation and real data demonstrate the feasibility and robustness of the proposed framework. Significance. This study offers the proof of principle required to implement this methodology to uncover VIM thalamic LFP-EMG interactions for (i) better understanding of the pathophysiology of tremor; (ii) objective selection of the DBS electrode contacts with the highest strength of association with the tremorogenic EMG, a particularly useful feature for the implementation of novel multicontact directional leads in clinical practice; and (iii) future research on DBS closed-loop devices.

Studying complex dynamic systems is usually very challenging due to limited prior knowledge and high complexity of relationships between interconnected components. Current methods either are like a “black box” that is difficult to understand and relate back to the underlying system or have limited universality and applicability due to too many assumptions. This paper proposes a time-varying Nonlinear Finite Impulse Response model to estimate the multiple features of correlation among measurements including direction, strength, significance, latency, correlation type, and nonlinearity. The dynamic behaviours of correlation are tracked through a sliding window approach based on the Blackman window rather than the simple truncation by a Rectangular window. This method is particularly useful for a system that has very little prior knowledge and the interaction between measurements is nonlinear, time-varying, rapidly changing, or of short duration. Simulation results suggest that the proposed tracking approach significantly reduces the sensitivity of correlation estimation against the window size. Such a method will improve the applicability and robustness of correlation analysis for complex systems. A real application to environmental changing data demonstrates the potential of the proposed method by revealing and characterising hidden information contained within measurements, which is usually “invisible” for conventional methods.

The analysis of retinal blood vessels present in fundus images, and the addressing of problems such as blood clot location, is important to undertake accurate and appropriate treatment of the vessels. Such tasks are hampered by the challenge of accurately tracing back problems along vessels to their source. This is due to the unresolved issue of distinguishing automatically between vessel bifurcations and vessel crossings in colour fundus photographs. In this paper, we present a new technique for addressing this problem using a convolutional neural network approach to firstly locate vessel bifurcations and crossings and then to classifying them as either bifurcations or crossings. Our method achieves high accuracies for junction detection and classification on the DRIVE dataset and we show further validation on an unseen dataset from which no data has been used for training. Combined with work in automated segmentation, this method has the potential to facilitate: reconstruction of vessel topography, classification of veins and arteries and automated localisation of blood clots and other disease symptoms leading to improved management of eye disease.

Our application concerns the automated detection of vessels in retinal images to improve understanding of the disease mechanism, diagnosis and treatment of retinal and a number of systemic diseases. We propose a new framework for segmenting retinal vasculatures with much improved accuracy and efficiency. The proposed framework consists of three technical components: Retinex-based image inhomogeneity correction, local phase-based vessel enhancement and graph cut-based active contour segmentation. These procedures are applied in the following order. Underpinned by the Retinex theory, the inhomogeneity correction step aims to address challenges presented by the image intensity inhomogeneities, and the relatively low contrast of thin vessels compared to the background. The local phase enhancement technique is employed to enhance vessels for its superiority in preserving the vessel edges. The graph cut-based active contour method is used for its efficiency and effectiveness in segmenting the vessels from the enhanced images using the local phase filter. We have demonstrated its performance by applying it to four public retinal image datasets (3 datasets of color fundus photography and 1 of fluorescein angiography). Statistical analysis demonstrates that each component of the framework can provide the level of performance expected. The proposed framework is compared with widely used unsupervised and supervised methods, showing that the overall framework outperforms its competitors. For example, the achieved sensitivity (0:744), specificity (0:978) and accuracy (0:953) for the DRIVE dataset are very close to those of the manual annotations obtained by the second observer.