Search

Compared with hardware-dependent methods, light field (LF) reconstruction algorithms enable requiring the densely sampled light field (DSLF) more economical and convenient. Most of the current LF reconstruction methods either directly apply multiple convolutional layers to the input views to generate DSLF or warp the input views to novel viewpoints based on the estimated depth and then blend to obtain the DSLF. These two types of methods produce either blurry results in texture areas or distortions near the boundaries of depth discontinuities. In this paper, we propose an end-to-end learning-based approach, which combines the characteristics of the above two methods from a parallel and complementary perspective, to reconstruct a high-quality light field. Our method consists of three sub-networks, i.e., two parallel sub-networks termed as ASR-Net and Warp-Net, and the refine sub-network termed as Refine-Net. ASR-Net directly learns an intermediate DSLF through deep convolutional layers, and Warp-Net warps the input views based on the estimated depth to obtain other intermediate DSLFs that maintain high-frequency texture information. These intermediate DSLFs are adaptively fused through the learned attention maps, and a fusion DSLF with the advantages of two types of intermediate DSLFs is obtained. Finally, the residual map learned from the intermediate DSLFs is added to the fusion DSLF to get a better one. Comprehensive experiments demonstrate the superiority of the proposed method on several LF datasets compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

With the continuous progress of remote sensing image object detection tasks in recent years, researchers in this field have gradually shifted the focus of their research from horizontal object detection to the study of object detection in arbitrary directions. It is worth noting that some properties are different from horizontal object detection during oriented object detection that researchers have yet to notice much. This article presents the design of a straightforward and efficient arbitrary-oriented detection system, leveraging the inherent properties of the orientation task, including the rotation angle and box aspect ratio. In the detection of low aspect ratio objects, the angle is of little importance to the orientation bounding box, and it is even difficult to define the angle information in extreme categories. Conversely, in the detection of objects with high aspect ratios, the angle information plays a crucial role and can have a decisive impact on the quality of the detection results. By exploiting the aspect ratio of different targets, this letter proposes a ratio-balanced angle loss that allows the model to make a better trade-off between low-aspect ratio objects and high-aspect ratio objects. The rotation angle of each oriented object, which we naturally embed into a two-dimensional Euclidean space for regression, thus avoids an overly redundant design and preserving the topological properties of the circular space. The performance of the UCAS-AOD, HRSC2016, and DLR-3K datasets show that the proposed model in this paper achieves a leading level in terms of both accuracy and speed. [ABSTRACT FROM AUTHOR]

High angular resolution light field (LF) enables exciting applications such as depth estimation, virtual reality, and augmented reality. Although many light field angular super-resolution methods have been proposed, the reconstruction problem of LF with a wide-baseline is far from being solved. In this paper, we propose an end-to-end learning-based approach to achieve angular super-resolution of the light field with a wide-baseline. Our model consists of three components. We first train a convolutional neural network to predict the depth map for each sub-aperture view. Then the estimated depth maps are used to warp the input views. In the final component, we first use a convolutional neural network to fuse the initial warped light fields, and then we propose an edge-aware inpainting network to modify the inaccurate pixels in the near-edge regions. Accordingly, we design an EdgePyramid structure that contains multi-scale edges to perform the inpainting of near-edge pixels. Moreover, we introduce a novel loss function to reduce the artifacts and further estimate the similarity in near-edge regions. Experimental results on various light field datasets including large-baseline light field images show that our method outperforms the state-of-the-art light field angular super-resolution methods, especially in the terms of visual performance near edges. [ABSTRACT FROM AUTHOR]

Motivation Phosphorylation is one of the most studied post-translational modifications, which plays a pivotal role in various cellular processes. Recently, deep learning methods have achieved great success in prediction of phosphorylation sites, but most of them are based on convolutional neural network that may not capture enough information about long-range dependencies between residues in a protein sequence. In addition, existing deep learning methods only make use of sequence information for predicting phosphorylation sites, and it is highly desirable to develop a deep learning architecture that can combine heterogeneous sequence and protein–protein interaction (PPI) information for more accurate phosphorylation site prediction. Results We present a novel integrated deep neural network named PhosIDN, for phosphorylation site prediction by extracting and combining sequence and PPI information. In PhosIDN, a sequence feature encoding sub-network is proposed to capture not only local patterns but also long-range dependencies from protein sequences. Meanwhile, useful PPI features are also extracted in PhosIDN by a PPI feature encoding sub-network adopting a multi-layer deep neural network. Moreover, to effectively combine sequence and PPI information, a heterogeneous feature combination sub-network is introduced to fully exploit the complex associations between sequence and PPI features, and their combined features are used for final prediction. Comprehensive experiment results demonstrate that the proposed PhosIDN significantly improves the prediction performance of phosphorylation sites and compares favorably with existing general and kinase-specific phosphorylation site prediction methods. Availability and implementation PhosIDN is freely available at https://github.com/ustchangyuanyang/PhosIDN. Supplementary information Supplementary data are available at Bioinformatics online. [ABSTRACT FROM AUTHOR]

Diabetic retinopathy (DR) is the leading cause of blindness among people of working age. Fundus lesions are clinical signs of DR, and their recognition and delineation are important for early screening, grading, and monitoring of the disease. We propose in this work a fully automatic deep convolutional neural network method for simultaneous segmentation of four different types of DR-related fundus lesions. To exploit multi-scale image information, we propose a collaborative architecture that comprises a contextual branch and a local branch. An attention mechanism is designed to fuse feature maps from all decoding layers in order to effectively and fully combine informative features from the two branches. Moreover, an auxiliary classification task with a novel supervision scheme is introduced to reduce model overfitting and further improve the accuracy of lesion segmentation. Extensive experiments are conducted using three public fundus datasets, and our method produces a mean AUC value of 0.677, 0.629, and 0.581 on them respectively. The results demonstrate the advantages of the proposed method, outperforming alternative strategies and other state-of-the-art methods in the literature. [ABSTRACT FROM AUTHOR]

A large-scale and well-annotated dataset is a key factor for the success of deep learning in medical image analysis. However, assembling such large annotations is very challenging, especially for histopathological images with unique characteristics (e.g., gigapixel image size, multiple cancer types, and wide staining variations). To alleviate this issue, self-supervised learning (SSL) could be a promising solution that relies only on unlabeled data to generate informative representations and generalizes well to various downstream tasks even with limited annotations. In this work, we propose a novel SSL strategy called semantically-relevant contrastive learning (SRCL), which compares relevance between instances to mine more positive pairs. Compared to the two views from an instance in traditional contrastive learning, our SRCL aligns multiple positive instances with similar visual concepts, which increases the diversity of positives and then results in more informative representations. We employ a hybrid model (CTransPath) as the backbone, which is designed by integrating a convolutional neural network (CNN) and a multi-scale Swin Transformer architecture. The CTransPath is pretrained on massively unlabeled histopathological images that could serve as a collaborative local–global feature extractor to learn universal feature representations more suitable for tasks in the histopathology image domain. The effectiveness of our SRCL-pretrained CTransPath is investigated on five types of downstream tasks (patch retrieval, patch classification, weakly-supervised whole-slide image classification, mitosis detection, and colorectal adenocarcinoma gland segmentation), covering nine public datasets. The results show that our SRCL-based visual representations not only achieve state-of-the-art performance in each dataset, but are also more robust and transferable than other SSL methods and ImageNet pretraining (both supervised and self-supervised methods). Our code and pretrained model are available at https://github.com/Xiyue-Wang/TransPath. [Display omitted] • The first Transformer-based unsupervised feature extractor for histopathological images. • A semantically-relevant contrastive learning approach is proposed. • A hybrid backbone (CTransPath) is constructed by combining CNN and Transformer. • CTransPath could be a general-purpose feature extractor for histopathological images. [ABSTRACT FROM AUTHOR]