Search

This article introduces a framework for predicting the survival of brain tumor patients by analyzing magnetic resonance images. The prediction of brain tumor survival is challenging due to the limited size of available datasets. To overcome the issue of overfitting, we propose a self-supervised learning method that involves identifying image patches from the same or different images. By recognizing intra- and inter-image differences, the network can learn the relationships between local spatial windows in the same image and across different images. In addition to analyzing local information, we also incorporate a global structure awareness network to capture global information from the entire image. Our proposed method shows a strong correlation between local spatial relationships and survivor class prediction in FLAIR MRI brain images. We evaluate our method using the BraTS 2020 validation dataset and observe that our method outperforms others in accuracy and SpearmanR correlation metrics.

Data augmentation is an important paradigm for boosting the generalization capability of deep learning in image classification tasks. Image augmentation using cut-and-paste strategies has shown very good performance improvement for deep learning. However, these existing methods often overlook the image's discriminative local context and rely on ad hoc regions consisting of square or rectangular local regions, leading to the loss of complete semantic object parts. In this work, we attempt to overcome these limitations and propose a superpixel-wise local-context-aware efficient image fusion approach for data augmentation. Our approach requires only one forward propagation using a superpixel attention-based label fusion with less computational complexity. The model is trained using a combination of a global classification of the fused (augmented) image loss, a superpixel-wise weighted local classification loss, and a superpixel-based weighted contrastive learning loss. The last two losses are based on the superpixel-aware attentive embeddings. Thus, the resulting deep encoder can learn both local and global features of the images while capturing object-part local context and information. Experiments on diverse benchmark image datasets indicate that the proposed method out-performs many region-based augmentation methods for visual recognition. We have demonstrated its effectiveness not only on CNN models but also on transformer models. The codes are accessible at https://github.com/DanielaPlusPlus/SAFuse. • We note the potential drawbacks of existing Cutmix augmentation strategies. • We present a superpixel-based pairwise image fusion for data augmentation. • It can capture the most discriminative semantically image regions. • Selected superpixels are contrasted and classified; fused images are classified. • We perform extensive experiments with different CNN and Transformer backbones. [ABSTRACT FROM AUTHOR]

Abstract. The saliency object detection is a hot topic of computer vision. Traditional saliency detection methods are overly dependent on handcrafted low-level features. The saliency detection methods based on deep learning can effectively solve the problem, which extracts high-level features automatically. However, there are some noises in the extracted high-level features that affect the detection performance. We propose a deep learning framework for saliency detection based on global prior and local context. First, we use feature maps generated by combining some middle-level features as the input of global-prior-based deep learning model, which can reduce the interference of distracting feature information for the saliency detection. Then, two deep learning models use respectively local contexts of color image and depth map as input, which combine global prior to generate the initial saliency map. Finally, the optimized saliency map can be obtained based on spatial consistence and appearance similarity. Experiments on two publicly available datasets show that the proposed method performs better than other nine state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Background: Protein secondary structure is the three dimensional form of local segments of proteins and its prediction is an important problem in protein tertiary structure prediction. Developing computational approaches for protein secondary structure prediction is becoming increasingly urgent. Results: We present a novel deep learning based model, referred to as CNNH_PSS, by using multi-scale CNN with highway. In CNNH_PSS, any two neighbor convolutional layers have a highway to deliver information from current layer to the output of the next one to keep local contexts. As lower layers extract local context while higher layers extract long-range interdependencies, the highways between neighbor layers allow CNNH_PSS to have ability to extract both local contexts and long-range interdependencies. We evaluate CNNH_PSS on two commonly used datasets: CB6133 and CB513. CNNH_PSS outperforms the multi-scale CNN without highway by at least 0.010 Q8 accuracy and also performs better than CNF, DeepCNF and SSpro8, which cannot extract long-range interdependencies, by at least 0.020 Q8 accuracy, demonstrating that both local contexts and long-range interdependencies are indeed useful for prediction. Furthermore, CNNH_PSS also performs better than GSM and DCRNN which need extra complex model to extract long-range interdependencies. It demonstrates that CNNH_PSS not only cost less computer resource, but also achieves better predicting performance. Conclusion: CNNH_PSS have ability to extracts both local contexts and long-range interdependencies by combing multi-scale CNN and highway network. The evaluations on common datasets and comparisons with state-of-the-art methods indicate that CNNH_PSS is an useful and efficient tool for protein secondary structure prediction. [ABSTRACT FROM AUTHOR]

Deep learning technology has been extensively explored by existing methods to improve the performance of target detection in remote sensing images, due to its powerful feature extraction and representation abilities. However, these methods usually focus on the interior features of the target, but ignore the exterior semantic information around the target, especially the object-level relationship. Consequently, these methods fail to detect and recognize targets in the complex background where multiple objects crowd together. To handle this problem, a diversified context information fusion framework based on convolutional neural network (DCIFF-CNN) is proposed in this paper, which employs the structured object-level relationship to improve the target detection and recognition in complex backgrounds. The DCIFF-CNN is composed of two successive sub-networks, i.e., a multi-scale local context region proposal network (MLC-RPN) and an object-level relationship context target detection network (ORC-TDN). The MLC-RPN relies on the fine-grained details of objects to generate candidate regions in the remote sensing image. Then, the ORC-TDN utilizes the spatial context information of objects to detect and recognize targets by integrating an attentional message integrated module (AMIM) and an object relational structured graph (ORSG). The AMIM is integrated into the feed-forward CNN to highlight the useful object-level context information, while the ORSG builds the relations between a set of objects by processing their appearance features and geometric features. Finally, the target detection method based on DCIFF-CNN effectively represents the interior and exterior information of the target by exploiting both the multiscale local context information and the object-level relationships. Extensive experiments are conducted, and experimental results demonstrate that the proposed DCIFF-CNN method improves the target detection and recognition accuracy in complex backgrounds, showing superiority to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]