Search

Deep convolutional neural networks (CNNs) have gained prominence in computer vision applications, including RGB salient object detection (SOD), owing to the advancements in deep learning. Nevertheless, the majority of deep CNNs employ either VGGNet or ResNet as their backbone architecture for extracting image information. This approach may lead to the following problems. 1) Variations between imaging modalities during feature extraction across layers. Cross-modal features across layers are often fused in a single step, resulting in inadequate cross-modal feature extraction. 2) Feature long-range dependence problem in multilayer feature decoding. 3) Image boundary blurring. To address these issues, we initially leverage the advantages offered by the VGGNet and ResNet architectures. Additionally, we present a novel hybrid VGG–ResNet feature encoder for RGB-T SOD. Specifically, we introduce a geometry information aggregation module that effectively combines and enhances the VGGNet spatial features of the RGB-T modalities from the bottom to the top. Moreover, we propose a innovative global saliency perception module that progressively refines the ResNet semantic features from the top to the bottom by integrating both local and global information. Furthermore, we introduce a Pearson-gated module to tackle the challenge of long-range dependence between features. This module utilizes gating to merge features by calculating the Pearson correlation coefficients of the fused features at multiple levels. Lastly, we devise an edge-aware module to precisely learn the contours of salient objects, thereby enhancing the clarity of the object boundaries. Extensive experiments conducted on three RGB-T SOD benchmarks demonstrate that our proposed network surpasses the performance of state-of-the-art methods for SOD. [ABSTRACT FROM AUTHOR]

With the wide propagation of cloud and mobile computing, screen content images (SCIs) have become more indispensable in our daily lives. Compared to natural scene images (NSIs), SCIs possess many particular characteristics, like mixed contents, extremely sharp edges, and text graphics. Consequently, more challenges occur in the feature extraction, which is used to reflect the distortion, during the quality assessment of SCIs. Recently, some convolutional neural network (CNN) models have been designed by automatically learning feature to evaluate the quality. In this paper, we develop a novel blind quality assessment method for SCIs via the CNN. First, compared with existing CNN-based methods, the proposed method avoids the disadvantage of training with image patches, and it is the pioneering attempt that takes the entire image as inputs. Second, instead of the image gray value, the original image is decomposed into two portions, i.e., the predicted and unpredicted portions, according to the internal generative mechanism (IGM) theory as the input of CNN. Through the CNN, all features of the image are learned automatically from beginning to end, and the network finally outputs the predicted score. Since existing SCI database is too small, to fully train the network, we collected 30000 SCIs and employed a high-accuracy full-reference quality assessment metric of SCI to compute scores as the training labels. Experimental results on SIQAD database demonstrate that the proposed method is comparable to reference-based SCI quality assessment metrics and is superior to the state-of-the-art NSI quality assessment metrics. [ABSTRACT FROM AUTHOR]