Search

Highlights • A hierarchical framework is proposed to mimic humans for handling image tags. • A tree structure is constructed for image labels using a learning method. • Our model treats simple labels and complex labels separately. • Experiments results show promising performance on different datasets. Abstract Artificial Intelligence has attracted much of researchers' attention in recent years. A question we always ask is: "Can machines replace human beings to some extent?" This paper aims to explore the knowledge learning for an image-annotation framework, which is an easy task for humans but a tough task for machines. This paper's research is based on an assumption that machines have two systems of thinking, each of which handles the labels of images at different abstract levels. Based on this, a new hierarchical model for image annotation is introduced. We explore not only the relationships between the labels and the features used, but also the relationships between labels. More specifically, we divide labels into several hierarchies for efficient and accurate labeling, which are constructed using our Associative Memory Sharing method, proposed in this paper. [ABSTRACT FROM AUTHOR]

Visible–infrared person re-identification (VI-ReID) is an important and practical task for full-time intelligent surveillance systems. Compared to visible person re-identification, it is more challenging due to the large cross-modal discrepancy. Existing VI-ReID methods suffer from heterogeneous structures and the different spectra of visible and infrared images. In this work, we propose the Spectrum-Insensitive Data Augmentation (SIDA) strategy, which effectively alleviates the disturbance in the visible and infrared spectra and forces the network to learn spectrum-irrelevant features. The network also compares samples with both global and local features. We devise a Feature Relation Reasoning (FRR) module to learn discriminative fine-grained representations according to the graph reasoning principle. Compared to the most commonly used uniform partition, our FRR better adopts to the case of VI-ReID, in which human bodies are difficult to align. Furthermore, we design the dual center loss for learning the global feature in order to maintain the intra-modality relations, while learning the cross-modal similarities. Our method achieves better convergence in training. Extensive experiments demonstrate that our method achieves state-of-the-art performance on two visible–infrared cross-modal Re-ID datasets. • Analyzing the cross-modality discrepancy and studying the data augmentation on spectra information, we propose a Spectrum-Insensitive Data Augmentation (SIDA) strategy. • We develop a Feature Relation Reasoning (FRR) module based on the graph reasoning principle, for extraction and alignment of the fine-grained representation. Through further transferring information among cross-modality samples on the part-level, FRR learns discriminative feature representations. • We present an effective solution for VI-ReID. The experiments demonstrate that our method achieves the state-of-the-art performance on two popular benchmarks of VI-ReID datasets. [ABSTRACT FROM AUTHOR]

Highlights • This paper provides a systematic review of current histogram-based local feature descriptors, which have been applied for facial-expression recognition. • The weaknesses and strengths of the existing descriptors are discussed and analysed. • A comprehensive evaluation of the performances of 27 local descriptors is presented. • The best overall performances are obtained by LPQ and LGBPHS. This paper aims to present histogram-based local descriptors applied to Facial Expression Recognition (FER) from static images and provide a systematic review and analysis of them. First, we describe the main steps in encoding binary patterns in a local patch, which are required in every histogram-based local descriptor. Then, we list the existing local descriptors, while analysing their strengths and weaknesses. Finally, we present the experimental results of all these descriptors on commonly used facial expression databases, with varying resolution, noise, occlusion, and number of sub-regions, as well as comparing them with the results obtained by the state-of-the-art deep learning methods. This paper aims to bring together different studies of the visual features for FER by evaluating their performances under the same experimental setup, and critically reviewing various classifiers making use of the local descriptors. [ABSTRACT FROM AUTHOR]

• A robust deep low-rank feature learning and encoding model for cross-age face recognition is proposed. • A manifold-constrained low-rank approximation algorithm is proposed to learn clean low-rank feature matrix for representation. • A locality-constrained feature encoding framework is proposed to represent the cross-age face images of the same identity by similar codewords. • The proposed method outperforms other state-of-the-art methods in terms of recognition rate. Cross-age face recognition (CAFR) is a challenging task, due to significant intra-personal variations. Furthermore, the training and testing data may contain random noise components. To address these issues, this paper proposes a deep low-rank feature learning and encoding method. Firstly, our method employs manifold learning in the low-rank optimization, which preserves the global and local structure of the data samples, while learning the clean low-rank features. Secondly, we encode the low-rank features using our locality-constrained feature encoding method, which learns an age-insensitive codebook from training data, and enables the intra-class samples to share the same local bases in a codebook. In the testing stage, the gallery and probe features are encoded by the learned codebook, which represents the images of the same identity by similar codewords for recognition. Furthermore, the periocular region of human faces is investigated for CAFR. Extensive experiments on five datasets demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Abstract: Example-based super-resolution is a promising approach to solving the image super-resolution problem. However, the learning process can be slow and prediction can be inaccurate. In this paper, we present a novel learning-based algorithm for image super-resolution to improve the computational speed and prediction accuracy. Our new method classifies image patches into several classes, for each class, a class-specific predictor is designed. A class-specific predictor takes a low-resolution image patch as input and predicts a corresponding high-resolution patch as output. The performances of the class-specific predictors are evaluated using different datasets formed by face images and natural-scene images. We present experimental results which demonstrate that the new method provides improved performances over existing methods. [Copyright &y& Elsevier]

Abstract: A novel algorithm based on spatial and statistical information is proposed for the display of high-dynamic range (HDR) images. In our proposed algorithm, an image is first decomposed into a base layer and a detailed layer, which represent its smoothed and fine details, respectively. The problem of overall impression preservation is regarded as a global issue in our algorithm. Statistical-based histogram adjustment is employed to deal with the base layer. The reproduction of visual details is regarded as a local issue. The detailed layer obtained using a spatial filter is adaptively enhanced according to the mapping function used for the base layer. The main contributions of our algorithm are that: (1) an adaptive detail-enhancement method is proposed and (2) a gain map is defined to combine the local and global issues. Experimental results show the superior performance of our approach in terms of visual quality. [Copyright &y& Elsevier]

• A novel method for Low-resolution (LR) face recognition is proposed. • Features are decomposed into a low-rank feature matrix and a corresponding sparse error matrix. • Learned low-rank features are projected onto a sparse feature space for recognition. • Experiments on five face datasets shows the effectiveness of our proposed method. In this paper, we propose a novel approach for low-resolution face recognition, under uncontrolled settings. Our approach first decomposes a multiple of extracted local features into a set of representative basis (low-rank matrix) and sparse error matrix, and then learns a projection matrix based on our proposed sparse-coding-based algorithm, which preserves the sparse structure of the learned low-rank features, in a low-dimensional feature subspace. Then, a coefficient vector, based on linear regression, is computed to determine the similarity between the projected gallery and query image's features. Furthermore, a new morphological pre-processing approach is proposed to improve the visual quality of images. Our experiments were conducted on five available face-recognition datasets, which contain images with variations in pose, facial expressions and illumination conditions. Experiment results show that our method outperforms other state-of–the-art low-resolution face recognition methods in terms of recognition accuracy. [ABSTRACT FROM AUTHOR]

Saliency detection has gained popularity in many applications, and many different approaches have been proposed. In this paper, we propose a new approach based on singular value decomposition (SVD) for saliency detection. Our algorithm considers both the human-perception mechanism and the relationship between the singular values of an image decomposed by SVD and its salient regions. The key concept of our proposed algorithms is based on the fact that salient regions are the important parts of an image. The singular values of an image are divided into three groups: large, intermediate, and small singular values. We propose the hypotheses that the large singular values mainly contain information about the non-salient background and slight information about the salient regions, while the intermediate singular values contain most or even all of the saliency information. The small singular values contain little or even none of the saliency information. These hypotheses are validated by experiments. By regularization based on the average information, regularization using the leading largest singular values or regularization based on machine learning, the salient regions will become more conspicuous. In our proposed approach, learning-based methods are proposed to improve the accuracy of detecting salient regions in images. Gaussian filters are also employed to enhance the saliency information. Experimental results prove that our methods based on SVD achieve superior performance compared to other state-of-the-art methods for human-eye fixations, as well as salient-object detection, in terms of the area under the receiver operating characteristic (ROC) curve (AUC) score, the linear correlation coefficient (CC) score, the normalized scan-path saliency (NSS) score, the F-measure score, and visual quality. [ABSTRACT FROM AUTHOR]

Saliency detection has extensive applications in daily life. In this paper, an efficient saliency-detection method based on wavelet transform and entropy theory is proposed. In the algorithm proposed in this paper, salient regions are viewed as uncommon regions in the background of an image. The uncommon regions can be caused by differences in color, orientation, texture, shape, or other factors. Considering the fact that wavelet coefficients can represent the local features of an image in different scales and orientations, the wavelet transform is therefore employed to identify the salient regions. Unlike those conventional wavelet-based methods, our proposed method need not perform the inverse wavelet transformation; this can reduce the computational requirements. In addition, because the different factors (i.e. color, orientation, texture, shape, etc.) stimulate different aspects of the human visual system, a saliency-map combination scheme based on the entropy theory is devised in this paper, which can evaluate the influence or significance of the different factors. Experimental results show that our method, based on wavelet transformation and entropy theory, can achieve excellent performance in terms of the area under the receiver operating characteristic curve (AUC) score, the linear correlation coefficient (CC), the normalized scan-path saliency (NSS) score, and visual performance, as compared to existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recently, over-the-top live gaming content video (GCV) services have significantly contributed to the overall internet traffic. Consequently, there is a growing demand of GCV quality assessment (GCVQA) to maintain service quality. Although recent literature has proposed a few GCVQA methods, these mainly focus on extracting spatial features and temporal fusion separately, limiting their performance due to the neglect of spatiotemporal feature learning, which is crucial for GCV as it typically shares spatial and temporal features across frames. To address this, we propose a novel GCVQA model, focusing on GCV spatiotemporal feature learning. First, we employ a multi-task self-supervised learning spatiotemporal pyramid convolutional neural network (STP-CNN) model to extract short-term spatiotemporal quality feature representations (STQFR) of GCVs. Our STP-CNN model specifically extracts multiscale spatiotemporal features from various temporal scales of multi-frames in pyramid mode, enabling dynamic learning of diverse spatiotemporal cues. Subsequently, we propose the differential Transformer model to process all short-term STQFR within a GCV, extracting global spatiotemporal features of GCV to assess the overall quality of GCV. To evaluate the effectiveness of our proposed method, we conducted experiments using four GCVQA datasets. The results demonstrate that our method outperforms existing approaches in predicting the perceived quality of GCV. • Propose a deep learning-based NR-GCVQA method tailored explicitly for GCV, emphasizing spatiotemporal feature learning in short-term and long-term. • Propose the multi-task, self-supervised STP-CNN model that employs a pyramid mode to extract multiscale spatiotemporal features, optimizing the representation of short-term spatiotemporal features. • Introduce the Diff-Transformer model to distill global spatiotemporal features of GCV to achieve a precise GCV quality prediction. • Provide experimental evidence to substantiate the accuracy and effectiveness of our proposed NR-GCVQA methodology. [ABSTRACT FROM AUTHOR]

Highlights • The selected proposals are utilized to calculate the background regions as the propagation seeds. • The texture and global structure of images are integrated to refine the saliency-detection results. • With ERW propagation, our method can distinguish salient objects from the background reliably. Abstract Recently, many graph-based algorithms are applied in the research of saliency detection, which use the border of an image as a background query. This frequently leads to undesired errors and retrieval outputs when the boundaries of the salient objects concerned touch, or connect with, the image's border. In this paper, a novel bottom-up saliency-detection algorithm is proposed to tackle and overcome the above issue. First, we utilize object proposals to collect the background seeds reliably. Then, the Extended Random Walk (ERW) algorithm is adopted to propagate the prior background labels to the rest of the pixels in an image. Finally, we refine the saliency map by taking both the textural and structure-information into consideration simultaneously. Experiments on publicly available data sets show that our proposed method achieves competitive results against the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel framework for underwater image saliency detection by exploiting Quaternionic Distance Based Weber Descriptor (QDWD), pattern distinctness, and local contrast. Our proposed algorithm incorporates quaternion number system and principal components analysis (PCA) simultaneously, so as to achieve superior performance. In our algorithm, QDWD, which was initially designed for detecting outliers in color images, is used to represent the directional cues in an underwater image. Then, PCA coordinate system is employed to compute pattern distinctness. Meanwhile, we utilize local contrast to further highlight salient regions and suppress background regions. Finally, by integrating QDWD, pattern distinctness, and local contrast, a reliable saliency map for underwater images can be computed and estimated. Experimental results, based on the publicly available OUC-VISION underwater image database, show that the proposed method can produce reliable and promising results, compared to other state-of-the-art saliency-detection models. [ABSTRACT FROM AUTHOR]