Search

Highlights • This paper provides a systematic review of current histogram-based local feature descriptors, which have been applied for facial-expression recognition. • The weaknesses and strengths of the existing descriptors are discussed and analysed. • A comprehensive evaluation of the performances of 27 local descriptors is presented. • The best overall performances are obtained by LPQ and LGBPHS. This paper aims to present histogram-based local descriptors applied to Facial Expression Recognition (FER) from static images and provide a systematic review and analysis of them. First, we describe the main steps in encoding binary patterns in a local patch, which are required in every histogram-based local descriptor. Then, we list the existing local descriptors, while analysing their strengths and weaknesses. Finally, we present the experimental results of all these descriptors on commonly used facial expression databases, with varying resolution, noise, occlusion, and number of sub-regions, as well as comparing them with the results obtained by the state-of-the-art deep learning methods. This paper aims to bring together different studies of the visual features for FER by evaluating their performances under the same experimental setup, and critically reviewing various classifiers making use of the local descriptors. [ABSTRACT FROM AUTHOR]

Face recognition across age progression remains one of the area's most challenging tasks, as the aging process affects both the shape and texture of a face. One possible solution is to apply a probabilistic model to represent a face simultaneously with its identity variable, which is stable through time, and its aging variable, which changes with time. However, as the aging process varies for different people, a person may look younger or older than another person, even though their ages are the same. Consequently, using the ‘real’ age labels given by existing face datasets for age-invariant face recognition will inevitably introduce ambiguity to learning algorithms. In this paper, an identity-inference model, based on age-subspace learning from appearance-age labels, is proposed. We first model human identity and aging variables simultaneously using Probabilistic Linear Discriminant Analysis (PLDA). Then, the aging subspace is learnt independently with the appearance-age labels, and the identity subspace is then determined iteratively with the Expectation-Maximization (EM) algorithm. We found that the learned aging subspace is insensitive to the training face images used, and is independent of the identity model. Consequently, the recognition of aging faces becomes simpler as identity inference no longer needs to consider age labels. Furthermore, in our algorithm, different identity features learnt from the identity model are further combined using Canonical Correlation Analysis (CCA), where their correlations are maximized for face recognition. A thorough experimental analysis of face recognition is performed on three public domain face-aging datasets: FGNET, MORPH, and CACD. Experiment results show that the proposed framework can achieve a comparable, or even better, performance against other state-of-the-art methods, especially when the age range is large. [ABSTRACT FROM AUTHOR]

Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one gallery per subject only. With the easy access to high-resolution (HR) face images nowadays, some HR face databases have recently been developed. However, few studies have tackled the use of HR information for face recognition or verification. In this paper, we propose a pose-invariant face-verification method, which is robust to alignment errors, using the HR information based on pore-scale facial features. A new keypoint descriptor, namely, pore-Principal Component Analysis (PCA)-Scale Invariant Feature Transform (PPCASIFT)—adapted from PCA-SIFT—is devised for the extraction of a compact set of distinctive pore-scale facial features. Having matched the pore-scale features of two-face regions, an effective robust-fitting scheme is proposed for the face-verification task. Experiments show that, with one frontal-view gallery only per subject, our proposed method outperforms a number of standard verification methods, and can achieve excellent accuracy even the faces are under large variations in expression and pose. [ABSTRACT FROM AUTHOR]

Establishing correct correspondences between two faces with different viewpoints has played an important role in 3D face reconstruction and other computer-vision applications. Usually, face images are considered to lack sufficient distinctive features to establish a large number of correspondences on uncalibrated images. In this paper, we investigate pore-scale facial features, which are formed from pores, fine wrinkles, and hair. These features have many characteristics that make them suitable for matching facial images under different variations. Using both biological observation and computer-vision consideration, a new framework is devised for pore-scale facial-feature extraction and matching. The matching difficulty under various skin appearances of different subjects and imaging distortion is also analyzed. For further improving the matching performance and tackling distortions such as varying illuminations and unfocused blurring, a pore-to-pore correspondences dataset is established for training a more distinctive and compact descriptor. Experiments are conducted on a face database containing 105 subjects, and the results prove that the pore-scale features are highly distinctive; face images with a minimum resolution of 600×700 (0.4 mega) pixels contain sufficient details to perform a reliable matching in different poses. Generally, our algorithm can establish between 500 and 2000 correct correspondences on a pair of uncalibrated face images of the same person. Furthermore, the proposed methods can be applied to face recognition, 3D reconstruction, etc. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, an efficient method based on singular values and potential-field representation is proposed for face-image retrieval. Firstly, we theoretically prove that the leading singular values of an image can be used as a rotation-shift-scale-invariant global feature. Then, for the feature-extraction stage, we exploit these special properties of the singular values to devise a compact, global feature for face-image representation. We also use the singular values of the potential field derived from edge gradients to enhance the retrieval performance. Experimental results based on the GTAV database show that the use of singular values as rotation-shift-scale-invariant global features is able to produce plausible retrieval results. [Copyright &y& Elsevier]

Abstract: For face recognition, image features are first extracted and then matched to those features in a gallery set. The amount of information and the effectiveness of the features used will determine the recognition performance. In this paper, we propose a novel face recognition approach using information about face images at higher and lower resolutions so as to enhance the information content of the features that are extracted and combined at different resolutions. As the features from different resolutions should closely correlate with each other, we employ the cascaded generalized canonical correlation analysis (GCCA) to fuse the information to form a single feature vector for face recognition. To improve the performance and efficiency, we also employ “Gabor-feature hallucination”, which predicts the high-resolution (HR) Gabor features from the Gabor features of a face image directly by local linear regression. We also extend the algorithm to low-resolution (LR) face recognition, in which the medium-resolution (MR) and HR Gabor features of a LR input image are estimated directly. The LR Gabor features and the predicted MR and HR Gabor features are then fused using GCCA for LR face recognition. Our algorithm can avoid having to perform the interpolation/super-resolution of face images and having to extract HR Gabor features. Experimental results show that the proposed methods have a superior recognition rate and are more efficient than traditional methods. [Copyright &y& Elsevier]

In this paper, we propose an efficient algorithm to reconstruct the 3D structure of a human face from one or more of its 2D images with different poses. In our algorithm, the nonlinear least-squares model is first employed to estimate the depth values of facial feature points and the pose of the 2D face image concerned by means of the similarity transform. Furthermore, different optimization schemes are presented with regard to the accuracy levels and the training time required. Our algorithm also embeds the symmetrical property of the human face into the optimization procedure, in order to alleviate the sensitivities arising from changes in pose. In addition, the regularization term, based on linear correlation, is added in the objective function to improve the estimation accuracy of the 3D structure. Further, a model-integration method is proposed to improve the depth-estimation accuracy when multiple nonfrontal-view face images are available. Experimental results on the 2D and 3D databases demonstrate the feasibility and efficiency of the proposed methods. [ABSTRACT FROM PUBLISHER]

Abstract: In this paper, we propose a wavelet-based eigentransformation method for human face hallucination. Our algorithm uses the wavelet transform to decompose interpolated low-resolution (LR) images in the wavelet domain to obtain high-frequency information in three different directions, and employs the eigentransformation method to reconstruct the corresponding finer high-frequency content of the high-resolution (HR) images. The low-frequency content of the HR images in the wavelet domain is estimated based on the interpolated images directly. The resulting high-quality HR faces can be synthesized by using the inverse wavelet transform, with all the estimated coefficients. By combining interpolation and eigentransformation, the reconstructed images are less dependent on the training set selected, and can better preserve the low-frequency content. Thus, the reconstructed images look more like the ground-true HR images, as compared to the original eigentransformation method. Experimental results show that our proposed algorithm outperforms the original eigentransformation and other existing methods for face hallucination in terms of both visual quality and objective measurements. [Copyright &y& Elsevier]

In this paper, an efficient algorithm for detecting human faces in color images is proposed. The first step of our algorithm is to segment the possible skin-like regions in an image by using color information. One of the major problems of using skin color is that a face region may not be detected under poor lighting conditions, or if the lighting conditions vary over the face region. Our approach considers the distributions of the color components of skin pixels under different illuminations. This information can be used to identify skin-color pixels reliably under varying lighting conditions. The skin-color regions are then clustered and verified as human face regions. In order to improve the reliability and accuracy, an eigenmask that has a large magnitude at the important facial features of a human face is used in the detection. Experimental results show that this algorithm can detect human faces under varying lighting conditions reliably and fast. [Copyright &y& Elsevier]

Saliency detection has gained popularity in many applications, and many different approaches have been proposed. In this paper, we propose a new approach based on singular value decomposition (SVD) for saliency detection. Our algorithm considers both the human-perception mechanism and the relationship between the singular values of an image decomposed by SVD and its salient regions. The key concept of our proposed algorithms is based on the fact that salient regions are the important parts of an image. The singular values of an image are divided into three groups: large, intermediate, and small singular values. We propose the hypotheses that the large singular values mainly contain information about the non-salient background and slight information about the salient regions, while the intermediate singular values contain most or even all of the saliency information. The small singular values contain little or even none of the saliency information. These hypotheses are validated by experiments. By regularization based on the average information, regularization using the leading largest singular values or regularization based on machine learning, the salient regions will become more conspicuous. In our proposed approach, learning-based methods are proposed to improve the accuracy of detecting salient regions in images. Gaussian filters are also employed to enhance the saliency information. Experimental results prove that our methods based on SVD achieve superior performance compared to other state-of-the-art methods for human-eye fixations, as well as salient-object detection, in terms of the area under the receiver operating characteristic (ROC) curve (AUC) score, the linear correlation coefficient (CC) score, the normalized scan-path saliency (NSS) score, the F-measure score, and visual quality. [ABSTRACT FROM AUTHOR]

• A constraint based on a 3D morphable model is proposed. • A large pore-to-pore correspondences dataset is established. • The Deeply Learned Pore-scale Facial Feature (DLPFF) is proposed. • The DLPFF has generalization on different high-resolution facial databases. Similar to fingerprints and irises, pore-scale facial features can be used to distinguish human identities effectively. However, without pore-to-pore correspondences dataset, there are no deep learning based methods for pore-scale facial features. Actually, it is hard to establish a large pore-to-pore correspondences dataset due to the existing high-resolution face databases are uncalibrated and nonsynchronous. In this paper, we employ a constraint based on 3D facial model and construct a large pore-to-pore correspondences dataset. This dataset is then used to train a Convolutional Neural Network (CNN) to generate the novel pore-scale facial features - Deeply Learned Pore-scale Facial Features (DLPFF). The experiments show that our learning based method achieves the state-of-the-art matching performance on the Bosphorus facial database and has good generalization. [ABSTRACT FROM AUTHOR]

Abstract: Gabor wavelets (GWs) are commonly used for extracting local features for various applications such as object detection, recognition and tracking. However, extracting Gabor features is computationally intensive, so the features are impractical for real-time applications. In this paper, we propose a simplified version of Gabor wavelets (SGWs) and an efficient algorithm for extracting the features based on an integral image. We evaluate the performance of the SGW features for face recognition. Experimental results show that using SGWs can achieve a performance level similar to using GWs, while the runtime for feature extraction using SGWs is, at most, 4.39 times faster than that of GWs implemented by using the fast Fourier transform (FFT). [Copyright &y& Elsevier]