Search

Facial beauty prediction (FBP) is an important and challenging problem in the fields of computer vision and machine learning. Not only it is easily prone to overfitting due to the lack of large-scale and effective data, but also difficult to quickly build robust and effective facial beauty evaluation models because of the variability of facial appearance and the complexity of human perception. Transfer Learning can be able to reduce the dependence on large amounts of data as well as avoid overfitting problems. Broad learning system (BLS) can be capable of quickly completing models building and training. For this purpose, Transfer Learning was fused with BLS for FBP in this paper. Firstly, a feature extractor is constructed by way of CNNs models based on transfer learning for facial feature extraction, in which EfficientNets are used in this paper, and the fused features of facial beauty extracted are transferred to BLS for FBP, called E-BLS. Secondly, on the basis of E-BLS, a connection layer is designed to connect the feature extractor and BLS, called ER-BLS. Finally, experimental results show that, compared with the previous BLS and CNNs methods existed, the accuracy of FBP was improved by E-BLS and ER-BLS, demonstrating the effectiveness and superiority of the method presented, which can also be widely used in pattern recognition, object detection and image classification. [ABSTRACT FROM AUTHOR]

Face recognition in visual surveillance systems is important for various applications to identify individuals who are behaving defiantly at the time of an event or for investigation purposes. Despite the dramatic improvements in facial recognition technology in recent years, it is difficult to recognize faces from surveillance feeds due to the presence of multiple people of different scales and orientations. This paper solves the task of low-resolution face recognition by combining exemplary techniques for extracting distinct features. This research utilizes the attributes learned by customized and learnable filters and injected in the training process to better match them with human brain functionality. The Gabor transform aims to convolve a facial image using a range of Gabor filter coefficients at various scales and orientations, resulting in scale and rotation invariant features. The tailored architecture with residual stream aims to enhance functional representation and prevent the gradient of the prediction engine from affecting the backbone network functional map. Experimental analysis is performed on the SCface and TinyFace databases and is reported with an accuracy of 89.21% on the SCface database and 56.68% on the TinyFace database. [ABSTRACT FROM AUTHOR]

Human facial expression and emotion play pivotal roles in our day-to-day communication, and detecting them are one of the formidable tasks in the field of human–computer interfaces (HCI). This paper presents a new facial expressions detection method by exploiting textural image features such as local binary patterns (LBP), local ternary patterns (LTP) and completed local binary pattern (CLBP). This paper utilizes the advantages of textural features which are highly correlated with the facial expression changes and thereby trains a convolution neural network (CNN) model to detect facial expressions. The CNN model is trained on the images from the extended Cohn-Kanade (CK +), JAFEE and FER2013 datasets that are converted into LBP, LTP and CLBP image features. The performance of our facial expression recognition system is validated on modified CK+, JAFEE and FER2013 dataset. The results reported here illustrates that the CNN model yields better efficiency when we train the model with textural images. Moreover, we have shown that the CNN model trained with CLBP outperforms than that of with LBP and LTP images. In case of CLBP images, accuracies are 91.0%, 82.2% and 64.5% for CK+, JAFFE and FER2013 dataset, respectively. In case of LBP, accuracies are 79.5%, 75% and 58.45% and in case of LTP images accuracies are 89.2%, 77.3% and 62.79% for the datasets CK+, JAFFE and FER2013, respectively. [ABSTRACT FROM AUTHOR]

This paper deals with the problem of video-based face recognition. Nowadays, facial recognition methods have made a big step forward, but video-based recognition with its poor quality, difficult lighting conditions, and real-time requirements is still a difficult and unfinished task. The paper uses the apparatus of convolutional networks for various stages of processing: for capturing and detecting a face, for constructing a feature vector, and finally for recognition. All algorithms are implemented and studied in the Matlab environment to simplify their further export to embedded applications. [ABSTRACT FROM AUTHOR]

The application of the facial expression recognition system in the human–computer interaction system refers to the recognition of human facial expressions through the human–computer interaction system in the real society, so as to be able to feel the specific situation of recognizing people. This is also one of the main directions of human–computer interaction system research. In this paper, the facial expression recognition system is designed by the algorithm that combines the expressions of the students in the classroom teaching with the system environment, so that the recognition of the facial expressions of the students in the classroom environment is more accurate. This article elaborates on the identification method of the system and conducts detailed experimental analysis on the specific functions of other modules in the system. The experimental results show that the security and stability of the system are very high. At the same time, the accuracy of the system in the classroom teaching environment is also very high in the recognition of student facial expressions. This is a modern intelligent face recognition system that enters education and teaching. Provide a strong theoretical basis and technical support during the work. [ABSTRACT FROM AUTHOR]

Face anti-spoofing has been attracting attention because of its prominent role in the security of face recognition systems. A face spoofing attack is launched on the authentication system by the attacker using the picture or video of the genuine user to get access to the services or information. This paper proposes local binary pattern (LBP) and convolutional neural network (CNN) based feature fusion model for classifying spoof face images. Initially, the images are pre-processed to remove the effect of the background in the feature extraction. The Viola-Jones algorithm is used for detecting and cropping the faces from the images. The color features of the pre-processed RGB images and LBP features are provided to the CNN. This feature combination reduces the requirement of a CNN-based deep learning architecture for providing a good classification performance. To validate the proposed model, it is trained and tested on NUAA and MSU-MFSD datasets. The experiment recorded the ACER of 0.21 and 0.20 on the MSU-MFSD and NUAA datasets, respectively. [ABSTRACT FROM AUTHOR]

Human emotions play a vital role in developing social interaction as well as interpersonal relationships. It is because humans express and convey their thoughts and feelings visually rather than using verbal communication. Recently, facial expression recognition is greatly adopted in the emerging human–computer interaction and human–robot interaction technologies. Although humans can identify facial emotions virtually, the reliable automated recognition of facial expressions becomes a complicated task. The main intention of this paper is to develop an automated facial expression recognition system. To attain the goal, in this work, an improved particle swarm optimization bat tracking search algorithm optimized quantum convolutional neural network (IPSOBSA-QCNN) approach is proposed to accurately identify and classify facial expressions such as fear, happy, disgust, sad, surprise, negative, positive, and dominated. Although classification performance is limited by hyperparameter tuning concerns, the QCNN system efficiently recognizes face expressions at a high processing speed. Therefore, the hyperparameters of the quantum convolutional neural network (QCNN) are optimized to enhance the classification accuracy using the improved particle swarm optimization backtracking search algorithm (IPSOBSA). The proposed IPSOBSA-QCNN approach is evaluated using three diverse datasets namely the real-world affective faces dataset, the Emotic dataset, and the facial expression comparison dataset. The effectiveness of the proposed approach is determined by comparing the proposed IPSOBSA-QCNN approach with other existing methods using different evaluation metrics. The experimental results display that the proposed IPSOBSA-QCNN approach achieves higher classification accuracy of about 98% for the real-world affective faces dataset, 97.5% for the Emotic dataset, and 97% for the facial expression comparison dataset. [ABSTRACT FROM AUTHOR]

Deep learning has become the main solution for face recognition applications due to its high accuracy and robustness. In recent years, a batch of research on lightweight convolutional neural networks (CNNs) has emerged, bringing new ideas to the economic application of face recognition systems. In this paper, a lightweight face recognition algorithm is proposed to reduce the number of parameters and calculations of the face feature extraction network. The most important part of our approach lies in designing a novel inverted residual shuffle unit (IR-Shuffle). After being trained by ArcFace loss on a GPU workstation, our model built on improved IR-Shuffle blocks of size 1.45 MB achieves an accuracy of 98.65%. In terms of running time, our model is 5 ms faster than the current fastest MobileFaceNet, with only about 0.5% drop in accuracy. The proposed algorithm is implemented and optimized on the Jetson Nano embedded platform, and accurate and real-time deployment of the face recognition system is realized. The system takes 37 ms to perform the complete face detection and recognition and is robust to complex backgrounds and ambient light changes. Experimental results show that our system is of practical application value. [ABSTRACT FROM AUTHOR]

Rapid advances in deep learning models have made it easier for public and crackers to generate hyper-realistic deepfake videos in which faces are swapped. Such deepfake videos may constitute a significant threat to the world if they are misused to blackmail public figures and to deceive systems of face recognition. As a result, distinguishing these fake videos from real ones has become fundamental. This paper introduces a new deepfake video detection method. You Only Look Once (YOLO) face detector is used to detect faces from video frames. A proposed hybrid method based on proposing two different feature extraction methods is applied to these faces. The first feature extraction method, a proposed Convolution Neural Network (CNN), is based on the Histogram of Oriented Gradient (HOG) method. The second one is an ameliorated XceptionNet CNN. The two extracted sets of features are merged together and fed as input to a sequence of Gated Recurrent Units (GRUs) to extract the spatial and temporal features and then individuate the authenticity of videos. The proposed method is trained on the CelebDF-FaceForencics++ (c23) dataset and evaluated on the CelebDF test set. The experimental results and analysis confirm the superiority of the suggested method over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Compared with common deep learning methods (e.g., convolutional neural networks), transfer learning is characterized by simplicity, efficiency and its low training cost, breaking the curse of small datasets. Medical image analysis plays an indispensable role in both scientific research and clinical diagnosis. Common medical image acquisition methods include Computer Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), X-Ray, etc. Although these medical imaging methods can be applied for non-invasive qualitative and quantitative analysis of patients—compared with image datasets in other computer vision fields such like faces—medical images, especially its labeling, is still scarce and insufficient. Therefore, more and more researchers adopted transfer learning for medical image processing. In this study, after reviewing one hundred representative papers from IEEE, Elsevier, Google Scholar, Web of Science and various sources published from 2000 to 2020, a comprehensive review is presented, including (i) structure of CNN, (ii) background knowledge of transfer learning, (iii) different types of strategies performing transfer learning, (iv) application of transfer learning in various sub-fields of medical image analysis, and (v) discussion on the future prospect of transfer learning in the field of medical image analysis. Through this review paper, beginners could receive an overall and systematic knowledge of transfer learning application in medical image analysis. And policymaker of related realm will benefit from the summary of the trend of transfer learning in medical imaging field and may be encouraged to make policy positive to the future development of transfer learning in the field of medical image analysis. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) has gone through more than forty years from the simulation stage to the embryonic stage, to the conception and the initial formation of the theory, until further simple applications. Firstly, deep convolution neural network is used to extract features, and then the extracted features are coded by Gaussian aggregation. Finally, the encoded features are input into the full connection layer to classify the image. The experiment in this paper focuses on the two-dimensional facial expression animation technology under DNN. This experiment mainly introduces the two important steps of image conversion, spatial mapping and resampling technology, as well as the detailed definition of the MPEG-4 standard. By using the feature points obtained in the experiment as the feature points set in the face definition parameters, in the deformation algorithm based on Delaunay triangulation, the inverse mapping technology and the quadratic linear interpolation technology are combined with the face and combined with facial animation. The parameter step can realize the conversion of facial expressions, and the generated facial expressions are more natural and delicate.The experimental data show that the facial animation parameters (FAP) as a complete set of basic facial movements can recognize the most subtle facial expressions under DNN. Change, different FAP combinations can form different facial expressions. The experimental results show that the 76 facial feature points located in the experiment have been tested for grid generation. When the segmentation threshold is larger, the merge method is closer to the sub-point insertion method; otherwise, it is closer to the divide and conquer method. When the threshold is 20, the algorithm belongs to the divide and conquer method; when the threshold is 80, the algorithm belongs to the point insertion method. Through research, this article finds that the innovation of DNN on the visual performance of animated films can provide more help for the production of sophisticated animation works, and can continuously improve the visual performance and cultural connotation of animated films. [ABSTRACT FROM AUTHOR]

Convolutional neural networks have been extensively used as the key role to address many computer vision applications. Traditionally, learning convolutional features is performed in a hierarchical manner along the dimension of network depth to create multi-scale feature maps. As a result, strong semantic features are derived at the top-level layers only. This paper proposes a novel feature pyramid fashion to produce semantic features at all levels of the network for specially addressing the problem of face detection. Particularly, a Semantic Convolutional Box (SCBox) is presented by merging the features from different layers in a bottom-up fashion. The proposed lightweight detector is stacked of alternating SCBox and Inception residual modules to learn the visual features in both the dimensions of network depth and width. In addition, the newly introduced objective functions (e.g., focal and CIoU losses) are incorporated to effectively address the problem of unbalanced data, resulting in stable training. The proposed model has been validated on the standard benchmarks FDDB and WIDER FACES, in comparison with the state-of-the-art methods. Experiments showed promising results in terms of both processing time and detection accuracy. For instance, the proposed network achieves an average precision of 96.8 % on FDDB, 82.4 % on WIDER FACES, and gains an inference speed of 106 FPS on a moderate GPU configuration or 20 FPS on a CPU machine. [ABSTRACT FROM AUTHOR]

Techniques for facial age progression and regression have many applications and a myriad of challenges. As such, automatic aged or de-aged face generation has become an important subject of study in recent times. Over the past decade or so, researchers have been working on developing face processing mechanisms to tackle the challenge of generating realistic aged faces for applications related to smart systems. In this paper, we propose a novel approach to try and address this problem. We use template faces based on the formulation of an average face of a given ethnicity and for a given age. Thus, given a face image, the target aged image for that face is generated by applying it to the relevant template face image. The resulting image is controlled by two parameters corresponding to the texture and the shape of the face. To validate our approach, we compute the similarity between aged images and the corresponding ground truth via face recognition. To do this, we have utilised a pre-trained convolutional neural network based on the VGG-face model for feature extraction, and we then use well-known classifiers to compare the features. We have utilised two datasets, namely the FEI and the Morph II, to test, verify and validate our approach. Our experimental results do suggest that the proposed approach achieves accuracy, efficiency and possess flexibility when it comes to facial age progression or regression. [ABSTRACT FROM AUTHOR]

Face recognition of realistic visual images (e.g., photos) has been well studied and made significant progress in the recent decade. However, face recognition between realistic visual images/photos and caricatures is still a challenging problem. Unlike the photos, the different artistic styles of caricatures introduce extreme non-rigid distortions of caricatures. The great representational gap between the different modalities of photos and caricatures is a big challenge for photo-caricature face recognition. In this paper, we propose to conduct cross-modal photo-caricature face recognition via multi-task learning, which can learn the features of different modalities with different tasks. Instead of manually setting the task weights as in conventional multi-task learning, this work proposes a dynamic weights learning module which can automatically generate/learn task weights according to the training importance of tasks. The learned task weights enable the network to focus on training the hard tasks instead of being stuck in the overtraining of easy tasks. The experimental results demonstrate the effectiveness of the proposed dynamic multi-task learning for cross-modal photo-caricature face recognition. The performance on the datasets CaVI and WebCaricature show the superiority over the state-of-art methods. The implementation code is provided here. (https://github.com/hengxyz/cari-visual-recognition-via-multitask-learning.git). [ABSTRACT FROM AUTHOR]

Binarized convolutional neural networks (BNNs) are widely used to improve the memory and computational efficiency of deep convolutional neural networks for to be employed on embedded devices. However, existing BNNs fail to explore their corresponding full-precision models' potential, resulting in a significant performance gap. This paper introduces a Rectified Binary Convolutional Network (RBCN) by combining full precision kernels and feature maps to rectify the binarization process in a generative adversarial network (GAN) framework. We further prune our RBCNs using the GAN framework to increase the model efficiency and promote flexibly in practical applications. Extensive experiments validate the superior performance of the proposed RBCN over state-of-the-art BNNs on tasks such as object classification, object tracking, face recognition, and person re-identification. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks have proven to be a highly efficient approach for face recognition. In this paper, we develop such a framework to learn a robust face verification in an unconstrained environment using aggressive data augmentation. Our objective is to learn a deep face representation from large-scale data with massive noisy and occluded face. Besides, we add an adaptive fusion of softmax loss and center loss as supervision signals, which are helpful to improve the performance and to conduct the final classification. The experiment results show that the suggested system achieves comparable performances with other state-of-the-art methods on the Labeled Faces in the Wild and YouTube face verification tasks. [ABSTRACT FROM AUTHOR]

Human behavior analysis from big multimedia data has become a trending research area with applications to various domains such as surveillance, medical, sports, and entertainment. Facial expression analysis is one of the most prominent clues to determine the behavior of an individual, however, it is very challenging due to variations in face poses, illuminations, and different facial tones. In this paper, we analyze human behavior using facial expressions by considering some famous TV-series videos. Firstly, we detect faces using Viola-jones algorithm followed by tracking through Kanade-Lucas-Tomasi (KLT) algorithm. Secondly, we use histogram of oriented gradients (HOG) features with support vector machine (SVM) classifier for facial recognition. Next, we recognize facial expressions using the proposed light-weight convolutional neural network (CNN). We utilize data augmentation techniques to overcome the issue of appearance of faces from different views and lightening conditions in video data. Finally, we predict human behaviors using an occurrence matrix acquired from facial recognition and expressions. The subjective and objective experimental evaluations prove better performance for both facial expression recognition and human behavior understanding. [ABSTRACT FROM AUTHOR]

In this paper, we propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks. As the human faces are highly structured and share unified facial components (e.g., eyes and mouths), such semantic information provides a strong prior for restoration. We incorporate face semantic labels as input priors and propose an adaptive structural loss to regularize facial local structures within an end-to-end deep convolutional neural network. Specifically, we first use a coarse deblurring network to reduce the motion blur on the input face image. We then adopt a parsing network to extract the semantic features from the coarse deblurred image. Finally, the fine deblurring network utilizes the semantic information to restore a clear face image. We train the network with perceptual and adversarial losses to generate photo-realistic results. The proposed method restores sharp images with more accurate facial features and details. Quantitative and qualitative evaluations demonstrate that the proposed face deblurring algorithm performs favorably against the state-of-the-art methods in terms of restoration quality, face recognition and execution speed. [ABSTRACT FROM AUTHOR]

Security issues of artificial intelligence attract many attention in many research fields and industries, such as face recognition, medical care, and client services. Federated learning is proposed by Google, which can prevent the leakage of data during the AI training because each enterprise only needs to exchange training parameters without data sharing. In this paper, we present a novel differentially private federated learning framework (DP-FL) for unbalanced data. In the cloud server, DP-FL framework considers the unbalanced data of different users to set different privacy budgets. In the user client, we design a novel differential private convolutional neural networks with adaptive gradient descent (DPAGD-CNN) algorithm to update each user's training parameters. Experimental results on several real-world datasets demonstrate that the DF-FL framework can protect data privacy with higher accuracy than existing works. [ABSTRACT FROM AUTHOR]

Deep learning technology has enabled successful modeling of complex facial features when high-quality images are available. Nonetheless, accurate modeling and recognition of human faces in real-world scenarios "on the wild" or under adverse conditions remains an open problem. Consequently, a plethora of novel deep network architectures addressing issues related to low-quality images, varying pose, illumination changes, emotional expressions, etc., have been proposed and studied over the last few years.This survey presents a comprehensive analysis of the latest developments in the field. A conventional deep face recognition system entails several main components: deep network, optimization loss function, classification algorithm, and train data collection. Aiming at providing a complete and comprehensive study of such complex frameworks, this paper first discusses the evolution of related network architectures. Next, a comparative analysis of loss functions, classification algorithms, and face datasets is given. Then, a comparative study of state-of-the-art face recognition systems is presented. Here, the performance of the systems is discussed using three benchmarking datasets with increasing degrees of complexity. Furthermore, an experimental study was conducted to compare several openly accessible face recognition frameworks in terms of recognition accuracy and speed. [ABSTRACT FROM AUTHOR]

In this paper, we propose a cancelable multi-biometric face recognition method that uses multiple convolutional neural networks (CNNs) to extract deep features from different facial regions. We also propose a new CNN architecture that exploits batch normalization, depth concatenation and a residual learning framework. The proposed method adopts a region-based technique in which face, eyes, nose and mouth regions are detected from the original face images. Multiple CNNs are used to extract deep features from each region, and then, a fusion network combines these features. Moreover, to provide user's privacy and increase the system resistance against spoof attacks, a cancelable biometric technique using bio-convolving encryption is performed on the final facial descriptor. Our experiments on the FERET, LFW and PaSC datasets show excellent and competitive results compared to state-of-the-art methods in terms of recognition accuracy, specificity, precision, recall and fscore. [ABSTRACT FROM AUTHOR]

Facial expression offers an important way of detecting the affective state of a human being. It plays a major role in various fields such as the estimation of students' attention level in online education, intelligent transportation systems and interactive games. This paper proposes a facial expression recognition system in which two channels of featured images are used to represent a 3D facial scan. Features are extracted from the local binary pattern and local directional pattern using a fine-tuned pre-trained AlexNet and a shallow convolutional neural network. The feature sets are then fused together using canonical correlation analysis. The fused feature set is fed into a multi-support vector machine (mSVM) classifier to classify the expressions into seven basic categories: anger, disgust, fear, happiness, neutral, sadness and surprise. Experiments were carried out on the Bosphorus database using tenfold cross-validation with mutually exclusive training and testing samples. The results show an average accuracy of 87.69% using an mSVM classifier with a polynomial kernel and demonstrate that the system performs better by characterizing the peculiarities in facial expressions than alternative state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Multi-face tracking in unconstrained videos is a challenging problem as faces of one person often can appear drastically different in multiple shots due to significant variations in scale, pose, expression, illumination, and make-up. Existing multi-target tracking methods often use low-level features which are not sufficiently discriminative for identifying faces with such large appearance variations. In this paper, we tackle this problem by learning discriminative, video-specific face representations using convolutional neural networks (CNNs). Unlike existing CNN-based approaches which are only trained on large-scale face image datasets offline, we automatically generate a large number of training samples using the contextual constraints for a given video, and further adapt the pre-trained face CNN to the characters in the specific videos using discovered training samples. The embedding feature space is fine-tuned so that the Euclidean distance in the space corresponds to the semantic face similarity. To this end, we devise a symmetric triplet loss function which optimizes the network more effectively than the conventional triplet loss. With the learned discriminative features, we apply an EM clustering algorithm to link tracklets across multiple shots to generate the final trajectories. We extensively evaluate the proposed algorithm on two sets of TV sitcoms and YouTube music videos, analyze the contribution of each component, and demonstrate significant performance improvement over existing techniques. [ABSTRACT FROM AUTHOR]

Identifying emotional sentiment projected in an image is a tedious task, considering the fact that sentiment represented by an image could depend on a very diverse set of factors. This paper presents a novel approach to predict the emotional sentiment of a group of people in a variety of environments. The proposed technique uses local facial features of subjects along with global scene features to estimate the type of emotional sentiment in group-level emotion recognition. Two separate convolutional neural networks based on different architectures are designed to predict group-level emotions into three categories: negative, neutral and positive. The first convolutional neural network referred as Scene-model, learns the global features in data. A novel partial fine-tuning process is proposed to train the model on task-specific data. The second convolutional model referred as Face-model is trained on facial expression datasets to learn the emotional status of subjects in an image. Joint distribution of the global (scene) and local (face) features is modeled using long short-term memory networks. This joint distribution is converted into class scores using softmax regression-based model. [ABSTRACT FROM AUTHOR]

Devising computational models for detecting abnormalities reflective of diseases from facial structures is a novel and emerging field of research in automatic face analysis. In this paper, we focus on automatic pain intensity estimation from faces. This has a paramount potential diagnosis values in healthcare applications. In this context, we present a novel 3D deep model for dynamic spatiotemporal representation of faces in videos. Using several convolutional layers with diverse temporal depths, our proposed model captures a wide range of spatiotemporal variations in the faces. Moreover, we introduce a cross-architecture knowledge transfer technique for training 3D convolutional neural networks using a pre-trained 2D architecture. This strategy is a practical approach for training 3D models, especially when the size of the database is relatively small. Our extensive experiments and analysis on two benchmarking and publicly available databases, namely the UNBC-McMaster shoulder pain and the BioVid, clearly show that our proposed method consistently outperforms many state-of-the-art methods in automatic pain intensity estimation. [ABSTRACT FROM AUTHOR]