Search

The effectiveness of facial expression recognition (FER) algorithms hinges on the model’s quality and the availability of a substantial amount of labeled expression data. However, labeling large datasets demands significant human, time, and financial resources. Although active learning methods have mitigated the dependency on extensive labeled data, a cold-start problem persists in small to medium-sized expression recognition datasets. This issue arises because the initial labeled data often fails to represent the full spectrum of facial expression characteristics. This paper introduces an active learning approach that integrates uncertainty estimation, aiming to improve the precision of facial expression recognition regardless of dataset scale variations. The method is divided into two primary phases. First, the model undergoes self-supervised pre-training using contrastive learning and uncertainty estimation to bolster its feature extraction capabilities. Second, the model is fine-tuned using the prior knowledge obtained from the pre-training phase to significantly improve recognition accuracy. In the pretraining phase, the model employs contrastive learning to extract fundamental feature representations from the complete unlabeled dataset. These features are then weighted through a self-attention mechanism with rank regularization. Subsequently, data from the low-weighted set is relabeled to further refine the model’s feature extraction ability. The pre-trained model is then utilized in active learning to select and label information-rich samples more efficiently. Experimental results demonstrate that the proposed method significantly outperforms existing approaches, achieving an improvement in recognition accuracy of 5.09% and 3.82% over the best existing active learning methods, Margin, and Least Confidence methods, respectively, and a 1.61% improvement compared to the conventional segmented active learning method. [ABSTRACT FROM AUTHOR]

Facial expression recognition is a crucial area of study in psychology that can be applied to many fields, such as intelligent healthcare, human-computer interaction, fuzzy control and other domains. However, current deep learning models usually encounter high complexity, expensive computational requirements and outsized parameters. These obstacles hinder the deployment of applications on resource-constrained mobile terminals. This paper proposes an improved lightweight MobileViT with channel expansion and attention mechanism for facial expression recognition to address these challenges. In this model, we adopt a channel expansion strategy to effectively extract more critical facial expression feature information from multi-scale feature maps. Furthermore, we introduce a channel attention module within the model to improve feature extraction performance. Compared with typical lightweight models, our proposed model significantly improves the accuracy rate while maintaining a lightweight network. Our proposed model achieves 94.35 and 87.41% accuracy on the KDEF and RAF-DB datasets, respectively, demonstrating superior recognition performance. [ABSTRACT FROM AUTHOR]

Summary: The ability of facial expression recognition, the ability to decipher human emotions from facial features, and we can judge the emotional state of human beings by analysing facial expressions. Empowered by this technology, the related research of student sentiment analysis has become a focal point in the realm of educational technology. Teachers can now read the emotional state of their students and estimate the effectiveness of their teaching strategies, enabling them to implement appropriate intervention techniques to enhance teaching outcomes. However, facial expression recognition techniques are currently limited by their shortcomings. Network performance degradation and loss of feature information are key issues that hinder the effectiveness of sentiment analysis of student feedback. To overcome these limitations, in this paper, we propose RFMNet, a novel network model based on deep learning theory. RFMNet, with its higher extraction capability, is designed to accurately analyse student expressions. It introduces a relation‐aware global attention (RGA) module, which facilitates the integration of more discriminative expression features in the image, leading to more refined sentiment analysis. This innovative model also employs a Mish activation function and a focal loss function. The Mish activation function is of particular interest because it helps to avoid the loss of feature information due to neuron deactivation when the ReLU gradient is. This approach results in a more robust and accurate facial expression recognition model. Our model was tested on the FERPlus public dataset. The model achieved an average recognition accuracy of 89.62%, which is a testament to its performance in decoding the facial expressions of students. This precision allows for intelligent processing of teaching information and real‐time feedback, enabling teachers to adapt their teaching strategies to better suit the needs of their students. [ABSTRACT FROM AUTHOR]

The objective of the article is to recognize users' emotions by classifying facial electromyographic (EMG) signals. A biomedical signal amplifier, equipped with eight active electrodes positioned in accordance with the Facial Action Coding System, was used to record the EMG signals. These signals were registered during a procedure where users acted out various emotions: joy, sadness, surprise, disgust, anger, fear, and neutral. Recordings were made for 16 users. The mean power of the EMG signals formed the feature set. We utilized these features to train and evaluate various classifiers. In the subject-dependent model, the average classification accuracies were 96.3% for KNN, 94.9% for SVM with a linear kernel, 94.6% for SVM with a cubic kernel, and 93.8% for LDA. In the subject-independent model, the classification results varied depending on the tested user, ranging from 91.4% to 48.6% for the KNN classifier, with an average accuracy of 67.5%. The SVM with a cubic kernel performed slightly worse, achieving an average accuracy of 59.1%, followed by the SVM with a linear kernel at 53.9%, and the LDA classifier at 41.2%. Additionally, the study identified the most effective electrodes for distinguishing between pairs of emotions. [ABSTRACT FROM AUTHOR]

Distinctive cognitive processes and analytical approaches are employed by humans and computers in the realm of emotion recognition, particularly when interpreting nuanced facial expressions. Leveraging these differences, we have engineered an advanced CAPTCHA system powered by a cutting-edge facial emotion recognition algorithm. This innovative CAPTCHA challenges users to replicate and align a specific facial expression with a reference image against a contextual backdrop. Following rigorous testing, we ascertained that the system exhibits less than a 1% susceptibility to security compromises. This method not only enhances the security mechanism of CAPTCHA but also introduces a more personalised interaction model, offering significant advantages for user groups such as the elderly. Through comprehensive user evaluations, we have confirmed that this CAPTCHA system surpasses traditional models in both recognition accuracy and processing velocity, demonstrating its substantial practical advantages. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes an improved strategy for the MobileNetV2 neural network(I-MobileNetV2) in response to problems such as large parameter quantities in existing deep convolutional neural networks and the shortcomings of the lightweight neural network MobileNetV2 such as easy loss of feature information, poor real-time performance, and low accuracy rate in facial emotion recognition tasks. The network inherits the characteristics of MobilenetV2 depthwise separated convolution, signifying a reduction in computational load while maintaining a lightweight profile. It utilizes a reverse fusion mechanism to retain negative features, which makes the information less likely to be lost. The SELU activation function is used to replace the RELU6 activation function to avoid gradient vanishing. Meanwhile, to improve the feature recognition capability, the channel attention mechanism (Squeeze-and-Excitation Networks (SE-Net)) is integrated into the MobilenetV2 network. Experiments conducted on the facial expression datasets FER2013 and CK + showed that the proposed network model achieved facial expression recognition accuracies of 68.62% and 95.96%, improving upon the MobileNetV2 model by 0.72% and 6.14% respectively, and the parameter count decreased by 83.8%. These results empirically verify the effectiveness of the improvements made to the network model.

This paper proposed a robust recognition model FFDNet for facial expression recognition in complex environments with occlusion and pose variation of the face. The algorithm used a multi-scale structure for feature fusion to address the differences in face region scales. It decomposed feature differences and fine-grained by fine-grained modules, and used encoders to capture features with discriminative power and small differences. Furthermore it proposed a diversity feature loss function to drive the model to mine richer fine-grained features. Experimental results show that FFDNet obtains 88.50% and 88.75% accuracy on the RAF-DB and FERPlus datasets, respectively, while outperforming some state-of-the-art models on both occlusion and pose variation datasets. The experimental results demonstrate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes an improved strategy for the MobileNetV2 neural network(I-MobileNetV2) in response to problems such as large parameter quantities in existing deep convolutional neural networks and the shortcomings of the lightweight neural network MobileNetV2 such as easy loss of feature information, poor real-time performance, and low accuracy rate in facial emotion recognition tasks. The network inherits the characteristics of MobilenetV2 depthwise separated convolution, signifying a reduction in computational load while maintaining a lightweight profile. It utilizes a reverse fusion mechanism to retain negative features, which makes the information less likely to be lost. The SELU activation function is used to replace the RELU6 activation function to avoid gradient vanishing. Meanwhile, to improve the feature recognition capability, the channel attention mechanism (Squeeze-and-Excitation Networks (SE-Net)) is integrated into the MobilenetV2 network. Experiments conducted on the facial expression datasets FER2013 and CK + showed that the proposed network model achieved facial expression recognition accuracies of 68.62% and 95.96%, improving upon the MobileNetV2 model by 0.72% and 6.14% respectively, and the parameter count decreased by 83.8%. These results empirically verify the effectiveness of the improvements made to the network model. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence technology, the digital transformation of student-oriented education becomes particularly important. How to promote real-time interaction between teachers and students in the classroom is an urgent issue which is needed to pay attention to. Based on the facial expression features of students in a classroom, this paper analyzes the changes in angles between facial expression feature points using Dlib. Additionally, this paper proposes a novel algorithm for extracting variable scale template edge trend features. The algorithm adaptively processes the template based on the edge trend features of expression feature points, and use the proposed template slope normalization algorithm to achieve multi-scale template edge trend extraction. Then, DNN are used to recognize different listening expressions. The experimental results show that the proposed algorithm has faster recognition speed and better robustness when applied to classroom expression recognition. By identifying students' class status to remind teachers to adjust their class progress, the goal of improving classroom learning effectiveness is achieved [ABSTRACT FROM AUTHOR]

Convolutional neural networks struggle to accurately handle changes in angles and twists in the direction of images, which affects their ability to recognize patterns based on internal feature levels. In contrast, CapsNet overcomes these limitations by vectorizing information through increased directionality and magnitude, ensuring that spatial information is not overlooked. Therefore, this study proposes a novel expression recognition technique called CAPSULE-VGG, which combines the strengths of CapsNet and convolutional neural networks. By refining and integrating features extracted by a convolutional neural network before introducing theminto CapsNet, ourmodel enhances facial recognition capabilities. Compared to traditional neural network models, our approach offers faster training pace, improved convergence speed, and higher accuracy rates approaching stability. Experimental results demonstrate that our method achieves recognition rates of 74.14% for the FER2013 expression dataset and 99.85% for the CK+ expression dataset. By contrasting these findings with those obtained using conventional expression recognition techniques and incorporating CapsNet's advantages, we effectively address issues associated with convolutional neural networks while increasing expression identification accuracy. [ABSTRACT FROM AUTHOR]

Although there has been much progress in the area of facial expression recognition (FER), most existing methods suffer when presented with images that have been captured from viewing angles that are non-frontal and substantially different from those used in the training process. In this article, we propose ViewFX, a novel view-invariant FER framework based on contrastive learning, capable of accurately classifying facial expressions regardless of the input viewing angles during inference. ViewFX learns view-invariant features of expression using a proposed self-supervised contrastive loss, which brings together different views of the same subject with a particular expression in the embedding space. We also introduce a supervised contrastive loss to push the learned view-invariant features of each expression away from other expressions. Since facial expressions are often distinguished with very subtle differences in the learned feature space, we incorporate the Barlow twins loss to reduce the redundancy and correlations of the representations in the learned representations. The proposed method is a substantial extension of our previously proposed CL-MEx, which only had a self-supervised loss. We test the proposed framework on two public multi-view facial expression recognition datasets, KDEF and DDCF. The experiments demonstrate that our approach outperforms previous works in the area and sets a new state-of-the-art for both datasets while showing considerably less sensitivity to challenging angles and the number of output labels used for training. We also perform detailed sensitivity and ablation experiments to evaluate the impact of different components of our model as well as its sensitivity to different parameters. [ABSTRACT FROM AUTHOR]

The aim of this study is designed an improved ResNet 50 network to achieve automatic classification model for pain expressions by elderly patients with hip fractures. This study built a dataset by combining the advantages of deep learning in image recognition, using a hybrid of the Multi-Task Cascaded Convolutional Neural Networks (MTCNN). Based on ResNet50 network framework utilized transfer learning to implement model function. This study performed the hyperparameters by Bayesian optimization in the learning process. This study calculated intraclass correlation between visual analog scale scores provided by clinicians independently and those provided by pain expression evaluation assistant(PEEA). The automatic pain expression recognition model in elderly patients with hip fractures, which constructed using the algorithm. The accuracy achieved 99.6% on the training set, 98.7% on the validation set, and 98.2% on the test set. The substantial kappa coefficient of 0.683 confirmed the efficacy of PEEA in clinic. This study demonstrates that the improved ResNet50 network can be used to construct an automatic pain expression recognition model for elderly patients with hip fractures, which has higher accuracy.

Facial expression recognition under classroom scenes can help the teacher to understand students' classroom learning status and improve teaching effectiveness. Aiming at the problem of low expression recognition accuracy in classroom scenarios, a novel multi-scale facial expression recognition algorithm based on improved Res2Net is proposed. Firstly, a bi-directional residual BiRes2Net module is proposed to achieve bi-directional multi-scale expression feature extraction at the fine-grained level, while a short-directed connection path is introduced to make the network have the self-closing capability and avoid extracting redundant information of expressions; Then the Fine-Grained Coordinate Attention (FGCA) mechanism is embedded to extract expression spatial location features and channel features at a fine-grained level by making full use of the prior knowledge of facial expressions; Finally, a multi-classification Focalloss loss function is used to alleviate the imbalance of expression data, and different weights are assigned to expression samples with different recognition difficulty so that the network is biased towards difficult sample feature extraction. The experimental results show that the recognition accuracy of the proposed method is 79.47%, 94.06%, and 96.67% in RAF-DB, JAFFE, and CK+ datasets respectively, and up to 72.71% in real classroom scenes, which are better than other comparative algorithms significantly. [ABSTRACT FROM AUTHOR]

The recognition of facial expression in images is one of the motional states in the observing forms and is one of the most frequent non-verbal routes in which a person transfers his inner emotional expressions on faces. The recognition of facial expressions in a wide range of fields including psychological and legal studies, animation, robotics, lip-reading, image and video conferencing, communications, telecommunications, and security protection whilst counterterrorism is used to identify individuals as well as human-machine confrontation. The general solution to this problem includes three general steps: images preprocessing, features extraction, and expression classification algorithms. A series of pre-processing steps must be performed to process the area on the face and then detect the expression, that is, a square in the face must be localized while the rest of the image must be removed. Then, Features extraction is used to classify. Each facial expression to a specific category. We divide our data, including images from different expressions, into two parts: training and testing. Different categories have been learned to specify different features that tested thereafter. In recent years, a number of researches has been performed on a facial expression analysis. Even though much progress has been made in this field since the recognition of facial expression with a high accuracy rate is difficult to achieve due to the complexity and variability. In this research article, we noticed that most of researchers are used JAFFE and CK+ databases due the diversification and high accuracy. Nevertheless most of researchers are used PSO, PCA, and LBP features as well as HOG that presented high accuracy. We also noticed that SVM and CNN classification algorithms have been used mostly due to high accuracy and response latency with few errors. [ABSTRACT FROM AUTHOR]

It is often assumed that the recognition of facial expressions is impaired in autism. However, recent evidence suggests that reports of expression recognition difficulties in autistic participants may be attributable to co-occurring alexithymia—a trait associated with difficulties interpreting interoceptive and emotional states—not autism per se. Due to problems fixating on the eye-region, autistic individuals may be more reliant on information from the mouth region when judging facial expressions. As such, it may be easier to detect expression recognition deficits attributable to autism, not alexithymia, when participants are forced to base expression judgements on the eye-region alone. To test this possibility, we compared the ability of autistic participants (with and without high levels of alexithymia) and non-autistic controls to categorise facial expressions (a) when the whole face was visible, and (b) when the lower portion of the face was covered with a surgical mask. High-alexithymic autistic participants showed clear evidence of expression recognition difficulties: they correctly categorised fewer expressions than non-autistic controls. In contrast, low-alexithymic autistic participants were unimpaired relative to non-autistic controls. The same pattern of results was seen when judging masked and unmasked expression stimuli. In sum, we find no evidence for an expression recognition deficit attributable to autism, in the absence of high levels of co-occurring alexithymia, either when participants judge whole-face stimuli or just the eye-region. These findings underscore the influence of co-occurring alexithymia on expression recognition in autism. [ABSTRACT FROM AUTHOR]

The complex facial expressions and small targets make it difficult for many existing artificial intelligence algorithms to achieve consistent identification. Thus, a multi-stream residual network integrated with an improved Support Vector Machine ( SVM) for facial expression recognition is proposed. The method consists of three steps, namely information enhancement, expression feature extraction and expression classification. First, an adaptive multi-stream information enhancement module is proposed to highlight key image information and enhance feature correlation. Next, a residual interaction module is proposed to extract the spatial information of feature images, highlight facial expression features, and output three feature images with diiferent scales. Last, improved SVM is employed to classify the expression of three-branch feature images to further enhance the robustness of the algorithm. The experimental results show that the accuracy of the proposed method on CK +, FER2013 and JAFFE data sets reaches 98.57%, 77.28% and 96. 4% respectively, all of which are superior to other classic and novel algorithms. Thus, it provides new insights in the recognition of facial expressions by AI. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Harbin University of Science & Technology is the property of Journal of Harbin University of Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Parkinson's Disease (PD) is associated with motor and non-motor symptoms. Among the latter are deficits in matching, identification, and recognition of emotional facial expressions. On one hand, this deficit has been attributed to a dysfunction in emotion processing. Another explanation (which does not exclude the former) links this deficit with reduced facial expressiveness in these patients, which prevents them from properly understanding or embodying emotions. To disentangle the specific contribution of emotion comprehension and that of facial expression processing in PD's observed deficit with emotions we performed two experiments on non-emotional facial expressions. In Experiment 1, a group of PD patients and a group of Healthy Controls (HC) underwent a task of non-emotional expression recognition in faces of different identity and a task of identity recognition in faces with different expression. No differences were observed between the two groups in accuracies. In Experiment 2, PD patients and Healthy Controls underwent a task where they had to recognize the identity of faces encoded through a non-emotional facial expression, through a rigid head movement, or as neutral. Again, no group differences were observed. In none of the two experiments hypomimia scores had a specific effect on expression processing. We conclude that in PD patients the observed impairment with emotional expressions is likely due to a specific deficit for emotions to a greater extent than for facial expressivity processing.

In order to achieve facial object detection and tracking in video, a method based on nonlinear sequence Monte Carlo filtering technology is proposed. The algorithm is simple, effective, and easy to operate, which can solve the problems of scale change and occlusion in the process of online learning tracking, so as to ensure the smooth implementation of learning effect evaluation. Experimental methods should be added to the article summary section. The results show that the algorithm in this study outperforms the basic KCF in terms of evaluation accuracy and success rate, as well as outperforms other tracker algorithms in benchmark, achieving scores of 0.837 and 0.705, respectively. In terms of overlapping accuracy, the reason why this study’s algorithm is higher than KCF is that this study determines the tracking status of the current target by calculating the primary side regulated (PSR) value when the target is obscured or lost, which does not make the tracking error to accumulate. The tracking algorithm in this study is not ranked first in the two attributes of motion blur and low resolution, but the rankings of all other nine attributes belong to the first. Compared with the KCF algorithm, the accuracy plots for the three attributes of scale change, occlusion, and leaving the field of view are improved by 10.26, 13.48, and 13.04%, respectively. Thus, it is proved that the method based on nonlinear sequence Monte Carlo filtering technology can achieve video facial object detection and tracking.

The objective of the article is to recognize users’ emotions by classifying facial electromyographic (EMG) signals. A biomedical signal amplifier, equipped with eight active electrodes positioned in accordance with the Facial Action Coding System, was used to record the EMG signals. These signals were registered during a procedure where users acted out various emotions: joy, sadness, surprise, disgust, anger, fear, and neutral. Recordings were made for 16 users. The mean power of the EMG signals formed the feature set. We utilized these features to train and evaluate various classifiers. In the subject-dependent model, the average classification accuracies were 96.3% for KNN, 94.9% for SVM with a linear kernel, 94.6% for SVM with a cubic kernel, and 93.8% for LDA. In the subject-independent model, the classification results varied depending on the tested user, ranging from 91.4% to 48.6% for the KNN classifier, with an average accuracy of 67.5%. The SVM with a cubic kernel performed slightly worse, achieving an average accuracy of 59.1%, followed by the SVM with a linear kernel at 53.9%, and the LDA classifier at 41.2%. Additionally, the study identified the most effective electrodes for distinguishing between pairs of emotions.

It takes ten years to grow trees and a hundred years to nurture people. Education, as the most indispensable link in establishing morality and cultivating people, has a very important role in modern society. The study explores the application of ancient Chinese educational concepts to modern educational models, proposes a modernized teaching model based on ancient educational concepts, and uses the YOLO model to construct a classroom teaching expression and action recognition model under ancient educational concepts. The model is used in the actual classroom to analyze the expressions and behaviors of students in the classroom to explore their classroom concentration, which is used to analyze the quality of classroom teaching. By comparing the differences and changes in the performance of the experimental group and the control group, the effect of the modernized teaching model based on the ancient education concept on actual teaching is explored. Through the model test, the classroom students’ concentration score is 72.71. The pre-test P-value of the experimental group and the control group is 0.846, and the post-test P-value is 0.000, which indicates that there is a significant difference in the performance of the two groups after the experiment. The increase in the performance of the students in the experimental group was 31.390 points, while in the control group, it was 5.175 points. The P-values of the students in the experimental group on the five dimensions of math level are less than 0.05, and the control group is greater than 0.05, which indicates that the modernized teaching model based on the ancient concept of education has a significant positive effect on improving the performance of the students.

In the education system, teachers and students as the main body of the classroom; their emotional state in the classroom school is an important indicator of the effectiveness of the classroom. This study first explores biometric recognition, based on the needs of the classroom curriculum and the classroom monitoring as a sensor, to propose a multimodal biometric fusion detection method based on the fusion of face and gait recognition. The PCA algorithm is used to optimize the face recognition as well as the occlusion situation in the classroom to improve gait recognition, and then the face and gait are fused based on the decision layer to achieve the detection and recognition of the identity situation of teachers and students. On this basis, an expression recognition model is established using the attention mechanism, and an emotion analysis system is designed for the classroom curriculum. According to the empirical evidence of multimodal biometric fusion sentiment analysis, the mAP accuracy of this paper’s fusion method is 100% in Euclidean distance, and the accuracy is higher than 99% in cosine distance, which is obviously better than other methods, and the accuracy of this paper’s fusion recognition is above 95% under any condition limitations. At the same time, the correct rate of recognition of emotions such as listening, appreciation, resistance, doubt, and inattention are all higher than 85%, and the five indexes of average absolute error, Pearson correlation coefficient, Accuarcy5, Accuarcy2, and F12 score of this paper’s sentiment analysis have achieved the best results comparing with other sentiment analysis models, which proves the generalization and validity of this paper’s sentiment analysis.

With the development of the times and the continuous innovation of modern education technology, the multimodal theory is developing deeply in the field of education. The first step in this study is to use the optimization principle to select modes that meet the needs of English translation teaching and then develop a multimodal analysis framework for external communication that incorporates multimedia communication methods. Then, we constructed an emotion recognition model for English translation teaching by using speech recognition and expression recognition combined with convolutional neural networks for the students’ classroom situation and attitude and carried out the empirical analysis. The recognition accuracy of the emotion recognition system constructed in this paper is 86.4%. In the multimodal analysis of English translation teaching, the teacher who uses the linguistic modality type for the most number of times 387 times, and the teacher who uses the mixed modality for the most number of times 271 times, the students have the best classroom status. It has been stable at about 5 points, which is a “more positive” state. The students in the class had the highest level of motivation. Through the analysis, it is found that multimodal teaching can effectively improve the students’ status in class and keep their attention, which verifies the appropriateness of the external communication path combined with multimedia communication in multimodal English translation teaching. It provides a practical path for the development of English translation teaching.

With the informatization of education, the traditional English teaching evaluation method is increasingly unable to meet the needs of cultivating talents in colleges and universities. In this paper, we constructed an informatized English teaching evaluation system using two modules: students’ classroom learning status evaluation and teachers’ teaching quality evaluation. Using informationized face detection and expression recognition technology, the evaluation axis of students’ classroom learning status is established as “students’ head-up rate - students’ expression - students’ concentration - classroom learning status”. The fuzzy comprehensive judgment model based on a genetic algorithm was established by constructing a judgment matrix and calculating the weights of teaching quality evaluation indexes using a hierarchical analysis method. The experimental and control groups analyzed the English teaching evaluation system’s application. Regarding English achievement, the post-experimental scores of the students in the experimental class were 8.57 points higher than those of the control class, with a highly significant difference (P

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This study examined whether three subtypes of anxiety (trait anxiety, state anxiety, and social anxiety) have different effects on recognition of facial expressions. One hundred and thirty-eight participants matched facial expressions of three intensity levels (20 %, 40 %, 100 %) with one of the six emotion labels (“happy”, “sad”, “fear”, “angry”, “disgust”, and “surprise”). While using a conventional method of analysis we were able to replicate some significant correlations between each anxiety type and recognition performance found in the literature. However, when we used partial correlation to isolate the effect of each anxiety type, most of these correlations were no longer significant, apart from the negative correlations between Beck Anxiety Inventory and reaction time to fearful faces displayed at 40 % intensity level, and the correlations between anxiety and categorisation errors. Specifically, social anxiety was positively correlated with misidentifying a happy face as a disgust face at 40 % intensity level, and state anxiety negatively correlated with misidentifying a happy face as a sad face at 20 % intensity level. However, these partial correlation analyses became non-significant after p value adjustment for multiple comparisons. Our eye tracking data also showed that state anxiety may be associated with reduced fixations on the eye regions of low-intensity sad or fearful faces. These analyses cast doubts on some effects reported in the previous studies because they are likely to reflect a mixture of influences from highly correlated anxiety subtypes.

Human facial behavior is an important information for communication. The study of facial behavior is one of the significant research topics in field of psychology, computer vision and artificial intelligence. In order to improve performance of facial expression and action unit recognition, a face recognition method based on deep-block network proposed in this paper. First, to improve the network performance, we preprocess the input of the network facial image, which includes two operations: face detection and face standardization. Second, deep-block network regards facial parts as the core of expression recognition rather than the whole face and key areas are in charge of specific action units to abate the weak correlation bias, which results in better classification and regression effect. Last, with the purpose of reducing impact of image independent factors, relevant feature map is applied to recognize the associated facial action units, which can promote the accuracy of detection to a certain extent. Experimental results on CK+ and MMI show that proposed method can not only capture the correlation of whole face regions globally, but also can increase network speed caused by too few pooling layers. [ABSTRACT FROM AUTHOR]

Face images in the logarithmic space can be considered as a sum of the texture component and lighting map component according to Lambert Reflection. However, it is still not easy to separate these two parts, because face contour boundaries and lighting change boundaries are difficult to distinguish. In order to enhance the separation quality of these to parts, this paper proposes an illumination standardization algorithm based on extreme L0 Gaussian difference regularization constraints, assuming that illumination is massively spread all over the image but illumination change boundaries are simple, regular, and sparse enough. The proposed algorithm uses an iterative L0 Gaussian difference smoothing method, which achieves a more accurate lighting map estimation by reserving the fewest boundaries. Thus, the texture component of the original image can be restored better by simply subtracting the lighting map estimated. The experiments in this paper are organized with two steps: the first step is to observe the quality of the original texture restoration, and the second step is to test the effectiveness of our algorithm for complex face classification tasks. We choose the facial expression classification in this step. The first step experimental results show that our proposed algorithm can effectively recover face image details from extremely dark or light regions. In the second step experiment, we use a CNN classifier to test the emotion classification accuracy, making a comparison of the proposed illumination removal algorithm and the state-of-the-art illumination removal algorithm as face image preprocessing methods. The experimental results show that our algorithm works best for facial expression classification at about 5 to 7 percent accuracy higher than other algorithms. Therefore, our algorithm is proven to provide effective lighting processing technical support for the complex face classification problems which require a high degree of preservation of facial texture. The contribution of this paper is, first, that this paper proposes an enhanced TV model with an L0 boundary constraint for illumination estimation. Second, the boundary response is formulated with the Gaussian difference, which strongly responds to illumination boundaries. Third, this paper emphasizes the necessity of reserving details for preprocessing face images. [ABSTRACT FROM AUTHOR]

Applying ensemble learning to facial expression recognition is an important research field nowadays, but all may not be better than many, the redundant learners in the classifier pool may hinder the ensemble system's performance, so ensemble pruning is needed. Ensemble pruning selects the most suitable subset of classifiers to classify test samples according to the classifier competence. However, the noisy and redundant samples in the validation set will often adversely affect the evaluation of the classifier, making it impossible to select the most suitable classifier. In this paper, a novel ensemble pruning algorithm based on clustering soft label optimization and sorting for facial expression recognition is proposed. First, to increase classifier evaluation objectivity, the novel method uses the clustering optimization model to perform prototype selection and classifier clustering simultaneously. Then the accuracy-based ordering is employed to remove the redundant or poor quality learners, and keep a balance between diversity and accuracy of the ensemble system. Experimental results show that the proposed method outperforms or competes with some state-of-the-art methods on several typical facial expression datasets. [ABSTRACT FROM AUTHOR]

Emotion is an important aspect of effective human communication, and hence, facial emotion recognition (FER) has become essential in human–computer interaction systems. The automation of FER has been carried out by many researchers using ML/DL techniques. However, the models developed using convolutional neural networks (CNNs) bagged high recognition accuracies among different FER approaches. Rather than its high performance, CNN has failed to encode different orientation features since the pooling operations used in CNN for feature extraction omit vital information. Due to omitting vital features, the performance will be reduced while recognizing the emotions from facial images that consists of different orientations. Subsequently, to reduce the problems of CNN such as encoding different orientation features and increased training time, Capsule Networks (CapsNet) were developed. CapsNet is capable of storing 8 such features vectors with the incorporation of dynamic routing approaches and squashing in place of pooling operations to mitigate the issue of rotational invariance. Hence in this paper, we proposed CapsNet for FER in order to enhance the accuracy. However, the facial images that consider for training consist of unwanted information that is not essential for FER, delay the convergence and take more iterations for training facial images. Hence, face localization (FL) is proposed to incorporate with CapsNet in our model to eliminate the back ground noise or unwanted information from the facial images for effective training process. The proposed FL-CapsNet is rigorously tested on benchmark datasets such as JAFFE, CK+, and FER2013 to evaluate the generalization of the proposed model, and it is evidenced that FL-CapsNet outperformed the existing CapsNet-based FER models. [ABSTRACT FROM AUTHOR]

Aiming at the problems that face images in uncontrollable environments are susceptible to complex factors such as illumination and pose changes, which in turn cause low face detection rate and poor expression recognition accuracy in face expression recognition, an expression recognition method with an embedded attention mechanism residual network is proposed. In the stage of face detection, the improved RetinaFace algorithm is used to complete multi-view face detection and obtain the face region. In the stage of feature extraction, ResNet-50 is used as the backbone network for feature extraction. Firstly, the pre- processed face images are sequentially passed through the channel attention network and spatial attention network of this model to explicitly model the global image interdependence. Secondly, in the shortcut connection of the dashed residual cells, an average ensemble layer is added for the downsampling operation. By fine-tuning the operation of the residual module, the mapping between the input features is enhanced, so that the extracted expression features can be passed between the networks more completely, so as to reduce the loss of feature information. Finally, the convolutional block attention module (CBAM) attention mechanism module is passed into the network again to enhance the channel dimension information and spatial dimension information of local expression features, strengthen the focus information of feature regions with high relevance to expressions in the feature map, and suppress the interference of irrelevant regions in the feature map, thus speeding up the convergence speed of the network and improving the expression recognition rate. Compared with the baseline algorithm, this method achieves 87.65% and 73.57% accuracy on the RAF-DB and FER2013 expression datasets, respectively. [ABSTRACT FROM AUTHOR]

Facial expression recognition is an important research direction of emotion computing and has broad application prospects in human-computer interaction. However, noise such as illumination and occlusion in natural environment brings many challenges to facial expression recognition. In order to solve the problems such as low recognition rate of facial expression in natural environment, unable to highlight the characteristics of facial expression in global facial research, and misclassification caused by the similarity between negative expressions. In this paper, a multi-region coordinate attentional residual expression recognition model (MrCAR) is proposed. The model is mainly composed of the following three parts: 1) multi-region input: MTCNN is used for face detection and alignment processing, and the eyes and mouth parts are further cropped to obtain multi-region pictures. Through multi-region input, local details and global features are more easily obtained, which reduces the influence of complex environmental noise and highlights the facial features. 2) Feature extraction module: On the basis of residual element, CA-Net and multi-scale convolution were added to obtain coordinate residual attention module, through which the model’s ability to distinguish subtle changes of expression and the utilization rate of key features were improved; 3) Classifier: Arcface Loss is used to enhance intra-class tightness and inter-class difference at the same time, thus reducing the wrong classification of negative expressions by the model. Finally, the accuracy rates of CK+, JAFFE, FER2013 and RAF-DB were 98.78%, 99.09%, 74.50% and 88.26%, respectively. The experimental results show that compared with many advanced models, the MrCAR model in this paper is more competent for the task of expression classification.

Article Highlights 1. In this paper, we describe a way to make a new face expression dataset by putting together open source and self-collected datasets. 2. This allows us to create the largest face expression dataset in the industry, which is better in quality and has more data. 3. When we use this new dataset to train a face expression recognition model, it performs the best in the current industry.

A single network model can't extract more complex and rich effective features. Meanwhile, the network structure is usually huge, and there are many parameters and consume more space resources, etc. Therefore, the combination of multiple network models to extract complementary features has attracted extensive attention. In order to solve the problems existing in the prior art that the network model can't extract high spatial depth features, redundant network structure parameters, and weak generalization ability, this paper adopts two models of Xception module and inverted residual structure to build the neural network. Based on this, a face expression recognition method based on improved depthwise separable convolutional network is proposed in the paper. Firstly, Gaussian filtering is performed by Canny operator to remove noise, and combined with two original pixel feature maps to form a three-channel image. Secondly, the inverted residual structure of MobileNetV2 model is introduced into the network structure. Finally, the extracted features are classified by Softmax classifier, and the entire network model uses ReLU6 as the nonlinear activation function. The experimental results show that the recognition rate is 70.76% in Fer2013 dataset (facial expression recognition 2013) and 97.92% in CK+ dataset (extended Cohn Kanade). It can be seen that this method not only effectively mines the deeper and more abstract features of the image, but also prevents network over-fitting and improves the generalization ability. [ABSTRACT FROM AUTHOR]