Search

This study presents an improved Facial Expression Recognition (FER) model using Swin transformers for enhanced performance in detecting mental health through facial emotion analysis. In addition, some techniques involving better dropout and layer-wise unfreezing were implemented to reduce model overfitting. This study evaluates the proposed models on benchmark datasets such as FER2013 and CK+ and real-time Genius HR data. Model A has no dropout layer, Model B has focal loss, and Model C has enhanced dropout and layer-wise unfreezing. Model C was the best among all proposed models, achieving test accuracies of 71.23% on FER2013 and 78.65% on CK+. Weighted cross-entropy loss and image augmentation were used to handle class imbalance. Based on Model C emotion predictions, a scoring mechanism was designed to analyze employees' mental health for the next 30 days. The higher the score, the higher the risk of mental health. This study demonstrates a practical version of the Swin transformer in FER models for detecting and early mental health intervention. [ABSTRACT FROM AUTHOR]

Facial Expression Recognition (FER) is crucial for human-computer interaction and has achieved satisfactory results on lab-collected datasets. However, occlusion and head pose variation in the real world make FER extremely challenging due to facial information deficiency. This paper proposes a novel Dual Stream Attention Network (DSAN) for occlusion and head pose robust FER. Specifically, DSAN consists of a Global Feature Element-based Attention Network (GFE-AN) and a Multi-Feature Fusion-based Attention Network (MFF-AN). A sparse attention block and a feature recalibration loss designed in GFE-AN selectively emphasize feature elements meaningful for facial expression and suppress those unrelated to facial expression. And a lightweight local feature attention block is customized in MFF-AN to extract rich semantic information from different representation sub-spaces. In addition, DSAN takes into account computation overhead minimization when designing model architecture. Extensive experiments on public benchmarks demonstrate that the proposed DSAN outperforms the state-of-the-art methods with 89.70% on RAF-DB, 89.93% on FERPlus, 65.77% on AffectNet-7, 62.13% on AffectNet-8. Moreover, the parameter size of DSAN is only 11.33M, which is lightweight compared to most of the recent in-the-wild FER algorithms. [ABSTRACT FROM AUTHOR]

Facial expression is the most direct way for human to express emotion and intention, and facial expression recognition (FER) plays a crucial role in machine intelligence. Currently, two common ways to process images includes: viewing facial images as Euclidean structures in space and spliting the images into patches represented with a continuous sequence. However, faces in images are usually irregular, and are not flexible enough to be processed in the form of grid or sequence. Meanwhile, discrete features such as background tend to have a negative impact and interfere with FER when their features are extracted. To address the above problems, an attentional visual graph neural network based FER method (AT-ViG) is proposed in this paper. First, AT-ViG represents the input face images as graphs, and applies pixel-based composition methods to process the input images, to solve the problem of inflexibility data processing methods. Second, AT-ViG employs an attention mechanism to enhance feature extraction, to minimize the influence of irrelevant features such as background and hair in facial images. This further highlights the features that are relevant to FER and suppresses irrelevant features, where the model can effectively capture the correlations between different facial regions, reduce redundant information and improve the accuracy and robustness of expression recognition. Finally, the AT-ViG is experimentally validated on three commonly used expression recognition datasets and achieved the accuracies of 99.60%, 92.03%, 74.35% and 88.15% in CK+, Oulu-CASIA NIR &VIS, FER2013 and RAF-DB, respectively. [ABSTRACT FROM AUTHOR]

Automatic Facial Expression Recognition (FER) systems provide an important way to express and interpret the emotional and mental states of human beings. These FER systems transform the facial image into a set of features to train a classifier capable of distinguishing between different classes of emotions. However, the problem often posed is that the extracted feature vectors sometimes contain irrelevant or redundant features, which decreases the accuracy of the induced classifier and increases the computation time. To overcome this problem, dimensionality must be reduced by selecting only the most relevant features. In this paper, we study the impact of adding the "Wrapper" selection approach and using the information provided by different local regions of the face such as the mouth, eyes and eyebrows, on the performance of a traditional FER system based on a local geometric feature-extraction method. The objective here is to test and analyze how this combination can improve the overall performance of the original traditional system. The obtained results, based on the Multimedia Understanding Group (MUG) database, showed that the FER system combined with the proposed feature-selection strategy gives better classification results than the original system for all four classification models; namely, K-Nearest Neighbor (KNN) classifier, Tree classifier, NB classifier and Linear Discriminant Analysis (LDA). Indeed, a considerable reduction (up to 50%) in the number of features used and an accuracy of 100%, using the LDA classifier, were observed, which represents a significant improvement in terms of computation time, efficiency and memory space. Furthermore, the majority of relevant features used are part of the "eyebrows' region", which proves the importance of using information from local regions of the face in emotion recognition tasks. [ABSTRACT FROM AUTHOR]

With the rapid development of sensors and sensor technology, facial expression recognition (FER) systems can be developed and applied to real-world scenarios. Vision scan sensors and ambient light sensors capture clear and noise-free images of faces. However, in the real world, annotating large facial expressions is challenging owing to inconsistent labels, which are caused by the annotators' subjectivity and the facial expressions' ambiguity. Moreover, current studies present limitations when addressing facial expression differences due to the gender gap. We not only rely on visual sensors for FER but also utilize nonvisual sensors. Therefore, in this paper, we propose a self-cure dual-branch network (SC-DBN) for FER, which automatically prevents deep networks from overfitting ambiguous samples. First, on the basis of SC-DBN, a two-branch training method is designed, taking full advantage of the gender information. Furthermore, a self-attention mechanism highlights the essential samples and weights, each with a regular weighting. Finally, a relabeling module is used to modify the labels of these samples in inconsistent labels. Many experiments on public datasets show that SC-DBN can effectively integrate gendered information and self-cure networks to improve performance. [ABSTRACT FROM AUTHOR]

Abstract Facial expression recognition (FER) has advanced applications in various disciplines, including computer vision, Internet of Things, and artificial intelligence, supporting diverse domains such as medical escort services, learning analysis, fatigue detection, and human-computer interaction. The accuracy of these systems is of utmost concern and depends on effective feature selection, which directly impacts their ability to accurately detect facial expressions across various poses. This research proposes a new hybrid approach called QIFABC (Hybrid Quantum-Inspired Firefly and Artificial Bee Colony Algorithm), which combines the Quantum-Inspired Firefly Algorithm (QIFA) with the Artificial Bee Colony (ABC) method to enhance feature selection for a multi-pose facial expression recognition system. The proposed algorithm uses the attributes of both the QIFA and ABC algorithms to enhance search space exploration, thereby improving the robustness of features in FER. The firefly agents initially move toward the brightest firefly until identified, then search transition to the ABC algorithm, targeting positions with the highest nectar quality. In order to evaluate the efficacy of the proposed QIFABC algorithm, feature selection is also conducted using QIFA, FA, and ABC algorithms. The evaluated features are utilized for classifying face expressions by utilizing the deep neural network model, ResNet-50. The presented FER system has been tested using multi-pose facial expression benchmark datasets, including RaF (Radboud Faces) and KDEF (Karolinska Directed Emotional Faces). Experimental results show that the proposed QIFABC with ResNet50 method achieves an accuracy of 98.93%, 94.11%, and 91.79% for front, diagonal, and profile poses on the RaF dataset, respectively, and 98.47%, 93.88%, and 91.58% on the KDEF dataset.

Abstract More virtual reality (VR) scenarios have become more prevalent in recent years. More and more people are getting into VR, meaning that objective physiological measures to assess a user's emotional state automatically are becoming more critical. Individuals’ emotional states impact their behaviour, opinions, emotions, and decisions. They may be used to analyze VR experiences and make systems react to and engage with the user’s emotions. VR environments require users to wear head-mounted displays (HMDs), blocking off their upper faces. That makes traditional Facial Expression Recognition (FER) approaches very limited in their usefulness. Thus, a Deep Learning (DL) solution combined with image processing is utilized to classify universal emotions: sadness, happiness, disgust, anger, fear and surprise. Hence, this paper suggests the Deep Automatic Facial Expression Recognition Model (DAFERM) for interactive virtual reality (VR) applications such as intelligent education, social networks, and virtual training. Two main parts comprise the system: one that uses deep neural networks (DNNs) for facial emotion identification and another that automatically tracks and segments faces. The system begins by following a marker on the front of the head-mounted display (HMD). With the help of the spatial data that has been retrieved, the positions and rotations of the face are estimated to segment the mouth. Finally, the system interacts with DNN using the pixels processed by the lips. It obtains the facial expression results in real time using an adaptive method for histogram-based mouth segmentation.

Abstract Existing facial expression recognition (FER) methods are mainly devoted to learning discriminative features from normal-light images. However, their performance drops sharply when they are used for low-light images. In this paper, we propose a novel low-light FER framework (termed LL-FER) that can simultaneously enhance the images and recognition tasks of low-light facial expression images. Specifically, we first meticulously design a low-light enhancement network (LLENet) to recover expressions images’ rich detail information. Then, we design a joint loss to train the LLENet with FER network in a cascade manner, so that the FER network can guide the LLENet to gradually perceive and restore discriminative features which are useful for FER during the training process. Extensive experiments show that the LLENet not only achieves competitive results both quantitatively and qualitatively, but also in the LL-FER framework, which can produce results more suitable for FER tasks, further improving the performance of the FER methods.

Abstract Noisy labels are unavoidable in facial expression recognition (FER) task, significantly hindering FER performance in real-world scenarios. Recent advances tackle this problem by leveraging uncertainty for sample partitioning or constructing label distributions. However, these approaches primarily depend on labels, leading to confirmation bias issues and performance degradation. We argue that mining both label-independent features and label-dependent information can mitigate the confirmation bias induced by noisy labels. In this paper, we propose MCR, that is, mining simple yet effective label-free consistency regularization (MCR) to learn robust representations against noisy labels. The proposed MCR incorporates three label-free consistency regularizations: instance-level embedding consistency regularization, pairwise distance consistency regularization, and neighbour consistency regularization. Initially, we employ instance-level embedding consistency regularization to learn instance-level discriminative information from identical facial samples under perturbations in an unsupervised manner. This facilitates the efficacy of mitigating inherent noise in data. Subsequently, a pairwise distance consistency regularization is constructed to regularize the classifier and alleviate bias induced by noisy labels. Finally, we use the neighbour consistency regularization to further strengthen the discriminative capability of the model against noise. Benefiting from the advantages of these three label-free consistency regularizations, MCR can learn discriminative and robust representations against noise. Extensive experimental results demonstrate the superior performance of MCR on three popular in-the-wild facial expression datasets, including RAF-DB, FERPlus, and AffectNet. Moreover, MCR demonstrates superior generalization capability on other datasets with noisy labels, such as CIFAR100 and Tiny-ImageNet.

Abstract By recognizing students’ facial expressions in actual classroom situations, the students’ emotional states can be quickly uncovered, which can help teachers grasp the students’ learning rate, which allows teachers to adjust their teaching strategies and methods, thus improving the quality and effectiveness of classroom teaching. However, most previous facial expression recognition methods have problems such as missing key facial features and imbalanced class distributions in the dateset, resulting in low recognition accuracy. To address these challenges, this paper proposes LCANet, a model founded on a fused attention mechanism and a joint loss function, which allows the recognition of students’ emotions in real classroom scenarios. The model uses ConvNeXt V2 as the backbone network to optimize the global feature extraction capability of the model, and at the same time, it enables the model to pay closer attention to the key regions in facial expressions. We incorporate an improved Channel Spatial Attention (CSA) module as a way to extract more local feature information. Furthermore, to mitigate the class distribution imbalance problem in the facial expression dataset, we introduce a joint loss function. The experimental results show that our LCANet model has good recognition rates on both the public emotion datasets FERPlus, RAF-DB and AffectNet, with accuracies of 91.43%, 90.03% and 64.43%, respectively, with good robustness and generalizability. Additionally, we conducted experiments using the model in real classroom scenarios, detecting and accurately predicting students’ classroom emotions in real time, which provides an important reference for improving teaching in smart teaching scenarios.

Abstract Continuous emotion recognition is to predict emotion states through affective information and more focus on the continuous variation of emotion. Fusion of electroencephalography (EEG) and facial expressions videos has been used in this field, while there are with some limitations in current researches, such as hand‐engineered features, simple approaches to integration. Hence, a new continuous emotion recognition model is proposed based on the fusion of EEG and facial expressions videos named residual multimodal Transformer (RMMT). Firstly, the Resnet50 and temporal convolutional network (TCN) are utilised to extract spatiotemporal features from videos, and the TCN is also applied to process the computed EEG frequency power to acquire spatiotemporal features of EEG. Then, a multimodal Transformer is used to fuse the spatiotemporal features from the two modalities. Furthermore, a residual connection is introduced to fuse shallow features with deep features which is verified to be effective for continuous emotion recognition through experiments. Inspired by knowledge distillation, the authors incorporate feature‐level loss into the loss function to further enhance the network performance. Experimental results show that the RMMT reaches a superior performance over other methods for the MAHNOB‐HCI dataset. Ablation studies on the residual connection and loss function in the RMMT demonstrate that both of them is functional.

To suppress the effects such as occlusions and posture variations on facial expression recognition in natural scenes, expression recognition network fusing global enhancement and local attention features (GE-LA) is proposed. Firstly, to acquire the enhanced global context information, an enhancement structure of channel-spatial global features is constructed, which uses channel flow module (CFM) and spatial flow module (SFM) to obtain symmetric multi-scale channel semantics and pixel-level spatial semantics, respectively, and combines these two types of semantics to generate global enhanced features. Secondly, to extract local detail features, an efficient channel attention (ECA) mechanism is improved to channel-spatial attention (CSA) mechanism, and a local attention module (LAM) is constructed based on this to obtain channel and spatial high-level semantics. Finally, to enhance the anti-interference ability of the proposed network against factors such as occlusions and posture variations, an adaptive strategy is designed to obtain the weighted fusion of global enhancement features and local attention features, and to achieve expression classification based on the adaptive fusion features. Experimental results on facial expression datasets RAF-DB and FERPlus in natural scenes show that the expression recognition rates of the proposed network are 89.82% and 89.93%, respectively, which are 13.39 percentage points and 10.62 percentage points higher than the baseline network ResNet50. Compared with the related methods, the proposed method, which reduces the influence of occlusions and posture variations, has a better expression recognition performance in natural scenes.

Facial expression recognition is vital for human behavior analysis, and deep learning has enabled models that can outperform humans. However, it is unclear how closely they mimic human processing. This study aims to explore the similarity between deep neural networks and human perception by comparing twelve different networks, including both general object classifiers and FER-specific models. We employ an innovative global explainable AI method to generate heatmaps, revealing crucial facial regions for the twelve networks trained on six facial expressions. We assess these results both quantitatively and qualitatively, comparing them to ground truth masks based on Friesen and Ekman's description and among them. We use Intersection over Union (IoU) and normalized correlation coefficients for comparisons. We generate 72 heatmaps to highlight critical regions for each expression and architecture. Qualitatively, models with pre-trained weights show more similarity in heatmaps compared to those without pre-training. Specifically, eye and nose areas influence certain facial expressions, while the mouth is consistently important across all models and expressions. Quantitatively, we find low average IoU values (avg. 0.2702) across all expressions and architectures. The best-performing architecture averages 0.3269, while the worst-performing one averages 0.2066. Dendrograms, built with the normalized correlation coefficient, reveal two main clusters for most expressions: models with pre-training and models without pre-training. Findings suggest limited alignment between human and AI facial expression recognition, with network architectures influencing the similarity, as similar architectures prioritize similar facial regions. [ABSTRACT FROM AUTHOR]

For solving the facial expression recognition (FER) problem, we introduce a novel feature extractor called the coordinate-based neighborhood attention mechanism (CNAM), which uses the coordinate attention (CA) method to capture the directional relationships in separate horizontal and vertical directions, the input features from a preprocessing unit, and then passes this to two residual blocks, one consisting of the neighborhood attention (NA) mechanism, which captures the local interaction of features within the neighborhood of a feature vector, while the other one contains a channel attention implemented by a multilayer perceptron (MLP). We apply the feature extractor, the CNAM module, to four FER benchmark datasets, namely, RAF-DB, AffectNet(7cls), AffectNet(8cls), and CK+, and through qualitative and quantitative analysis techniques, we conclude that the insertion of the CNAM module could decrease the intra-cluster distances and increase the inter-cluster distances among the high-dimensional feature vectors. The CNAM compares well with other state-of-the-art (SOTA) methods, being the best-performing method for the AffectNet(7cls) and CK+ datasets, while for the RAF-DB and AffectNet(8cls) datasets, its performance is among the top-performing SOTA methods. [ABSTRACT FROM AUTHOR]

Facial expressions exhibit inherent similarities, variability, and complexity. In real-world scenarios, challenges such as partial occlusions, illumination changes, and individual differences further complicate the task of facial expression recognition (FER). To further improve the accuracy of FER, a Multi-head Attention Affinity and Diversity Sharing Network (MAADS) is proposed in this paper. MAADS comprises a Feature Discrimination Network (FDN), an Attention Distraction Network (ADN), and a Shared Fusion Network (SFN). To be specific, FDN first integrates attention weights into the objective function to capture the most discriminative features by using the proposed sparse affinity loss. Then, ADN employs multiple parallel attention networks to maximize diversity within spatial attention units and channel attention units, which guides the network to focus on distinct, non-overlapping facial regions. Finally, SFN deconstructs facial features into generic parts and unique parts, which allows the network to learn the distinctions between these features without having to relearn complete features from scratch. To validate the effectiveness of the proposed method, extensive experiments were conducted on several widely used in-the-wild datasets including RAF-DB, AffectNet-7, AffectNet-8, FERPlus, and SFEW. MAADS achieves the accuracy of 92.93%, 67.14%, 64.55%, 91.58%, and 62.41% on these datasets, respectively. The experimental results indicate that MAADS not only outperforms current state-of-the-art methods in recognition accuracy but also has a relatively low computational complexity. [ABSTRACT FROM AUTHOR]

Deep metric-based center loss has been widely used to enhance inter-class separability and intra-class compactness of network features and achieved promising results in facial expression recognition (FER) recently. However, existing center loss does not take the complexity of expression samples into consideration, which deteriorates the representativeness of the generated center vectors resulting in suboptimal performance. To solve this problem, we propose a novel complexity aware center loss for FER. Specifically, a multi-category division module is firstly devised to distinguish simple samples and difficult samples for each category based on the entropy value of sample prediction results. Then, an exact representative center module is employed on simple samples to generate a more representative center vector for each category by encouraging greater differences between different categories. Finally, an adaptive distance adjustment module is proposed to reduce the interference of difficult samples in the model learning process to further improve the accuracy of FER by maintaining a suitable distance between difficult samples and their corresponding center vector. Extensive experimental results on two benchmark datasets demonstrate the effectiveness, universality and superiority of our methods. The code will be available at https://github.com/sanjiaobo/CACL. [ABSTRACT FROM AUTHOR]

In studying the joint object detection and classification problem for facial expression recognition (FER) deploying the YOLOX framework, we introduce a novel feature extractor, called neighborhood coordinate attention Mamba (NCAMamba) to substitute for the original feature extractor in the Feature Pyramid Network (FPN). NCAMamba combines the background information reduction capabilities of Mamba, the local neighborhood relationship understanding of neighborhood attention, and the directional relationship understanding of coordinate attention. The resulting FER-YOLO-NCAMamba model, when applied to two unaligned FER benchmark datasets, RAF-DB and SFEW, obtains significantly improved mean average precision (mAP) scores when compared with those obtained by other state-of-the-art methods. Moreover, in ablation studies, it is found that the NCA module is relatively more important than the Visual State Space (VSS), a version of using Mamba for image processing, and in visualization studies using the grad-CAM method, it reveals that regions around the nose tip are critical to recognizing the expression; if it is too large, it may lead to erroneous prediction, while a small focused region would lead to correct recognition; this may explain why FER of unaligned faces is such a challenging problem. [ABSTRACT FROM AUTHOR]

In smart classroom environments, accurately recognizing students' facial expressions is crucial for teachers to efficiently assess students' learning states, timely adjust teaching strategies, and enhance teaching quality and effectiveness. In this paper, we propose a student facial expression recognition model based on multi-scale and deep fine-grained feature attention enhancement (SFER-MDFAE) to address the issues of inaccurate facial feature extraction and poor robustness of facial expression recognition in smart classroom scenarios. Firstly, we construct a novel multi-scale dual-pooling feature aggregation module to capture and fuse facial information at different scales, thereby obtaining a comprehensive representation of key facial features; secondly, we design a key region-oriented attention mechanism to focus more on the nuances of facial expressions, further enhancing the representation of multi-scale deep fine-grained feature; finally, the fusion of multi-scale and deep fine-grained attention-enhanced features is used to obtain richer and more accurate facial key information and realize accurate facial expression recognition. The experimental results demonstrate that the proposed SFER-MDFAE outperforms the existing state-of-the-art methods, achieving an accuracy of 76.18% on FER2013, 92.75% on FERPlus, 92.93% on RAF-DB, 67.86% on AffectNet, and 93.74% on the real smart classroom facial expression dataset (SCFED). These results validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

To better apply deep convolutional neural networks for expression recognition in UAV-enabled B5G/6G networks, we propose a deep network expression recognition method based on transfer learning and fine-tuning on facial expression datasets. Initially, we train our model on a large-scale facial attribute dataset and subsequently fine-tune it on a facial expression dataset. This strategy effectively lowers the high costs and dependence associated with annotating facial expression datasets, while enhancing the accuracy and training speed of our model. This method is particularly suited for UAVs equipped with on-board cameras and image processing capabilities, enabling real-time expression recognition for various applications such as crowd monitoring, search and rescue, and human-UAV interaction. Compared to traditional methods that train exclusively on facial expression datasets, our method significantly reduces the number of training iterations on facial expression datasets and significantly improves the generalization ability of the model, especially for UAV applications. We use a large-scale facial attribute dataset, which is more closely related to the facial expression recognition task, as our source dataset, forming a contrast with methods that typically use a facial recognition dataset as the transfer learning source dataset. The experimental results on the CK + dataset, integrated with UAV-enabled B5G/6G networks, show that our method achieves a facial expression recognition accuracy of 97.6%, a significant improvement over the 97.3% accuracy rate of methods that only train on facial expression datasets, with less training time as well. [ABSTRACT FROM AUTHOR]

Objective: To investigate the accuracy of facial emotion recognition in the Iranian community, a face database validated in this community is required. To do this, we conducted a validation study on the Radboud face database. The primary objective of this study was to evaluate the accuracy of recognizing emotions through faces in an Iranian sample and then to choose the pictures with high agreement in terms of detecting emotions. Method: This is a cross sectional study recruiting a total number of 142 males and females aged between 20 and 50 years old (Mean ± SD of age 31.7 ± 7.07). The participants were instructed to detect the type of emotion of each face as well as its valence and arousal. The percentage of participants' agreement on evaluating each picture was assessed. To evaluate the effect of different variables on participants' accuracy, one way and repeated measure ANOVA analyses were also used. Results: Emotional faces were recognized by around 84% of the participants. The highest accuracy belongs to happy (Mean ± SD of 98 ± 6.1%) and the lowest one to neutral (75 ± 18.06%) faces. The accuracy for detecting other emotions were as follows: sad (91 ± 8.7%), surprised (87 ± 10.64%), angry (77 ± 15.6%), and fearful (76 ± 15.26%). Additionally, we found no differences between male and female participants in terms of recognizing emotions. Then we selected the pictures with high agreement (above 85 percent) in labeling emotions among the participants. Conclusion: The current study provided a valid emotional face database based on Iranian participants' responses in terms of recognizing basic emotions. The selected pictures can be used in designing tasks to evaluate emotion recognition ability in clinical and nonclinical populations. It can also be used in designing applications to improve detecting emotion in clinical samples such as individuals with autism spectrum disorder. [ABSTRACT FROM AUTHOR]

Video‐based facial expression recognition (VFER) technique intends to categorize an input video into different kinds of emotions. It remains a challenging issue because of the gap between visual features and emotions, problems in handling the delicate movement of muscles, and restricted datasets. One of the effective solutions to solve this problem is the exploitation of efficient features defining facial expressions to carry out FER. Generally, the VFER find useful in several areas like unmanned driving, venue management, urban safety management, and senseless attendance. Recent advances in computer vision and deep learning (DL) techniques enable the design of automated VFER models. In this aspect, this study establishes a new Marine Predators Optimization with Deep Learning Model for Video‐based Facial Expression Recognition (MPODL‐VFER) technique. The presented MPODL‐VFER technique mainly aims to classify different kinds of facial emotions in the video. To accomplish this, the presented MPODL‐VFER technique derives features using the deep convolutional neural network based densely connected network (DenseNet) model. The presented MPODL‐VFER technique employs MPO technique for the hyperparameter adjustment of the DenseNet model. Finally, Elman Neural Network (ENN) model is exploited for emotion recognition purposes. For assuring the enhanced recognition performance of the MPODL‐VFER approach, a comparison study was developed on benchmark dataset. The comprehensive results have shown the significant outcome of MPODL‐VFER model over other approaches. [ABSTRACT FROM AUTHOR]

The research on technologies and methodologies for (accurate, real-time, spontaneous, three-dimensional...) facial expression recognition is ongoing and has been fostered in the past decades by advances in classification algorithms like deep learning, which makes them part of the Artificial Intelligence literature. Still, despite its upcoming application to contexts such as human–computer interaction, product and service design, and marketing, only a few literature studies have investigated the willingness of end users to share their facial data with the purpose of detecting emotions. This study investigates the level of awareness and interest of 373 potential consumers towards this technology in the car insurance sector, particularly in the contract drafting phase, with a focus on differentiating the respondents between generation Y and Z. Results show that younger people, individuals with higher levels of education, and social network users feel more confident about this innovative technology and are more likely to share their expressive facial data. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) holds significant practical implications in real-world scenarios such as human–computer interaction, fatigue driving detection, and learning engagement analysis. Nonetheless, acquiring large-scale and high-quality annotated facial expression datasets is profoundly challenging due to the inherent ambiguity of facial images and concerns over privacy. Consequently, this paper introduces a self-supervised facial expression recognition method based on mask image modeling. This method can learn multi-level facial feature representations without expensive labels and achieves commendable facial expression recognition performance through further fine-grained feature selection. Specifically, we propose the multi-level feature selector (MFS). The MFS comprises two pivotal components: the multi-level feature combiner and the feature selector. During the pre-training stage, the multi-level feature combiner is employed to integrate multi-level features, effectively addressing the vision transformer's deficiencies in capturing high-frequency facial semantics. Subsequently, in the fine-tuning stage, the feature selector can automatically differentiate highly discriminative regions, extracting fine-grained features. Subsequently, we use graph convolutional networks to further mine the latent connections among fine-grained features, ultimately deriving an integrated feature with enhanced discriminative capabilities. Through such fine-grained facial feature selection, we can mitigate performance degradation induced by inter-class similarities and intra-class variations. Experimental results on the RAF-DB, AffectNet, and FER + datasets demonstrate that our approach significantly outperforms other self-supervised methods in recognition performance and closely approaches the state-of-the-art methods in supervised learning. The code is available at https://github.com/Greysahy/MFS. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) plays a crucial role in various applications, including human–computer interaction and affective computing. However, the joint training of an FER network with multiple datasets is a promising strategy to enhance its performance. Nevertheless, widespread annotation inconsistencies and class imbalances among FER datasets pose significant challenges to this approach. This paper proposes a new multi-dataset joint training method, Sample Selection and Paired Augmentation Joint Training (SSPA-JT), to address these challenges. SSPA-JT models annotation inconsistency as a label noise problem and selects clean samples from auxiliary datasets to expand the overall dataset size while maintaining consistent annotation standards. Additionally, a dynamic matching algorithm is developed to pair clean samples of the tail class with noisy samples, which enriches the tail classes with diverse background information. Experimental results demonstrate that SSPA-JT achieved superior or comparable performance compared with the existing methods by addressing both annotation inconsistencies and class imbalance during multi-dataset joint training. It achieved state-of-the-art performance on RAF-DB and CAER-S datasets with accuracies of 92.44% and 98.22%, respectively, reflecting improvements of 0.2% and 3.65% over existing methods. [ABSTRACT FROM AUTHOR]

This paper proposes a facial expression recognition network called the Lightweight Facial Network with Spatial Bias (LFNSB). The LFNSB model effectively balances model complexity and recognition accuracy. It has two key components: a lightweight feature extraction network (LFN) and a Spatial Bias (SB) module for aggregating global information. The LFN introduces combined channel operations and depth-wise convolution techniques, effectively reducing the number of parameters while enhancing feature representation capability. The Spatial Bias module enables the model to focus on local facial features while capturing the dependencies between different facial regions. Additionally, a new loss function called Cosine-Harmony Loss is designed. This function optimizes the relative positions of feature vectors in high-dimensional space, resulting in better feature separation and clustering. Experimental results on the AffectNet and RAF-DB datasets demonstrate that the LFNSB model achieves competitive recognition accuracy, with 63.12% accuracy on AffectNet-8, 66.57% accuracy on AffectNet-7, and 91.07% accuracy on RAF-DB, while significantly reducing the model complexity. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) has caught the research community's attention recently because it can affect many real-life applications. Multiple studies have focused on automatic FER, most of which use a machine learning methodology, FER has continued to be a difficult and exciting issue in computer vision. Deep learning has recently drawn increased attention as a solution to several practical issues, including facial expression recognition. This article introduces an efficient method for FER (FERDCNN) verified on five different pre-trained deep CNN (DCNN) models (AlexNet, GoogleNet, ResNet-18, ResNet-50, and ResNet-101). In the proposed method, firstly the input image has been pre-processed using face detection, resizing, gamma correction, and histogram equalization techniques. Secondly, the images go through DCNN to extract deep features. Finally, support vector machine (SVM) and transfer learning are used to classify generated features. Recent methods have been employed to evaluate and contrast the performance of the proposed approach on two publicly standard databases namely, CK+ and JAFFE on the seven classes of fundamental emotions, including anger, disgust, fear, happiness, sadness, and surprise beside neutrality for CK+ and contempt for JAFFE. The suggested method tested Four different traditional supervised classifiers with deep features, Experimental found that AlexNet excels as a feature extractor, while SVM demonstrates superiority as a classifier because of this combination achieving the highest accuracy rates of 99.0% and 95.16% for the CK+ database and the JAFFE datasets, respectively. [ABSTRACT FROM AUTHOR]

Continuous emotion recognition is to predict emotion states through affective information and more focus on the continuous variation of emotion. Fusion of electroencephalography (EEG) and facial expressions videos has been used in this field, while there are with some limitations in current researches, such as hand‐engineered features, simple approaches to integration. Hence, a new continuous emotion recognition model is proposed based on the fusion of EEG and facial expressions videos named residual multimodal Transformer (RMMT). Firstly, the Resnet50 and temporal convolutional network (TCN) are utilised to extract spatiotemporal features from videos, and the TCN is also applied to process the computed EEG frequency power to acquire spatiotemporal features of EEG. Then, a multimodal Transformer is used to fuse the spatiotemporal features from the two modalities. Furthermore, a residual connection is introduced to fuse shallow features with deep features which is verified to be effective for continuous emotion recognition through experiments. Inspired by knowledge distillation, the authors incorporate feature‐level loss into the loss function to further enhance the network performance. Experimental results show that the RMMT reaches a superior performance over other methods for the MAHNOB‐HCI dataset. Ablation studies on the residual connection and loss function in the RMMT demonstrate that both of them is functional. [ABSTRACT FROM AUTHOR]

Dynamic ensemble pruning (DEP) can select the best set of models for an unseen sample to improve a classification system's prediction accuracy and robustness. However, little work has been done to apply DEP to the facial expression recognition (FER) system and reduce DEP's computational burden. This paper proposes a semi-dynamic ensemble pruning algorithm (SDEP) to implement the FER task. First, a subspace-based classifier sequences selection method is proposed to break DEP's neighborhood construction limitation and avoid the exhaustive similarity calculation. Moreover, a pre-prediction mechanism is introduced to calculate the relationship between an unseen sample and data subspaces. Ultimately, the calculated relationship scores and their credibility will be used as the weights for fusion. SDEP reduces the DEP's computational burden while retaining the dynamic characteristics through changing the classifier selection into the subspace-based classifier sequences combination. SDEP demonstrates superior classification accuracy and efficiency to several state-of-the-art dynamic and static EP algorithms in benchmark FER datasets Fer2013, JAFFE, CK+ using a homogeneous ensemble of 799 CNN classifiers. [ABSTRACT FROM AUTHOR]

Facial expression recognition using convolutional neural networks (CNNs) is a prevalent research area, and the network's complexity poses obstacles for deployment on devices with limited computational resources, such as mobile devices. To address these challenges, researchers have developed lightweight networks with the aim of reducing model size and minimizing parameters without compromising accuracy. The LiteFer method introduced in this study incorporates depth-separable convolution and a lightweight attention mechanism, effectively reducing network parameters. Moreover, through comprehensive comparative experiments on the RAFDB and FERPlus datasets, its superior performance over various state-of-the-art lightweight expression-recognition methods is evident. [ABSTRACT FROM AUTHOR]

Automatic pain intensity estimation from facial expression analysis has important applications in medical and healthcare areas. Most of the existing works tend to directly transfer the typical face recognition models to pain estimation task, which may not obtain good performances because the facial expression of pain is spontaneous with subtle facial variations. Pain estimation from facial video is still challenging because it relies on modeling semantic parts and extraction of fine-grained and dynamic features. In this study, we propose a hierarchical global and local transformer (HGLT) model for pain estimation from facial expression videos. HGLT model consists of an image frame embedding subnetwork and a temporal embedding subnetwork for extraction of spatio-temporal features. In the frame embedding subnetwork, we propose a multi-head local attention mechanism to extract the local fine-grained features related to micro variations of pain, followed by a hierarchical self-attention pooling to integrate the global and local features. In the temporal embedding subnetwork, a transformer encoder with temporal attention is proposed to model the temporal relationships of video frames and capture the dynamic facial variations. A correlation loss is proposed to alleviate the problem of long-tailed imbalance in the distribution of pain intensities. Our proposed method is tested with UNBC-McMaster Shoulder Pain, BioVid Heart Pain dataset, and DAiSEE dataset. Experimental results indicate that our method achieves competitive performances compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) in the wild is one of the most challenging visual tasks owing to various uncontrolled factors such as occlusion, pose, and subtle variation in real scenes. These factors can directly affect the robust performance of current networks, especially as most single-feature learning space methods lack the extraction of potential discriminative features and fail to provide a deeper understanding of expressions. To address the above issues, we propose a novel hybrid attention-aware learning network (HALNet), which comprises a feature compactness network (FCN), a hybrid attention enhancement network (HAEN), and a joint loss optimization strategy. First, FCN performs basic expression feature extraction and optimizes intra- and inter-class distributions simultaneously. Afterward, HAEN constructs a multi-level feature enhancement space by fusing hybrid attention based on CNN and transformer in parallel to effectively improve the profound understanding of expressions. Finally, the expression classification is performed by supervised optimization with joint loss. Extensive experiments are assessed on some of the widest employed wild expression datasets, and results indicate our method is superior to several present state-of-the-art methods, obtaining accuracies of 90.29%, 90.04%, and 61.75% on RAF-DB, FERPlus, and AffectNet, respectively. The cross-dataset and occlusion and pose variation datasets assessment further substantiate our approach's sound generalization and robustness. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) plays a vital role for applications based on human–computer interaction. In the past few years, many deep learning models have been proposed for FER, but their performance is limited due to challenges such as variation in head pose, occlusion, illumination, etc. Moreover, existing models consider input image through holistic view without giving attention to features relevant to expressions. In this work, we propose a deep fusion framework for FER which employs self-attention mechanism to resolve this issue. Furthermore, to improve feature representation of the images, the proposed model transforms the input image to wavelet domain through discrete wavelet transform. The framework employs two parallel branches for shallow and deep features which are fused for better feature representation. The proposed model is evaluated on posed and in-the-wild datasets: CK+, JAFFE, MUG, YALE, RAF, and experimental results validate the effectiveness of the model. [ABSTRACT FROM AUTHOR]

In this paper, we study facial expression recognition (FER) using three modalities obtained from a light field camera: sub-aperture (SA), depth map, and all-in-focus (AiF) images. Our objective is to construct a more comprehensive and effective FER system by investigating multimodal fusion strategies. For this purpose, we employ EfficientNetV2-S, pre-trained on AffectNet, as our primary convolutional neural network. This model, combined with a BiGRU, is used to process SA images. We evaluate various fusion techniques at both decision and feature levels to assess their effectiveness in enhancing FER accuracy. Our findings show that the model using SA images surpasses state-of-the-art performance, achieving 88.13% ± 7.42% accuracy under the subject-specific evaluation protocol and 91.88% ± 3.25% under the subject-independent evaluation protocol. These results highlight our model's potential in enhancing FER accuracy and robustness, outperforming existing methods. Furthermore, our multimodal fusion approach, integrating SA, AiF, and depth images, demonstrates substantial improvements over unimodal models. The decision-level fusion strategy, particularly using average weights, proved most effective, achieving 90.13% ± 4.95% accuracy under the subject-specific evaluation protocol and 93.33% ± 4.92% under the subject-independent evaluation protocol. This approach leverages the complementary strengths of each modality, resulting in a more comprehensive and accurate FER system. [ABSTRACT FROM AUTHOR]

Achieving safe collaboration between humans and robots in an industrial work-cell requires effective communication. This can be achieved through a robot perception system developed using data-driven machine learning. The challenge for human–robot communication is the availability of extensive, labelled datasets for training. Due to the variations in human behaviour and the impact of environmental conditions on the performance of perception models, models trained on standard, publicly available datasets fail to generalize well to domain and application-specific scenarios. Thus, model personalization involving the adaptation of such models to the individual humans involved in the task in the given environment would lead to better model performance. A novel framework is presented that leverages robust modes of communication and gathers feedback from the human partner to auto-label the mode with the sparse dataset. The strength of the contribution lies in using in-commensurable multimodes of inputs for personalizing models with user-specific data. The personalization through feedback-enabled human–robot communication (PF-HRCom) framework is implemented on the use of facial expression recognition as a safety feature to ensure that the human partner is engaged in the collaborative task with the robot. Additionally, PF-HRCom has been applied to a real-time human–robot handover task with a robotic manipulator. The perception module of the manipulator adapts to the user's facial expressions and personalizes the model using feedback. Having said that, the framework is applicable to other combinations of multimodal inputs in human–robot collaboration applications. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) has been recognised as a promising technology for methodological progress and theoretical advancement in learning sciences. However, there remains few empirical investigations into how AI could be applied in learning sciences research. This study aims to utilize AI facial recognition to inform the learning regulation behaviors in synchronous online collaborative learning environments. By studying groups of university students (N = 36) who participated in their online classes under the COVID-19 social distancing mandates, we strive to understand the interrelation between individual affective states and their collaborative group members. Theoretically underpinned by the socially shared regulation of learning framework, our research features a cutting-edge insight into how learners socially shared regulation in group-based tasks. Findings accentuate fundamental added values of AI application in education, whilst indicating further interesting patterns about student self-regulation in the collaborative learning environment. Implications drawn from the study hold strong potential to provide theoretical and practical contributions to the exploration of AI supportive roles in designing and personalizing learning needs, as well as fathom the motion and multiplicity of collaborative learning modes in higher education. [ABSTRACT FROM AUTHOR]

To solve the problem of poor recognition accuracy due to the complex network structure in the facial expression recognition algorithm, an improved lightweight network based on FasterNet is proposed: FGENet. Firstly, the PConv in the FasterNet block module is replaced with the RFE module to enhance the network's ability to perceive key features of facial expressions. Secondly, the GSConv module is introduced into the residual branch, which helps to strengthen the feature modeling of different regions and improves the expression feature extraction ability of the network. Finally, the ECA attention mechanism is introduced into the Ghost module to design the deep attention mechanism DEA, which is applied to the standard convolution after global pooling to better capture the global context information. FGENet is designed to keep the network lightweight while improving performance. Through a series of experiments, it has been proven that FGENet has achieved performance improvements on the three data sets of Fer2013, CK+ and RAF, reaching recognition accuracy rates of 70.49%, 97.89% and 86.72% respectively, further verifying its effectiveness in facial expression recognition tasks. [ABSTRACT FROM AUTHOR]

Copyright of Ordnance Industry Automation is the property of Editorial Board for Ordnance Industry Automation and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The development of facial expression recognition (FER) and facial expression generation (FEG) systems is essential to enhance human-robot interactions (HRI). The facial action coding system is widely used in FER and FEG tasks, as it offers a framework to relate the action of facial muscles and the resulting facial motions to the execution of facial expressions. However, most FER and FEG studies are based on measuring and analyzing facial motions, leaving the facial muscle component relatively unexplored. This study introduces a novel framework using surface electromyography (sEMG) signals from facial muscles to recognize facial expressions and estimate the displacement of facial keypoints during the execution of the expressions. For the facial expression recognition task, we studied the coordination patterns of seven muscles, expressed as three muscle synergies extracted through non-negative matrix factorization, during the execution of six basic facial expressions. Muscle synergies are groups of muscles that show coordinated patterns of activity, as measured by their sEMG signals, and are hypothesized to form the building blocks of human motor control. We then trained two classifiers for the facial expressions based on extracted features from the sEMG signals and the synergy activation coefficients of the extracted muscle synergies, respectively. The accuracy of both classifiers outperformed other systems that use sEMG to classify facial expressions, although the synergy-based classifier performed marginally worse than the sEMG-based one (classification accuracy: synergy-based 97.4%, sEMG-based 99.2%). However, the extracted muscle synergies revealed common coordination patterns between different facial expressions, allowing a low-dimensional quantitative visualization of the muscle control strategies involved in human facial expression generation. We also developed a skin-musculoskeletal model enhanced by linear regression (SMSM-LRM) to estimate the displacement of facial keypoints during the execution of a facial expression based on sEMG signals. Our proposed approach achieved a relatively high fidelity in estimating these displacements (NRMSE 0.067). We propose that the identified muscle synergies could be used in combination with the SMSM-LRM model to generate motor commands and trajectories for desired facial displacements, potentially enabling the generation of more natural facial expressions in social robotics and virtual reality.

Facial expressions are the most basic non-verbal method people use to communicate feelings, intentions and reactions without words. Recognizing these facial expressions accurately is essential for a variety of applications — such as tools that use our faces to interact with computers (human-computer interaction, or HCI), security systems and emotionally intelligent artificial intelligence technologies. As the complexities surrounding these relationships have become better understood, it has allowed us to develop increasingly more complex systems for identifying and detecting facial expressions of different emotions. This paper presents an improved performance of the Facial Expression Recognition (FER) systems via augmentation in Artificial Neural Networks and Genetic Algorithms, two renowned artificial intelligence techniques possessing disparate strengths. ANNS are inspired by the neural architecture of human brain capable of learning and recognizing patterns in unchartered data after trained examples, on the other hand GAs come from fundamental principles underlying natural selection perform optimization process based-on evolutionary methods which includes fitness evaluation, comparison, selection, crossover, and mutation. The research is an effort to mitigate the problems pertaining with conventional methods, like overfitting and generalization fault in order design FER model which has potential for performing much more accurately. A hybrid ANN-GA model that uses Petri Nets and production systems is proposed for the real-time video sequence analysis with high precision in predicting different dynamic facial activities of anger, surprise, disgust, joy, sadness and fear from emotion faces. Importantly, results on the study show that this integrated model has a large-scale promoting effect in emotion detection upon varied scenes and is therefore generalizable to many domains from security and surveillance over biomedicine up to interactive AI-driven systems. Implications for implementing real-time and context-aware recognition of human emotions based on AI technologies are far-reaching as they demonstrate the potential that hybrid AI systems offer at enhancing emotion deciphering.

Facial expression recognition is important for many applications, including sentiment analysis, human-computer interaction, and interactive systems in areas such as security, healthcare, and entertainment. However, this task is fraught with challenges, mainly due to large differences in lighting conditions, viewing angles, and differences in individual eye structures. These factors can drastically affect the appearance of facial expressions, making it difficult for traditional recognition systems to consistently and accurately identify emotions. Variations in lighting can alter the visibility of facial features, while different angles can obscure critical details necessary for accurate expression detection. This study addresses these issues by employing transfer learning with ResNet-50 and effective pre-processing techniques. The dataset consists of grayscale images with a 48 x 48 pixels resolution. It includes a total of 680 samples categorized into seven classes: anger, contempt, disgust, fear, happy, sadness, and surprise. The dataset was divided so that 80% was allocated for training and 20% for testing to ensure robust model evaluation. The results demonstrate that the model utilizing transfer learning achieved an exceptional performance level, with accuracy at 99.49%, precision at 99.49%, recall at 99.71%, and an F1-score of 99.60%, significantly outperforming the model without transfer learning. Future research will focus on implementing real-time facial recognition systems and exploring other advanced transfer learning models to further enhance accuracy and operational efficiency.

In synchronous online lectures, the lecturers often provide the lecture material directly through video conference technology. On the other hand, there are many students who do not pay attention to the lecturers when they are participating in online lectures. As a consequence, in this research, an application was developed to assist lecturers in gathering data regarding the degree to which students who participate in online lectures pay attention to the presented information. The application employed a convolutional neural network (CNN) model to recognize each student's facial expressions and place them into one of two classes: either engaged or disengaged. The captured student facial image was preprocessed to facilitate the classification process. The preprocessing stage consisted of image conversion to gray scale, face detection using the Haar-Cascade Classifier model, and a median filter to reduce noise. In the process of designing a CNN model, three different hyperparameter tuning scenarios were implemented. These tuning scenarios aimed to obtain the best possible CNN model by determining which CNN model hyperparameters were the most optimal. The results of the experiments indicate that the CNN model from the second scenario has the highest level of accuracy in terms of recognizing facial expressions, coming in at 86%. The results of this research have been tested to measure the level of student participation in online lectures. The trial results show that the proposed application can help lecturers evaluate student engagement during online lectures.

Facial expression recognition (FER) plays a crucial role in affective computing, enhancing human-computer interaction by enabling machines to understand and respond to human emotions. Despite advancements in deep learning, current FER systems often struggle with challenges such as occlusions, head pose variations, and motion blur in natural environments. These challenges highlight the need for more robust FER solutions. To address these issues, we propose the Attention-Enhanced Multi-Layer Transformer (AEMT) model, which integrates a dual-branch Convolutional Neural Network (CNN), an Attentional Selective Fusion (ASF) module, and a Multi-Layer Transformer Encoder (MTE) with transfer learning. The dual-branch CNN captures detailed texture and color information by processing RGB and Local Binary Pattern (LBP) features separately. The ASF module selectively enhances relevant features by applying global and local attention mechanisms to the extracted features. The MTE captures long-range dependencies and models the complex relationships between features, collectively improving feature representation and classification accuracy. Our model was evaluated on the RAF-DB and AffectNet datasets. Experimental results demonstrate that the AEMT model achieved an accuracy of 81.45% on RAF-DB and 71.23% on AffectNet, significantly outperforming existing state-of-the-art methods. These results indicate that our model effectively addresses the challenges of FER in natural environments, providing a more robust and accurate solution. The AEMT model significantly advances the field of FER by improving the robustness and accuracy of emotion recognition in complex real-world scenarios. This work not only enhances the capabilities of affective computing systems but also opens new avenues for future research in improving model efficiency and expanding multimodal data integration. [ABSTRACT FROM AUTHOR]

In recent years, significant progress has been made in facial expression recognition methods. However, tasks related to facial expression recognition in real environments still require further research. This paper proposes a tri-cross-attention transformer with a multi-feature fusion network (TriCAFFNet) to improve facial expression recognition performance under challenging conditions. By combining LBP (Local Binary Pattern) features, HOG (Histogram of Oriented Gradients) features, landmark features, and CNN (convolutional neural network) features from facial images, the model is provided with a rich input to improve its ability to discern subtle differences between images. Additionally, tri-cross-attention blocks are designed to facilitate information exchange between different features, enabling mutual guidance among different features to capture salient attention. Extensive experiments on several widely used datasets show that our TriCAFFNet achieves the SOTA performance on RAF-DB with 92.17%, AffectNet (7 cls) with 67.40%, and AffectNet (8 cls) with 63.49%, respectively. [ABSTRACT FROM AUTHOR]