Search

Drawing parallels from the TV series Suits, this paper investigates the role of emotional intelligence (EQ) in fostering positive workplace culture, specifically within academic libraries. While the high-octane world of corporate law might seem disparate from the calm corridors of academic libraries, the interpersonal dynamics and challenges faced are strikingly similar. Central to this discussion is the recurring motif in Suits: the power of dialogue. Through characters like Harvey Specter and Donna Paulsen, the show emphasizes that open communication can be the key to resolution, even in high-pressure environments. Moreover, with the introduction of "The Donna," a unique AI tool, Suits provides a lens to examine the interplay between technology and EQ, raising questions about the true essence of human communication and its irreplaceability. This paper explores these themes, offering practical strategies for academic libraries to bolster EQ, enhance user experience, and ensure technology complements, rather than replaces, genuine human interaction. [ABSTRACT FROM AUTHOR]

Customer care is important for its role in relationship building. This role has traditionally been performed by human customer agents; however, the emergence of interactive generative AI (GenAI) shows potential for using AI for customer care in emotionally charged interactions. Bridging practice and the academic literatures in marketing and computer science, this article develops an AI-enabled customer care journey, from accurate emotion recognition to empathetic response, emotional management support, and, finally, the establishment of an emotional connection. Marketing requirements for each of the stages are derived from in-depth interviews with top managers and a survey of chief marketing officers. By juxtaposing these requirements against the current feeling capabilities of GenAI, the authors highlight the technological challenges engineers must tackle. The article concludes with a set of marketing tenets for implementing and researching the caring machine. These include verifying emotion recognition accuracy using marketing emotion theories through multiple emotion signals and methods, utilizing prompt engineering to enhance GenAI's emotion understanding, employing "response engineering" to personalize emotion management recommendations, and strategically deploying GenAI for emotional connection to simultaneously enhance customer emotional well-being and customer lifetime value. [ABSTRACT FROM AUTHOR]

For the purpose of improving the user experience of mobile smart terminals for the elderly, accessibility tools have become an indispensable component of intelligent system frameworks to meet the varying needs of senior citizens for assistance. Nonetheless, the complexity of accessibility assistive tools complicates the lives of some elderly with a lack of cognitive experience to use these tools independently. Our purpose is to study the relationship between the accessibility design elements of smartphones, the emotions, and touch behaviors of elderly users, and to optimize the accessibility design methods of mobile smart terminals. First, we obtained the four interface elements with the most influence on users through a Semantic Differential evaluation involving 50 participants. Second, in a touch event collection experiment, 20 participants' touch data was collected, and an SVM-based correlation model between interface elements, user emotions, and touch behaviors was established. Third, the prototype of the assistive recommender was iterated employing the combination of the correlation model and Genetic Algorithms. Ultimately, the effectiveness of the six touch media recommendations was compared through usability testing. The experimental results indicates that: (1) Emotion-aware accessibility assistive recommendation systems based on haptic recognition can enhance the elderly's ability to access information through mobile terminals. (2) Middle-aged and elderly users have a more robust negative emotion reflection for dragging and swiping on touch screens. (3) Negative emotions of elderly users can assist identify design defects of accessibility tools. Our research work distilled a set of design suggestions for digital accessibility improvement, thereby enhancing the usability and inclusiveness of assistive tools in diverse contexts, and reducing the psychological pressure caused by unfamiliar interfaces. This study supplies a reference for improving the emotional experience of elderly people employing mobile terminals and extending the approach to accessible design. [ABSTRACT FROM AUTHOR]

This paper introduces a neural network‐based model designed for classifying emotional states by leveraging multimodal physiological signals. The model utilizes data from the AMIGOS and SEED‐V databases. The AMIGOS database integrates inputs from electroencephalogram (EEG), electrocardiogram (ECG), and galvanic skin response (GSR) to analyze emotional responses, while the SEED‐V database continuously updates EEG signals. We implemented a sequential neural network architecture featuring two hidden layers, which underwent substantial hyperparameter tuning to achieve optimal performance. Our model's effectiveness was tested through binary classification tasks focusing on arousal and valence, as well as a more complex four‐class classification that delineates emotional quadrants for the emotional tags: happy, sad, neutral, and disgust. In these varied scenarios, the model consistently demonstrated accuracy levels ranging from 79% to 86% in the AMIGOS database and up to 97% in SEED‐V. A notable aspect of our approach is the model's ability to accurately recognize emotions without the need for extensive signal preprocessing, a common challenge in multimodal emotion analysis. This feature enhances the practical applicability of our model in real‐world scenarios where rapid and efficient emotion recognition is essential. [ABSTRACT FROM AUTHOR]

Automatic emotion recognition is a burgeoning field of research and has its roots in psychology and cognitive science. This article comprehensively reviews multimodal emotion recognition, covering various aspects such as emotion theories, discrete and dimensional models, emotional response systems, datasets, and current trends. This article reviewed 179 multimodal emotion recognition literature papers from 2017 to 2023 to reflect on the current trends in multimodal affective computing. This article covers various modalities used in emotion recognition based on the emotional response system under four categories: subjective experience comprising text and self‐report; peripheral physiology comprising electrodermal, cardiovascular, facial muscle, and respiration activity; central physiology comprising EEG, neuroimaging, and EOG; behavior comprising facial, vocal, whole‐body behavior, and observer ratings. This review summarizes the measures and behavior of each modality under various emotional states. This article provides an extensive list of multimodal datasets and their unique characteristics. The recent advances in multimodal emotion recognition are grouped based on the research focus areas such as emotion elicitation strategy, data collection and handling, the impact of culture and modality on multimodal emotion recognition systems, feature extraction, feature selection, alignment of signals across the modalities, and fusion strategies. The recent multimodal fusion strategies are detailed in this article, as extracting shared representations of different modalities, removing redundant features from different modalities, and learning critical features from each modality are crucial for multimodal emotion recognition. This article summarizes the strengths and weaknesses of multimodal emotion recognition based on the review outcome, along with challenges and future work in multimodal emotion recognition. This article aims to serve as a lucid introduction, covering all aspects of multimodal emotion recognition for novices. This article is categorized under:Fundamental Concepts of Data and Knowledge > Human Centricity and User InteractionTechnologies > Cognitive ComputingTechnologies > Artificial Intelligence [ABSTRACT FROM AUTHOR]

As the world evolves, human-AI teams (HAT) have become increasingly more capable in their ability to complete task objectives. Due to this rising importance, it has become essential to understand the interpersonal dynamism between humans and AI to further optimise their performance potential. Given the demonstrated utility of emotional communication within human-human team structures, this research investigates the nature of AI-sourced positive emotions on human teammates. Through 47 interviews, our findings show that for these AI teammates to be accepted, human teammates have preferences on understanding the emotional utility prior to its presentation, as well as which emotions are situationally acceptable. Also, findings show that integrating emotions within AI teammates has a positive influence on human perceptions and behaviour in a task. In further detail, emotions act as status updates that allow human teammates to not only better understand their teammates' mental states but also understand how their AI teammates perceive the situation around them. Together, this gives insight into how AI emotional expressions influence the perception of social support on the wider Human-AI team. Mainly how emotions can be used to increase acceptance of AI teammates and improve the overall experience human teammates have within the task. [ABSTRACT FROM AUTHOR]

Automatic Speech Emotion Recognition (SER) plays a vital role in making human–computer interactions more natural and effective. A significant challenge in SER development is the limited availability of diverse emotional speech datasets, which hinders the application of advanced deep learning models. Transfer learning is a machine learning technique that helps address this issue by utilizing knowledge from pre-trained models to improve performance on a new task in a target domain, even with limited data. This study investigates the use of transfer learning from various pre-trained networks, including speaker embedding models such as d-vector, x-vector, and r-vector, and image classification models like AlexNet, GoogLeNet, SqueezeNet, ResNet-18, and ResNet-50. We also propose enhanced versions of the x-vector and r-vector models incorporating Multi-Head Attention Pooling and Angular Margin Softmax, alongside other architectural improvements. Additionally, reverberation from the Room Impulse Response datasets was added to the speech utterances to diversify and augment the available data. Notably, the enhanced r-vector model achieved classification accuracies of 74.05% Unweighted Accuracy (UA) and 73.68% Weighted Accuracy (WA) on the IEMOCAP dataset, and 80.25% UA and 79.81% WA on the CREMA-D dataset, outperforming the existing state-of-the-art methods. This study shows that using cross-domain transfer learning is beneficial for low-resource emotion recognition. The enhanced models developed in other domains (for non-emotional tasks) can further improve the accuracy of SER. [ABSTRACT FROM AUTHOR]

AbstractAffect-adaptive video games are increasingly concerned with changing game material based on the emotional states of player to best elicit the optimal experience. However, conceptual uncertainties and unreliable methodologies make this process currently difficult and results inconclusive. Here we present a methodological approach to explicitly test mapping assumptions between in-game variables, player input, and subjective experience to build affect-adaptive video games. In Experiment 1, 161 participants played through a newly developed 2D action arcade game. Their data was used to identify associations between game input, presented material, and affective valence. Results made it possible to identify two potential pathways for emotion adaptation which were implemented and evaluated in Experiment 2 (N = 158) against a non-adaptive version of the same game. The player-based emotional adaptation showed a significant increase in self-reported valence compared to the control condition, providing evidence for the utility of the process. [ABSTRACT FROM AUTHOR]

WEMAC is a unique open multi-modal dataset that comprises physiological, speech, and self-reported emotional data records of 100 women, targeting Gender-based Violence detection. Emotions were elicited through visualizing a validated video set using an immersive virtual reality headset. The physiological signals captured during the experiment include blood volume pulse, galvanic skin response, and skin temperature. The speech was acquired right after the stimuli visualization to capture the final traces of the perceived emotion. Subjects were asked to annotate among 12 categorical emotions, several dimensional emotions with a modified version of the Self-Assessment Manikin, and liking and familiarity labels. The technical validation proves that all the targeted categorical emotions show a strong statistically significant positive correlation with their corresponding reported ones. That means that the videos elicit the desired emotions in the users in most cases. Specifically, a negative correlation is found when comparing fear and not-fear emotions, indicating that this is a well-portrayed emotional dimension, a specific, though not exclusive, purpose of WEMAC towards detecting gender violence [ABSTRACT FROM AUTHOR]

Contactless physiological signal measurement has great applications in various fields, such as affective computing and health monitoring. Physiological measurements based on remote photoplethysmography (rPPG) are realized by capturing the weak periodic color changes. The changes are caused by the variation in the light absorption of skin surface during systole and diastole stages of a functioning heart. This measurement mode has advantages of contactless measurement, simple operation, low cost, etc. In recent years, several deep learning-based rPPG measurement methods have been proposed. However, the features learned by deep learning models are vulnerable to motion and illumination artefacts, and are unable to fully exploit the intrinsic temporal characteristics of the rPPG. This paper presents an efficient spatiotemporal modeling-based rPPG recovery method for physiological signal measurements. First, two modules are utilized in the rPPG task: 1) 3D central difference convolution for temporal context modeling with enhanced representation and generalization capacity, and 2) Huber loss for robust intensity-level rPPG recovery. Second, a dual branch structure for both motion and appearance modeling and a soft attention mask are adapted to take full advantage of the central difference convolution. Third, a multi-task setting for joint cardiac and respiratory signals measurements is introduced to benefit from the internal relevance between two physiological signals. Last, extensive experiments performed on three public databases show that the proposed method outperforms prior state-of-the-art methods with the Pearson's correlation coefficient higher than 0.96 on all three datasets. The generalization ability of the proposed method is also evaluated by cross-database and video compression experiments. The effectiveness and necessity of each module are confirmed by ablation studies. [ABSTRACT FROM AUTHOR]

Electroencephalography (EEG)-based Brain-Computer Interface (BCI) systems for emotion recognition have the potential to assist the enrichment of human–computer interaction with implicit information since they can enable understanding of the cognitive and emotional activities of humans. Therefore, these systems have become an important research topic today. This study aims to present trends and gaps on this topic by performing a systematic literature review based on the 216 published scientific literature gathered from various databases including ACM, IEEE Xplore, PubMed, Science Direct, and Web of Science from 2016 to 2020. This review gives an overview of all the components of EEG based BCI system from the signal stimulus module which includes the employed device, signal stimuli, and data processing modality, to the signal processing module which includes signal acquisition, pre-processing, feature extraction, feature selection, classification algorithms, and performance evaluation. Thus, this study provides an overview of all components of an EEG-based BCI system for emotion recognition and examines the available evidence in a clear, concise, and systematic way. In addition, the findings are aimed to inform researchers about the issues on what are research trends and the gaps in this field and guide them in their research directions. [ABSTRACT FROM AUTHOR]

Emotions significantly impact human thinking, judgment, health, and communication. EEG-based emotion detection has advanced with the use of Brain-Computer Interface (BCI) technology, proving more effective than other physiological data. Despite progress in affective computing, emotion recognition remains a challenge. However, it's increasingly common in brain–machine interfaces, and research shows EEG brain waves are valuable in identifying emotional states. This research introduces a novel automated system for emotion recognition using deep learning techniques on EEG data collected from the GAMEEMO dataset, where participants played emotional assessment games. The system is designed to identify four emotions experienced during gameplay. The proposed model, called EEGER, was trained exclusively on EEG signals and demonstrated a 99.99% accuracy with minimal computational time. Key to its efficiency is the use of LSTM (Long Short-Term Memory) classifiers, which simplify the process by automatically extracting relevant features. The system was tested across different learning rates and epoch values, showing that 10 epochs with a learning rate of 0.0001 were sufficient to achieve the best accuracy. EEGER was also compared with other methods like K-Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), and Adaptive Boosting (AdaBoost), outperforming them in both accuracy and efficiency. These findings suggest that EEGER offers a promising new approach to EEG-based emotion recognition, optimizing performance with lower complexity and computation time. [ABSTRACT FROM AUTHOR]

Emotion recognition in user‐generated videos plays an essential role in affective computing. In general, visual information directly affects human emotions, so the visual modality is significant for emotion recognition. Most classic approaches mainly focus on local temporal information of videos, which potentially restricts their capacity to encode the correlation of long‐range context. To address this issue, a novel network is proposed to recognize emotions in videos. To be specific, a spatio‐temporal correlation‐aware block is designed to depict the long‐range correlations between input tokens, where the convolutional layers are used to learn the local correlations and the inter‐image cross‐attention is designed to learn the long‐range and spatio‐temporal correlations between input tokens. To generate diverse and challenging samples, a dual‐augmentation fusion layer is devised, which fuses each frame with its corresponding frame in the temporal domain. To produce rich video clips, a long‐range sampling layer is designed, which generates clips in a wide range of spatial and temporal domains. Extensive experiments are conducted on two challenging video emotion datasets, namely VideoEmotion‐8 and Ekman‐6. The experimental results demonstrate that the proposed method obtains better performance than baseline methods. Moreover, the proposed method achieves state‐of‐the‐art results on the two datasets. The source code of the proposed network is available at: https://github.com/JinChow/LRCANet. [ABSTRACT FROM AUTHOR]

Brain-Computer Interface (BCI)-based affect recognition in humans utilizing physiological signals is typically guided by a supervised method that relies heavily on the presence of appropriate affect representations. In this paper, we assess the effectiveness of contemporary affect representation methodologies for establishing ground truth for reliable affective BCI systems. On the DEAP dataset, two experiments were conducted to compare the observed emotion clusters based on subjective ratings with the clustering of corresponding physiological responses based on EEG characteristics. The outcomes of the two investigations are inconsistent with one another. The findings suggest the subjective use of emotion categories as opposed to their general application in establishing ground truth for supervised approaches and must encourage researchers to examine affect representation techniques critically before considering them as concrete representations of a person's affective state. [ABSTRACT FROM AUTHOR]

Images are capable of conveying emotions, but emotional experience is highly subjective. Advances in artificial intelligence have enabled the generation of images based on emotional descriptions. However, the level of agreement between the generative images and human emotional responses has not yet been evaluated. In order to address this, 20 artistic landscapes were generated using StyleGAN2-ADA. Four variants evoking positive emotions (contentment and amusement) and negative emotions (fear and sadness) were created for each image, resulting in 80 pictures. An online questionnaire was designed using this material, in which 61 observers classified the generated images. Statistical analyses were performed on the collected data to determine the level of agreement among participants between the observers' responses and the generated emotions by AI. A generally good level of agreement was found, with better results for negative emotions. However, the study confirms the subjectivity inherent in emotional evaluation. [ABSTRACT FROM AUTHOR]

Facial emotion recognition (FER) represents a significant outcome of the rapid advancements in artificial intelligence (AI) technology. In today's digital era, the ability to decipher emotions from facial expressions has evolved into a fundamental mode of human interaction and communication. As a result, FER has penetrated diverse domains, including but not limited to medical diagnosis, customer feedback analysis, the automation of automobile driver systems, and the evaluation of student comprehension. Furthermore, it has matured into a captivating and dynamic research field, capturing the attention and curiosity of contemporary scholars and scientists. The primary objective of this paper is to provide an exhaustive review of FER systems. Its significance goes beyond offering a comprehensive resource; it also serves as a valuable guide for emerging researchers in the FER domain. Through a meticulous examination of existing FER systems and methodologies, this review equips them with essential insights and guidance for their future research pursuits. Moreover, this comprehensive review contributes to the expansion of their knowledge base, facilitating a profound understanding of this rapidly evolving field. In a world increasingly dependent on technology for communication and interaction, the study of FER holds a pivotal role in human‐computer interaction (HCI). It not only provides valuable insights but also unlocks a multitude of possibilities for future innovations and applications. As we continue to integrate AI and facial emotion recognition into our daily lives, the importance of comprehending and enhancing FER systems becomes increasingly evident. This paper serves as a stepping stone for researchers, nurturing their involvement in this exciting and ever‐evolving field. [ABSTRACT FROM AUTHOR]

Inter-species emotional relationships, particularly the symbiotic interaction between humans and dogs, are complex and intriguing. Humans and dogs share fundamental mammalian neural mechanisms including mirror neurons, crucial to empathy and social behavior. Mirror neurons are activated during the execution and observation of actions, indicating inherent connections in social dynamics across species despite variations in emotional expression. This study explores the feasibility of using deep-learning Artificial Intelligence systems to accurately recognize canine emotions in general environments, to assist individuals without specialized knowledge or skills in discerning dog behavior, particularly related to aggression or friendliness. Starting with identifying key challenges in classifying pleasant and unpleasant emotions in dogs, we tested advanced deep-learning techniques and aggregated results to distinguish potentially dangerous human--dog interactions. Knowledge transfer is used to fine-tune different networks, and results are compared on original and transformed sets of frames from the Dog Clips dataset to investigate whether DogFACS action codes detailing relevant dog movements can aid the emotion recognition task. Elaborating on challenges and biases, we emphasize the need for bias mitigation to optimize performance, including different image preprocessing strategies for noise mitigation in dog recognition (i.e., face bounding boxes, segmentation of the face or body, isolating the dog on a white background, blurring the original background). Systematic experimental results demonstrate the system's capability to accurately detect emotions and effectively identify dangerous situations or signs of discomfort in the presence of humans. [ABSTRACT FROM AUTHOR]

Social deficits in schizophrenia have been attributed to an impaired attunement to mutual interaction, or "interaffectivity". While impairments in emotion recognition and facial expressivity in schizophrenia have been consistently reported, findings on mimicry and social synchrony are inconsistent, and previous studies have often lacked ecological validity. To investigate interaffective behavior in dyadic interactions in a real-world-like setting, 20 individuals with schizophrenia and 20 without mental disorder played a cooperative board game with a previously unacquainted healthy control participant. Facial expression analysis was conducted using Affectiva Emotion AI in iMotions 9.3. The contingency and state space distribution of emotional facial expressions was assessed using Mangold INTERACT. Psychotic symptoms, subjective stress, affectivity and game experience were evaluated through questionnaires. Due to a considerable between-group age difference, age-adjusted ANCOVA was performed. Overall, despite an unchanged subjective experience of the social interaction, individuals with schizophrenia exhibited reduced responsiveness to positive affective stimuli. Subjective game experience did not differ between groups. Descriptively, facial expressions in schizophrenia were generally more negative, with increased sadness and decreased joy. Facial mimicry was impaired specifically regarding joyful expressions in schizophrenia, which correlated with blunted affect as measured by the SANS. Dyadic interactions involving persons with schizophrenia were less attracted toward mutual joyful affective states. Only unadjusted for age, in the absence of emotional stimuli from their interaction partner, individuals with schizophrenia showed more angry and sad expressions. These impairments in interaffective processes may contribute to social dysfunction in schizophrenia and provide new avenues for future research. [ABSTRACT FROM AUTHOR]

Recently, zero-shot image captioning (ZSIC) has gained significant attention, given its potential to describe unseen objects in images. This is important for real-world applications such as human–computer interaction, intelligent education, and service robots. However, the zero-shot image captioning method based on large-scale pretrained models may generate descriptions containing objects that are not present in the image, which is a phenomenon termed "object hallucination". This is because large-scale models tend to predict words or phrases with high frequency, as seen in the training phase. Additionally, the method set a limitation to the description length, which often leads to an improper ending. In this paper, a novel approach is proposed to address and reduce the object hallucination and improper ending problem in the ZSIC task. We introduce additional emotion signals as guidance for sentence generation, and we find that proper emotion will filter words that do not appear in the image. Moreover, we propose a novel strategy that gradually extends the number of words in a sentence to confirm the generated sentence is properly completed. Experimental results show that the proposed method achieves the leading performance on unsupervised metrics. More importantly, the subjective examples illustrate the effect of our method in improving hallucination and generating properly ending sentences. [ABSTRACT FROM AUTHOR]

Music is a powerful language capable of eliciting a variety of emotions in individuals. Understanding and recognizing these emotions is pivotal for applications ranging from personalized music recommendations and music therapy to automatic music composition and affective computing. Presently, deep learning for music emotion recognition is gaining popularity, primarily relying on timbre features to capture local spatial information. However, there is an untapped potential in incorporating other pertinent audio features and global correlations in the feature space to capture the repetitive temporal information of music for emotion classification. This study introduces GlocalEmoNet as a method to capture both local and global correlations in music, utilizing timbre and Chroma audio features for tasks related to emotion classification and segmentation. The neural network underwent training and testing on approximately six thousand music audio samples, encompassing six music-emotion categories. The utilization of a genetic algorithm is employed for optimizing the hyperparameters of the proposed neural networks, aiming to attain optimal performance, efficiency, and generalization. The best classifier demonstrated superior performance, surpassing previously published results by a significant margin of approximately 14%. The optimal classifier achieved an accuracy score of 81.66%, an f1-score of 0.812, and an area under the curve score of 0.956. The evaluation of classification and segmentation outcomes also involved the use of visual representations. [ABSTRACT FROM AUTHOR]

The V-Cockpit platform aims to transform the design, testing, and validation of car infotainment systems from the physical realm to virtual reality. It uniquely integrates various aspects of the creative phases—from conceptualization to evaluation—streamlining the process and reducing time and costs compared to traditional methods that focus on individual activities and rely heavily on physical prototyping. This technical note provides a comprehensive overview of the main platform's aspects, highlighting the integration of hardware and behavioral analysis algorithms to improve user experience and detect potential design flaws early on. The V-Cockpit platform, composed of six key components, leverages virtual reality and digital twins, promising significant cost savings and enhanced design efficiency. This work details the system architecture, implementation, and first benefits of this innovative approach through the analysis of three use cases. [ABSTRACT FROM AUTHOR]

Human cognitive functions such as perception, attention, learning, memory, reasoning, and problem-solving are all significantly influenced by emotion. Emotion has a particularly potent impact on attention, modifying its selectivity in particular and influencing behavior and action motivation. Artificial Emotional Intelligence (AEI) technologies enable computers to understand a user's emotional state and respond appropriately. These systems enable a realistic dialogue between people and machines. The current generation of adaptive user interference technologies is built on techniques from data analytics and machine learning (ML), namely deep learning (DL) artificial neural networks (ANN) from multimodal data, such as videos of facial expressions, stance, and gesture, voice, and bio-physiological data (such as eye movement, ECG, respiration, EEG, FMRT, EMG, eye tracking). In this study, we reviewed existing literature based on ML and data analytics techniques being used to detect emotions in speech. The efficacy of data analytics and ML techniques in this unique area of multimodal data processing and extracting emotions from speech. This study analyzes how emotional chatbots, facial expressions, images, and social media texts can be effective in detecting emotions. PRISMA methodology is used to review the existing survey. Support Vector Machines (SVM), Naïve Bayes (NB), Random Forests (RF), Recurrent Neural Networks (RNN), Logistic Regression (LR), etc., are commonly used ML techniques for emotion extraction purposes. This study provides a new taxonomy about the application of ML in SER. The result shows that Long-Short Term Memory (LSTM) and Convolutional Neural Networks (CNN) are found to be the most useful methodology for this purpose. [ABSTRACT FROM AUTHOR]

This article explores the use of artificial intelligence (AI) in mental healthcare, specifically focusing on its impact on the therapist-patient relationship. AI technologies, such as chatbots, have the potential to provide personalized support and improve access to mental health services. However, there are concerns about transference, where patients may project their feelings onto the AI. It is important to carefully manage transference dynamics to avoid harm and maintain ethical standards. The development of AI systems that can recognize and appropriately respond to transference is necessary, and collaboration between AI developers and mental health professionals is crucial. Additionally, regulatory frameworks and ethical guidelines should be implemented to ensure responsible and effective use of AI in mental healthcare. Interdisciplinary collaboration, cultural sensitivity, and continuous monitoring and evaluation of AI interactions are also emphasized. [Extracted from the article]

Purpose: The purpose of this study is to investigate consumers' emotional responses to artificial intelligence (AI) defeating people. Meanwhile, the authors investigate the negative spillover effect of AI defeating people on consumers' attitudes toward AI companies. The authors also try to alleviate this spillover effect. Design/methodology/approach: Using four studies to test the hypotheses. In Study 1, the authors use the fine-tuned Bidirectional Encoder Representations from the Transformers algorithm to run a sentiment analysis to investigate how AI defeating people influences consumers' emotions. In Studies 2 to 4, the authors test the effect of AI defeating people on consumers' attitudes, the mediating effect of negative emotions and the moderating effect of different intentions. Findings: The authors find that AI defeating people increases consumers' negative emotions. In terms of downstream consequences, AI defeating people induces a spillover effect on consumers' unfavorable attitudes toward AI companies. Emphasizing the intention of helping people can effectively mitigate this negative spillover effect. Practical implications: The authors' findings remind governments, policymakers and AI companies to pay attention to the negative effect of AI defeating people and take reasonable steps to alleviate this negative effect. The authors help consumers rationally understand this phenomenon and correctly control and reduce unnecessary negative emotions in the AI era. Originality/value: This paper is the first study to examine the adverse effects of AI defeating humans. The authors contribute to research on the dark side of AI, the outcomes of competition matches and the method to analyze emotions in user-generated content (UGC). [ABSTRACT FROM AUTHOR]

As artificial intelligence (AI) integration becomes more prominent in society, the infosphere becomes increasingly complex. A better understanding of the human-AI relationship is crucial during these rapid technological and social changes. Emotions are complex responses of the human mind – a highly advanced information processing system. Thus, examining how humans perceive AI's emotional capability may provide helpful insights. Human physiological properties such as humanoid appearance and gender are suggested to be potential influencing factors. Following the information processing approach, we employed Bayesian Mindsponge Framework (BMF) analytics on a dataset of 266 residents of the United States. Results show that interacting with an AI agent with human-like physical features is positively associated with the belief in that AI agent's capability of experiencing emotional pain and pleasure. This effect is stronger in males than females. A possible explanation for these patterns is reliance on cognitive strategies used in interpersonal interactions, including gender-specific physiological properties. This study also demonstrates the effectiveness of using a compatible information processing analytical method such as BMF for studying humans' perceptions of AI. [ABSTRACT FROM AUTHOR]

This work introduces a revolutionary YOLOv9 deep learning architecture-based method for facial emotion identification. Happy, Sad, Angry, Disgust, Natural, and Surprise are the six emotional classifications the research divides the 21,263 images into which the research divides. The model was trained on 88% of the dataset; 4% was used for testing and 8% for validation. The preprocessing operations were auto-orientation, zoom, rotation, and 416x416 pixel scaling. Training time was cut down significantly because the experiment was carried out with a T4 GPU from Kaggle. Five epochs later, the average mAP score of 0.85, average precision of 0.74, and average recall of 0.84 indicate encouraging emotional performance. At a mAP score of 0.98, the model showed exceptionally high accuracy in identifying disgust. With its reliable and effective approach to real-time emotion recognition in various applications, this work advances the field of affective computing. [ABSTRACT FROM AUTHOR]

Speech emotion recognition (SER) is a vital component of the human–computer interaction system. The traditional Deep learning-based speech SER schemes show poor time-domain representation, class imbalance issues due to uneven samples in the training datasets, less feature distinctiveness, and inferior long-term dependency on global and local attributes of the speech. This article introduces lightweight, long short memory (LSTM) along with multiple acoustic features such as Mel frequency spectrum coefficients (MFCC), chroma, root mean square (RMS), and Tonnetz features for boosting the time-domain representation and long-term dependency of emotional speech. Further, it used data augmentation techniques such as noise addition, shifting, stretching, and feature extraction to lessen the class imbalance problem. The proposed methods have an average accuracy of 95% for SAVEE, 94.57% for EMOVO, 97.16% for EMODB, and 96.66% for BAVED, which has shown noteworthy improvement over the traditional state of arts. [ABSTRACT FROM AUTHOR]

In this work, we present a method for personalizing human-robot interaction by using emotive facial expressions to generate affective robot movements. Movement is an important medium for robots to communicate affective states, but the expertise and time required to craft new robot movements promotes a reliance on fixed preprogrammed behaviors. Enabling robots to respond to multimodal user input with newly generated movements could stave off staleness of interaction and convey a deeper degree of affective understanding than current retrieval-based methods. We use autoencoder neural networks to compress robot movement data and facial expression images into a shared latent embedding space. Then, we use a reconstruction loss to generate movements from these embeddings and triplet loss to align the embeddings by emotion classes rather than data modality. To subjectively evaluate our method, we conducted a user survey and found that generated happy and sad movements could be matched to their source face images. However, angry movements were most often mismatched to sad images. This multimodal data-driven generative method can expand an interactive agent's behavior library and could be adopted for other multimodal affective applications. [ABSTRACT FROM AUTHOR]

Multimodal speech emotion recognition can utilize features from different modalities simultaneously to improve the modeling capabilities in affective computing. However, the rough feature combining method may not effectively promote interaction and facilitate learning between different modalities. This paper proposes a novel multimodal Speech Emotion Recognition (SER) framework that maps two feature vectors from different modalities to the same feature space by a Relative Entropy Alignment (REA) mechanism, which facilitates the modalities to complement mutually in learning emotional representations. Specifically, we employ KLD (Kullback-Leibler Divergence) to perform the feature alignment, capturing the temporal correlation between audio and text while ensuring that the features of both modalities tend to align in the feature space, thereby alleviating modality conflicts. Meanwhile, we construct a Multi-level Attention Fusion (MAF) mechanism to capture the emotion representations from different modalities, facilitating information exchange between different modalities while mitigating some redundant features. Furthermore, we extract multi-level acoustic information by wavelet packet transform to enrich the audio modality's emotional features further. Experimental results on some multimodal emotion datasets, such as Interactive Emotional Dyadic Motion Capture (IEMOCAP) and Multi-modal Emotion Lines Dataset (MELD), demonstrate that our proposed method outperforms the state-of-the-art model. [ABSTRACT FROM AUTHOR]

Extended reality (XR) systems are about to be integrated into our daily lives and will provide support in a variety of fields such as education and coaching. Enhancing user experience demands agents that are capable of displaying realistic affective and social behaviors within these systems, and, as a prerequisite, with the capability of understanding their interaction partner and responding appropriately. Based on our literature review of recent works published in the field of co-speech gesture generation, researchers have developed complex models capable of generating gestures characterized by a high level of human-likeness and speaker appropriateness. Nevertheless, this is only true in settings where the agent has an active status (i.e., the agent acts as the speaker), or it is delivering a monologue in a non-interactive setting. However, as illustrated in multiple works and competitions like the GENEA Challenge, these models remain inadequate in generating interlocutor-aware gestures. We consider interlocutor-aware gesture generation the process of displaying gestures that take into account the conversation partner's behavior. Moreover, in settings where the agent is the listener, generated gestures lack the level of naturalness that we expect from a face-to-face conversation. To overcome these issues, we have designed a pipeline, called TAG2G, composed of a diffusion model, which was demonstrated to be a stable and powerful tool in gesture generation, and a vector-quantized variational auto-encoder (VQVAE), widely employed to produce meaningful gesture embeddings. Refocusing from monadic to dyadic multimodal input settings (i.e., taking into account text, audio, and previous gestures of both participants of a conversation) allows us to explore and infer the complex interaction mechanisms that lie in a balanced two-sided conversation. As per our results, a multi-agent conversational input setup improves the generated gestures' appropriateness with respect to the conversational counterparts. Conversely, when the agent is speaking, a monadic approach performs better in terms of the generated gestures' appropriateness in relation to the speech. [ABSTRACT FROM AUTHOR]

The dynamic expressions of emotion convey both the emotional and functional states of an individual's interactions. Recognizing the emotional states helps us understand human feelings and thoughts. Systems and frameworks designed to recognize human emotional states automatically can use various affective signals as inputs, such as visual, vocal and physiological signals. However, emotion recognition via a single modality can be affected by various sources of noise that are specific to that modality and the fact that different emotion states may be indistinguishable. This review examines the current state of multimodal emotion recognition methods that integrate visual, vocal or physiological modalities for practical emotion computing. Recent empirical evidence on deep learning methods used for fine-grained recognition is reviewed, with discussions on the robustness issues of such methods. This review elaborates on the profound learning challenges and solutions required for a high-quality emotion recognition system, emphasizing the benefits of dynamic expression analysis, which aids in detecting subtle micro-expressions, and the importance of multimodal fusion for improving emotion recognition accuracy. The literature was comprehensively searched via databases with records covering the topic of affective computing, followed by rigorous screening and selection of relevant studies. The results show that the effectiveness of current multimodal emotion recognition methods is affected by the limited availability of training data, insufficient context awareness, and challenges posed by real-world cases of noisy or missing modalities. The findings suggest that improving emotion recognition requires better representation of input data, refined feature extraction, and optimized aggregation of modalities within a multimodal framework, along with incorporating state-of-the-art methods for recognizing dynamic expressions. [ABSTRACT FROM AUTHOR]

Emotion artificial intelligence (AI) is shown to vary systematically in its ability to accurately identify emotions, and this variation creates potential biases. In this paper, we conduct an experiment involving three commercially available emotion AI systems and a group of human labelers tasked with identifying emotions from two image data sets. The study focuses on the alignment between facial expressions and the emotion labels assigned by both the AI and humans. Importantly, human labelers are given the AI's scores and informed about its algorithmic fairness measures. This paper presents several key findings. First, the labelers' scores are affected by the emotion AI scores, consistent with the anchoring effect. Second, information transparency about the AI's fairness does not uniformly affect human labeling across different emotions. Moreover, information transparency can even increase human inconsistencies. Plus, significant inconsistencies in the scoring among different emotion AI models cast doubt on their reliability. Overall, the study highlights the limitations of individual decision making and information transparency regarding algorithmic fairness measures in addressing algorithmic fairness. These findings underscore the complexity of integrating emotion AI into practice and emphasize the need for careful policies on emotion AI. Emotion artificial intelligence (AI) or emotion recognition AI may systematically vary in its recognition of facial expressions and emotions across demographic groups, creating inconsistencies and disparities in its scoring. This paper explores the extent to which individuals can compensate for these disparities and inconsistencies in emotion AI considering two opposing factors; although humans evolved to recognize emotions, particularly happiness, they are also subject to cognitive biases, such as the anchoring effect. To help understand these dynamics, this study tasks three commercially available emotion AIs and a group of human labelers to identify emotions from faces in two image data sets. The scores generated by emotion AI and human labelers are examined for inference inconsistencies (i.e., misalignment between facial expression and emotion label). The human labelers are also provided with the emotion AI's scores and with measures of its scoring fairness (or lack thereof). We observe that even when human labelers are operating in this context of information transparency, they may still rely on the emotion AI's scores, perpetuating its inconsistencies. Several findings emerge from this study. First, the anchoring effect appears to be moderated by the type of inference inconsistency and is weaker for easier emotion recognition tasks. Second, when human labelers are provided with information transparency regarding the emotion AI's fairness, the effect is not uniform across emotions. Also, there is no evidence that information transparency leads to the selective anchoring necessary to offset emotion AI disparities; in fact, some evidence suggests that information transparency increases human inference inconsistencies. Lastly, the different models of emotion AI are highly inconsistent in their scores, raising doubts about emotion AI more generally. Collectively, these findings provide evidence of the potential limitations of addressing algorithmic bias through individual decisions, even when those individuals are supported with information transparency. History: Alessandro Acquisti, Senior Editor; Monideepa Tarafdar, Associate Editor. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2019.0493. [ABSTRACT FROM AUTHOR]

The ability to effectively classify human emotion states is critically important for human‐computer or human‐robot interactions. However, emotion classification with physiological signals is still a challenging problem due to the diversity of emotion expression and the characteristic differences in different modal signals. A novel learning‐based network architecture is presented that can exploit four‐modal physiological signals, electrocardiogram, electrodermal activity, electromyography, and blood volume pulse, and make a classification of emotion states. It features two kinds of attention modules, feature‐level, and semantic‐level, which drive the network to focus on the information‐rich features by mimicking the human attention mechanism. The feature‐level attention module encodes the rich information of each physiological signal. While the semantic‐level attention module captures the semantic dependencies among modals. The performance of the designed network is evaluated with the open‐source Wearable Stress and Affect Detection dataset. The developed emotion classification system achieves an accuracy of 83.88%. Results demonstrated that the proposed network could effectively process four‐modal physiological signals and achieve high accuracy of emotion classification. [ABSTRACT FROM AUTHOR]

Abstract Micro-expressions are spontaneous, rapid and subtle facial movements that can hardly be suppressed or fabricated. Micro-expression recognition (MER) is one of the most challenging topics in affective computing. It aims to recognize subtle facial movements which are quite difficult for humans to perceive in a fleeting period. Recently, many deep learning-based MER methods have been developed. However, how to effectively capture subtle temporal variations for robust MER still perplexes us. We propose a counterfactual discriminative micro-expression recognition (CoDER) method to effectively learn the slight temporal variations for video-based MER. To explicitly capture the causality from temporal dynamics hidden in the micro-expression (ME) sequence, we propose ME counterfactual reasoning by comparing the effects of the facts w.r.t. original ME sequences and the counterfactuals w.r.t. counterfactually-revised ME sequences, and then perform causality-aware prediction to encourage the model to learn those latent ME temporal cues. Extensive experiments on four widely-used ME databases demonstrate the effectiveness of CoDER, which results in comparable and superior MER performance compared with that of the state-of-the-art methods. The visualization results show that CoDER successfully perceives the meaningful temporal variations in sequential faces.

Speech emotion recognition (SER) is attractive in several domains, such as automated translation, call centres, intelligent healthcare, and human–computer interaction. Deep learning models for emotion identification need considerable labelled data, which is only sometimes available in the SER industry. A database needs enough speech samples, good features, and a better classifier to identify emotions efficiently. This study uses data augmentation to enhance the amount of input voice samples and address the data shortage issue. The database capacity increases by adding white noise to the speech signals by data augmentation. In this work, the Mel-frequency Cepstral Coefficient (MFCC) and Mel-frequency Magnitude Coefficient (MFMC) features, along with a one-dimensional convolutional neural network (1D-CNN), are used to classify speech emotions. The datasets utilized to estimate the model's enactment were AESDD, CAFE, EmoDB, IEMOCAP, and MESD. The data augmentation with the 1D-CNN (MFMC) model performed best, with an average accuracy of 99.2% for AESDD, 99.5% for CAFE, 97.5% for EmoDB, 92.4% for IEMOCAP and 96.9% for the MESD database. The proposed 1D-CNN (MFMC) with data augmentation outperforms the 1D-CNN (MFCC) without data augmentation in emotion recognition.

Abstract The ability to effectively classify human emotion states is critically important for human‐computer or human‐robot interactions. However, emotion classification with physiological signals is still a challenging problem due to the diversity of emotion expression and the characteristic differences in different modal signals. A novel learning‐based network architecture is presented that can exploit four‐modal physiological signals, electrocardiogram, electrodermal activity, electromyography, and blood volume pulse, and make a classification of emotion states. It features two kinds of attention modules, feature‐level, and semantic‐level, which drive the network to focus on the information‐rich features by mimicking the human attention mechanism. The feature‐level attention module encodes the rich information of each physiological signal. While the semantic‐level attention module captures the semantic dependencies among modals. The performance of the designed network is evaluated with the open‐source Wearable Stress and Affect Detection dataset. The developed emotion classification system achieves an accuracy of 83.88%. Results demonstrated that the proposed network could effectively process four‐modal physiological signals and achieve high accuracy of emotion classification.

Actualmente, asistimos a la consolidación de un capitalismo de plataformas que basa gran parte de su economía en la producción de perfiles de comportamiento que buscan dirigir las acciones de los usuarios hacia fines privados. Mediante la asociación de técnicas de conductismo radical con tecnologías algorítmicas de procesamiento de datos se ha intensificado y consolidado una fuerza que Bernard Stiegler ha recogido bajo el nombre de psicopoder. Este artículo tiene como objetivo mostrar cómo ello se consigue a partir del despliegue de dos tecnologías de poder en la arquitectura de las plataformas digitales: el Affective Computing y el Hook Model. Mediante un estudio etnográfico de la red BeReal mostramos cómo estas dos tecnologías consiguen, primero, capturar la atención de los usuarios y crear hábitos de uso y, segundo, promover la circulación de emociones para que éstas puedan ser relacionadas con contextos concretos y datificadas para elaborar perfiles de comportamiento. Finalmente, realizamos un ejercicio teórico para argumentar que ambas tecnologías de control son elementos clave de un nuevo dispositivo de control al que llamamos “pulsional” y mediante el cual se consigue disparar una acción en los individuos que no pasa por su reflexión consciente, lo que tiene consecuencias nocivas para el ejercicio de su libertad.

Large language models have been extensively utilized for addressing general problems across various tasks. However, their current implementation in large language models still exhibits constraints in Chinese comprehension and reasoning. This study aims to overcome these limitations through enhancing the understanding and reasoning capabilities of Chinese language models while minimizing resource demands. We introduce LLaMA-LoRA, an engineered framework driven by thoughtful guidance, which is constructed upon the LLaMA-13B model and incorporates low-rank adaptation (LoRA) for meticulous fine-tuning. To overcome the inherent constraint of token-based left-to-right decision making during the inference process, we introduce the tree of thought to further augment our model. By considering diverse reasoning paths and engaging in self-evaluation during the decision-making process, Tree of thought facilitates deliberate decision making and facilitates the determination of subsequent action plans. Our experimental results demonstrate that the integration of this strategic mechanism significantly enhances Chinese comprehension and reasoning abilities, surpassing the performance of GPT-3.5, Chatglm, and other relevant language models in specific tasks. Meanwhile, it demonstrates excellent performance in sentiment calculation tasks such as multiclass sentiment analysis. Moreover, the availability of our open-source code facilitates further advancements in the field of Chinese text logical inference. [ABSTRACT FROM AUTHOR]

The purpose of this research was to identify the comfort levels of professionals with AI in various humanlike roles. A survey of 787 full-time working adults showed that more active AI users are comfortable with AI in many humanlike roles, such as a teammate or a performance coach. Less active AI users, however, are uncomfortable with AI in these roles. Leaders, managers, and educators should prepare employees and students to responsibly address the social and psychological outcomes of increasingly humanlike AI. [ABSTRACT FROM AUTHOR]

Technological advancements in web platforms allow people to express and share emotions toward textual write-ups written and shared by others. This brings about different interesting domains for analysis, emotion expressed by the writer and emotion elicited from the readers. In this paper, we propose a novel approach for readers' emotion detection from short-text documents using a deep learning model called REDAffectiveLM. Within state-of-the-art NLP tasks, it is well understood that utilizing context-specific representations from transformer-based pre-trained language models helps achieve improved performance. Within this affective computing task, we explore how incorporating affective information can further enhance performance. Toward this, we leverage context-specific and affect enriched representations by using a transformer-based pre-trained language model in tandem with affect enriched Bi-LSTM+Attention. For empirical evaluation, we procure a new dataset REN-20k, besides using RENh-4k and SemEval-2007. We evaluate the performance of our REDAffectiveLM rigorously across these datasets, against a vast set of state-of-the-art baselines, where our model consistently outperforms baselines and obtains statistically significant results. Our results establish that utilizing affect enriched representation along with context-specific representation within a neural architecture can considerably enhance readers' emotion detection. Since the impact of affect enrichment specifically in readers' emotion detection isn't well explored, we conduct a detailed analysis over affect enriched Bi-LSTM+Attention using qualitative and quantitative model behavior evaluation techniques. We observe that compared to conventional semantic embedding, affect enriched embedding increases the ability of the network to effectively identify and assign weightage to the key terms responsible for readers' emotion detection to improve prediction. [ABSTRACT FROM AUTHOR]

In a multi-objective problem, no single solution optimally satisfies all objectives. Thus, the challenge is to find a balance between conflicting objectives. The decision-making necessarily requires human intervention. The person responsible for selecting the most appropriate solution among all the trade-off solutions is the decision maker (DM). The DM seeks to approach only the solutions that best suit her/his preferences. Since there is plenty of specialized literature showing that emotions play a critical role in decision-making, we aim to incorporate them into the decision-making process. To elicit emotions that can be quantifiable, we propose the Emotional Assessment Method. The method presents a simulation of the objectives to be optimized that represents the consequences of each decision. Using this methodology, the decision maker assesses the emotions evoked by each presented simulation to guide his/her search for solutions that satisfy his/her preferences. As a case study, we aim to identify subjects’ preferences towards robot behaviors. Seventy-two subjects with varying levels of familiarity with robots (divided into two datasets) participated in the experiments. We concluded that the method elicits subjects’ emotions while observing the consequences of the robot’s performance. Also, we found out that it is possible to identify subjects’ preferences based on both the context and the emotions to select the robot’s behavior. [ABSTRACT FROM AUTHOR]

Abstract Attention mechanism has been a successful method for multimodal affective analysis in recent years. Despite the advances, several significant challenges remain in fusing language and its nonverbal context information. One is to generate sparse attention coefficients associated with acoustic and visual modalities, which helps locate critical emotional semantics. The other is fusing complementary cross‐modal representation to construct optimal salient feature combinations of multiple modalities. A Conditional Transformer Fusion Network is proposed to handle these problems. Firstly, the authors equip the transformer module with CNN layers to enhance the detection of subtle signal patterns in nonverbal sequences. Secondly, sentiment words are utilised as context conditions to guide the computation of cross‐modal attention. As a result, the located nonverbal features are not only salient but also complementary to sentiment words directly. Experimental results show that the authors’ method achieves state‐of‐the‐art performance on several multimodal affective analysis datasets.

We propose that the emotional quality of people’s interactions with acquaintances (i.e. weak ties) and strangers contributes to well-being. We test whether a new micro-intervention can raise the quality of these interactions. We randomized young adults (N = 335) to this connectedness micro-intervention or a control intervention. Both interventions were delivered via a psychoeducational video followed by a brief conversation with a virtual human, with whom participants developed if-then plans to carry out their assigned behavioral goal. Pre-intervention, high-quality weak-tie and stranger interactions were associated with lower loneliness and greater mental health, independent of strong-tie interaction quality. Experimental data showed that the connectedness intervention improved the emotional quality of participants’ interactions with weak ties and strangers over 2 days, evident in participants’ episodic self-reports and faster in-lab conversational response time. Discussion centers on implications for developing scalable behavioral interventions to improve well-being. [ABSTRACT FROM AUTHOR]