Search

Reinforcement learning (RL) is an effective method in training dialogue policies to steer the conversation towards successful task completion. However, most RL-based methods only rely on semantic inputs that lack empathy as they ignore the user emotional information. Moreover, these methods suffer from delayed rewards caused by the user simulator returning valuable results only at dialogue end. Recently, some methods have been proposed to learn the reward function together with user emotions, but they omit considering user emotion in each dialogue turn. In this paper, we proposed an emotion-sensitive dialogue policy model (ESDP), it incorporates user emotions information into dialogue policy and selects the optimal action by the combination of top-k actions with the user emotions. The user emotion information in each turn is used as an immediate reward for the current dialogue state to solve sparse rewards and the dependency on termination. Extensive experiments validate that our method outperforms the baseline approaches when combined with different Q-Learning algorithms, and also surpasses other popular existing dialog policies' performance., (© 2024. The Author(s).)

Undercover videos have become a popular tool among NGOs to influence public opinion and generate engagement for the NGO's cause. These videos are seen as a powerful and cost-effective way of bringing about social change, as they provide first-hand evidence and generate a strong emotional response among those who see them. In this paper, we empirically assess the impact of undercover videos on support for the cause. We in addition analyze whether the increased engagement among viewers is driven by the negative emotional reactions produced by the video. To do so, we design an online experiment that enables us to estimate both the total and emotion-mediated treatment effects on engagement by randomly exposing participants to an undercover video (of animal abuse) and randomly introducing a cooling-off period. Using a representative sample of the French population (N=3,310), we find that the video successfully increases actions in favor of animals (i.e., donations to NGOs and petitions), but we fail to prove that this effect is due to the presence of primary emotions induced by the video. Last, we investigate whether activists correctly anticipate their undercover videos' (emotional) impact via a prediction study involving activists (exploratory analysis). Protocol registration: This manuscript is a Stage-2 working paper of a Registered Report that received In-Principle-Acceptance from Scientific Reports on November 20th, 2023 [Link to Stage-1]. The Stage-1 that received In-Principal-Acceptance can be found here: https://osf.io/8cg2d. [ABSTRACT FROM AUTHOR]

Better communication with older people requires not only improving speech intelligibility but also understanding how well emotions can be conveyed and the effect of age and hearing loss (HL) on emotion perception. In this paper, emotion discrimination experiments were conducted using a vocal morphing method and an HL simulator in young normal hearing (YNH) and older participants. Speech sounds were morphed to represent intermediate emotions between all combinations of happiness, sadness, and anger. Discrimination performance was compared when the YNH listened to normal sounds, when the same YNH listened to HL simulated sounds, and when older people listened to the same normal sounds. The results showed that there was no significant difference between discrimination with and without HL simulation, suggesting that peripheral HL may not affect emotion perception. The discrimination performance of the older participants was significantly worse only for the anger-happiness pair than for the other emotion pairs and for the YNH. It was also found that the difficulty increases with age, not just with hearing level., (© 2024. The Author(s).)

Speech emotion recognition (SER) technology involves feature extraction and prediction models. However, recognition efficiency tends to decrease because of gender differences and the large number of extracted features. Consequently, this paper introduces a SER system based on gender. First, gender and emotion features are extracted from speech signals to develop gender recognition and emotion classification models. Second, according to gender differences, distinct emotion recognition models are established for male and female speakers. The gender of speakers is determined before executing the corresponding emotion model. Third, the accuracy of these emotion models is enhanced by utilizing an advanced differential evolution algorithm (ADE) to select optimal features. ADE incorporates new difference vectors, mutation operators, and position learning, which effectively balance global and local searches. A new position repairing method is proposed to address gender differences. Finally, experiments on four English datasets demonstrate that ADE is superior to comparison algorithms in recognition accuracy, recall, precision, F1-score, the number of used features and execution time. The findings highlight the significance of gender in refining emotion models, while mel-frequency cepstral coefficients are important factors in gender differences., (© 2024. The Author(s).)

Affect recognition in a real-world, less constrained environment is the principal prerequisite of the industrial-level usefulness of this technology. Monitoring the psychological profile using smart, wearable electroencephalogram (EEG) sensors during daily activities without external stimuli, such as memory-induced emotions, is a challenging research gap in emotion recognition. This paper proposed a deep learning framework for improved memory-induced emotion recognition leveraging a combination of 1D-CNN and LSTM as feature extractors integrated with an Extreme Learning Machine (ELM) classifier. The proposed deep learning architecture, combined with the EEG preprocessing, such as the removal of the average baseline signal from each sample and extraction of EEG rhythms (delta, theta, alpha, beta, and gamma), aims to capture repetitive and continuous patterns for memory-induced emotion recognition, underexplored with deep learning techniques. This work has analyzed EEG signals using a wearable, ultra-mobile sports cap while recalling autobiographical emotional memories evoked by affect-denoting words, with self-annotation on the scale of valence and arousal. With extensive experimentation using the same dataset, the proposed framework empirically outperforms existing techniques for the emerging area of memory-induced emotion recognition with an accuracy of 65.6%. The EEG rhythms analysis, such as delta, theta, alpha, beta, and gamma, achieved 65.5%, 52.1%, 65.1%, 64.6%, and 65.0% accuracies for classification with four quadrants of valence and arousal. These results underscore the significant advancement achieved by our proposed method for the real-world environment of memory-induced emotion recognition., (© 2024. The Author(s).)

Recent research has focused extensively on employing Deep Learning (DL) techniques, particularly Convolutional Neural Networks (CNN), for Speech Emotion Recognition (SER). This study addresses the burgeoning interest in leveraging DL for SER, specifically focusing on Punjabi language speakers. The paper presents a novel approach to constructing and preprocessing a labeled speech corpus using diverse social media sources. By utilizing spectrograms as the primary feature representation, the proposed algorithm effectively learns discriminative patterns for emotion recognition. The method is evaluated on a custom dataset derived from various Punjabi media sources, including films and web series. Results demonstrate that the proposed approach achieves an accuracy of 69%, surpassing traditional methods like decision trees, Naïve Bayes, and random forests, which achieved accuracies of 49%, 52%, and 61% respectively. Thus, the proposed method improves accuracy in recognizing emotions from Punjabi speech signals., (© 2024. The Author(s).)

Speech emotion recognition (SER) has gained an increased interest during the last decades as part of enriched affective computing. As a consequence, a variety of engineering approaches have been developed addressing the challenge of the SER problem, exploiting different features, learning algorithms, and datasets. In this paper, we propose the application of the graph theory for classifying emotionally-colored speech signals. Graph theory provides tools for extracting statistical as well as structural information from any time series. We propose to use the mentioned information as a novel feature set. Furthermore, we suggest setting a unique feature-based identity for each emotion belonging to each speaker. The emotion classification is performed by a Random Forest classifier in a Leave-One-Speaker-Out Cross Validation (LOSO-CV) scheme. The proposed method is compared with two state-of-the-art approaches involving well known hand-crafted features as well as deep learning architectures operating on mel-spectrograms. Experimental results on three datasets, EMODB (German, acted) and AESDD (Greek, acted), and DEMoS (Italian, in-the-wild), reveal that our proposed method outperforms the comparative methods in these datasets. Specifically, we observe an average UAR increase of almost [Formula: see text], [Formula: see text] and [Formula: see text], respectively., (© 2024. The Author(s).)

The negative impact of used batteries of new energy vehicles on the environment has attracted global attention, and how to effectively deal with used batteries of new energy vehicles has become a hot issue. This paper combines the rank-dependent expected utility with the evolutionary game theory, constructs an evolutionary game model based on the interaction mechanism between decision makers' emotions and decision making, and studies the recycling strategy of new energy automobile trams under the heterogeneous combination of emotions. The study shows that: (1) In addition to the establishment of effective external norms, the subjective preference of decision makers can also positively affect the recycling strategy of new energy vehicle batteries. (2) Fairness preferences can have a significant nonlinear effect on new energy vehicle battery recycling strategies by changing the utility function of decision makers. (3) When new energy vehicle manufacturers remain optimistic and new energy vehicle demanders remain rational or pessimistic, the new energy vehicle battery recycling strategy can reach the optimal steady state. [ABSTRACT FROM AUTHOR]

The role of learner emotions in language learning has long been observed and researched. Of various emotions, foreign language anxiety, enjoyment and boredom have been the most often researched, while research on other emotions like pride and sadness is hardly available in the current literature. Research on emotions in students with special characteristics can be hardly found as well. Hence, this study collected interview data from 26 and survey data from 520 top students in two universities in China to examine their emotions in English language class. Analyses of the data revealed the following major findings: (a) The participants experienced a diversity of emotions in their English language class due to both learner-internal and external reasons, but the most often reported emotions were anxiety, enjoyment and boredom, (b) significant correlations existed among the students' English language classroom anxiety, enjoyment and boredom, (c) English language classroom anxiety and boredom significantly negatively while enjoyment significantly positively predicted the students' English test performance, and (d) English language classroom anxiety debilitated English learning but motivated students to study harder as well; enjoyment facilitated English learning but students might forget much of what had been learned after class; though boredom caused some students to be absent-minded in class, most students would study on their own when feeling bored. These findings further pinpoint the important role and complex nature of learner emotions in second/foreign language learning. Based on these findings, specific suggestions for language teachers and learners are discussed. [ABSTRACT FROM AUTHOR]

While alterations in both physiological responses to others' emotions as well as interoceptive abilities have been identified in autism, their relevance in altered emotion recognition is largely unknown. We here examined the role of interoceptive ability, facial mimicry, and autistic traits in facial emotion processing in non-autistic individuals. In an online Experiment 1, participants (N = 99) performed a facial emotion recognition task, including ratings of perceived emotional intensity and confidence in emotion recognition, and reported on trait interoceptive accuracy, interoceptive sensibility and autistic traits. In a follow-up lab Experiment 2 involving 100 participants, we replicated the online experiment and additionally investigated the relationship between facial mimicry (measured through electromyography), cardiac interoceptive accuracy (evaluated using a heartbeat discrimination task), and autistic traits in relation to emotion processing. Across experiments, neither interoception measures nor facial mimicry accounted for a reduced recognition of specific expressions with higher autistic traits. Higher trait interoceptive accuracy was rather associated with more confidence in correct recognition of some expressions, as well as with higher ratings of their perceived emotional intensity. Exploratory analyses indicated that those higher intensity ratings might result from a stronger integration of instant facial muscle activations, which seem to be less integrated in intensity ratings with higher autistic traits. Future studies should test whether facial muscle activity, and physiological signals in general, are correspondingly less predictive of perceiving emotionality in others in individuals on the autism spectrum, and whether training interoceptive abilities might facilitate the interpretation of emotional expressions. [ABSTRACT FROM AUTHOR]

Dogs can discriminate stressed from non-stressed human odour samples, but the effect on their cognition is unstudied. Using a cognitive bias task, we tested how human odours affect dogs' likelihood of approaching a food bowl placed at three ambiguous locations ("near-positive", "middle" and "near-negative") between trained "positive" (rewarded) and "negative" (unrewarded) locations. Using odour samples collected from three unfamiliar volunteers during stressful and relaxing activities, we tested eighteen dogs under three conditions: no odour, stress odour and relaxed odour, with the order of test odours counterbalanced across dogs. When exposed to stress odour during session three, dogs were significantly less likely to approach a bowl placed at one of the three ambiguous locations (near-negative) compared to no odour, indicating possible risk-reduction behaviours in response to the smell of human stress. Dogs' learning of trained positive and negative locations improved with repeated testing and was significant between sessions two and three only when exposed to stress odour during session three, suggesting odour influenced learning. This is the first study to show that without visual or auditory cues, olfactory cues of human stress may affect dogs' cognition and learning, which, if true, could have important consequences for dog welfare and working performance. [ABSTRACT FROM AUTHOR]

While perceiving the emotional state of others may be crucial for our behavior even when this information is present outside of central vision, emotion perception studies typically focus on central visual field. We have recently investigated emotional valence (pleasantness) perception across the parafovea (≤ 4°) and found that for briefly presented (200 ms) emotional face images (from the established KDEF image-set), positive (happy) valence was the least affected by eccentricity (distance from the central visual field) and negative (fearful) valence the most. Furthermore, we found that performance at 2° predicted performance at 4°. Here we tested (n = 37) whether these effects replicate with face stimuli of different identities from a different well-established image-set (NimStim). All our prior findings replicated and eccentricity-based modulation magnitude was smaller with NimStim (~ 16.6% accuracy reduction at 4°) than with KDEF stimuli (~ 27.3% reduction). Our current investigations support our earlier findings that for briefly presented parafoveal stimuli, positive and negative valence perception are differently affected by eccentricity and may be dissociated. Furthermore, our results highlight the importance of investigating emotions beyond central vision and demonstrate commonalities and differences across different image sets in the parafovea, emphasizing the contribution of replication studies to substantiate our knowledge about perceptual mechanisms. [ABSTRACT FROM AUTHOR]

Humans use pictures to model the world. The structure of a picture maps to mind space to form a concept. When an internal structure matches the corresponding external structure, an observation functions. Whether effective or not, the observation is self-consistent. In epistemology, people often differ from each other in terms of whether a concept is probabilistic or certain. Based on the effect of the presented IG and pull anti algorithm, we attempt to provide a comprehensive answer to this problem. Using the characters of hidden structures, we explain the difference between the macro and micro levels and the same difference between semantics and probability. In addition, the importance of attention is highlighted through the combination of symmetry and asymmetry included and the mechanism of chaos and collapse revealed in the presented model. Because the subject is involved in the expression of the object, representationalism is not complete. However, people undoubtedly reach a consensus based on the objectivity of the representation. Finally, we suggest that emotions could be used to regulate cognition. [ABSTRACT FROM AUTHOR]

This study delves into expressing primary emotions anger, happiness, sadness, and fear through drawings. Moving beyond the well-researched color-emotion link, it explores under-examined aspects like spatial concepts and drawing styles. Employing Python and OpenCV for objective analysis, we make a breakthrough by converting subjective perceptions into measurable data through 728 digital images from 182 university students. For the prominent color chosen for each emotion, the majority of participants chose red for anger (73.11%), yellow for happiness (17.8%), blue for sadness (51.1%), and black for fear (40.7%). Happiness led with the highest saturation (68.52%) and brightness (75.44%) percentages, while fear recorded the lowest in both categories (47.33% saturation, 48.78% brightness). Fear, however, topped in color fill percentage (35.49%), with happiness at the lowest (25.14%). Tangible imagery prevailed (71.43–83.52%), with abstract styles peaking in fear representations (28.57%). Facial expressions were a common element (41.76–49.45%). The study achieved an 81.3% predictive accuracy for anger, higher than the 71.3% overall average. Future research can build on these results by improving technological methods to quantify more aspects of drawing content. Investigating a more comprehensive array of emotions and examining factors influencing emotional drawing styles will further our understanding of visual-emotional communication. [ABSTRACT FROM AUTHOR]

Social decision-making is known to be influenced by predictive emotions or the perceived reciprocity of partners. However, the connection between emotion, decision-making, and contextual reciprocity remains less understood. Moreover, arguments suggest that emotional experiences within a social context can be better conceptualised as prosocial rather than basic emotions, necessitating the inclusion of two social dimensions: focus, the degree of an emotion's relevance to oneself or others, and dominance, the degree to which one feels in control of an emotion. For better representation, these dimensions should be considered alongside the interoceptive dimensions of valence and arousal. In an ultimatum game involving fair, moderate, and unfair offers, this online study measured the emotions of 476 participants using a multidimensional affective rating scale. Using unsupervised classification algorithms, we identified individual differences in decisions and emotional experiences. Certain individuals exhibited consistent levels of acceptance behaviours and emotions, while reciprocal individuals' acceptance behaviours and emotions followed external reward value structures. Furthermore, individuals with distinct emotional responses to partners exhibited unique economic responses to their emotions, with only the reciprocal group exhibiting sensitivity to dominance prediction errors. The study illustrates a context-specific model capable of subtyping populations engaged in social interaction and exhibiting heterogeneous mental states. [ABSTRACT FROM AUTHOR]

Trauma scientists have raised the alarm about the devastating consequences of the Ukraine war on mental health. We examined how higher education students—as indirect victims—coped with this conflict and how they emotionally reacted during 2022. We involved 2314 students from 16 countries through an online survey. A structural equation model indicated significant relations between war-related worry about military and macroeconomics domains and two coping strategies (opposition, support giving), in turn significantly linked with six emotions. The model was strongly invariant across gender, study field, and geographic area. The most frequent emotions were anger and anxiety, followed by two future-centred emotions (hopelessness and hope). Emotions were more frequent for females and students of the countries geographically close to the war region. Our findings call for evidence-based policy recommendations to be implemented by institutions to combat the negative short and long-term psychological sequelae of being witnesses of armed conflicts. [ABSTRACT FROM AUTHOR]

Music is ubiquitous in our everyday lives, and lyrics play an integral role when we listen to music. The complex relationships between lyrical content, its temporal evolution over the last decades, and genre-specific variations, however, are yet to be fully understood. In this work, we investigate the dynamics of English lyrics of Western, popular music over five decades and five genres, using a wide set of lyrics descriptors, including lyrical complexity, structure, emotion, and popularity. We find that pop music lyrics have become simpler and easier to comprehend over time: not only does the lexical complexity of lyrics decrease (for instance, captured by vocabulary richness or readability of lyrics), but we also observe that the structural complexity (for instance, the repetitiveness of lyrics) has decreased. In addition, we confirm previous analyses showing that the emotion described by lyrics has become more negative and that lyrics have become more personal over the last five decades. Finally, a comparison of lyrics view counts and listening counts shows that when it comes to the listeners' interest in lyrics, for instance, rock fans mostly enjoy lyrics from older songs; country fans are more interested in new songs' lyrics. [ABSTRACT FROM AUTHOR]

Among disabling post-traumatic stress symptoms (PTSS) are irritability, aggressive behavior, distressing memories and general impaired cognition and negative mood. Art therapy interventions, including mask-making, can potentially alleviate these symptoms. We tested the hypothesis that art conveys emotions and predicted that blinded viewers would be able to perceive changes in theoretically derived emotional profiles expressed in art made by military personnel with PTSS from the onset to the end of therapy. Five service members and veterans exhibiting PTSS were enrolled in an 8-session art therapy protocol, during which they artistically transformed papier-mâché masks at the beginning and end of the protocol. We found that blinded viewers without knowledge of the masks' creation stage (onset or end of therapy) read initial masks as conveying more negative emotions (e.g., angry, upset, and challenged) and later masks as conveying more positive emotions (calm and pleasure). Based on the assessments from the blinded evaluators, we infer the emotional transition experienced by the participants was expressed in the masks. In an exploratory arm of the study, we also found that viewers were better able to empathize with the negative emotions experienced by participants with PTSS when asked to explicitly take their perspective. [ABSTRACT FROM AUTHOR]

Developmental prosopagnosia (DP) is characterised by deficits in face identification. However, there is debate about whether these deficits are primarily perceptual, and whether they extend to other face processing tasks (e.g., identifying emotion, age, and gender; detecting faces in scenes). In this study, 30 participants with DP and 75 controls completed a battery of eight tasks assessing four domains of face perception (identity; emotion; age and gender; face detection). The DP group performed worse than the control group on both identity perception tasks, and one task from each other domain. Both identity perception tests uniquely predicted DP/control group membership, and performance on two measures of face memory. These findings suggest that deficits in DP may arise from issues with face perception. Some non-identity tasks also predicted DP/control group membership and face memory, even when face identity perception was accounted for. Gender perception and speed of face detection consistently predicted unique variance in group membership and face memory; several other tasks were only associated with some measures of face recognition ability. These findings indicate that face perception deficits in DP may extend beyond identity perception. However, the associations between tasks may also reflect subtle aspects of task demands or stimuli. [ABSTRACT FROM AUTHOR]

A person with impaired emotion recognition is not able to correctly identify facial expressions represented by other individuals. The aim of the present study is to assess eyes gaze and facial emotion recognition in a healthy population using dynamic avatars in immersive virtual reality (IVR). For the first time, the viewing of each area of interest of the face in IVR is studied by gender and age. This work in healthy people is conducted to assess the future usefulness of IVR in patients with deficits in the recognition of facial expressions. Seventy-four healthy volunteers participated in the study. The materials used were a laptop computer, a game controller, and a head-mounted display. Dynamic virtual faces randomly representing the six basic emotions plus neutral expression were used as stimuli. After the virtual human represented an emotion, a response panel was displayed with the seven possible options. Besides storing the hits and misses, the software program internally divided the faces into different areas of interest (AOIs) and recorded how long participants looked at each AOI. As regards the overall accuracy of the participants' responses, hits decreased from the youngest to the middle-aged and older adults. Also, all three groups spent the highest percentage of time looking at the eyes, but younger adults had the highest percentage. It is also noteworthy that attention to the face compared to the background decreased with age. Moreover, the hits between women and men were remarkably similar and, in fact, there were no statistically significant differences between them. In general, men paid more attention to the eyes than women, but women paid more attention to the forehead and mouth. In contrast to previous work, our study indicates that there are no differences between men and women in facial emotion recognition. Moreover, in line with previous work, the percentage of face viewing time for younger adults is higher than for older adults. However, contrary to earlier studies, older adults look more at the eyes than at the mouth.Consistent with other studies, the eyes are the AOI with the highest percentage of viewing time. For men the most viewed AOI is the eyes for all emotions in both hits and misses. Women look more at the eyes for all emotions, except for joy, fear, and anger on hits. On misses, they look more into the eyes for almost all emotions except surprise and fear. [ABSTRACT FROM AUTHOR]

The ability to understand and regulate one's emotions is an integral part of human development. As part of this learning process, emotion socialization is understood as the dynamic mechanism by which caregivers mediate and influence the child's emotional competence. Failures in emotion socialization have been associated with antisocial behavior, peer rejection, and mental health issues in both children and adults, which underscores the importance of this process. It has been suggested that emotion socialization is strongly influenced by the socio-cultural features of the caregivers. This Collection compiles recent works that unravel the underlying complex mechanisms of emotion socialization and related life outcomes. It emphasizes the crucial role that cultural and individual traits play in the process of emotion socialization. Looking ahead, combining insights from neuro-physiological and socio-cultural perspectives promises to enrich our comprehension of emotional processes and emotional competence development. [ABSTRACT FROM AUTHOR]