1. A novel conversational hierarchical attention network for speech emotion recognition in dyadic conversation.
- Author
-
Tellai, Mohammed, Gao, Lijian, Mao, Qirong, and Abdelaziz, Mounir
- Subjects
EMOTION recognition ,SPEECH perception ,AFFECTIVE computing ,HUMAN-computer interaction ,HIERARCHICAL Bayes model ,SPEECH ,ATTENTION - Abstract
Speech is one of the most fundamental mediums for human-to-human interaction, thereby playing a pivotal role in shaping the landscape of next-generation human-computer interaction (HCI). The development of an accurate speech emotion recognition (SER) system for human conversation is a critical yet challenging task. Most state-of-the-art research work in SER predominantly centers around the individual modeling of vocal attributes within each discrete speech utterance, often overlooking the integration of transactional cues intrinsic to the broader interactive context. In this paper, we introduce an innovative dual-level framework designed for the recognition of speech emotions, which leverages the complementary attributes of MFCC features and Mel-spectrograms. Furthermore, we propose a hierarchical attention mechanism designed to effectively include contextual information, hence improving the accuracy of emotion recognition. Our experimentation, conducted on the widely recognized IEMOCAP emotional benchmark dataset, yields promising results. Compared to state-of-the-art methods in four-class emotion recognition, our model demonstrates a substantial advancement, achieving a weighted accuracy of 75.0% and an unweighted accuracy of 75.9%. This marks a notable enhancement of 5.8% in terms of unweighted accuracy, underscoring the efficacy of our approach. This work contributes to the advancement of SER by effectively utilizing multiple audio representations and contextual information. The significant improvements underscore the efficacy of our approach, promising more accurate emotion recognition in human-computer interaction and affective computing. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF