Back to Search
Start Over
'You made me feel this way': Investigating Partners' Influence in Predicting Emotions in Couples' Conflict Interactions using Speech Data
- Source :
- arXiv, Companion Publication of the 2021 International Conference on Multimodal Interaction (ICMI ’21 Companion)
- Publication Year :
- 2021
- Publisher :
- arXiv, 2021.
-
Abstract
- How romantic partners interact with each other during a conflict influences how they feel at the end of the interaction and is predictive of whether the partners stay together in the long term. Hence understanding the emotions of each partner is important. Yet current approaches that are used include self-reports which are burdensome and hence limit the frequency of this data collection. Automatic emotion prediction could address this challenge. Insights from psychology research indicate that partners' behaviors influence each other's emotions in conflict interaction and hence, the behavior of both partners could be considered to better predict each partner's emotion. However, it is yet to be investigated how doing so compares to only using each partner's own behavior in terms of emotion prediction performance. In this work, we used BERT to extract linguistic features (i.e., what partners said) and openSMILE to extract paralinguistic features (i.e., how they said it) from a data set of 368 German-speaking Swiss couples (N = 736 individuals) who were videotaped during an 8-minutes conflict interaction in the laboratory. Based on those features, we trained machine learning models to predict if partners feel positive or negative after the conflict interaction. Our results show that including the behavior of the other partner improves the prediction performance. Furthermore, for men, considering how their female partners spoke is most important and for women considering what their male partner said is most important in getting better prediction performance. This work is a step towards automatically recognizing each partners' emotion based on the behavior of both, which would enable a better understanding of couples in research, therapy, and the real world.<br />Comment: 5 pages. Accepted at the 2nd Workshop on Social Affective Multimodal Interaction for Health (SAMIH) at ICMI 2021
- Subjects :
- FOS: Computer and information sciences
Computer Science - Computation and Language
J.4
health sciences
05 social sciences
050109 social psychology
couples
emotion recognition
multimodal fusion
linguistic
paralinguistic
conflict
computer science
information management
behavioral science
02 engineering and technology
Couple, emotion, prediction
16. Peace & justice
couples, emotion recognition, multimodal fusion, linguistic, par- alinguistic, conflict
5. Gender equality
0202 electrical engineering, electronic engineering, information engineering
020201 artificial intelligence & image processing
0501 psychology and cognitive sciences
social sciences
Computation and Language (cs.CL)
Subjects
Details
- Database :
- OpenAIRE
- Journal :
- arXiv, Companion Publication of the 2021 International Conference on Multimodal Interaction (ICMI ’21 Companion)
- Accession number :
- edsair.doi.dedup.....7809f831249290a3957ad6fa94b9a6d7
- Full Text :
- https://doi.org/10.48550/arxiv.2106.01526