Search

We present and evaluate a new Visual Voice Activity Detection method based on Spatiotemporal Gabor filters (STem-VVAD). Since Spatiotemporal Gabor filters are dynamic, they offer an attractive method to separate speech from non-speech frames in video, even though they have not been used for this purpose before. We evaluate our method on two datasets, which differ in the ratio of speech to non-speech frames (high versus low), as well as in the head orientation of the speakers (frontal versus profile). We compare models on different regions (applied to the mouth, the head or the entire video frame), and do so both for speaker-dependent, individual models and speaker-independent, generic models. In general, best performances are obtained for speaker-dependent STem-VVAD applied to the mouth region, and combining information from different speeds. In all these cases, the system outperforms two reference systems, relying on frame differencing and static Gabor filters respectively, showing that Spatiotemporal Gabor filters indeed are beneficial for visual voice detection. [ABSTRACT FROM AUTHOR]

Affective robots and embodied conversational agents require convincing facial expressions to make them socially acceptable. To be able to virtually generate facial expressions, we need to investigate the relationship between technology and human perception of affective and social signals. Facial landmarks, the locations of the crucial parts of a face, are important for perception of the affective and social signals conveyed by facial expressions. Earlier research did not use that kind of technology, but rather used analogue technology to generate point-light faces. The goal of our study is to investigate whether digitally extracted facial landmarks contain sufficient information to enable the facial expressions to be recognized by humans. This study presented participants with facial expressions encoded in moving landmarks, while these facial landmarks correspond to the facial-landmark videos that were extracted by face analysis software from full-face videos of acted emotions. The facial-landmark videos were presented to 16 participants who were instructed to classify the sequences according to the emotion represented. Results revealed that for three out of five facial-landmark videos (happiness, sadness and anger), participants were able to recognize emotions accurately, but for the other two facial-landmark videos (fear and disgust), their recognition accuracy was below chance, suggesting that landmarks contain information about the expressed emotions. Results also show that emotions with high levels of arousal and valence are better recognized than those with low levels of arousal and valence. We argue that the question of whether these digitally extracted facial landmarks are a basis for representing facial expressions of emotions is crucial for the development of successful human-robot interaction in the future. We conclude by stating that landmarks provide a basis for the virtual generation of emotions in humanoid agents, and discuss how additional facial information might be included to provide a sufficient basis for faithful emotion identification. [ABSTRACT FROM AUTHOR]