Back to Search
Start Over
Speech-driven facial animation using polynomial fusion of features
- Publication Year :
- 2019
-
Abstract
- Speech-driven facial animation involves using a speech signal to generate realistic videos of talking faces. Recent deep learning approaches to facial synthesis rely on extracting low-dimensional representations and concatenating them, followed by a decoding step of the concatenated vector. This accounts for only first-order interactions of the features and ignores higher-order interactions. In this paper we propose a polynomial fusion layer that models the joint representation of the encodings by a higher-order polynomial, with the parameters modelled by a tensor decomposition. We demonstrate the suitability of this approach through experiments on generated videos evaluated on a range of metrics on video quality, audiovisual synchronisation and generation of blinks.
Details
- Database :
- arXiv
- Publication Type :
- Report
- Accession number :
- edsarx.1912.05833
- Document Type :
- Working Paper