Back to Search
Start Over
Isochrony-Aware Neural Machine Translation for Automatic Dubbing
- Publication Year :
- 2021
-
Abstract
- We introduce the task of isochrony-aware machine translation which aims at generating translations suitable for dubbing. Dubbing of a spoken sentence requires transferring the content as well as the speech-pause structure of the source into the target language to achieve audiovisual coherence. Practically, this implies correctly projecting pauses from the source to the target and ensuring that target speech segments have roughly the same duration of the corresponding source speech segments. In this work, we propose implicit and explicit modeling approaches to integrate isochrony information into neural machine translation. Experiments on English-German/French language pairs with automatic metrics show that the simplest of the considered approaches works best. Results are confirmed by human evaluations of translations and dubbed videos.<br />Comment: Published at InterSpeech 2022 (https://interspeech2022.org) - scheduled for September 18-22 2022, Incheon Korea
Details
- Database :
- OAIster
- Publication Type :
- Electronic Resource
- Accession number :
- edsoai.on1333738673
- Document Type :
- Electronic Resource