1. Joint Generation of Captions and Subtitles with Dual Decoding
- Author
-
Xu, Jitao, Buet, François, Crego, Josep, Elise Bertin-Lemée, Yvon, François, Traitement du Langage Parlé (TLP ), Laboratoire Interdisciplinaire des Sciences du Numérique (LISN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Sciences et Technologies des Langues (STL), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS), SYSTRAN, and Xu, Jitao
- Subjects
FOS: Computer and information sciences ,Computer Science - Computation and Language ,[INFO.INFO-CL] Computer Science [cs]/Computation and Language [cs.CL] ,Computation and Language (cs.CL) ,[INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL] - Abstract
As the amount of audio-visual content increases, the need to develop automatic captioning and subtitling solutions to match the expectations of a growing international audience appears as the only viable way to boost throughput and lower the related post-production costs. Automatic captioning and subtitling often need to be tightly intertwined to achieve an appropriate level of consistency and synchronization with each other and with the video signal. In this work, we assess a dual decoding scheme to achieve a strong coupling between these two tasks and show how adequacy and consistency are increased, with virtually no additional cost in terms of model size and training complexity., Comment: Accepted at IWSLT 2022
- Published
- 2022