Search

International audience; This papers aims at improving spoken language modeling (LM) using very large amount of automatically transcribed speech. We leverage the INA (French National Audiovisual Institute 1) collection and obtain 19GB of text after applying ASR on 350,000 hours of diverse TV shows. From this, spoken language models are trained either by fine-tuning an existing LM (FlauBERT 2) or through training a LM from scratch. The new models (FlauBERT-Oral) are shared with the community 3 and are evaluated not only in terms of word prediction accuracy but also for two downstream tasks: classification of TV shows and syntactic parsing of speech. Experimental results show that FlauBERT-Oral is better than its initial FlauBERT version demonstrating that, despite its inherent noisy nature, ASR-Generated text can be useful to improve spoken language modeling.

In this paper, we focus on end-to-end architectures designed to tackle spoken language understanding problems. We propose encoder-decoder architectures with an attention mechanism that focuses on relevant contextual acoustic features to hypothesize semantic contents. A first architecture has been built in order to extract pronounced words and concepts from speech. Tested on the M EDIA dataset, it obtains good results, and combined with a language model, it lowers the error by 2.8 points from the state-of-the-art results with end-to-end systems. With this same architecture, we propose a new configuration allowing to predict both concepts and their values. Lastly, a new architecture is proposed, composed of multiple chained decoders for a single encoder. With this architecture, we aim to improve the decoder with both linguistic and acoustic informations., Dans cet article, nous nous intéressons au problème de la compréhension de la parole et à sa résolution dans le cadre d'architectures dites bout-en-bout. Les différentes architectures proposées, basées sur des modèles neuronaux encodeurs-décodeurs avec mécanisme d'attention permettent d'émettre des hypothèses de contenus sémantiques directement à partir des caractéristiques acoustiques. Une première architecture a été conc ¸ue afin d'extraire à la fois les mots prononcés et les concepts. Testée sur le corpus MEDIA, elle permet une réduction d'erreur en absolu de 2,8 points par rapport à l'état de l'art. Avec cette même architecture, nous proposons une configuration originale permettant d'émettre également des hypothèses sur les valeurs des concepts. Enfin, une architecture composée de plusieurs décodeurs neuronaux chaînés pour un seul encodeur est testée dans l'objectif d'enrichir le décodeur d'informations linguistiques en plus des informations acoustiques.

In this paper, we propose a novel end-to-end sequence-to-sequence spoken language understanding model using an attention mechanism. It reliably selects contextual acoustic features in order to hypothesize semantic contents. An initial architecture capable of extracting all pronounced words and concepts from acoustic spans is designed and tested. With a shallow fusion language model, this system reaches a 13.6 concept error rate (CER) and an 18.5 concept value error rate (CVER) on the French MEDIA corpus, achieving an absolute 2.8 points reduction compared to the state-of-the-art. Then, an original model is proposed for hypothesizing concepts and their values. This transduction reaches a 15.4 CER and a 21.6 CVER without any new type of context., Comment: Accepted at IEEE ICASSP 2021