1. aMV-LSTM
- Author
-
Mohand Boughanem, Taoufiq Dkaki, Jose G. Moreno, Thiziri Belkacem, Institut National Polytechnique de Toulouse - INPT (FRANCE), Centre National de la Recherche Scientifique - CNRS (FRANCE), Université Toulouse III - Paul Sabatier - UT3 (FRANCE), Université Toulouse - Jean Jaurès - UT2J (FRANCE), Université Toulouse 1 Capitole - UT1 (FRANCE), Recherche d’Information et Synthèse d’Information (IRIT-IRIS), Institut de recherche en informatique de Toulouse (IRIT), Université Toulouse 1 Capitole (UT1), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées-Centre National de la Recherche Scientifique (CNRS)-Institut National Polytechnique (Toulouse) (Toulouse INP), Université Fédérale Toulouse Midi-Pyrénées-Université Toulouse 1 Capitole (UT1), Université Fédérale Toulouse Midi-Pyrénées, Université Toulouse - Jean Jaurès (UT2J), Université Toulouse III - Paul Sabatier (UT3), and Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
- Subjects
Matching (statistics) ,Process (engineering) ,Computer science ,Attention models ,Text matching ,02 engineering and technology ,computer.software_genre ,Position (vector) ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,Layer (object-oriented design) ,Traitement du texte et du document ,Positional ,business.industry ,020207 software engineering ,Text representation ,Weighting ,[INFO.INFO-TT]Computer Science [cs]/Document and Text Processing ,Identification (information) ,Artificial intelligence ,business ,computer ,Feature learning ,Word (computer architecture) ,Natural language processing - Abstract
National audience; Deep models are getting a wide interest in recent NLP and IR state-of-the-art. Among the proposed models, position-based models and attention-based models take into account the word position in the text, in the former, and the importance of a word among other words in the latter. The positional information are some of the important features that help text representation learning. However, the importance of a given word among others in a given text, which is an important aspect in text matching, is not considered in positional features. In this paper, we propose a model that combines position-based representation learning approach with the attention-based weighting process. The latter learns an importance coefficient for each word of the input text. We propose an extension of a position-based model MV-LSTM with an attention layer, allowing a parameterizable architecture. We believe that when the model is aware of both word position and importance, the learned representations will get more relevant features for the matching process. Our model, namely aMV-LSTM, learns the attention based coefficients to weight words of the different input sentences, before computing their position-based representations. Experimental results, in question/answer matching and question pairs identification tasks, show that the proposed model outperforms the MV-LSTM baseline and several state-of-the-art models.
- Published
- 2019
- Full Text
- View/download PDF