Evaluation of the Coherence of Polish Texts Using Neural Network Models.

Featured Application: The designed models of the coherence evaluation of Polish texts may be utilized during the discrimination of human-written and machine-generated texts in order to detect fake documents. Moreover, these models can be applied in a medicine area in order to reveal incoherent speech of a person that can indicate some symptoms of mental illness. There is no severe dependency on the language in considered models, thus, they can be trained and utilized for the solving of typical coherence evaluation tasks for other languages. Coherence evaluation of texts falls into a category of natural language processing tasks. The evaluation of texts' coherence implies the estimation of their semantic and logical integrity; such a feature of a text can be utilized during the solving of multidisciplinary tasks (SEO analysis, medicine area, detection of fake texts, etc.). In this paper, different state-of-the-art coherence evaluation methods based on machine learning models have been analyzed. The investigation of the effectiveness of different methods for the coherence estimation of Polish texts has been performed. The impact of text's features on the output coherence value has been analyzed using different approaches of a semantic similarity graph. Two neural networks based on LSTM layers and a pre-trained BERT model correspondingly have been designed and trained for the coherence estimation of input texts. The results obtained may indicate that both lexical and semantic components should be taken into account during the coherence evaluation of Polish documents; moreover, it is advisable to analyze corresponding documents in a sentence-by-sentence manner taking into account word order. According to the retrieved accuracy of the proposed neural networks, it can be concluded that suggested models may be used in order to solve typical coherence estimation tasks for a Polish corpus. [ABSTRACT FROM AUTHOR]