Search

Automatic Grammar Error Correction (GEC) models identify and correct a wide range of grammatical errors. Various strategies have been proposed for GEC, with the Neural Machine Translation (NMT) approach being the most effective. However, NMT-based GEC models with encoderdecoder layers rely heavily on the highest layer, leading to potential inaccuracies. Additionally, during inference, exposure bias can cause the model to substitute previously targeted words with incorrect alternatives. Another challenge is data scarcity. This paper introduces a GEC model leveraging the seq-to-seq Transformer framework, specifically designed for low-resource languages like Arabic. We propose a method to generate noise in the text to create synthetic parallel data, addressing the data constraints. Inspired by Capsule Networks (CapsNet), we incorporate CapsNet in GEC to dynamically aggregate information from multiple layers. In order to mitigate exposure bias, we incorporated a bidirectional training approach and a regularization term using Kullback-Leibler divergence to align left-toright and right-to-left models. Experiments on two benchmarks demonstrate that our model outperforms current Arabic GEC models, achieving the highest scores. The code is available on GitHub (https://github.com/Zainabobied/ArabicGEC). [ABSTRACT FROM AUTHOR]

The correction of grammatical errors in natural language processing is a crucial task as it aims to enhance the accuracy and intelligibility of written language. However, developing a grammatical error correction (GEC) framework for low-resource languages presents significant challenges due to the lack of available training data. This article proposes a novel GEC framework for low-resource languages, using Arabic as a case study. To generate more training data, we propose a semi-supervised confusion method called the equal distribution of synthetic errors (EDSE), which generates a wide range of parallel training data. Additionally, this article addresses two limitations of the classical seq2seq GEC model, which are unbalanced outputs due to the unidirectional decoder and exposure bias during inference. To overcome these limitations, we apply a knowledge distillation technique from neural machine translation. This method utilizes two decoders, a forward decoder right-to-left and a backward decoder left-to-right, and measures their agreement using Kullback-Leibler divergence as a regularization term. The experimental results on two benchmarks demonstrate that our proposed framework outperforms the Transformer baseline and two widely used bidirectional decoding techniques, namely asynchronous and synchronous bidirectional decoding. Furthermore, the proposed framework reported the highest F1 score, and generating synthetic data using the equal distribution technique for syntactic errors resulted in a significant improvement in performance. These findings demonstrate the effectiveness of the proposed framework for improving grammatical error correction for low-resource languages, particularly for the Arabic language. [ABSTRACT FROM AUTHOR]