Error correction algorithm for grating Moiré fringes based on QM-ANN.

• This paper suggests a grating Moiré fringe-based error correction approach that uses an artificial neural network (ANN) and the quartile method (QM). • The approach uses QM for data cleaning operations to eliminate outliers and creates the ANN model with a structure of fully connected layer-normalization layer-fully connected layer for regression training in order to achieve lightweight and stability. • The proposed QM-ANN error correction model's mean absolute error (MAE) can reach 3.3 nm once the QM data has been cleaned and the ANN model has been trained. Many interference factors of the complex and changeable working environment of the grating sensor are the main factors affecting the accuracy; a grating Moiré fringe error correction model based on the quartile method (QM) and artificial neural network (ANN) is proposed in this paper. Firstly, the grating sensor's signals in the working process are collected, and three kinds of interference signals (temperature, humidity and vibration) from the grating sensor working environment are collected simultaneously. Secondly, the QM is used to detect outliers from the collected data and delete data outliers directly. Subsequently, the processed data are input into the ANN model for training. The model is lightweight and stable, because it adopts the cascaded structure of full connection layer-normalization layer-full connection layer, which does not need deploy the complex convolution layer module etc. Finally, the corrected data are output after being processed by the QM-ANN error correction model. The experimental results show that the mean absolute error (MAE) of the proposed QM-ANN error correction model is better than QM-convolutional neural network (CNN), QM-long short-term memory (LSTM) and ANN, CNN, LSTM models without QM. It shows that the proposed model has better robustness and practicability. [ABSTRACT FROM AUTHOR]