Bayesian variational transformer: A generalizable model for rotating machinery fault diagnosis.

[Display omitted] • Bayesian learning is introduced to Transformer to randomize its attention weights. • A generalizable model called Bayesformer model is proposed. • A variational self-attention mechanism is designed to model the posterior and prior distributions of attention weights. • An ELBO optimization objective is defined to guide model training. Transformer has been widely applied in the research of rotating machinery fault diagnosis due to its ability to explore the internal correlation of vibration signals. However, challenges still exist despite the countless efforts. Generally, Transformer is more prone to overfitting than CNN on small-scale datasets. In practical engineering, collecting sufficient fault samples for training is difficult, resulting in poor generalization of Transformer. In addition, the measured signals are often accompanied with severe noise, further reducing the generalization performance of the model. Meanwhile, the collected signals often follow different distributions due to the changing operating conditions, which places higher demands on the generalizability of Transformer. This paper proposes a Bayesian variational Transformer (Bayesformer) to cope with the abovementioned problems. In Bayesformer, all the attention weights are treated as latent random variables, rather than determined values as the previous studies. This allows to train an ensemble of networks, instead of a single one, enhancing the generalizability of the model. Three experimental studies are conducted to illustrate the developed model and superior diagnostic performance is showed throughout the experiments. [ABSTRACT FROM AUTHOR]