Review of Research on Application of Transformer in Domain Adaptation.

Domain adaptation, the important branch of transfer learning, aims to solve the problem that the performance of traditional machine learning algorithms drops sharply when the training and test samples obey different data distributions. Transformer is a deep learning framework based on a self-attention mechanism, which has strong global feature extraction ability and modeling ability. In recent years, the combination of Transformer and domain adaptation has also become a research hotspot. Although many relative methods have been published, the review of Transformer application in domain adaptation has not been reported. In order to fill the gap in this field and provide reference for relevant research, this paper summarizes and analyzes some typical domain adaptation methods based on Transformer in recent years. This paper summarizes the concepts related to domain adaptation and the basic structure of the Transformer, sorts out various domain adaptation methods based on Transformer from four applications, i.e. image classification, image semantic segmentation, object detection and medical image analysis and compares the domain adaptation methods in image classification. Finally, the challenges of the current domain adaptation Transformer model are summarized, and the feasible research directions in the future are discussed. [ABSTRACT FROM AUTHOR]