MFGTN: A multi-modal fast gated transformer for identifying single trawl marine fishing vessel.

In order to achieve sustainable development of marine fishery resources, effective supervise of trawl fishing during forbidden fishing period is of great significance. This paper addresses the challenges of poor generalization and the lack of unstructured information in the precise identification of single trawler fishing behavior. We propose a Transformer network with multi-source information fusion processing (MFGTN), which accurately classifies fishing vessels as single trawl or non-single trawl vessels. Firstly, a private fishing dataset of single trawl behavior is constructed by integrating AIS data with radar data, named HaiNan_SingleTrawlVessel(HN_STV). Subsequently, as fused data lacks unstructured information, it undergoes transformation into trajectory point images and recurrence plot images to reveal the internal structure of the fused data. As such, a visual module is introduced to handle the trajectory point images and recurrence plot images as a branch. Simultaneously, the fused data are input into a Double-Tower Transformer with Dual-gate structures to extract information in different dimensions of the time series and feature space as two separate branches. The Fast Attention module replaces the traditional Attention module to improve network speed and reduce memory consumption. Ultimately, the output of the three branches are fused and controlled by a Dual-gate structure that can autonomously learn to determine the network output. Experimental results show that compared to the current best-performing methods, the method discussed herein on the HN_STV dataset has improved the accuracy, recall, precision, and F1-score performance indicators by 2.34%, 2.46%, 0.97%, and 1.39%, respectively. The AUC area on the ROC curve increased by 4%. In a public dataset including three fishing activities, the proposed method improved accuracy, recall, precision, and F1-score by 2.95%, 2.59%, 2.25%, and 2.70%, respectively, and the AUC area on the ROC curve increased by 3%. And in all experiments, our network incurs the lowest time cost. Therefore, the method proposed herein demonstrates its advanced performance. • The HN_STV dataset is constructed, which extracts ship trajectories from the surrounding sea area with Hainan Province as the center, and effectively integrates the ship's AIS data and radar data to create temporal fishing fusion data sequences. • A double-tower Transformer network is proposed to process the information in different dimensions of time series and feature space of temporal fishing fusion data sequences. The traditional Attention module is modified to a Fast Attention module to reduce training time and memory consumption. • A Fusion former visual module is introduced to process the unstructured information of input data, including trajectory point images and recurrence plot images. • A double-layer learnable gate structure is proposed to continuously optimize the weights of different information sources according to the training results of neural networks, so that the network can independently choose the weight ratio of different information sources. [ABSTRACT FROM AUTHOR]