Learning a Twofold Siamese Network for RGB-T Object Tracking.

Visual object tracking works as a key component for many instrumentation and measurement applications such as UAV systems, optical tracking and measuring systems. This paper investigates how to implement accurate RGB-T tracking by integrating the complementary information from RGB and thermal sources. Inspired by the success of Siamese networks in the RGB tracking field, we design a twofold Siamese network for RGB-T tracking, which is composed of an RGB branch and a thermal branch. Each branch is a Siamese network, which can be utilized to compute similarities between the search image and the exemplar image. The parameters in the RGB branch are kept the same as SiamFC. The thermal branch is initialized with parameter weights of the trained network in SiamFC and fine-tuned with constructed thermal image pairs to better capture the target characteristics in the thermal data. Two criteria are further proposed to measure the confidence degrees of response maps obtained by these two modalities. The final response map is computed by adaptively fusing them according to their confidence degrees. The maximum location on the final response map is identified as the target location, and the target scale is obtained through a simple multi-scale search. Experiments on the recently public benchmark RGB-T234 demonstrate the effectiveness of our proposed method when compared to other state-of-the-art trackers. The high performance of our proposed trakcer makes it easy to be implemented in embedded system. [ABSTRACT FROM AUTHOR]