Search

Recently few-shot object detection (FSOD) in remote sensing images (RSIs) has drawn increasing attention. However, the current FSOD methods in RSIs merely focus on the detection performance of few-shot novel classes while ignoring the severe degradation of the base class performance. Generalized few-shot object detection (G-FSOD) aims to solve the FSOD problem without forgetting previous knowledge. In this paper, we focus on the G-FSOD in RSIs and propose a Generalized Few-Shot Detector (G-FSDet) that can learn novel knowledge without forgetting. Through the comprehensive analysis of each component in the detector, a novel efficient transfer-learning framework is presented as the foundation of our G-FSDet, which is more suitable for FSOD in remote sensing scenes. Considering the greater intra-class diversity and lower inter-class separability of geospatial objects, we design a metric-based discriminative loss to learn a more discriminative classifier in the few-shot fine-tuning stage. Furthermore, a representation compensation module is proposed to alleviate the catastrophic forgetting problem by decoupling the representation learning of previous and novel knowledge. Extensive experiments on DIOR and NWPU VHR-10.v2 datasets demonstrate that our proposed G-FSDet achieves competitive novel class performance with minor degradation in the base class, reaching state-of-the-art overall performance among all few-shot settings. The source code is available at (https://github.com/RSer-XDU/G-FSDet). [ABSTRACT FROM AUTHOR]

For remote sensing image (RSI) captioning tasks, two-stage RSI captioning methods have achieved high performance because they introduce the results of other RSI tasks, such as image classification as prior information. However, most previous works treat image classification in the two-stage RSI captioning method as a single-label classification task, which cannot adequately describe the entire contents of complex RSIs and may cause semantic ambiguity. To settle this problem and further refine image feature representation, we introduce multi-label classification into two-stage RSI captioning to provide sufficient and accurate prior semantic information and propose a multi-label semantic feature fusion (MLSFF) framework. Specifically, we design a robust multi-label semantic attribute extractor to extract multi-label semantic attributes of RSIs. To obtain discriminative feature representations for RSI captioning, we propose two cross-modal semantic feature fusion operators that fuse the extracted semantic attributes and the image feature extracted by the convolutional neural network. The results of extensive numerical experiments show that the proposed method can achieve state-of-the-art performance on the UCM-Captions, Sydney-Captions, and RSICD datasets. Specifically, on the UCM-Captions dataset, our method achieves a gain of 8.2% in S m score over the SAT (LAM) method (Zhang et al., 2019c). On the Sydney-Captions dataset, our method improves the S m score by 17.4% compared with the TCE loss-based method (Li et al., 2020a). On the RSICD, our method outperforms the multi-level attention method (Li et al., 2020b) by 3.2% in terms of the S m score. Code is available at https://github.com/xtye5025/MLSFF. [ABSTRACT FROM AUTHOR]