Search

The remote sensing data is difficult to collect and lack of diversity, which extremely limits the performance of object detection on remote sensing images. In this paper, a multi-branch conditional generative adversarial network (MCGAN) is proposed to augment data for object detection in optical remote sensing images, which is the first GANs-based data augmentation framework proposed for this topic. We use MCGAN to generate the diverse objects based on the existing remote sensing datasets. The multi-branch dilated convolution and the classification branch are adopted into MCGAN to help the generator to generate the diverse and high-quality images. Meanwhile, an adaptive samples selection strategy based on the Faster R-CNN is proposed to select the samples for data augmentation from the objects generated by MCGAN, which can ensure the quality of new augmented training sets and improve the diversity of samples. Experiments based on NWPU VHR-10 and DOTA show that the objects generated by MCGAN have the higher quality compared with the objects generated by WGAN and LSGAN. And the mean average precision detected by the state-of-the-art object detection models used in the experiments has the satisfactory improvement after the MCGAN based data augmentation, which indicates that data augmentation by MCGAN can effectively improve the accuracy of remote sensing images object detection. [ABSTRACT FROM AUTHOR]

• We propose an efficient hybrid ReID backbone for discriminative feature extraction. • We formulate an efficient PW-MSA block to free positions from the fixed length. • We put forward a novel attention-guided GCN model to encode person attributes and body parts into embedding representations. [Display omitted] The Transformer has been applied into computer vision to explore long-range dependencies with multi-head self-attention strategy, therefore numerous Transformer-based methods for person re-identification (ReID) are designed for extracting effective as well as robust representation. However, the memory and computational complexity of scaled dot-product attention in Transformer cost vast overheads. To overcome these limitations, this paper presents ResT-ReID method, which designs a hybrid backbone Res-Transformer based on ResNet-50 and Transformer block for effective identify information. Specifically, we use global self-attention in place of depth-wise convolution in the fourth layer's residual bottleneck of ResNet-50. For fully exploiting the entire knowledge of the person, we devise attention-guided Graph Convolution Networks (GCNs) with side information embedding (SIE-AGCN), which has an attention layer located into two GCN layers. The quantified experiments on two large-scale ReID benchmarks demonstrate that the proposed ResT-ReID achieves competitive results compared with several state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Photo-realistic image synthesis is an attracting idea for person re-identification (ReID) and data augmentation on human pose estimation. However, existing advances manipulating human image synthesis lack texture details for varying poses or appearances. This paper presents a person image synthesis Siamese generative adversarial network (PS2GAN), which re-synthesizes person image by changing the pose of that person to a given pose, modeled in a Siamese structure with image generative network and pair conditional discriminative networks in single-branch. For pose transfer, the proposed PS2GAN adopts Siamese structure consisting of two image generative networks and a novel contrastive-pose loss regularizing the generation process. Additionally, a nearest-neighbor loss computes the difference between fake and real images to make high-level information closer. Furthermore, the proposed PS2GAN is competitive to the state-of-the-art performance on Market-1501 and DeepFashion datasets via qualitatively and quantitatively comparing with prior works, and synthetic images of the PS2GAN can alleviate data insufficiency for person ReID. [ABSTRACT FROM AUTHOR]

Person image synthesis based on shape and appearance using deep generative models opens the door in mickle applications, such as person re-identification (ReID) and movie industry. The methods of image synthesis are driven by producing the image of an object directly, which fail to recover spatial deformations when images are generated. In this paper, we present a conditional variational generative adversarial network (CVGAN) to synthesize desired images guided by target shape by modeling the inherent interplay between shape and appearance. Firstly, the shape and appearance of the given images are disentangled by adopting variational inference, which enables us to generate person images with arbitrary shapes. Secondly, to preserve the details and generate photo-realistic images, the Kullback–Leibler (KL) loss is adopted to reduce the gap between the condition image and generated image. Thirdly, to prevent partly gradient vanishing problem for training our framework stably, we propose combined general learning method, where the discriminative network leverages least squares loss. In addition, we experiment on COCO, DeepFashion and Market-1501 datasets, and results demonstrate that VGAN significantly improves the synthesis of images on discriminability, diversity and quality over the existing methods. [ABSTRACT FROM AUTHOR]