Search

The emergence of low-cost depth cameras creates potential for RGB-D based human action recognition. However, most of the existing RGB-D based approaches simply concatenate original heterogeneous features without discovering the latent relations among different modalities. We propose a discriminative common structure learning (DCSL) model for human action recognition from RGB-D sequences. Specifically, we extract deep learning-based features and hand-crafted features from multimodal data (skeleton, depth, and RGB). In particular, we propose a deep architecture based on 3-D convolutional neural network to automatically extract deep spatiotemporal features from raw sequences. The proposed DCSL model utilizes a generalized version of collective matrix factorization to learn shared features among different modalities. To perform supervised learning and preserve intermodal similarity, we formulate a graph regularization term by considering both label information and similar geometric structure of multimodal data, which intends to improve the discriminative power of shared features. Moreover, we solve the objective function using an iterative optimization algorithm. Then, an improved collaborative representation classifier is employed to perform computationally efficient action recognition. Experimental results on four action datasets demonstrate the superior performance of the proposed method. [ABSTRACT FROM AUTHOR]

End-to-end multiple object tracking (MOT) has been very popular recently in the field of computer vision. However, when performing the inter-frame data association, the majority of algorithms ignore the diversity among trajectories and treat them uniformly. In the joint-detection-and-tracking (JDT) paradigm, there exists an optimization conflict between detection and re-identification due to their different optimization directions. In this paper, we propose a dynamic trajectory quantification (DTQ) strategy for MOT with feature rearrangement. Initially, the DTQ strategy quantifies each trajectory to show its quality, and the score is renewed every frame. To adapt to different situations, we dynamically adopt different updating mechanisms. Additionally, we provide a channel-enhanced feature rearrangement module to alleviate the optimization conflict between subtasks in the JDT paradigm, which obtains more convincing detection results for the DTQ strategy. We evaluate our proposed tracker on several benchmarks, i.e., MOT15, MOT16, MOT17, and MOT20. Extensive experimental results demonstrate that our method achieves competitive results while still maintaining a sizable real-time tracking speed. [ABSTRACT FROM AUTHOR]

Most existing multi-object tracking models only focus on the spatial information generated by image-level input while ignoring the necessary temporal information. The temporal motion information between consecutive frames can effectively reflect the target's motion status, which is essential to improving the performance in dealing with occlusion and motion blur of the model. We propose MISTracker to realize the motion information supplement to the original tracking model. Specifically, we divide multi-scale feature maps into two categories from the perspective of space and channel information. Meanwhile, the spatial-level frame differences processing (SFDP) module and the channel-level frame differences processing (CFDP) module are proposed to deal with these differences between continuous frames, respectively. The SFDP processes the differences from the perspective of spatial information and supplements the motion information through the perception of pixel-level information changes in the feature maps. The CFDP processes the differences from the perspective of channels and enhances the information of motion-sensitive channels through the overall pixel differences of different channels. Eventually, temporal and motion information are complementary to each other after upsampling fusion. The whole process is realized by simple convolution, which reduces the computational force as much as possible and enhances the tracking performance of the model. [ABSTRACT FROM AUTHOR]

The skeleton-based action recognition task has recently drawn much attention in computer vision, and graph convolutional networks (GCNs) have shown great advantages in this task. However, in shallow layers of GCNs, the sharing of convolution kernels in regular graph convolution (RGC) may result in the underutilization of some convolution kernels due to the particularity of the skeleton data. To capture richer locally discriminative action features, we propose the exclusive kernel graph convolution (EKGC) for the shallow layers of GCNs, which assigns the vertexes exclusive convolution kernels for graph convolution. At the same time, following the mixed graph convolution strategy, RGC is retained in the deep layers of the network, and the mixed graph convolutional network (MGCN) is proposed to utilize EKGC and RGC reasonably in all layers of the network. In addition, as a new form of skeleton information in GCN-based methods, angle information is explored in our work. Compared with joint data and bone data, angle data have better invariance against rotation and scaling, and they expand the relationships between skeletons and enrich the data representation of action. Finally, our final model, the multi-stream MGCN, is validated on two challenging datasets, achieving the state-of-the-art performance on the NTU-RGBD dataset and making competitive performance on a kinetics-skeleton dataset. [ABSTRACT FROM AUTHOR]