Search

Single image super-resolution (SISR) is a classic ill-posed problem in computer vision. In recent years, deep-learning-based (DL-based) models have achieved promising results with the SISR problem. However, most existing methods suffer from an intrinsic trade-off between distortion and perceptual quality. To satisfy the requirements in different real-world situations, the balance of distortion and visual quality for image super-resolution is a critical issue. In DL-based models, the uses of hybrid loss (i.e., the combination of the distortion loss and the perceptual loss) and network interpolation are two common approaches to balancing the distortion and perceptual quality of super-resolved images. However, these two kinds of methods lack flexibility and hold strict constraints on network architectures. In this paper, we propose an image-fusion interpolation method for image super-resolution, which can balance the distortion and visual quality of super-resolved images, based on the optimal transport theory in the wavelet domain. The advantage of our proposed method is that it can be applied to any pretrained DL-based model, without any requirement from the network architecture and parameters. In addition, our proposed method is parameter-free and can run fast without using a GPU. Compared with existing state-of-the-art SISR methods, experiment results show that our proposed method can achieve a better balance between the distortion and visual quality in super-resolved images. [ABSTRACT FROM AUTHOR]

End-to-end multiple object tracking (MOT) has been very popular recently in the field of computer vision. However, when performing the inter-frame data association, the majority of algorithms ignore the diversity among trajectories and treat them uniformly. In the joint-detection-and-tracking (JDT) paradigm, there exists an optimization conflict between detection and re-identification due to their different optimization directions. In this paper, we propose a dynamic trajectory quantification (DTQ) strategy for MOT with feature rearrangement. Initially, the DTQ strategy quantifies each trajectory to show its quality, and the score is renewed every frame. To adapt to different situations, we dynamically adopt different updating mechanisms. Additionally, we provide a channel-enhanced feature rearrangement module to alleviate the optimization conflict between subtasks in the JDT paradigm, which obtains more convincing detection results for the DTQ strategy. We evaluate our proposed tracker on several benchmarks, i.e., MOT15, MOT16, MOT17, and MOT20. Extensive experimental results demonstrate that our method achieves competitive results while still maintaining a sizable real-time tracking speed. [ABSTRACT FROM AUTHOR]

The skeleton-based action recognition task has recently drawn much attention in computer vision, and graph convolutional networks (GCNs) have shown great advantages in this task. However, in shallow layers of GCNs, the sharing of convolution kernels in regular graph convolution (RGC) may result in the underutilization of some convolution kernels due to the particularity of the skeleton data. To capture richer locally discriminative action features, we propose the exclusive kernel graph convolution (EKGC) for the shallow layers of GCNs, which assigns the vertexes exclusive convolution kernels for graph convolution. At the same time, following the mixed graph convolution strategy, RGC is retained in the deep layers of the network, and the mixed graph convolutional network (MGCN) is proposed to utilize EKGC and RGC reasonably in all layers of the network. In addition, as a new form of skeleton information in GCN-based methods, angle information is explored in our work. Compared with joint data and bone data, angle data have better invariance against rotation and scaling, and they expand the relationships between skeletons and enrich the data representation of action. Finally, our final model, the multi-stream MGCN, is validated on two challenging datasets, achieving the state-of-the-art performance on the NTU-RGBD dataset and making competitive performance on a kinetics-skeleton dataset. [ABSTRACT FROM AUTHOR]

As a challenging task of video classification, action recognition has become a significant topic of computer vision community. The most popular methods based on two-stream architecture up to now are still simply fusing the prediction scores of each stream. In that case, the complementary characteristics of two streams cannot be fully utilized and the effect of shallower features is often overlooked. In addition, the equal treatment to features may weaken the role of the feature contributing significantly to the classification. Accordingly, a novel network called Multiple Depth-levels Features Fusion Enhanced Network (MDFFEN) is proposed. It improves on two aspects of two-stream architecture. In terms of the two-stream interaction mechanism, multiple depth-levels features fusion (MDFF) is formed to aggregate spatial–temporal features extracted from several sub-modules of original two streams by spatial–temporal features fusion (STFF). And with respect to further refining the spatiotemporal features, we propose a group-wise spatial-channel enhance (GSCE) module to highlight the meaningful regions and expressive channels automatically by priority assignment. The competitive results are achieved after we validate MDFFEN on three public challenging action recognition datasets, HDMB51, UCF101 and ChaLearn LAP IsoGD. • The proposed Two-Stream Features Fusion extracts diverse hybrid features. • The fusion of three types of features learns more discriminative information. • The Group-wise Spatial-Channel Enhance mechanism reduces interference information. • The proposed network achieves competitive performance. [ABSTRACT FROM AUTHOR]