Search

• A pose-position attention strategy is proposed to update the bi-modal visual graph. • A linguistic-embedding-based semantic graph is presented to model label relations. • A semantic-preserving loss is designed to maintain the semantics consistency. • Both visual and semantic information are fused for group activity recognition. Video-based group activities typically contain interactive contexts among diverse visual modalities between multiple persons, and semantic relationships between individual actions. Nevertheless, majority of the existing methods for recognizing group activity either captures the relationships among different persons by utilizing a solely RGB modality or neglect to exploit the label hierarchies between individual actions and the group activity. To tackle these issues, we propose a visual-semantic graph neural network, with pose-position attentive learning (VSGNN-PAL), for group activity recognition. Specifically, we first extract the individual-level appearance and motion representations from RGB and optical-flow inputs, to build a bi-modal visual graph. Two attentive aggregators are further proposed to integrate both the pose and position information to measure the relevance scores between persons, and dynamically refine the representation of each visual node from both modality-specific and cross-modal perspectives. To model a semantic hierarchy from a label space, we construct a semantic graph based on the linguistic embeddings of individual actions and group activity labels. We further employ a bi-directional mapping learning scheme, to integrate the label-relation-aware semantic context into the visual representations. Besides, a global reasoning module is introduced to progressively generate the group-level representations with the scene description maintained. Furthermore, we formulate a semantic-preserving loss, to maintain the consistency between the learned high-level representations and the semantics of the ground-truth labels. Experimental results on three group activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SISR) is a classic ill-posed problem in computer vision. In recent years, deep-learning-based (DL-based) models have achieved promising results with the SISR problem. However, most existing methods suffer from an intrinsic trade-off between distortion and perceptual quality. To satisfy the requirements in different real-world situations, the balance of distortion and visual quality for image super-resolution is a critical issue. In DL-based models, the uses of hybrid loss (i.e., the combination of the distortion loss and the perceptual loss) and network interpolation are two common approaches to balancing the distortion and perceptual quality of super-resolved images. However, these two kinds of methods lack flexibility and hold strict constraints on network architectures. In this paper, we propose an image-fusion interpolation method for image super-resolution, which can balance the distortion and visual quality of super-resolved images, based on the optimal transport theory in the wavelet domain. The advantage of our proposed method is that it can be applied to any pretrained DL-based model, without any requirement from the network architecture and parameters. In addition, our proposed method is parameter-free and can run fast without using a GPU. Compared with existing state-of-the-art SISR methods, experiment results show that our proposed method can achieve a better balance between the distortion and visual quality in super-resolved images. [ABSTRACT FROM AUTHOR]

High dynamic range (HDR) imaging is an important task in image processing that aims to generate well-exposed images in scenes with varying illumination. Although existing multi-exposure fusion methods have achieved impressive results, generating high-quality HDR images in dynamic scenes remains difficult. The primary challenges are ghosting artifacts caused by object motion between low dynamic range images and distorted content in underexposure and overexposed regions. In this paper, we propose a deep progressive feature aggregation network for improving HDR imaging quality in dynamic scenes. To address the issues of object motion, our method implicitly samples high-correspondence features and aggregates them in a coarse-to-fine manner for alignment. In addition, our method adopts a densely connected network structure based on the discrete wavelet transform, which aims to decompose the input features into multiple frequency subbands and adaptively restore corrupted contents. Experiments show that our proposed method can achieve state-of-the-art performance under different scenes, compared to other promising HDR imaging methods. Specifically, the HDR images generated by our method contain cleaner and more detailed content, with fewer distortions, leading to better visual quality. [ABSTRACT FROM AUTHOR]

• The ADJST realizes users' anomalous electricity consumption behaviors detection. • The MS-GCN achieves extracting spatial features and preventing feature coupling. • The MS-TCN module reduces the loss of time sequence information in long sequences. • Enhance the model's stability and generalization by joint spatio-temporal learning. The use of electric energy is an integral part of people's daily life. Anomaly detection of electricity consumption data, as a classification problem, has always been a hot research topic of scholars. Anomaly detection can not only reduce energy waste, but also prevent small problems from becoming overwhelming problems. At present, most anomaly detection algorithms mainly focus on the time series information of electricity consumption data, while ignoring the spatial feature of electricity consumption data. To fill this research gap, the paper proposes an Anomaly Detection based on Joint Spatio-Temporal learning (ADJST) method for building electricity consumption. First, a Multi-Scale Graph Convolutional Network (MS-GCN) is proposed to learn the spatial features of building electricity consumption data. Specifically, two types of graphs are constructed to extract short-term correlation features and long-term regularity features of building electricity consumption data. Second, a Multi-Scale Temporal Convolutional Network (MS-TCN) is proposed to learn the temporal features of building electricity consumption data. Adopt a multi-scale vanilla convolution structure to extract multi-scale time series information from building electricity consumption data. Third, the combination of temporal features and spatial features detects anomalous electricity consumption of marked users. Final, taken the user electricity consumption data collected by the State Grid Corporation's smart meter as examples, compared with a variety of classical anomaly detection algorithms, the results of F1-score and AUC of the proposed method are 0.935 and 0.977 respectively, which proves the superiority of the method. The model shows good stability in dealing with extreme imbalance of data, and is proved to be generalized by experiments and can be transferred to other datasets. [ABSTRACT FROM AUTHOR]