Search

Fine-grained object recognition (FGOR) aims to learn discriminative features that can identify the subtle distinctions between visually similar objects. However, less effort has been devoted to overcoming the impact of object's personalized differences, e.g., varying posture or perspective. We argue that the personalized differences could decline the network's perception of discriminative features, thus discarding some discriminative clues and degrading the FGOR performance accordingly. This motivates us to explore the intrinsic structure knowledge: the fixed spatial correlation between object parts, and thus apply this knowledge to associate diverse semantic parts and recover the missing discriminative details caused by the personalized differences accordingly. In this paper, we propose an end-to-end Structure-driven Relation Graph Network (SRGN) for fine-grained object recognition, and target at exploring and exploiting the object structure information without any additional annotations to associate diverse semantic parts, making the network sensitive to discriminative details influenced by personalized differences. Specifically, the core of SRGN is a Structure-aware Axial Graph (SAG) module, which first infers the structure embedding by establishing the correlation between position information and visual features along the axial direction, and then applies this embedding as aggregation weights to emphasize each discriminative representation by weighted reassembling all relevant features to it. Additionally, our SAG can be readily extensible to a multi-graph schema, that leverages the complementary advantages of different structure embeddings between the position information and visual content, further improving SAG. In this way, our SRGN can demonstrate remarkable robustness in scenarios characterized by extreme distribution perturbations, ultimately leading to superior performance. Extensive experiments and explainable visualizations validate the efficacy of the proposed approach on widely-used fine-grained benchmarks. [ABSTRACT FROM AUTHOR]

Graph convolution networks (GCNs) based methods for 3D human pose estimation usually aggregate immediate features of single-hop nodes, which are unaware of the correlation of multi-hop nodes and therefore neglect long-range dependency for predicting complex poses. In addition, they typically operate either on single-scale or sequential down-sampled multi-scale graph representations, resulting in the loss of contextual information or spatial details. To address these problems, this paper proposes a parallel hop-aware graph attention network (PHGANet) for 3D human pose estimation, which learns enriched hop-aware correlation of the skeleton joints while maintaining the spatially-precise representations of the human graph. Specifically, we propose a hop-aware skeletal graph attention (HSGAT) module to capture the semantic correlation of multi-hop nodes, which first calculates skeleton-based 1-hop attention and then disseminates it to arbitrary hops via graph connectivity. To alleviate the redundant noise introduced by the interactions with distant nodes, HSGAT uses an attenuation strategy to separate attention from distinct hops and assign them learnable attenuation weights according to their distances adaptively. Upon HSGAT, we further build PHGANet with multiple parallel branches of stacked HSGAT modules to learn the enriched hop-aware correlation of human skeletal structures at different scales. In addition, a joint centrality encoding scheme is proposed to introduce node importance as a bias in the learned graph representation, which makes the core joints (e.g., neck and pelvis) more influential during node aggregation. Experimental results indicate that PHGANet performs favorably against state-of-the-art methods on the Human3.6M and MPI-INF-3DHP benchmarks. Models and code are available at https://github.com/ChenyangWang95/PHGANet/. [ABSTRACT FROM AUTHOR]

Deploying convolutional neural networks (CNNs) on mobile devices is difficult due to the limited memory and computation resources. We aim to design efficient neural networks for heterogeneous devices including CPU and GPU, by exploiting the redundancy in feature maps, which has rarely been investigated in neural architecture design. For CPU-like devices, we propose a novel CPU-efficient Ghost (C-Ghost) module to generate more feature maps from cheap operations. Based on a set of intrinsic feature maps, we apply a series of linear transformations with cheap cost to generate many ghost feature maps that could fully reveal information underlying intrinsic features. The proposed C-Ghost module can be taken as a plug-and-play component to upgrade existing convolutional neural networks. C-Ghost bottlenecks are designed to stack C-Ghost modules, and then the lightweight C-GhostNet can be easily established. We further consider the efficient networks for GPU devices. Without involving too many GPU-inefficient operations (e.g., depth-wise convolution) in a building stage, we propose to utilize the stage-wise feature redundancy to formulate GPU-efficient Ghost (G-Ghost) stage structure. The features in a stage are split into two parts where the first part is processed using the original block with fewer output channels for generating intrinsic features, and the other are generated using cheap operations by exploiting stage-wise redundancy. Experiments conducted on benchmarks demonstrate the effectiveness of the proposed C-Ghost module and the G-Ghost stage. C-GhostNet and G-GhostNet can achieve the optimal trade-off of accuracy and latency for CPU and GPU, respectively. MindSpore code is available at https://gitee.com/mindspore/models/pulls/1809, and PyTorch code is available at https://github.com/huawei-noah/CV-Backbones. [ABSTRACT FROM AUTHOR]

It is an important problem to design resource-efficient neural architectures. One solution is adjusting the number of channels in each layer and the number of blocks in each network stage. This paper presents a novel framework named network adjustment which considers accuracy as a function of the computational resource (e.g., FLOPs or parameters), so that architecture design becomes an optimization problem and can be solved with the gradient-based optimization method. The gradient is defined as the resource utilization ratio (RUR) of each changeable module (layer or block) in a network and is accurate only in a small neighborhood of the current status. Therefore, we estimate it using Dropout, a probabilistic operation, and optimize the network architecture iteratively. The computational overhead of the entire process is comparable to that of re-training the final model from scratch. We investigate two versions of RUR where the resource usage is measured by FLOPs and latency. Experiments on standard image classification datasets and a few base networks including ResNet and EfficientNet demonstrate the effectiveness of our approach, which consistently outperforms the pruning-based counterparts. [ABSTRACT FROM AUTHOR]

In computer vision, an important challenge to deep neural networks comes from adjusting the varying properties of different image domains. To study this problem, researchers have been investigating a practical setting in which the deep neural networks are trained on a labeled source domain and then transferred to an unlabeled or even unseen target domain. The major difficulty lies in the potential domain gap, which essentially arises from the overfitting in the source domain. Hence, it is important to introduce generalized priors to alleviate the issue. From this perspective, this paper presents a novel framework that forces visual features to align with domain-agnostic priors (DAP). Specifically, we study two kinds of priors, (i) language-guided embedding and (ii) class-level relationship, and we believe that more such priors can be constructed. Our framework, referred to as DAP, is evaluated on both unsupervised domain adaptation (UDA) and domain generalization (DG) where the target domain is unlabeled and even unseen, respectively. We use the standard benchmark that performs transfer semantic segmentation on synthesized datasets (i.e., GTAv and SYNTHIA) and a real dataset (i.e., Cityscapes). Experiments validate the effectiveness of DAP with competitive accuracy in all tasks. In particular, language-guided priors work sufficiently well for UDA, while class-level priors serve as useful complements for DG. The proposed frameworks shed light that domain transfer benefits from better proxies, possibly from other modalities. [ABSTRACT FROM AUTHOR]