Search

• The ADNMT uses two asymmetric networks to generate pseudo-labels for each other by clustering and iterative training. • Based on alternate training and mutual teaching to optimize the pseudo-labels. • The SCIC compensates the similarity distance of images captured by inter-camera. • The SSIC further suppresses the similarity distance by calculate the similarity distribution. Unsupervised person re-identification (Re-ID) has always been challenging in computer vision. It has received much attention from researchers because it does not require any labeled information and can be freely deployed to new scenarios. Most unsupervised person Re-ID research studies produce and optimize pseudo-labels by iterative clustering algorithms on a single network. However, these methods are easily affected by noisy labels and feature variations caused by camera shifts, which will limit the optimization of pseudo-labels. In this paper, we propose an Asymmetric Double Networks Mutual Teaching (ADNMT) architecture that uses two asymmetric networks to generate pseudo-labels for each other by clustering, and the pseudo-labels are updated and optimized by alternate training. Specifically, ADNMT contains two asymmetric networks. One network is a multiple granularity network, which extracts pedestrian features of multiple granularity that correspond to numerous classifiers, and the other network is a conventional backbone network, which extracts pedestrian features that correspond to a classifier. Furthermore, because the camera style changes seriously affect the generalization ability of the proposed model, this paper designs Similarity Compensation of Inter-Camera (SCIC) and Similarity Suppression of Intra-Camera (SSIC) according to the camera ID of the pedestrian images to optimize the similarity measure. Extensive experiments on multiple Re-ID benchmark datasets show that our proposed method achieves superior performance compared with the state‐of‐the‐art unsupervised person re-identification methods. [ABSTRACT FROM AUTHOR]

Deep neural networks have achieved outstanding performance in computer vision tasks. Convolutional Neural Networks (CNNs) typically operate in the spatial domain with raw images, but in practice, images are usually stored and transmitted in their compressed representation where JPEG is one of the most widely used encoder. Also, these networks are computationally intensive and slow. This paper proposes performing the learning and inference processes in the compressed domain in order to reduce the computational complexity and improve the speed of popular CNNs. For this purpose, a novel graph-based frequency channel selection method is proposed to identify and select the most important frequency channels. The computational complexity is reduced by retaining the important frequency components and discarding the insignificant ones as well as eliminating the unnecessary layers of the network. Experimental results show that the modified ResNet-50 operating in the compressed domain is up to 70% faster than the spatial-based traditional ResNet-50 while resulting in similar classification accuracy. Moreover, this paper proposes a preprocessing step with partial encoding to improve the resilience to distortions caused by low-quality encoded images. Finally, we show that training a network with highly compressed data can achieve a good classification accuracy with up to 93% reduction in the storage requirements of the training data. [ABSTRACT FROM AUTHOR]

Image Salient Object Detection (SOD) is a fundamental research topic in the area of computer vision. Recently, the multimodal information in RGB, Depth (D), and Thermal (T) modalities has been proven to be beneficial to the SOD. However, existing methods are only designed for RGB-D or RGB-T SOD, which may limit the utilization in various modalities, or just finetuned on specific datasets, which may bring about extra computation overhead. These defects can hinder the practical deployment of SOD in real-world applications. In this paper, we propose an end-to-end U nified T riplet D ecoder Net work, dubbed UTDNet, for both RGB-T and RGB-D SOD tasks. The intractable challenges for the unified multimodal SOD are mainly two-fold, i.e., (1) accurately detecting and segmenting salient objects, and (2) preferably via a single network that fits both RGB-T and RGB-D SOD. First, to deal with the former challenge, we propose the multi-scale feature extraction unit to enrich the discriminative contextual information, and the efficient fusion module to explore cross-modality complementary information. Then, the multimodal features are fed to the triplet decoder, where the hierarchical deep supervision loss further enable the network to capture distinctive saliency cues. Second, as to the latter challenge, we propose a simple yet effective continual learning method to unify multimodal SOD. Concretely, we sequentially train multimodal SOD tasks by applying Elastic Weight Consolidation (EWC) regularization with the hierarchical loss function to avoid catastrophic forgetting without inducing more parameters. Critically, the triplet decoder separates task-specific and task-invariant information, making the network easily adaptable to multimodal SOD tasks. Extensive comparisons with 26 recently proposed RGB-T and RGB-D SOD methods demonstrate the superiority of the proposed UTDNet. [ABSTRACT FROM AUTHOR]

Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or events that deviate from normal behavior. In recent years, significant progress has been made in this field due to the remarkable growth of deep learning models. Notably, the advent of self-supervised learning has sparked the development of novel AD algorithms that outperform the existing state-of-the-art approaches by a considerable margin. This paper aims to provide a comprehensive review of the current methodologies in self-supervised anomaly detection. We present technical details of the standard methods and discuss their strengths and drawbacks. We also compare the performance of these models against each other and other state-of-the-art anomaly detection models. Finally, the paper concludes with a discussion of future directions for self-supervised anomaly detection, including the development of more effective and efficient algorithms and the integration of these techniques with other related fields, such as multi-modal learning. • We present a cohesive overview of self-supervised methods for anomaly detection. • We categorize existing self-supervised anomaly detection algorithms. their proxy tasks. • We conduct a comprehensive performance comparison. • We discuss future directions in self-supervised anomaly detection. [ABSTRACT FROM AUTHOR]

Although 3D human pose estimation has recently made strides, it is still difficult to precisely recreate a 3D human posture from a single image without the aid of 3D annotation for the following reasons. Firstly, the process of reconstruction inherently suffers from ambiguity, as multiple 3D poses can be projected onto the same 2D pose. Secondly, accurately measuring camera rotation without laborious camera calibration is a difficult task. While some approaches attempt to address these issues using traditional computer vision algorithms, they are not differentiable and cannot be optimized through training. This paper introduces two modules that explicitly leverage geometry to overcome these challenges, without requiring any 3D ground-truth or camera parameters. The first module, known as the relative depth estimation module, effectively mitigates depth ambiguity by narrowing down the possible depths for each joint to only two candidates. The second module, referred to as the differentiable pose alignment module, calculates camera rotation by aligning poses from different views. The use of these geometrically interpretable modules reduces the complexity of training and yields superior performance. By adopting our proposed method, we achieve state-of-the-art results on standard benchmark datasets, surpassing other self-supervised methods and even outperforming several fully-supervised approaches that heavily rely on 3D annotations. [ABSTRACT FROM AUTHOR]

Supervised learning-based image classification in computer vision relies on visual samples containing a large amount of labeled information. Considering that it is labor-intensive to collect and label images and construct datasets manually, Zero-Shot Learning (ZSL) achieves knowledge transfer from seen categories to unseen categories by mining auxiliary information, which reduces the dependence on labeled image samples and is one of the current research hotspots in computer vision. However, most ZSL methods fail to properly measure the relationships between classes, or do not consider the differences and similarities between classes at all. In this paper, we propose Adaptive Relation-Aware Network (ARAN), a novel ZSL approach that incorporates the improved triplet loss from deep metric learning into a VAE-based generative model, which helps to model inter-class and intra-class relationships for different classes in ZSL datasets and generate an arbitrary amount of high-quality visual features containing more discriminative information. Moreover, we validate the effectiveness and superior performance of our ARAN through experimental evaluations under ZSL and more practical GZSL settings on three popular datasets AWA2, CUB, and SUN. [ABSTRACT FROM AUTHOR]

Vision transformers (ViTs) have become one of the dominant frameworks for vision tasks in recent years because of their ability to efficiently capture long-range dependencies in image recognition tasks using self-attention. In fact, both CNNs and ViTs have advantages and disadvantages in vision tasks, and some studies suggest that the use of both may be an effective way to balance performance and computational cost. In this paper, we propose a new hybrid network based on CNN and transformer, using CNN to extract local features and transformer to capture long-distance dependencies. We also proposed a new feature map resolution reduction based on Discrete Cosine Transform and self-attention, named DCT-Attention Down-sample (DAD). Our DctViT-L achieves 84.8% top-1 accuracy on ImageNet 1K, far outperforming CMT, Next-ViT, SpectFormer and other state-of-the-art models, with lower computational costs. Using DctViT-B as the backbone, RetinaNet can achieve 46.8% mAP on COCO val2017, which improves mAP by 2.5% and 1.1% with less calculation cost compared with CMT-S and SpectFormer as the backbone. • We propose a DAD block for feature map compression and feature extraction using DCT. • We design new convolutional blocks and transformer blocks. • We propose a CNN and Transformer hybrid neural network family DctViT. • We experimentally demonstrate the superior performance of the DctViT network. [ABSTRACT FROM AUTHOR]

Vision Transformer (ViT) has performed remarkably in various computer vision tasks. Nonetheless, affected by the massive amount of parameters, ViT usually suffers from serious overfitting problems with a relatively limited number of training samples. In addition, ViT generally demands heavy computing resources, which limit its deployment on resource-constrained devices. As a type of model-compression method, model binarization is potentially a good choice to solve the above problems. Compared with the full-precision one, the model with the binarization method replaces complex tensor multiplication with simple bit-wise binary operations and represents full-precision model parameters and activations with only 1-bit ones, which potentially solves the problem of model size and computational complexity, respectively. In this paper, we investigate a binarized ViT model. Empirically, we observe that the existing binarization technology designed for Convolutional Neural Networks (CNN) cannot migrate well to a ViT's binarization task. We also find that the decline of the accuracy of the binary ViT model is mainly due to the information loss of the Attention module and the Value vector. Therefore, we propose a novel model binarization technique, called G roup S uperposition B inarization (GSB), to deal with these issues. Furthermore, in order to further improve the performance of the binarization model, we have investigated the gradient calculation procedure in the binarization process and derived more proper gradient calculation equations for GSB to reduce the influence of gradient mismatch. Then, the knowledge distillation technique is introduced to alleviate the performance degradation caused by model binarization. Analytically, model binarization can limit the parameter's search space during parameter updates while training a model. Therefore, the binarization process can actually play an implicit regularization role and help solve the problem of overfitting in the case of insufficient training data. Experiments on three datasets with limited numbers of training samples demonstrate that the proposed GSB model achieves state-of-the-art performance among the binary quantization schemes and exceeds its full-precision counterpart on some indicators. Code and models are available at: https://github.com/IMRL/GSB-Vision-Transformer. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks (CNNs) have revolutionized image classification through their innovative design and training methodologies in computer vision. Differential convolutional neural network with simultaneous multidimensional filter realization improved the performance of the convolutional neural network with calculation cost drawback. This paper introduces logarithmic learning integration into the differential Convolutional neural network to overcome the drawback by supplying faster error minimization and convergence. This task is done by incorporating LogRelu activation, a Logarithmic Cost Function, and unique logarithmic learning method. The effectiveness of the proposed approaches are evaluated by using various datasets and SGD/Adam optimizers. The first step is the adaptation of LogRelu activation function to convolutional and differential convolutional neural networks. The experiment results show that LogRelu integration to convolutional neural network and differential convolutional neural network yields performance improvements ranging from 1.61% to 5.44%. The same integration on ResNet-18, ResNet-34, and ResNet-50 enhances top-1 accuracy in the range of 3.07% and 9.96%. In addition to LogRelu activation function, a Logarithmic Cost Function with logarithmic learning method is also proposed and adapted to differential convolutional neural network. These improvements lead to a new differential convolutional neural network named as Logarithmic Differential Convolutional Neural Network (LDiffCNN), It consistently outperforms standard CNN by increasing the accuracy up to 3.02%. Notably, Logarithmic Differential Convolutional Neural Network demonstrates reduced training iterations up to 38% with faster convergence. The experimental results proved the efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

Recent advances in unsupervised domain adaptation have shown that mitigating the domain divergence by extracting the domain-invariant features could significantly improve the generalization of a model with respect to a new data domain. However, current methodologies often neglect to retain domain private information, which is the unique information inherent to the unlabeled new domain, compromising generalization. This paper presents a novel method that utilizes mutual information to protect this domain-specific information, ensuring that the latent features of the unlabeled data not only remain domain-invariant but also reflect the unique statistics of the unlabeled domain. We show that simultaneous maximization of mutual information and reduction of domain divergence can effectively preserve domain-private information. We further illustrate that a neural estimator can aptly estimate the mutual information between the unlabeled input space and its latent feature space. Both theoretical analysis and empirical results validate the significance of preserving such unique information of the unlabeled domain for cross-domain generalization. Comparative evaluations reveal our method's superiority over existing state-of-the-art techniques across multiple benchmark datasets. [ABSTRACT FROM AUTHOR]

Medical image segmentation faces challenges because of the small sample size of the dataset and the fact that images often have noise and artifacts. In recent years, diffusion models have proven very effective in image generation and have been used widely in computer vision. This paper presents a new feature map denoising module (FMD) based on the diffusion model for feature refinement, which is plug-and-play, allowing flexible integration into popular used segmentation networks for seamless end-to-end training. We evaluate the performance of the FMD module on four models, UNet, UNeXt, TransUNet, and IB-TransUNet, by conducting experiments on four datasets. The experimental data analysis shows that adding the FMD module significantly positively impacts the model performance. Furthermore, especially for small lesion areas and minor organs, adding the FMD module allows users to obtain more accurate segmentation results than the original model. [ABSTRACT FROM AUTHOR]

Although great success has been achieved in various computer vision tasks, deep neural networks (DNNs) suffer dramatic performance degradation when evaluated on out-of-distribution data. Domain generalization (DG) is proposed to handle this problem by learning domain-agnostic information from multiple source domains to generalize well on unseen target domains. Several methods resort to Fourier transform due to its simplicity and efficiency. They argue that amplitude spectra imply domain-specific information, which should be suppressed, while phase counterparts imply domain-agnostic information, which should be preserved. However, these methods only suppress the domain-specific information in source domains and neglect the relationship with target domains, leading to the persistence of the domain gap. Besides, these methods preserve domain-agnostic information by keeping phase components unchanged, causing them to be underutilized. In this paper, we propose Dual Branch Augmentation Module (DBAM) by leveraging Fourier transform and taking advantage of both amplitude and phase spectra. For the amplitude branch, we propose Inner-domain Amplitude Distribution Rectification (IADR) and Cross-domain Amplitude Dirichlet Mixup (CADM) to stabilize the training process and explore more feature space. In addition, we propose Test-time Amplitude Prototype Calibration (TAPC) to construct the connection between source and target domains during evaluation to further mitigate the domain gap. For the phase branch, we propose Random Symmetric Phase Perturbation (RSPP) to enhance the robustness for recognizing domain-agnostic information. With the combined contributions of the two branches, DBAM significantly surpasses other state-of-the-art (SOTA) methods. Extensive experiments on four benchmarks and further analysis demonstrate the effectiveness of DBAM. [ABSTRACT FROM AUTHOR]

Many large and complex deep neural networks have been shown to provide higher performance on various computer vision tasks. However, very little is known about the relationship between the complexity of the input data along with the type of noise and the depth needed for correct classification. Existing studies do not address the issue of common corruptions adequately, especially in understanding what impact these corruptions leave on the individual part of a deep neural network. Therefore, we can safely assume that the classification (or misclassification) might be happening at a particular layer(s) of a network that accumulates to draw a final correct or incorrect prediction. In this paper, we introduce a novel concept of corruption depth , which identifies the location of the network layer/depth until the misclassification persists. We assert that the identification of such layers will help in better designing the network by pruning certain layers in comparison to the purification of the entire network which is computationally heavy. Through our extensive experiments, we present a coherent study to understand the processing of examples through the network. Our approach also illustrates different philosophies of example memorization and a one-dimensional view of sample or query difficulty. We believe that the understanding of the corruption depth can open a new dimension of model explainability and model compression , where in place of just visualizing the attention map, the classification progress can be seen throughout the network. • A novel metric corruption depth is proposed to explain functioning of neural network. • The proposed research helps in deciphering the flow of classification decisions. • The proposed research overcomes the limitations of understanding example difficulty. • Corruption depth notation helps in pruning the networks while improving the accuracy. [ABSTRACT FROM AUTHOR]