Search

Deep neural networks have achieved outstanding performance in computer vision tasks. Convolutional Neural Networks (CNNs) typically operate in the spatial domain with raw images, but in practice, images are usually stored and transmitted in their compressed representation where JPEG is one of the most widely used encoder. Also, these networks are computationally intensive and slow. This paper proposes performing the learning and inference processes in the compressed domain in order to reduce the computational complexity and improve the speed of popular CNNs. For this purpose, a novel graph-based frequency channel selection method is proposed to identify and select the most important frequency channels. The computational complexity is reduced by retaining the important frequency components and discarding the insignificant ones as well as eliminating the unnecessary layers of the network. Experimental results show that the modified ResNet-50 operating in the compressed domain is up to 70% faster than the spatial-based traditional ResNet-50 while resulting in similar classification accuracy. Moreover, this paper proposes a preprocessing step with partial encoding to improve the resilience to distortions caused by low-quality encoded images. Finally, we show that training a network with highly compressed data can achieve a good classification accuracy with up to 93% reduction in the storage requirements of the training data. [ABSTRACT FROM AUTHOR]

Although great success has been achieved in various computer vision tasks, deep neural networks (DNNs) suffer dramatic performance degradation when evaluated on out-of-distribution data. Domain generalization (DG) is proposed to handle this problem by learning domain-agnostic information from multiple source domains to generalize well on unseen target domains. Several methods resort to Fourier transform due to its simplicity and efficiency. They argue that amplitude spectra imply domain-specific information, which should be suppressed, while phase counterparts imply domain-agnostic information, which should be preserved. However, these methods only suppress the domain-specific information in source domains and neglect the relationship with target domains, leading to the persistence of the domain gap. Besides, these methods preserve domain-agnostic information by keeping phase components unchanged, causing them to be underutilized. In this paper, we propose Dual Branch Augmentation Module (DBAM) by leveraging Fourier transform and taking advantage of both amplitude and phase spectra. For the amplitude branch, we propose Inner-domain Amplitude Distribution Rectification (IADR) and Cross-domain Amplitude Dirichlet Mixup (CADM) to stabilize the training process and explore more feature space. In addition, we propose Test-time Amplitude Prototype Calibration (TAPC) to construct the connection between source and target domains during evaluation to further mitigate the domain gap. For the phase branch, we propose Random Symmetric Phase Perturbation (RSPP) to enhance the robustness for recognizing domain-agnostic information. With the combined contributions of the two branches, DBAM significantly surpasses other state-of-the-art (SOTA) methods. Extensive experiments on four benchmarks and further analysis demonstrate the effectiveness of DBAM. [ABSTRACT FROM AUTHOR]

Many large and complex deep neural networks have been shown to provide higher performance on various computer vision tasks. However, very little is known about the relationship between the complexity of the input data along with the type of noise and the depth needed for correct classification. Existing studies do not address the issue of common corruptions adequately, especially in understanding what impact these corruptions leave on the individual part of a deep neural network. Therefore, we can safely assume that the classification (or misclassification) might be happening at a particular layer(s) of a network that accumulates to draw a final correct or incorrect prediction. In this paper, we introduce a novel concept of corruption depth , which identifies the location of the network layer/depth until the misclassification persists. We assert that the identification of such layers will help in better designing the network by pruning certain layers in comparison to the purification of the entire network which is computationally heavy. Through our extensive experiments, we present a coherent study to understand the processing of examples through the network. Our approach also illustrates different philosophies of example memorization and a one-dimensional view of sample or query difficulty. We believe that the understanding of the corruption depth can open a new dimension of model explainability and model compression , where in place of just visualizing the attention map, the classification progress can be seen throughout the network. • A novel metric corruption depth is proposed to explain functioning of neural network. • The proposed research helps in deciphering the flow of classification decisions. • The proposed research overcomes the limitations of understanding example difficulty. • Corruption depth notation helps in pruning the networks while improving the accuracy. [ABSTRACT FROM AUTHOR]