Search

The gradients of CNN are traditionally utilized for optimization and visualization. In this paper, we find that a discriminative representation hides in the gradients of convolution filters. Based on this, we propose a corresponding feature extraction and aggregation method for fine-grained image retrieval (FGIR). Firstly, we propose a metric to evaluate manually-designed loss functions and design a loss function originating from Grad-CAM in the testing phase based on it to extract the gradients of the convolution filters. Secondly, we take the gradients as the new features and design a succinct approach to aggregate them into a compact vector, which is named as Convolution Filters Gradient Aggregation (CFGA) feature. CFGA features can be extracted from pre-trained and fine-tuned CNN models. Extensive experiments are conducted on FGIR to verify the effectiveness of our proposed CFGA approach, compared with five supervised state-of-the-art methods and two unsupervised methods on two standard fine-grained retrieval datasets. Moreover, we generalize the CFGA method designed for CNN to Swin Transformer, and propose the Transformer parameter gradients aggregation (TPGA) method, which proves the applicability of the core idea of CFGA/TPGA to mainstream feature extraction models. We achieve state-of-the-art FGIR performance on CUB-200-2011 dataset and CARS196 dataset. • We design a novel feature aggregation method to attain the representation. • We propose a visual metric for evaluating. • The proposed method has extensive adaptability. [ABSTRACT FROM AUTHOR]

Standard deep learning models such as convolutional neural networks (CNNs) lack the ability of generalizing to domains which have not been seen during training. This problem is mainly due to the common but often wrong assumption of such models that the source and target data come from the same i.i.d. distribution. Recently, Vision Transformers (ViTs) have shown outstanding performance for a broad range of computer vision tasks. However, very few studies have investigated their ability to generalize to new domains. This paper presents a first Token-level Feature Stylization (TFS-ViT) approach for domain generalization, which improves the performance of ViTs to unseen data by synthesizing new domains. Our approach transforms token features by mixing the normalization statistics of images from different domains. We further improve this approach with a novel strategy for attention-aware stylization, which uses the attention maps of class (CLS) tokens to compute and mix normalization statistics of tokens corresponding to different image regions. The proposed method is flexible to the choice of backbone model and can be easily applied to any ViT-based architecture with a negligible increase in computational complexity. Comprehensive experiments show that our approach is able to achieve state-of-the-art performance on five challenging benchmarks for domain generalization, and demonstrate its ability to deal with different types of domain shifts. The implementation is available at this repository. • A new token-level stylization method for Domain Generalization is introduced. • We further enhance the method with a novel attention-driven styling strategy. • Our method is flexible and easily adaptable to any ViT, adding minimal complexity. • Our method shows superior performance in most cases on five challenging datasets. [ABSTRACT FROM AUTHOR]

Handwritten mathematical expression recognition (HMER) has attracted extensive attention. Despite the significant progress achieved in recent years attributed to the development of deep learning approaches, HMER remains a challenge due to the complex spatial structure and variable writing styles. Encoder–decoder models with attention mechanism, which treats HMER as an image-to-sequence (i.e. LaTeX) generation task, have boosted the accuracy, but suffer from low interpretability in that the symbols are not segmented explicitly. Symbol segmentation is desired for facilitating post-processing and human interaction in real applications. In this paper, we formulate the mathematical expression as a graph and propose a Graph-Encoder-Transformer-Decoder (GETD) approach for HMER. For constructing the graph from input image, candidate symbols are first detected using an object detector and represented as the nodes of a graph, called symbol graph, and the edges of the graph encodes the between-symbol relationship. The spatial information is aggregated in a graph neural network (GNN), and a Transformer-based decoder is used to identify the symbol classes and structure from the graph. Experiments on public datasets demonstrate that our GETD model achieves competitive expression recognition performance while offering good interpretability compared with previous methods. • A Graph-Encoder-Transformer-Decoder (GETD) approach is proposed for offline HMER, which has good interpretability. • The approach consists of a graph encoder (GNN) for aggregating spatial structure information and a Transformer decoder for generating recognition result. • Experiments on public datasets demonstrate that the proposed method achieves competitive recognition performance. [ABSTRACT FROM AUTHOR]

Long-term forecasting is of paramount importance in numerous scenarios, including predicting future energy, water, and food consumption. For instance, extreme weather events and natural disasters can profoundly impact infrastructure operations and pose severe safety concerns. Traditional CNN-based models often struggle to capture long-distance dependencies effectively. In contrast, Transformers-based models have shown significant promise in long-term forecasting. This paper investigates the long-term forecasting problem and identifies a common limitation in existing Transformer-based models: they tend to reduce computational complexity at the expense of time information aggregation capability. Moreover, the order of time series plays a crucial role in accurate predictions, but current Transformer-based models lack sensitivity to time series order, rendering them unreasonable. To address these issues, we propose a novel Deformable-Local (DL) aggregation mechanism. This mechanism enhances the model's ability to aggregate time information and allows the model to adaptively adjust the size of the time aggregation window. Consequently, the model can discern more complex time patterns, leading to more accurate predictions. Additionally, our model incorporates a Frequency Selection module to reinforce effective features and reduce noise. Furthermore, we introduce Position Weights to mitigate the order-insensitivity problem present in existing methods. In extensive evaluations of long-term forecasting tasks, we conducted benchmark tests on six datasets covering various practical applications, including energy, traffic, economics, weather, and disease. Our method achieved state-of-the-art (SOTA) results, demonstrating significant improvements. For instance, on the ETT dataset, our model achieved an average MSE improvement of approximately 19% and an average MAE improvement of around 27%. Remarkably, for predicted lengths of 96 and 192, we achieved outstanding MSE and MAE improvements of 32.1% and 30.9% , respectively. • Empirical Evaluation of High-Performance Time Series Forecasting Techniques for Single and Multivariate Data. • Our deformable mechanism-based module significantly enhances the performance of the model. • Our position-sensitive mechanism-based module brings a significant improvement to the model. [ABSTRACT FROM AUTHOR]

The rapid growth of Internet technology renders intrusion detection an important research topic in the field of pattern recognition. Considering that traffic data relate to not only temporal information, but also attack frequency, this paper presents a novel deep learning framework termed the multi-frequency intrusion detection network (MF-Net). MF-Net regards the pattern of Internet traffic as a superposition of sequential data with various frequencies, and is able to recognize the multi-frequency nature of network traffic data. The core of MF-Net is the multi-frequency LSTM (MF-LSTM) and multi-frequency transformer (MF-Transformer) module, both of which consist of high-frequency and low-frequency layers. In comparison with other state-of-the-art approaches on 4 public datasets, namely UNSW-NB15, KDD Cup 99, NSL-KDD and CICIDS 2017, as well as an IPv6 traffic dataset we created, MF-Net has shown better result in both binary and multi-class classification, which demonstrates the superiority of MF-Net over other compared approaches on network traffic intrusion detection. [Display omitted] • MF-Net captures multiple frequencies features with both long-term and short-term dependencies. • Both MF-LSTM and MF-Transformer detect network intrusion data with different frequencies. • MF-Net achieve state-of-the-art performance on 5 network intrusion datasets. [ABSTRACT FROM AUTHOR]

6DoF camera localization is an important component of autonomous driving and navigation. Deep learning has achieved impressive results in localization, but its robustness in dynamic environments has not been adequately addressed. In this paper, we propose a framework based on hybrid attention mechanism which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. Specifically, we propose a novel Transformer Bottleneck (TBo) block including convolution, channel attention, and a position-aware self-attention mechanism, which extracts more geometrically robust features by capturing the corresponding long-term dependencies between pixels. Furthermore, we introduce shuffle attention (SA) before the pose regressor, which integrates feature information in both spatial and channel dimensions, forcing the network to learn geometrically robust features, reducing the effects of dynamic objects and illumination conditions to improve camera localization accuracy. We evaluate our method on commonly benchmarked indoor and outdoor datasets and the experimental results show that our proposed method can significantly improve localization performance compared compare favorably to contemporary pose regressors schemes. In addition, extensive ablation evaluations are conducted to prove the effectiveness of our proposed hybrid attention bottleneck block for pose regression networks. • We propose a novel Transformer Bottleneck block with self-attention and channel attention to overcome the limitations of convolution. This coupling allows them to be optimized in a mutually reinforcing manner, significantly improving fine-grained feature extraction for accurate localization. • We propose a novel end-to-end hybrid attention network for single image localization, which improves the accuracy and robustness of camera localization, especially in dynamic scenes. • We conduct extensive experiments in both indoor and outdoor dataset, which shows that our model performs better than the existing competitive methods. [ABSTRACT FROM AUTHOR]