Search

Vision transformers have achieved encouraging progress in various computer vision tasks. A common belief is that this is attributed to the competence of self-attention in modeling the global dependencies among feature tokens. Unfortunately, self-attention still faces some challenges in dense prediction tasks, such as the high computational complexity and absence of desirable inductive bias. To address these issues, we revisit the potential benefits of integrating vision transformer with Gabor filter, and propose a Learnable Gabor Filter (LGF) by using convolution. As an alternative to self-attention, we employ LGF to simulate the response of simple cells in the biological visual system to input images, prompting models to focus on discriminative feature representations of targets from various scales and orientations. Additionally, we design a Bionic Focal Vision (BFV) block based on the LGF. This block draws inspiration from neuroscience and introduces a Multi-Path Feed Forward Network (MPFFN) to emulate the working way of biological visual cortex processing information in parallel. Furthermore, we develop a unified and efficient pyramid backbone network family called Focal Vision Transformers (FViTs) by stacking BFV blocks. Experimental results show that FViTs exhibit highly competitive performance in various vision tasks. Especially in terms of computational efficiency and scalability, FViTs show significant advantages compared with other counterparts. Code is available at https://github.com/nkusyl/FViT, Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Thanks to the advancement of deep learning technology, vision transformers has demonstrated competitive performance in various computer vision tasks. Unfortunately, vision transformers still faces some challenges such as high computational complexity and absence of desirable inductive bias. To alleviate these issues, we propose a novel Bi-Fovea Self-Attention (BFSA) inspired by the physiological structure and visual properties of eagle eyes. This BFSA is used to simulate the shallow and deep fovea of eagle vision, prompting the network to learn the feature representation of targets from coarse to fine. Additionally, we design a Bionic Eagle Vision (BEV) block based on BFSA. It combines the advantages of convolution and introduces a novel Bi-Fovea Feedforward Network (BFFN) to mimic the working way of biological visual cortex processes information in hierarchically and parallel. Furthermore, we develop a unified and efficient pyramid backbone network family called Eagle Vision Transformers (EViTs) by stacking BEV blocks. Experimental results show that EViTs exhibit highly competitive performance in various computer vision tasks such as image classification, object detection and semantic segmentation. Especially in terms of performance and computational efficiency, EViTs show significant advantages compared with other counterparts. Code is available at https://github.com/nkusyl/EViT, Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible