Search

Using 3D reconstruction techniques within computer vision frameworks can result in more robust and accurate solutions. However, the main challenge lies in the high computation and memory resources required by such methods. To reduce the complexity of these frameworks, a practical solution is to use geometric landmarks instead of the entire image. Therefore, in this paper the problem of 3D shape recovery of a set of standard 2D landmarks, on a single human face image is faced using multi-dimensional scaling (MDS) approach to find a 3D embedding for a set of standard 2D points. Hence, MDS approach is used for the first time in this study to establish an unbiased mapping from 2D landmark space to the corresponding 3D shape space. A deep neural network learns the pairwise 3D dissimilarity among standard 2D landmarks. This scheme leads to find a symmetric dissimilarity matrix, to be fed into the MDS approach to appropriately recovering the 3D shape of corresponding 2D landmarks. In the case of complex input image formations like posedness or perspective projection causing occlusion in the input image, an autoencoder component is used in the proposed framework, as an occlusion removal part, which turns different input views of the human face into a profile view. The results of performance evaluation using variety of synthetic and real-world human face datasets, including NoW dataset, Besel Face Model (BFM), CelebA, CoMA - FLAME, and CASIA-3D, indicates the superiority of the proposed framework, despite its small number of training parameters, with the related state-of-the-art and recent 3D shape recovery of landmark methods from the literature, in terms of unbiasedness, independent projection, efficiency and accuracy. Further, ablation study is performed to find the best training scheme of the deep learning components. All codes and public data of our paper are publicly available for research purposes at https://github.com/s2kamyab/DeepMDS. [ABSTRACT FROM AUTHOR]

• Proposing a shift-insensitive perceptual feature based on GM and LoG signals, and theoretically selecting a ratio parameter to balance them. • Building FR-IQA models based on the proposed QGL feature maps that achieve good performance on three benchmark databases. • Achieving the best shift-insensitive performance on images with spatial changes compared with other metrics. • Combining QGL with deep neural networks and validating that QGL can help improving the accuracy in texture classification. Most existing full-reference (FR) Image quality assessment (IQA) models work in the premise of that the two images should be well registered. Shifting an image would lead to an inaccurate evaluation of image quality, because small spatial shifts are far less noticeable than structural distortion for human observers. To this regard, we propose to study an IQA feature that is shift-insensitive to the basic primitive structure of images, i.e., image edge. According to previous studies, the image gradient magnitude (GM) and the Laplacian of Gaussian (LoG) operator that depict the edge profiles of natural images are highly efficient structural features in IQA tasks. In this paper, we find that the Quadratic sum of the normalized GM and the LoG signals (QGL) has excellent shift-insensitive property in representing image edges after theoretically solving the selection problem of a ratio parameter to balance the GM and LoG signals. Based on the proposed QGL feature, two FR-IQA models can be built directly by measuring the similarity map with mean and standard deviation pooling strategies, named mQGL and sQGL, respectively. Experimental results show that the proposed sQGL and mQGL work robustly on four benchmark IQA databases, and QGL-based models show great shift-insensitive property to spatial translation and image rotation while judging the image quality. In addition, we explore the feasibility of combining QGL feature with deep neural networks, and verify that it can help to promote image pattern recognition in texture classification tasks. [ABSTRACT FROM AUTHOR]

Accurate 3D human pose estimation is a challenging task due to occlusion and depth ambiguity. In this paper, we introduce a multi-hop graph transformer network designed for 2D-to-3D human pose estimation in videos by leveraging the strengths of multi-head self-attention and multi-hop graph convolutional networks with disentangled neighborhoods to capture spatio-temporal dependencies and handle long-range interactions. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix, and a multi-hop graph convolutional block comprised of multi-hop convolutional and dilated convolutional layers. The combination of multi-head self-attention and multi-hop graph convolutional layers enables the model to capture both local and global dependencies, while the integration of dilated convolutional layers enhances the model's ability to handle spatial details required for accurate localization of the human body joints. Extensive experiments demonstrate the effectiveness and generalization ability of our model, achieving competitive performance on benchmark datasets. [ABSTRACT FROM AUTHOR]

Gait recognition is an important biometric technology that allows for the remote collection of stakeholders' characteristics, without requiring their explicit cooperation. It has gained considerable attention in the fields of criminal investigation and intelligent security. Previous studies have shown that local gait features can enhance gait recognition performance by improving robustness to disturbances. However, global gait features also play a crucial role in gait recognition. Many researchers have utilized convolutional operations to extract global features, but these operations tend to focus on features within the receptive field, neglecting those outside of it. Therefore, the potential of global gait features has not been fully explored. In this paper, we propose a gait recognition framework based on vision transformers, aiming to enhance the extraction of global gait features. We introduce an adaptive multi-frame global feature mapping (AMGM) method to address the challenge of inconsistent feature dimensions caused by variations in the number of gait frames when fusing global and local features. We evaluate our model on the latest datasets, and the experimental results demonstrate a significant breakthrough. Notably, our model achieves state-of-the-art recognition accuracy, particularly in scenarios where subjects are wearing coats. Additionally, our model achieves remarkable improvements in recognition accuracy through training with small sample sets. [ABSTRACT FROM AUTHOR]

Seam carving is a method of resizing images based on content awareness. It can realize image retargeting while retaining the main content of the image. However, it may also be maliciously used to tamper with images, such as changing image semantic content by object removal. Therefore, seam carving detection has become important in image forensics. In this paper, a noise residual-based deep learning method is proposed to detect seam carving images. We try to learn the local noise in-consistency of images to recognize them. Firstly, a noise residual extraction segment is used to learn local noise features in the image, and a noise augmentation module is designed to enrich the features, which leverages the noise features extracted from a steganalysis rich model filter to discover the noise in-consistency between authentic and tampered regions. Then, through the feature dimensionality reduction section, the features are further learned and the size of feature maps are reduced. Finally, the output is obtained through global average pooling and a fully-connected layer. A careful testing strategy is further proposed, which greatly improves the detection performance, especially for seam carving with small scaling ratios. The experimental results demonstrate that our method achieves state-of-the-art performance at various scales, and has good robustness and generalization compared with other methods. [ABSTRACT FROM AUTHOR]

Convolutional neural networks have achieved remarkable success in single image dehazing tasks, and previous studies verified the dehazing performance of the U-shaped framework. However, most existing U-shaped architecture dehazing networks still face challenges in sufficiently dealing with a large area of haze with low visibility. In this paper, we propose a novel dehazing network named Double U-Net(DU-Net). Specifically, to reduce the interference of haze features in the encoder to the recovery stage when skip-connecting to the decoder directly, we develop a new architecture firstly, which is composed of an extended encoder–decoder. Besides, the hierarchical depth-wise convolution block(HDCB) is designed to gradually increase the receptive field by leveraging the depth-wise convolution, enriching the global information. Moreover, we propose a multi-branch interactive fusion(MIF) which achieves efficient cross-branch and cross-channel interaction through parallel multiple 1D convolutions. Extensive experiments on both synthetic and real-world hazy images demonstrate the effectiveness of our proposed method. • We construct a Double U-Net for image dehazing, by extending the encoder–decoder. • We build hierarchical convolution blocks to obtain a multi-scale receptive field. • A branch fusion is proposed to enhance cross-branch and cross-channel interaction. [ABSTRACT FROM AUTHOR]

Due to the absorption and scattering of light on the water surface, underwater images often face challenges such as low contrast, color deviation, insufficient exposure, and blurred details, exacerbating the difficulty of underwater tasks. In recent years, underwater image enhancement has become increasingly crucial in marine applications. Among existing underwater enhancement methods, the focus has often been on pixel-level learning, which may lead to image noise and an inability to finely adjust the image. In this paper, we propose a dual-branch Transformer-CNN Parameter Filtering network for underwater image enhancement, referred to as DTCPF. Specifically, to better aggregate window information, we introduce an overlapping window self-attention module to enhance interaction between windows. Additionally, we employ an improved Transformer encoder and decoder, utilizing long-distance attention and reversible neural networks to extract low-frequency and high-frequency information from the image. Moreover, we introduce a regression parameter filtering group for regression prediction, using the predicted parameters to enhance the image and obtain a reliable underwater enhancement model. Our approach undergoes qualitative and quantitative analyses on four real underwater datasets, demonstrating outstanding performance. [ABSTRACT FROM AUTHOR]

Recent self-supervised learning methods, where instance discrimination task is a fundamental way of pretraining convolutional neural networks (CNN), excel in transfer learning performance. Even though instance discrimination task is a well suited pretraining method for classification with its image-level learning, lack of dense representation learning makes it sub-optimal for localization tasks such as object detection. In this paper, we aim to mitigate this shortcoming of instance discrimination task by extending it to jointly learn dense representations alongside image-level representations. We add a segmentation branch parallel to the image-level learning to predict class-agnostic masks, enhancing location-awareness of the representations. We show the effectiveness of our pretraining approach on localization tasks by transferring the learned representations to object detection and segmentation tasks, providing relative improvements by up to 1.7% AP on PASCAL VOC and 0.8% AP on COCO object detection, 0.8% AP on COCO instance segmentation and 3.6% mIoU on PASCAL VOC semantic segmentation respectively. • Convolutional Neural Networks lose localization info with instance discrimination task. • Augmenting instance discrimination with an extra dense task enhances localization. • Unsupervised foreground segmentation boosts localization in (non-)contrastive methods. • Improved localization in SSL enhances transfer learning in dense downstream tasks. [ABSTRACT FROM AUTHOR]

Video anomaly detection aims to identify a set of abnormal events in videos. Deep reconstruction and prediction-based models have been employed to detect anomalies. Deep reconstruction models sometimes recreate the abnormal events along with the normal ones. However, the prediction-based approaches have demonstrated encouraging results. This paper presents a video vision transformer (ViViT) based generative adversarial network (GAN), TransGANomaly, a novel approach for detecting anomalies. The proposed framework is a video frame predictor and trained only on normal video data adversarially. The generator of the GAN is a ViViT network that receives 3D input tokens from the video snippets. The generator aims to predict the future frame based on past sequences. After that, the predicted and original frames are given to the model's discriminator for binary classification. Extensive experiments have been performed on UCSD Pedestrian, CUHK Avenue, and ShanghiaTech datasets to validate the efficacy of the proposed method. [ABSTRACT FROM AUTHOR]

Deep learning methods utilizing Convolutional Neural Networks (CNNs) and Transformers have achieved remarkable success in volumetric medical image analysis. While successful, the symmetrical structure of numerous networks pays insufficient attention to the encoding phase, and the large amount of memory occupied by voxels leads to unnecessary redundancy in the network. In this paper, we present a novel approach to handle volumetric medical images by converting them into point cloud and introduce a new asymmetrical segmentation architecture. We propose a dual-path encoder that fully captures both dense and sparse representations of the input point cloud sampled from volumes. Moreover, the two obtained representations are subtracted at the skip connection as a complementary feature during the decoding stage. Experimental results on the Brain Tumor Segmentation (BraTS) and the Multi-sequence Cardiac MR Segmentation tasks demonstrate the great potential of our point-based method for volumetric medical image segmentation. [ABSTRACT FROM AUTHOR]

The images in foggy weather are often degraded which may affect the object detection process. Hence, in recent times, several schemes have been designed to eliminate the haze effect and enhance the performance of computer vision systems. Although these methods reduce the haze effect, they also produce over-saturation and over-degradation on most occasions. These problems occur due to inadequate utilization of the regional haze properties during the dehazing process. Therefore, a new dehazing model is proposed in this paper to address the issues faced by existing dehazing algorithms. The residual convolutional neural network (RCNN) with 14 layers is developed in the present work. In this model, features to indicate the extent of haze are extracted using the regional characteristics of a hazy image and are used as the fourth channel along with the three colour channels of a hazy image. The proposed RCNN is trained with 4-channel hazy images as input and their corresponding haze-free images as output. The results of our model are obtained with several images taken from three different datasets. Through the experimentation, quantitative and qualitative improvements are observed when compared to state-of-the-art dehazing methods. • The local-haze characteristics driven dehazing method is developed in this work. • The regional haze properties are extracted using pixel-wise local max-min difference and pixel intensities of the hazy image. • A 4-channel hazy image is developed by concatenating the regional haze properties map to the hazy image. • A 14-layered RCNN is developed with the 4-channel hazy image as input and the haze-free image as output. [ABSTRACT FROM AUTHOR]

In this paper, an eye-tracking dataset of movie viewing in the immersive environment is developed, which contains 256 movie clips with 2K QHD resolution and corresponding movie genre labels from IMDb (Internet Movie Database). The dataset provides the audio-visual clues for studying the human visual attention when watching movie using a VR headset, by recording the eye movements using integrated eye tracker. To provide benchmarks for a saliency prediction for movie viewing in the immersive environment, fifteen computational models are evaluated on the dataset, including a newly developed multi-stream audio-visual saliency prediction model based on deep neural networks, named as MSAV. Detailed quantitative and qualitative comparisons and analyses are also provided. The developed dataset and benchmarks could help to facilitate the studies of visual saliency prediction for movie viewing in the immersive environments. [ABSTRACT FROM AUTHOR]

Accurate segmentation of nasopharyngeal carcinoma (NPC) in magnetic resonance (MR) images is crucial for radiotherapy planning. However, vanilla 2D/3D deep convolutional networks fail to gain satisfying NPC segmentation results, since 3D methods suffer from inter-slice discontinuities and 2D methods lack inter-slice context learning. To address this problem, this paper proposes a 2.5D learning Selective Inter-slice Context Network (SICNet) for NPC segmentation. The proposed SICNet comprises a main encoder that extracts target features from the middle slice and an auxiliary encoder that learns inter-slice context from adjacent slices. The features from two encoders are fused by the Inter-slice Context Fusion (ICF) module and then sent to the decoder for predictions. Furthermore, a knowledge distillation module named Mask-Guided Self-knowledge Distillation (MGSD) is proposed to help the auxiliary encoder learn context selectively. Specifically, with the guidance of mask labels, MGSD tells the auxiliary encoder where to look and how to learn, thus achieving selective inter-slice context learning and boosting segmentation accuracy. The proposed SICNet achieves a DSC of 74.38 ± 11.99, an HD95 of 9.31 ± 2.68, and an ASSD of 1.46 ± 0.76 on an in-house NPC dataset, outperforming other state-of-the-art segmentation methods. [ABSTRACT FROM AUTHOR]

Domain generalizable person re-identification (DG ReID) aims to obtain a model that can be applied directly to unseen domains once trained on a set of source domains (datasets collected from different camera networks). Among current DG ReID methods, instance normalization is a promising solution to reduce the effect of domain bias, but it inevitably filters out some discriminative information. Besides, since most pioneering approaches cannot effectively address the loss of discriminative information, this paper proposes a Style Elimination and Information Restitution (SEIR) module and constructs a generalizable framework. Specifically, we utilize instance normalization to reinforce the generalization capability on unseen domains. Then, the instance-specific mean and variance are employed to construct a feature vector that includes discriminative and style information. Besides, an encoder–decoder structure is designed to restitute the discriminative information to ensure a high recognition rate. Finally, a dual optimization strategy is devised to update the proposed module by simulating a real train-test process to enhance the generalization robustness further and prevent SEIR from overfitting to the source domain. Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art methods by a large margin on the public DG ReID benchmarks. • A Style Elimination and Information Restitution module for handling domain bias. • An optimization strategy for improving the generalization ability. • We propose a framework for generalizable person re-identification. • Significant results were achieved for addressing the generalizable person re-identification. [ABSTRACT FROM AUTHOR]

Lossy compression introduces artifacts, and many conventional in-loop filters have been adopted in the AV1 standard to reduce these artifacts. Researchers have explored deep learning-based filters to remove artifacts in the compression loop. However, the high computational complexity of CNN-based filters remains a challenge. In this paper, a Texture- and Motion-Aware Perception (TMAP) in-loop filter is proposed to addresses this issue by selectively applying CNNs to texture-rich and high-motion regions, while utilizing non-learning methods to detect these regions. The proposed method introduces a new CNN structure, the Dense-Dual-Field Network (DDFN), which leverages a larger receptive field to enhance the quality of reconstructed frames by incorporating more contextual information. Furthermore, to improve perceptual quality, a novel loss function integrating wavelet-based perceptual information is presented. Experimental results demonstrate the superiority of our proposed models over other lightweight CNN models, and the effectiveness of the perceptual loss function is validated using the VMAF metric. [ABSTRACT FROM AUTHOR]

In the last decade, the exploration of deep-sea ecosystems has surged, offering exciting prospects for discovering untapped resources such as medical drugs, food and energy sources, and renewable energy products. Consequently, research in underwater image processing has witnessed substantial growth. However, underwater imaging poses significant challenges, particularly without sophisticated, specialized cameras. Traditional cameras are impacted by absorption and scattering in the aquatic environment, producing hazy images with a blue–green tint. This phenomenon holds implications for marine research and other disciplines that rely on underwater imaging. While hardware advancements have been made over the years, image processing remains a valuable, cost-effective, and practical approach for underwater enhancement. Despite the existence of state-of-the-art techniques for underwater enhancement and restoration, their performance is often inconsistent. While some methods excel in contrast restoration, color restoration remains a pervasive challenge. In this paper, we introduce Sea-Pix-GAN, a Generative Adversarial Network (GAN)-based model that addresses these issues in underwater image enhancement. We redefine the problem as an image-to-image translation task and tailor the objective and loss functions to achieve color, content, and style transfer. The model is trained on a large dataset of underwater scenes, encompassing the diverse color dynamics of underwater subjects. Sea-Pix-GAN demonstrates promising results in restoring color, contrast, texture, and saturation. To validate its effectiveness, we compare the performance of Sea-Pix-GAN quantitatively based on metrics like PSNR, SSIM, and UIQM and qualitatively against several existing techniques. [ABSTRACT FROM AUTHOR]

Recent improvements in dynamic hand gesture recognition have seen a shift from traditional convolutional architectures to attention-based networks. These attention networks have been proven to outclass CNN + LSTM architectures, showing higher accuracy as well as reduced model parameters. Especially, skeleton-based attention networks have been shown to outperform visual-based networks due to the rich information from skeleton-based hand features. However, there is an opportunity to introduce complementary features from other modalities like RGB, depth, and optical flow images to enhance the recognition capability of skeleton-based networks. This paper aims to explore the addition of a multimodal fusion network to a skeleton-based Hierarchical Self-Attention Network (MF-HAN) and test for increased model effectiveness. Unlike traditional fusion techniques, this fusion network uses features derived from other sources of multimodal data in a reduced feature space using a cross-attention layer. The model outperforms its root model and other state-of-the-art models on the SHREC'17 track dataset, especially in the 28 gestures setting by more than 1 % in gesture classification accuracy. The experimentation was tested on the DHG dataset as well. [ABSTRACT FROM AUTHOR]

In social networks, people usually communicate online by posting and passing images, people will find that there are many similar or even visually identical images. When the images determine whether the published information is true or false, original images became crucial, and at the same time, deep learning excelled in image identification. Combining the original image identification and deep learning methods, this paper proposes the original image identification method based on the deep network models, aiming to use the deep network models to identify the original images between the original images and a bunch of variant images of the original images, and obtain satisfactory results. In this way, through the trained network models, the original images can be identified and can help the viewers to determine the correctness of the information. Experimental results verify the feasibility, effectiveness and security of the proposed method. • The method can prevent the images from being stolen during transmission. • The method can achieve good identification results on the image databases. • The method is feasible and flexible in actual operation. [ABSTRACT FROM AUTHOR]

• An image fusion method is proposed that can effectively highlight infrared target features and retain visible visual information. • Infrared targets are analyzed based on the saliency weight map. • The target contour and background texture are refined by a method based on weight refinement of guided filtering. • A multi-layer fusion strategy is used to fuse saliency weight maps at multiple scales. Image fusion is the fusion of multiple images from the same scene to produce a more informative image, and infrared and visible image fusion is an important branch of image fusion. To tackle the issues of diminished luminosity in the infrared target, inconspicuous target features, and blurred texture of the fused image after the fusion of infrared and visible images. This paper introduces a novel effective fusion framework that merges multi-scale Convolutional Neural Networks (CNN) with saliency weight maps. First, the method measures the source image features to estimate the initial saliency weight map. Then, the initial weight map is segmented and optimized using a guided filter before being further processed by CNN. Next, a trained Siamese convolutional network is used to solve the two key problems of activity measure and weight assignment. Meanwhile, a multi-layer fusion strategy is designed to effectively retain the luminance of the infrared target and the texture information in the visible background. Finally, adaptive adjustment of the fusion coefficients is achieved by employing saliency. The experimental results show that the method outperforms the state-of-the-art algorithms in terms of both subjective visual quality and objective evaluation effects. [ABSTRACT FROM AUTHOR]

• A novel attention module is introduced to capture important action features in a video both at pix level and channel level. • Another novel attention module is proposed to highlight discriminative convolutional features at channel level. • Based on the above two attention modules, a new network architecture is proposed for action classification in videos. Human action recognition in videos is a fundamental and important topic in computer vision, and modeling spatial–temporal dynamics in a video is crucial for action classification. In this paper, a novel attention module named Channel-wise Non-local Attention Module (CNAM) is proposed to highlight the important features both spatially and temporally. Besides, another new attention module named Channel-wise Attention Recalibration Module (CARM) is developed to focus on capturing discriminative features at channel level. Based on these two attention modules, a novel convolutional neural network named Residual Attention Fusion Network (RAFN) is proposed to model long-range temporal structure and capture more discriminative action features at the same time. More specifically, first, a sparse temporal sampling strategy is adopted to uniformly sample video data as input to RAFN along the temporal dimension. Secondly, the attention modules CNAM and CARM are plugged into residual network for highlighting important action regions around actors. Finally, the classification scores of four streams of RAFN are combined by late fusion. The experimental results on HMDB51 and UCF101 demonstrate the effectiveness and excellent recognition performance of our proposed method. [ABSTRACT FROM AUTHOR]

Previous deep learning-based single image dehazing methods are implemented by using real-valued convolutional neural network (RV-CNN), which ignore the phase information of the image, and tend to cause models to perform well on synthetic datasets but low generalization on real-world datasets. In this paper, we propose a novel Complex-Valued convolutional Dehazing Network (CVD-Net), which considers both amplitude and phase information of image for haze removal in real-world scenes. Specifically, we construct complex-valued transformation module, including complex-valued convolutional layer and residual block embedded into the middle of the network to extract features of image efficiently, and also design a complex-valued selected fusion module (CVSF) and complex-valued attention module (CVAM) to promote the interaction between different scale features for preserving image detailed information. Both qualitative and quantitative experimental results show that the proposed CVD-Net can effectively remove the haze, and has good generalization in real-world hazy images. [ABSTRACT FROM AUTHOR]

To overcome challenges in multiple-object tracking (MOT) tasks, recent algorithms use interaction cues alongside motion and appearance features. These algorithms use graph neural networks or transformers to extract interaction features that lead to high computation costs. In this paper, a novel interaction cue based on geometric features is presented aiming to detect occlusion and reidentify lost targets with low computational costs. Moreover, in the majority of algorithms, camera motion is considered negligible, which is a strong assumption that is not always true and can lead to identity (ID) switching or mismatching of targets. In this paper, a method for measuring camera motion is presented that efficiently reduces its effect on tracking. The proposed algorithm is evaluated on MOT17 and MOT20 datasets and achieves state-of-the-art performance on MOT17 with comparable results on MOT20. The code is also publicly available. 1 1 https://github.com/mhnasseri/for_tracking. • Cascade association based on the detection score. • A new similarity metric that considers detection's size. • A novel interaction model for occlusion handling and target re-identification. • A novel algorithm for estimating camera motion and removing it. • State-of-the-art results on MOT17 and comparable results on MOT20. [ABSTRACT FROM AUTHOR]

This paper proposes a novel robust duplicated-diffusion watermarking model based on deep network (RDD-net). RDD-net employs the encoder-noise-decoder structure for end-to-end training to obtain high image watermarking invisibility and robustness. Firstly, a duplicated-diffusion strategy is employed in the encoder to replicate the original watermark iteratively until all copies are diffused to the whole image so that the generalized ability in resisting various noises is obtained. Then, a channel connection technique is designed to extract inherent image features for fusing watermark for robustness by mining correlations of RGB three channels of the color image. Meanwhile, each channel is fused with watermark to increase robustness. Another attempt to improve the watermarking performance is the optimizer, which optimizes the watermark distribution by evaluating the similarities between the encoded image and the original image, as well as between the encoded image and the noised image. Our extensive experimental results demonstrate that the proposed RDD-net not only resists different noises, but also obtains better image quality and higher robustness than the existing watermarking models. [ABSTRACT FROM AUTHOR]

Person Re-Identification (PR-Id) encounters misclassification issues when re-identifying persons with different backpacks. These bags manifest as large and distinct regions in the images surpassing other finer details of a person. As a result, a CNN model swiftly detects and prioritizes these image regions for re-identification. However, the bags are subject to alterations or may be similar among multiple persons, resulting in misclassification. To ensure that a CNN model does not consider bags as unique features of specific persons and prioritize them for re-identification, images of persons with diverse backpacks are crucial in the training dataset. Moreover, these images enhance the model's focus on other unique regions of a person. Although the current datasets show potential, incorporating such images could enhance their effectiveness. Therefore, in this paper, we propose an indoor PR-Id dataset named "With Bag/Without Bag-ReID" (WB/WoB-ReID). The set "with_bag" in WB/WoB-ReID dataset includes identities with different backpacks for the first time. We also incorporate identities without bags and with varying numbers of image counts in three other sets, namely "without_bag", "both_small," and "both_large". We assess WB/WoB-ReID and three other PR-Id datasets: Market1501, CUHK03, and DukeMTMC-reID on various existing approaches. The highest mAP achieved on the"with_bag" is 74%, "without_bag" is 96.7% and other datasets are 97.78%, 95.20% and 92.4%. The results show that incorporating identities with diverse bags reduces the mAP, highlighting the misclassifications that arise specifically in the presence of bags. [ABSTRACT FROM AUTHOR]

Despite the remarkable results achieved by deep convolutional neural networks (CNNs) in single image super-resolution (SISR), their computational cost increase exponentially as CNN models get deeper and wider. Consequently, depthwise separable convolutions (DSConvs) have emerged as the fundamental building basis for various contemporary lightweight network architectures. However, depthwise convolution is sub-optimal for restoring missing high-frequency details. In this paper, we commence with the vanilla depthwise convolution and progressively develop it to alleviate the above problem. Specifically, we introduce an effective method, kernel attention injection, to integrate intra-kernel correlation into the depthwise convolution layer by incorporating the learned kernel attention into the depthwise kernel. We find that the determinant is capable of capturing the correlation between diagonal elements of the depthwise kernel by summing the products of the diagonals. Therefore, we utilize the determinant as the kernel pooling method. Furthermore, we propose to remove the non-linear activation function behind the depthwise convolution (i.e. , linear depthwise convolution) to preserve informative features for reconstructing faithful high-resolution (HR) images. Built on this recipe, we introduce kernel-attentive linear depthwise separable convolutions (KDSConvs), which exhibits promising super-resolution performance. Our experimental results underline the potential of depthwise convolution in super-resolution tasks by demonstrating that the proposed KDSConvs significantly outperforms DSConvs in terms of both quantitative measurements and visual quality. The code will be released at https://github.com/AwesomeHwang/KALDN. • We propose kernel attention injection to integrate intra-kernel correlation. • We utilize the determinant as the kernel pooling method. • We introduce kernel-attentive linear depthwise separable convolutions. [ABSTRACT FROM AUTHOR]

Feature pyramid network (FPN) is a typical detector commonly for solving the issue of object detection at different scales. However, the lateral connections in FPN lead to the loss of feature information due to the reduction of feature channels. Moreover, the top-down feature fusion will weaken the feature representation in the process of feature delivery because of features with different semantic information. In this paper, we propose a feature pyramid network with channel and content adaptive feature enhancement module (CCA-FPN), which uses a channel adaptive guided mechanism module (CAGM) and multi-scale content adaptive feature enhancement module (MCAFEM) to alleviate these problems. We conduct comprehensive experiments on the MS COCO dataset. By replacing FPN with CCA-FPN in ATSS, our models achieve 1.3 percentage points higher Average Precision (AP) when using ResNet50 as backbone. Furthermore, our CCA-FPN achieves 0.3 percentage points higher than the AugFPN which is the state-of-the-art FPN-based detector. • Targets of different scales often use feature pyramid networks for hierarchical detection. • Using deep learning for object detection, feature enhancement is beneficial for improving detection performance. • When performing feature fusion on the feature layer, it is necessary to reduce the channel dimension. • The reduction of the channel dimension can use the attention mechanism to mine the correlation between channels for compression. • In the learning process of neural networks, the learning target features are adapted using different weight sizes. [ABSTRACT FROM AUTHOR]

Over the past few years, skeleton-based action recognition has attracted great success because the skeleton data is immune to illumination variation, view-point variation, background clutter, scaling, and camera motion. However, effective modeling of the latent information of skeleton data is still a challenging problem. Therefore, in this paper, we propose a novel idea of action embedding with a self-attention Transformer network for skeleton-based action recognition. Our proposed technology mainly comprises of two modules as, (i) action embedding and (ii) self-attention Transformer. The action embedding encodes the relationship between corresponding body joints (e.g., joints of both hands move together for performing clapping action) and thus captures the spatial features of joints. Meanwhile, temporal features and dependencies of body joints are modeled using Transformer architecture. Our method works in a single-stream (end-to-end) fashion, where multiple-layer perceptron (MLP) is used for classification. We carry out an ablation study and evaluate the performance of our model on a small-scale SYSU-3D dataset and large-scale NTU-RGB+D and NTU-RGB+D 120 datasets where the results establish that our method performs better than other state-of-the-art architectures. • Action recognition has several computer applications such as surveillance, content analysis, healthcare, etc. • We have proposed a new action embedding scheme using a graph convolution and link prediction method that can help exploit the relation between distant joints. • We tune the embeddings using a self-attention transformer to preserve longer spatial–temporal dependencies across frames for final multiclass classification. • We have evaluated our method of different public datasets, including SYSU-3D, NTU RGB+D, and NTU RGB+D 120, where it surpasses state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In the field of face anti-spoofing (FAS), how to extract the representative features to distinguish between real and spoof faces and train the corresponding deep networks are two vital issues. In this paper, we propose a simple but effective end-to-end FAS model based on an innovative texture extractor and a depth auxiliary supervision mechanism. In the feature extraction stage, we first design the residual gradient convolutions based on the redesigned gradient operators, which are used to extract fine-grained texture features. The extraction of texture features is based on multiple scales by dividing the texture differences between living and spoofing faces into three levels reasonably. Then we construct a multiscale residual gradient attention (MRGA) to obtain representative texture features from multiple levels texture features. By combining the proposed feature extractor MRGA and existing vision transformer (ViT), the MRGA-ViT is proposed to generate related semantics and obtain final classification results. In the training stage, we also propose a local depth auxiliary supervision based on a novel adjacent depth loss, which utilizes the correlation information of adjacent pixels adequately compared with traditional depth loss. The proposed MRGA-ViT model achieves competitive performance in generalization and stability ability, e.g., the ACER(%) values of intra testing on OULU-NPU database are 1.8, 2.6, 1.6 ± 1.2 and 1.9 ± 2.7 respectively, the AUC(%) of cross type testing attains 99.45 ± 0.57, the ACER(%) values of cross dataset testing are 28.1 and 36.7 respectively. Experimental results prove that the proposed model is competitive to other state-of-the-art works on generalization and stability performance. • The multi-scale feature extraction based on the multiple levels texture differences. • The gap elimination of representation between local textures and global semantics. • The adjacent depth loss and local depth supervision are effective and generalizable. • The FAS method is stable and generalizable to detect the spoofing face. [ABSTRACT FROM AUTHOR]

Although the deep CNN-based super-resolution methods have achieved outstanding performance, their memory cost and computational complexity severely limit their practical employment. Knowledge distillation (KD), which can efficiently transfer knowledge from a cumbersome network (teacher) to a compact network (student), has demonstrated its advantages in some computer vision applications. The representation of knowledge is vital for knowledge transferring and student learning, which is generally defined in hand-crafted manners or uses the intermediate features directly. In this paper, we propose a model-agnostic meta knowledge distillation method under the teacher–student architecture for the single image super-resolution task. It provides a more flexible and accurate way to help teachers transmit knowledge in accordance with the abilities of students via knowledge representation networks (KRNets) with learnable parameters. Specifically, the texture-aware dynamic kernels are generated from local information to decompose the distillation problem into texture-wise supervision for further promoting the recovery quality of high-frequency details. In addition, the KRNets are optimized in a meta-learning manner to ensure the knowledge transferring and the student learning are beneficial to improving the reconstructed quality of the student. Experiments conducted on various single image super-resolution datasets demonstrate that our proposed method outperforms existing defined knowledge representation-related distillation methods and can help super-resolution algorithms achieve better reconstruction quality without introducing any extra inference complexity. [ABSTRACT FROM AUTHOR]

Although convolutional neural network-based methods have achieved significant performance improvement for Single Image Super-Resolution (SISR), their vast computational cost hinders real-world environment application. Thus, the interest in light networks for SISR is rising. Since existing SISR light models mainly focus on extracting fine local features using convolution operation, they have a limitation in that networks hardly capture global information. To capture the long-range dependency, Non-Local (NL) attention and Transformers have been explored in the SISR task. However, they are still suffering from a balancing problem between performance and computational cost. In this paper, we propose Fast Non-Local attention NETwork (FNLNET) for a super light SISR, which can capture the global representation. To acquire global information, we propose The Fast Non-Local Attention (FNLA) module that has low computational complexity while capturing global representation that reflects long-distance relationships between patches. Then, FNLA requires only 16 times lower computational cost than conventional NL networks while improving performance. In addition, we propose a powerful module called Global Self-Intension Mining (GSIM) that fuses the multi-information resources such as local, and global representation. Our FNLNET shows outstanding performance with fewer parameters and computational costs in the experiments on the benchmark datasets against state-of-the-art light SISR models. [Display omitted] • We propose a novel approach to Non-Local attention in SISR. • We propose a powerful module for SISR that designs multi-branch architecture. • The proposed method demonstrates notable improvements, despite having fewer parameters and MAC. [ABSTRACT FROM AUTHOR]

In contrast to Convolutional Neural Networks (CNNs), Vision Transformers (ViT) cannot capture sequence ordering of input tokens and require position embeddings. As a learnable fixed-dimension vector, the position embedding improves accuracy while limiting the migration of the model between different input sizes. Hence, this paper conducts an empirical study on position embeddings of pre-trained models, which mainly focuses on two questions: (1) What do the position embeddings learn from training? (2) How do the position embeddings affect the self-attention modules? This paper analyzes the pattern of position embedding in pre-trained models and finds that the linear combination of Gabor filters and edge markers can fit the learned position embeddings well. The Gabor filters and edge markers can occupy some channels to append the position information, and the edge markers have flowed to values in self-attention modules. The experimental results can guide future work to choose suitable position embeddings. • Position Embeddings are the linear combination of Gabor Filters and Edge Markers. • The main part of the positional embeddings is input-independent. • Relative position embeddings requires adding edge markers to the value matrix. [ABSTRACT FROM AUTHOR]

Dynamic hand gesture recognition is still an interesting topic for the computer vision community. A set of feature vectors can represent any hand gesture. A Recurrent Neural Network (RNN) can recognize these feature vectors as a hand gesture that analyzes the temporal and contextual information of the gesture sequence. Thus, we proposed a hybrid deep learning framework to recognize dynamic hand gestures. In the Hybrid model GoogleNet is pipelined with a Bidirectional GRU unit to recognize the dynamic hand gesture. Dynamic hand gestures consist of many frames, and features of each frame need to be extracted to get the temporal and dynamic information of the performed gesture. As RNN takes input as a sequence of feature vectors, we extract features from videos using pretrained GoogleNet. As Gated Recurrent Unit is one of the variants of RNN to classify the sequential data, we created a feature vector that corresponds to each video and passed it to the bidirectional GRU (BGRU) network to classify the gestures. We evaluate our model on four publicly available hand gesture datasets. The proposed method performs well and is comparable with the existing methods. For instance, we achieved 98.6% accuracy on Northwestern University Hand Gesture(NWUHG), 99.6% on SKIG, 99.4% on Cambridge Hand Gesture (CHG) datasets respectively. We performed our experiments on DHG14/28 dataset and achieved an accuracy of 97.8% with 14-gesture classes and 92.1% on 28-gesture classes. DHG14/28 dataset contains skeleton and depth data, and our proposed model used depth data and achieved comparable accuracy. • The paper presented a two-stream hybrid model that uses a convolutional neural network to extract the frame-level features and used a BGRU to classify the dynamic hand gesture. • A deeply coupled hybrid pipeline is used to extract spatial and temporal features. • The paper presented a comprehensive review of the BLSTM and BGRU model with different batch sizes. • Experimental results shows that the BGRU model with optical flow and RGB videos performs better than state-of-the-art. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have dominated the field of computer vision for nearly a decade. However, due to their limited receptive field, CNNs fail to model the global context. On the other hand, transformers, an attention-based architecture, can model the global context easily. Despite this, there are limited studies that investigate the effectiveness of transformers in crowd counting. In addition, the majority of the existing crowd-counting methods are based on the regression of density maps which requires point-level annotation of each person present in the scene. This annotation task is laborious and also error-prone. This has led to an increased focus on weakly-supervised crowd-counting methods, which require only count-level annotations. In this paper, we propose a weakly-supervised method for crowd counting using a pyramid vision transformer. We have conducted extensive evaluations to validate the effectiveness of the proposed method. Our method achieves state-of-the-art performance. More importantly, it shows remarkable generalizability. • A pyramid vision transformer-based weakly-supervised crowd counting method. • An MLP based multiscale feature aggregation and regression module. • New benchmarks for cross-dataset performance in crowd counting. • A novel end-to-end attention-based crowd-counting method employing smooth- L 1 loss. [ABSTRACT FROM AUTHOR]

With the continuous development of deep learning, neural networks have made great progress in license plate recognition (LPR). Nevertheless, there is still room to improve the performance of license plate recognition for low-resolution and relatively blurry images in remote surveillance scenarios. When it is difficult to enhance the recognition algorithm, we choose super-resolution (SR) to improve the quality of license plate images and thereby provide clearer input for the subsequent recognition stage. In this paper, we propose an automatic super-resolution license plate recognition (SRLPR) network which consists of four parts separately: license plate detection, character detection, single character super-resolution, and recognition. In the training stage, firstly, LP detection model needs to be trained alone and then its detection results will be used to successively train the three subsequent modules. During the test phase, for each input image, the network can get its LP number automatically. We also collect an applicable and challenging LPR dataset called SRLP, which is collected from real remote traffic surveillance. The experimental results demonstrate that our method achieves comprehensive quality of SR images and higher recognition accuracy compared with state-of-the-art methods. The SRLP dataset and the code for training and testing SRLPR network are available at https://pan.baidu.com/s/1vnhRa-c-dBj6jlfBZV5w4g. • An automatic Super-Resolution License Plate Recognition (SRLPR) method is proposed. • DBAGAN is proposed for single character super-resolution in the SRLPR framework. • A challenging SRLP dataset is collected for LPR in remote surveillance. • Experimental results demonstrate the effectiveness of our SRLPR. [ABSTRACT FROM AUTHOR]

Neural network pruning techniques can be effective in accelerating neural network models, making it possible to deploy them on edge devices. In this paper, we propose to prune neural networks using data variance. Unlike other existing methods, this method is somewhat robust and does not invalidate our criteria depending on the number of data batches and the number of training sessions. We also propose a pruning compensation technique. This technique fuses the pruned convolutional information into the remaining convolutional kernel close to it. This fusion operation can effectively help retain the pruned information. We evaluate the proposed method on a number of standard datasets and compare it with several current state-of-the-art methods. Our method always achieves better performance. For example, on Tiny ImageNet, our method can prune 54.2% FLOPs of ResNet50 while obtaining a 0.22% accuracy improvement. • A data-driven network pruning method with high robustness is proposed. • The proposed data-driven pruning method consumes less resources and is highly efficient. • A pruning compensation technique is proposed to reduce the negative impact of pruning. • The effectiveness of the proposed method is validated on several datasets. [ABSTRACT FROM AUTHOR]

The outstanding performance of convolutional neural networks (CNNs) shown in single image super-resolution (SISR) strongly depends on network's depth, which hampers its application in low-power computing devices. In this paper, a lightweight and efficient network (LESR) is proposed for SISR by constructing the shallow feature extraction block (SFBlock), the cascaded sparse mask blocks (SMBlocks) and the feature fusion block (FFBlock). The SFBlock efficiently extracts global informative features from the original low resolution image using sparse self-attention, SMBlock skips the redundant computation in extracted features, and more meaningful information is distilled for the sequential reconstruction block by the FFBlock. In addition, a recently proposed activation function called ACON-C is used to replace the ReLU function to ease the training difficulty. Extensive experiments show that our proposed network performs better than most advanced lightweight SISR algorithms with comparable parameters and less FLOPs on benchmark database for × 2 / 3 / 4 SISR. • A lightweight and efficient network for single image super-resolution is proposed. • It adopts sparse self-attention to extract global information from shallow layers. • It is spare mask blocks skips the redundant computation. • A new active function ACON-C is adopted to enhance nonlinear mapping ability. [ABSTRACT FROM AUTHOR]

• Rethinking the PASCAL-VOC and MS-COCO dataset for small object detection. • There are different labeling errors in these two benchmarks by visual analysis. • Building several specific datasets to analyze small object detection. • Data quality is another factor that affects the performance of small object detection. The data and the algorithm are critical to deep learning-based small object detectors. In this paper, we rethink the PASCAL-VOC and MS-COCO dataset for small object detection. By visual analysis of the original annotations, we find that there are different labeling errors in these two datasets. To solve these problems, we build specific datasets, including SDOD, Mini6K, Mini2022 and Mini6KClean. The experimental results of several typical algorithms (e.g. SSD, YOLOv5, Faster RCNN and Deformable DETR) on the datasets show that data labeling errors (such as missing labels, category label errors, inappropriate labels) are another factor that affects the detection performance of small objects. [ABSTRACT FROM AUTHOR]

Weakly supervised semantic segmentation (WSSS) with only image-level supervision is a challenging task. Most existing methods exploit Class Activation Maps (CAM) to generate pixel-level pseudo labels for supervised training. However, due to the local receptive field of Convolution Neural Networks (CNN), CAM applied to CNNs often suffers from partial activation — highlighting the most discriminative part instead of the entire object area. In order to capture both local features and global representations, the Conformer has been proposed to combine a visual transformer branch with a CNN branch. In this paper, we propose TransCAM, a Conformer-based solution to WSSS that explicitly leverages the attention weights from the transformer branch of the Conformer to refine the CAM generated from the CNN branch. TransCAM is motivated by our observation that attention weights from shallow transformer blocks are able to capture low-level spatial feature similarities while attention weights from deep transformer blocks capture high-level semantic context. Despite its simplicity, TransCAM achieves competitive performance of 69.3% and 69.6% on the respective PASCAL VOC 2012 validation and test sets, showing the effectiveness of transformer attention-based refinement of CAM for WSSS. • We propose a transformer-based solution for Weakly Supervised Semantic Segmentation. • We utilize the attention weights from the transformer to refine the CAM. • We find different blocks' attention weights capture distinct feature affinities. • Our method is simple yet effective, showing competitive results on PASCAL VOC 2012. [ABSTRACT FROM AUTHOR]

To solve the problem of low sign language recognition rate under the condition of small samples, a simple and effective static gesture recognition method based on an attention mechanism is proposed. The method proposed in this paper can enhance the features of both the details and the subject of the gesture image. The input of the proposed method depends on the intermediate feature map generated by the original network. Also, the proposed convolutional model is a lightweight general module, which can be seamlessly integrated into any CNN(Convolutional Neural Network) architecture and achieve significant performance gains with minimal overhead. Experiments on two different datasets show that the proposed method is effective and can improve the accuracy of sign language recognition of the benchmark model, making its performance better than the existing methods. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach for low-light image enhancement. We propose a deep simultaneous estimation network (DSE-Net), which simultaneously estimates the reflectance and illumination for low-light image enhancement. The proposed network contains three modules: image decomposition, illumination adjustment, and image refinement module. The DSE-Net uses a novel branched encoder–decoder based image decomposition module for simultaneous estimation. The proposed decomposition module uses a separate decoder to estimate illumination and reflectance. DSE-Net improves the estimated illumination using the illumination adjustment module and feeds it to the proposed refinement module. The image refinement module aims to produce sharp and natural-looking output. Extensive experiments conducted on a range of low-light images demonstrate the efficacy of the proposed model and show its supremacy over various state-of-the-art alternatives. [ABSTRACT FROM AUTHOR]

• AMSFF-Net has ability to pay more attention to the informative features at different multi-stream resolution levels using pixel attention mechanism. • AMSFF-Net has ability to capture semantic information and sharp textural details from the extracted features and retain high-quality image from coarse-to-fine using mixed-convolution attention mechanism. • A sharp image can be recovered by the good kernel estimation. Mixed convolution attention mechanism highlights the task-relevant features and retain the dehazed images with sharp texture details and more vibrant colors. • Deep semantic loss highlights the boundary information and the spatial correlation in the deep features. In this paper, an end-to-end convolutional neural network is proposed to recover haze-free image named as Attention-Based Multi-Stream Feature Fusion Network (AMSFF-Net). The encoder-decoder network structure is used to construct the network. An encoder generates features at three resolution levels. The multi-stream features are extracted using residual dense blocks and fused by feature fusion blocks. AMSFF-Net has ability to pay more attention to informative features at different resolution levels using pixel attention mechanism. A sharp image can be recovered by the good kernel estimation. Further, AMSFF-Net has ability to capture semantic and sharp textural details from the extracted features and retain high-quality image from coarse-to-fine using mixed-convolution attention mechanism at decoder. The skip connections decrease the loss of image details from the larger receptive fields. Moreover, deep semantic loss function emphasizes more semantic information in deep features. Experimental findings prove that the proposed method outperforms in synthetic and real-world images. [ABSTRACT FROM AUTHOR]

In recent years, the research method of depth estimation of target images using Convolutional Neural Networks (CNN) has been widely recognized in the fields of artificial intelligence, scene understanding and three-dimensional (3D) reconstruction. The fusion of semantic segmentation information and depth estimation will further improve the quality of acquired depth images. However, how to deeply combine image semantic information with image depth information and use image edge information more accurately to improve the accuracy of depth image is still an urgent problem to be solved. For this purpose, we propose a novel depth estimation model based on semantic segmentation to estimate the depth of monocular images in this paper. Firstly, a shared parameter model of semantic segmentation information and depth estimation information is built, and the semantic segmentation information is used to guide depth acquisition in an auxiliary way. Then, through the multi-scale feature fusion module, the feature information contained in the neural network on different layers is fused, and the local feature information and global feature information are effectively used to generate high-resolution feature maps, so as to achieve the goal of improving the quality of depth image by optimizing the semantic segmentation model. The experimental results show that the model can fully extract and combine the image feature information, which improves the quality of monocular depth vision estimation. Compared with other advanced models, our model has certain advantages. [ABSTRACT FROM AUTHOR]

In the action recognition, a proper frame sampling method can not only reduce redundant video information, but also improve the accuracy of action recognition. In this paper, an action density based non-isometric frame sampling method, namely NFS, is proposed to discard the redundant video information and sample the rational frames in videos for neural networks to achieve great accuracy on human action recognition, in which action density is introduced in our method to indicate the intensity of actions in videos. Particularly, the action density determination mechanism, focused-clips division mechanism, and reinforcement learning based frame sampling (RLFS) mechanism are proposed in NFS method. Via the evaluations with various neural networks and datasets, our results show that the proposed NFS method can achieve great effectiveness in frame sampling and can assist in achieving better accuracy on action recognition in comparison with existing methods. [ABSTRACT FROM AUTHOR]

Image denoising requires both spatial details and global contextualized information to recover a clean version from the deteriorative one. Previous deep convolution networks usually focus on modeling the local feature and stacked convolution blocks to expand the receptive field, which can catch the long-distance dependencies. However, contrary to the expectation, the extracted local feature incapacity recovers the global details by traditional convolution while the stacked blocks hinder the information flow. To tackle these issues, we introduce the M atrix Factorization D enoising Module (MD) to model the interrelationship between the global context aggregating process and the reconstructed process to attain the context details. Besides, we redesign a new basic block to ease the information flow and maintain the network performance. In addition, we conceive the F eature Fu sion Module (FFU) to fuse the information from the different sources. Inspired by the multi-stage progressive restoration architecture, we adopt two-stage convolution branches progressively reconstructing the denoised image. In this paper, we propose an original and efficient neural convolution network dubbed MFU. Experimental results on various image denoising datasets: SIDD, DND, and synthetic Gaussian noise datasets show that our MFU can produce comparable visual quality and accuracy results with state-of-the-art methods. • We propose MFU, an effective and efficient end-to-end multi-stage progressive convolution network with a new basic block for image denoising. • We represent the global structured information as the low-rank matrix and use the MD module to learn the contextually-enriched and globally structured feature. • Designing a module (FFU) fuses the multi-source information to promote the network reconstruct the image. [ABSTRACT FROM AUTHOR]

Video compression is essential for uploading videos to online platforms which usually have bandwidth limitations. However, the compression reduces the visual quality. To overcome this problem, the visual quality of the low bitrate compressed videos for various standards, including H.264 and HEVC in decoders, needs to be improved. Accordingly, this paper proposes a novel method for improving video quality based on 3D convolutional neural networks (CNNs). This method is totally compatible with the encoders of video compression standards, i.e., H.264, VVC, and HEVC, and can be implemented easily. In particular, the proposed neural network model receives five frames of the low bitrate compressed video as input and subsequently predicts the compression error of frames using the first and fifth frames. Finally, it reconstructs an improved version of the frame with high quality. The CNN is an Additive (3D) model that can predict the eliminated inter-frame redundancies resulting from compression. Our goal is to increase the peak signal to noise ratio (PSNR) and structural index similarity (SSIM) of the luminance (Y) and chrominance (U, V) frames in the video. Additive 3D-CNN achieves an average of 12.4%, 9.9% and 5% BD-rate increases for LP, LB and RA for the Y component. The results indicate that the new proposed algorithm outperforms the previous methods in terms of PSNR, SSIM, and BD-rate. [ABSTRACT FROM AUTHOR]

Recently, Convolutional Neural Networks (CNNs) have achieved great success in Single Image Super-Resolution (SISR). In particular, the recursive networks are now widely used. However, existing recursion-based SISR networks can only make use of multi-scale features in a layer-wise manner. In this paper, a Deep Recursive Multi-Scale Feature Fusion Network (DRMSFFN) is proposed to address this issue. Specifically, we propose a Recursive Multi-Scale Feature Fusion Block (RMSFFB) to make full use of multi-scale features. Besides, a Progressive Feature Fusion (PFF) technique is proposed to take advantage of the hierarchical features from the RMSFFB in a global manner. At the reconstruction stage, we use a deconvolutional layer to upscale the feature maps to the desired size. Extensive experimental results on benchmark datasets demonstrate the superiority of the proposed DRMSFFN in comparison with the state-of-the-art methods in both quantitative and qualitative evaluations. [ABSTRACT FROM AUTHOR]

With the great success of deep neural networks, combining deep learning with traditional dictionary learning has become a hot issue. However, the performance of these methods is still limited for several reasons. First, some existing methods update dictionary learning and classifier as two independent modules, which limits the classification performance. Second, the non-attention dictionary is learned to represent all images, reducing the model representation flexibility. In this paper, we design a novel end-to-end model named Multi-layer Attention Dictionary Pair Learning Network (MADPL-net), which integrates the learning schemes of the convolutional neural network, deep encoder learning, and attention dictionary pair learning (ADicL) into a unified framework. The encoder layer contains the ADicL block, which selects more image-attentive atoms in the dictionary pair block via the softmax function to ensure MADPL-net classification capability. In addition, ADicL schema can yield discriminative dictionary atoms and feature maps with high inter-class separation and high intra-class compactness. To improve the sparse representation learning performance, MADPL-net adds l 1 − norm constraint of the analysis dictionary to the cross-entropy loss function. Extensive experiments show that MADPL-net can achieve excellent performance over other state-of-the-arts. • A powerful architecture, called the multi-layer attention dictionary pair learning network (MADPL-net), is proposed for image classification. • The MADPL-net integrates convolutional neural network, deep encoder learning, and attention dictionary pair learning into a unified framework. • The MADPL-net applies the attentional dictionary learning block to select more image-attentive atoms via the soft-max function. • The MADPL-net adds the l 1 − norm constraint of analysis dictionary to the crossentropy loss function. • The MADPL-net can ensure discriminative dictionary atoms and feature maps with high inter-class separation and high intra-class compactness. [ABSTRACT FROM AUTHOR]

• Channel attention model captures important channels for each feature map position. • Multi-scale features combined from multiple layers handle different person sizes. • Adaptive loss improves the accuracy of crowd counting for sparse crowd samples. Crowd counting algorithms have recently incorporated attention mechanisms into convolutional neural networks (CNNs) to achieve significant progress. The channel attention model (CAM), as a popular attention mechanism, calculates a set of probability weights to select important channel-wise feature responses. However, most CAMs roughly assign a weight to the entire channel-wise map, which makes useful and useless information being treat indiscriminately, thereby limiting the representational capacity of networks. In this paper, we propose a multi-scale and spatial position-based channel attention network (MS-SPCANet), which integrates spatial position-based channel attention models (SPCAMs) with multiple scales into a CNN. SPCAM assigns different channel attention weights to different positions of channel-wise maps to capture more informative features. Furthermore, an adaptive loss, which uses adaptive coefficients to combine density map loss and headcount loss, is constructed to improve network performance in sparse crowd scenes. Experimental results on four public datasets verify the superiority of the scheme. [ABSTRACT FROM AUTHOR]

As a framework of optical machine learning, all-optical diffractive neural network (D 2 NN) has delivered an ideal outcome of feature detection and target classification, currently raising high interest in the optics and photonics community. In this paper, we applied an improved D 2 NN architecture to the field of gesture recognition, which features more complicated contour than the common MNIST handwriting recognition in the previous literature. The proposed network structure incorporates the wavelet-like phase modulation pattern technique and the highway network on the basis of all-optical neural network. Through modulating the phase of incident light, the wavelet-like pattern can substantially reduce the parameters in the network layer. In addition, a highway network is employed to address the vanishing gradient phenomenon in the training process. In the experiment, we numerically achieved blind testing accuracy of 95.6% for identifying ten different gestures, and the number of parameters is only 3% of the regular D 2 NN. Reliability test and analysis show that the proposed method is a high-efficiency solution with low-parameters expecting for implementation of various machine learning tasks. • All-optical diffractive deep neural network (D2NN) for gesture recognition. • The wavelet-like phase modulation pattern for D2NN to reduce the parameters. • The highway network structure to address the vanishing gradient phenomenon. [ABSTRACT FROM AUTHOR]

Due to the copyright issues often involved in the recapture of LCD screen content, recaptured screen image identification has received lots of concerns in image source forensics. This paper analyzes the characteristics of convolutional neural network (CNN) and vision transformer (ViT) in extracting features and proposes a cascaded network structure that combines local-feature and global-feature extraction modules to detect the recaptured screen image from original images with or without demoiréing operation. We first extract the local features of the input images with five convolutional layers and feed the local features into the ViT to enhance the local perception capability of the ViT module, and further extract the global features of the input images. Through thorough experiments, our method achieves a detection accuracy rate of 0.9691 in our generated dataset and 0.9940 in the existing mixture dataset, both showing the best performance among the compared methods. [Display omitted] • First method to identify recaptured screen images with demoiréing operation. • CNN and ViT are set as the local-feature and global-feature extraction modules, respectively. • The attention mechanism and small receptive fields are used to extract global and small-scale moiré features. • Achieve a detection accuracy rate of 0.9691 in our generated demoiréing dataset. [ABSTRACT FROM AUTHOR]