Search

• Proposing a shift-insensitive perceptual feature based on GM and LoG signals, and theoretically selecting a ratio parameter to balance them. • Building FR-IQA models based on the proposed QGL feature maps that achieve good performance on three benchmark databases. • Achieving the best shift-insensitive performance on images with spatial changes compared with other metrics. • Combining QGL with deep neural networks and validating that QGL can help improving the accuracy in texture classification. Most existing full-reference (FR) Image quality assessment (IQA) models work in the premise of that the two images should be well registered. Shifting an image would lead to an inaccurate evaluation of image quality, because small spatial shifts are far less noticeable than structural distortion for human observers. To this regard, we propose to study an IQA feature that is shift-insensitive to the basic primitive structure of images, i.e., image edge. According to previous studies, the image gradient magnitude (GM) and the Laplacian of Gaussian (LoG) operator that depict the edge profiles of natural images are highly efficient structural features in IQA tasks. In this paper, we find that the Quadratic sum of the normalized GM and the LoG signals (QGL) has excellent shift-insensitive property in representing image edges after theoretically solving the selection problem of a ratio parameter to balance the GM and LoG signals. Based on the proposed QGL feature, two FR-IQA models can be built directly by measuring the similarity map with mean and standard deviation pooling strategies, named mQGL and sQGL, respectively. Experimental results show that the proposed sQGL and mQGL work robustly on four benchmark IQA databases, and QGL-based models show great shift-insensitive property to spatial translation and image rotation while judging the image quality. In addition, we explore the feasibility of combining QGL feature with deep neural networks, and verify that it can help to promote image pattern recognition in texture classification tasks. [ABSTRACT FROM AUTHOR]

Domain generalizable person re-identification (DG ReID) aims to obtain a model that can be applied directly to unseen domains once trained on a set of source domains (datasets collected from different camera networks). Among current DG ReID methods, instance normalization is a promising solution to reduce the effect of domain bias, but it inevitably filters out some discriminative information. Besides, since most pioneering approaches cannot effectively address the loss of discriminative information, this paper proposes a Style Elimination and Information Restitution (SEIR) module and constructs a generalizable framework. Specifically, we utilize instance normalization to reinforce the generalization capability on unseen domains. Then, the instance-specific mean and variance are employed to construct a feature vector that includes discriminative and style information. Besides, an encoder–decoder structure is designed to restitute the discriminative information to ensure a high recognition rate. Finally, a dual optimization strategy is devised to update the proposed module by simulating a real train-test process to enhance the generalization robustness further and prevent SEIR from overfitting to the source domain. Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art methods by a large margin on the public DG ReID benchmarks. • A Style Elimination and Information Restitution module for handling domain bias. • An optimization strategy for improving the generalization ability. • We propose a framework for generalizable person re-identification. • Significant results were achieved for addressing the generalizable person re-identification. [ABSTRACT FROM AUTHOR]

In the last decade, the exploration of deep-sea ecosystems has surged, offering exciting prospects for discovering untapped resources such as medical drugs, food and energy sources, and renewable energy products. Consequently, research in underwater image processing has witnessed substantial growth. However, underwater imaging poses significant challenges, particularly without sophisticated, specialized cameras. Traditional cameras are impacted by absorption and scattering in the aquatic environment, producing hazy images with a blue–green tint. This phenomenon holds implications for marine research and other disciplines that rely on underwater imaging. While hardware advancements have been made over the years, image processing remains a valuable, cost-effective, and practical approach for underwater enhancement. Despite the existence of state-of-the-art techniques for underwater enhancement and restoration, their performance is often inconsistent. While some methods excel in contrast restoration, color restoration remains a pervasive challenge. In this paper, we introduce Sea-Pix-GAN, a Generative Adversarial Network (GAN)-based model that addresses these issues in underwater image enhancement. We redefine the problem as an image-to-image translation task and tailor the objective and loss functions to achieve color, content, and style transfer. The model is trained on a large dataset of underwater scenes, encompassing the diverse color dynamics of underwater subjects. Sea-Pix-GAN demonstrates promising results in restoring color, contrast, texture, and saturation. To validate its effectiveness, we compare the performance of Sea-Pix-GAN quantitatively based on metrics like PSNR, SSIM, and UIQM and qualitatively against several existing techniques. [ABSTRACT FROM AUTHOR]

In social networks, people usually communicate online by posting and passing images, people will find that there are many similar or even visually identical images. When the images determine whether the published information is true or false, original images became crucial, and at the same time, deep learning excelled in image identification. Combining the original image identification and deep learning methods, this paper proposes the original image identification method based on the deep network models, aiming to use the deep network models to identify the original images between the original images and a bunch of variant images of the original images, and obtain satisfactory results. In this way, through the trained network models, the original images can be identified and can help the viewers to determine the correctness of the information. Experimental results verify the feasibility, effectiveness and security of the proposed method. • The method can prevent the images from being stolen during transmission. • The method can achieve good identification results on the image databases. • The method is feasible and flexible in actual operation. [ABSTRACT FROM AUTHOR]

• An image fusion method is proposed that can effectively highlight infrared target features and retain visible visual information. • Infrared targets are analyzed based on the saliency weight map. • The target contour and background texture are refined by a method based on weight refinement of guided filtering. • A multi-layer fusion strategy is used to fuse saliency weight maps at multiple scales. Image fusion is the fusion of multiple images from the same scene to produce a more informative image, and infrared and visible image fusion is an important branch of image fusion. To tackle the issues of diminished luminosity in the infrared target, inconspicuous target features, and blurred texture of the fused image after the fusion of infrared and visible images. This paper introduces a novel effective fusion framework that merges multi-scale Convolutional Neural Networks (CNN) with saliency weight maps. First, the method measures the source image features to estimate the initial saliency weight map. Then, the initial weight map is segmented and optimized using a guided filter before being further processed by CNN. Next, a trained Siamese convolutional network is used to solve the two key problems of activity measure and weight assignment. Meanwhile, a multi-layer fusion strategy is designed to effectively retain the luminance of the infrared target and the texture information in the visible background. Finally, adaptive adjustment of the fusion coefficients is achieved by employing saliency. The experimental results show that the method outperforms the state-of-the-art algorithms in terms of both subjective visual quality and objective evaluation effects. [ABSTRACT FROM AUTHOR]

Previous deep learning-based single image dehazing methods are implemented by using real-valued convolutional neural network (RV-CNN), which ignore the phase information of the image, and tend to cause models to perform well on synthetic datasets but low generalization on real-world datasets. In this paper, we propose a novel Complex-Valued convolutional Dehazing Network (CVD-Net), which considers both amplitude and phase information of image for haze removal in real-world scenes. Specifically, we construct complex-valued transformation module, including complex-valued convolutional layer and residual block embedded into the middle of the network to extract features of image efficiently, and also design a complex-valued selected fusion module (CVSF) and complex-valued attention module (CVAM) to promote the interaction between different scale features for preserving image detailed information. Both qualitative and quantitative experimental results show that the proposed CVD-Net can effectively remove the haze, and has good generalization in real-world hazy images. [ABSTRACT FROM AUTHOR]

Person Re-Identification (PR-Id) encounters misclassification issues when re-identifying persons with different backpacks. These bags manifest as large and distinct regions in the images surpassing other finer details of a person. As a result, a CNN model swiftly detects and prioritizes these image regions for re-identification. However, the bags are subject to alterations or may be similar among multiple persons, resulting in misclassification. To ensure that a CNN model does not consider bags as unique features of specific persons and prioritize them for re-identification, images of persons with diverse backpacks are crucial in the training dataset. Moreover, these images enhance the model's focus on other unique regions of a person. Although the current datasets show potential, incorporating such images could enhance their effectiveness. Therefore, in this paper, we propose an indoor PR-Id dataset named "With Bag/Without Bag-ReID" (WB/WoB-ReID). The set "with_bag" in WB/WoB-ReID dataset includes identities with different backpacks for the first time. We also incorporate identities without bags and with varying numbers of image counts in three other sets, namely "without_bag", "both_small," and "both_large". We assess WB/WoB-ReID and three other PR-Id datasets: Market1501, CUHK03, and DukeMTMC-reID on various existing approaches. The highest mAP achieved on the"with_bag" is 74%, "without_bag" is 96.7% and other datasets are 97.78%, 95.20% and 92.4%. The results show that incorporating identities with diverse bags reduces the mAP, highlighting the misclassifications that arise specifically in the presence of bags. [ABSTRACT FROM AUTHOR]

Despite the remarkable results achieved by deep convolutional neural networks (CNNs) in single image super-resolution (SISR), their computational cost increase exponentially as CNN models get deeper and wider. Consequently, depthwise separable convolutions (DSConvs) have emerged as the fundamental building basis for various contemporary lightweight network architectures. However, depthwise convolution is sub-optimal for restoring missing high-frequency details. In this paper, we commence with the vanilla depthwise convolution and progressively develop it to alleviate the above problem. Specifically, we introduce an effective method, kernel attention injection, to integrate intra-kernel correlation into the depthwise convolution layer by incorporating the learned kernel attention into the depthwise kernel. We find that the determinant is capable of capturing the correlation between diagonal elements of the depthwise kernel by summing the products of the diagonals. Therefore, we utilize the determinant as the kernel pooling method. Furthermore, we propose to remove the non-linear activation function behind the depthwise convolution (i.e. , linear depthwise convolution) to preserve informative features for reconstructing faithful high-resolution (HR) images. Built on this recipe, we introduce kernel-attentive linear depthwise separable convolutions (KDSConvs), which exhibits promising super-resolution performance. Our experimental results underline the potential of depthwise convolution in super-resolution tasks by demonstrating that the proposed KDSConvs significantly outperforms DSConvs in terms of both quantitative measurements and visual quality. The code will be released at https://github.com/AwesomeHwang/KALDN. • We propose kernel attention injection to integrate intra-kernel correlation. • We utilize the determinant as the kernel pooling method. • We introduce kernel-attentive linear depthwise separable convolutions. [ABSTRACT FROM AUTHOR]

• We present a novel discriminative model to learn similarity measurement between source domain and target domain. • We freeze part of the network layers to extract well-defined representations from source domain to target domain. • We use auxiliary images labelled by perspective-class to improve categorization performance. Fine-grained categorization is challenging due to its small inter-class and large intra-class variance. Moreover, requiring domain expertise makes fine-grained labelled data much more expensive to acquire. Existing models predominantly require extra information such as bounding box and part annotation in addition to the image category labels, which involves heavy manual labor. In this paper, we propose a novel hierarchical deep transfer learning model, based on a compression convolutional neural network. Our model transfers the learned image representation from large-scale labelled fine-grained datasets to micro fine-grained datasets, which avoids using expensive annotations and realizes visual categorization task effectively. Firstly, we introduce a cohesion domain to measure correlation degree between source domain and target domain. Secondly, the source-domain convolutional neural network is adjusted according to its metrical feedback, in order to select task-specific features that are suitable for transferring to the target domain. Finally, we make most of perspective-class labels, which are inherent attributes of fine-grained data for multi-task learning and learn all the attributes through joint learning to extract more discriminative representations. The proposed model not only economizes training time effectively and achieves high categorization accuracy, but also verifies that the inter-domain feature transition can accelerate learning and optimization. [ABSTRACT FROM AUTHOR]

Abstract With the development of information transmission technology and computer technology, information acquisition mode is mainly converted from character to image nowadays. However, in the process of acquiring and transmitting images, image damage and quality decrease due to various factors. Therefore, how to restore image has become a research hotspot in the field of image processing. This paper establishes an image restoration model based on BP neural network. The simulation results show that the proposed method has made a great improvement compared with the traditional image restoration method. [ABSTRACT FROM AUTHOR]

Recently, very deep convolution neural network (CNN) has shown strong ability in single image super-resolution (SISR) and has obtained remarkable performance. However, most of the existing CNN-based SISR methods rarely explicitly use the high-frequency information of the image to assist the image reconstruction, thus making the reconstructed image looks blurred. To address this problem, a novel contour enhanced Image Super-Resolution by High and Low Frequency Fusion Network (HLFN) is proposed in this paper. Specifically, a contour learning subnetwork is designed to learn the high-frequency information, which can better learn the texture of the image. In order to reduce the redundancy of the contour information learned by the contour learning subnetwork during fusion, the spatial channel attention block (SCAB) is introduced, which can select the required high-frequency information adaptively. Moreover, a contour loss is designed and it is used with the ℓ 1 loss to optimize the network jointly. Comprehensive experiments demonstrate the superiority of our HLFN over state-of-the-art SISR methods. • High-frequency information will be lost in the restoration process of super-resolution reconstruction. • By using image contour information to assist super-resolution reconstruction. • Introducing spatial channel attention to enhance high-frequency information further. • Use proportion of image contour information is controlled by contour loss. [ABSTRACT FROM AUTHOR]

Object trackers based on Siamese network usually transform the tracking task into a matching problem between the candidate samples and the target template. However, with the increasing depth and width of backbone networks, researches on Siamese trackers using backbone networks are not very advanced. Therefore, it is necessary for us to further investigate the characteristics of backbone network. As a fact, the ability of backbone network to extract features can directly determine the performance of object tracker. Given this, in this paper, we first propose an asymmetric convolutional network to improve the representational capability of backbone network. And then, the strip convolution is employed to enhance the operational capability of square kernel convolution in the backbone network. Besides, we also construct a novel module named Feature Dropblock (i.e., FD) to simulate the occlusion of hidden space, which goal is to improve the performance of backbone network in the target tracking under occlusion. To demonstrate the effectiveness of the proposed tracker, extensive ablation studies are conducted. Better results are obtained on the tracking benchmarks OTB100 and VOT2018, compared to other state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

Recently, Siamese trackers have received widespread attention for visual object tracking owing to their good balance between speed and performance. Those Siamese trackers heavily depend on target template while conventional practice fixes the template to initial frame. This strategy makes it unable to cope with variation of target appearance, which often leads to tracking failures and causes the gap in performance from other tracking methods. Despite the performance gain achieved by few template update methods with target templates generated by the tracked results, these tracked templates are easy to accumulate errors and cause tracking drift. In this paper, we propose two template update mechanisms to effectively adapt the target template during the tracking process which is dubbed as DTDU (Dynamic Template with Dual Update). Unlike predecessors that directly use the tracked template, we use initial template to perform similar transformation to the tracked template. Then the similar transformed template and the tracked template are combined linearly to capture the variation of target appearance. These updated templates are stored in a memory bank and retrieved to generate the final target template. In order to enhance quick update of memory bank to accommodate the target appearance, we use the retrieved template to further update the templates in memory bank for subsequent tracking. Extensive experiments on OTB-2015, VOT2016, VOT2018 and GOT-10k datasets have proved the effectiveness of these two update mechanisms and the proposed tracker achieves a real-time speed of 44 fps. • We update template to integrate initial and variation appearance of target for robustness. • The retrieved template is to update templates in memory bank for more dynamic change. • Extensive evaluations on four benchmarks show the effectiveness of the proposed DTDU. [ABSTRACT FROM AUTHOR]

Vehicle re-identification (V-ReID) aims at discovering an image of a specific vehicle from a set of images typically captured by different cameras. Vehicles are one of the most important objects in cross-camera target recognition systems, and recognizing them is one of the most difficult tasks due to the subtle differences in the visible characteristics of vehicle rigid objects. Compared to various methods that can improve re-identification accuracy, data augmentation is a more straightforward and effective technique. In this paper, we propose a novel data synthesis method for V-ReID based on local-region perspective transformation, transformation state adversarial learning and a candidate pool. Specifically, we first propose a parameter generator network, which is a lightweight convolutional neural network, to generate the transformation states. Secondly, an adversarial module is designed in our work, it ensures that noise information is added as much as possible while keeping the labeling and structure of the dataset intact. With this adversarial module, we are able to promote the performance of the network and generate more proper and harder training samples. Furthermore, we use a candidate pool to store harder samples for further selection to improve the performance of the model. Our system pays more balanced attention to the features of vehicles. Extensive experiments show that our method significantly boosts the performance of V-ReID on the VeRi-776, VehicleID and VERI-Wild datasets. • A data augmentation method is designed that integrates a deep learning framework. • Local-region perspective transformation is optimized with an adversarial model. • More balanced attention is paid to the overall features of vehicles. • Data augmentation and the recognition model are jointly optimized. • Amplified samples are more efficient due to the automatic learning process. • A candidate pool stores the augmented images through the dynamic learning process. • Datasets are augmented by lifting the difficulty rather than the quantity. • The structure of the original dataset is reserved including the size and labels. [ABSTRACT FROM AUTHOR]

Single image dehazing is a critical image pre-processing step for many practical vision systems. Most existing dehazing methods solve this problem utilizing various of hand-crafted priors or by supervised training on the synthetic hazy image information (such as haze-free image, transmission map and atmospheric light). However, the assumptions on the hand-crafted priors are easily violated and collecting realistic transmission map and atmospheric light are unpractical. In this paper, we propose a novel weakly supervised network based on the multi-level multi-scale block. The proposed network reduces the constraint on the training data and automatically estimates the transmission map and the atmospheric light as well as the intermediate haze-free image without using any realistic transmission map and atmospheric light as supervision. Moreover, the estimated intermediate haze-free image helps to generate accurate transmission map and atmospheric light by embedding the physical-model, which presents reliable restoration of the final haze-free image. In particular, our network also can be trained on the real-world dataset to fine-tune the model and the fine-tuning operation improves the dehazing performance on the real-world dataset. Quantitative and qualitative experimental results demonstrate the proposed method performs on par with the supervised methods. • We propose a weakly supervised single image dehazing method. • The network can simultaneously estimate the transmission map and atmospheric light. • The network can be fine-tuned to improve the real-world dehazing result. [ABSTRACT FROM AUTHOR]

• A new animation method is proposed to generate a video animation from still image based on intelligent computing. • The active objects can generate a virtual driving force to drive the passive objects. • An innovative analytical method is designed to generate a virtual driving force for active objects. • Two types of animation driving force was specified in our method: virtual driving force and virtual wind force. Bringing a single still image into reality is a challenging topic in computer animation because the driven and structural information in single still image is inadequate. In this paper, we present an image animating method for enhancing single still image in social media with virtual realistic and animated motions without prior information. We imitate the interaction between the active objects in an image and their neighboring passive objects. The existing actions in the image and the virtual specified force are employed to animate the active objects. Observing that the change between two subsequent motions of the active objects derives a motion tendency, we can calculate a virtual driving force based on the motion tendency. By virtue of the virtual driving force, the stochastic motion texture is used to animate the passive objects. Finally, the convolutional neural network is employed to optimize the virtual motion animations. In this way, the proposed method produces visually natural results while guaranteeing motion harmony between active objects and passive objects. To demonstrate the applicability and rationality of virtual animation driving force, our method generates several animations from still images in Social Media. [ABSTRACT FROM AUTHOR]

• Retaining high frequency detail information is important for image denoising. • Removing batch normalization (BN) is beneficial for image denoising. • Image restoration tasks can be finished by a single end-to-end network. Compared with the traditional image denoising method, although the convolutional neural network (CNN) has better denoising performance, there is an important issue that has not been well resolved: the residual image obtained by learning the difference between noisy image and clean image pairs contains abundant image detail information, resulting in the serious loss of detail in the denoised image. In this paper, in order to relearn the lost image detail information, a mathematical model is deducted from a minimization problem and an end-to-end detail retaining CNN (DRCNN) is proposed. Unlike most denoising methods based on CNN, DRCNN is not only focus to image denoising, but also the integrity of high frequency image content. DRCNN needs less parameters and storage space, therefore it has better generalization ability. Moreover, DRCNN can also adapt to different image restoration tasks such as blind image denoising, single image superresolution (SISR), blind deburring and image inpainting. Extensive experiments show that DRCNN has a better effect than some classic and novel methods. [ABSTRACT FROM AUTHOR]

• We propose spectral pyramid transform to model actions sufficiently. • We design a sampling strategy to make actions with various duration aligned. • We propose a feature pyramid to fuse CNN features and spectral features. In this paper, we propose a novel Deep Spectral Feature Pyramid in the Frequency domain (DSFP) to share the merits of deep features and spectral approaches for long-term action recognition. More specifically, in the spatial domain, deep features of sparse sampled frames are extracted by Convolutional Neural Networks (CNNs) to cover long-term temporal structure. In the frequency domain, appearance features of sampled frames are partitioned recursively along the time dimension and spectral transform is applied to each partitioned feature respectively. All coefficients of partitioned features are then concatenated into a video-level feature to better model the spatio-temporal structure of actions in the form of a pyramid. So DSFP could model actions from both microcosmic and macroscopic aspects. Extensive experiments conducted on two challenging action benchmarks UCF101 and HMDB51 show that our proposed DSFP is effective for spatio-temporal representation of actions and achieves comparable performance with the state-of-the-arts. [ABSTRACT FROM AUTHOR]