Search

• An image fusion method is proposed that can effectively highlight infrared target features and retain visible visual information. • Infrared targets are analyzed based on the saliency weight map. • The target contour and background texture are refined by a method based on weight refinement of guided filtering. • A multi-layer fusion strategy is used to fuse saliency weight maps at multiple scales. Image fusion is the fusion of multiple images from the same scene to produce a more informative image, and infrared and visible image fusion is an important branch of image fusion. To tackle the issues of diminished luminosity in the infrared target, inconspicuous target features, and blurred texture of the fused image after the fusion of infrared and visible images. This paper introduces a novel effective fusion framework that merges multi-scale Convolutional Neural Networks (CNN) with saliency weight maps. First, the method measures the source image features to estimate the initial saliency weight map. Then, the initial weight map is segmented and optimized using a guided filter before being further processed by CNN. Next, a trained Siamese convolutional network is used to solve the two key problems of activity measure and weight assignment. Meanwhile, a multi-layer fusion strategy is designed to effectively retain the luminance of the infrared target and the texture information in the visible background. Finally, adaptive adjustment of the fusion coefficients is achieved by employing saliency. The experimental results show that the method outperforms the state-of-the-art algorithms in terms of both subjective visual quality and objective evaluation effects. [ABSTRACT FROM AUTHOR]

• A layout plane is a closed loop surrounded by layout keypoints. • The wall structure can be parametrization as an undirected graph. • Layout keypoint prediction is relative to human pose estimation. • Cuboid room assumption is limited in real life. Room layout estimation aims to predict the spatial structure of a room from a single image. Most existing methods relying on 2D cues are suitable for cuboid rooms, but non-cuboid rooms. 3D layout estimation methods can reconstruct 3D models of general rooms, but they are trained with depth information on high collection costs, consume large computation resources, and run slowly. This paper considers an undirected graph representation method of the general room layouts, which includes cuboid and non-cuboid rooms consisting of a single ceiling, a single floor, and multiple walls. To this end, we first predict the positions of the layout vertices and then use the network automatically learn the connection relationship between the vertices. The final layout is obtained through a simple layout inference post-processing algorithm. The experimental results both on cuboid and non-cuboid datasets validate the effectiveness and efficiency of our method. The code is available at https://github.com/Hui-Yao/2D-graph-layout-estimation. [ABSTRACT FROM AUTHOR]

Human activity recognition is one of the most studied topics in the field of computer vision. In recent years, with the availability of RGB-D sensors and powerful deep learning techniques, research on human activity recognition has gained momentum. From simple human atomic actions, the research has advanced towards recognizing more complex human activities using RGB-D data. This paper presents a comprehensive survey of the advanced deep learning based recognition methods and categorizes them in human atomic action, human–human interaction, human–object interaction. The reviewed methods are further classified based on the individual modality used for recognition i.e. RGB based, depth based, skeleton based, and hybrid. We also review and categorize recent challenging RGB-D datasets for the same. In addition, the paper also briefly reviews RGB-D datasets and methods for online activity recognition. The paper concludes with a discussion on limitations, challenges, and recent trends for promising future directions. • A comprehensive categorical survey of RGB-D human action/interaction recognition. • Categorized RGB-D datasets, methods into human actions, human–object interactions. • In-depth analysis of well-known deep learning methods for actions and interactions. • Examined distinct characteristics of RGB-D modalities for actions/interactions. • Discussed limitations, challenges, and promising future directions in the domain. [ABSTRACT FROM AUTHOR]

Document examination is a vital mission for revealing illegal modifications that assist in the detection and resolution of criminal acts. Addition and alteration are more frequently used in handwritten documents. However, most of the documents have been modified with similar inks, and it is tough to detect or observe them with human eyes. As a result, there is a need for methods to automatically detect handwriting forgery to reach an accurate detection efficiently. In this paper, a novel and efficient method is proposed for automatically detecting altered handwritten documents and locating the fake part. Therefore, DE-Net is proposed to identify the forged document using a digitally scanned version of the document. Unlike the existing methods, a further localization schema is applied to locate the forged parts in the candidate forged document accurately. Where each forged document is segmented into objects. Color histograms of R, G, and B channels are used to generate a fused feature vector for each object. Then a structural similarity index (SSIM) is applied to detect the lower similarity parts as forged. The experimental results demonstrate that the proposed method can identify and localize foreign ink in handwritten documents with high performance. • Propose a novel and efficient method for exposing altered documents via DE-Net. • Localize the foreign ink based on image processing rather than analytical chemistry. • SSIM of RGB color histograms is used as handcrafted features for localization. • An integrated dataset is introduced for document forensics and machine learning use. [ABSTRACT FROM AUTHOR]

Nowadays, stereoscopic image quality assessment (SIQA) based on convolutional neural network (CNN) has become the mainstream model of image quality assessment (IQA). Compared with the two-dimensional quality evaluation model, stereoscopic image quality evaluation is more challenging due to the effects of depth and parallax information. In this paper, we propose a two-stream interactive network model to perform quality evaluation, which can well simulate the process of human stereo visual perception. Meanwhile, we enhance the extraction of local and global features of images by asymmetric convolution kernel and interactive sub-networks of inter-layers, respectively, which can further optimize our network model. Our proposed algorithm was evaluated on four public databases. The final experimental results show that our proposed algorithm exhibits good performance not only on the whole database but also on each single distortion type. [ABSTRACT FROM AUTHOR]

This paper proposes a novel two-stream encoder–decoder network that utilizes both the high-level and the low-level image features for precisely localizing forged regions in a manipulated image. This is motivated by the fact that the forgery creation process generally introduces both the high-level artefacts (e.g. , unnatural contrast) and the low-level artefacts (e.g. , noise inconsistency) to the forged images. In the proposed two-stream network, one stream learns the low-level manipulation-related features in the encoder side by extracting noise residuals through a set of high-pass filters in the first layer. In the second stream, the encoder learns the high-level image manipulation features from the input image RGB values. The coarse feature maps each encoder are upsampled by the corresponding decoder network to produce the dense feature maps. The dense feature maps of the two streams are concatenated and fed to a final convolutional layer with sigmoidal activation to produce the pixel-wise prediction. We have carried out experimental analyses on multiple standard forensics datasets to evaluate the performance of the proposed method. The experimental results show the efficacy of the proposed method with respect to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Instance search is an interesting task as well as a challenging issue due to the lack of effective feature representation. In this paper, an instance level feature representation built upon fully convolutional instance-aware segmentation is proposed. The feature is ROI-pooled from the segmented instance region. So that instances in various sizes and layouts are represented by deep features in uniform length. This representation is further enhanced by the use of deformable ResNeXt blocks. Superior performance is observed in terms of its distinctiveness and scalability on a challenging evaluation dataset built by ourselves. In addition, the proposed enhancement on the network structure also shows superior performance on the instance segmentation task. [ABSTRACT FROM AUTHOR]

Human detection in crowded scenes is challenging since the objects occlude and overlap each other. Compared to general pedestrian detection, there is also more variation in human posture. This paper proposes a real-time human detection network, Dynamic Dual-Peak Network (DDPNet), which specifically addresses human object detection in overlapping and crowded scenes. We design a deep cascade fusion module to enhance the feature extraction capability of the anchor-free model for small objects in crowded scenes. In the meantime, the head–body dual-peak activation module is used to improve the prediction score of the central region of the occluded individual through low occlusion components. By this improvement strategy, the network's ability is enhanced to discriminate individuals in crowded scenes and alleviate the problem caused by individual posture variation. Ultimately, we propose a novel Exhale–Inhale method to adjust the feature mapping ranges for various scale objects dynamically. In the process of ground truth mapping, the overlapping of individual feature information is reduced. Our DDPNet achieves competitive performance on the CrowdHuman dataset and executes real-time inference of almost 3x ∼ 7x faster than competitive methods. [ABSTRACT FROM AUTHOR]