Search

Due to hardware limitations, existing hyperspectral (HS) camera often suffer from low spatial/temporal resolution. Recently, it has been prevalent to super‐resolve a low resolution (LR) HS image into a high resolution (HR) HS image with a HR RGB (or multispectral) image guidance. Previous approaches for this guided super‐resolution task often model the intrinsic characteristic of the desired HR HS image using hand‐crafted priors. Recently, researchers pay more attention to deep learning methods with direct supervised or unsupervised learning, which exploit deep prior only from training dataset or testing data. In this article, an efficient convolutional neural network‐based method is presented to progressively super‐resolve HS image with RGB image guidance. Specifically, a progressive HS image super‐resolution network is proposed, which progressively super‐resolve the LR HS image with pixel shuffled HR RGB image guidance. Then, the super‐resolution network is progressively trained with supervised pre‐training and unsupervised adaption, where supervised pre‐training learns the general prior on training data and unsupervised adaptation generalises the general prior to specific prior for variant testing scenes. The proposed method can effectively exploit prior from training dataset and testing HS and RGB images with spectral‐spatial constraint. It has a good generalisation capability, especially for blind HS image super‐resolution. Comprehensive experimental results show that the proposed deep progressive learning method outperforms the existing state‐of‐the‐art methods for HS image super‐resolution in non‐blind and blind cases. [ABSTRACT FROM AUTHOR]

Object detection is one of the most challenging tasks in computer vision. With the advances in semiconductor devices and chip technology, hardware accelerators have been widely used. Field-programmable gate arrays (FPGAs) are a highly flexible hardware platform that allows customized reconfiguration of the integrated circuit, which has the potential to improve the efficiency of object detection accelerators. However, few reviews summarize FPGA-based object detection accelerators. Also, there is no general principle for realizing object detection according to FPGA characteristics. In this paper, the current hardware accelerators are introduced and compared. Then, the typical deep learning-based object detectors are summarized. Next, the questions of "Why choose FPGA," "The design goals of FPGA accelerators" and "The design methods for FPGA accelerators" are discussed in detail. Finally, the challenges of object detection algorithms, hardware, and co-design are presented. In addition, an online platform (https://github.com/vivian13maker/) is constructed to provide specific information on all advanced works. [ABSTRACT FROM AUTHOR]

Effective 3D shape retrieval and recognition are challenging but important tasks in computer vision research field, which have attracted much attention in recent decades. Although recent progress has shown significant improvement of deep learning methods on 3D shape retrieval and recognition performance, it is still under investigated of how to jointly learn an optimal representation of 3D shapes considering their relationships. To tackle this issue, we propose a multi-scale representation learning method on hypergraph for 3D shape retrieval and recognition, called multi-scale hypergraph neural network (MHGNN). In this method, the correlation among 3D shapes is formulated in a hypergraph and a hypergraph convolution process is conducted to learn the representations. Here, multiple representations can be obtained through different convolution layers, leading to multi-scale representations of 3D shapes. A fusion module is then introduced to combine these representations for 3D shape retrieval and recognition. The main advantages of our method lie in 1) the high-order correlation among 3D shapes can be investigated in the framework and 2) the joint multi-scale representation can be more robust for comparison. Comparisons with state-of-the-art methods on the public ModelNet40 dataset demonstrate remarkable performance improvement of our proposed method on the 3D shape retrieval task. Meanwhile, experiments on recognition tasks also show better results of our proposed method, which indicate the superiority of our method on learning better representation for retrieval and recognition. [ABSTRACT FROM AUTHOR]

Salient object detection from RGB-D images aims to utilize both the depth view and RGB view to automatically localize objects of human interest in the scene. Although a few earlier efforts have been devoted to the study of this paper in recent years, two major challenges still remain: 1) how to leverage the depth view effectively to model the depth-induced saliency and 2) how to implement an optimal combination of the RGB view and depth view, which can make full use of complementary information among them. To address these two challenges, this paper proposes a novel framework based on convolutional neural networks (CNNs), which transfers the structure of the RGB-based deep neural network to be applicable for depth view and fuses the deep representations of both views automatically to obtain the final saliency map. In the proposed framework, the first challenge is modeled as a cross-view transfer problem and addressed by using the task-relevant initialization and adding deep supervision in hidden layer. The second challenge is addressed by a multiview CNN fusion model through a combination layer connecting the representation layers of RGB view and depth view. Comprehensive experiments on four benchmark datasets demonstrate the significant and consistent improvements of the proposed approach over other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Large scale image understanding is a challenging but significant task to comprehend image contents on the internet. The de-facto standard methods based on machine learning or computer vision still suffer from a phenomenon of visual polysemia and concept polymorphism (VPCP). To resolve the VPCP, semantic dictionary has been proposed to characterize the membership distribution between visual appearances and semantic concepts. In this paper, we propose an online semantic dictionary learning algorithm on the base of both locality sensitive hashing (LSH) and stochastic approximations, which can scale up to large scale datasets with millions of training samples and speedup the efficiency of follow-up processing. With the help of the LSH-based semantic dictionary, we develop an extension of the spatial pyramid matching (SPM) kernel method by generalizing the dictionary as a basic semantic description. The efficiency of our approach is validated in the experiments of web-scale semantic image search and image classification on the ImageNet dataset and Caltech-256 dataset. [ABSTRACT FROM AUTHOR]

With the development of productivity, people set higher demands on the quality of steel. In recent years, artificial intelligence, especially deep learning-based computer vision technology has attracted great attention, which can be used for detecting steel surface defects such as scratches, patches, and rust spots. However, due to the complexity of the strip steel surface, it is still a challenge to accurately and effectively detect defect regions by the existing defects detection methods. Therefore, we propose a unique saliency model, i.e., deeper feature integration network to highlight the defect regions on the strip steel surface. To be specific, after each encoder stage, we introduce the multiscale global feature extraction module to elevate the multiscale deep features from the encoder. Meanwhile, we deploy the deeper feature extraction module, which contains a bidirectional feature extraction unit, to dig the effective representation for defects. Particularly, the forward one is equipped with a channel-space weighted module and the backward one is equipped with a split attention module. After that, the features from the two branches are progressively integrated by the decoder, yielding the final high-quality saliency maps that give a good depiction of the strip steel surface defects. Extensive experiments are executed on the public dataset, and the comparison results show that our model performs better than the 15 state-of-the-art methods, which proves the effectiveness and superiority of the proposed model. [ABSTRACT FROM AUTHOR]

The dense connection is a powerful technique to build wider and deeper convolution neural networks (CNNs) for handling several computer vision tasks. Despite the excellent performance, it consumes numerous parameters and produces a large weight model file. We studied the distribution of convolution layers and proposed a hard parameter sharing approach known as convolution pool (CP) for compressing dense-connection-based image restoration CNN models. CP is used to reallocate the parameters to specific convolution layers to ensure that some can be shared in different layers. We design a set of dense-connection-based baselines for three typical image restoration tasks, including image denoising, super-resolution, and JPEG deblocking, to validate the performance of the proposed method. Moreover, we comprehensively analyze the potential problems by introducing CP, including group convolution, dilated convolution, and modeling efficiency. Experimental results demonstrate that the proposed method can efficiently achieve an impressive compression rate with negligible performance reduction. [ABSTRACT FROM AUTHOR]

The assessment of image is a critical task in computer vision, particularly in a no-referenced context, which presents numerous challenges. Though Deep neural networks have exhibited excellent performance on no-reference quality assessment (NR-IQA), the distortion-aware NR-IQA methods are limited to specific distortion type and struggle to handle realistic distortion scenarios. In this work, we propose a feature rectification and enhancement convolutional neural network namely FREIQA for NR-IQA. Our approach targets to fully utilize the multi-stage semantic features to produce a well fused and rectified score vector for effective image quality regression. Specifically, the multi-stage semantic features extracted by a pre-trained backbone would be rectified with a multi-level channel attention module before integrating to a global score vector. The fused score vector then will be gradually optimized through the perception feature provided by the multi-stage semantic feature, and finally sent to quality regression network to obtain the image quality score. Experimental results on public benchmarks including LIVE, TID2013, CSIQ, LIVEC, KADID-10k, KonIQ-10k and Waterloo exploration database show that our FREIQA achieve the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Surface defect detection has become more and more important in the industrial manufacture and engineering application in recent years. However, due to the lack of overall perception and interaction among features layers, lots of computer vision-based detection methods cannot grab the complete details of defects when dealing with complex scenes, such as low contrast and irregular shape. Therefore, in this paper, we propose a Context-aware Aggregation Network (CANet) to accurately pop-out the defects, where we focus on the mining of context cues and the fusion of multiple context features. To be specific, embarking on the multi-level deep features extracted by encoder, we first deploy a sufficient exploration to dig the context information by deploying the weighted convolution pyramid (WCP) module, which extracts multi-scale context features, transfers the information flow between different resolution features, and fuses the features with same resolution. By this way, we can obtain the effective context pyramid features. Then, the decoder deploys the weighted context attention (WCA) module to filter the irrelevant information from context features and employs the cascaded fusion structure (CFS) to aggregate the multiple context cues in a hierarchical way. Following this way, the generated high-quality saliency maps can highlight the defects accurately and completely. Extensive experiments are performed on four public datasets, and the results firmly prove the effectiveness and superiority of the proposed CANet under different evaluation metrics. [ABSTRACT FROM AUTHOR]