Search

Real-time dense SLAM techniques aim to reconstruct the dense three-dimensional geometry of a scene in real time with an RGB or RGB-D sensor. An indoor scene is an important type of working environment for these techniques. The planar prior can be used in this scenario to improve the reconstruction quality, especially for large low-texture regions that commonly occur in an indoor scene. This article fully explores the planar prior in a dense SLAM pipeline. First, we propose a novel plane detection and segmentation method that runs at 200 Hz on a modern graphics processing unit. Our algorithm for constructing global plane constraints is very efficient; hence, we use it in the process of each input frame for the camera pose estimation while maintaining the real-time performance. Second, we propose herein a plane-based map representation that greatly reduces the memory footprint of plane regions while keeping the geometric details on planes. The experiments reveal that our system yields superior reconstruction results with planar information running at more than 30 fps. Aside from speed and storage improvements, our technique also handles the low-texture problem in plane regions. [ABSTRACT FROM AUTHOR]

Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin. [ABSTRACT FROM AUTHOR]

3D layout is crucial for scene understanding and reconstruction, and very useful in applications like real estate and furniture design. In this paper, we propose a fully automatic solution to estimate 3D layout of an indoor scene from a single 2D image. Our technique contains two key components. Firstly, we train a neural network that directly estimates room structure lines from the input image. Secondly, we propose a novel technique to automatically identify the layout topology of an input image, followed by a nonlinear optimization with equality constraints to estimate the final 3D layout of a scene. Based on our knowledge, this is the first fully automatic technique to achieve single image-based 3D layout estimation of an indoor scene. We evaluate our method on the public datasets $LSUN$ , $Hedau$ and $3DGP$ and the results show that the proposed method achieves accurate 3D layout reconstruction on various images with different layout topologies. [ABSTRACT FROM AUTHOR]

Frame rate up-conversion (FRUC) usually suffers from unreliable motion vectors due to the absence of the current frame to be interpolated. In addition, since the majority of video sequences are usually compressed by various coding standards to reduce the data volume, the quality of the generated frames in the FRUC will be further impaired. To address this problem, we proposed two FRUC algorithms based on deep residual network. We first present a deep residual network for the FRUC (DRNFRUC), which consists of feature extraction, feature recursive analysis, and image restoration parts with a skip connection between the input and the output of the network. The proposed DRNFRUC takes the result of an arbitrary existing FRUC method as the input and is able to significantly reduce the edge blurring and blocking artifacts when the motion of the block is violent. In addition, we proposed a deep residual network with weighted convolutional motion compensation (DRNWCMC) for the FRUC, where the convolution operations can be embedded into the motion compensation interpolation (MCI) in any existing MCI-based FRUC method. In DRNWCMC, we first devise two convolutional neural networks corresponding to the forward and backward motion compensated frames, respectively. And then, the adaptive interpolation coefficients for motion compensation are designed as two $1\times1$ convolutional kernels. Finally, the interpolation result of WCMC is fed into another convolutional neural network to further improve the performance. All the parameters involved in the DRNWCMC are trained simultaneously under the same cost function. The experimental results show that the two proposed algorithms can remarkably improve both the objective and subjective quality of the interpolated frames. [ABSTRACT FROM AUTHOR]

A predefined permission source region (PSR) is commonly used in bioluminescence tomography (BLT) as a priori information to reduce the ill-posedness of source reconstruction. However, the PSR is usually hard to determine and may miss the actual source region. Simultaneously, a linear or nonlinear based reconstruction method commonly have a relatively poor performance in the terms of low photon rate and strong noise. Here, we propose an alternative statistical-based-reconstruction framework with a cluster-algorithm-optimized PSR for BLT. In the alternative reconstruction framework, a global reconstruction is first conducted overall the solving domain to generate a rough estimation of source distribution by using a global based maximum likelihood expectation maximum (MLEM) method. Subsequently, a k-means cluster method is used to learn and find likely source support regions, which generates the optimal PSR. Finally, a local MLEM based reconstruction is performed to refine the source distribution on the optimal PSR. We first verify the performance of the proposed method with a series of numerical simulations, and then explore the potential by imaging in situ liver cancer in living mouse. [ABSTRACT FROM AUTHOR]

Image restoration methods based on convolutional neural networks have shown great success in the literature. However, since most of networks are not deep enough, there is still some room for the performance improvement. On the other hand, though some models are deep and introduce shortcuts for easy training, they ignore the importance of location and scaling of different inputs within the shortcuts. As a result, existing networks can only handle one specific image restoration application. To address such problems, we propose a novel adaptive residual network (ARN) for high-quality image restoration in this paper. Our ARN is a deep residual network, which is composed of convolutional layers, parametric rectified linear unit layers, and some adaptive shortcuts. We assign different scaling parameters to different inputs of the shortcuts, where the scaling is considered as part parameters of the ARN and trained adaptively according to different applications. Due to the special construction of ARN, it can solve many image restoration problems and have superior performance. We demonstrate its capabilities with three representative applications, including Gaussian image denoising, single image super resolution, and JPEG image deblocking. Experimental results prove that our model greatly outperforms numerous state-of-the-art restoration methods in terms of both peak signal-to-noise ratio and structure similarity index metrics, e.g., it achieves 0.2–0.3 dB gain in average compared with the second best method at a wide range of situations. [ABSTRACT FROM AUTHOR]

This paper proposes a 3D laser scanning technology for the pinhole camera with lens distortion. On the basis of the pinhole camera mathematical model, we improve the extraction algorithm of Harris corner points, and introduce the professional 2D RAC calibration, 3D reconstruction, and 3D volume calculation. With the help of stepper motor, and the camera, we can capture a series of images and obtain the object 3D surface information. Furthermore, we use the line laser with Gaussian distribution as the measurement illuminant, and select the samples at the peak of intensity distribution. This brings both high accuracy and speed of measurement. [ABSTRACT FROM AUTHOR]

The dense connection is a powerful technique to build wider and deeper convolution neural networks (CNNs) for handling several computer vision tasks. Despite the excellent performance, it consumes numerous parameters and produces a large weight model file. We studied the distribution of convolution layers and proposed a hard parameter sharing approach known as convolution pool (CP) for compressing dense-connection-based image restoration CNN models. CP is used to reallocate the parameters to specific convolution layers to ensure that some can be shared in different layers. We design a set of dense-connection-based baselines for three typical image restoration tasks, including image denoising, super-resolution, and JPEG deblocking, to validate the performance of the proposed method. Moreover, we comprehensively analyze the potential problems by introducing CP, including group convolution, dilated convolution, and modeling efficiency. Experimental results demonstrate that the proposed method can efficiently achieve an impressive compression rate with negligible performance reduction. [ABSTRACT FROM AUTHOR]

Thanks to the powerful modeling capabilities of multi-head self attention (MSA), transformers have shown significant performance gains in vision tasks. However, as transformers require heavy computation, more efficient designs are required. In this paper, we present an efficient transformer architecture named GoLDFormer for image restoration. GoLDFormer extends the capability of window-based self-attention through two core designs. First, We propose a globally-enhanced window-based transformer block (G-WTB), which applies transposed attention to a compressed window representation rather than the spatial features, thus establishing connections between all windows with less computational complexity. Second, since the interactions between image content and window attention weights can be interpreted as spatially varying convolution, we introduce an adaptive filter structure into transformer models and propose a deformable filtering block (DFB) to enable cross-window connections. By adjusting the shape of the generated filters in the DFB, we can balance the computational costs and the degree of adjacent window interaction. Extensive experiments on several image restoration tasks demonstrate that GoLDFormer achieves competitive results against recent methods but with optimal computational costs. • We propose GoLDFormer, including two blocks, to enhance W-MSA globally and locally. • Globally-enhanced attention block captures global dependencies between all windows. • Deformable filtering block captures local dependencies by modifying filter size. • Our GoLDFormer achieves competitive results against recent lightweight methods. [ABSTRACT FROM AUTHOR]