Search

Classification algorithm is the crucial technique for machine vision system. The Transfer Component Analysis has been introduced for workpiece classification system. The dataset with five classes of workpieces is constructed for experiments, and the data augmentation strategies have been employed to prevent class imbalance problems. The Histogram of the Oriented Gradient features are extracted over workpiece images, and they are transformed by the Transfer Component Analysis and the Principal Component Analysis separately. The Maximum Mean Discrepancy distances have been employed to measure the distribution distances between the training set and the testing set. And the results indicate that the Transfer Component Analysis has dramatically reduced the distribution differences in a dimensionality reduction subspace, better than the Principal Component Analysis. The Error Correction Output Code is combined with Support Vector Machine and K-Nearest Neighbor respectively as the classifiers. The macro-averaged F 1 measure is adopted to evaluate the classifiers performance. Experiment results show that, the Transfer Component Analysis is particularly beneficial for improving the classification performance, and the Support Vector Machine is more favorable than the K-Nearest Neighbor for workpiece classification. The method of the Transfer Component Analysis combined with the Support Vector Machine possesses the best classification performance among these methods in this paper. [ABSTRACT FROM AUTHOR]

Single image haze removal is an important task in computer vision. However, haze removal is an extremely challenging problem due to its massively ill-posed, which is that at each pixel we must estimate the transmission and the global atmospheric light from a single color measurement. In this paper, we propose a new deep learning-based method for removing haze from single input image. First, we estimate a transmission map via joint estimation of clear image detail and transmission map, which is different from traditional methods only estimating a transmission map for a hazy image. Second, we use a global regularization method to eliminate the halos and artifacts. Experimental results on synthetic dataset and real-world images show our method outperforms the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

It is a classical task to automatically extract road networks from very high-resolution (VHR) images in remote sensing. This paper presents a novel method for extracting road networks from VHR remotely sensed images in complex urban scenes. Inspired by image segmentation, edge detection, and object skeleton extraction, we develop a multitask convolutional neural network (CNN), called RoadNet, to simultaneously predict road surfaces, edges, and centerlines, which is the first work in such field. The RoadNet solves seven important issues in this vision problem: 1) automatically learning multiscale and multilevel features [gained by the deeply supervised nets (DSN) providing integrated direct supervision] to cope with the roads in various scenes and scales; 2) holistically training the mentioned tasks in a cascaded end-to-end CNN model; 3) correlating the predictions of road surfaces, edges, and centerlines in a network model to improve the multitask prediction; 4) designing elaborate architecture and loss function, by which the well-trained model produces approximately single-pixel width road edges/centerlines without nonmaximum suppression postprocessing; 5) cropping and bilinear blending to deal with the large VHR images with finite-computing resources; 6) introducing rough and simple user interaction to obtain desired predictions in the challenging regions; and 7) establishing a benchmark data set which consists of a series of VHR remote sensing images with pixelwise annotation. Different from the previous works, we pay more attention to the challenging situations, in which there are lots of shadows and occlusions along the road regions. Experimental results on two benchmark data sets show the superiority of our proposed approaches. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel method for creating a seamless mosaic from a set of geometrically aligned images captured from the scene with dynamic objects at different times. The artifacts caused by dynamic objects and geometric misalignments can be effectively concealed in our proposed seamline detection algorithm. In addition, we simultaneously compensate the image regions of dynamic objects based on the optimal seamline detection in the graph cuts energy minimization framework and create the mosaic with a relatively clean background. To ensure the high quality of the optimal seamline, the energy functions adopted in graph cuts combine the pixel-level similarities of image characteristics, including intensity and gradient, and the texture complexity. To successfully compensate the image regions covered by dynamic objects for creating a mosaic with a relatively clean background, we initially detect them in overlap regions between images based on pixel-level and region-level similarities, then refine them based on segments, and determine their image source in probability based on contour matching. We finally integrate all of these into the energy minimization framework to detect optimal seamlines. Experimental results on different dynamic scenes demonstrate that our proposed method is capable of generating high-quality mosaics with relatively clean backgrounds based on the detected optimal seamlines. [ABSTRACT FROM AUTHOR]

We present an efficient 3D scene representation method from a set of 3D range scans captured from a large-scale indoor or outdoor scene based on range planar segmentation and model fusion. In our method, range images are partitioned into planar patches and non-planar regions. We first partition the range image into a set of rectangle blocks and fit a planar patch to all points of each block. Blocks that are not successfully fitted as planar patches, are iteratively partitioned into sub-blocks until reaching a minimum size. Second, we iteratively merge the planar patches and identify unclassified rectangle blocks or points. The segmentation is then refined by relabelling the boundary points of fitted planar patches with respect to the neighbouring planar patches. We further simplify the scene model representation by fusing range scans and welding neighbouring planar patches with clean straight boundaries. An efficient texture mapping approach is proposed to automatically map the reflectance/colour images onto the fused scene model composed of a set of planar patches. Finally we successfully demonstrate the performance of our algorithms on several challenging range data sets. [ABSTRACT FROM AUTHOR]

The rapid development of augmented reality (AR), 3D reconstruction, simultaneous localization and mapping (SLAM), and autonomous driving requires off-the-shelf camera calibration solutions that are adaptable to cameras of different configurations in different complex scenarios. To this end, we propose a generic, robust, and accurate camera calibration framework, called Meta-Calib, by using single or multiple novel designed ArUco-encoded meta-board(s), which is dedicated to estimate accurate camera intrinsic parameters and extrinsic transformations of different multi-camera configurations. The ArUco calibration board has been redesigned to facilitate learning-based robust detection and obtain higher precision control point coordinates, which is termed the meta-board. This completely replaces the widely-used chessboard based on the corner extraction scheme to greatly alleviate the impact of image distortion on control points, especially when it is located at the boundary area of the fish-eye camera. A robust two-stage deep learning detection strategy is applied to reliably localize the ArUco-encoded inner coding region of the meta-board followed by identifying two categories of circular shapes representing "0" and "1" encoded in the ArUco pattern for decoding and orientation determination. The center points of circular shapes on the meta-board in the distorted image taken under the perspective view can be approximated through elliptical fitting with contour edges. The deviation between the fitting center points and ground-truth can be greatly suppressed when the refined sub-pixel contour edges extracted on the original image are projected to the orthographic projection view based on the camera intrinsic parameters, distortion coefficients and the prior information of the meta-board. Based on this observation, we propose a systematic iterative refinement approach to achieve the high-precision intrinsic calibration of a camera. This process involves improving the estimation of camera intrinsic parameters and fitting the center control points of circular shapes on the meta-boards in an iterative manner. The progressive nature of our approach permits reliably calibrate large distortion camera models under the presence of noisy measurements, which ensures good convergence. In addition, we also propose a graph-based multi-camera extrinsic calibration method via the corrected control points to reliably estimate both the relative poses of the meta-boards and cameras in the multi-camera system. The proposed method is not constrained by the number of cameras and meta-boards used, which makes our strategy accessible even with inflexible computer vision experts. Furthermore, we have derived the mathematical form for computing the covariance of the extrinsic transformation, which makes it possible to evaluate the uncertainty of the calibration results. Extensive experiments on a large number of both real and synthetic datasets, including perspective, fish-eye, and multiple overlapping cameras, are performed to prove the effectiveness and robustness of the developed Meta-Calib calibration framework. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) building roof reconstruction from airborne LiDAR point clouds is an important task in photogrammetry and computer vision. To automatically reconstruct the 3D building models at Level of Detail 2 (LoD-2) from airborne LiDAR point clouds, the data-driven approaches usually need to be performed in two steps: geometric primitive extraction and roof structure inference. Obviously, the traditional approaches are not end-to-end, the accumulated errors in different stages cannot be avoided and the final 3D roof models may not be optimal. In addition, the results of 3D roof models largely depend on the accuracy of geometric primitives (planes, lines, etc.). To solve these problems, we present a deep learning-based approach to directly reconstruct building roofs from airborne LiDAR point clouds, named Point2Roof. In our method, we start by extracting the deep features for each input point using PointNet++. Then, we identify a set of candidate corner points from the input point clouds using the extracted deep features. In addition, we also regress the offset for each candidate corner point to refine their locations. After that, these candidates are clustered into a set of initial vertices, and we further refine their locations to obtain the final accurate vertices. Finally, we propose a Paired Point Attention (PPA) module to predict the true model edges from an exhaustive set of candidate edges between the vertices. Unlike traditional roof modeling approaches, the proposed Point2Roof is end-to-end. However, due to the lack of a building reconstruction dataset, we construct a large-scale synthetic dataset to verify the effectiveness and robustness of the proposed Point2Roof. The experimental results conducted on the synthetic benchmark demonstrate that the proposed Point2Roof significantly outperforms the traditional roof modeling approaches. The experiments also show that the network trained on the synthetic dataset can be applied to the real point clouds after fine-tuning the trained model on a small real dataset. The large-scale synthetic dataset, the small real dataset and the source code of our approach are publicly available in https://github.com/Li-Li-Whu/Point2Roof. [ABSTRACT FROM AUTHOR]

This paper investigates the generation of large field-of-view (FOV) and high-resolution (HR) panorama images for smartphones. Existing techniques like image stitching struggle to produce satisfactory results due to geometry misalignment and inconsistent appearance. To circumvent the inherent challenges of image stitching methods and generate high-quality panoramas, we treat the image stitching problem as a multiple-reference-based super-resolution problem. Specifically, one large-FOV low-resolution (LR) image and several overlapped small-FOV HR images are taken as inputs, where the LR image acts as a base and multi-view references provide rich HR information. Building on this foundation, a novel multi-view references image super-resolution framework (MVRefSR) is proposed. Within this framework, to address the residual geometric misalignment between the LR-Ref image pairs after coarse alignment, a flow-based RefSR network (FlowSRNet) is proposed, which super-resolves LR patches with corresponding HR references. To facilitate adaptive feature fusion and minimize distortion in structured regions, a fusion weight estimation module and a gradient branch are introduced in FlowSRNet. Finally, the large-FOV HR image is generated by combining these SR patches together. Furthermore, the lack of real-world RefSR datasets for smartphones is addressed by designing an innovative dataset construction pipeline. Extensive experiments demonstrate the superior performance of FlowSRNet and MVRefSR over the compared SR methods and image stitching software, which proves the effectiveness of generating panoramas from a new perspective. • A novel multi-view references image super-resolution framework (MVRefSR) is proposed to generate the large field-of-view and high-resolution panoramas from a new perspective. • Two modules, a fusion weight estimation module and a gradient branch, are adopted to enhance the performance of our reference-based super-resolution (RefSR) methods within our MVRefSR framework. • A new RefSR dataset generation pipeline and a real-world RefSR dataset are presented to address the lack of datasets and facilitate the study of real-world RefSR. • Experimental results demonstrate the superiority of our RefSR network and our framework MVRefSR, which proves the effectiveness of generating panoramas from a new perspective. [ABSTRACT FROM AUTHOR]