Search

The rapid development of augmented reality (AR), 3D reconstruction, simultaneous localization and mapping (SLAM), and autonomous driving requires off-the-shelf camera calibration solutions that are adaptable to cameras of different configurations in different complex scenarios. To this end, we propose a generic, robust, and accurate camera calibration framework, called Meta-Calib, by using single or multiple novel designed ArUco-encoded meta-board(s), which is dedicated to estimate accurate camera intrinsic parameters and extrinsic transformations of different multi-camera configurations. The ArUco calibration board has been redesigned to facilitate learning-based robust detection and obtain higher precision control point coordinates, which is termed the meta-board. This completely replaces the widely-used chessboard based on the corner extraction scheme to greatly alleviate the impact of image distortion on control points, especially when it is located at the boundary area of the fish-eye camera. A robust two-stage deep learning detection strategy is applied to reliably localize the ArUco-encoded inner coding region of the meta-board followed by identifying two categories of circular shapes representing "0" and "1" encoded in the ArUco pattern for decoding and orientation determination. The center points of circular shapes on the meta-board in the distorted image taken under the perspective view can be approximated through elliptical fitting with contour edges. The deviation between the fitting center points and ground-truth can be greatly suppressed when the refined sub-pixel contour edges extracted on the original image are projected to the orthographic projection view based on the camera intrinsic parameters, distortion coefficients and the prior information of the meta-board. Based on this observation, we propose a systematic iterative refinement approach to achieve the high-precision intrinsic calibration of a camera. This process involves improving the estimation of camera intrinsic parameters and fitting the center control points of circular shapes on the meta-boards in an iterative manner. The progressive nature of our approach permits reliably calibrate large distortion camera models under the presence of noisy measurements, which ensures good convergence. In addition, we also propose a graph-based multi-camera extrinsic calibration method via the corrected control points to reliably estimate both the relative poses of the meta-boards and cameras in the multi-camera system. The proposed method is not constrained by the number of cameras and meta-boards used, which makes our strategy accessible even with inflexible computer vision experts. Furthermore, we have derived the mathematical form for computing the covariance of the extrinsic transformation, which makes it possible to evaluate the uncertainty of the calibration results. Extensive experiments on a large number of both real and synthetic datasets, including perspective, fish-eye, and multiple overlapping cameras, are performed to prove the effectiveness and robustness of the developed Meta-Calib calibration framework. [ABSTRACT FROM AUTHOR]

Background: Freezing injury is a devastating yet common damage that occurs to winter rapeseed during the overwintering period which directly reduces the yield and causes heavy economic loss. Thus, it is an important and urgent task for crop breeders to find the freezing-tolerant rapeseed materials in the process of breeding. Existing large-scale freezing-tolerant rapeseed material recognition methods mainly rely on the field investigation conducted by the agricultural experts using some professional equipments. These methods are time-consuming, inefficient and laborious. In addition, the accuracy of these traditional methods depends heavily on the knowledge and experience of the experts. Methods: To solve these problems of existing methods, we propose a low-cost freezing-tolerant rapeseed material recognition approach using deep learning and unmanned aerial vehicle (UAV) images captured by a consumer UAV. We formulate the problem of freezing-tolerant material recognition as a binary classification problem, which can be solved well using deep learning. The proposed method can automatically and efficiently recognize the freezing-tolerant rapeseed materials from a large number of crop candidates. To train the deep learning network, we first manually construct the real dataset using the UAV images of rapeseed materials captured by the DJI Phantom 4 Pro V2.0. Then, five classic deep learning networks (AlexNet, VGGNet16, ResNet18, ResNet50 and GoogLeNet) are selected to perform the freezing-tolerant rapeseed material recognition. Result and conclusion: The accuracy of the five deep learning networks used in our work is all over 92%. Especially, ResNet50 provides the best accuracy (93.33 % ) in this task. In addition, we also compare deep learning networks with traditional machine learning methods. The comparison results show that the deep learning-based methods significantly outperform the traditional machine learning-based methods in our task. The experimental results show that it is feasible to recognize the freezing-tolerant rapeseed using UAV images and deep learning. [ABSTRACT FROM AUTHOR]

Background: The flowering period is a critical time for the growth of rape plants. Counting rape flower clusters can help farmers to predict the yield information of the corresponding rape fields. However, counting in-field is a time-consuming and labor-intensive task. To address this, we explored a deep learning counting method based on unmanned aircraft vehicle (UAV). The proposed method developed the in-field counting of rape flower clusters as a density estimation problem. It is different from the object detection method of counting the bounding boxes. The crucial step of the density map estimation using deep learning is to train a deep neural network that maps from an input image to the corresponding annotated density map. Results: We explored a rape flower cluster counting network series: RapeNet and RapeNet+. A rectangular box labeling-based rape flower clusters dataset (RFRB) and a centroid labeling-based rape flower clusters dataset (RFCP) were used for network model training. To verify the performance of RapeNet series, the paper compares the counting result with the real values of manual annotation. The average accuracy (Acc), relative root mean square error (rrMSE) and R 2 of the metrics are up to 0.9062, 12.03 and 0.9635 on the dataset RFRB, and 0.9538, 5.61 and 0.9826 on the dataset RFCP, respectively. The resolution has little influence for the proposed model. In addition, the visualization results have some interpretability. Conclusions: Extensive experimental results demonstrate that the RapeNet series outperforms other state-of-the-art counting approaches. The proposed method provides an important technical support for the crop counting statistics of rape flower clusters in field. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) building roof reconstruction from airborne LiDAR point clouds is an important task in photogrammetry and computer vision. To automatically reconstruct the 3D building models at Level of Detail 2 (LoD-2) from airborne LiDAR point clouds, the data-driven approaches usually need to be performed in two steps: geometric primitive extraction and roof structure inference. Obviously, the traditional approaches are not end-to-end, the accumulated errors in different stages cannot be avoided and the final 3D roof models may not be optimal. In addition, the results of 3D roof models largely depend on the accuracy of geometric primitives (planes, lines, etc.). To solve these problems, we present a deep learning-based approach to directly reconstruct building roofs from airborne LiDAR point clouds, named Point2Roof. In our method, we start by extracting the deep features for each input point using PointNet++. Then, we identify a set of candidate corner points from the input point clouds using the extracted deep features. In addition, we also regress the offset for each candidate corner point to refine their locations. After that, these candidates are clustered into a set of initial vertices, and we further refine their locations to obtain the final accurate vertices. Finally, we propose a Paired Point Attention (PPA) module to predict the true model edges from an exhaustive set of candidate edges between the vertices. Unlike traditional roof modeling approaches, the proposed Point2Roof is end-to-end. However, due to the lack of a building reconstruction dataset, we construct a large-scale synthetic dataset to verify the effectiveness and robustness of the proposed Point2Roof. The experimental results conducted on the synthetic benchmark demonstrate that the proposed Point2Roof significantly outperforms the traditional roof modeling approaches. The experiments also show that the network trained on the synthetic dataset can be applied to the real point clouds after fine-tuning the trained model on a small real dataset. The large-scale synthetic dataset, the small real dataset and the source code of our approach are publicly available in https://github.com/Li-Li-Whu/Point2Roof. [ABSTRACT FROM AUTHOR]

Multi-focus image fusion uses multiple images focused on different depths to generate a clear image covering the whole scene. The existing multi-focus image fusion methods do not consider the defocus successive variation in close-range photography and the camera shaking in sequence image shooting, and most methods cannot process multiple images simultaneously. We proposed an end-to-end deep learning network to generate the all-in-focus image from multi-focus and misaligned images to solve these problems. Specifically, taking multiple multi-focus and misaligned source images as input, our Defocus and Similarity Attention Fusion Network (DSAF-Net) first generates the corresponding defocus map through the Defocus-Net, and warps the source images and defocus maps to a unified camera view according to the optical flow estimated by the OpticalFlow-Net. Finally, our model applies a coarse-to-fine correspondence matching scheme to obtain the similarity weight maps, combined with the warped source images and defocus maps to fuse into a clear image. For training and testing our DSAF-Net, a multi-focus and misaligned cultural heritage photography dataset (WHU-MFM) is constructed using Blender Cycles renderer. Experiments results demonstrate that our method outperforms the state-of-the-art methods both qualitatively and quantitatively. DSAF-Net is available at https://github.com/PeimingCHEN/DSAF-Net. And WHU-MFM is available at https://github.com/PeimingCHEN/WHU-MFM-Dataset. • Proposed an end-to-end neural network based on defocus and similarity attention. • It can fuse several multi-focus and misaligned images simultaneously into a clear image. • Designed an attentional aggregation module for reducing pixel collisions and holes. • Built a first-of-its-kind multi-focus and misaligned close-range photography dataset. [ABSTRACT FROM AUTHOR]

The license plate recognition is an important part of the intelligent traffic management system, and the application of deep learning to the license plate recognition system can effectively improve the speed and accuracy of recognition. Aiming at the problems of traditional license plate recognition algorithms such as the low accuracy, slow speed, and the recognition rate being easily affected by the environment, a Convolutional Neural Network- (CNN-) based license plate recognition algorithm-Fast-LPRNet is proposed. This algorithm uses the nonsegment recognition method, removes the fully connected layer, and reduces the number of parameters. The algorithm—which has strong generalization ability, scalability, and robustness—performs license plate recognition on the FPGA hardware. Increaseing the depth of network on the basis of the Fast-LPRNet structure, the dataset of Chinese City Parking Dataset (CCPD) can be recognized with an accuracy beyond 90%. The experimental results show that the license plate recognition algorithm has high recognition accuracy, strong generalization ability, and good robustness. [ABSTRACT FROM AUTHOR]