Search

Three-dimensional reconstruction is a key technology employed to represent virtual reality in the real world, which is valuable in computer vision. Large-scale 3D models have broad application prospects in the fields of smart cities, navigation, virtual tourism, disaster warning, and search-and-rescue missions. Unfortunately, most image-based studies currently prioritize the speed and accuracy of 3D reconstruction in indoor scenes. While there are some studies that address large-scale scenes, there has been a lack of systematic comprehensive efforts to bring together the advancements made in the field of 3D reconstruction in large-scale scenes. Hence, this paper presents a comprehensive overview of a 3D reconstruction technique that utilizes multi-view imagery from large-scale scenes. In this article, a comprehensive summary and analysis of vision-based 3D reconstruction technology for large-scale scenes are presented. The 3D reconstruction algorithms are extensively categorized into traditional and learning-based methods. Furthermore, these methods can be categorized based on whether the sensor actively illuminates objects with light sources, resulting in two categories: active and passive methods. Two active methods, namely, structured light and laser scanning, are briefly introduced. The focus then shifts to structure from motion (SfM), stereo matching, and multi-view stereo (MVS), encompassing both traditional and learning-based approaches. Additionally, a novel approach of neural-radiance-field-based 3D reconstruction is introduced. The workflow and improvements in large-scale scenes are elaborated upon. Subsequently, some well-known datasets and evaluation metrics for various 3D reconstruction tasks are introduced. Lastly, a summary of the challenges encountered in the application of 3D reconstruction technology in large-scale outdoor scenes is provided, along with predictions for future trends in development. [ABSTRACT FROM AUTHOR]

Currently, three-dimensional (3D) point clouds are widely used in the field of remote sensing and mapping, including the measurement of living vegetation volume (LVV) in cities. However, the existing quantitative methods for LVV measurement are mainly based on single tree modeling or on the calculation of single tree species' growth parameters, which cannot be applied to the many tree species and complex forest layer structures present in urban regions, and thus are unsuitable for broad application. LVV measurement is based primarily on vegetation point cloud data, which can be obtained through many methods and often lack some information, thus posing problems in the use of traditional LVV measurement methods. To address the above problems, this paper proposes a novel LVV estimation model, which combines the voxel measurement method with an organizing point cloud based on an octree structure (we called it VO-LVV), to estimate the LVV of typical vegetation and landforms in cities. The point cloud data of single plants and multiple plants were obtained through preprocessing to verify the improvement in the calculation efficiency and accuracy of the proposed method. The results indicated that the VO-LVV estimation method, compared with the traditional method, enabled substantial efficiency improvement and higher calculation accuracy. Furthermore, the new method can be simultaneously applied to scenarios of single plants and multiple plants, and can be used for the calculation of LVV in areas with various vegetation types in cities. [ABSTRACT FROM AUTHOR]

3D reconstruction of urban scenes is an important research topic in remote sensing. Neural Radiance Fields (NeRFs) offer an efficient solution for both structure recovery and novel view synthesis. The realistic 3D urban models generated by NeRFs have potential future applications in simulation for autonomous driving, as well as in Augmented and Virtual Reality (AR/VR) experiences. Previous NeRF methods struggle with large-scale, urban environments. Due to the limited model capability of NeRF, directly applying them to urban environments may result in noticeable artifacts in synthesized images and inferior visual fidelity. To address this challenge, we propose a sparse voxel-based NeRF. First, our approach leverages LiDAR odometry to refine frame-by-frame LiDAR point cloud alignment and derive accurate initial camera pose through joint LiDAR-camera calibration. Second, we partition the space into sparse voxels and perform voxel interpolation based on 3D LiDAR point clouds, and then construct a voxel octree structure to disregard empty voxels during subsequent ray sampling in the NeRF, which can increase the rendering speed. Finally, the depth information provided by the 3D point cloud on each viewpoint image supervises our NeRF model, which is further optimized using a depth consistency loss function and a plane constraint loss function. In the real-world urban scenes, our method significantly reduces the training time to around an hour and enhances reconstruction quality with a PSNR improvement of 1–2 dB, outperforming other state-of-the-art NeRF models. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) reconstruction is an essential technique to visualize and monitor the growth of agricultural and forestry plants. However, inspecting tall plants (trees) remains a challenging task for single-camera systems. A combination of low-altitude remote sensing (an unmanned aerial vehicle) and a terrestrial capture platform (a mobile robot) is suggested to obtain the overall structural features of trees including the trunk and crown. To address the registration problem of the point clouds from different sensors, a registration method based on a fast point feature histogram (FPFH) is proposed to align the tree point clouds captured by terrestrial and airborne sensors. Normal vectors are extracted to define a Darboux coordinate frame whereby FPFH is calculated. The initial correspondences of point cloud pairs are calculated according to the Bhattacharyya distance. Reliable matching point pairs are then selected via random sample consensus. Finally, the 3D transformation is solved by singular value decomposition. For verification, experiments are conducted with real-world data. In the registration experiment on noisy and partial data, the root-mean-square error of the proposed method is 0.35% and 1.18% of SAC-IA and SAC-IA + ICP, respectively. The proposed method is useful for the extraction, monitoring, and analysis of plant phenotypes. [ABSTRACT FROM AUTHOR]

Urban environments are regions in which spectral variability and spatial variability are extremely high, with a huge range of shapes and sizes, and they also demand high resolution images for applications involving their study. Due to the fact that these environments can grow even more over time, applications related to their monitoring tend to turn to autonomous intelligent systems, which together with remote sensing data could help or even predict daily life situations. The task of mapping cities by autonomous operators was usually carried out by aerial optical images due to its scale and resolution; however new scientific questions have arisen, and this has led research into a new era of highly-detailed data extraction. For many years, using artificial neural models to solve complex problems such as automatic image classification was commonplace, owing much of their popularity to their ability to adapt to complex situations without needing human intervention. In spite of that, their popularity declined in the mid-2000s, mostly due to the complex and time-consuming nature of their methods and workflows. However, newer neural network architectures have brought back the interest in their application for autonomous classifiers, especially for image classification purposes. Convolutional Neural Networks (CNN) have been a trend for pixel-wise image segmentation, showing flexibility when detecting and classifying any kind of object, even in situations where humans failed to perceive differences, such as in city scenarios. In this paper, we aim to explore and experiment with state-of-the-art technologies to semantically label 3D urban models over complex scenarios. To achieve these goals, we split the problem into two main processing lines: first, how to correctly label the façade features in the 2D domain, where a supervised CNN is used to segment ground-based façade images into six feature classes, roof, window, wall, door, balcony and shop; second, a Structure-from-Motion (SfM) and Multi-View-Stereo (MVS) workflow is used to extract the geometry of the façade, wherein the segmented images in the previous stage are then used to label the generated mesh by a "reverse" ray-tracing technique. This paper demonstrates that the proposed methodology is robust in complex scenarios. The façade feature inferences have reached up to 93% accuracy over most of the datasets used. Although it still presents some deficiencies in unknown architectural styles and needs some improvements to be made regarding 3D-labeling, we present a consistent and simple methodology to handle the problem. [ABSTRACT FROM AUTHOR]

In recent years, the application of geomatics tools in archaeology has proved to be very useful to obtain meaningful knowledge of the 3D reconstruction of archaeological remains and semantic classification of the 3D surface. These techniques have proven to be an effective solution for the 3D modeling and the extraction of many spatial features on an archaeological site. However, novel methodologies as well as new data exploitation strategies are required to exploit these geospatial data for natural and cultural heritage documentation, monitoring, and preservation. In this paper, we have studied unique archaeological ruins, a Mozarab church in Al-Andalus, using high-resolution RGB images, which was taken by a drone. Thus, a 3D reconstruction of the ruins and the surrounding environment is carried out in order to characterize it on a dense point cloud. Then, a digital elevation model (DEM) was calculated in order to identify critical slope lines, which are significant to determine where the structure of the church was built. Our results can be used for the development of an architectural project and thus a virtual recreation of these archaeological ruins was performed. [ABSTRACT FROM AUTHOR]

The importance of three-dimensional (3D) point cloud technologies in the field of agriculture environmental research has increased in recent years. Obtaining dense and accurate 3D reconstructions of plants and urban areas provide useful information for remote sensing. In this paper, we propose a novel strategy for the enhancement of 3D point clouds from a single 4D light field (LF) image. Using a light field camera in this way creates an easy way for obtaining 3D point clouds from one snapshot and enabling diversity in monitoring and modelling applications for remote sensing. Considering an LF image and associated depth map as an input, we first apply histogram equalization and histogram stretching to enhance the separation between depth planes. We then apply multi-modal edge detection by using feature matching and fuzzy logic from the central sub-aperture LF image and the depth map. These two steps of depth map enhancement are significant parts of our novelty for this work. After combing the two previous steps and transforming the point–plane correspondence, we can obtain the 3D point cloud. We tested our method with synthetic and real world image databases. To verify the accuracy of our method, we compared our results with two different state-of-the-art algorithms. The results showed that our method can reliably mitigate noise and had the highest level of detail compared to other existing methods. [ABSTRACT FROM AUTHOR]

With extensive applications of Unmanned Aircraft Vehicle (UAV) in the field of remote sensing, 3D reconstruction using aerial images has been a vibrant area of research. However, fast large-scale 3D reconstruction is a challenging task. For aerial image datasets, large scale means that the number and resolution of images are enormous, which brings significant computational cost to the 3D reconstruction, especially in the process of Structure from Motion (SfM). In this paper, for fast large-scale SfM, we propose a clustering-aligning framework that hierarchically merges partial structures to reconstruct the full scene. Through image clustering, an overlapping relationship between image subsets is established. With the overlapping relationship, we propose a similarity transformation estimation method based on joint camera poses of common images. Finally, we introduce the closed-loop constraint and propose a similarity transformation-based hybrid optimization method to make the merged complete scene seamless. The advantage of the proposed method is a significant efficiency improvement without a marginal loss in accuracy. Experimental results on the Qinling dataset captured over Qinling mountain covering 57 square kilometers demonstrate the efficiency and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Accurate 3D reconstruction/modelling from unmanned aerial vehicle (UAV)-based imagery has become the key prerequisite in various applications. Although current commercial software has automated the process of image-based reconstruction, a transparent system, which can be incorporated with different user-defined constraints, is still preferred by the photogrammetric research community. In this regard, this paper presents a transparent framework for the automated aerial triangulation of UAV images. The proposed framework is conducted in three steps. In the first step, two approaches, which take advantage of prior information regarding the flight trajectory, are implemented for reliable relative orientation recovery. Then, initial recovery of image exterior orientation parameters (EOPs) is achieved through either an incremental or global approach. Finally, a global bundle adjustment involving Ground Control Points (GCPs) and check points is carried out to refine all estimated parameters in the defined mapping coordinate system. Four real image datasets, which are acquired by two different UAV platforms, have been utilized to evaluate the feasibility of the proposed framework. In addition, a comparative analysis between the proposed framework and the existing commercial software is performed. The derived experimental results demonstrate the superior performance of the proposed framework in providing an accurate 3D model, especially when dealing with acquired UAV images containing repetitive pattern and significant image distortions. [ABSTRACT FROM AUTHOR]

The shift to the early Holocene in northern Europe is strongly associated with major environmental and climatic changes that influenced hunter-gatherers' activities and occupation during the Mesolithic period. The ancient lake Duvensee (10,000–6500 cal. BCE) has been studied for almost a century, providing archaeological sites consisting of bark mats and hazelnut-roasting hearths situated on small sand banks deposited by the glacier. No method is yet available to locate these features before excavation. Therefore, a key method for understanding the living conditions of hunter-gatherer groups is to reconstruct the paleoenvironment with a focus on the identification of areas that could possibly host Mesolithic camps and well-preserved archaeological artefacts. We performed a 16-channel MALÅ Imaging Radar Array (MIRA) system survey aimed at understanding the landscape surrounding the find spot Duvensee WP10, located in a hitherto uninvestigated part of the bog. Using an integrated approach of high-resolution ground radar mapping and targeted excavations enabled us to derive a 3D spatio-temporal landscape reconstruction of the investigated sector, including paleo-bathymetry, stratigraphy, and shorelines around the Mesolithic camps. Additionally, we detected previously unknown islands as potential areas for yet unknown dwelling sites. We found that the growth rates of the islands were in the order of approximately 0.3 m2/yr to 0.7 m2/yr between the late Preboreal and the Subboreal stages. The ground-penetrating radar surveying performed excellently in all aspects of near-surface landscape reconstruction as well as in identifying potential dwellings; however, the direct identification of small-scale artefacts, such as fireplaces, was not successful because of their similarity to natural structures. [ABSTRACT FROM AUTHOR]

Remote sensing data provide a huge number of sea surface observations, but cannot give direct information on deeper ocean layers, which can only be provided by sparse in situ data. The combination of measurements collected by satellite and in situ sensors represents one of the most effective strategies to improve our knowledge of the interior structure of the ocean ecosystems. In this work, we describe a Multi-Layer-Perceptron (MLP) network designed to reconstruct the 3D fields of ocean temperature and chlorophyll-a concentration, two variables of primary importance for many upper-ocean bio-physical processes. Artificial neural networks can efficiently model eventual non-linear relationships among input variables, and the choice of the predictors is thus crucial to build an accurate model. Here, concurrent temperature and chlorophyll-a in situ profiles and several different combinations of satellite-derived surface predictors are used to identify the optimal model configuration, focusing on the Mediterranean Sea. The lowest errors are obtained when taking in input surface chlorophyll-a, temperature, and altimeter-derived absolute dynamic topography and surface geostrophic velocity components. Network training and test validations give comparable results, significantly improving with respect to Mediterranean climatological data (MEDATLAS). 3D fields are then also reconstructed from full basin 2D satellite monthly climatologies (1998–2015) and resulting 3D seasonal patterns are analyzed. The method accurately infers the vertical shape of temperature and chlorophyll-a profiles and their spatial and temporal variability. It thus represents an effective tool to overcome the in-situ data sparseness and the limits of satellite observations, also potentially suitable for the initialization and validation of bio-geophysical models. [ABSTRACT FROM AUTHOR]

Generating 3D point clouds from a single image has attracted full attention from researchers in the field of multimedia, remote sensing and computer vision. With the recent proliferation of deep learning, various deep models have been proposed for the 3D point cloud generation. However, they require objects to be captured with absolutely clean backgrounds and fixed viewpoints, which highly limits their application in the real environment. To guide 3D point cloud generation, we propose a novel network, RealPoint3D, to integrate prior 3D shape knowledge into the network. Taking additional 3D information, RealPoint3D can handle 3D object generation from a single real image captured from any viewpoint and complex background. Specifically, provided a query image, we retrieve the nearest shape model from a pre-prepared 3D model database. Then, the image, together with the retrieved shape model, is fed into RealPoint3D to generate a fine-grained 3D point cloud. We evaluated the proposed RealPoint3D on the ShapeNet dataset and ObjectNet3D dataset for the 3D point cloud generation. Experimental results and comparisons with state-of-the-art methods demonstrate that our framework achieves superior performance. Furthermore, our proposed framework works well for real images in complex backgrounds (the image has the remaining objects in addition to the reconstructed object, and the reconstructed object may be occluded or truncated) with various viewing angles. [ABSTRACT FROM AUTHOR]

Geometric three-dimensional (3D) reconstruction has emerged as a powerful tool for plant phenotyping and plant breeding. Although laser scanning is one of the most intensely used sensing techniques for 3D reconstruction projects, it still has many limitations, such as the high investment cost. To overcome such limitations, in the present study, a low-cost, novel, and efficient imaging system consisting of a red-green-blue (RGB) camera and a photonic mixer detector (PMD) was developed, and its usability for plant phenotyping was demonstrated via a 3D reconstruction of a soybean plant that contains color information. To reconstruct soybean canopies, a density-based spatial clustering of applications with noise (DBSCAN) algorithm was used to extract canopy information from the raw 3D point cloud. Principal component analysis (PCA) and iterative closest point (ICP) algorithms were then used to register the multisource images for the 3D reconstruction of a soybean plant from both the side and top views. We then assessed phenotypic traits such as plant height and the greenness index based on the deviations of test samples. The results showed that compared with manual measurements, the side view-based assessments yielded a determination coefficient (R2) of 0.9890 for the estimation of soybean height and a R2 of 0.6059 for the estimation of soybean canopy greenness index; the top view-based assessment yielded a R2 of 0.9936 for the estimation of soybean height and a R2 of 0.8864 for the estimation of soybean canopy greenness. Together, the results indicated that an assembled 3D imaging device applying the algorithms developed in this study could be used as a reliable and robust platform for plant phenotyping, and potentially for automated and high-throughput applications under both natural light and indoor conditions. [ABSTRACT FROM AUTHOR]

Many studies have been focusing on reconstructing the branch skeleton of a three-dimensional (3D) tree structure that is based on photos or point clouds scanned by a terrestrial laser scanner (TLS), but leaves, as the important component of a tree, are often ignored or simplified because of their complexity. Therefore, we develop a voxel-based method to add leaves to a reconstructed 3D branches structure based on TLS point clouds. The location and size of each leaf depend on the spatial distribution and density of leaves points in the voxel. We reconstruct a small 3D scene with four realistic 3D trees and a virtual tree (including trunk, branches, and leaves), and validate the structure of each tree through the directional gap fractions calculated based on simulated point clouds of this reconstructed scene by the ray-tracing algorithm. The results show good coherence with those from measured point clouds data. The relative errors of the directional gap fractions are no more than 4.1%, though the method is limited by the effects of point occlusion. Therefore, this method is shown to give satisfactory consistency both visually and in the quantitative evaluation of the 3D structure. [ABSTRACT FROM AUTHOR]