Search

High-quality agricultural multi-view stereo reconstruction technology is the key to precision and informatization in agriculture. Multi-view stereo reconstruction methods are an important part of 3D vision technology. In the multi-view stereo 3D reconstruction method based on deep learning, the effect of feature extraction directly affects the accuracy of reconstruction. Aiming at the actual problems in orchard fruit tree reconstruction, this paper designs an improved multi-view stereo structure based on the combination of remote sensing and artificial intelligence to realize the accurate reconstruction of jujube tree trunks. Firstly, an automatic key frame extraction method is proposed for the DSST target tracking algorithm to quickly recognize and extract high-quality data. Secondly, a composite U-Net feature extraction network is designed to enhance the reconstruction accuracy, while the DRE-Net feature extraction enhancement network improved by the parallel self-attention mechanism enhances the reconstruction completeness. Comparison tests show different levels of improvement on the Technical University of Denmark (DTU) dataset compared to other deep learning-based methods. Ablation test on the self-constructed dataset, the MVSNet + Co U-Net + DRE-Net_SA method proposed in this paper improves 20.4% in Accuracy, 12.8% in Completion, and 16.8% in Overall compared to the base model, which verifies the real effectiveness of the scheme.

The image is a particularly valuable data carrier in medical forensic and forensic analyses. One of the analyses, as mentioned above, is to assess whether a graphically captured object is the same object examined in reality. This is a complicated process due to perspective foreshortening, making it difficult to determine the scale and proportion of objects in the frame, as well as the subsequent correct reading of their actual measurements. This paper presented a method for the 3D reconstruction of silhouettes of people recorded in a photo or video, with the aim of identifying these people through subsequent comparative studies. The authors presented an algorithm for dealing with graphic evidence, using the example of the analysis of spatial correlation of the silhouette of the perpetrator of the actual event (recorded via CCTV footage) with the silhouette of the suspect (scanned in 3D in custody). In this paper, the authors posed the thesis that the isometric (devoid of perspective foreshortening) display mode that 3D platforms offer, and the animation of the figure to the desired identical poses, provides the possibility of not only obtaining linear measurements of the person but also of orthophotographic visualization of body proportions, allowing their comparison with another silhouette, which is difficult to achieve in perspective view of the studied image.

This paper presents an autonomous grasping approach for complex-shaped objects using an anthropomorphic robotic hand. Although human-like robotic hands have a number of distinctive advantages, most of the current autonomous robotic pickup systems still use relatively simple gripper setups such as a two-finger gripper or even a suction gripper. The main difficulty of utilizing human-like robotic hands lies in the sheer complexity of the system; it is inherently tough to plan and control the motions of the high degree of freedom (DOF) system. Although data-driven approaches have been successfully used for motion planning of various robotic systems recently, it is hard to directly apply them to high-DOF systems due to the difficulty of acquiring training data. In this paper, we propose a novel approach for grasping complex-shaped objects using a high-DOF robotic manipulation system consisting of a seven-DOF manipulator and a four-fingered robotic hand with 16 DOFs. Human demonstration data are first acquired using a virtual reality controller with 6D pose tracking and individual capacitive finger sensors. Then, the 3D shape of the manipulation target object is reconstructed from multiple depth images recorded using the wrist-mounted RGBD camera. The grasping pose for the object is estimated using a residual neural network (ResNet), K-means clustering (KNN), and a point-set registration algorithm. Then, the manipulator moves to the grasping pose following the trajectory created by dynamic movement primitives (DMPs). Finally, the robot performs one of the object-specific grasping motions learned from human demonstration. The suggested system is evaluated by an official tester using five objects with promising results.

Potholes, a kind of road defect, can damage vehicles and negatively affect drivers’ safe driving, and in severe cases can lead to traffic accidents. Efficient and preventive management of potholes in a complex road environment plays an important role in securing driver safety. It is also expected to contribute to the prevention of traffic accidents and the smooth flow of traffic. In the past, pothole detection was mainly performed via visual inspection by human experts. Recently, automated pothole-detection methods apply various technologies that converge basic technologies such as sensors and signal processing. The automated pothole-detection methods can be classified into three types according to the technology used in the pothole-recognition process: a vision-based method, a vibration-based method, and a 3D reconstruction-based method. In this paper, three methods are compared, and the strengths and weaknesses of each method are summarized. The detection process and technology proposed in the latest research related to automated pothole detection are described for each method. The development plans of future technology that is connected with those studies are also presented in this paper.

This paper introduces a real-time 3D reconstruction of a human captured using a depth sensor and has integrated it with a holographic telepresence application. Holographic projection is widely recognized as one of the most promising 3D display technologies, and it is expected to become more widely available in the near future. This technology may also be deployed in various ways, including holographic prisms and Z-Hologram, which this research has used to demonstrate the initial results by displaying the reconstructed 3D representation of the user. The realization of a stable and inexpensive 3D data acquisition system is a problem that has yet to be solved. When we involve multiple sensors we need to compress and optimize the data so that it can be sent to a server for a telepresence. Therefore the paper presents the processes in real-time 3D reconstruction, which consists of data acquisition, background removal, point cloud extraction, and a surface generation which applies a marching cube algorithm to finally form an isosurface from the set of points in the point cloud which later texture mapping is applied on the isosurface generated. The compression results has been presented in this paper, and the results of the integration process after sending the data over the network also have been discussed.

Three-dimensional reconstruction based on optical satellite images has always been a research hotspot in the field of photogrammetry. In particular, the 3D reconstruction of building areas has provided great help for urban planning, change detection and emergency response. The results of 3D reconstruction of satellite images are greatly affected by the input images, and this paper proposes an improvement method for 3D reconstruction of satellite images based on the generative adversarial network (GAN) image enhancement. In this method, the perceptual loss function is used to optimize the network, so that it can output high-definition satellite images for 3D reconstruction, so as to improve the completeness and accuracy of the reconstructed 3D model. We use the public benchmark dataset of satellite images to test the feasibility and effectiveness of the proposed method. The experiments show that compared with the satellite stereo pipeline (S2P) method and the bundle adjustment (BA) method, the proposed method can automatically reconstruct high-quality 3D point clouds.

The identification of a person is a natural way to gain access to information or places. A face image is an essential element of visual validation. In this paper, we present the Card3DFace application, which captures a single-shot image of a person’s face. After reconstructing the 3D model of the head, the application generates several images from different perspectives, which, when printed on a card with a layer of lenticular lenses, produce a 3D visualization effect of the face. The image acquisition is achieved with a regular consumer 3D camera, either using plenoptic, stereo or time-of-flight technologies. This procedure aims to assist and improve the human visual recognition of ID cards and travel documents through an affordable and fast process while simultaneously increasing their security level. The whole system pipeline is analyzed and detailed in this paper. The results of the experiments performed with polycarbonate ID cards show that this end-to-end system is able to produce cards with realistic 3D visualization effects for humans.

In the era of Industry 4.0, digital twin technology is revolutionizing traditional manufacturing paradigms. However, the adoption of this technology in modern manufacturing systems is fraught with challenges due to the scarcity of labeled data. Specifically, existing supervised machine learning algorithms, with their reliance on voluminous training data, find their applicability constrained in real-world production settings. This paper introduces an unsupervised 3D reconstruction approach tailored for industrial applications, aimed at bridging the data void in creating digital twin models. Our proposed model, by ingesting high-resolution 2D images, autonomously reconstructs precise 3D digital twin models without the need for manual annotations or prior knowledge. Through comparisons with multiple baseline models, we demonstrate the superiority of our method in terms of accuracy, speed, and generalization capabilities. This research not only offers an efficient approach to industrial 3D reconstruction but also paves the way for the widespread adoption of digital twin technology in manufacturing.

An artificial compound eye consists of multiple apertures that allow for a large field of view (FOV) while maintaining a small size. Each aperture captures a sub-image, and multiple sub-images are needed to reconstruct the full FOV. The reconstruction process is depth-related due to the parallax between adjacent apertures. This paper presents an all-in-focus 3D reconstruction method for a specific type of artificial compound eye called the electronic cluster eye (eCley). The proposed method uses edge matching to address the edge blur and large textureless areas existing in the sub-images. First, edges are extracted from each sub-image, and then a matching operator is applied to match the edges based on their shape context and intensity. This produces a sparse matching result that is then propagated to the whole image. Next, a depth consistency check and refinement method is performed to refine the depth of all sub-images. Finally, the sub-images and depth maps are merged to produce the final all-in-focus image and depth map. The experimental results and comparative analysis demonstrate the effectiveness of the proposed method.

SLAM (Simultaneous Localization and Mapping), primarily relying on camera or LiDAR (Light Detection and Ranging) sensors, plays a crucial role in robotics for localization and environmental reconstruction. This paper assesses the performance of two leading methods, namely ORB-SLAM3 and SC-LeGO-LOAM, focusing on localization and mapping in both indoor and outdoor environments. The evaluation employs artificial and cost-effective datasets incorporating data from a 3D LiDAR and an RGB-D (color and depth) camera. A practical approach is introduced for calculating ground-truth trajectories and during benchmarking, reconstruction maps based on ground truth are established. To assess the performance, ATE and RPE are utilized to evaluate the accuracy of localization; standard deviation is employed to compare the stability during the localization process for different methods. While both algorithms exhibit satisfactory positioning accuracy, their performance is suboptimal in scenarios with inadequate textures. Furthermore, 3D reconstruction maps established by the two approaches are also provided for direct observation of their differences and the limitations encountered during map construction. Moreover, the research includes a comprehensive comparison of computational performance metrics, encompassing Central Processing Unit (CPU) utilization, memory usage, and an in-depth analysis. This evaluation revealed that Visual SLAM requires more CPU resources than LiDAR SLAM, primarily due to additional data storage requirements, emphasizing the impact of environmental factors on resource requirements. In conclusion, LiDAR SLAM is more suitable for the outdoors due to its comprehensive nature, while Visual SLAM excels indoors, compensating for sparse aspects in LiDAR SLAM. To facilitate further research, a technical guide was also provided for the researchers in related fields.

Chronic wounds or wounds that are not healing properly are a worldwide health problem that affect the global economy and population. Alongside with aging of the population, increasing obesity and diabetes patients, we can assume that costs of chronic wound healing will be even higher. Wound assessment should be fast and accurate in order to reduce the possible complications, and therefore shorten the wound healing process. Contact methods often used by medical experts have drawbacks that are easily overcome by non-contact methods like image analysis, where wound analysis is fully or partially automated. Two major tasks in wound analysis on images are segmentation of the wound from the healthy skin and background, and classification of the most important wound tissues like granulation, fibrin, and necrosis. These tasks are necessary for further assessment like wound measurement or healing evaluation based on tissue representation. Researchers use various methods and algorithms for image wound analysis with the aim to outperform accuracy rates and show the robustness of the proposed methods. Recently, neural networks and deep learning algorithms have driven considerable performance improvement across various fields, which has a led to a significant rise of research papers in the field of wound analysis as well. The aim of this paper is to provide an overview of recent methods for non-contact wound analysis which could be used for developing an end-to-end solution for a fully automated wound analysis system which would incorporate all stages from data acquisition, to segmentation and classification, ending with measurement and healing evaluation.

This work proposed a method to reconstruct the 3D bubble shape in a transparent medium utilizing the three orthographic digital images. The bubble was divided into several ellipse slices. The azimuth angle and projection parameters were extracted from the top view image, while the formulas for dimensionless semi-axes were derived according to the geometric projection relationship. The elliptical axes of each layer were calculated by substituting the projection width into the formulas. All layers of slices were stacked to form the 3D bubble shape. Reconstruction accuracy was evaluated with spheres, ellipsoids, and inverted teardrops. The results show that the position contributes greatly to the reconstruction accuracy of the bubbles with serious horizontal deformation. The method in Bian et al. (2013) is sensitive to both horizontal and vertical deformations. The vertical deformation has little influence on the method in Fujiwara et al. (2004), whereas the horizontal deformation greatly impacts its accuracy. The method in this paper is negligibly affected by vertical deformation, but it does better in reconstructing single bubbles with large horizontal deformation. The azimuth angle affects the accuracy of the methods in Bian et al. (2013) and Fujiwara et al. (2004) more than the method in this paper.

The prevalence of lower back pain and radicular pain in the population requires more and more accurate diagnostic methods to more effectively prevent and treat patients with these ailments. In this paper, we focused on one of the causes of lower back pain and radicular pain—lumbar foraminal stenosis (LFS). The aim of the study is to assess the morphometry of the intervertebral canals in the lumbar spine at the levels of the L4/L5 and L5/S1 motor segments. The obtained results showed correlations between the circumference and the surface area on individual cross-sections of the intervertebral canals at the L4/L5 and L5/S1 levels and determined the approximate shape of the root canal and its variability. On this basis, we were able to determine the influences of the patient’s age and sex on the morphometric parameters of the intervertebral canals at the L4/L5 and L5/S1 levels. Further research is needed in this area, taking into account additional factors influencing the shape of intervertebral canals.

In urban underground projects, such as urban drainage systems, the real-time acquisition and generation of 3D models of pipes can provide an important foundation for pipe safety inspection and maintenance. The simultaneous localization and mapping (SLAM) technique, compared to the traditional structure from motion (SfM) reconstruction technique, offers high real-time performance and improves the efficiency of 3D object reconstruction. Underground pipes are situated in complex environments with unattended individuals and often lack natural lighting. To address this, this paper presents a real-time and cost-effective 3D perception and reconstruction system that utilizes an unmanned aerial vehicle (UAV) equipped with Intel RealSense D435 depth cameras and an artificial light-supplementation device. This system carries out real-time 3D reconstruction of underground pipes using the RTAB-Map (real-time appearance-based mapping) method. RTAB-Map is a graph-based visual SLAM method that combines closed-loop detection and graph optimization algorithms. The unique memory management mechanism of RTAB-Map enables synchronous mapping for multiple sessions during UAV flight. Experimental results demonstrate that the proposed system, based on RTAB-Map, exhibits the robustness, textures, and feasibility for 3D reconstruction of underground pipes.

Euler’s elastica energy regularizer, initially employed in mathematical and physical systems, has recently garnered much attention in image processing and computer vision tasks. Due to the non-convexity, non-smoothness, and high order of its derivative, however, the term has yet to be effectively applied in 3D reconstruction. To this day, the industry is still searching for 3D reconstruction systems that are robust, accurate, efficient, and easy to use. While implicit surface reconstruction methods generally demonstrate superior robustness and flexibility, the traditional methods rely on initialization and can easily become trapped in local minima. Some low-order variational models are able to overcome these issues, but they still struggle with the reconstruction of object details. Euler’s elastica term, on the other hand, has been found to share the advantages of both the TV regularization term and the curvature regularization term. In this paper, we aim to address the problems of missing details and complex computation in implicit 3D reconstruction by efficiently using Euler’s elastica term. The main contributions of this article can be outlined in three aspects. Firstly, Euler’s elastica is introduced as a regularization term in 3D point cloud reconstruction. Secondly, a new dual algorithm is devised for the proposed model, significantly improving solution efficiency compared to the commonly used TV model. Lastly, numerical experiments conducted in 2D and 3D demonstrate the remarkable performance of Euler’s elastica in enhancing features of curved surfaces during point cloud reconstruction. The reconstructed point cloud surface adheres more closely to the initial point cloud surface when compared to the classical TV model. However, it is worth noting that Euler’s elastica exhibits a lesser capability in handling local extrema compared to the TV model.

Turbine blades are crucial components exposed to harsh conditions, such as high temperatures, high pressures, and high rotational speeds. It is of great significance to accurately predict the life of blades for reducing maintenance cost and improving the reliability of gas turbine systems. A rapid and accurate blade life assessment method holds significant importance in the maintenance plan of gas turbine engines. In this paper, a novel on-line remaining useful life (RUL) prediction method for high-temperature blades is proposed based on 3D reconstruction technology and data-driven surrogate mode. Firstly, the 3D reconstruction technology was employed to establish the geometric model of real turbine blades, and the fluid–thermal–solid analysis under actual operational conditions was carried out in ANSYS software. Six checkpoints were selected to estimate the RUL according to the stress–strain distribution of the blade surface. The maximum equivalent stress was 1481.51 MPa and the highest temperature was 1393.42 K. Moreover, the fatigue-creep lifetime was calculated according to the parameters of the selected checkpoints. The RUL error between the simulation model and commercial software (Control and Engine Health Management (CEHM)) was less than 0.986%. Secondly, different data-driven surrogate models (BP, DNN, and LSTM algorithms) were developed according to the results from numerical simulation. The maximum relative errors of BP, DNN, and LSTM models were 0.030%, 0.019%, and 0.014%. LSTM demonstrated the best performance in predicting the RUL of turbine blades with time-series characteristics. Finally, the LSTM model was utilized for predicting the RUL within a gas turbine real operational process that involved five start–stop cycles.

In this paper, an innovative approach concerning the investigation of the human heart is introduced, employing state-of-the-art technologies. In particular, sophisticated algorithms were developed to automatically reconstruct a 3D model of a human heart based on DICOM data and to segment the main parts that constitute it. Regarding the reconstructed 3D model, a diagnosis of the examined patient can be derived, whereas in the present study, a clinical case involving the coarctation of the aorta was inspected. Moreover, numerical approaches that are able to simulate flows on complex shapes were considered. Thereupon, the outcomes of the computation analysis coupled with the segmented patient-specific 3D model were inserted in a virtual reality environment, where the clinicians can visualize the blood flow at the vessel walls and train on real-life medical scenarios, enhancing their procedural understanding prior to the actual operation. The physical model was 3D-printed via the MultiJet 3D printing process utilizing materials possessing an adequate mechanical response replicating the mechanical properties and the geometrical characteristics of the human heart. The presented tools aim at the creation of an innovative digital environment, where gaining surgical experience and developing pre-operative strategies could be achieved without the risk and anxiety of actual surgery.

There has emerged substantial research in addressing single-view 3D reconstruction and the majority of the state-of-the-art implicit methods employ CNNs as the backbone network. On the other hand, transformers have shown remarkable performance in many vision tasks. However, it is still unknown whether transformers are suitable for single-view implicit 3D reconstruction. In this paper, we propose the first end-to-end single-view 3D reconstruction network based on the Pyramid Vision Transformer (PVT), called ED2IF2-Net, which disentangles the reconstruction of an implicit field into the reconstruction of topological structures and the recovery of surface details to achieve high-fidelity shape reconstruction. ED2IF2-Net uses a Pyramid Vision Transformer encoder to extract multi-scale hierarchical local features and a global vector of the input single image, which are fed into three separate decoders. A coarse shape decoder reconstructs a coarse implicit field based on the global vector, a deformation decoder iteratively refines the coarse implicit field using the pixel-aligned local features to obtain a deformed implicit field through multiple implicit field deformation blocks (IFDBs), and a surface detail decoder predicts an enhanced displacement field using the local features with hybrid attention modules (HAMs). The final output is a fusion of the deformed implicit field and the enhanced displacement field, with four loss terms applied to reconstruct the coarse implicit field, structure details through a novel deformation loss, overall shape after fusion, and surface details via a Laplacian loss. The quantitative results obtained from the ShapeNet dataset validate the exceptional performance of ED2IF2-Net. Notably, ED2IF2-Net-L stands out as the top-performing variant, exhibiting the highest mean IoU, CD, EMD, ECD-3D, and ECD-2D scores, reaching impressive values of 61.1, 7.26, 2.51, 6.08, and 1.84, respectively. The extensive experimental evaluations consistently demonstrate the state-of-the-art capabilities of ED2IF2-Net in terms of reconstructing topological structures and recovering surface details, all while maintaining competitive inference time.

Reconstructing 3D models from a single image has numerous applications in fields such as VR/AR, medical imaging, and gaming. However, most mesh-based methods are limited by the use of 0-genus initial templates, which makes it difficult to reconstruct 3D meshes with complex topologies. Additionally, existing methods often prioritize reconstructing the overall shape and neglect to study local meshes with varying curvatures, resulting in a lack of correct and detailed local features in the generated meshes. This paper proposes a 3D reconstruction framework that transitions from global to local and incorporates MLP and GCN. The framework introduces a mesh pruning strategy under a 2D hole constraint to restore the correct mesh topology. Moreover, the framework fine-tunes local details by separately learning corresponding mapping functions on high-frequency and low-frequency local extended patches. The experiment with the proposed network on the ShapeNet dataset shows that the network has a CD value of 1.763 and an F-score of 85.40. The results from extensive experiments demonstrate that our proposed method outperforms existing methods in topology correction and local detail reconstruction.

Image matching is a basic task in three-dimensional reconstruction, which, in recent years, has attracted extensive attention in academic and industrial circles. However, when dealing with large-scale image datasets, these methods have low accuracy and slow speeds. To improve the effectiveness of modern image matching methods, this paper proposes an image matching method for 3D reconstruction. The proposed method can obtain high matching accuracy through hash index in a very short amount of time. The core of hash matching includes two parts: creating the hash table and hash index. The former is used to encode local feature descriptors into hash codes, and the latter is used to search candidates for query feature points. In addition, the proposed method is extremely robust to image scaling and transformation by using various verifications. A comprehensive experiment was carried out using several challenging datasets to evaluate the performance of hash matching. Experimental results show that the HashMatch presents excellent results compared to the state-of-the-art methods in both computational efficiency and matching accuracy.

Augmented reality is a mature technology that uses the real world as a substrate and extends it by overlaying computer-generated information. It has been applied to several domains. In particular, the technology was proven to be useful for the management and preservation of Cultural Heritage. This study provides an overview of the last decade of the use of augmented reality in cultural heritage through a detailed review of the scientific papers in the field. We analyzed the applications published on Scopus and Clarivate Web of Science databases over a period of 9 years (2012–2021). Bibliometric data consisted of 1201 documents, and their analysis was performed using various tools, including ScientoPy, VOS Viewer, and Microsoft Excel. The results revealed eight trending topics of applying augmented reality technology to cultural heritage: 3D reconstruction of cultural artifacts, digital heritage, virtual museums, user experience, education, tourism, intangible cultural heritage, and gamification. Each topic is discussed in detail in the article sections, providing insight into existing applications and research trends for each application field.

With the rapid development of the geographic information service industry, point cloud data are widely used in various fields, such as architecture, planning, cultural relics protection, mining engineering, etc. Despite that there are many approaches to collecting point clouds, we are facing the problem of point cloud holes caused by the inability of a 3D laser scanner to collect data completely in the narrow space of the mine access shaft. Thus, this paper uses RGB-D cameras to collect data and reconstruct the hole in the point cloud. We used a 3D laser scanner and RGB-D depth camera to collect the 3D point cloud data of the access shaft roadway. The maximum error was 2.617 cm and the minimum error was 0.031 cm by measuring the distance between the feature points, which satisfied the visualization repair of the missing parts of the 3D laser scanner data collection. We used the FPTH + ICP algorithm, ISS + ICP algorithm, SVD + ICP algorithm, and 3D-NDT algorithm to perform registration and fusion of the processed 3D point cloud and the original point cloud and finally repaired the hole. The study results show that the ISS + ICP registration algorithm had the most matching points and the lowest RMSE value of 13.8524 mm. In addition, in the closed and narrow roadway, the RGB-D camera was light and easy to operate and the point data acquired by it had relatively high precision. The three-dimensional point cloud of the repaired access shaft roadway has a good fit and can meet the repair requirements.

To address the lack of fast and high-precision in situ measurement of large-sized pipes in current industrial applications, a pipe-diameter-measurement method based on stereo vision is designed in this paper. By using multiple sets of binocular cameras to perform 3D reconstruction and integration of multiple laser markers projected on measured cross sections of pipes, the pipe diameter can be estimated. In this method, a measurement algorithm is adopted to enable automatic matching of feature points through affine distance transformation, and an optimized point-cloud-registration algorithm with normal-vector constraints is used to ensure measurement robustness. To verify the feasibility of the method, an experimental system was built under laboratory conditions, and three types of pipes with outer diameters from 285 mm to 325 mm were measured. The experimental results show that the relative error is within ±0.570% and the maximum repeatability standard deviation is 0.551 mm. The experimental results basically meet industrial standards, and the proposed method therefore has good application prospects.

In this paper, an approach through a Deep Learning architecture for the three-dimensional reconstruction of outdoor environments in challenging terrain conditions is presented. The architecture proposed is configured as an Autoencoder. However, instead of the typical convolutional layers, some differences are proposed. The Encoder stage is set as a residual net with four residual blocks, which have been provided with the necessary knowledge to extract the feature maps from aerial images of outdoor environments. On the other hand, the Decoder stage is set as a Generative Adversarial Network (GAN) and called a GAN-Decoder. The proposed network architecture uses a sequence of the 2D aerial image as input. The Encoder stage works for the extraction of the vector of features that describe the input image, while the GAN-Decoder generates a point cloud based on the information obtained in the previous stage. By supplying a sequence of frames that a percentage of overlap between them, it is possible to determine the spatial location of each generated point. The experiments show that with this proposal it is possible to perform a 3D representation of an area flown over by a drone using the point cloud generated with a deep architecture that has a sequence of aerial 2D images as input. In comparison with other works, our proposed system is capable of performing three-dimensional reconstructions in challenging urban landscapes. Compared with the results obtained using commercial software, our proposal was able to generate reconstructions in less processing time, with less overlapping percentage between 2D images and is invariant to the type of flight path.

In this paper, a surface measurement method based on dual-scan positioning strategy is presented to address the challenges of irregular surface patterns and complex geometries. A confocal sensor with an internal scanning mechanism was used in this study. By synchronizing the local scan, enabled by the internal actuator in the confocal sensor, and the global scans, enabled by external positioners, the developed system was able to perform noncontact line scan and area scan. Thus, this system was able to measure both surface roughness and surface uniformity. Unlike laboratory surface measurement equipment, the proposed system is reconfigurable for in situ measurement and able to scan free-form surfaces with a proper stand-off distance and approaching angle. For long-travel line scan, which is needed for rough surfaces, a surface form tracing algorithm was developed to ensure that the data were always captured within the sensing range of the confocal sensor. It was experimentally verified that in a scanning length of 100 mm, where the surface fluctuation in vertical direction is around 10 mm, the system was able to perform accurate surface measurement. For area scan, XY coordinates provided by the lateral positioning system and the Z coordinate captured by the confocal sensor were plotted into one coordinate system for 3D reconstruction. A coherence scanning interferometer and a confocal microscope were employed as the reference measurement systems to verify the performance of the proposed system in a scanning area of 1 mm by 1 mm. Experimental data showed that the proposed system was able to achieve comparable accuracy with laboratory systems. The measurement deviation was within 0.1 µm. Because line scan mechanisms are widely used in sensor design, the presented work can be generalized to expand the applications of line scan sensors.

Estimating the 3D shape of a scene from differently focused set of images has been a practical approach for 3D reconstruction with color cameras. However, reconstructed depth with existing depth from focus (DFF) methods still suffer from poor quality with textureless and object boundary regions. In this paper, we propose an improved depth estimation based on depth from focus iteratively refining 3D shape from uniformly focused image set (UFIS). We investigated the appearance changes in spatial and frequency domains in iterative manner. In order to achieve sub-frame accuracy in depth estimation, optimal location of focused frame in DFF is estimated by fitting a polynomial curve on the dissimilarity measurements. In order to avoid wrong depth values on texture-less regions we propose to build a confidence map and use it to identify erroneous depth estimations. We evaluated our method on public and our own datasets obtained from different types of devices, such as smartphones, medical, and normal color cameras. Quantitative and qualitative evaluations on various test image sets show promising performance of the proposed method in depth estimation.

A multi-medium motion capture system based on markers’ visual detection is developed and experimentally demonstrated for monitoring underwater intelligent agents such as fish biology and bionic robot-fish. Considering the refraction effect between air and water, a three-dimensional (3D) reconstruction model is established, which can be utilized to reconstruct the 3D coordinate of markers underwater from 2D data. Furthermore, the process of markers matching is undertaken through the multi-lens fusion perception prediction combined K-Means clustering algorithm. Subsequently, in order to track the marker of being occluded, according to the kinematics information of fish, an improved Kalman filtering algorithm is proposed. Finally, the feasibility and effectiveness of proposed system are verified through experimental results. The main models and methods in this paper can provide a reference and inspiration for measurement of underwater intelligent agents.

In the architecture, engineering, and construction (AEC) industry, creating an indoor model of existing buildings has been a challenging task since the introduction of building information modeling (BIM). Because the process of BIM is primarily manual and implies a high possibility of error, the automated creation of indoor models remains an ongoing research. In this paper, we propose a fully automated method to generate 2D floorplan computer-aided designs (CADs) from 3D point clouds. The proposed method consists of two main parts. The first is to detect planes in buildings, such as walls, floors, and ceilings, from unstructured 3D point clouds and to classify them based on the Manhattan-World (MW) assumption. The second is to generate 3D BIM in the industry foundation classes (IFC) format and a 2D floorplan CAD using the proposed line-detection algorithm. We experimented the proposed method on 3D point cloud data from a university building, residential houses, and apartments and evaluated the geometric quality of a wall reconstruction. We also offer the source code for the proposed method on GitHub.

As a classical method widely used in 3D reconstruction tasks, the multi-source Photometric Stereo can obtain more accurate 3D reconstruction results compared with the basic Photometric Stereo, but its complex calibration and solution process reduces the efficiency of this algorithm. In this paper, we propose a multi-source Photometric Stereo 3D reconstruction method based on the fully convolutional network (FCN). We first represent the 3D shape of the object as a depth value corresponding to each pixel as the optimized object. After training in an end-to-end manner, our network can efficiently obtain 3D information on the object surface. In addition, we added two regularization constraints to the general loss function, which can effectively help the network to optimize. Under the same light source configuration, our method can obtain a higher accuracy than the classic multi-source Photometric Stereo. At the same time, our new loss function can help the deep learning method to get a more realistic 3D reconstruction result. We have also used our own real dataset to experimentally verify our method. The experimental results show that our method has a good effect on solving the main problems faced by the classical method.

Among various building information model (BIM) reconstruction methods for existing building, image-based method can identify building components from scanned as-built drawings and has won great attention due to its lower cost, less professional operators and better reconstruction performance. However, this kind of method will cost a great deal of time to design and extract features. Moreover, the manually extracted features have poor robustness and contain less non-geometric information. In order to solve this problem, this paper proposes a deep learning-based method to detect building components from scanned 2D drawings. Taking structural drawings as an example, in this article, 1500 images of structural drawings were firstly collected and preprocessed to guarantee the quality of data. After that, the neural network model—You Only Look Once (YOLO) was trained, verified and tested. In addition, a series of metrics were utilized to evaluate the performance of recognition. The results of test experiments show that the components in structural drawings (e.g., grid reference, column and beam) can be successfully detected, while the average detection accuracy of the whole image is over 80% and the average detection time for each image is 0.71 s. The experimental results demonstrate that the proposed method is robust and timesaving, which provides a good basis for the reconstruction of BIM from 2D drawings.

The domain of non-destructive testing (NDT) or thermal characterization is currently often done by using contactless methods based on the use of an IR camera to monitor the transient temperature response of a system or sample warmed by using any heat source. Though many techniques use optical excitation (flash lamps, lasers, etc.), some techniques use volumetric sources such as acoustic or induction waves. In this paper, we propose a new inverse processing method, which allows for the estimation of 3D fields of heat sources from surface temperature measurements. This method should be associated with volumetric heat source generation. To validate the method, a volumetric source was generated by the Joule effect in a homogeneous PVC sample using an electrical thin cylindrical wire molded in the material. The inverse processing allows us to retrieve the depth of the wire and its geometrical shape and size. This tool could be a new procedure for retrieving 3D defects on NDT.

With the advent of the era of artificial intelligence and big data, intelligent security robots not only improve the efficiency of the traditional intelligent security industry but also propose higher requirements for intelligent security. Aiming to solve the problem of long recognition time and high equipment cost of intelligent security robots, we propose a new face recognition method for intelligent security in this paper. We use the Goldstein branching method for phase unwrapping, which can improve the three-dimensional (3D) face reconstruction effect. Subsequently, by using the three-dimensional face recognition method based on face radial curve elastic matching, different weights are assigned to different curve recognition similarity for weighted fusion as the total similarity for recognition. Experiments show that the method has a higher face recognition rate and is robust to attitude, illumination, and noise.

The depth estimation of the 3D deformable object has become increasingly crucial to various intelligent applications. In this paper, we propose a feature-based approach for accurate depth estimation of a deformable 3D object with a single camera, which reduces the problem of depth estimation to a pose estimation problem. The proposed method needs to reconstruct the target object at the very beginning. With the 3D reconstruction as an a priori model, only one monocular image is required afterwards to estimate the target object’s depth accurately, regardless of pose changes or deformability of the object. Experiments are taken on an NAO robot and a human to evaluate the depth estimation accuracy by the proposed method.

The creation of as-built Building Information Models requires the acquisition of the as-is state of existing buildings. Laser scanners are widely used to achieve this goal since they permit to collect information about object geometry in form of point clouds and provide a large amount of accurate data in a very fast way and with a high level of details. Unfortunately, the scan-to-BIM (Building Information Model) process remains currently largely a manual process which is time consuming and error-prone. In this paper, a semi-automatic approach is presented for the 3D reconstruction of indoors of existing buildings from point clouds. Several segmentations are performed so that point clouds corresponding to grounds, ceilings and walls are extracted. Based on these point clouds, walls and slabs of buildings are reconstructed and described in the IFC format in order to be integrated into BIM software. The assessment of the approach is proposed thanks to two datasets. The evaluation items are the degree of automation, the transferability of the approach and the geometric quality of results of the 3D reconstruction. Additionally, quality indexes are introduced to inspect the results in order to be able to detect potential errors of reconstruction.

In this paper, we present a new stereo vision-based system and its efficient hardware implementation for real-time underwater environments exploration throughout 3D sparse reconstruction based on a number of feature points. The proposed underwater 3D shape reconstruction algorithm details are presented. The main concepts and advantages are discussed and comparison with existing systems is performed. In order to achieve real-time video constraints, a hardware implementation of the algorithm is performed using Xilinx System Generator. The pipelined stereo vision system has been implemented using Field Programmable Gate Arrays (FPGA) technology. Both timing constraints and mathematical operations precision have been evaluated in order to validate the proposed hardware implementation of our system. Experimental results show that the proposed system presents high accuracy and execution time performances.