Search

This paper presents STNeRF, a method for synthesizing novel views of vehicles from single-view 2D images without the need for 3D ground truth data, such as point clouds, depth maps, CAD models, etc., as prior knowledge. A significant challenge in this task arises from the characteristics of CNNs and the utilization of local features can lead to a flattened representation of the synthesized image when training and validation with images from a single viewpoint. Many current methodologies tend to overlook local features and rely on global features throughout the entire reconstruction process, potentially resulting in the loss of fine-grained details in the synthesized image. To tackle this issue, we introduce Symmetric Triplane Neural Radiance Fields (STNeRF). STNeRF employs a triplane feature extractor with spatially aware convolution to extend 2D image features into 3D. This decouples the appearance component, which includes local features, and the shape component, which consists of global features, and utilizes them to construct a neural radiance field. These neural priors are then employed for rendering novel views. Furthermore, STNeRF leverages the symmetric properties of vehicles to liberate the appearance component from reliance on the original viewpoint and to align it with the symmetry of the target space, thereby enhancing the neural radiance field network’s ability to represent the invisible regions. The qualitative and quantitative evaluations demonstrate that STNeRF outperforms existing solutions in terms of both geometry and appearance reconstruction. More supplementary materials and the implementation code are available for access at the following link: . [ABSTRACT FROM AUTHOR]

Monostatic rapid single-view radar imaging technology is a technique that employs single incidence angle and single frequency point information to implement rapid monostatic radar imaging within a small angular field. Owing to its analytical expression, this technique can substitute the traditional frequency-angle- scanning imaging in a small angular range, facilitating the rapid generation of highly realistic radar imaging data slices for complex targets and environments. This technology has been significantly applied in scatter hotspot diagnostics and target recognition. In order to achieve the windowing effect equivalent to that of frequency-angle-scanning imaging, and to enhance the scattering feature of monostatic imaging while controlling sidelobes, this paper derives analytic windowed imaging formulas for monostatic radar. It then obtains analytical expressions for various typical monostatic windowing rapid radar imaging scenarios. This enables the monostatic rapid imaging technology to maintain high efficiency in its analytical expressions while achieving the windowing effect equivalent to traditional imaging. The validity and correctness of the analytical formula and software implementation have been confirmed through 1D, 2D, and 3D imaging verifications. This technology can provide a vast amount of training data for modern radars. [ABSTRACT FROM AUTHOR]

3D garment reconstruction has a wide range of applications in apparel design, digital human body, and virtual try-on. Reconstructing 3D shapes from single-view images is a completely undefined and challenging problem. Recent single-view methods require only single-view images of static or dynamic objects and mine the potential multi-view information in single-view images by statistical, geometric, and physical prior knowledge, which is tedious to obtain. In this paper, we use an implicit function of pixel alignment represented by a neural network to correlate 2D image pixels with the corresponding 3D clothing information for end-to-end training without any relevant prior model. The qualitative and quantitative analysis of the experimental results showed that our results reduced the relative error by an average of 2.6% and the chamfer distance by 2.37% compared to the previous method. Experiments show that our model can reasonably reconstruct the 3D model of garments from their collections of single-view images. Our method can not only capture the overall geometry of the garment but also extract the tiny but important wrinkle details of the fabric. Even with low-resolution input images, our model still achieves available results. In addition, experiments show that the reconstructed 3D models of garments can be used for texture migration and virtual fitting. [ABSTRACT FROM AUTHOR]

There has emerged substantial research in addressing single-view 3D reconstruction and the majority of the state-of-the-art implicit methods employ CNNs as the backbone network. On the other hand, transformers have shown remarkable performance in many vision tasks. However, it is still unknown whether transformers are suitable for single-view implicit 3D reconstruction. In this paper, we propose the first end-to-end single-view 3D reconstruction network based on the Pyramid Vision Transformer (PVT), called ED 2 IF 2 -Net, which disentangles the reconstruction of an implicit field into the reconstruction of topological structures and the recovery of surface details to achieve high-fidelity shape reconstruction. ED 2 IF 2 -Net uses a Pyramid Vision Transformer encoder to extract multi-scale hierarchical local features and a global vector of the input single image, which are fed into three separate decoders. A coarse shape decoder reconstructs a coarse implicit field based on the global vector, a deformation decoder iteratively refines the coarse implicit field using the pixel-aligned local features to obtain a deformed implicit field through multiple implicit field deformation blocks (IFDBs), and a surface detail decoder predicts an enhanced displacement field using the local features with hybrid attention modules (HAMs). The final output is a fusion of the deformed implicit field and the enhanced displacement field, with four loss terms applied to reconstruct the coarse implicit field, structure details through a novel deformation loss, overall shape after fusion, and surface details via a Laplacian loss. The quantitative results obtained from the ShapeNet dataset validate the exceptional performance of ED 2 IF 2 -Net. Notably, ED 2 IF 2 -Net-L stands out as the top-performing variant, exhibiting the highest mean IoU, CD, EMD, ECD-3D, and ECD-2D scores, reaching impressive values of 61.1, 7.26, 2.51, 6.08, and 1.84, respectively. The extensive experimental evaluations consistently demonstrate the state-of-the-art capabilities of ED 2 IF 2 -Net in terms of reconstructing topological structures and recovering surface details, all while maintaining competitive inference time. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Data Acquisition & Processing / Shu Ju Cai Ji Yu Chu Li is the property of Editorial Department of Journal of Nanjing University of Aeronautics & Astronautics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The single-view 3D voxel reconstruction approach that relies on deep learning is inherently constrained by its single-input nature, which fails to account for the disparities in data distribution between the image and voxel domains. This leads to insufficient model details in the reconstruction. Currently, there is an absence of an end-to-end single-view 3D voxel reconstruction technique that effectively addresses the data distribution variances across domains while streamlining the training and inference processes. To mitigate the challenges arising from the disparities in data distribution across various domains, we introduce a novel single-view 3D voxel reconstruction model that leverages cross-domain feature fusion, termed SV3D-CDFF. SV3D-CDFF utilizes cross-domain feature clustering to mitigate the impact of data distribution disparities and employ a feature supervision method to learns voxel features. Additionally, it incorporates attention mechanism to fuse image and voxel features and utilizes residual network for 3D voxel reconstruction. Quantitative and qualitative experimental results demonstrate that SV3D-CDFF effectively integrates image and voxel features, eliminates data distribution disparities across various domains, and improves the quality of reconstructed voxel models in terms of overall structure and local detail. SV3D-CDFF outperforms existing state-of-the-arts methods in terms of the IoU and F-Score, with improvements of 0.005 and 0.045, respectively. [ABSTRACT FROM AUTHOR]

Copyright of Lubrication Engineering (0254-0150) is the property of Editorial Office of LUBRICATION ENGINEERING and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

There has emerged substantial research in addressing single-view 3D reconstruction and the majority of the state-of-the-art implicit methods employ CNNs as the backbone network. On the other hand, transformers have shown remarkable performance in many vision tasks. However, it is still unknown whether transformers are suitable for single-view implicit 3D reconstruction. In this paper, we propose the first end-to-end single-view 3D reconstruction network based on the Pyramid Vision Transformer (PVT), called ED2IF2-Net, which disentangles the reconstruction of an implicit field into the reconstruction of topological structures and the recovery of surface details to achieve high-fidelity shape reconstruction. ED2IF2-Net uses a Pyramid Vision Transformer encoder to extract multi-scale hierarchical local features and a global vector of the input single image, which are fed into three separate decoders. A coarse shape decoder reconstructs a coarse implicit field based on the global vector, a deformation decoder iteratively refines the coarse implicit field using the pixel-aligned local features to obtain a deformed implicit field through multiple implicit field deformation blocks (IFDBs), and a surface detail decoder predicts an enhanced displacement field using the local features with hybrid attention modules (HAMs). The final output is a fusion of the deformed implicit field and the enhanced displacement field, with four loss terms applied to reconstruct the coarse implicit field, structure details through a novel deformation loss, overall shape after fusion, and surface details via a Laplacian loss. The quantitative results obtained from the ShapeNet dataset validate the exceptional performance of ED2IF2-Net. Notably, ED2IF2-Net-L stands out as the top-performing variant, exhibiting the highest mean IoU, CD, EMD, ECD-3D, and ECD-2D scores, reaching impressive values of 61.1, 7.26, 2.51, 6.08, and 1.84, respectively. The extensive experimental evaluations consistently demonstrate the state-of-the-art capabilities of ED2IF2-Net in terms of reconstructing topological structures and recovering surface details, all while maintaining competitive inference time.

3D reconstruction from a single image is one of the core computer vision problems. Thanks to the development of deep learning, 3D reconstruction of a single image has demonstrated impressive progress in recent years. Existing researches use Chamfer distance as a loss function to guard the training of the neural network. However, the Chamfer loss will give equal weights to all points inside the 3D point clouds. It tends to sacrifice fine-grained and thin structures to avoid incurring a high loss, which will lead to visually unsatisfactory results. This paper proposes a framework that can recover a detailed three-dimensional point cloud from a single image by focusing more on boundaries (edge and corner points). Experimental results demonstrate that the proposed method outperforms existing techniques significantly, both qualitatively and quantitatively, and has fewer training parameters. [Display omitted] • Reconstructing the full 3D shape of an object from a single image. • Using a differentiable point cloud projection module to get the projected images. • Increasing visual reconstruction quality. [ABSTRACT FROM AUTHOR]

This paper summarized the research status, imaging model, systems calibration, distortion correction, and panoramic expansion of panoramic vision systems, pointed out the existing problems and put forward the prospect of future research. According to the research status of panoramic vision systems, a panoramic vision system with single viewpoint of refraction and reflection is designed. The systems had the characteristics of fast acquisition, low manufacturing cost, fixed single-view imaging, integrated imaging, and automatic switching depth of field. Based on these systems, an improved nonlinear optimization polynomial fitting method is proposed to calibrate the monocular HOVS, and the binocular HOVS is calibrated with the Aruco label. This method not only improves the robustness of the calibration results, but also simplifies the calibration process. Finally, a real-time method of panoramic map of multi-function vehicle based on vcam is proposed.

This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize fully high-fidelity 3D / 4D organ geometric models from single-view medical images with complicated background in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to segment or extract the accurate 3D organ models subsequently. The computational time and imaging dose can be reduced by decreasing the number of projections, but the reconstructed image quality is degraded accordingly. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models; while, all current deep learning based approaches on the shape reconstruction from a single image cannot. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples, including both synthetic phantom and real patient datasets. The efficiency of the proposed method shows that it only needs several milliseconds to generate organ meshes with 10K vertices, which has great potential to be used in real-time image guided radiation therapy (IGRT). [ABSTRACT FROM AUTHOR]

The approximation of 3D geometry through single image is a particular instance of 3D reconstruction from several images. The advance information or user input should be provided to retrieve or conclude depth information. This research presents a novel automatic method to enumerated 3D affine measurements from a single perspective image. The least geometric information has been resolute through the image of the scene. The vanishing line and a vanishing point are two required information to reconstruct from an image of a scene. The affine scene structure can be reconstructed through the image of a scene. The proposed approach has many advantages; there is no need of the camera's intrinsic matrix and the explicit correlation among camera and scene (pose), no need for selecting Vx, Vy, Vz points, novel dexterous robot architecture for manipulation. In this paper, the following approaches have been implemented: (1) the three sets of vanishing points in X, Y, and Z axis; (2) the vanishing lines of the image; (3) distance among planes that parallel to the reference plane; (4) image wrapping; (5) corner detection (algorithm has been implemented in order to make the process automatic). The indigenous data set has been taken for the experiment. The results are compared with Zhang- and ArUco-based calibration. This novel approach has been used to perform tracking and manipulation of an object in real-time environment. [ABSTRACT FROM AUTHOR]

We present a new motion tracking technique to robustly reconstruct non-rigid geometries and motions from a single view depth input recorded by a consumer depth sensor. The idea is based on the observation that most non-rigid motions (especially human-related motions) are intrinsically involved in articulate motion subspace. To take this advantage, we propose a novel $L_0$ based motion regularizer with an iterative solver that implicitly constrains local deformations with articulate structures, leading to reduced solution space and physical plausible deformations. The $L_0$ strategy is integrated into the available non-rigid motion tracking pipeline, and gradually extracts articulate joints information online with the tracking, which corrects the tracking errors in the results. The information of the articulate joints is used in the following tracking procedure to further improve the tracking accuracy and prevent tracking failures. Extensive experiments over complex human body motions with occlusions, facial and hand motions demonstrate that our approach substantially improves the robustness and accuracy in motion tracking. [ABSTRACT FROM PUBLISHER]

This article proposes a real-time method that uses a single-view RGB-D input (a depth sensor integrated with a color camera) to simultaneously reconstruct a casual scene with a detailed geometry model, surface albedo, per-frame non-rigid motion, and per-frame low-frequency lighting, without requiring any template or motion priors. The key observation is that accurate scene motion can be used to integrate temporal information to recover the precise appearance, whereas the intrinsic appearance can help to establish true correspondence in the temporal domain to recover motion. Based on this observation, we first propose a shading-based scheme to leverage appearance information for motion estimation. Then, using the reconstructed motion, a volumetric albedo fusing scheme is proposed to complete and refine the intrinsic appearance of the scene by incorporating information from multiple frames. Since the two schemes are iteratively applied during recording, the reconstructed appearance and motion become increasingly more accurate. In addition to the reconstruction results, our experiments also show that additional applications can be achieved, such as relighting, albedo editing, and free-viewpoint rendering of a dynamic scene, since geometry, appearance, and motion are all reconstructed by our technique. [ABSTRACT FROM AUTHOR]

In this paper we tackle the problem of 3D modeling for urban environment using a modular, flexible and powerful approach driven from procedural generation. To this end, typologies of architectures are modeled through shape grammars that consist of a set of derivation rules and a set of shape/dictionary elements. Appearance (from statistical point of view with respect to the individual pixel's properties) of the dictionary elements is then learned using a set of training images. Image classifiers are trained towards recovering image support with respect to the semantics. Then, given a new image and the corresponding footprint, the modeling problem is formulated as a search of the space of shapes, that can be generated on-the-fly by deriving the grammar on the input axiom. Defining an image-based score function for the produced instances using the trained classifiers, the best rules are selected, making sure that we keep exploring the space by allowing some rules to be randomly selected. New rules are then generated by resampling around the selected rules. At the finest level, these rules define the 3D model of the building. Promising results on complex and varying architectural styles demonstrate the potential of the presented method. [ABSTRACT FROM AUTHOR]

This paper summarized the research status, imaging model, systems calibration, distortion correction, and panoramic expansion of panoramic vision systems, pointed out the existing problems and put forward the prospect of future research. According to the research status of panoramic vision systems, a panoramic vision system with single viewpoint of refraction and reflection is designed. The systems had the characteristics of fast acquisition, low manufacturing cost, fixed single-view imaging, integrated imaging, and automatic switching depth of field. Based on these systems, an improved nonlinear optimization polynomial fitting method is proposed to calibrate the monocular HOVS, and the binocular HOVS is calibrated with the Aruco label. This method not only improves the robustness of the calibration results, but also simplifies the calibration process. Finally, a real-time method of panoramic map of multi-function vehicle based on vcam is proposed. [ABSTRACT FROM AUTHOR]

Cotraining, a paradigm of semisupervised learning, is promised to alleviate effectively the shortage of labeled examples in supervised learning. The standard two-view cotraining requires the data set to be described by two views of features, and previous studies have shown that cotraining works well if the two views satisfy the sufficiency and independence assumptions. In practice, however, these two assumptions are often not known or ensured (even when the two views are given). More commonly, most supervised data sets are described by one set of attributes (one view). Thus, they need be split into two views in order to apply the standard two-view cotraining. In this paper, we first propose a novel approach to empirically verify the two assumptions of cotraining given two views. Then, we design several methods to split single view data sets into two views, in order to make cotraining work reliably well. Our empirical results show that, given a whole or a large labeled training set, our view verification and splitting methods are quite effective. Unfortunately, cotraining is called for precisely when the labeled training set is small. However, given small labeled training sets, we show that the two cotraining assumptions are difficult to verify, and view splitting is unreliable. Our conclusions for cotraining's effectiveness are mixed. If two views are given, and known to satisfy the two assumptions, cotraining works well. Otherwise, based on small labeled training sets, verifying the assumptions or splitting single view into two views are unreliable; thus, it is uncertain whether the standard cotraining would work or not. [ABSTRACT FROM AUTHOR]