Search

This paper introduces OpenGaussian, a method based on 3D Gaussian Splatting (3DGS) capable of 3D point-level open vocabulary understanding. Our primary motivation stems from observing that existing 3DGS-based open vocabulary methods mainly focus on 2D pixel-level parsing. These methods struggle with 3D point-level tasks due to weak feature expressiveness and inaccurate 2D-3D feature associations. To ensure robust feature presentation and 3D point-level understanding, we first employ SAM masks without cross-frame associations to train instance features with 3D consistency. These features exhibit both intra-object consistency and inter-object distinction. Then, we propose a two-stage codebook to discretize these features from coarse to fine levels. At the coarse level, we consider the positional information of 3D points to achieve location-based clustering, which is then refined at the fine level. Finally, we introduce an instance-level 3D-2D feature association method that links 3D points to 2D masks, which are further associated with 2D CLIP features. Extensive experiments, including open vocabulary-based 3D object selection, 3D point cloud understanding, click-based 3D object selection, and ablation studies, demonstrate the effectiveness of our proposed method. Project page: https://3d-aigc.github.io/OpenGaussian, Comment: technical report, 15 pages

Single image-to-3D generation is pivotal for crafting controllable 3D assets. Given its under-constrained nature, we attempt to leverage 3D geometric priors from a novel view diffusion model and 2D appearance priors from an image generation model to guide the optimization process. We note that there is a disparity between the generation priors of these two diffusion models, leading to their different appearance outputs. Specifically, image generation models tend to deliver more detailed visuals, whereas novel view models produce consistent yet over-smooth results across different views. Directly combining them leads to suboptimal effects due to their appearance conflicts. Hence, we propose a 2D-3D hybrid Fourier Score Distillation objective function, hy-FSD. It optimizes 3D Gaussians using 3D priors in spatial domain to ensure geometric consistency, while exploiting 2D priors in the frequency domain through Fourier transform for better visual quality. hy-FSD can be integrated into existing 3D generation methods and produce significant performance gains. With this technique, we further develop an image-to-3D generation pipeline to create high-quality 3D objects within one minute, named Fourier123. Extensive experiments demonstrate that Fourier123 excels in efficient generation with rapid convergence speed and visually-friendly generation results.

3D Gaussian Splatting (3DGS) has marked a significant breakthrough in the realm of 3D scene reconstruction and novel view synthesis. However, 3DGS, much like its predecessor Neural Radiance Fields (NeRF), struggles to accurately model physical reflections, particularly in mirrors that are ubiquitous in real-world scenes. This oversight mistakenly perceives reflections as separate entities that physically exist, resulting in inaccurate reconstructions and inconsistent reflective properties across varied viewpoints. To address this pivotal challenge, we introduce Mirror-3DGS, an innovative rendering framework devised to master the intricacies of mirror geometries and reflections, paving the way for the generation of realistically depicted mirror reflections. By ingeniously incorporating mirror attributes into the 3DGS and leveraging the principle of plane mirror imaging, Mirror-3DGS crafts a mirrored viewpoint to observe from behind the mirror, enriching the realism of scene renderings. Extensive assessments, spanning both synthetic and real-world scenes, showcase our method's ability to render novel views with enhanced fidelity in real-time, surpassing the state-of-the-art Mirror-NeRF specifically within the challenging mirror regions. Our code will be made publicly available for reproducible research., Comment: 22 pages, 7 figures

We experimentally investigated the optical responses of a superconducting niobium resonator. It was found that, with increasing radiation power, the resonance frequency increases monotonically below around 500 mK, decreases monotonically above around 1 K and exhibits a nonmonotonic behavior at around 700 mK. These observations show that one can operate the irradiated resonator in three temperature regimes, depending on whether two-level system (TLS) effects or kinetic inductance effects dominate. Furthermore, we found that the optical responses at ultra-low temperatures can be qualitatively regarded as a photon-induced thermalization effect of TLSs, which could be utilized to achieve thermal sensitive photon detections., Comment: Accepted by the Journal of Applied Physics

We designed and successfully fabricated an absorption-type of superconducting coplanar waveguide (CPW) resonators. The resonators are made from a Niobium film (about 160 nm thick) on a high-resistance Si substrate, and each resonator is fabricated as a meandered quarter-wavelength transmission line (one end shorts to the ground and another end is capacitively coupled to a through feedline). With a vector network analyzer we measured the transmissions of the applied microwave through the resonators at ultra-low temperature (e.g., at 20 mK), and found that their loaded quality factors are significantly high, i.e., up to 10^6. With the temperature increases slowly from the base temperature (i.e., 20 mK), we observed the resonance frequencies of the resonators are blue shifted and the quality factors are lowered slightly. In principle, this type of CPW-device can integrate a series of resonators with a common feedline, making it a promising candidate of either the data bus for coupling the distant solid-state qubits or the sensitive detector of single photons., Comment: Accepted by Chinese Science Bulletin