Back to Search Start Over

Self-Evolving Neural Radiance Fields

Authors :
Jung, Jaewoo
Han, Jisang
Kang, Jiwon
Kim, Seongchan
Kwak, Min-Seop
Kim, Seungryong
Publication Year :
2023

Abstract

Recently, neural radiance field (NeRF) has shown remarkable performance in novel view synthesis and 3D reconstruction. However, it still requires abundant high-quality images, limiting its applicability in real-world scenarios. To overcome this limitation, recent works have focused on training NeRF only with sparse viewpoints by giving additional regularizations, often called few-shot NeRF. We observe that due to the under-constrained nature of the task, solely using additional regularization is not enough to prevent the model from overfitting to sparse viewpoints. In this paper, we propose a novel framework, dubbed Self-Evolving Neural Radiance Fields (SE-NeRF), that applies a self-training framework to NeRF to address these problems. We formulate few-shot NeRF into a teacher-student framework to guide the network to learn a more robust representation of the scene by training the student with additional pseudo labels generated from the teacher. By distilling ray-level pseudo labels using distinct distillation schemes for reliable and unreliable rays obtained with our novel reliability estimation method, we enable NeRF to learn a more accurate and robust geometry of the 3D scene. We show and evaluate that applying our self-training framework to existing models improves the quality of the rendered images and achieves state-of-the-art performance in multiple settings.<br />Comment: 34 pages, 21 figures Our project page can be found at : https://ku-cvlab.github.io/SE-NeRF/

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2312.01003
Document Type :
Working Paper