Self-Adapting NeRF: Non-ideal Video Based NeRF for High-Quality Novel View Synthesis
摘要
We propose Self-Adapting NeRF for high-quality novel view synthesis based on non-ideal video input. We first using Lie algebra to encode camera poses, which were dynamically controlled by parameters from the precomputed Fourier transform encoding in the NeRF network input, achieving joint optimization of poses and the model. Subsequently, by monitoring intermediate training results, we supplement areas with poor performance, implementing a training strategy based on keyframe supplementation and gradient prioritization. This addresses the challenge of achieving high-quality novel view synthesis with NeRF-series in non-ideal input scenarios. Finally, we employ a strategy of geometry and material information separation, along with a reflection lighting model, to address issues in scenes with specular reflections.