Scholarlink

<p>NeRF (Neural Radiance Fields) exhibits the capability to synthesize images from unknown views. However, it faces challenges stemming from factors such as occlusion, non-Lambertian surfaces, sparse inputs, and weak textures commonly encountered in multi-view images. These complexities often result in fitting erroneous scene geometries, leading to suboptimal novel view synthesis quality. This paper addresses this challenge by harnessing the potential of feature-based Multi-View Stereo Matching (MVS) priors to assist NeRF in perceiving deeper information within the scene. The method is distinguished by its adaptive construction of Gaussian functions based on MVS-estimated depth values, uncertainties, and depth intervals, offering flexibility and adaptability to arbitrarily scaled scenes. Building upon this, the optimization of NeRF’s training process is achieved by comparing the difference between this distribution and the weight distribution of sampled points on the volume rendering ray. Furthermore, we propose an effective Riemann sum approximation strategy to further enhance the performance of depth loss. Quantitative metrics applied to three real-scene datasets, namely LLFF, IBRNet, and DTU, demonstrate that the method presented in this paper significantly improves the quality of novel view synthesis compared to current advanced methods, achieving enhancements ranging from 3.8% to 26.9%. Visualization experiments reveal robust optimization results, particularly in challenging regions where conventional NeRF encounters difficulties.</p> Graphical abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Robust novel view synthesis from multi-view feature stereo matching priors

  • 期刊论文

  • 发表日期：

    2025年3月25日

查看全文
  • Jianxin Wang,
  • Haijian Shao,
  • Xing Deng,
  • Shuheng Lian

摘要

NeRF (Neural Radiance Fields) exhibits the capability to synthesize images from unknown views. However, it faces challenges stemming from factors such as occlusion, non-Lambertian surfaces, sparse inputs, and weak textures commonly encountered in multi-view images. These complexities often result in fitting erroneous scene geometries, leading to suboptimal novel view synthesis quality. This paper addresses this challenge by harnessing the potential of feature-based Multi-View Stereo Matching (MVS) priors to assist NeRF in perceiving deeper information within the scene. The method is distinguished by its adaptive construction of Gaussian functions based on MVS-estimated depth values, uncertainties, and depth intervals, offering flexibility and adaptability to arbitrarily scaled scenes. Building upon this, the optimization of NeRF’s training process is achieved by comparing the difference between this distribution and the weight distribution of sampled points on the volume rendering ray. Furthermore, we propose an effective Riemann sum approximation strategy to further enhance the performance of depth loss. Quantitative metrics applied to three real-scene datasets, namely LLFF, IBRNet, and DTU, demonstrate that the method presented in this paper significantly improves the quality of novel view synthesis compared to current advanced methods, achieving enhancements ranging from 3.8% to 26.9%. Visualization experiments reveal robust optimization results, particularly in challenging regions where conventional NeRF encounters difficulties.

Graphical abstract