The vanilla NeRF and its subsequent variants usually assume that the path of the scene ray is a straight line. For scenes with refractive media, e.g., underwater scenes, the refracted scene rays would violate the assumption and bring geometrically incorrect modeling of the rendering procedure, resulting in low rendering quality. In particular, for underwater scenes with complicated refractive surfaces, an accurate ray-by-ray estimation of the refractive surface normal would be crucial for rendering the refracted rays. To address the limitations in current underwater neural rendering techniques, in this paper, we propose to realize high-quality neural rendering of scenes with refractive media while accurately reconstructing the complex refractive surface. Firstly, to accurately model the scene refraction, a multilayer perceptron is utilized to estimate the distance from the camera center to the refractive surface, which is used to fit a Bezier surface to achieve more accurate normal vector estimation. Secondly, a learnable parameter is designed to control the ratio of coarse-to-fine sampling and achieve adaptive sampling of the refracted ray. Furthermore, to enhance underwater low-frequency view-dependent information, the spherical harmonics is adopted for directional position encoding of the refracted ray. Extensive experimental results on public and self-built datasets demonstrate the rendering quality and surface reconstruction accuracy in various refractive scenes with complex refractive surfaces.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Refractive Neural Rendering of Underwater Scenes via Complex Refractive Surface Reconstruction

  • Xiaoqiang Zhang,
  • Zhixin Zhang,
  • Lingyan Ran,
  • Xinmin Li

摘要

The vanilla NeRF and its subsequent variants usually assume that the path of the scene ray is a straight line. For scenes with refractive media, e.g., underwater scenes, the refracted scene rays would violate the assumption and bring geometrically incorrect modeling of the rendering procedure, resulting in low rendering quality. In particular, for underwater scenes with complicated refractive surfaces, an accurate ray-by-ray estimation of the refractive surface normal would be crucial for rendering the refracted rays. To address the limitations in current underwater neural rendering techniques, in this paper, we propose to realize high-quality neural rendering of scenes with refractive media while accurately reconstructing the complex refractive surface. Firstly, to accurately model the scene refraction, a multilayer perceptron is utilized to estimate the distance from the camera center to the refractive surface, which is used to fit a Bezier surface to achieve more accurate normal vector estimation. Secondly, a learnable parameter is designed to control the ratio of coarse-to-fine sampling and achieve adaptive sampling of the refracted ray. Furthermore, to enhance underwater low-frequency view-dependent information, the spherical harmonics is adopted for directional position encoding of the refracted ray. Extensive experimental results on public and self-built datasets demonstrate the rendering quality and surface reconstruction accuracy in various refractive scenes with complex refractive surfaces.