<p>The existing Neural Radiance Fields (NeRF) uses deep learning only for the part that infers density during the sampling-network-rendering process. It is known that learning by configuring end-to-end can achieve better performance than learning only part of it. This paper proposes a transformer rendering technique that does not require multi-views, unlike the existing NeRF method using a transformer structure. The structure is built to mimic the volume rendering technique used in NeRF and can be used as a replacement for volume rendering. When used, the network shows color composition ability similar to NeRF and about 6% improvement in depth inference ability. In particular, it shows that the depth inference ability is about 35% better, even when the neural field network is small. Although it has the disadvantage of increasing learning parameters, it has the advantage of reducing actual memory usage by about 36%. This indicates that the rendering process can be sufficiently replaced with a network.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Neural radiance fields with transformer rendering

  • Jin-Woo Kim,
  • Jong-Eun Ha

摘要

The existing Neural Radiance Fields (NeRF) uses deep learning only for the part that infers density during the sampling-network-rendering process. It is known that learning by configuring end-to-end can achieve better performance than learning only part of it. This paper proposes a transformer rendering technique that does not require multi-views, unlike the existing NeRF method using a transformer structure. The structure is built to mimic the volume rendering technique used in NeRF and can be used as a replacement for volume rendering. When used, the network shows color composition ability similar to NeRF and about 6% improvement in depth inference ability. In particular, it shows that the depth inference ability is about 35% better, even when the neural field network is small. Although it has the disadvantage of increasing learning parameters, it has the advantage of reducing actual memory usage by about 36%. This indicates that the rendering process can be sufficiently replaced with a network.