Dynamic Neural Radiance Fields (Dynamic NeRF) have become increasingly important for 3D scene reconstruction, particularly in modeling dynamic environments. However, they often face challenges in rendering high-frequency details with sufficient quality. To address this issue, we propose a novel frequency-aware approach using the Mixture of TriPlanes (MoT) framework. Inspired by the Mixture of Experts (MoE) paradigm, our method combines high-low frequency and dynamic-static triplanes, allowing for adaptive handling of different scene regions. This design enables specialized triplanes to process varying frequency and temporal components, resulting in more precise and flexible 3D reconstructions. Additionally, we propose a frequency-based feature fusion mechanism that dynamically adjusts weights for blending high and low-frequency information, improving the representation of complex scenes. Extensive experiments validate the effectiveness of our approach, demonstrating significant improvements over existing dynamic NeRF methods, particularly in capturing high-frequency details and dynamic elements.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MoT: A Mixture of TriPlanes Framework for Frequency-Aware Dynamic Neural Radiance Fields

  • Zhiwei Liu,
  • Shengfan Wang,
  • Fei Hu,
  • Wei Zhong,
  • Li Fang

摘要

Dynamic Neural Radiance Fields (Dynamic NeRF) have become increasingly important for 3D scene reconstruction, particularly in modeling dynamic environments. However, they often face challenges in rendering high-frequency details with sufficient quality. To address this issue, we propose a novel frequency-aware approach using the Mixture of TriPlanes (MoT) framework. Inspired by the Mixture of Experts (MoE) paradigm, our method combines high-low frequency and dynamic-static triplanes, allowing for adaptive handling of different scene regions. This design enables specialized triplanes to process varying frequency and temporal components, resulting in more precise and flexible 3D reconstructions. Additionally, we propose a frequency-based feature fusion mechanism that dynamically adjusts weights for blending high and low-frequency information, improving the representation of complex scenes. Extensive experiments validate the effectiveness of our approach, demonstrating significant improvements over existing dynamic NeRF methods, particularly in capturing high-frequency details and dynamic elements.