<p>Implicit neural representation (INR) has recently emerged as a promising paradigm for signal representation and processing, demonstrating outstanding performance across numerous computer vision tasks. However, existing multi-layer perceptron based INR models face two inherent issues: (i) individually consider each coordinate while neglecting the spatial relations, thus impairing the representational capacity; (ii) learn the parameters directly while ignoring the connections between weights and inputs of different channels, thus hindering better model expressiveness. Thus, this paper proposes to introduce attention mechanisms into INR models, termed as INRAM, to capture more comprehensive and context-aware representations. Specifically, INRAM introduces Spatial Recombination Attention to make the model attend to relevant locations and recombine global spatial information. Moreover, Channel Recalibration Attention mechanism is designed to adjust the model weights of different channels according to the dynamic inputs. In this way, INRAM can enhance the representational capacity by capturing spatial and channel relations. Extensive experiments are conducted on three tasks, including image fitting and compression, computed tomography reconstruction, and magnetic resonance imaging. It is demonstrated that INRAM can achieve superior performance compared to the existing INR methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

INRAM: Implicit Neural Representation with Attention Mechanism

  • Chengyang Yan,
  • Zhicheng Cai,
  • Hao Zhu

摘要

Implicit neural representation (INR) has recently emerged as a promising paradigm for signal representation and processing, demonstrating outstanding performance across numerous computer vision tasks. However, existing multi-layer perceptron based INR models face two inherent issues: (i) individually consider each coordinate while neglecting the spatial relations, thus impairing the representational capacity; (ii) learn the parameters directly while ignoring the connections between weights and inputs of different channels, thus hindering better model expressiveness. Thus, this paper proposes to introduce attention mechanisms into INR models, termed as INRAM, to capture more comprehensive and context-aware representations. Specifically, INRAM introduces Spatial Recombination Attention to make the model attend to relevant locations and recombine global spatial information. Moreover, Channel Recalibration Attention mechanism is designed to adjust the model weights of different channels according to the dynamic inputs. In this way, INRAM can enhance the representational capacity by capturing spatial and channel relations. Extensive experiments are conducted on three tasks, including image fitting and compression, computed tomography reconstruction, and magnetic resonance imaging. It is demonstrated that INRAM can achieve superior performance compared to the existing INR methods.