<p>In this paper, we propose ASTCAN, an adaptive sparse triple convolutional attention network, designed to enhance visual question answering (VQA) by introducing innovative modifications to the standard Transformer architecture. Traditional VQA models often struggle with noise interference from irrelevant regions due to their inability to dynamically filter out extraneous features. ASTCAN addresses this limitation through an adaptive threshold sparse attention mechanism, which dynamically filters irrelevant features during training, significantly improving focus and efficiency. Additionally, we introduce a triple convolutional attention module, which extends the Transformer by capturing cross-dimensional interactions between spatial and channel features, further enhancing the model’s reasoning ability. Extensive experiments on benchmark datasets demonstrate that ASTCAN outperforms most existing end-to-end methods, particularly in scenarios without pre-training, highlighting its effectiveness and potential for real-world applications. The code and datasets are publicly available to facilitate reproducibility and further research.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive sparse triple convolutional attention for enhanced visual question answering

  • Ronggui Wang,
  • Hong Chen,
  • Juan Yang,
  • Lixia Xue

摘要

In this paper, we propose ASTCAN, an adaptive sparse triple convolutional attention network, designed to enhance visual question answering (VQA) by introducing innovative modifications to the standard Transformer architecture. Traditional VQA models often struggle with noise interference from irrelevant regions due to their inability to dynamically filter out extraneous features. ASTCAN addresses this limitation through an adaptive threshold sparse attention mechanism, which dynamically filters irrelevant features during training, significantly improving focus and efficiency. Additionally, we introduce a triple convolutional attention module, which extends the Transformer by capturing cross-dimensional interactions between spatial and channel features, further enhancing the model’s reasoning ability. Extensive experiments on benchmark datasets demonstrate that ASTCAN outperforms most existing end-to-end methods, particularly in scenarios without pre-training, highlighting its effectiveness and potential for real-world applications. The code and datasets are publicly available to facilitate reproducibility and further research.