Visual Question Answering with Cascade of Self- and Co-Attention Blocks
摘要
Recent advancements in Visual Question Answering (VQA) have been driven by the integration of complex attention mechanisms. This work introduces a novel approach aimed at enhancing multi-modal representations through dense interactions between visual and textual inputs in an alternating fashion. The proposed model features an attention framework that incorporates both self-attention and co-attention mechanisms, strategically applied to image and text modalities. Self-attention modules capture contextual dependencies among objects in images and words in questions, crucial for accurate inference of answers. Meanwhile, co-attention mechanisms facilitate effective cross-modal interactions between images and text. To extract fine-grained information from both modalities, we introduce a Cascade of Self- and Co-Attention blocks (CSCA). This architecture is evaluated extensively on prominent benchmarks including VQA2.0, TDIUC, and GQA datasets. Experimental results, including comprehensive ablation studies, highlight the effectiveness of the model’s key components and the cascading nature of attention mechanisms in enhancing performance across diverse VQA tasks.