ARDN: Attention Re-distribution Network for Visual Question Answering

The Transformer-based architecture, with its efficient parallel computation, long-range dependency modeling, and context-aware capabilities, has showcased remarkable performance in various single-modal tasks. However, when dealing with complex multi-modal tasks like Visual Question Answering (VQA), the conventional Transformer-based attention mechanism encounters challenges in learning optimal weight distributions for intricate modal image tokens customized to specific tasks, resulting in suboptimal performance. To tackle this concern, we introduce an innovative approach known as the Attention Re-distribution Network (ARDN), tailored specifically for addressing VQA challenges. ARDN reallocates token weights via dual strategies, emphasizing salient visual elements for enhanced attention. Within the indirect approach, we leverage the exponential function to smoothen the distribution of feature maps utilized for calculating image token self-correlation. This empowers the model to extract meaningful information even from seemingly unrelated tokens, thus promoting the rationality of token distribution and the overall robustness of the model. Differing from the indirect approach, the direct approach introduces a temperature coefficient, adjusting image token weights directly, ensuring a more rational distribution. Experimental results showcase ARDN’s superiority in VQA tasks on VQA-v2 and GQA datasets, outperforming conventional Transformer-based methods. Enhanced attention distribution improves information capture and reasoning, leading to significant performance enhancements. Source code is available at https://github.com/yassels/ARDN.