DAME-Cap: a dual-teacher and parallel attention–mambablock framework for image captioning
摘要
Image captioning aims to generate natural language descriptions for visual content by bridging computer vision and natural language processing. Recent large-scale visual-language pre-training models and Transformer-based architectures have significantly improved image captioning performance. However, the self-attention mechanism in Transformers struggles to balance modeling global dependencies with capturing local semantic continuity. Furthermore, existing methods commonly face the issue of supervision signals being biased toward a single modality during cross-modal training, resulting in insufficient visual-linguistic alignment. To address these limitations, we propose the Parallel Attention-MambaBlock module, which fuses the global context modeling capability of self-attention with the fine-grained sequence representation advantage of MambaBlock to achieve complementary feature learning. Simultaneously, we introduce a dual-teacher distillation framework that leverages visual and linguistic teachers to provide balanced and complementary supervision signals, comprehensively enhancing cross-modal representation capabilities. Finally, we develop an adaptive gated fusion mechanism that dynamically balances the weights of attention and MambaBlock outputs. This mechanism enables the model to adjust fusion weights based on contextual demands, efficiently integrating their optimized outputs. Extensive experiments on the MSCOCO benchmark demonstrate that our DAME-Cap model outperforms most state-of-the-art models.