Scholarlink

<p>The rapid spread of offensive and violent speech over social media platforms adds a significant threat to community harmony. In particular, hateful memes are multimodal artefacts that combine images with text to convey implicit or sarcastic hate cues that remain difficult to detect because they often bypass traditional unimodal detection methods. To address this problem, we proposed Dynamic Dual Cross-Attention Fusion (DDCAF), a novel multimodal framework for detecting hate speech that integrates profound semantic comprehension from both visual and textual modalities. It uses a dual-stream architecture consisting of a RoBERTa-based text encoder and a Vision Transformer-based image encoder. Using a bidirectional cross-attention mechanism, the model dynamically computes text-guided visual attention and visual-guided text attention, enabling it to prioritize semantically aligned features across modalities. The dynamic attention-driven fusion mechanism is able to identify subtle, context-dependent cues of hate intent. The proposed framework is primarily tested on multimodal benchmark datasets such as Hateful Memes and MMHS150K, while also assessing unimodal baselines (HateEval and OLID). The experimental findings reveal that it surpasses existing approaches, providing an accuracy of 89.35% on Hateful Memes and 91.20% on MMHS150K. Furthermore, ablation studies are carried out to demonstrate the impact of the subcomponent within the DDCAF, which emphasises the value of dynamic adaptive gating in capturing intermodal dependencies.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DDCAF: dynamic dual cross-attention fusion framework for multimodal hate speech detection

  • 期刊论文

  • 发表日期：

    2026年5月16日

查看全文
  • Gauri Kitukale,
  • Navneet Pratap Singh,
  • Sidharth Quamara

摘要

The rapid spread of offensive and violent speech over social media platforms adds a significant threat to community harmony. In particular, hateful memes are multimodal artefacts that combine images with text to convey implicit or sarcastic hate cues that remain difficult to detect because they often bypass traditional unimodal detection methods. To address this problem, we proposed Dynamic Dual Cross-Attention Fusion (DDCAF), a novel multimodal framework for detecting hate speech that integrates profound semantic comprehension from both visual and textual modalities. It uses a dual-stream architecture consisting of a RoBERTa-based text encoder and a Vision Transformer-based image encoder. Using a bidirectional cross-attention mechanism, the model dynamically computes text-guided visual attention and visual-guided text attention, enabling it to prioritize semantically aligned features across modalities. The dynamic attention-driven fusion mechanism is able to identify subtle, context-dependent cues of hate intent. The proposed framework is primarily tested on multimodal benchmark datasets such as Hateful Memes and MMHS150K, while also assessing unimodal baselines (HateEval and OLID). The experimental findings reveal that it surpasses existing approaches, providing an accuracy of 89.35% on Hateful Memes and 91.20% on MMHS150K. Furthermore, ablation studies are carried out to demonstrate the impact of the subcomponent within the DDCAF, which emphasises the value of dynamic adaptive gating in capturing intermodal dependencies.