Scholarlink

Hate memes pose a significant challenge for automated content moderation due to the implicit nature of hateful content conveyed through the interplay of text and images. To address this, we investigate multimodal fusion strategies, combining various vision and language models, and enhance text representations with image captions to improve contextual understanding. We evaluate six language models (BERT, HateBERT, RoBERTa, DistilBERT, ALBERT, XLM-R), paired with four vision models (Swin, ConvNeXt, ViT, EfficientNet), conducting a comprehensive analysis of their combinations. We also independently assess the performance of the CLIP model to examine its effectiveness in multimodal text-image understanding. Using the HarMeme dataset, which contains hateful memes related to COVID-19, we systematically compare these models and explore ensemble techniques to improve accuracy. Our research aims to identify the most effective combination of models for robust hate meme detection.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Hate Meme Detection Using Transformer-Based Text and Vision Models: A Multimodal Approach

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Urvashi Sinha,
  • Siddharth Goel,
  • Teeksha Harish,
  • Varsha Sisaudia

摘要

Hate memes pose a significant challenge for automated content moderation due to the implicit nature of hateful content conveyed through the interplay of text and images. To address this, we investigate multimodal fusion strategies, combining various vision and language models, and enhance text representations with image captions to improve contextual understanding. We evaluate six language models (BERT, HateBERT, RoBERTa, DistilBERT, ALBERT, XLM-R), paired with four vision models (Swin, ConvNeXt, ViT, EfficientNet), conducting a comprehensive analysis of their combinations. We also independently assess the performance of the CLIP model to examine its effectiveness in multimodal text-image understanding. Using the HarMeme dataset, which contains hateful memes related to COVID-19, we systematically compare these models and explore ensemble techniques to improve accuracy. Our research aims to identify the most effective combination of models for robust hate meme detection.