Scholarlink

<p>Multimodal large language models (MLLMs) could enhance the accuracy of automated content moderation by integrating contextual information. This study examines how MLLMs evaluate hate speech through a series of conjoint experiments. Models are provided with a hate speech policy and shown simulated social media posts that systematically vary in slur usage, user demographics and other attributes. The decisions from MLLMs are benchmarked against judgements by human participants (<i>n</i> = 1,854). The results demonstrate that larger, more advanced models can make context-sensitive evaluations that are closely aligned with human judgement. However, pervasive demographic and lexical biases remain, particularly among smaller models. Further analyses show that context sensitivity can be amplified via prompting but not eliminated, and that some models are especially responsive to visual identity cues. These findings highlight the benefits and risks of using MLLMs for content moderation and demonstrate the utility of conjoint experiments for auditing artificial intelligence in complex, context-dependent applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal large language models can make context-sensitive hate speech evaluations aligned with human judgement

  • 期刊论文

  • 发表日期：

    2025年12月15日

查看全文
  • Thomas Davidson

摘要

Multimodal large language models (MLLMs) could enhance the accuracy of automated content moderation by integrating contextual information. This study examines how MLLMs evaluate hate speech through a series of conjoint experiments. Models are provided with a hate speech policy and shown simulated social media posts that systematically vary in slur usage, user demographics and other attributes. The decisions from MLLMs are benchmarked against judgements by human participants (n = 1,854). The results demonstrate that larger, more advanced models can make context-sensitive evaluations that are closely aligned with human judgement. However, pervasive demographic and lexical biases remain, particularly among smaller models. Further analyses show that context sensitivity can be amplified via prompting but not eliminated, and that some models are especially responsive to visual identity cues. These findings highlight the benefits and risks of using MLLMs for content moderation and demonstrate the utility of conjoint experiments for auditing artificial intelligence in complex, context-dependent applications.