Scholarlink

In this study, we examine regional biases in Large Language Models (LLMs) by assessing their evaluations of U.S. state residents on work ethic and morality. We utilized four distinct models—two general-purpose (GPT-4o and DeepSeek-Chat V3) and two reasoning-focused (o3-mini and DeepSeek-Reasoner R1)—and collected 25 independent ratings per question for each state to support a broad statistical analysis. Our findings reveal that while general-purpose models offer fairly uniform evaluations across regions, reasoning-focused models—most notably the o3-mini—demonstrate increased rating variability and occasionally refuse to respond. The observed refusal patterns, which correlate with lower ratings for certain states, point to an implicit bias emerging from the model’s advanced reasoning mechanisms. Additionally, the low correlation between work ethic and morality ratings suggests that the biases are trait-specific, reflecting distinct cultural and regional stereotypes embedded within the training data. Overall, our research indicates that enhancements in reasoning capability do not inherently reduce bias and may, in fact, intensify specific preexisting stereotypes. These results call for the integration of robust bias detection and mitigation frameworks in LLM design, offering valuable guidance for researchers and practitioners dedicated to developing and using AI systems.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

New York, New York - Unraveling Bias in Large Language Models: Investigating Differences Between Standard and Reasoning-Based Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Marek Opuszko,
  • Paul Böhm

摘要

In this study, we examine regional biases in Large Language Models (LLMs) by assessing their evaluations of U.S. state residents on work ethic and morality. We utilized four distinct models—two general-purpose (GPT-4o and DeepSeek-Chat V3) and two reasoning-focused (o3-mini and DeepSeek-Reasoner R1)—and collected 25 independent ratings per question for each state to support a broad statistical analysis. Our findings reveal that while general-purpose models offer fairly uniform evaluations across regions, reasoning-focused models—most notably the o3-mini—demonstrate increased rating variability and occasionally refuse to respond. The observed refusal patterns, which correlate with lower ratings for certain states, point to an implicit bias emerging from the model’s advanced reasoning mechanisms. Additionally, the low correlation between work ethic and morality ratings suggests that the biases are trait-specific, reflecting distinct cultural and regional stereotypes embedded within the training data. Overall, our research indicates that enhancements in reasoning capability do not inherently reduce bias and may, in fact, intensify specific preexisting stereotypes. These results call for the integration of robust bias detection and mitigation frameworks in LLM design, offering valuable guidance for researchers and practitioners dedicated to developing and using AI systems.