Scholarlink

Objectives <p>Large language models (LLMs) are increasingly explored as decision-support tools in medical imaging. However, their ability to align with country-specific guidelines, which often diverge, remains uncertain. We set out to evaluate the geographic neutrality of three state-of-the-art LLMs—GPT-o3, Mistral Large, and DeepSeek R1—and a biomedical LLM (MedGemma 1.5 4B), when applied to neuroradiology scenarios with conflicting U.S. and non-U.S. recommendations.</p> Materials and methods <p>Vignettes derived from contradictory international guidelines were presented to each model under two conditions: an implicit setting, where no guideline was specified and vignettes were provided in English and French; and an explicit setting, where prompts directed models to follow a named guideline. Performance was reviewed against the target guideline, and mitigation strategies were tested.</p> Results <p>Thirty clinical vignettes presenting conflicting guidelines were evaluated by GPT-o3, Mistral Large, and DeepSeek R1. In the implicit setting, all models favored U.S. guidelines, with GPT-o3, Mistral, and DeepSeek aligning with them in 27 of 30 scenarios (90.0%; 95% CI, 74.4–96.5). In the explicit setting, adherence declined sharply for non-U.S. recommendations for all models. Providing the complete guideline text was the most effective mitigation strategy, restoring accuracies above 90% across all models.</p> Conclusion <p>Across languages and model origins, LLMs exhibited a systematic bias toward U.S. neuroradiology guidelines, even when explicitly instructed otherwise. This U.S.-centrism likely reflects training data imbalances and raises concerns for safe global deployment. Strategies for local contextualization, such as guideline integration at deployment, are necessary to ensure context-appropriate clinical decision support.</p> Key Points <p><Emphasis Type="BoldItalic">Question</Emphasis> <i>Do large language models display geographical neutrality in neuroradiology decision support?</i></p> <p><Emphasis Type="BoldItalic">Findings</Emphasis> <i>Even models developed in France and China systematically preferred United States guidelines, aligning with them in most implicit scenarios while failing to follow explicit guidelines from other sources.</i></p> <p><Emphasis Type="BoldItalic">Clinical relevance</Emphasis> <i>This systematic United States-centric bias poses clinical and legal risks for global deployment. Safe implementation requires specific localization strategies, such as providing full guideline texts, to ensure recommendations align with local practice standards.</i></p> Graphical Abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cultural bias in large language models’ ability to follow neuroradiology guidelines

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月26日

查看全文
  • Naël Bazerbachi,
  • Raphaël Bentegeac,
  • Giada Pistilli,
  • Su-Hwan Kim,
  • Philippe Amouyel,
  • Jean-Pierre Pruvo,
  • Lotfi Hacein-Bey,
  • Aghiles Hamroun,
  • Grégory Kuchcinski,
  • Bastien Le Guellec

摘要

Objectives

Large language models (LLMs) are increasingly explored as decision-support tools in medical imaging. However, their ability to align with country-specific guidelines, which often diverge, remains uncertain. We set out to evaluate the geographic neutrality of three state-of-the-art LLMs—GPT-o3, Mistral Large, and DeepSeek R1—and a biomedical LLM (MedGemma 1.5 4B), when applied to neuroradiology scenarios with conflicting U.S. and non-U.S. recommendations.

Materials and methods

Vignettes derived from contradictory international guidelines were presented to each model under two conditions: an implicit setting, where no guideline was specified and vignettes were provided in English and French; and an explicit setting, where prompts directed models to follow a named guideline. Performance was reviewed against the target guideline, and mitigation strategies were tested.

Results

Thirty clinical vignettes presenting conflicting guidelines were evaluated by GPT-o3, Mistral Large, and DeepSeek R1. In the implicit setting, all models favored U.S. guidelines, with GPT-o3, Mistral, and DeepSeek aligning with them in 27 of 30 scenarios (90.0%; 95% CI, 74.4–96.5). In the explicit setting, adherence declined sharply for non-U.S. recommendations for all models. Providing the complete guideline text was the most effective mitigation strategy, restoring accuracies above 90% across all models.

Conclusion

Across languages and model origins, LLMs exhibited a systematic bias toward U.S. neuroradiology guidelines, even when explicitly instructed otherwise. This U.S.-centrism likely reflects training data imbalances and raises concerns for safe global deployment. Strategies for local contextualization, such as guideline integration at deployment, are necessary to ensure context-appropriate clinical decision support.

Key Points

Question Do large language models display geographical neutrality in neuroradiology decision support?

Findings Even models developed in France and China systematically preferred United States guidelines, aligning with them in most implicit scenarios while failing to follow explicit guidelines from other sources.

Clinical relevance This systematic United States-centric bias poses clinical and legal risks for global deployment. Safe implementation requires specific localization strategies, such as providing full guideline texts, to ensure recommendations align with local practice standards.

Graphical Abstract