Scholarlink

This paper evaluates DeepSeek-R1’s performance on Brazil’s National High School Exam (ENEM), a culturally-specific educational benchmark that tests reasoning across multiple domains. Using the pass@k metric across three years (2022–2024) of ENEM questions, we found strong reasoning capabilities, particularly in Human Sciences, where 2023–2024 performance exceeded 0.98 pass@1 score. Mathematics showed the most variability, with diverse scores across the three years. The model demonstrated sophisticated self-translation capabilities when handling Brazilian Portuguese language questions without explicit translation instructions. Despite strong overall performance, inconsistencies across subject domains persist. Our findings suggest that recent advances in AI reasoning extend effectively beyond typical AI benchmarks to diverse cultural contexts, with implications for AI deployment in various global settings. This evaluation contributes to understanding how generative AI systems perform when facing multidisciplinary, culturally-situated reasoning challenges.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Assessing DeepSeek-R1’s Performance on Brazil’s Defining National Education Benchmark

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Alexandre Thurow Bender,
  • Gabriel Almeida Gomes,
  • Ulisses Brisolara Corrêa,
  • Ricardo Matsumura Araujo

摘要

This paper evaluates DeepSeek-R1’s performance on Brazil’s National High School Exam (ENEM), a culturally-specific educational benchmark that tests reasoning across multiple domains. Using the pass@k metric across three years (2022–2024) of ENEM questions, we found strong reasoning capabilities, particularly in Human Sciences, where 2023–2024 performance exceeded 0.98 pass@1 score. Mathematics showed the most variability, with diverse scores across the three years. The model demonstrated sophisticated self-translation capabilities when handling Brazilian Portuguese language questions without explicit translation instructions. Despite strong overall performance, inconsistencies across subject domains persist. Our findings suggest that recent advances in AI reasoning extend effectively beyond typical AI benchmarks to diverse cultural contexts, with implications for AI deployment in various global settings. This evaluation contributes to understanding how generative AI systems perform when facing multidisciplinary, culturally-situated reasoning challenges.