Scholarlink

<p>This study evaluated the performance of ChatGPT-4o in predicting the country and region of origin from personal names. A cross-sectional analysis was conducted using 2515 names from 63 countries of origin, generated by randomly combining popular first and last names from a public GitHub dataset. Predictions were obtained in two rounds, conducted five days apart (25 and 30 November 2024). Performance metrics included errorCoded (proportion of incorrect or non-classifications), errorCodedWithoutNA (proportion of incorrect classifications, excluding non-classifications), and naCoded (proportion of non-classifications). Cohen’s Kappa was used to assess agreement between rounds. ChatGPT correctly identified the country of origin for 69.7% and 70.3% of names in the first and second rounds, respectively (errorCoded = 0.303 and 0.297). Regional predictions were more accurate, with 95.8% of names correctly assigned in both rounds (errorCoded = 0.042). No non-classifications occurred (naCoded = 0), and agreement between rounds was high (Cohen’s Kappa: 0.841 for countries and 0.986 for regions). Performance varied by linguistic complexity, with high accuracy for linguistically distinct countries (e.g., Japan) and lower accuracy for countries with overlapping linguistic, cultural, or geographic features (e.g., English, French, and Spanish-speaking countries). In conclusion,&#xa0;ChatGPT-4o demonstrated high accuracy in predicting regions of origin and moderate performance at the country level when classifying names from a diverse international dataset. These findings highlight the potential and current limitations of large language models for name-based origin detection and suggest promising avenues for future applications in research, demography, and public health.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An analysis of ChatGPT’s performance in predicting countries and regions of origin from personal names

  • 期刊论文

  • 发表日期：

    2026年3月19日

查看全文
  • Paul Sebo

摘要

This study evaluated the performance of ChatGPT-4o in predicting the country and region of origin from personal names. A cross-sectional analysis was conducted using 2515 names from 63 countries of origin, generated by randomly combining popular first and last names from a public GitHub dataset. Predictions were obtained in two rounds, conducted five days apart (25 and 30 November 2024). Performance metrics included errorCoded (proportion of incorrect or non-classifications), errorCodedWithoutNA (proportion of incorrect classifications, excluding non-classifications), and naCoded (proportion of non-classifications). Cohen’s Kappa was used to assess agreement between rounds. ChatGPT correctly identified the country of origin for 69.7% and 70.3% of names in the first and second rounds, respectively (errorCoded = 0.303 and 0.297). Regional predictions were more accurate, with 95.8% of names correctly assigned in both rounds (errorCoded = 0.042). No non-classifications occurred (naCoded = 0), and agreement between rounds was high (Cohen’s Kappa: 0.841 for countries and 0.986 for regions). Performance varied by linguistic complexity, with high accuracy for linguistically distinct countries (e.g., Japan) and lower accuracy for countries with overlapping linguistic, cultural, or geographic features (e.g., English, French, and Spanish-speaking countries). In conclusion, ChatGPT-4o demonstrated high accuracy in predicting regions of origin and moderate performance at the country level when classifying names from a diverse international dataset. These findings highlight the potential and current limitations of large language models for name-based origin detection and suggest promising avenues for future applications in research, demography, and public health.