This chapter reports on a novel approach to estimating word difficulty based on a Large Language Model (LLM). In the present study, we prompted GPT-4o to determine the relative difficulty of English words for Japanese learners. By fine-tuning the GPT-4o model with Rasch item difficulty data from the Updated Vocabulary Levels Test (Webb, S., Sasao, Y., & Ballance, O., The updated vocabulary levels test: Developing and validating two new forms of the VLT. ITL—International Journal of Applied Linguistics, 168(1), 33–69, 2017), we achieved an overall prediction accuracy of approximately 90% for pairwise comparisons of word difficulty. A neural network model was constructed to convert the data of these pairwise comparisons into difficulty scores for individual words that are considered to be equivalent to Rasch item difficulty estimates. The results showed that 83% of the word difficulty predictions fell within the 99% confidence interval. This suggests that with a representative dataset of word difficulties derived from human responses, LLMs have the potential to reliably quantify the difficulty of a wider range of words, which is expected to have pedagogical value in areas such as vocabulary acquisition, curriculum design, materials development, and language assessment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

New Directions in Vocabulary Learning and Teaching

  • Yosuke Sasao,
  • Zhen Liang

摘要

This chapter reports on a novel approach to estimating word difficulty based on a Large Language Model (LLM). In the present study, we prompted GPT-4o to determine the relative difficulty of English words for Japanese learners. By fine-tuning the GPT-4o model with Rasch item difficulty data from the Updated Vocabulary Levels Test (Webb, S., Sasao, Y., & Ballance, O., The updated vocabulary levels test: Developing and validating two new forms of the VLT. ITL—International Journal of Applied Linguistics, 168(1), 33–69, 2017), we achieved an overall prediction accuracy of approximately 90% for pairwise comparisons of word difficulty. A neural network model was constructed to convert the data of these pairwise comparisons into difficulty scores for individual words that are considered to be equivalent to Rasch item difficulty estimates. The results showed that 83% of the word difficulty predictions fell within the 99% confidence interval. This suggests that with a representative dataset of word difficulties derived from human responses, LLMs have the potential to reliably quantify the difficulty of a wider range of words, which is expected to have pedagogical value in areas such as vocabulary acquisition, curriculum design, materials development, and language assessment.