Scholarlink

<p>There are currently few established scales for evaluating learning progression in generative artificial intelligence (GenAI). This study validates an instrument designed to assess the progression of the GenAI concept using a diverse cohort of 635 participants in Hong Kong, including 355 adolescents (under 18) and 280 adults (over 18), comprising students, teachers, parents, and administrative staff. The findings indicate that a three-parameter logistic model provides the best fit for the instrument when compared to other Item Response Theory (IRT) models. Analysis of the difficulty, discrimination, and pseudo-guessing parameter confirms that the instrument effectively differentiates individuals with various abilities. The study’s contribution is the validated Generative Artificial Intelligence Concept Test—a practical and sensitive tool applicable for assessing GenAI learning achievements across both adolescents and adults. To evaluate educational impact, a paired statistical analysis comparing performance before and after a 30-h GenAI literacy course revealed significant learning progression because of the intervention. Both groups demonstrated notable improvements, with adults showing greater improvement than adolescents. The findings highlight an urgent need to unleash the potential of adults and prioritise GenAI literacy by deepening their understanding and application of these technologies.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Validation of a Generative Artificial Intelligence Learning Progression Instrument Using Item Response Theory: Unleashing the Potential of Adult Learners

  • 期刊论文

  • 发表日期：

    2026年7月4日

查看全文
  • Siu Cheung Kong,
  • Chunyu Hou

摘要

There are currently few established scales for evaluating learning progression in generative artificial intelligence (GenAI). This study validates an instrument designed to assess the progression of the GenAI concept using a diverse cohort of 635 participants in Hong Kong, including 355 adolescents (under 18) and 280 adults (over 18), comprising students, teachers, parents, and administrative staff. The findings indicate that a three-parameter logistic model provides the best fit for the instrument when compared to other Item Response Theory (IRT) models. Analysis of the difficulty, discrimination, and pseudo-guessing parameter confirms that the instrument effectively differentiates individuals with various abilities. The study’s contribution is the validated Generative Artificial Intelligence Concept Test—a practical and sensitive tool applicable for assessing GenAI learning achievements across both adolescents and adults. To evaluate educational impact, a paired statistical analysis comparing performance before and after a 30-h GenAI literacy course revealed significant learning progression because of the intervention. Both groups demonstrated notable improvements, with adults showing greater improvement than adolescents. The findings highlight an urgent need to unleash the potential of adults and prioritise GenAI literacy by deepening their understanding and application of these technologies.