Research on Multidimensional Evaluation Technology of Teachers’ Digital Literacy for LLM as a Judge
摘要
This research explores the multidimensional evaluation technology for assessing teachers’ digital literacy using Large Language Models (LLMs) as judges. The study proposes a novel framework for teacher digital literacy evaluation, which integrates international evaluation standards based on the EU digcampedu framework. The framework includes three key modules: the data fusion module, the LLM-based evaluation module, and the feedback module. The data fusion module combines teacher data from various scenarios and dimensions into a unified metadata set, while the evaluation module inputs this data into the LLM, which maps evaluation dimensions and rules to assess teachers’ digital competencies. The feedback module then provides multidimensional evaluations and targeted improvement suggestions. The LLM-based approach offers two key advantages: scalability and interpretability. It reduces the need for human intervention and enables rapid, scalable assessments, while also offering clear explanations for the evaluation results. This makes the evaluation process more transparent and understandable for educators. By utilizing LLMs as judges, the model provides personalized, adaptive, and comprehensive evaluations of teachers’ digital literacy, ensuring alignment with international standards. The findings of this research contribute to the development of a robust and scalable framework for teacher digital literacy assessment, offering valuable insights for both educational practitioners and policymakers. Future research will focus on further refining the evaluation algorithms, incorporating dynamic tracking and personalized feedback systems, and expanding the model to address emerging challenges in digital literacy assessment. This study represents a significant step toward leveraging advanced AI technology to enhance teacher professional development and digital competency evaluation.