<p>In large-scale teaching scenarios, the problems of inefficient evaluation of virtual simulation training process and low quality of teaching intervention are still outstanding; such issues make it difficult to effectively support the implementation of normalized training. Traditional automatic scoring methods mostly rely on pure data-driven models. These models are easy to learn the statistical correlation in behavioral data; however, they ignore normative logic (e.g., step dependence, sequence constraints and safety taboos). Thus, this leads to the deviation between scoring results and training norms; meanwhile, relevant feedback lacks traceable evidence. In addition, manual playback and manual scoring methods have limitations such as high costs, long cycles, and strong subjectivity. In order to solve the above problems, this study builds an end-to-end closed-loop framework of “log standardization-neural scoring-rule verification-diagnosis attribution-retrieval enhancement-controllable generation; this can achieve a consistent linkage between scoring results and feedback content. From the viewpoint of method positioning, this study proposes a neural-symbol fusion method for process evaluation tasks. By combining sequence representation learning with explicit symbolic rule constraints, the accuracy, standard consistency and feedback support ability of process scoring are improved. Considering the lack of publicly available standardized log data for virtual simulation training processes, this study selects EdNet-KT1 as a surrogate process log dataset with temporal behavioral features for preliminary verification of the proposed method. Experimental results show that in three data subsets, the proposed model’s mean absolute error is 0.13, 0.14 and 0.16; the root mean square error reaches 0.21, 0.23 and 0.22; the Spearman rank correlation coefficients are 0.79, 0.77 and 0.78, respectively. In the ranking task of key error positions, the average accuracy records 0.71, 0.69 and 0.73; the performance is better than the comparison model as a whole. The findings reveal that the proposed method can effectively reduce the scoring error and improve the consistency of process quality ranking; it also enhances the ability of identifying and prioritizing key error positions. Further analysis indicates that the introduction of symbolic rule constraints helps the model to better deal with the problems of step dependence and specification consistency; it thus enhances the structural correspondence between scoring decisions and feedback generation. Therefore, this study can offer some reference for the evaluation of virtual simulation training process and the generation of intelligent teaching feedback.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic scoring and feedback generation system for virtual simulation training process with integrated neuro-symbolic reasoning

  • Liping Dai,
  • Chunxiang Fan,
  • Lei Lu

摘要

In large-scale teaching scenarios, the problems of inefficient evaluation of virtual simulation training process and low quality of teaching intervention are still outstanding; such issues make it difficult to effectively support the implementation of normalized training. Traditional automatic scoring methods mostly rely on pure data-driven models. These models are easy to learn the statistical correlation in behavioral data; however, they ignore normative logic (e.g., step dependence, sequence constraints and safety taboos). Thus, this leads to the deviation between scoring results and training norms; meanwhile, relevant feedback lacks traceable evidence. In addition, manual playback and manual scoring methods have limitations such as high costs, long cycles, and strong subjectivity. In order to solve the above problems, this study builds an end-to-end closed-loop framework of “log standardization-neural scoring-rule verification-diagnosis attribution-retrieval enhancement-controllable generation; this can achieve a consistent linkage between scoring results and feedback content. From the viewpoint of method positioning, this study proposes a neural-symbol fusion method for process evaluation tasks. By combining sequence representation learning with explicit symbolic rule constraints, the accuracy, standard consistency and feedback support ability of process scoring are improved. Considering the lack of publicly available standardized log data for virtual simulation training processes, this study selects EdNet-KT1 as a surrogate process log dataset with temporal behavioral features for preliminary verification of the proposed method. Experimental results show that in three data subsets, the proposed model’s mean absolute error is 0.13, 0.14 and 0.16; the root mean square error reaches 0.21, 0.23 and 0.22; the Spearman rank correlation coefficients are 0.79, 0.77 and 0.78, respectively. In the ranking task of key error positions, the average accuracy records 0.71, 0.69 and 0.73; the performance is better than the comparison model as a whole. The findings reveal that the proposed method can effectively reduce the scoring error and improve the consistency of process quality ranking; it also enhances the ability of identifying and prioritizing key error positions. Further analysis indicates that the introduction of symbolic rule constraints helps the model to better deal with the problems of step dependence and specification consistency; it thus enhances the structural correspondence between scoring decisions and feedback generation. Therefore, this study can offer some reference for the evaluation of virtual simulation training process and the generation of intelligent teaching feedback.