The pandemic shift to distance learning has revealed a lack of reliable tools for online assessment of learning outcomes. Existing distance learning platforms do not provide effective tools for mass and objective assessment of competences. This paper proposes the use of an intelligent assessment system that combines the massiveness of computer-based testing and the personalised approach of an oral examination. The aim of the research is to develop an adaptive methodology for assessing the level of students’ competence formation in one testing session. The methodology is based on the modernised Rasch model and uses a Bayesian algorithm to classify students according to the level of competence formation. After each task, the probability of a student belonging to certain partners (sets of values of formed competences) is calculated. In order to speed up the estimation process, an adaptive job selection is used, oriented to maximise the information function for the partner whose probability of affiliation to the previous step was maximal. To improve the stability of the Bayesian algorithm, a regularisation method based on the analysis of the change in the entropy of the distribution of the probability of belonging to partners is used. In case of atypical entropy change, it is suggested to re-run the task with the same difficulty parameters. This is analogous to asking a clarifying question during a face-to-face exam, which increases the robustness of the Bayesian algorithm. The simulation experiment has shown that the developed model allows us to reliably measure the level of competences formed during one testing session by performing several tasks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Digital Transformation of the Procedure for Assessing Competencies in a Pandemic

  • Victor N. Gusyatnikov,
  • Tatyana N. Sokolova,
  • Aleksey I. Bezrukov,
  • Inna V. Kayukova

摘要

The pandemic shift to distance learning has revealed a lack of reliable tools for online assessment of learning outcomes. Existing distance learning platforms do not provide effective tools for mass and objective assessment of competences. This paper proposes the use of an intelligent assessment system that combines the massiveness of computer-based testing and the personalised approach of an oral examination. The aim of the research is to develop an adaptive methodology for assessing the level of students’ competence formation in one testing session. The methodology is based on the modernised Rasch model and uses a Bayesian algorithm to classify students according to the level of competence formation. After each task, the probability of a student belonging to certain partners (sets of values of formed competences) is calculated. In order to speed up the estimation process, an adaptive job selection is used, oriented to maximise the information function for the partner whose probability of affiliation to the previous step was maximal. To improve the stability of the Bayesian algorithm, a regularisation method based on the analysis of the change in the entropy of the distribution of the probability of belonging to partners is used. In case of atypical entropy change, it is suggested to re-run the task with the same difficulty parameters. This is analogous to asking a clarifying question during a face-to-face exam, which increases the robustness of the Bayesian algorithm. The simulation experiment has shown that the developed model allows us to reliably measure the level of competences formed during one testing session by performing several tasks.