An Automated Hybrid Exam Evaluation Framework for Textual Courses Using AI
摘要
Evaluation and assessment systems stand to benefit from Artificial Intelligence (AI) driven automated open-ended exam grading technologies. However, despite remarkable achievements in Natural Language Processing (NLP) research, streamlining of those achievements into production-quality systems has yet to be seen. In this paper we present the specification, design and a prototype implementation of an open, extensible and comprehensive software platform and illustrate its performance via the results of our initial experimentation with real data obtained in collaboration with an operating college in Turkey. Our system treats question authoring, answer specification design, key (or reference) answer association, answer collection, evaluation, and grading processes holistically in a single framework, including the maintenance and management of all the data generated during those processes.