<p>Structured query language (SQL) queries are an important aspect of database concepts in the information technology (IT) domain. Evaluation of SQL queries ensures that the learners can understand and apply various SQL concepts correctly. However, this can be a laborious task when carried out manually by course instructors at universities, which often does not scale well. To address these limitations, this study proposes a web-based application, SQL autograder, which can be used by instructors of a university course to evaluate assessments and enhance the quality of education and learning outcomes. We propose a framework that makes use of large language models (LLMs) to assess the correctness of SQL queries submitted by students. We train a variety of open-source LLMs of varying sizes on a diverse dataset of SQL queries, with queries ranging from simple ones that include a single JOIN statement to more complex ones involving multiple SQL features. We implemented and tested our LLM-based framework in real-world educational settings for a university course, which shows promising results in enhancing the learning experience for students by providing instant feedback on areas needing improvement. We tested our application on 88 participants and found that the autograder is 180x faster than the instructor, with an average accuracy of 96.77%. After taking the qualitative feedback from the participants, 97% of them found it to be useful. The proposed framework reduces the workload of instructors by offering a more scalable and consistent evaluation process that enhances the performance of students.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SQL Autograder: Web-based LLM-powered Autograder for Assessment of SQL Queries

  • Karan Manikani,
  • Radhika Chapaneri,
  • Dharmik Shetty,
  • Divyata Shah

摘要

Structured query language (SQL) queries are an important aspect of database concepts in the information technology (IT) domain. Evaluation of SQL queries ensures that the learners can understand and apply various SQL concepts correctly. However, this can be a laborious task when carried out manually by course instructors at universities, which often does not scale well. To address these limitations, this study proposes a web-based application, SQL autograder, which can be used by instructors of a university course to evaluate assessments and enhance the quality of education and learning outcomes. We propose a framework that makes use of large language models (LLMs) to assess the correctness of SQL queries submitted by students. We train a variety of open-source LLMs of varying sizes on a diverse dataset of SQL queries, with queries ranging from simple ones that include a single JOIN statement to more complex ones involving multiple SQL features. We implemented and tested our LLM-based framework in real-world educational settings for a university course, which shows promising results in enhancing the learning experience for students by providing instant feedback on areas needing improvement. We tested our application on 88 participants and found that the autograder is 180x faster than the instructor, with an average accuracy of 96.77%. After taking the qualitative feedback from the participants, 97% of them found it to be useful. The proposed framework reduces the workload of instructors by offering a more scalable and consistent evaluation process that enhances the performance of students.