The test-case-based approach is widely used for automated evaluation of students’ lab submissions in Data Structures and Algorithms (e.g., HackerRank, CodeChef). However, this method has key limitations: distinguishing between O(log n) and O(n) time solutions requires large test cases, and even optimal solutions fail sometimes under high server load conditions. As a result, an alternative method for evaluating time complexity is necessary. This study explores the potential of using a Large Language Model (LLM), specifically ChatGPT-3.5, for determining the time complexity of student programs. Using a dataset from four DSA lab problems with 259 student submissions, we compared ChatGPT-3.5’s predictions with the manually calculated ground truth. Our findings demonstrate that the ChatGPT-3.5-based approach achieves an average accuracy of 99.6% in correctly identifying time complexity. This suggests that an LLM-based method can effectively replace the large test cases typically required to determine the time complexity of submitted solutions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Leveraging ChatGPT-3.5 for Automated Evaluation of Time Complexity in Programming Assignments

  • Raja Kumar,
  • Sushant Phalke,
  • Deependra Singh Rajput,
  • Sujith Thomas

摘要

The test-case-based approach is widely used for automated evaluation of students’ lab submissions in Data Structures and Algorithms (e.g., HackerRank, CodeChef). However, this method has key limitations: distinguishing between O(log n) and O(n) time solutions requires large test cases, and even optimal solutions fail sometimes under high server load conditions. As a result, an alternative method for evaluating time complexity is necessary. This study explores the potential of using a Large Language Model (LLM), specifically ChatGPT-3.5, for determining the time complexity of student programs. Using a dataset from four DSA lab problems with 259 student submissions, we compared ChatGPT-3.5’s predictions with the manually calculated ground truth. Our findings demonstrate that the ChatGPT-3.5-based approach achieves an average accuracy of 99.6% in correctly identifying time complexity. This suggests that an LLM-based method can effectively replace the large test cases typically required to determine the time complexity of submitted solutions.