How Effectively Can Off-the-Shelf Generative AI Tools Solve Circuit Design Problems?
摘要
Large language models (LLMs) are easy for students to access via off-the-shelf Generative AI tools such as ChatGPT-3.5 and ChatGPT-4o. The ability of these tools to effectively translate natural language into code has forced educators to rethink assessments in the context of Computer Science courses. In this study, we examine whether such tools can generate code in Verilog for circuit design problems. We analyse 153 Verilog questions from the instructional website HDLBits. These problems range from simple combinational circuits to complex sequential circuits. Our results indicate that off-the-shelf tools can often generate code successfully in their first attempts. Further, when the resulting code fails, these tools can often rectify errors by using error and warning messages from the HDLBits simulation results, with little or no additional human input. Thus, our results suggest that educators may need to rethink assessments involving circuit design problems.