Pretrained Language Models for Automatic Source Code Generation: Evaluation and Application
摘要
This study explores the capabilities of the ChatGPT artificial intelligence language model, with the GPT-3 architecture developed by OpenAI, in the task of generating JavaScript source code from Spanish instructions. Transformer language models, as exemplars of deep learning, are effective at learning contextual representations of words and phrases. This allows the model to understand not only individual programming terms but also how they combine into larger structures such as loops and functions. Through a set of unique programming feature requests from a selected set of cases prepared for this work, we examine the model’s ability to transcribe these high-level specifications into executable and functional source code. The results of the model were evaluated using a compiler, seeking an objective evaluation of the functionality of the generated code in predefined unit test cases. Overall, the model achieves 100% compilable code and 90% successful individual problem resolution. This work explores the intersection of AI and programming, paving the way for effective automation of code development from sentences in Spanish. While our analysis is centered on Spanish, the insights gained could be relevant for other non-English languages as well. It is expected that this work will contribute to the growing body of literature focusing on code generation and natural language understanding in AI language models.