This study explores the capabilities of the ChatGPT artificial intelligence language model, with the GPT-3 architecture developed by OpenAI, in the task of generating JavaScript source code from Spanish instructions. Transformer language models, as exemplars of deep learning, are effective at learning contextual representations of words and phrases. This allows the model to understand not only individual programming terms but also how they combine into larger structures such as loops and functions. Through a set of unique programming feature requests from a selected set of cases prepared for this work, we examine the model’s ability to transcribe these high-level specifications into executable and functional source code. The results of the model were evaluated using a compiler, seeking an objective evaluation of the functionality of the generated code in predefined unit test cases. Overall, the model achieves 100% compilable code and 90% successful individual problem resolution. This work explores the intersection of AI and programming, paving the way for effective automation of code development from sentences in Spanish. While our analysis is centered on Spanish, the insights gained could be relevant for other non-English languages as well. It is expected that this work will contribute to the growing body of literature focusing on code generation and natural language understanding in AI language models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Pretrained Language Models for Automatic Source Code Generation: Evaluation and Application

  • Adrian Bender,
  • Santiago Nicolet

摘要

This study explores the capabilities of the ChatGPT artificial intelligence language model, with the GPT-3 architecture developed by OpenAI, in the task of generating JavaScript source code from Spanish instructions. Transformer language models, as exemplars of deep learning, are effective at learning contextual representations of words and phrases. This allows the model to understand not only individual programming terms but also how they combine into larger structures such as loops and functions. Through a set of unique programming feature requests from a selected set of cases prepared for this work, we examine the model’s ability to transcribe these high-level specifications into executable and functional source code. The results of the model were evaluated using a compiler, seeking an objective evaluation of the functionality of the generated code in predefined unit test cases. Overall, the model achieves 100% compilable code and 90% successful individual problem resolution. This work explores the intersection of AI and programming, paving the way for effective automation of code development from sentences in Spanish. While our analysis is centered on Spanish, the insights gained could be relevant for other non-English languages as well. It is expected that this work will contribute to the growing body of literature focusing on code generation and natural language understanding in AI language models.