Scholarlink

<p>With the rapid development of IT technology, the software information industry is growing rapidly in the world. As a core function and basic component of an integrated development environment, code completion can greatly improve software development efficiency and has always been a research hotspot in the field of software development. To predict the next token of a code snippet, existing code-completion research focuses on modeling the code snippet itself and its abstract syntax tree (AST), but ignores the position information of leaf nodes in the AST. Therefore, This paper proposes a code completion model based on rotary position embedding (RoPE), named CRPE, which aims to encode the position information of leaf nodes through RoPE, thereby effectively improving the accuracy of predicting the next token. CRPE model first utilizes an LSTM model to extract information from the root path of leaf nodes, which is then fused with leaf node information to obtain code representations. Next, RoPE injects position information from leaf nodes into the code representations in the form of a rotation matrix within the attention mechanism. Experimental results on two major public datasets in Python and JavaScript show that, compared with the baseline model, the CRPE model achieves the best performance in both Top-k and MRR indicators; in addition, the RoPE in the CRPE model performs better than the Absolute Position Embedding and Relative Position Embedding, which provide better results.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Advancing code completion through rotary position embedding

  • 期刊论文

  • 发表日期：

    2025年7月31日

查看全文
  • Yiming Yin,
  • Jianxun Liu,
  • Yi Liu,
  • Jia Deng

摘要

With the rapid development of IT technology, the software information industry is growing rapidly in the world. As a core function and basic component of an integrated development environment, code completion can greatly improve software development efficiency and has always been a research hotspot in the field of software development. To predict the next token of a code snippet, existing code-completion research focuses on modeling the code snippet itself and its abstract syntax tree (AST), but ignores the position information of leaf nodes in the AST. Therefore, This paper proposes a code completion model based on rotary position embedding (RoPE), named CRPE, which aims to encode the position information of leaf nodes through RoPE, thereby effectively improving the accuracy of predicting the next token. CRPE model first utilizes an LSTM model to extract information from the root path of leaf nodes, which is then fused with leaf node information to obtain code representations. Next, RoPE injects position information from leaf nodes into the code representations in the form of a rotation matrix within the attention mechanism. Experimental results on two major public datasets in Python and JavaScript show that, compared with the baseline model, the CRPE model achieves the best performance in both Top-k and MRR indicators; in addition, the RoPE in the CRPE model performs better than the Absolute Position Embedding and Relative Position Embedding, which provide better results.