Scholarlink

<p>High-quality code comments can improve the efficiency of code comprehension and maintenance. Previous studies have proposed various automatic source code summarization methods. A recent trend is to improve the generative comment capabilities of neural network models with Information Retrieval (IR), i.e., incorporating comments of similar code snippets as an additional source. However, these approaches rely primarily on sparse vector representations of code, which often fail to retrieve the most analogous code snippets effectively. Moreover, such methods usually represent source code as code token sequences, abstract syntax tree (AST) sequences, or a combination of both, and neglect important information such as data flow. In this paper, we propose a novel code summarization approach named FTIR by deeply fusing the Transformer model and IR. FTIR employs the pre-trained model CodeBERT to represent source code with dense vectors to retrieve the most similar code snippet more effectively. The comment of the retrieved code is regarded as the external feature to guide the Transformer model in generating a more accurate comment. FTIR integrates multiple features of source code, including semantic (code tokens), syntactic and structural information (e.g. AST), and global syntactic structures (e.g., data flow) as internal features. This integration aims to strengthen token-level alignment across various code representations. To effectively leverage external features, we propose a cross-attention mechanism that guides the fusion of the current decoding state with this feature. Finally, a deep fusion of both internal and external features is performed to determine the probability distribution over the vocabulary. Summarization tokens are then generated using a beam search. Experiments on a commonly used benchmark dataset show that FTIR achieves BLEU, ROUGE-L, and METEOR scores of 46.17, 55.90, and 28.15, respectively, significantly outperforming existing hybrid approaches. These scores represent improvements of 0.28-18.57 for BLEU, 0.11-14.8 for ROUGE-L, and 0.3-15.54 for METEOR compared to the state-of-the-art approaches.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deeply fusing transformer model and information retrieval with cross attention for source code summarization

  • 期刊论文

  • 发表日期：

    2026年2月27日

查看全文
  • Rongcun Wang,
  • Chenkun Chang,
  • Xiang Chen,
  • Yuan Tian,
  • Rubing Huang

摘要

High-quality code comments can improve the efficiency of code comprehension and maintenance. Previous studies have proposed various automatic source code summarization methods. A recent trend is to improve the generative comment capabilities of neural network models with Information Retrieval (IR), i.e., incorporating comments of similar code snippets as an additional source. However, these approaches rely primarily on sparse vector representations of code, which often fail to retrieve the most analogous code snippets effectively. Moreover, such methods usually represent source code as code token sequences, abstract syntax tree (AST) sequences, or a combination of both, and neglect important information such as data flow. In this paper, we propose a novel code summarization approach named FTIR by deeply fusing the Transformer model and IR. FTIR employs the pre-trained model CodeBERT to represent source code with dense vectors to retrieve the most similar code snippet more effectively. The comment of the retrieved code is regarded as the external feature to guide the Transformer model in generating a more accurate comment. FTIR integrates multiple features of source code, including semantic (code tokens), syntactic and structural information (e.g. AST), and global syntactic structures (e.g., data flow) as internal features. This integration aims to strengthen token-level alignment across various code representations. To effectively leverage external features, we propose a cross-attention mechanism that guides the fusion of the current decoding state with this feature. Finally, a deep fusion of both internal and external features is performed to determine the probability distribution over the vocabulary. Summarization tokens are then generated using a beam search. Experiments on a commonly used benchmark dataset show that FTIR achieves BLEU, ROUGE-L, and METEOR scores of 46.17, 55.90, and 28.15, respectively, significantly outperforming existing hybrid approaches. These scores represent improvements of 0.28-18.57 for BLEU, 0.11-14.8 for ROUGE-L, and 0.3-15.54 for METEOR compared to the state-of-the-art approaches.