In programming languages, learning models often treat code as a natural language, causing them to learn syntax and semantics implicitly. The paper proposes, as a proof of concept, a new approach to generate source code embeddings based on Abstract Syntax Graphs, a more compact and less redundant representation than traditional ASTs, particularly at the leaf node level. Thus, a Graph Attention Network model is employed for learning to reconstruct ASGs and internalize structural information. To evaluate the quality of the resulting embeddings, the embedded data is clustered using k-means, DBSCAN, and Spectral Clustering, using the cosine similarity distance metric. Performance is assessed using silhouette scores, outperforming those achieved by state-of-the-art models such as CodeBERT and CodeT5 on the same task.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

CodeASG: An Approach for Extracting Code Embeddings from Abstract Syntax Graphs

  • Alexandru-Gabriel Sîrbu,
  • Gabriela Czibula

摘要

In programming languages, learning models often treat code as a natural language, causing them to learn syntax and semantics implicitly. The paper proposes, as a proof of concept, a new approach to generate source code embeddings based on Abstract Syntax Graphs, a more compact and less redundant representation than traditional ASTs, particularly at the leaf node level. Thus, a Graph Attention Network model is employed for learning to reconstruct ASGs and internalize structural information. To evaluate the quality of the resulting embeddings, the embedded data is clustered using k-means, DBSCAN, and Spectral Clustering, using the cosine similarity distance metric. Performance is assessed using silhouette scores, outperforming those achieved by state-of-the-art models such as CodeBERT and CodeT5 on the same task.