Structure-Aware Transformer for Code Snippet Summarization

Code summarization refers to the automatic generation of natural language descriptions from source code, a task that plays a crucial role in program understanding and software lifecycle support. Existing models often struggle to effectively capture the structural and control-flow semantics embedded in code snippets. In this work, we propose a structure-aware Transformer model for code snippet summarization. Our model incorporates a sparse graph attention mechanism that captures explicit structural dependencies, such as data flow, by integrating a carefully constructed adjacency matrix into the self-attention computation. We also introduce a control flow label embedding mechanism to model control structures, such as loops and conditionals, enhancing the model's ability to understand control logic. Experiments on a cleaned and curated Java dataset show that our model outperforms baseline models, including Transformer-based and structure-aware models, on several evaluation metrics such as BLEU, ROUGE-L, and METEOR. Ablation studies further validate the contribution of each component. These results demonstrate that our model captures structural and semantic dependencies more effectively, resulting in higher-quality summaries.