A Re-assessment of Code2Vec

Code2Vec has emerged as a powerful tool for analyzing source code by leveraging distributed representations. Code2Vec has demonstrated substantial capabilities in capturing semantic information from source code; however, its sensitivity to variable names has been identified as a significant limitation. This sensitivity raises concerns about the robustness of the model’s performance in different code bases with varying naming conventions. In response to this limitation, our study focuses on evaluating the impact of a variable name anonymisation technique. The anonymization process was guided by the fundamental concept that a program’s semantics remain consistent despite changes in variable names. This idea underscores the potential for exploring Code2Vec’s learning performance with anonymous variables. As operations within a program constitute its semantics, and certain paths encapsulate these operations, it prompts a natural inquiry into whether Code2Vec’s learning process can be enhanced by prioritizing operations over variable names. By anonymizing variable names, we aim to enhance Code2Vec’s generalization performance, especially when confronted with non-human-generated source code, such as that produced by genetic programming.