On the Relevance of Graph2Vec Source Code Embeddings for Software Defect Prediction

Software defects prediction is a crucial activity related to software development and an extensively studied subject that remains challenging. One of the difficulties lies in the fact the most prevalent software metrics are not sufficiently relevant for accurately predicting defects. In this paper we propose employing the Graph2Vec embeddings unsupervisedly learnt from the source code, as well as a novel suite of syntactic coupling metrics derived from Graph2Vec embeddings, named SYNMET, as a foundation for software defect prediction. The dependability of the Graph2Vec embeddings is evaluated in contrast to that of the alternative embeddings based on Doc2Vec and LSI. The assessment is conducted through multiple experiments performed on 16 versions of Apache Calcite. Three distinct classification models, namely FastAI as a deep learning model, Multilayer Perceptron as an untuned conventional model, and Random Forests with hyperparameter tuning as a tuned conventional model, are employed in the experimental study. The findings indicate a synergistic relationship among Graph2Vec, Doc2Vec, and LSI-based embeddings. When comparatively assessing the performance of the three classifiers, the empirical results underscore the supremacy of tuned Random Forests over FastAI and Multilayer Perceptron. This observation substantiates the effectiveness of hyperparameter optimization. The relevance of the SYNMET suite is also assessed through multiple comparative analyses, the results confirming that they enhance the software defect prediction performance and reconfirming the interplay between software coupling and defect proneness.