Defect-scanner: a comparative empirical study on language model and deep learning approach for software vulnerability detection

The complex and rapidly evolving nature of modern software landscapes introduces challenges such as increasingly sophisticated cyber threats, the diversity in programming languages and coding styles, and the need to identify subtle patterns indicative of vulnerabilities. These hurdles underscore the necessity for advanced techniques that can effectively cope with the intricacies of software security. Hence, this paper gives a comparative empirical study in harnessing the potential of cutting-edge natural language processing (NLP) advancements, namely Word2Vec and CodeBERT to detect vulnerabilities in C and C++ programs in the proposed Defect-Scanner framework. With the capability of converting code components and source code into contextual embedding vectors, various potential NLP techniques are combined with several DL models to evaluate the precision and accuracy of identifying vulnerabilities within software systems. Moreover, the experimentations are conducted using datasets with different representation types of codes, aiming to figure out the best combination of NLP techniques and DL models to work with each form of input. As a result, besides the outperformance of CodeBERT-based models with accuracies of approximately 90%, this comparative study also provides a comprehensive evaluation of NLP-based software vulnerability detection in the face of intricate security challenges.