Anomaly Detection Method of Source Code Vulnerability Detection Tools Based on Differential Testing
摘要
Deep learning-based source code vulnerability detection tools have gained popularity among developers for identifying security flaws in source code. However, these tools often exhibit poor generalization, achieving high accuracy only on datasets similar to their training data, and performing poorly (less than 20% accuracy) in real-world environments. This performance gap is attributed not only to training data issues but also to anomalies in tool design, implementation, and robustness—factors typically overlooked in current research. Most existing works focus on improving datasets and algorithms, ignoring the fact that such tools, as software systems, also require systematic testing. This paper views deep learning-based detection tools as software and introduces a novel anomaly detection framework based on differential testing. Unlike prior approaches, our method targets not only common software faults but also deep learning-specific anomalies and those unique to vulnerability detection tools. We further propose a new test case mutation strategy to enhance differential testing effectiveness. Our system identifies, quantifies, and provides fixes for detected anomalies, supporting automated and efficient anomaly detection and repair. This work bridges the gap between software testing and AI-based vulnerability detection, providing a more holistic and robust evaluation and improvement framework.