Exploring the Capability of LLMs for Vulnerability Knowledge Management
摘要
Large Language Models (LLMs), such as GPT and DeepSeek, have made significant advancements in software engineering, particularly in code review and generation. However, their potential in vulnerability knowledge management remains underexplored. To bridge this research gap, we conduct a comprehensive evaluation of LLMs’ performance in vulnerability knowledge management, focusing on two critical tasks: (1) establishing static associations across vulnerability databases and (2) integrating fragmented vulnerability information from multiple sources. Experimental results demonstrate that LLMs are capable of understanding and analyzing the nature of vulnerabilities, showing significant promise in supporting vulnerability knowledge management. Furthermore, our study reveals several challenges when applying LLMs into this domain and offers insights for future research. For instance, for static classification tasks, simply extending the reasoning chain does not necessarily lead to good performance; instead, designing efficient prompts tailored for such tasks may be a promising direction. In addition, LLMs themselves exhibit inherent issues, such as hallucinations, which lead to unreliable or inaccurate outputs, posing potential risks in practical applications.