SECCODEPLT: A Unified Evaluation Platform for Code GenAI Security Risks
摘要
The emergence of Code GenAI tools raises concerns about their security implications, particularly in generating insecure code and aiding cyberattacks. Current evaluation benchmarks face significant limitations, including reliance on static metrics, limited scalability, and a lack of comprehensive testing. SECCODEPLT addresses these gaps by introducing a unified platform that evaluates the security risks of Code GenAI in two critical areas: insecure coding and cyberattack facilitation. By combining expert-verified methodologies with automated data generation, SECCODEPLT ensures both scalability and data quality. It also incorporates dynamic metrics for more accurate risk assessments. Extensive experiments demonstrate that SECCODEPLT outperforms the CYBERSECEVAL benchmark in security relevance and precision, revealing significant vulnerabilities in state-of-the-art models such as Cursor. This platform sets a new standard for evaluating the risks posed by Code GenAI, providing valuable insights for both academic and industry applications.