The emergence of Code GenAI tools raises concerns about their security implications, particularly in generating insecure code and aiding cyberattacks. Current evaluation benchmarks face significant limitations, including reliance on static metrics, limited scalability, and a lack of comprehensive testing. SECCODEPLT addresses these gaps by introducing a unified platform that evaluates the security risks of Code GenAI in two critical areas: insecure coding and cyberattack facilitation. By combining expert-verified methodologies with automated data generation, SECCODEPLT ensures both scalability and data quality. It also incorporates dynamic metrics for more accurate risk assessments. Extensive experiments demonstrate that SECCODEPLT outperforms the CYBERSECEVAL benchmark in security relevance and precision, revealing significant vulnerabilities in state-of-the-art models such as Cursor. This platform sets a new standard for evaluating the risks posed by Code GenAI, providing valuable insights for both academic and industry applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SECCODEPLT: A Unified Evaluation Platform for Code GenAI Security Risks

  • Muhammed Abeed Nabith

摘要

The emergence of Code GenAI tools raises concerns about their security implications, particularly in generating insecure code and aiding cyberattacks. Current evaluation benchmarks face significant limitations, including reliance on static metrics, limited scalability, and a lack of comprehensive testing. SECCODEPLT addresses these gaps by introducing a unified platform that evaluates the security risks of Code GenAI in two critical areas: insecure coding and cyberattack facilitation. By combining expert-verified methodologies with automated data generation, SECCODEPLT ensures both scalability and data quality. It also incorporates dynamic metrics for more accurate risk assessments. Extensive experiments demonstrate that SECCODEPLT outperforms the CYBERSECEVAL benchmark in security relevance and precision, revealing significant vulnerabilities in state-of-the-art models such as Cursor. This platform sets a new standard for evaluating the risks posed by Code GenAI, providing valuable insights for both academic and industry applications.