Accurately locating security vulnerabilities in ever-growing codebases is critical for protecting modern software ecosystems. Yet real-world detectors must contend with three intertwined hurdles: extreme class imbalance, thousand-token functions that exceed Transformer limits, and vulnerability patterns that span semantic, contextual, and syntactic dimensions. We propose Sparse-MoE, a syntax-aware, multi-view mixture-of-experts framework that addresses these obstacles within a single architecture. The pipeline (i) splits ultra-long functions along abstract-syntax-tree boundaries to preserve long-range semantics without quadratic attention, (ii) encodes each block through complementary semantic (SBERT), contextual (code2vec), and syntactic (CodeBERT-C) views, and (iii) routes embeddings to a small set of lightweight experts via a risk-aware sparse gate while enforcing load balance. Training combines cross-entropy with block-level InfoNCE alignment and an expert-usage regularizer. Sparse-MoE attains F1 = 0.94, Precision = 0.97 on the highly imbalanced BigVul benchmark and 0.70 accuracy on the balanced Devign dataset, while analysing 2 k-line functions with just 1.24 GB GPU memory and 13 ms latency per inference. These results demonstrate that syntax-aware splitting, risk-driven sparse gating, and multi-view contrastive alignment jointly yield an efficient and scalable solution for large-scale vulnerability screening.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Sparse-MoE: Syntax-Aware Multi-view Mixture of Experts for Long-Sequence Software Vulnerability Detection

  • Yifan Wu,
  • Lei Xiao

摘要

Accurately locating security vulnerabilities in ever-growing codebases is critical for protecting modern software ecosystems. Yet real-world detectors must contend with three intertwined hurdles: extreme class imbalance, thousand-token functions that exceed Transformer limits, and vulnerability patterns that span semantic, contextual, and syntactic dimensions. We propose Sparse-MoE, a syntax-aware, multi-view mixture-of-experts framework that addresses these obstacles within a single architecture. The pipeline (i) splits ultra-long functions along abstract-syntax-tree boundaries to preserve long-range semantics without quadratic attention, (ii) encodes each block through complementary semantic (SBERT), contextual (code2vec), and syntactic (CodeBERT-C) views, and (iii) routes embeddings to a small set of lightweight experts via a risk-aware sparse gate while enforcing load balance. Training combines cross-entropy with block-level InfoNCE alignment and an expert-usage regularizer. Sparse-MoE attains F1 = 0.94, Precision = 0.97 on the highly imbalanced BigVul benchmark and 0.70 accuracy on the balanced Devign dataset, while analysing 2 k-line functions with just 1.24 GB GPU memory and 13 ms latency per inference. These results demonstrate that syntax-aware splitting, risk-driven sparse gating, and multi-view contrastive alignment jointly yield an efficient and scalable solution for large-scale vulnerability screening.