Large language models (LLMs) now drive applications in e-commerce, healthcare, law, and education, yet they expose systems to threats such as prompt injection and jailbreak-style manipulation. Many defenses—ranging from API-level controls to semantic or behavioral detectors—have been proposed, but their effectiveness is rarely measured under consistent conditions. This work presents an empirical benchmark of nine representative techniques using a unified testbed. The framework issues benign and adversarial prompts to measure detection accuracy, false-positive rates, and latency across both rule-based modules and machine learning-based detectors. Our results show that no single defense provides comprehensive coverage: rule-based methods offer low-latency filtering for structural anomalies, while ML-based detectors capture indirect semantic attacks at the cost of higher overhead and increased false positives. These findings highlight the need for modular, context-aware defense pipelines in practical deployments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Benchmarking Defense Techniques for Securing Large Language Models

  • Abdulkadir Korkmaz,
  • Timoteo Kelly,
  • Praveen Rao

摘要

Large language models (LLMs) now drive applications in e-commerce, healthcare, law, and education, yet they expose systems to threats such as prompt injection and jailbreak-style manipulation. Many defenses—ranging from API-level controls to semantic or behavioral detectors—have been proposed, but their effectiveness is rarely measured under consistent conditions. This work presents an empirical benchmark of nine representative techniques using a unified testbed. The framework issues benign and adversarial prompts to measure detection accuracy, false-positive rates, and latency across both rule-based modules and machine learning-based detectors. Our results show that no single defense provides comprehensive coverage: rule-based methods offer low-latency filtering for structural anomalies, while ML-based detectors capture indirect semantic attacks at the cost of higher overhead and increased false positives. These findings highlight the need for modular, context-aware defense pipelines in practical deployments.