Benchmarking Defense Techniques for Securing Large Language Models
摘要
Large language models (LLMs) now drive applications in e-commerce, healthcare, law, and education, yet they expose systems to threats such as prompt injection and jailbreak-style manipulation. Many defenses—ranging from API-level controls to semantic or behavioral detectors—have been proposed, but their effectiveness is rarely measured under consistent conditions. This work presents an empirical benchmark of nine representative techniques using a unified testbed. The framework issues benign and adversarial prompts to measure detection accuracy, false-positive rates, and latency across both rule-based modules and machine learning-based detectors. Our results show that no single defense provides comprehensive coverage: rule-based methods offer low-latency filtering for structural anomalies, while ML-based detectors capture indirect semantic attacks at the cost of higher overhead and increased false positives. These findings highlight the need for modular, context-aware defense pipelines in practical deployments.