Ensuring the security and ethical alignment of large language models (LLMs) is critical as adversarial attacks, such as prompt injections and jailbreak exploits, continue to evolve. Pre-detection mechanisms have emerged as a promising defense, filtering adversarial prompts before they reach the LLM. However, existing pre-detectors exhibit limitations in distinguishing genuinely harmful queries from legitimate prompts that resemble adversarial inputs, leading to high false positive rates (FPR). To address this, we propose a Two-Axis Pre-Detector (TAPD) that independently classifies harmfulness and jailbreakness, enhancing detection granularity. Furthermore, we introduce a conditional Warning Wrapper mechanism (CWW), a conditional self-reminder that mitigates false positives while maintaining LLM alignment. Our empirical evaluation demonstrates that TAPD significantly reduces FPR while preserving robust security measures, improving both pre-detection reliability and usability in real-world AI applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Countering Jailbreak Attacks with Two-Axis Pre-detection and Conditional Warning Wrappers

  • Hyunsik Na,
  • Hajun Kim,
  • Dooshik Yoon,
  • Daeseon Choi

摘要

Ensuring the security and ethical alignment of large language models (LLMs) is critical as adversarial attacks, such as prompt injections and jailbreak exploits, continue to evolve. Pre-detection mechanisms have emerged as a promising defense, filtering adversarial prompts before they reach the LLM. However, existing pre-detectors exhibit limitations in distinguishing genuinely harmful queries from legitimate prompts that resemble adversarial inputs, leading to high false positive rates (FPR). To address this, we propose a Two-Axis Pre-Detector (TAPD) that independently classifies harmfulness and jailbreakness, enhancing detection granularity. Furthermore, we introduce a conditional Warning Wrapper mechanism (CWW), a conditional self-reminder that mitigates false positives while maintaining LLM alignment. Our empirical evaluation demonstrates that TAPD significantly reduces FPR while preserving robust security measures, improving both pre-detection reliability and usability in real-world AI applications.