Scholarlink

<p>The rapid advancement of Artificial Intelligence (AI) has positioned Large Language Models (LLMs) such as ChatGPT at the forefront of innovation, showcasing unprecedented capabilities in text understanding and generation. However, the flexibility of LLMs also introduces vulnerabilities, particularly in generating malicious content, posing significant cybersecurity threats. Despite OpenAI’s implementation of security systems to mitigate misuse, adversarial techniques like "jailbreak prompts" continue to bypass these safeguards, underscoring the need for more advanced detection systems. This paper presents a novel machine-learning model designed to detect malicious cybersecurity prompts. The study involved the creation of a comprehensive dataset representing diverse cybersecurity threats across seven domains, including malware, phishing, and social engineering. The dataset was developed through a three-phased approach: generating malicious prompts using ChatGPT, collecting benign prompts from reputable cybersecurity websites via web scraping, and crafting advanced prompts to simulate sophisticated jailbreak attacks. The resulting dataset contains 3,354 samples, offering a diverse and realistic representation of potential LLM exploitations. Two machine-learning experiments were conducted using various Natural Language Processing (NLP) techniques for binary and multiclass classification tasks. The binary classification achieved a 97% accuracy rate using a Support Vector Machine (SVM), while the multiclass classification achieved 99% accuracy with a sophisticated Voting technique. These results demonstrate the model’s efficacy in detecting malicious prompts and contribute significantly to the cybersecurity community by establishing a robust foundation for defending against LLM-based threats.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Adding extra security layer to chatGPT: machine learning based model to detect malicious cybersecurity prompts

  • 期刊论文

  • 发表日期：

    2025年9月15日

查看全文
  • Ibrahim Obeidat,
  • Rabee Alquran,
  • Alla Mughaid,
  • Ala Obeidat

摘要

The rapid advancement of Artificial Intelligence (AI) has positioned Large Language Models (LLMs) such as ChatGPT at the forefront of innovation, showcasing unprecedented capabilities in text understanding and generation. However, the flexibility of LLMs also introduces vulnerabilities, particularly in generating malicious content, posing significant cybersecurity threats. Despite OpenAI’s implementation of security systems to mitigate misuse, adversarial techniques like "jailbreak prompts" continue to bypass these safeguards, underscoring the need for more advanced detection systems. This paper presents a novel machine-learning model designed to detect malicious cybersecurity prompts. The study involved the creation of a comprehensive dataset representing diverse cybersecurity threats across seven domains, including malware, phishing, and social engineering. The dataset was developed through a three-phased approach: generating malicious prompts using ChatGPT, collecting benign prompts from reputable cybersecurity websites via web scraping, and crafting advanced prompts to simulate sophisticated jailbreak attacks. The resulting dataset contains 3,354 samples, offering a diverse and realistic representation of potential LLM exploitations. Two machine-learning experiments were conducted using various Natural Language Processing (NLP) techniques for binary and multiclass classification tasks. The binary classification achieved a 97% accuracy rate using a Support Vector Machine (SVM), while the multiclass classification achieved 99% accuracy with a sophisticated Voting technique. These results demonstrate the model’s efficacy in detecting malicious prompts and contribute significantly to the cybersecurity community by establishing a robust foundation for defending against LLM-based threats.