Development Generative AI for Cybersecurity: Evaluating Script Generation and Attack Classification in Penetration Testing

The application of Artificial Intelligence (AI) in cybersecurity is designed to address the growing complexity and volume of cyber threats, which create significant challenges for traditional penetration testing methods. Utilizing advanced Natural Language Processing (NLP) models can boost automation, consistency, and efficiency in detecting and mitigating vulnerabilities. This paper focuses on generating payloads for penetration testers using tuned pre-trained models such as ChatGPT-2, CodeBERT, and T5 models on a customized dataset. The effectiveness of the outperforming model is compared with Support Vector Machines (SVM) and Logistic Regression for dataset classification. By achieving these objectives, this study aims to enhance penetration testing processes, contributing to stronger cybersecurity defenses. The results of this study prove the effectiveness of using CodeBERT for the payload generation and as a classifier.