Assessing the Security of Vibe Coding: Baseline Vs. Security-Oriented Prompts in LLM Code Generation

Large Language Models (LLMs) are increasingly used in software development through so-called “vibe coding,” where developers specify tasks in natural language and rely on the model to produce executable code. While this paradigm lowers barriers to entry and accelerates prototyping, it raises concerns about security. Prior studies show that a substantial fraction of AI-generated code contains exploitable vulnerabilities, and functional correctness does not guarantee safety. This paper investigates whether security-oriented prompting improves the security of LLM-generated code and whether these effects generalize across models and sampling variability. We design ten representative Python tasks inspired by OWASP Top 10 and CWE categories, and evaluate outputs from GPT-OSS 20B with ten random seeds and from a cross-model comparison between GPT-OSS 20B and Gemma-3 27B. Outputs are analyzed using static analysis (Bandit) and lightweight runtime probes. Results show that default prompts consistently yield insecure code, with vulnerabilities present in most tasks. Security-informed prompts reduce both the prevalence and severity of weaknesses, particularly in command execution, deserialization, and secrets management. However, residual risks persist, such as insecure file upload handling and incomplete authentication checks, and improvements are not uniform across models. These findings suggest that prompt engineering can mitigate but not eliminate vulnerabilities.