Secure Code Generation with Parameter-Efficient Fine-Tuning of LLMs
摘要
Large Language Models (LLMs) have demonstrated strong capabilities in code generation, yet studies reveal that their outputs may contain critical security vulnerabilities. This study investigates the effectiveness of Parameter-Efficient Fine-Tuning (PEFT) techniques in enhancing the secure code generation capabilities of LLMs. Using the CodeGen-Multi 2B model as the base, applied two state-of-the-art PEFT methods–LoRA and IA3–to fine-tune the model on a curated dataset of C and C++ code. The objective is to evaluate whether fine-tuned models can reduce vulnerability incidence and to compare the relative performance of LoRA and IA3. Security evaluations are conducted using prompts designed to elicit eight common vulnerabilities from the MITRE CWE Top 25 list. Findings show that PEFT fine-tuning significantly improves the model’s ability to generate secure code, with both LoRA and IA3 outperforming the base model. Among the two, LoRA exhibits a slight advantage in reducing vulnerabilities. This research highlights the potential of lightweight fine-tuning methods to mitigate security risks in LLM-generated code and offers practical insights for future advancements in secure AI-assisted software development.