In the current digital age, many threats have emerged, most notably software vulnerabilities, which require new and advanced mechanisms to detect or prevent threats before they occur. This research studies the use of large language models to mitigate these challenges. We used large language models (GPT and Llama) to analyze and classify program code as vulnerable or non-vulnerable. We used the DiverseVul dataset and took test data from it with three different sizes (1000, 5000, and 20,000 records). We compared the performance of GPT and Llama in zero-shot and few-shot cases and observed that GPT outperformed Llama in all cases whether accuracy or speed. This study’s main contribution is a comprehensive comparison between GPT and Llama models in the Zero-shot and Few-shot scenarios to offer deeper insights into their performance and reliability. This is accomplished by two major experiments: (1) Consistency Check to ensure the stability of the models by repeating the experiment several times. (2) Comparative Analysis of GPTs vs. LLaMA in Zero-shot and Few-shot settings for evaluating predictive accuracy of models in software vulnerability tasks. These experiments are intended to afford a more comprehensive assessment of the models from multiple angles, thereby improving the reliability of the findings. This study enhances cybersecurity by carefully and accurately processing large language models to detect vulnerabilities and software breaches. It also suggests prospects for improving the results and expanding the study. The code and the dataset used for this study is available at https://github.com/walaashe/software-vulnerability-detection-using-LLM .