This paper analyzes the efficacy of fine-tuning pre-trained code models for predicting software bug severity, using only the bug report title and description, which is readily available to the bug reporter at submission. We explore the use of various text and code generation language models for this classification task, utilizing the Apacheś Impala project bug dataset, focusing on resolved bug reports for accurate severity classifications. GraphCodeBERT demonstrates superior performance (F1: 49.9%) compared to other models studied, such as CodeBERT, CodeT5, and CodeGPT2 (F1 scores ranging from 36.4% to 46.5%), and traditional machine learning methods like Random Forest and XGBoost. Additionally, the Parameter-Efficient Fine-Tuning (PEFT) technique, Low-Rank Adaptation (LoRA), is applied which optimizes computational efficiency. Experiments reveal that GraphCodeBERT, even with LoRA (F1: 45.5%), maintains robust accuracy comparable to its non-LoRA performance.