This study investigates the fine-tuning of open-source large language models (LLMs) for domain-specific tasks, such as question-answering in cybersecurity and IT support. It focuses on two fine-tuning techniques: Supervised Fine-Tuning (SFT) and Parameter-Efficient Fine-Tuning (PEFT), specifically Low-Rank Adaptation (LoRA). The research compares the performance of 21 open-source LLMs, ranging from 2 to 9 billion parameters models like Llama2-7B, Llama3.1-7B, Mistral-7B, Falcon-7B, Phi-3.5, and Gemma2-9B. SFT consistently delivers high accuracy and low train-evaluation loss, while LoRA significantly reduces GPU memory usage and computational costs without compromising performance. The research findings emphasize the importance of selecting optimal fine-tuning techniques and model architectures for domain-specific tasks and also highlight advancements in fine-tuning LLMs for efficient and scalable AI solutions in production.