Depression is a prevalent mental health problem that affects millions of people worldwide, and early detection is crucial for effective intervention and treatment. The rise of digital communication and social media provides new opportunities for monitoring mental health through text analysis. In this paper, we evaluate the performance of popular open-source Large Language Models for text classification of depression through supervised fine-tuning with LoRA and QLoRA techniques. In addition, the study provides a comprehensive comparison of traditional Machine Learning algorithms, Deep Learning models, and Large Language Models. The study shows that Mistral-7B with full 16-bit precision (0.976), 8-bit (0.977), and 4-bit (0.975) weight quantization all achieve the highest F1 score, indicating that Mistral-7B is greatly compatible with depression diagnosis tasks. The findings will contribute to further research on Large Language Models for depression detection by providing insights into model selection, fine-tuning strategies, and the impact of quantization on performance. This will aid researchers in optimizing model architectures and adaptation techniques for improved accuracy and efficiency in mental health monitoring applications. Additionally, the findings will provide insights into the performance differences between traditional Machine Learning, Deep Learning, and LLMs for depression monitoring.