Large language models (LLMs) have shown exceptional performance in the domain of composite artificial intelligence tasks, offering a preliminary insight into the potential of general artificial intelligence. The fine-tuning process for LLMs necessitates significant computational resources, often surpassing those available from standard consumer-grade GPUs. To this end, we introduce the Adaptive Quantization Low-Rank Adaptation fine-tuning (AQLoRA), a method that reduces memory demands during fine-tuning by utilizing quantization coupled with pruning techniques. This dual strategy not only reduces memory usage but also preserves accuracy. AQLoRA refines the original Low-Rank Adaptation fine-tuning (LoRA) method by efficiently quantizing LLMs weights, prioritizing computational resource allocation based on weight importance, and effectively integrating the quantized model with auxiliary weights post fine-tuning. Applying AQLoRA to the ChatGLM2-6B model, we demonstrate its effectiveness in both natural language generation (NLG) and natural language understanding (NLU) across diverse fine-tuning datasets and scenarios. Our findings reveal that AQLoRA achieves balance between performance and memory efficiency, reducing memory consumption by 25% in NLG tasks. For NLU tasks, it enhances performance by 10% and reduces memory consumption by 10% compared to state-of-the-art methods.