Gender bias mitigation is an integral aspect of ensuring fair and equitable technological outcomes from artificial intelligence (AI) and natural language processing (NLP) systems. While large language models (LLMs) have demonstrated exceptional performance across diverse domains, their reliance on few-shot and zero-shot inference paradigms often fails to address inherent biases, posing risks of perpetuating discriminatory outputs. To address this issue, we propose a multi-task framework called Qwen-Gender, designed for parameter-efficient fine-tuning of the Qwen2.5-7B-Instruct LLM to automate gender bias detection, classification, and mitigation. First, a multi-task chain-of-thought (CoT) prompting strategy that generates CoT-based analyses to guide model fine-tuning while preserving the model’s foundational analytical capabilities. Next, a low-rank adaptation (LoRA) and a 4-bit quantization strategy are utilized to optimize multi-task collaboration without overburdening computational resources. Experimental results on the enhanced CORGI-PM dataset show that Qwen-Gender achieved second place in NLPCC 2025 Shared Task 7, demonstrating strong scalability and interpretability in Chinese gender bias mitigation tasks, and providing an effective pathway toward building fairer language model systems.