The implementation of Artificial Intelligence in Large Language Models (LLMs) faces challenges due to high computational demands. While fine-tuning and knowledge distillation help, they still struggle with information loss and performance degradation, especially when the student model can’t fully inherit the teacher model’s knowledge. To address these issues, this study proposes a knowledge distillation method, GKCoT, based on Global Keywords and Chain of Thought (CoT). GKCoT clarifies decision-making by breaking down large model decisions into CoT, helping understand the model’s internal mechanisms, and extracting global keywords to ensure the student model retains essential information. Experiments on multiple question-answering datasets show that the GKCoT method offers significant advantages. When the student model has over 200 million parameters, GKCoT outperforms both standard fine-tuning and baseline approaches. On the cqa, causal, and revqa datasets, the T5-Large student model distilled using GKCoT surpasses the performance of the GPT-3.5-turbo model. Moreover, GKCoT maintains strong performance even with limited labeled samples. On the causal and revqa datasets, it achieves higher accuracy using only 50% of the training data compared to standard fine-tuning trained on the full dataset.