Domain generalization aims to enhance the performance of neural network models on unseen target domains by addressing the distribution shift between source and target domains. The rise of large model technologies, particularly the emergence of vision-language models (VLMs), has provided new solutions for domain generalization. Through knowledge distillation, the powerful generalization capabilities of VLMs can be transferred to compact student models, making them adaptable for resource-constrained scenarios. Nevertheless, current studies on visual-language knowledge distillation often overlook the potential inconsistency between visual and textual encodings, leading to limited generalization of student models. To this end, we propose the CLIP-based multi-modal calibration distillation (termed CMCD). The framework firstly fuses visual and textual encodings according to their cross-modal similarity (referred to as encoding calibration), then distills the calibrated encodings into the student model. Additionally, we apply normalization to eliminate domain-specific style features in visual encodings, thus enhancing the generalization capability of the student model. Extensive experiments on multiple domain generalization benchmarks verify that CMCD outperforms the state-of-the-art methods, demonstrating an average classification accuracy improvement of 4.67%.