Existing cross-domain few-shot hyperspectral image classification methods often encounter negative transfer due to domain shift, especially when target domain samples are limited and cross-domain differences are significant. To address this challenge, this paper proposes a method named Cross-Modal Domain Shared Feature Learning (CMDSFL). This method adopts a multi-branch feature disentanglement strategy to effectively separate domain-shared and domain-specific features in the source and target domains. It leverages independent branches for extracting domain-specific features and shared branches for domain-shared features. By incorporating adversarial learning, it optimizes the extraction of domain-shared features and strengthens the generalization capacity of the model. Orthogonal loss reduces feature redundancy, while alignment loss reinforces inter-domain consistency. Additionally, a cross-modal alignment strategy is employed to utilize textual information for assisting in the classification of image features, achieving complementary and synergistic effects between features, thereby improving classification performance. Experimental results demonstrate that the CMDSFL method achieves superior performance on three publicly available hyperspectral image datasets, thus verifying the effectiveness of the proposed approach.