<p>Cross-domain few-shot learning (CD-FSL) addresses the challenges of few-shot learning (FSL) across extremely different domains and has garnered extensive attention in recent years. Existing CD-FSL studies with transfer learning primarily focus on extracting general features from source domain, neglecting the presence of redundant information that hinders adaptation to the target domain due to the significant domain shift between the source and target domain. To address this limitation and enable the model to better acquire target-domain-specific feature representations, this paper employs few unlabeled target domain data during the training phase. Given this setup and the challenge of redundancy, we propose a novel gradient-weighted pruning method, which operates during the training process to effectively eliminate redundant information from the source domain. Our method jointly prunes weights with gradient significance and bias parameters, ensuring the removal of redundant components while retaining transferable and critical information. By dynamically identifying and pruning uninformative weights, our proposed method enhances the model’s capacity to adapt to the target domain with minimal reliance on labeled data. Extensive experiments conducted on the BSCD-FSL (Broader Study of CD-FSL) benchmark dataset show that the proposed method achieves competitive average accuracy of 58.56% and 68.42% on 1-shot and 5-shot classification tasks. The results validate the efficacy of our approach in mitigating the negative effects of domain shift and redundancy and reveal the critical role of gradient-weighted pruning in improving the model’s adaptability and overall performance in CD-FSL.</p>