Deep learning based palmprint recognition has gained wide adoption in identity authentication and security systems owing to its unique and stable biometric features. However, conventional methods rely on centralized data storage, where multi-source datasets are aggregated to train deep learning models, raising significant privacy leakage and security concerns. To mitigate these risks, we propose DAFL-Palm, a novel Federated Learning framework for palmprint recognition enhanced with Domain Adaptation (DA). DAFL-Palm employs a distributed training paradigm, allowing clients to retain raw data locally while leveraging a single-source domain adaptation technique that aligns feature distributions at both pixel and frequency levels. The framework generates a global model and an anchor model through federated aggregation, followed by joint optimization to boost recognition performance. Experimental results demonstrate that DAFL-Palm significantly improves accuracy while robustly preserving user privacy and data security.