Federated learning allows clients to collaboratively train models while safeguarding the privacy of their data. Existing methods typically assume that data from different clients originates from the same domain or distribution. Nonetheless, owing to regional constraints, data features from diverse clients demonstrate notable variations, termed as domain heterogeneity. The naive aggregation of models trained on such heterogeneous data can result in a global model that is biased towards dominant domains and generalizes poorly to others. Therefore, we expect the global model to have better generalization performance in different domains. In this paper, we propose a federated dual-clustered prototype learning(FedCPL) framework, a novel approach designed to counteract domain heterogeneity and improve model generalization. The key insight is to construct a shareable global prototype through dual-clustering, effectively minimizing the discrepancy among feature representations from disparate domains. On the client side, we introduce weighted contrastive learning and feature fusion to align local features, thereby mitigating domain-specific biases during model training. On the server side, an adaptive weighted aggregation strategy is introduced to prioritize contributions from more challenging domains. Extensive experiments on multiple benchmark datasets demonstrate that FedCPL significantly outperforms existing methods in scenarios with domain heterogeneity.