<p>Federated learning preserves data privacy by sharing model parameters instead of raw data. However, a single global model often struggles to accommodate the diversity of client-side data, particularly in large-scale, high-performance computing, and real-time scenarios. This paper proposes a personalized federated learning framework that integrates clustering and knowledge distillation to enable heterogeneous model optimization. Specifically, the framework supports architecturally distinct models on the server and client sides. The server initializes a global model and distributes it to clients for local training. Based on gradient similarity, the trained client models are clustered and aggregated into multiple cluster-specific models, which are further fused to construct an adaptive global model. To enhance both global generalization performance and local personalization capability, bidirectional knowledge distillation is performed between the global and client models. Extensive experimental results demonstrate that the proposed framework significantly improves the overall performance of federated learning systems while maintaining personalization, thus providing an effective solution for heterogeneous model scenarios.</p>