Federated learning (FL) has emerged as a novel framework of distributed machine learning in edge computing for ensuring data privacy meanwhile reducing communication traffic. Recently, an enhanced FL technology namely Clustered Federated Learning (CFL) has been developed to address the challenges posed by non-independent and identically distributed (non-IID) data across clients in FL. Nevertheless, existing CFL strategies face significant challenges in model parameter aggregation, because the aggregation weights are calculated according to the size of the dataset at each client instead of its contribution to the global model. In this paper, we propose a Model Similarity based Clustering Federated Learning (MS-CFL) framework in edge computing architecture. Specifically, we design an aggregation strategy based on client clustering, and we propose a model-similarity client clustering algorithm tailored to select optimal base stations for clients located in areas with overlapping resources. To precisely evaluate each cluster’s impact on the global model, we apply the Deep Reinforcement Learning (DRL) techniques to ensure a balanced consideration of dataset size and testing accuracy on global model during the aggregation process. Finally, we evaluate our approach based on open-source datasets, and the experimental results show the superiority of MS-CFL over several state-of-the-art methodologies in terms of communication and computation efficiency.