Due to the privacy preserving capabilities, federated learning (FL) has emerged as an efficient decentralized deep learning paradigm. Traditional FL attempts to learn a single global model with the cooperation of many clients under the coordination of a central server. However, learning a single global model cannot achieve satisfactory model accuracy for all clients joining in FL on heterogeneous data. In addition, in the process of collaborative learning of the global model among a large number of clients, massive data exchange between the clients and server results in high communication cost, which is unlikely to be affordable for resource constrained mobile devices or Internet of Things (IoT) devices. Therefore, we propose a novel Personalized Federated Learning by Model Pruning (PFLMP) which uses batch-normalization (BN) layers of clients to personalize the global model to solve the challenge that arise with data heterogeneity across clients, and uses BN layers of server to prune the global model to reduce the communication cost. Extensive experiments show that PFLMP outperforms current state-of-the-art personalized algorithms and pruning algorithms in FL in terms of the model accuracy and the communication overheads.