Federated Learning (FL) is a distributed collaborative training paradigm that enables model training without sharing local data, thereby preserving user privacy. However, FL remains susceptible to privacy inference attacks. To mitigate these threats, Differential Privacy (DP) mechanisms have been integrated into FL frameworks. Despite their advantages, the inherent noise injection in DP inevitably degrades model performance, particularly in scenarios with Non-Independent and Identically Distributed (Non-IID) data. In this paper, we propose an improved personalized federated learning approach named FedFIP (Fisher-based Information for Privacy), which trains personalized models on the client side to adapt to local data distributions while incorporating dynamic references to the global model through L2-norm regularization, thereby balancing personalization and generalization. In addition, the FedFIP scheme utilizes Fisher information as a parameter importance measure to protect sensitive parameters from being interfered with by noisy global parameters. At the same time, parameter sparsification is used to further reduce the negative impact of noise on model performance and reduce communication overhead. Experimental results on the CIFAR-10 and MNIST datasets demonstrate that FedFIP significantly improves model accuracy and accelerates convergence speed. These findings offer a robust solution for optimizing both privacy protection and model utility in federated learning.