The heightened demands for privacy protection, combined with legal mandates granting users the right to be forgotten, emphasize the growing importance of endowing models with the capability to forget specific sample information. Unlearning is akin to the reverse process of model training, which involves the elimination of the influence of training data on a machine learning model. In the context of federated learning, conducting unlearning is notably more demanding, as data is dispersed across multiple client endpoints. Due to privacy and security concerns, many conventional machine unlearning methods are no longer applicable. This paper draws inspiration from the forgetting processes in the human brain, highlighting that memory attrition is not solely a passive process. Therefore, we dichotomize the federated unlearning process into two components: active forgetting and passive forgetting. We make respective modifications to the updates of these two components to achieve efficient unlearning. We propose an approach called FedScale, which only sacrifices a fraction of storage space, enabling a rapid federated unlearning process without the need for retraining the federated model when it becomes necessary to forget a user’s data. Experimental evaluations are conducted on four datasets, demonstrating the effectiveness of FedScale unlearning. Furthermore, we show its resilience against backdoor attacks and membership inference attacks.