Federated learning is a machine learning technique that allows multiple distributed clients to collaboratively train an ML model without sharing their private data with any of the parties involved. However, ensuring the privacy of client data during the FL process remains an ongoing concern. In this study, we propose a homomorphic-encryption-based privacy-preserving FL protocol for multilayer perceptrons, which is shown to be secure under the presence of colluding honest-but-curious clients. The possibility of client collusion attacks is eliminated by utilizing the inherent permutability of neural networks. Our results indicate that our protocol does not incur any considerable loss in accuracy during the training process. Furthermore, it offers minimal computation costs by utilizing the batching technique of homomorphic operation and employing only the inexpensive homomorphic addition operation for the aggregation process.