Evaluating Data Distribution Strategies in Federated Learning: A Trade-Off Analysis Between Privacy and Performance for IoT Security

Federated learning is an effective approach for training a global machine learning model. It uses locally acquired data without having to share local data with the centralised server. This method provides a machine learning model beneficial for all parties. It ensures that individual parties do not compromise their privacy or disclose sensitive or personal data. From a cyber security perspective, machine learning with federated learning can highlight intrusions or anomalous activity on a device, without the individual device owner having to reveal characteristics of their own personal usage that would then breach their own privacy. In this paper, we conduct an exploratory investigation into two public datasets, Edge-IIoTset, and CICIoT2023, and we highlight the strengths and limitations of these datasets as currently presented. We then conduct further experimentation on the CICIoT2023 dataset, that previously has only been used for developing centralised learning models. We investigate machine learning performance under various distributions of the data across a set of federated clients, including stratified, leave-one-out, one-class, and half-benign strategies. Specifically, we examine whether a comparable model can be developed using federated learning, and how little data is required by each client to maintain privacy whilst also offering comparable performance against a centralised model.