A Novel Approach for Continual and Federated Network Anomaly Detection

Nowadays, systems present an ever-increasing surface area, decentralised nature of the distributed systems and data privacy concerns, among other characteristics, making security a primary concern. Being able to identify anomalous traffic in such challenging conditions while ensuring the privacy of the analysed data is quite a challenging task. This work presents a combination of a Federated Learning-based approach with continual learning using unsupervised machine learning techniques for network anomaly detection. This also includes the discussion of a Holistic Security and Privacy Framework and its evaluation in a Kubernetes environment. Indeed, the Continual Learning concepts were applied to enable a quick adaptation to the ever-changing network traffic characteristics by performing frequent training sessions with the existent Machine Learning models, which are supported by collected data, whilst simultaneously performing network anomaly detection and never exposing the original network information. For its evaluation, we validated it using a micro-services-oriented application, where the generated normal traffic was used to train the different Machine Learning models, which were trained in several training periods and frequencies. In addition, we considered four different types of attacks: Denial of Service, Port Scan, Brute Force and SQL Injection to further evaluate the capability of the detection module to distinguish normal and anomalous traffic. Throughout the different validation scenarios, the detection module achieved an f1-score of 93.80% for one of the targeted components and a percentage of normal flows correctly identified of 99,13%.