Enhancing the Parallel UC2B Framework: Approach Validation and Scalability Study

Anomaly detection is a critical aspect of uncovering unusual patterns in data analysis. This involves distinguishing between normal patterns and abnormal ones, which inherently involves uncertainty. This paper presents an enhanced version of the parallel UC2B framework for anomaly detection, previously introduced in a different context. In this work, we present an extension of the framework and present its large-scale evaluation on the Supercomputer Fugaku. The focus is on assessing its scalability by leveraging a great number of nodes to process large-scale datasets within the cybersecurity domain, using the UNSW-NB15 dataset. The ensemble learning techniques and inherent parallelizability of the Unite and Conquer approach are highlighted as key components, contributing to the framework’s computational efficiency, scalability, and accuracy. This study expands upon the framework’s capabilities and emphasizes its potential integration into an existing Security Orchestration, Automation, and Response (SOAR) system for enhancing cyber threat detection and response.