A2N-SFS: an advanced statistical adaptive feature selection method for enhancing intrusion detection systems in IoT systems
摘要
The rapid growth of the Internet of Things (IoT) has resulted in complex, high-dimensional network traffic data, posing significant challenges for intrusion detection systems (IDSs). The abundance of irrelevant and redundant data hampers IDS effectiveness, making efficient feature selection essential to improve detection accuracy. Existing methods, however, have limitations: traditional filter and embedded approaches often depend on fixed thresholds requiring domain knowledge and lack adaptability, while wrapper methods, although accurate, are computationally intensive and unsuitable for large-scale IoT environments. Moreover, single-feature selection techniques may introduce bias, leading to suboptimal outcomes. To address these issues, we propose the Advanced Adaptive Neighborhood-based Feature Selection (A2N-SFS), a flexible method that dynamically adjusts its feature selection threshold based on the evolving properties of the active feature subset. A2N-SFS leverages local statistical information from neighboring features to capture subtle interdependencies often missed by conventional methods. This adaptability enables precise and efficient selection tailored to the dataset’s unique characteristics. We validate A2N-SFS on benchmark intrusion detection datasets—NSL-KDD, UNSW-NB15, CSECICIDS-2018, BoT-IoT, and UNBCICIoT-23—where it consistently outperforms traditional machine learning classifiers, achieving accuracy rates of 99.4, 98.3, 99.9, 99.9, and 99.6% respectively. Further comparisons with statistical methods such as ANOVA, Pearson correlation, and Chi-square tests demonstrate A2N-SFS’s superior ability to manage complex, high-dimensional data, highlighting its effectiveness over existing feature selection techniques.