Balancing the act? Resampling versus imbalanced data for Wi-Fi IDS

Ensuring the security of Wi-Fi networks is a top priority and an ongoing research topic for academic and industry research groups. This work investigates the impact of class distribution in benchmark datasets on the detection performance of machine learning (ML)-driven wireless intrusion detection systems (WIDS). More specifically, we aim to address a critical question not yet explored in the literature: To what extent does class distribution influence WIDS performance? In pursuit of answers, we consider a multiclass problem and utilize several resampling schemes, including oversampling, undersampling, and both under- and oversampling, exploring their impingement on the classification results in terms of standard metrics. We evaluate a variety of ML models, both traditional and deep ones, and contrast their performance against those yielded when the algorithms are trained on the original imbalanced dataset. The key finding is that training on the original imbalanced dataset significantly reduces false positives vis-á-vis a balanced dataset created by any resampling technique. This reduction in terms of macro-averaged false positive rate is particularly noteworthy, approximately 139 and 43 times lower false positives compared to the worst and best resampling scheme, respectively. However, while resampled balanced datasets can lead to fewer false negatives (missed attacks), the improvement is less pronounced, at about 1.2 times lower relative to the imbalanced dataset.