Fusion of Chi-Square and Z-Test Statistics for Feature Selection with Machine Learning Techniques in Intrusion Detection

With the proliferation of internet applications in daily life, intrusion detection has emerged as a prominent research area in network security. Due to technological advancements, undiscovered or zero-day attacks have the potential to threaten computer and network systems. Although the usage of machine learning in intrusion detection system has demonstrated profound improvements and defense mechanisms against the detection of such attacks, its success depends on how well the model is trained with the data. Processing of high-dimensional data often leads to more computational overhead and time. This paper proposes and implements a ranking-based novel feature selection approach for fast and effective selection of features for intrusion detection. Chi-square and Z-test are utilized for rapid selection of features based on their ranking to infer relevant information. The experiment further minimizes the 41features of NSL-KDD dataset to 19 features, while maintaining a high detection rate. The performance of the proposed approach is then assessed with eight classifiers. Moreover, we have compared the model build time (MBT) with and without feature selection and it is revealed that the proposed method converges to reduced time. Other performance metrics like accuracy, precision, detection rate and ROC curve justify the result.