Anomaly Detection for Intrusion Detection Systems Using Machine Learning: Experimental Study and Feature Reduction Approach

The introduction of artificial intelligence (AI) methods for effective intrusion detection or prevention systems (IDS/IPS) is a promising task. This study focuses on the possibilities of anomaly detection using machine and deep learning (ML/DL) methods. Two main groups of methods for intrusion detection, namely signature analysis and anomaly analysis exist. While signature analysis requires precise knowledge of network protocols, anomaly analysis leverages artificial intelligence techniques when accurate protocol knowledge is unavailable. This study utilizes public datasets, NSL-KDD and UNSW-NB15, containing information about various types of anomalies, occurring at different frequencies in the dataset. Additionally, the study discusses the possibility of reducing the dimensions of datasets and the impact of such reduction on accuracy and performance. The authors build upon a previous study that demonstrated satisfactory accuracy in detecting anomalies like DoS attacks using machine learning methods. Based on numerical experiments, the study formulates a hypothesis suggesting that the main set of features identified using an AHP-like method forms ensembles with different weight coefficients. The results confirm the hypothesis, indicating that this approach can be applied to all types of anomaly detection and ML/DL methods. The modified method proposed in the article establishes that there is a minimum dimension of feature sets, allowing the recognition of anomalies with achievable accuracy, albeit limited.