Sample Similarity Based Incremental Clustering: An Effective Methodology for Anomaly Detection in Networks
摘要
The day to day increase of devices connected to the internet has led to an increase in cybersecurity threats, making it essential to develop advanced intrusion detection systems (IDS) that can recognize new and evolving attacks. Traditional methods for clustering in IDS, like K-means and DBSCAN, have their drawbacks, including rigid cluster formation, and difficulties in processing real-time data streams. To address this gap, this paper proposes an unsupervised learning based approach called Incremental Cosine Similarity-based Clustering (ICS) to detecting anomalies in network environments. ICS uses cosine similarity to allow clusters to grow adaptively, update centroids efficiently, and score anomalies dynamically. When evaluated on the BCAST IDS and NSL-KDD datasets, ICS showed impressive results, outperforming incremental K-Means variants in terms of cluster quality (Silhouette Score: 0.504 vs 0.201), runtime efficiency (17.45 s vs 20.66 s), and adaptability (3 clusters vs 2 clusters). The findings show that ICS is effective for detecting intrusion, providing enhanced accuracy, precision, and specificity while effectively identifying outliers and adapting to new attack patterns.