Scholarlink

<p>Modern enterprise networks generate high-volume, high-velocity traffic, making timely and accurate intrusion detection increasingly challenging for traditional batch-oriented Intrusion Detection Systems (IDS). These systems often struggle to detect sophisticated and evolving cyber threats due to delayed analysis, limited scalability, and the inability to provide continuous monitoring under dynamic workloads. This paper presents a real-time Network Intrusion Detection framework built on a distributed stream-processing architecture using Zeek, Apache Kafka, and PySpark. Network packets are captured and transformed into structured logs using Zeek, streamed reliably through Kafka, and processed in real time using PySpark for feature extraction, rule-based analysis, and machine learning–based classification. A Random Forest model is employed to identify both benign and malicious traffic across multiple attack categories, while Elasticsearch and Grafana provide low-latency storage, visualization, and alerting capabilities for operational security monitoring. The proposed system enables end-to-end real-time detection with high throughput and low latency, helping bridge the gap between rapid attack execution and timely threat response. Experimental results demonstrate the feasibility of scalable prototype IDS deployment for continuous monitoring, rapid analytics, and actionable security insights, while indicating strong promise for broader deployment with further real-world validation under diverse operational conditions.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Low-Latency Streaming-Based Network Intrusion Detection System

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月21日

查看全文
  • M. Shreya,
  • Shreyas Jain,
  • Shivakumar Shetty,
  • S. Anupama Kumar,
  • Narasimha Swamy S

摘要

Modern enterprise networks generate high-volume, high-velocity traffic, making timely and accurate intrusion detection increasingly challenging for traditional batch-oriented Intrusion Detection Systems (IDS). These systems often struggle to detect sophisticated and evolving cyber threats due to delayed analysis, limited scalability, and the inability to provide continuous monitoring under dynamic workloads. This paper presents a real-time Network Intrusion Detection framework built on a distributed stream-processing architecture using Zeek, Apache Kafka, and PySpark. Network packets are captured and transformed into structured logs using Zeek, streamed reliably through Kafka, and processed in real time using PySpark for feature extraction, rule-based analysis, and machine learning–based classification. A Random Forest model is employed to identify both benign and malicious traffic across multiple attack categories, while Elasticsearch and Grafana provide low-latency storage, visualization, and alerting capabilities for operational security monitoring. The proposed system enables end-to-end real-time detection with high throughput and low latency, helping bridge the gap between rapid attack execution and timely threat response. Experimental results demonstrate the feasibility of scalable prototype IDS deployment for continuous monitoring, rapid analytics, and actionable security insights, while indicating strong promise for broader deployment with further real-world validation under diverse operational conditions.