This paper presents a stream anomaly detection app that assists in real-time data processing and forecasting because information is rapidly growing. Traces the findings of unusual data streams passed to Kafka using Deep Learning (DL) deep model called an Autoencoder neural network. The model will test its ability to predict the data and seek errors (reconstruction error) to identify outliers or outliers. In the event that the model detects anything out of the ordinary, the model puts a mark on the data and saves it in a database to be checked on later. This assists them in distinguishing normal data from odd data so that it becomes easier to use correct data in the reports or isolate it when the time arises. The app offers real-time data and status of the data on a dashboard comprising charts. It is run within a Kubernetes system, which enables it to handle much information without many problems or inefficiencies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Time Anomaly Detection in Big Data Streams with Deep Learning

  • Krishnam Narsepalle,
  • Sreepal Reddy Bolla

摘要

This paper presents a stream anomaly detection app that assists in real-time data processing and forecasting because information is rapidly growing. Traces the findings of unusual data streams passed to Kafka using Deep Learning (DL) deep model called an Autoencoder neural network. The model will test its ability to predict the data and seek errors (reconstruction error) to identify outliers or outliers. In the event that the model detects anything out of the ordinary, the model puts a mark on the data and saves it in a database to be checked on later. This assists them in distinguishing normal data from odd data so that it becomes easier to use correct data in the reports or isolate it when the time arises. The app offers real-time data and status of the data on a dashboard comprising charts. It is run within a Kubernetes system, which enables it to handle much information without many problems or inefficiencies.