Scholarlink

<p>Accurate anomaly detection in system operations is essential for maintaining security and efficiency. Anomalies, unexpected deviations in system behavior, can indicate software bugs, cyberattacks, or hardware failures, making timely detection crucial to preventing significant damage and optimizing performance. Traditional anomaly detection techniques often rely on signature-based algorithms and face challenges in processing large volumes of data. This paper introduces a novel unsupervised approach to anomaly detection that leverages system logs and Generative Adversarial Networks (GANs) integrated with the Log2Vec framework, addressing the limitations of traditional methods and improving detection accuracy, recall, and runtime efficiency. Our proposed GAN-based model trains exclusively on normal data, enabling a precise reconstruction of log sequences. This approach mitigates class imbalance and reduces false alarms by generating realistic data distributions, which enhances the model’s ability to differentiate between normal and anomalous patterns. Reconstruction accuracy serves as the key metric for identifying anomalies. We also integrate the Log2Vec framework, which converts unstructured system logs into semantic vectors. This transformation captures underlying patterns and features indicative of anomalies, improving the model’s ability to identify complex deviations. Log2Vec effectively handles various log formats and out-of-vocabulary (OOV) words through domain-specific embeddings and a dedicated OOV processor, overcoming the limitations of traditional NLP-based methods like Word2Vec. This ensures a robust handling of complex and varied log data, improving the adaptability of the model to different operational contexts. Our proposed model demonstrates superior performance on several benchmarks. In the OpenStack dataset, it outperforms established models such as Deeplog, LogBERT, LogRobust and LogContrast, achieving a 97% accuracy rate and detecting 94% anomalous events. In the HDFS dataset, our model achieves a 92% F1 score, surpassing models such as LogGPT, Trine and LogBERT, and offers a runtime advantage over LSTM-based models such as as Deeplog and LogRobust due to its optimized architecture. In conclusion, our model processes massive datasets efficiently and excels in accuracy, recall, and scalability, positioning it as a robust solution to enhance the security and performance of large-scale complex information systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhanced anomaly detection in log-based systems using GAN-based models

  • 期刊论文

  • 发表日期：

    2025年9月11日

查看全文
  • Ebrahim Farjamfard,
  • Alireza Shameli-Sendi

摘要

Accurate anomaly detection in system operations is essential for maintaining security and efficiency. Anomalies, unexpected deviations in system behavior, can indicate software bugs, cyberattacks, or hardware failures, making timely detection crucial to preventing significant damage and optimizing performance. Traditional anomaly detection techniques often rely on signature-based algorithms and face challenges in processing large volumes of data. This paper introduces a novel unsupervised approach to anomaly detection that leverages system logs and Generative Adversarial Networks (GANs) integrated with the Log2Vec framework, addressing the limitations of traditional methods and improving detection accuracy, recall, and runtime efficiency. Our proposed GAN-based model trains exclusively on normal data, enabling a precise reconstruction of log sequences. This approach mitigates class imbalance and reduces false alarms by generating realistic data distributions, which enhances the model’s ability to differentiate between normal and anomalous patterns. Reconstruction accuracy serves as the key metric for identifying anomalies. We also integrate the Log2Vec framework, which converts unstructured system logs into semantic vectors. This transformation captures underlying patterns and features indicative of anomalies, improving the model’s ability to identify complex deviations. Log2Vec effectively handles various log formats and out-of-vocabulary (OOV) words through domain-specific embeddings and a dedicated OOV processor, overcoming the limitations of traditional NLP-based methods like Word2Vec. This ensures a robust handling of complex and varied log data, improving the adaptability of the model to different operational contexts. Our proposed model demonstrates superior performance on several benchmarks. In the OpenStack dataset, it outperforms established models such as Deeplog, LogBERT, LogRobust and LogContrast, achieving a 97% accuracy rate and detecting 94% anomalous events. In the HDFS dataset, our model achieves a 92% F1 score, surpassing models such as LogGPT, Trine and LogBERT, and offers a runtime advantage over LSTM-based models such as as Deeplog and LogRobust due to its optimized architecture. In conclusion, our model processes massive datasets efficiently and excels in accuracy, recall, and scalability, positioning it as a robust solution to enhance the security and performance of large-scale complex information systems.