Scholarlink

<p>System logs are used to record the operational status of a system and significant events, and by performing anomaly detection on these logs, system faults can be rapidly and accurately identified. However, existing anomaly detection methods encounter difficulties with features that exhibit complex relationships, thereby limiting detection accuracy. Furthermore, the majority of methods depend on supervised learning, which hinders the detection of abnormal logs in large, unlabeled datasets. To address these limitations, this paper proposes a novel semi-supervised log anomaly detection model, termed LogCTBL (CNN-TCN-Bi-LSTM). Firstly, the model parses raw logs using the Drain3 tool. Secondly, it applies BERT for semantic embedding, thereby addressing the issue of log statement discreteness. Thirdly, the model further employs the HDBSCAN (hierarchical density-based spatial clustering of applications with noise) algorithm to estimate dummy tags for unlabeled data in the training set, thereby addressing the challenge of insufficient labeled data. Finally, the hybrid model is then applied to anomaly detection, and the efficacy of the proposed method is evaluated on the BGL and Thunderbird datasets. The findings demonstrate that the proposed method surpasses alternative approaches, attaining F1 scores of 99.87% and 99.78% in the two datasets, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

LogCTBL: a hybrid deep learning model for log-based anomaly detection

  • 期刊论文

  • 发表日期：

    2025年1月30日

查看全文
  • Hong Huang,
  • Wengang Luo,
  • Yunfei Wang,
  • Yinghang Zhou,
  • Weitao Huang

摘要

System logs are used to record the operational status of a system and significant events, and by performing anomaly detection on these logs, system faults can be rapidly and accurately identified. However, existing anomaly detection methods encounter difficulties with features that exhibit complex relationships, thereby limiting detection accuracy. Furthermore, the majority of methods depend on supervised learning, which hinders the detection of abnormal logs in large, unlabeled datasets. To address these limitations, this paper proposes a novel semi-supervised log anomaly detection model, termed LogCTBL (CNN-TCN-Bi-LSTM). Firstly, the model parses raw logs using the Drain3 tool. Secondly, it applies BERT for semantic embedding, thereby addressing the issue of log statement discreteness. Thirdly, the model further employs the HDBSCAN (hierarchical density-based spatial clustering of applications with noise) algorithm to estimate dummy tags for unlabeled data in the training set, thereby addressing the challenge of insufficient labeled data. Finally, the hybrid model is then applied to anomaly detection, and the efficacy of the proposed method is evaluated on the BGL and Thunderbird datasets. The findings demonstrate that the proposed method surpasses alternative approaches, attaining F1 scores of 99.87% and 99.78% in the two datasets, respectively.