<p>Logs produced by heterogeneous systems are essential for analysing system behaviour and ensuring operational efficiency and security. Traditional log anomaly detection approaches often struggle with the variety of log formats and structures, as they heavily rely on log parsing, which can result in incomplete or flawed information extraction. In this work, we enhance our previously proposed HEART framework (<b>HE</b>terogeneous Log <b>A</b>nomaly Detection using <b>R</b>obust <b>T</b>ransformers), a parsing-independent, end-to-end methodology that leverages Transfer Learning (TL) and Transformer models to process raw log events from diverse systems directly. We propose X-HEART (e<b>X</b>plainable <b>HE</b>terogeneous Log <b>A</b>nomaly Detection using <b>R</b>obust <b>T</b>ransformers) to address the limitations of previous approaches and improve model transparency and interpretability. X-HEART incorporates SHAP (SHapley Additive exPlanations) innovatively by aggregating and analysing SHAP values from various components, offering a thorough understanding of model decisions for improved explainability. Our framework is evaluated on seven distinct log datasets, allowing for a more comprehensive intra- and cross-system analysis. Additionally, we employ extensive performance evaluation metrics to assess the framework’s effectiveness. Our findings indicate improvements in anomaly detection capabilities, with enhanced interpretability and reliability, marking a significant contribution to log anomaly detection in heterogeneous environments.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

X-HEART: eXplainable heterogeneous log anomaly detection using robust transformers

  • Paul K. Mvula,
  • Paula Branco,
  • Guy-Vincent Jourdan,
  • Herna L. Viktor

摘要

Logs produced by heterogeneous systems are essential for analysing system behaviour and ensuring operational efficiency and security. Traditional log anomaly detection approaches often struggle with the variety of log formats and structures, as they heavily rely on log parsing, which can result in incomplete or flawed information extraction. In this work, we enhance our previously proposed HEART framework (HEterogeneous Log Anomaly Detection using Robust Transformers), a parsing-independent, end-to-end methodology that leverages Transfer Learning (TL) and Transformer models to process raw log events from diverse systems directly. We propose X-HEART (eXplainable HEterogeneous Log Anomaly Detection using Robust Transformers) to address the limitations of previous approaches and improve model transparency and interpretability. X-HEART incorporates SHAP (SHapley Additive exPlanations) innovatively by aggregating and analysing SHAP values from various components, offering a thorough understanding of model decisions for improved explainability. Our framework is evaluated on seven distinct log datasets, allowing for a more comprehensive intra- and cross-system analysis. Additionally, we employ extensive performance evaluation metrics to assess the framework’s effectiveness. Our findings indicate improvements in anomaly detection capabilities, with enhanced interpretability and reliability, marking a significant contribution to log anomaly detection in heterogeneous environments.