LogLAA: an adaptive integrated log anomaly analysis framework
摘要
Modern systems generate massive amounts of logs during operation, which are the key foundation for anomaly log analysis. However, existing research typically breaks down log analysis into multiple isolated tasks, which lacks flexibility in complex application scenarios and requires significant manpower. Furthermore, the increasing diversity and complexity of log formats place higher demands on the accuracy of log analysis. To achieve a more robust, accurate, and comprehensive log analysis method, we propose an integrated framework, called LogLAA. We construct a log parser based on length and word frequency that runs stably in most log systems with minimal parameter tuning, supporting both offline and online parsing in various scenarios. By introducing variable substitution and combining it with a similarity prefix tree, we achieve high accuracy and efficiency. We introduce counting embeddings, sequence embeddings, and semantic embeddings, and combining them with a CNN-LSTM model based on a dual-attention mechanism, we significantly improve the precision of anomaly detection. To ensure the interpretability of anomaly logs, we combine them with a large language model (LLM) for analysis. Experimental results show that our log parsing method achieves a 0.8% improvement over the SOTA model and anomaly detection achieves 6% improvement over the average precision of other advanced methods. We use the weighted matching score to evaluate anomaly analysis. LogLAA scores 0.7, placing it at an upper-middle level.