Log Anomaly Detection Based on Semantic Features and Topic Features

System logs serve as crucial data sources for monitoring system performance and enhancing service quality. Many existing log-based anomaly detection methods primarily focus on detecting anomalies through parsed log templates. However, they often overlook valuable information such as the components and anomaly levels present in the original logs. These details provide crucial context about the logs’ origin and importance, serving as effective auxiliary information for the anomaly detection task. And recent studies only focus on log semantics, with a single log feature and the performance of the model is easily affected by template changes. Therefore, we propose a log anomaly detection method based on semantic feature and topic feature, LogST. LogST combines component and level information on the basis of log templates to construct a multi-information fused log sequence. The BERT model is used to extract log sentence vectors, and the SVD algorithm is used to reduce dimensionality to obtain efficient semantic features. Meanwhile, the LDA topic model is used to extract the topic features of log sequences and fully explore the distribution of different templates in normal and abnormal sequences. In addition, in order to highlight the key features and make full use of the anomaly log knowledge, we design a weighted residual-connected TCN model, which consists of a multilayer convolutional architecture for log sequences and uses attention mechanism to weight and aggregate hidden features of different convolutional layers. The experimental results on public datasets indicate that LogST has better performance than existing methods.