An Automated and Verbose Approach for Detecting Anomalies in Cloud Computing Platform Using Logs

Logs represent a valuable source of runtime information, serving various purposes such as monitoring, diagnosing issues, assessing performance, and facilitating maintenance. However, the task of automating real-time anomaly detection based on logs remains highly challenging. The process of parsing log messages is intricate and prone to errors. Furthermore, identifying relationships between logs is often unfeasible, particularly in complex systems like cloud computing platforms. We propose here Monilog, an automated system designed for log-based anomaly detection in cloud computing environments. Monilog introduces a novel approach to parsing and encoding logs, enabling the creation of appropriate inputs for traffic forecasting models. By generating comprehensive summaries of detected anomalies, Monilog empowers practitioners to develop intelligent alerting systems on top of it. Our evaluation, conducted using real-life log data obtained from Kernel Based Virtual Machine at the scale of a cloud region, highlights the effectiveness of Monilog in accurately forecasting server failures and providing meaningful insights into reported anomalies.