Failures in the client-server model spread server failures to clients. For example, in a system using an NFS server and clients, when file system usage on the server exceeds 80%, the file system usage on the clients also reaches 80%. Both the NFS server and client file system usage are monitored, and alerts are sent from both locations. Failure location and unaffected locations are included when an alert is sent. On-call engineers take time to identify the alerts related to the root cause of the system failure. This paper proposes an algorithm that reduces alerts by analyzing connection relationships for each application. The algorithm calculates weights based on the established connection generated by each application. The number of alerts is reduced by focusing only on the target with the highest weight. The experiment reproduced failures in NFS server and client and RKE2 cluster. Evaluation compared the number of alerts between using Alertmanager and a combination of Alertmanager and the proposed method. Four alerts were sent using Alertmanager, and one alert was sent using the combination method in NFS server and client failures. Three alerts were sent using Alertmanager, and one alert was sent using the combination method in RKE2 cluster failure. The experiment results showed that the proposed method reduces redundant alerts and limits alerts to the failure location only.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Reduction of Redundant Alerts by Relationship Discovery Based on Port Usage

  • Manato Hirao,
  • Tomoyuki Koyama,
  • Takayuki Kushida

摘要

Failures in the client-server model spread server failures to clients. For example, in a system using an NFS server and clients, when file system usage on the server exceeds 80%, the file system usage on the clients also reaches 80%. Both the NFS server and client file system usage are monitored, and alerts are sent from both locations. Failure location and unaffected locations are included when an alert is sent. On-call engineers take time to identify the alerts related to the root cause of the system failure. This paper proposes an algorithm that reduces alerts by analyzing connection relationships for each application. The algorithm calculates weights based on the established connection generated by each application. The number of alerts is reduced by focusing only on the target with the highest weight. The experiment reproduced failures in NFS server and client and RKE2 cluster. Evaluation compared the number of alerts between using Alertmanager and a combination of Alertmanager and the proposed method. Four alerts were sent using Alertmanager, and one alert was sent using the combination method in NFS server and client failures. Three alerts were sent using Alertmanager, and one alert was sent using the combination method in RKE2 cluster failure. The experiment results showed that the proposed method reduces redundant alerts and limits alerts to the failure location only.