KPIs (key performance indicators) reflect the operational state of Cloud services and infrastructure. With the rapid development of cloud services, many dimensional KPIs are constantly generated during service operations. Recently, deep-learning-based Multivariate KPI Anomaly Detection (MAD) methods have become the mainstream trend to meet the requirements of effective anomaly detection. As the number of KPI dimensions grows, the redundant dimensions in anomaly detection increase. Data with high dimensional complexity increases computational costs and storage requirements. This phenomenon is called the “Curse of dimensionality", which prevents common MAD strategies from being efficient. Therefore, dimensionality reduction is important in MAD. However, there are few researches on dimensionality reduction methods specifically designed for MAD. In this paper, we propose a feature selection KPI dimensionality reduction framework ITRMD, which reduces multivariate KPI series from two aspects: inter-metric dimensionality selection and temporal dimensionality selection. ITRMD preserves features of original data by selecting KPI dimensions with effective information and discards redundant KPI dimensions. Experiments on real KPI datasets show that ITRMD can greatly improve the detection accuracy of the state-of-the-art MAD and reduce the computation cost of model training.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ITRMD: A Dimensionality Reduction Framework for Accurate and Efficient Multivariate KPI Anomaly Detection

  • Tianrun Gao,
  • Decheng Zuo,
  • Yanjun Shu,
  • Zhan Zhang,
  • Dongxin Wen,
  • Yutong Qu

摘要

KPIs (key performance indicators) reflect the operational state of Cloud services and infrastructure. With the rapid development of cloud services, many dimensional KPIs are constantly generated during service operations. Recently, deep-learning-based Multivariate KPI Anomaly Detection (MAD) methods have become the mainstream trend to meet the requirements of effective anomaly detection. As the number of KPI dimensions grows, the redundant dimensions in anomaly detection increase. Data with high dimensional complexity increases computational costs and storage requirements. This phenomenon is called the “Curse of dimensionality", which prevents common MAD strategies from being efficient. Therefore, dimensionality reduction is important in MAD. However, there are few researches on dimensionality reduction methods specifically designed for MAD. In this paper, we propose a feature selection KPI dimensionality reduction framework ITRMD, which reduces multivariate KPI series from two aspects: inter-metric dimensionality selection and temporal dimensionality selection. ITRMD preserves features of original data by selecting KPI dimensions with effective information and discards redundant KPI dimensions. Experiments on real KPI datasets show that ITRMD can greatly improve the detection accuracy of the state-of-the-art MAD and reduce the computation cost of model training.