<p>Online change detection forms the foundation for numerous applications across various fields, including robotics, finance, and data mining. Despite being a long-standing research problem, it remains pertinent today. Numerous strategies and methods have been developed to address this issue. However, most struggle in the presence of outliers or heavy-tailed noise, including the two-sample t-statistic test. To address this, data often requires preprocessing to remove outliers, a task that becomes particularly challenging when data is presented as evolving streams over time. In this work, we introduce a novel method that enhances the performance of the latter, assuming that the observations follow either a normal distribution or a t-distribution with unknown parameters. Our approach involves proposing a new estimator in the sense of Maximum a Posteriori (MAP), which maximizes the conditional probability of the observations considering a Markov Random Field (MRF). By applying Bayes’ theorem, we reduce the problem to a global optimization problem that is straightforward to solve. Our numerical results demonstrate a significant improvement in the proportion of false positives while maintaining the same average detection delay compared to the state-of-the-art methods. We further assess the effectiveness of our method through the online analysis of real well-log data and DNA copy number variation data.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Online change detection for a shift in mean in the presence of outliers

  • Zakariae Drabech,
  • Mohammed Douimi,
  • Elmoukhtar Zemmouri

摘要

Online change detection forms the foundation for numerous applications across various fields, including robotics, finance, and data mining. Despite being a long-standing research problem, it remains pertinent today. Numerous strategies and methods have been developed to address this issue. However, most struggle in the presence of outliers or heavy-tailed noise, including the two-sample t-statistic test. To address this, data often requires preprocessing to remove outliers, a task that becomes particularly challenging when data is presented as evolving streams over time. In this work, we introduce a novel method that enhances the performance of the latter, assuming that the observations follow either a normal distribution or a t-distribution with unknown parameters. Our approach involves proposing a new estimator in the sense of Maximum a Posteriori (MAP), which maximizes the conditional probability of the observations considering a Markov Random Field (MRF). By applying Bayes’ theorem, we reduce the problem to a global optimization problem that is straightforward to solve. Our numerical results demonstrate a significant improvement in the proportion of false positives while maintaining the same average detection delay compared to the state-of-the-art methods. We further assess the effectiveness of our method through the online analysis of real well-log data and DNA copy number variation data.