<p>We introduce distance-based CUSUM statistics for detecting change points in high dimensional data streams. Unlike the standard CUSUM statistic which can mainly detect linear changes such as shifts in the mean of observations, the distance-based CUSUM statistics are constructed based on pairwise dissimilarity distances between observations and hence capable of detecting more general types of change points including linear and non-linear changes in a data stream, such as changes in the mean, variance, correlation, or other changes in the shape of distribution over time. Moreover, the distance-based CUSUM method is particularly useful for high dimensional low sample size (HDLSS) data in which the number of observations is very small but the dimension is very large. Detecting change points in such high dimensional data is an understudied problem. We study the properties of our proposed distance-based CUSUM statistic and use it to develop a non-parametric test to determine statistical significance of the estimated change point locations. Our approach does not require normality or any other distribution for the data. We provide theoretical guarantees for our method and demonstrate its empirical performance in comparison with some of the recent methods via extensive simulation studies and two real data applications. We provide an R package called <Emphasis FontCategory="NonProportional">distCUSUM</Emphasis> for implementation of the proposed method.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Distance-based CUSUM statistics for high dimensional change points

  • Lupeng Zhang,
  • Reza Drikvandi

摘要

We introduce distance-based CUSUM statistics for detecting change points in high dimensional data streams. Unlike the standard CUSUM statistic which can mainly detect linear changes such as shifts in the mean of observations, the distance-based CUSUM statistics are constructed based on pairwise dissimilarity distances between observations and hence capable of detecting more general types of change points including linear and non-linear changes in a data stream, such as changes in the mean, variance, correlation, or other changes in the shape of distribution over time. Moreover, the distance-based CUSUM method is particularly useful for high dimensional low sample size (HDLSS) data in which the number of observations is very small but the dimension is very large. Detecting change points in such high dimensional data is an understudied problem. We study the properties of our proposed distance-based CUSUM statistic and use it to develop a non-parametric test to determine statistical significance of the estimated change point locations. Our approach does not require normality or any other distribution for the data. We provide theoretical guarantees for our method and demonstrate its empirical performance in comparison with some of the recent methods via extensive simulation studies and two real data applications. We provide an R package called distCUSUM for implementation of the proposed method.