<p>Concept drift represents a challenge for online machine learning models, resulting in a loss of predictive performance if the models cannot quickly adapt to the changes in data distribution. Despite numerous efforts dealing with this issue in the literature, few of them have presented robust results on data stream-based industrial processes that usually present noise, outliers, and non-annotated data. In fact, most drift detection methods are supervised, focused on abrupt changes, and often struggle to deal with noisy data. Moreover, abrupt drifts are comparatively easier to identify than gradual or incremental. This study proposes an unsupervised drift detection method called the noise-resistant equal-intensity drift detection method (NR-EIDDM), which is focused on bringing robustness related to noise and incremental drifts. It comprises two clustering algorithms followed by a statistical distribution dissimilarity test. The proposed method was compared with well-known unsupervised drift detection methods using synthetic and real-world industrial datasets. The results showed that it was able to identify incremental drifts in a noisy environment. Moreover, NR-EIDDM outperformed the other unsupervised detection methods in the presence of noise and incremental drifts.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An unsupervised noise-resistant method for detection of incremental drifts

  • Daniel da Silva Diogo Lara,
  • Rodrigo Amador Coelho,
  • Cristiano Leite de Castro,
  • Walmir Matos Caminhas

摘要

Concept drift represents a challenge for online machine learning models, resulting in a loss of predictive performance if the models cannot quickly adapt to the changes in data distribution. Despite numerous efforts dealing with this issue in the literature, few of them have presented robust results on data stream-based industrial processes that usually present noise, outliers, and non-annotated data. In fact, most drift detection methods are supervised, focused on abrupt changes, and often struggle to deal with noisy data. Moreover, abrupt drifts are comparatively easier to identify than gradual or incremental. This study proposes an unsupervised drift detection method called the noise-resistant equal-intensity drift detection method (NR-EIDDM), which is focused on bringing robustness related to noise and incremental drifts. It comprises two clustering algorithms followed by a statistical distribution dissimilarity test. The proposed method was compared with well-known unsupervised drift detection methods using synthetic and real-world industrial datasets. The results showed that it was able to identify incremental drifts in a noisy environment. Moreover, NR-EIDDM outperformed the other unsupervised detection methods in the presence of noise and incremental drifts.