An unsupervised noise-resistant method for detection of incremental drifts
摘要
Concept drift represents a challenge for online machine learning models, resulting in a loss of predictive performance if the models cannot quickly adapt to the changes in data distribution. Despite numerous efforts dealing with this issue in the literature, few of them have presented robust results on data stream-based industrial processes that usually present noise, outliers, and non-annotated data. In fact, most drift detection methods are supervised, focused on abrupt changes, and often struggle to deal with noisy data. Moreover, abrupt drifts are comparatively easier to identify than gradual or incremental. This study proposes an unsupervised drift detection method called the noise-resistant equal-intensity drift detection method (NR-EIDDM), which is focused on bringing robustness related to noise and incremental drifts. It comprises two clustering algorithms followed by a statistical distribution dissimilarity test. The proposed method was compared with well-known unsupervised drift detection methods using synthetic and real-world industrial datasets. The results showed that it was able to identify incremental drifts in a noisy environment. Moreover, NR-EIDDM outperformed the other unsupervised detection methods in the presence of noise and incremental drifts.