Open-Source Drift Detection Tools in Action: Insights from Two Use Cases

Data drifts pose a critical challenge in the lifecycle of machine learning (ML) models, affecting their performance and reliability. In response to this challenge, we present a benchmark study, called D3Bench  (Source code is available at https://github.com/mohamedyd/D3Bench ), which evaluates the efficacy of open-source drift detection tools. D3Bench examines the capabilities of Evidently AI, NannyML, and Alibi-Detect, leveraging real-world data from two smart building use cases. We prioritize assessing the functional suitability of these tools to identify and analyze data drifts. Our findings reveal that Evidently AI stands out for its general data drift detection, whereas NannyML excels at pinpointing the precise timing of shifts and evaluating their consequent effects on predictive accuracy.