Monitoring Concept Drift in Continuous Federated Learning Platforms

Continuous federated learning (CFL), a recently emerging learning paradigm that facilitates collaborative, yet privacy-preserving machine learning (ML), bears the potential to shape the future of distributed ML. In spite of its great potential, it is - similar to continuous ML - prone to suffer from concept drift (a change in data properties over time). In turn, CFL can greatly benefit from employing drift detection to react adequately to emerging drifts. Although various such approaches exist, respective research lacks application of drift detection to CFL with dynamic client participation as well as detailed analysis of the advantages of different drift detection approaches such as error-based or data-based drift detection. To this end, we apply these drift detection approaches to a CFL platform that allows new clients to join even after the training has started and measure the negative impact of concept drift on model performance. Moreover, we uncover distinct differences between the error- and data-based drift detection. In particular, we find the former ones to be more suitable to detect the point in time where the joint models stops benefiting from concept drift whereas the latter allows for a more precise detection of the first occurrence of concept drift.