Provenance-Based Dynamic Fine-Tuning of Cross-Silo Federated Learning

Federated Learning (FL) is a distributed technique that allows multiple users to train models collaboratively without accessing private and sensitive data. Iteratively, each user trains a “local” model in a specific machine consuming private data and then sends the model updates to a server for their fusion into a centralized one. Although FL represents a step forward, the training duration in each iteration directly depends on the several configurations set, e.g., hyperparameters. Analyzing hyperparameters during the FL workflow allows for dynamic fine-tuning that can improve the performance of FL regarding training time and quality of results. However, due to its exploratory nature, the user may lose track of which configurations have been used to train the model with the best accuracy if the choices are not correctly registered. Provenance is the natural choice to represent data derivation traces to help hyperparameters fine-tuning by providing a global data-oriented picture of the FL workflow. Yet, the existing FL frameworks do not provide dynamic fine-tuning nor support provenance capturing. Therefore, this paper introduces an FL framework named Flower-PROV that uses provenance data for tracking configurations and evaluation metrics during the FL execution to allow for dynamic fine-tuning of hyperparameters, thus saving training time. We show a use case with Cross-Silo FL where Flower-PROV dynamic fine-tuning reduced the FL training time up to 94.24% when compared with the fine-tuning using grid-search.