Federated Machine Learning

While presenting the basics of machine learning in the previous chapter, we assumed that we had access to a training sample \(\mathcal {S}\) comprising data points that are (independently) drawn from the ground-truth distribution \(\mathcal {D}\) . We did not, however, explicitly mention how this dataset was collected or stored. A dominant practice in machine learning for many years has been to collect and store user-generated data on a single machine, which then uses the data to train a model. Although this approach is effective in training accurate models, it raises serious privacy concerns about the utilization of user data. Furthermore, training the model centrally, i.e., on a single machine, is becoming increasingly challenging due to the growing size of the models and the amount of data involved in modern machine learning tasks. To circumvent these limitations, a new paradigm has emerged: federated machine learning. This learning scheme consists in having several machines train a model in collaboration while keeping their data local. The computational load is distributed over multiple machines, and users can retain control over their local data. This paradigm is also sometimes coined as collaborative or simply distributed learning. In this chapter, we formalize the problem of federated machine learning in a server-based framework and present how to adapt gradient-descent methods in this context.