Federated Learning for Privacy-Preserving Next-Word Prediction on Mobile Devices

Due to the increase in the use of mobile devices, the usability and the probability of input errors is improved via the use of predictive text and even next-word suggestions. However, the typical approaches used in next-word predictions require a lot of user data which causes problems arising from privacy concerns. This article proposes a method of federated learning (FL) for next-word prediction that is carried out on the devices themselves so as to avoid centralizing any user data. The data from the Shakespeare dataset is used to demonstrate a federated training paradigm implemented over devices in a TFF environment. In this case, an existing pre-trained RNN model is fine-tuned to cater for the data from each client. Clients, instead of sending raw data, send encrypted model updates to a central aggregator. This helps to protect privacy while still working toward the improvement of the global model. As enhancements, tokenization, vocabulary lookups, and length constraints are implemented, since these are required for use in mobile devices which are limited in resources. The experiments back the usefulness of this FL-based approach for privacy-aware text prediction (Thakre et al., in Int J Innov Res Comput Commun Eng 11(10):11332–11343, 2023), as it does not lose significantly in prediction metrics. The research shows federated learning can deliver secure and customized experiences across a range of mobile devices. To improve adaptability and user satisfaction, future work will focus on increasing the model’s context awareness and lexicon growth.