A Survey on Advances of Foundation Models in Federated Learning

Foundation models (FM) have achieved significant progress in recent years, with large scale architectures proving to be highly effective in enhancing model performance. Federated learning (FL) offers a compelling paradigm to train powerful foundation models by leveraging high-quality, private data from edge users, overcoming the limitations of traditional training pipelines, which are often restricted to public data due to privacy concerns. However, integrating these massive models, often with hundreds of billions of parameters, into federated systems presents substantial challenges, most notably prohibitive communication overhead and the severe computational constraints of edge devices. This survey provides a comprehensive review of advancements in Foundation Models in Federated Learning (FMFL), charting critical progress across key dimensions. We explore pioneering efforts in federated pre-training, techniques for efficient fine-tuning, strategies for ensuring trustworthiness, and examine real-world applications. By mapping the landscape and identifying key challenges, this survey offers timely insights into this rapidly evolving field and illuminates future research directions for developing advanced and practical FMFL systems.