Scholarlink

<p>Federated Learning (FL) is a promising alternative to traditional machine learning, enabling multiple entities to collaboratively learn a model without sharing sensitive data with a central server. Most of the current FL methods focus on datasets from a single modality, such as images or text. However, the proliferation of different types of sensors, wearable devices, and other varieties of data collection methods has led to a growing interest in Multi-Modal Federated Learning (MMFL). In this survey, we provide a comprehensive review of 75 centralized MMFL systems, introducing a taxonomy that facilitates the comparison of existing solutions. We classify multimodal federated systems as homogeneous, hybrid, or heterogeneous, based on data modalities and model architectures. We analyze the role of the server and clients in managing multimodality and resulting challenges, such as modality alignment, missing modalities, and statistical heterogeneity. Furthermore, we discuss the strategies employed for multimodal aggregation, including direct model averaging, knowledge distillation, and adaptive aggregation schemes. Finally, we highlight key research directions essential for making MMFL more robust and reliable for real-world applications, with the aim of fostering future developments in the field.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A survey on multimodal federated learning

  • 期刊论文

  • 发表日期：

    2026年5月27日

查看全文
  • Silvia Grosso,
  • Nawel Benarba,
  • Sara Bouchenak,
  • Roberto Esposito,
  • Mirko Polato

摘要

Federated Learning (FL) is a promising alternative to traditional machine learning, enabling multiple entities to collaboratively learn a model without sharing sensitive data with a central server. Most of the current FL methods focus on datasets from a single modality, such as images or text. However, the proliferation of different types of sensors, wearable devices, and other varieties of data collection methods has led to a growing interest in Multi-Modal Federated Learning (MMFL). In this survey, we provide a comprehensive review of 75 centralized MMFL systems, introducing a taxonomy that facilitates the comparison of existing solutions. We classify multimodal federated systems as homogeneous, hybrid, or heterogeneous, based on data modalities and model architectures. We analyze the role of the server and clients in managing multimodality and resulting challenges, such as modality alignment, missing modalities, and statistical heterogeneity. Furthermore, we discuss the strategies employed for multimodal aggregation, including direct model averaging, knowledge distillation, and adaptive aggregation schemes. Finally, we highlight key research directions essential for making MMFL more robust and reliable for real-world applications, with the aim of fostering future developments in the field.