Handling Missing Modalities in Multimodal Federated Learning for Healthcare Data Analytics

Accurate disease diagnosis increasingly depends on integrating diverse clinical data such as health records, medical images, and clinical notes. However, real-world medical datasets are often incomplete and distributed across hospitals under strict privacy regulations, which limiting the development of robust Machine Learning (ML) models. This study investigates multimodal Federated Learning (FL) as a privacy-preserving framework for collaborative model training without sharing raw patient data. Using a selected subset of 312 patients from the MIMIC-IV database containing complete EHR, medical imaging, and radiology note modalities, neural encoders were trained for each modality and distributed across simulated hospital nodes to simulate federated training. The predictive objective was to classify in-hospital mortality. Generative AI tools, including ChatGPT and Claude, were employed to synthesize missing radiology notes to address incomplete modalities. The results indicate that multimodal inputs yield higher predictive accuracy than unimodal input. Employing generative AI to synthesize missing modalities problem effectively restores model performance, while federated learning preserves patient privacy without compromising predictive accuracy.