SummFuseCare: extractive summarization enhancement and attention-based dual-modal fusion network for clinical risk prediction
摘要
Leveraging multimodal data from electronic health records (EHR) is crucial for improving clinical risk prediction, yet existing methods often fall short in effectively integrating structured data (e.g., lab tests) and unstructured clinical notes. Challenges such as underutilization of textual information and redundant learning across modalities persist, limiting model performance and interpretability. To address these issues, we propose SummFuseCare, a novel model designed to enhance extractive summarization and integrate multimodal EHR data via an attention-based fusion network. Our approach first employs extractive summarization on clinical notes to distill key sentences that capture core medical events, thereby simplifying textual input while preserving critical semantic context. Secondly, a dynamic-static feature learning module processes structured data, separately capturing static patient demographics and the dynamic trajectories of laboratory indicators to represent both health status and temporal disease progression. Finally, an attention-based dual-modal fusion network integrates the summarized text features with the structured features. This module uses a cross-modal masking mechanism to suppress redundancy and encourage complementary feature learning between modalities. Extensive experiments on the public MIMIC-III and MIMIC-IV datasets for hospital mortality prediction demonstrate that SummFuseCare significantly outperforms strong baseline models. The results validate the effectiveness of our method in achieving more accurate and robust clinical risk assessment by making fuller and more efficient use of multimodal EHR data.