A Multimodal Fusion Framework Employing Full-Stage Attention for Survival Prediction
摘要
Current approaches to multimodal survival analysis based on pathology and genomics often apply attention mechanisms at a single stage, thereby missing fine-grained intra-modal details and cross-modal synergies. We propose a novel framework—Multimodal Fusion Framework with Full-stage Attention (MFF-FA)—which integrates attention across three stages: feature extraction, modality alignment, and modality fusion. Specifically, we introduce En-UNI for pathology image sampling and representation, and employ attention modules to enhance latent genomic feature interactions. A convolution-based cross-attention module is further developed to model inter-modal correlations between pathology and genomics. Finally, a gating-attention fusion strategy is applied to retain modality-specific characteristics while enhancing their integration. Comparative evaluations demonstrate that MFF-FA outperforms existing state-of-the-art models in predictive accuracy for cancer survival analysis.