Rapid and accurate prediction of clinical deterioration in emergency medicine is challenging due to heterogeneous data modalities, severe class imbalance, and complex temporal dynamics. We present MediFusion-Flex, an adaptive multimodal deep learning framework that integrates time-series physiological signals, categorical demographic and clinical features, and unstructured clinical notes through specialized encoders and a dynamic multihead attention fusion mechanism. A novel composite loss function combining focal, dice, and contrastive losses addresses class imbalance and enhances the quality of representation. Evaluated on three large-scale datasets: CNUH, MIMIC-III and eICU, our approach achieves state-of-the-art performance with average AUROC scores of 0.730, 0.968 and 0.858 and AUPRC of 0.172, 0.522 and 0.571, respectively. The framework reduces late alarm rates by 34% while maintaining sensitivity above 72%, demonstrating superior early warning capabilities compared to traditional clinical scores and contemporary machine learning baselines.