This study introduces a fatigue monitoring model based on multimodal physiological parameters. The proposed model utilizes a multi-branch convolutional neural network (CNN) architecture, incorporating four distinct feature extraction branches: a global index branch, a video behavior index branch, an eye movement index branch, and an electrocardiogram (ECG) index branch. By integrating 2D convolutional layers, residual modules, and 1D convolutional layers, the model effectively captures multi-scale spatial features and temporal dynamic features from diverse modalities. Through dual-level fusion of extracted features and decision-making processes, combined with adaptive decision optimization, the model achieves precise classification of multimodal physiological data. Experimental results demonstrate the model’s superior performance in 5-fold cross-validation, achieving an average accuracy of 91.63%, precision of 91.88%, recall of 91.63%, F1 score of 0.92, and a Kappa coefficient of 0.89. These findings validate the robustness and effectiveness of the model for long-endurance flight fatigue monitoring tasks.