Mental Workload Assessment in Human–Computer Interaction Multitasking Environment Based on Multimodal Physiological Signals

Based on the second generation of Multi-Attribute Task Battery-II (MATB-II) experimental paradigm of concurrent multi-task, this paper collects the multi-modal physiological signals and subjective mental workload data of NASA-TLX scale during the task completion process of experimental objects, proposes a mental workload classification model based on multi-modal physiological signal feature analysis and pattern recognition, and compares the classification and recognition effects of different modal physiological signals and their combinations in three typical machine learning algorithms (Random forests, decision trees, and k-nearest neighbor models). The results show that, among the classification models based on single-modal physiological signals, the classification models based on skin electrical, electrocardiographic, and EEG signals increase in accuracy in turn; the classification models based on multi-modal physiological signals are generally better than the single-modal classification models; the random forest classification model based on the three modalities of EEG, ECG, and skin electrical physiological signals has the highest classification accuracy. In occupational settings, the interaction between perceived mental workload and physical health effects should be considered, as workers often face both physical and mental demands at the same time. Controlling the mental workload of operators within reasonable limits can reduce human error.