EEG and EMG-Based Multimodal Driver Drowsiness Detection: A CWT and Improved VGG-16 Pipeline

Monitoring distracted driving is crucial for ensuring road safety and avoiding the economic and social consequences of loss. Most past research has focused solely on EEG or EMG; hence, they are liable for delivering inaccurate results. Integrated methods utilize the advantages of EEG and EMG while eliminating their disadvantages. Therefore, it is preferable to merge as many options as possible to improve the accuracy of sleepiness identification. This article presents a novel electroencephalogram (EEG) and electromyogram (EMG)-based multimodal drowsiness detection system. Initially, an experiment simulating driving was undertaken to gather EEG and EMG data in alert and drowsy states. Continuous wavelet transformation (CWT) has been utilized for EEG and EMG signals, converting time-domain representations to time–frequency representations. A pre-trained VGG16 model has been used to classify the discriminating features. In the architecture of the enhanced VGG16, several convolutional layers have been eliminated, and new layers have been incorporated into the fully connected unit. In addition, the annotated time–frequency images are utilized in the process of fine-tuning the higher levels of the neural network design. The proposed multimodal system detected driver drowsiness with a validation accuracy of 92.50%. Therefore, the proposed EEG and EMG-based multimodal system with CWT and an enhanced VGG-16 pipeline will result in a more reliable method for detecting driver drowsiness.