Multi-input CNN Based Classification of EEG and NIRS Signal During Voluntary Hand Movement

The brain-computer interface (BCI) is a communication method between the brain and computer or electronic devices. An electroencephalography (EEG) signal is a popular way to create a traditional BCI system. However, EEG signals can be sensitive and prone to interference from both external and internal noise, complicating system design and potentially leading to malfunctions in the BCI system. Near-infrared spectroscopy (NIRS), which is less susceptible to external noise, complements EEG in BCI systems. However, the sampling frequencies of EEG and NIRS signals differ significantly. It needs some complicated process to combine them or design a system that considers the characteristics of both data. This study explores the use of a flexible convolutional neural network (CNN) architecture to mitigate the impact of disparate sampling frequencies on accurately classifying left- and right-hand movements using raw EEG and NIRS data. By employing a multi-input CNN approach, the study aims for significant improvements in classification accuracy. The results suggest that the proposed model achieves high accuracy in classifying tasks using participant-dependent cross-validation. Therefore, the proposed model can classify a label with high accuracy, and the utilization of multi-input methods supports addressing measurements-related issues, such as temporal resolution, in BCI systems. EEG is superior to NIRS in estimating the localization of brain function. However, NIRS excels in estimating relative changes in blood flow. Creating a system that exploits both advantages is essential for BCI applications.