Functional near-infrared spectroscopy (fNIRS) is an important non-invasive neuroimaging technique used in cognitive neuroscience research to deeply analyze brain activity by measuring oxygenation and hemodynamic changes. Some current fNIRS classification models are simply adapted from deep learning models such as Transformer and fail to fully utilize the temporal and spatial features of fNIRS. In this paper, we propose a hybrid fNIRS classification network based on a novel spatial convolutional neural network (CNN), a temporal CNN, and Transformer, named as fNIRS-STCT. Primarily, to comprehensively capture temporal dynamics and aggregate spatial channel information, we employ two distinct large convolutional kernels for temporal and spatial CNN to extract temporal and spatial channel features, respectively. Second, spatial and temporal CNNs are respectively integrated with Transformer by feeding CNN-extracted features as input to Transformer and then concatenating CNN and Transformer features. Third, spatial and temporal features are fused by element-wise addition ways. The performance of the proposed fNIRS-STCT model is validated by using publicly available unilateral finger- and foot-tapping datasets. Across subject-dependent, subject-semi-dependent and subject-independent evaluation schemes, fNIRS-STCT model consistently outperforms other compared models for all given performance metrics. Specifically, our model achieves accuracies of 82.53%, 81.73%, and 79.60% across three evaluation schemes, surpassing the second-best model by margins of 3%, 2%, and 1%, respectively. These results highlight the capability of fNIRS-STCT to effectively learn and interpret mental states and brain activity across different subjects.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Spatio-Temporal Feature Classification Method for fNIRS Signals Based on a Hybrid CNN-Transformer Architecture

  • Li-Dan Kuang,
  • Yi-Xiao Wang,
  • Junwu Xie

摘要

Functional near-infrared spectroscopy (fNIRS) is an important non-invasive neuroimaging technique used in cognitive neuroscience research to deeply analyze brain activity by measuring oxygenation and hemodynamic changes. Some current fNIRS classification models are simply adapted from deep learning models such as Transformer and fail to fully utilize the temporal and spatial features of fNIRS. In this paper, we propose a hybrid fNIRS classification network based on a novel spatial convolutional neural network (CNN), a temporal CNN, and Transformer, named as fNIRS-STCT. Primarily, to comprehensively capture temporal dynamics and aggregate spatial channel information, we employ two distinct large convolutional kernels for temporal and spatial CNN to extract temporal and spatial channel features, respectively. Second, spatial and temporal CNNs are respectively integrated with Transformer by feeding CNN-extracted features as input to Transformer and then concatenating CNN and Transformer features. Third, spatial and temporal features are fused by element-wise addition ways. The performance of the proposed fNIRS-STCT model is validated by using publicly available unilateral finger- and foot-tapping datasets. Across subject-dependent, subject-semi-dependent and subject-independent evaluation schemes, fNIRS-STCT model consistently outperforms other compared models for all given performance metrics. Specifically, our model achieves accuracies of 82.53%, 81.73%, and 79.60% across three evaluation schemes, surpassing the second-best model by margins of 3%, 2%, and 1%, respectively. These results highlight the capability of fNIRS-STCT to effectively learn and interpret mental states and brain activity across different subjects.