Understanding the Phonetic Similarity in Imagined Speech Using Cross-Task Saliency Mapping
摘要
Imagined speech electroencephalogram (EEG) provides a promising avenue for decoding internal language representations in brain-computer interfaces, yet the relationship between phonologically similar words and their corresponding EEG patterns during imagined articulation remains underexplored. To address this question, Multi-Scale EEGNet (MS-EEGNet), a modified version of the established EEGNet architecture with multi-scale frequency-aware capabilities, was developed. This framework was evaluated on two classification paradigms: binary classification grouping /iy/-phoneme-containing words and multi-class classification treating each word distinctly, while Gradient-weighted Class Activation Mapping (Grad-CAM) was employed to generate saliency maps for assessing activation overlap across tasks. Consistent performance exceeding 97% on all binary tasks and 96.43% on multi-class imagined speech decoding was demonstrated. Importantly, significant temporal overlap in saliency maps from different tasks was exhibited, as quantified through Jaccard similarity and validated statistically. This indicates the presence of task-invariant neural features linked to phonological structure. These findings suggest that imagined speech representations are organized along phonetic dimensions, offering new insights into neural language encoding and promising directions for efficient EEG-based speech decoding systems.