Silent Speech Interaction (SSI) technology decodes facial electromyography (EMG) signals or strain signals to provide novel communication solutions for high-noise environments and patients with aphasia. Traditional methods rely solely on independent time-domain features or frequency-domain features, limiting classifier performance in nonlinearly separable scenarios. This paper introduces a multimodal fusion method based on time-frequency joint representation. The proposed method combines four fundamental components: the time-domain feature extraction block, the frequency-domain feature extraction block, the multi-head self-attention based multimodal feature fusion block and the temporal convolution block. Experiments show that proposed approach achieves significant performance improvements across multiple open-source strain signal datasets. On the 20-class common vocabulary dataset, it attains an accuracy of 95.75%. On the 10-class confusable words dataset, it attains an accuracy of 96.50%. On the 5-class speech rate dataset, it achieves 100% accuracy. On the 100-class generalization dataset, it attains an accuracy of 90.15%. Multimodal fusion boosts silent speech recognition accuracy and robustness.