When SpikeKANFormer Meets Angular Contrastive Learning

In recent years, Spiking Transformer has emerged as a promising architecture for neuromorphic computing, leveraging MLPs as self-attention computation units to achieve competitive performance. However, this approach faces two critical limitations: the inherent constraints of MLPs in modeling highly nonlinear spatio-temporal dynamics in spiking data, and the lack of robust supervised representation learning mechanisms for spike-based paradigms. To address these challenges, we propose SpikeKANFormer, a novel architecture that replaces traditional MLPs with Kolmogorov-Arnold Networks (KANs). KANs enhance the model’s capacity to approximate complex spike-timing dependencies, thanks to their theoretically grounded function composition and adaptive structure, while also improving interpretability. Furthermore, we introduce Angular Contrastive Learning Loss (ACL) to tackle the representation learning bottleneck. By exploiting the temporal dynamics of Spiking Neural Networks (SNNs), ACL constructs positive and negative sample pairs across different time steps and optimizes their angular separability in feature space. This approach not only mitigates the reliance on labeled data but also strengthens feature discrimination for downstream tasks. Extensive experiments demonstrate that SpikeKANFormer, combined with Angular Contrastive Learning Loss, achieves state-of-the-art performance across multiple neuromorphic vision and text benchmarks.