Contrastive learning has made significant progress in sentence representation, where methods such as DiffCSE have been evidenced to be able to enhance model representation capability in various prediction tasks. However, existing methods still have limitations in nonlinear feature transformation and sentence information aggregation. This paper proposes two key improvements: A novel framework InfoKANCSE to implement edge activation functions via B-spline curves based on Kolmogorov-Arnold Networks to enhance nonlinear modeling capability, and a multi-head attention mechanism to overcome mask token limitations by incorporating alignment and information maximization losses. Experimental results on seven semantic textual similarity (STS) tasks and BEIR benchmark demonstrate that these two improvements significantly boost the effectiveness the proposed method, achieving state-of-the-art performance on multiple downstream applications. Our proposed InfoKANCSE not only improves the representation expressiveness and interpretability but also offers valuable insights for sentence embedding research. InfoKANCSE achieves an average Spearman correlation of 82.4 on STS tasks and strong zero-shot performance on BEIR, surpassing prior methods such as InfoCSE and SimCSE.