Low-Rank Guided Attention with Wavelet Augmentation for Sequential Recommendation

Transformer-based deep learning models have achieved remarkable success in sequential recommendation. However, their attention mechanism still faces two inherent challenges: its high representational capacity tends to overfit noise; and it suffers from the oversmoothing problem, making it insufficiently sensitive to high-frequency signals in user behavior sequences. To address these issues, we propose LoWARec, which introduces structural priors into the Transformer through a low-rank attention to alleviate the impact of noise and captures multi-scale user behavior patterns in the frequency domain to mitigate the oversmoothing problem. Specifically, LoWARec first employs a low-rank attention to compress long interaction sequences into compact latent interests, thereby introducing structural priors. It then fuses these latent interests with the original sequences, seamlessly integrating low-rank representations into the Transformer to alleviate overfitting. Finally, it applies Fourier and wavelet transforms to extract low- and high-frequency components, thereby refining user preference modeling and mitigating the oversmoothing problem. Extensive experiments on five public datasets show that LoWARec consistently outperforms eight baselines, achieving up to 4.48% and 7.07% HR@5 improvement on Sports and LastFM, respectively. The code is released at https://github.com/juexuan/LoWARec .