<p>Sequential recommendation aims to predict users’ next interactions based on their historical behavior. However, existing models suffer from three major limitations: (1) reliance solely on collaborative filtering signals while overlooking item semantic information; (2) vulnerability to high-frequency noise in behavior sequences, which interferes with representation learning; and (3) computational inefficiency in Transformer-based bidirectional architectures. To address these challenges, we propose AFNBM (Adaptive Fusion via Noise-filtered Bidirectional Mamba), a novel framework that introduces three key innovations. <b>First</b>, we develop an adaptive semantic-collaborative fusion module that leverages pre-trained language models to extract item semantic embeddings and integrates them with collaborative embeddings through learnable parameters <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\varvec{\alpha }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">α</mi> </mrow> </math></EquationSource> </InlineEquation> and <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\varvec{\beta }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">β</mi> </mrow> </math></EquationSource> </InlineEquation>. This mechanism effectively mitigates data sparsity and cold-start problems. <b>Second</b>, we integrate selective frequency modulation (SFM) into a bidirectional Mamba encoder, making this the first work to combine frequency-domain adaptive denoising with state space models (SSMs) for sequential recommendation. The bidirectional encoder captures temporal dependencies in both directions during training while the Mamba blocks themselves maintain linear complexity <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\varvec{O(L)}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">O</mi> <mo mathvariant="bold" stretchy="false">(</mo> <mi mathvariant="bold-italic">L</mi> <mo mathvariant="bold" stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation>. Each Mamba block incorporates an SFM layer with learnable parameter <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\varvec{\gamma }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">γ</mi> </mrow> </math></EquationSource> </InlineEquation>, enabling adaptive control over high-frequency component retention. The SFM operation introduces an <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(\varvec{O(L \log L)}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">O</mi> <mo mathvariant="bold" stretchy="false">(</mo> <mi mathvariant="bold-italic">L</mi> <mo mathvariant="bold">log</mo> <mi mathvariant="bold-italic">L</mi> <mo mathvariant="bold" stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> factor, making the overall framework complexity <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(\varvec{O(L \log L)}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">O</mi> <mo mathvariant="bold" stretchy="false">(</mo> <mi mathvariant="bold-italic">L</mi> <mo mathvariant="bold">log</mo> <mi mathvariant="bold-italic">L</mi> <mo mathvariant="bold" stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation>, which remains significantly more efficient than Transformer-based models with <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(\varvec{O(L}^{\varvec{2}}\varvec{)}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mrow> <mi mathvariant="bold-italic">O</mi> <mo mathvariant="bold" stretchy="false">(</mo> <mi mathvariant="bold-italic">L</mi> </mrow> <mrow> <mn mathvariant="bold">2</mn> </mrow> </msup> <mrow> <mo mathvariant="bold" stretchy="false">)</mo> </mrow> </mrow> </math></EquationSource> </InlineEquation> complexity while simultaneously improving computational efficiency and noise robustness. <b>Third</b>, we construct a comprehensive end-to-end framework built upon the adaptive fusion and bidirectional Mamba-SFM architecture. For prediction, we employ a gated linear unit (GLU) to selectively enhance salient features, followed by a masked-terminal prediction layer. Extensive experiments validate the effectiveness of our design: ablation studies quantify each component’s contribution (e.g., GLU yields +2.2% to +5.2% gains across datasets), while AFNBM achieves 1.02%-11.76% improvements in Hit@10 and 2.84%-7.57% improvements in NDCG@10 over state-of-the-art baselines across four benchmark datasets, with particularly strong results on sparse and cold-start scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive multi-embedding fusion via denoising bidirectional mamba for sequential recommendation

  • Tongfei Zeng,
  • Quan Wen,
  • Yujian Huang,
  • Yanmei Hu,
  • Cheng Wang

摘要

Sequential recommendation aims to predict users’ next interactions based on their historical behavior. However, existing models suffer from three major limitations: (1) reliance solely on collaborative filtering signals while overlooking item semantic information; (2) vulnerability to high-frequency noise in behavior sequences, which interferes with representation learning; and (3) computational inefficiency in Transformer-based bidirectional architectures. To address these challenges, we propose AFNBM (Adaptive Fusion via Noise-filtered Bidirectional Mamba), a novel framework that introduces three key innovations. First, we develop an adaptive semantic-collaborative fusion module that leverages pre-trained language models to extract item semantic embeddings and integrates them with collaborative embeddings through learnable parameters \(\varvec{\alpha }\) α and \(\varvec{\beta }\) β . This mechanism effectively mitigates data sparsity and cold-start problems. Second, we integrate selective frequency modulation (SFM) into a bidirectional Mamba encoder, making this the first work to combine frequency-domain adaptive denoising with state space models (SSMs) for sequential recommendation. The bidirectional encoder captures temporal dependencies in both directions during training while the Mamba blocks themselves maintain linear complexity \(\varvec{O(L)}\) O ( L ) . Each Mamba block incorporates an SFM layer with learnable parameter \(\varvec{\gamma }\) γ , enabling adaptive control over high-frequency component retention. The SFM operation introduces an \(\varvec{O(L \log L)}\) O ( L log L ) factor, making the overall framework complexity \(\varvec{O(L \log L)}\) O ( L log L ) , which remains significantly more efficient than Transformer-based models with \(\varvec{O(L}^{\varvec{2}}\varvec{)}\) O ( L 2 ) complexity while simultaneously improving computational efficiency and noise robustness. Third, we construct a comprehensive end-to-end framework built upon the adaptive fusion and bidirectional Mamba-SFM architecture. For prediction, we employ a gated linear unit (GLU) to selectively enhance salient features, followed by a masked-terminal prediction layer. Extensive experiments validate the effectiveness of our design: ablation studies quantify each component’s contribution (e.g., GLU yields +2.2% to +5.2% gains across datasets), while AFNBM achieves 1.02%-11.76% improvements in Hit@10 and 2.84%-7.57% improvements in NDCG@10 over state-of-the-art baselines across four benchmark datasets, with particularly strong results on sparse and cold-start scenarios.