Conversational recommender systems (CRS) aim to capture user preferences through multi-turn dialogues and provide high-quality recommendations. Due to the limited conversation context and background knowledge, present CRS usually rely on external sources such as knowledge graphs to enrich the context and model entities based on their interrelations. However, while these methods mainly focus on current dialogue understanding, they fail to make adequate use of entities’ intrinsic information and users’ rich historical dialogue information. To address these issues, we propose Multimodal Knowledge-enhanced Sequence Modeling (MKSM). Specifically, we first develop a multimodal entity representation framework integrating knowledge graph(KG), text, and image features, which uses cross-modal alignment to solve semantic sparsity. Secondly, we adopt a position-aware self-attention mechanism to model entity mention order and capture temporal interest evolution. Then, we design an attention-weighted dynamic filtering strategy to select relevant historical segments based on the current dialogue context, which co-models short-term intents and long-term interests. The experimental results show that MKSM attains remarkable enhancements in both recommendation accuracy and response generation quality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Knowledge-Enhanced Sequence Modeling Method for Conversational Recommender Systems

  • Weina Zhang,
  • Wenmin Zu,
  • Zhongqin Bi,
  • Dan Dai

摘要

Conversational recommender systems (CRS) aim to capture user preferences through multi-turn dialogues and provide high-quality recommendations. Due to the limited conversation context and background knowledge, present CRS usually rely on external sources such as knowledge graphs to enrich the context and model entities based on their interrelations. However, while these methods mainly focus on current dialogue understanding, they fail to make adequate use of entities’ intrinsic information and users’ rich historical dialogue information. To address these issues, we propose Multimodal Knowledge-enhanced Sequence Modeling (MKSM). Specifically, we first develop a multimodal entity representation framework integrating knowledge graph(KG), text, and image features, which uses cross-modal alignment to solve semantic sparsity. Secondly, we adopt a position-aware self-attention mechanism to model entity mention order and capture temporal interest evolution. Then, we design an attention-weighted dynamic filtering strategy to select relevant historical segments based on the current dialogue context, which co-models short-term intents and long-term interests. The experimental results show that MKSM attains remarkable enhancements in both recommendation accuracy and response generation quality.