<p>This study improves tag recommendation for marketer-generated contents (MGCs) by integrating multimodal information from both textual descriptions and visual information. To address this issue, we develop a behavior-driven multimodal neural network that jointly captures content semantics and marketer-specific tagging preferences. Specifically, we first introduce a text-guided attention mechanism to model the interaction between the textual contents and images in MGCs. Second, we incorporate internal attention modules to identify the impacts of specific words, image regions, and individual images on tag prediction. Third, we design a time-decaying attention mechanism to account for marketers’ historical tagging behaviors, thereby capturing the temporal dynamics and heterogeneity in their preferences. We evaluate our model using a large-scale real-world dataset collected from Taobao. Quantitative results demonstrate that our approach significantly outperforms state-of-the-art baselines in tag recommendation. Qualitative analyses further reveal interpretable insights into how marketers’ historical behaviors and multimodal cues contribute to their tagging decisions.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic tagging services for marketer-generated contents: a multimodal neural network

  • Yang Qian,
  • Sipeng Wang,
  • Wang Xu,
  • Yuanchun Jiang,
  • Yidong Chai,
  • Haifeng Ling

摘要

This study improves tag recommendation for marketer-generated contents (MGCs) by integrating multimodal information from both textual descriptions and visual information. To address this issue, we develop a behavior-driven multimodal neural network that jointly captures content semantics and marketer-specific tagging preferences. Specifically, we first introduce a text-guided attention mechanism to model the interaction between the textual contents and images in MGCs. Second, we incorporate internal attention modules to identify the impacts of specific words, image regions, and individual images on tag prediction. Third, we design a time-decaying attention mechanism to account for marketers’ historical tagging behaviors, thereby capturing the temporal dynamics and heterogeneity in their preferences. We evaluate our model using a large-scale real-world dataset collected from Taobao. Quantitative results demonstrate that our approach significantly outperforms state-of-the-art baselines in tag recommendation. Qualitative analyses further reveal interpretable insights into how marketers’ historical behaviors and multimodal cues contribute to their tagging decisions.