To maximize click-through rates, e-commerce platforms need to display advertising videos tailored to user preferences by splicing relevant segments from raw videos and incorporating valid user descriptions. Video grounding, the most related task, can locate video segments given an exact textual description. However, in advertising video editing, user descriptions can be noisy and may contain unrelated content, necessitating to discard those invalid descriptions. In this paper, we propose a cross video-text grounding network based on the Transformer, which can identify pairs of matched video segments and textual descriptions despite potential noisy. Specifically, we utilize learnable query embeddings to interact with video and text, obtaining updated query embeddings, among which the valid query embeddings correspond to the selling points present in both the raw product video and user descriptions. These valid query embeddings are used to predict video segments and textual description masks, allowing us to filter out invalid descriptions and obtain matched video segments for the remaining valid descriptions. Furthermore, we contribute Selling Points, a large-scale dataset for advertising video editing, comprising 22,948 videos totaling 250 h and featuring 183k descriptions. The performance of our proposed method on our contributed dataset demonstrates its effectiveness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Cross Video-Text Grounding for Advertising Video Editing

  • Qingyang Liu

摘要

To maximize click-through rates, e-commerce platforms need to display advertising videos tailored to user preferences by splicing relevant segments from raw videos and incorporating valid user descriptions. Video grounding, the most related task, can locate video segments given an exact textual description. However, in advertising video editing, user descriptions can be noisy and may contain unrelated content, necessitating to discard those invalid descriptions. In this paper, we propose a cross video-text grounding network based on the Transformer, which can identify pairs of matched video segments and textual descriptions despite potential noisy. Specifically, we utilize learnable query embeddings to interact with video and text, obtaining updated query embeddings, among which the valid query embeddings correspond to the selling points present in both the raw product video and user descriptions. These valid query embeddings are used to predict video segments and textual description masks, allowing us to filter out invalid descriptions and obtain matched video segments for the remaining valid descriptions. Furthermore, we contribute Selling Points, a large-scale dataset for advertising video editing, comprising 22,948 videos totaling 250 h and featuring 183k descriptions. The performance of our proposed method on our contributed dataset demonstrates its effectiveness.