Scholarlink

<p>The rapid growth of multimedia technologies has led to massive amounts of video data, thus necessitating effective and fast Video Summarization (VS) methods. Although recent VS methods have shown significant progress, many existing approaches based on frame-wise analysis have performed limited semantic alignment, thus resulting in redundant keyframe selection. To address this limitation, this paper proposes a novel user-centric, query-based video summarization framework. Initially, input video and textual queries are collected and pre-processed. Then, deep visual features extracted from segmented video frames are fused with sentence-transformer-based textual features using Hadamard element-wise multiplication and Multimodal Compact Bilinear (MCB) pooling to compute relevance-aware keyframe rankings and eliminate redundancy. Pre-trained EfficientNet and Residual Network (ResNet18) models are employed to generate customized summaries. Experimental results on the MMSum dataset show that the proposed EfficientNet_b0_MiniLM model achieves 99.41% accuracy and a 98.13% F1-score, yielding over 27% improvement in F1-score compared to ResNet34_BOW baselines. These results demonstrate the effectiveness and robustness of the proposed approach for query-controllable video summarization.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Query-centric video summarization model based on sentence transformers and pre-trained efficient net

  • 期刊论文

  • 发表日期：

    2026年5月13日

查看全文
  • Rakhi D. Akhare,
  • Subhash K. Shinde

摘要

The rapid growth of multimedia technologies has led to massive amounts of video data, thus necessitating effective and fast Video Summarization (VS) methods. Although recent VS methods have shown significant progress, many existing approaches based on frame-wise analysis have performed limited semantic alignment, thus resulting in redundant keyframe selection. To address this limitation, this paper proposes a novel user-centric, query-based video summarization framework. Initially, input video and textual queries are collected and pre-processed. Then, deep visual features extracted from segmented video frames are fused with sentence-transformer-based textual features using Hadamard element-wise multiplication and Multimodal Compact Bilinear (MCB) pooling to compute relevance-aware keyframe rankings and eliminate redundancy. Pre-trained EfficientNet and Residual Network (ResNet18) models are employed to generate customized summaries. Experimental results on the MMSum dataset show that the proposed EfficientNet_b0_MiniLM model achieves 99.41% accuracy and a 98.13% F1-score, yielding over 27% improvement in F1-score compared to ResNet34_BOW baselines. These results demonstrate the effectiveness and robustness of the proposed approach for query-controllable video summarization.