Scholarlink

The fusion of computer vision (CV) and natural language processing (NLP) has revolutionized artificial intelligence. In 2022, over 3.37 billion internet users engaged with video content, accentuating its significance. However, video-question-answering (Video-QA) remains a formidable challenge due to the complexity of video data and the necessity to comprehend visual and textual modalities simultaneously. Our research examines how large language models (LLMs) like T5 or Gemini can boost video-QA accuracy by processing questions and subtitles to generate answers. This research aims to leverage contextual understanding from these models to extract relevant information from textual cues. Additionally, multi-modal fusion techniques have been explored, combining visual and textual features to enhance answer accuracy in Video-QA tasks further. Our results demonstrate that LLMs outperform traditional approaches when provided with question and subtitle inputs, indicating the effectiveness of contextualized language representations in capturing nuanced relationships between textual cues. Interestingly, our findings highlight the significance of textual information in video-QA datasets, as our approach, relying solely on subtitles, surpassed previous state-of-the-art visual models. This challenges the traditional emphasis on visual data, showcasing the potential of leveraging textual cues in improving Video-QA performance. In pursuit of improving accuracy for Video-QA, our approach achieved 89.75% and 91.37% accuracy on validation data for both PORORO-QA and TVQA datasets, respectively.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leveraging LLM’s Mastery in Video Question Answering

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • R. T. Rakshitha,
  • Padmashree Desai,
  • C. Sujatha,
  • Shankar Gangisetty

摘要

The fusion of computer vision (CV) and natural language processing (NLP) has revolutionized artificial intelligence. In 2022, over 3.37 billion internet users engaged with video content, accentuating its significance. However, video-question-answering (Video-QA) remains a formidable challenge due to the complexity of video data and the necessity to comprehend visual and textual modalities simultaneously. Our research examines how large language models (LLMs) like T5 or Gemini can boost video-QA accuracy by processing questions and subtitles to generate answers. This research aims to leverage contextual understanding from these models to extract relevant information from textual cues. Additionally, multi-modal fusion techniques have been explored, combining visual and textual features to enhance answer accuracy in Video-QA tasks further. Our results demonstrate that LLMs outperform traditional approaches when provided with question and subtitle inputs, indicating the effectiveness of contextualized language representations in capturing nuanced relationships between textual cues. Interestingly, our findings highlight the significance of textual information in video-QA datasets, as our approach, relying solely on subtitles, surpassed previous state-of-the-art visual models. This challenges the traditional emphasis on visual data, showcasing the potential of leveraging textual cues in improving Video-QA performance. In pursuit of improving accuracy for Video-QA, our approach achieved 89.75% and 91.37% accuracy on validation data for both PORORO-QA and TVQA datasets, respectively.