Scholarlink

<p>Today, 83% of the world’s population owns a smartphone. The use of smartphones as personal assistants for people who are visually impaired (VI) is also emerging. One of the objectives of such assistive technology is to manage day-to-day activities and provide an independent shopping experience using visual question answering (VQA). This article proposes a new video dataset suitable for such tasks, namely the smartphone product video dataset (SFPVD). The dataset contains handheld product videos captured by VI individuals using a smartphone. With the ultimate goal of improving assistive technology for the VI, the dataset is designed to facilitate question-answering based on both textual and visual features available in the videos. We propose a pipeline that combines pre-processing and various information extraction techniques using pre-trained models, leading to an approximate 17% improvement in VQA accuracy compared to the best-performing single pre-trained model. We believe the dataset presents new challenges and will attract many computer vision researchers. This dataset will be made available at <a href="https://github.com/Ratnabali-Pal/SFPVD_VI">https://github.com/Ratnabali-Pal/SFPVD_VI</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Improving visual question answering for visually impaired people leveraging pre-trained models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月29日

查看全文
  • Ratnabali Pal,
  • Samarjit Kar,
  • Dilip K. Prasad,
  • Arif Ahmed Sekh

摘要

Today, 83% of the world’s population owns a smartphone. The use of smartphones as personal assistants for people who are visually impaired (VI) is also emerging. One of the objectives of such assistive technology is to manage day-to-day activities and provide an independent shopping experience using visual question answering (VQA). This article proposes a new video dataset suitable for such tasks, namely the smartphone product video dataset (SFPVD). The dataset contains handheld product videos captured by VI individuals using a smartphone. With the ultimate goal of improving assistive technology for the VI, the dataset is designed to facilitate question-answering based on both textual and visual features available in the videos. We propose a pipeline that combines pre-processing and various information extraction techniques using pre-trained models, leading to an approximate 17% improvement in VQA accuracy compared to the best-performing single pre-trained model. We believe the dataset presents new challenges and will attract many computer vision researchers. This dataset will be made available at https://github.com/Ratnabali-Pal/SFPVD_VI.