<p>Individuals with visual impairments continue to face challenges in performing everyday tasks, despite ongoing advances in assistive technologies. In the context of the Cybathlon 2024 Vision Assistance Race, we developed and evaluated a smart vision assistance system combining lightweight wearable hardware, multimodal sensing, and computer vision algorithms. This article presents the technical development process of the system, including hardware design, sensor integration, feedback mechanisms, real-time object detection and text recognition pipelines, and task-oriented interaction strategies adapted to the competition tasks. The system achieved reliable performance across most race tasks, although consistent results for the Forest task remained challenging. To investigate usability beyond the competition environment, we conducted a user study using simplified versions of selected race tasks. The study evaluated task completion time and success rate to assess how effectively the system could be used with minimal training. Results showed an average success rate of 93.46%, with reductions in task completion times for the evaluated tasks: Doorbell, Empty Seats, Grocery, Colours, and Touchscreen. This work serves as a technical report on the design, implementation, and evaluation of a Cybathon vision assistance system, providing insights into the integration of wearable hardware, multimodal feedback, and machine learning methods for assistive applications.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A smart vision assistance system designed for the cybathlon 2024 vision assistance race to support individuals with visual impairments

  • Simon Senoner,
  • Simon Winkler,
  • Lukas Neururer,
  • Leon Ochtendung,
  • Julia Jackermaier,
  • Marcel Naderer,
  • Bernhard Tschulnigg,
  • Bernhard Hollaus,
  • Yeongmi Kim

摘要

Individuals with visual impairments continue to face challenges in performing everyday tasks, despite ongoing advances in assistive technologies. In the context of the Cybathlon 2024 Vision Assistance Race, we developed and evaluated a smart vision assistance system combining lightweight wearable hardware, multimodal sensing, and computer vision algorithms. This article presents the technical development process of the system, including hardware design, sensor integration, feedback mechanisms, real-time object detection and text recognition pipelines, and task-oriented interaction strategies adapted to the competition tasks. The system achieved reliable performance across most race tasks, although consistent results for the Forest task remained challenging. To investigate usability beyond the competition environment, we conducted a user study using simplified versions of selected race tasks. The study evaluated task completion time and success rate to assess how effectively the system could be used with minimal training. Results showed an average success rate of 93.46%, with reductions in task completion times for the evaluated tasks: Doorbell, Empty Seats, Grocery, Colours, and Touchscreen. This work serves as a technical report on the design, implementation, and evaluation of a Cybathon vision assistance system, providing insights into the integration of wearable hardware, multimodal feedback, and machine learning methods for assistive applications.