Scholarlink

<p>It is challenging for wearable obstacle avoidance devices to simultaneously meet practical demands of high reliability, rapid response, long-lasting duration, and usable design. Here we report a wearable obstacle avoidance device, comprising a set of self-developed glasses (weighing &#xa0;~400 grams, including an &#xa0;~80 grams battery) and a common smartphone. Specifically, the glasses collect the multi-modal data for comprehensive environmental perception, including video and depth modalities, and implement a depth-aided video compression module. This module not only adaptively compresses video data to reduce transmission delay to the smartphone, but also operates on a customized FPGA board featuring a multi float-point vector unit streaming processing architecture, thereby facilitating responsive and energy-efficient obstacle detection. Additionally, we design a cross-modal obstacle detection module on the smartphone, which ensures reliable detection and provides user-friendly auditory and tactile alerts by utilizing cross-modal learning based on modal correlations. Multiple indoor and outdoor experimental results demonstrate 100% collision avoidance rates, delay of less than 320 ms, and duration of approximately 11 hours.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A wearable obstacle avoidance device for visually impaired individuals with cross-modal learning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年3月24日

查看全文
  • Yun Gao,
  • Dan Wu,
  • Jie Song,
  • Xueyi Zhang,
  • Bangbang Hou,
  • Hengfa Liu,
  • Junqi Liao,
  • Liang Zhou

摘要

It is challenging for wearable obstacle avoidance devices to simultaneously meet practical demands of high reliability, rapid response, long-lasting duration, and usable design. Here we report a wearable obstacle avoidance device, comprising a set of self-developed glasses (weighing  ~400 grams, including an  ~80 grams battery) and a common smartphone. Specifically, the glasses collect the multi-modal data for comprehensive environmental perception, including video and depth modalities, and implement a depth-aided video compression module. This module not only adaptively compresses video data to reduce transmission delay to the smartphone, but also operates on a customized FPGA board featuring a multi float-point vector unit streaming processing architecture, thereby facilitating responsive and energy-efficient obstacle detection. Additionally, we design a cross-modal obstacle detection module on the smartphone, which ensures reliable detection and provides user-friendly auditory and tactile alerts by utilizing cross-modal learning based on modal correlations. Multiple indoor and outdoor experimental results demonstrate 100% collision avoidance rates, delay of less than 320 ms, and duration of approximately 11 hours.