<p>Prompt-based control is a powerful paradigm for directing computer vision systems to solve diverse and complex tasks. In single-object segmentation scenarios, a laser pointer can serve as an intuitive, precise navigation module for generating corresponding object masks. However, the computational demands of these systems have traditionally confined them to desktop workstations equipped with dedicated GPUs, limiting their portability and widespread applicability. To address this challenge, a real-time, laser-guided object segmentation system built on a mobile cloud architecture is introduced in this study. The system employs a smartphone as a portable client while offloading computationally intensive tasks to a cloud-based server. The proposed architecture integrates a custom laser pointer detection model based on You Only Look Once version 5n (YOLOv5n-LPD): a lightweight laser spot detector with 65k parameters that achieves a 2.5-ms inference time with an mAP@50 of 91% and an mAP@50:95 of 62%. For the segmentation task, the highly efficient and accurate fast segment-anything model (FastSAM) is selected on the basis of an experimental evaluation. When tested on distant cloud servers, the system operates with an end-to-end latency of approximately 120 ms, whereas the client application maintains a frame rate of 10–20 fps. This work demonstrates the viability of combining mobile devices with cloud-based controlled computer vision, offering new possibilities for everyday, industrial, and academic applications.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Laser-prompted real-time object segmentation on smartphones via cloud computing

  • Kyrylo Shliaiev,
  • Jinfu Yang

摘要

Prompt-based control is a powerful paradigm for directing computer vision systems to solve diverse and complex tasks. In single-object segmentation scenarios, a laser pointer can serve as an intuitive, precise navigation module for generating corresponding object masks. However, the computational demands of these systems have traditionally confined them to desktop workstations equipped with dedicated GPUs, limiting their portability and widespread applicability. To address this challenge, a real-time, laser-guided object segmentation system built on a mobile cloud architecture is introduced in this study. The system employs a smartphone as a portable client while offloading computationally intensive tasks to a cloud-based server. The proposed architecture integrates a custom laser pointer detection model based on You Only Look Once version 5n (YOLOv5n-LPD): a lightweight laser spot detector with 65k parameters that achieves a 2.5-ms inference time with an mAP@50 of 91% and an mAP@50:95 of 62%. For the segmentation task, the highly efficient and accurate fast segment-anything model (FastSAM) is selected on the basis of an experimental evaluation. When tested on distant cloud servers, the system operates with an end-to-end latency of approximately 120 ms, whereas the client application maintains a frame rate of 10–20 fps. This work demonstrates the viability of combining mobile devices with cloud-based controlled computer vision, offering new possibilities for everyday, industrial, and academic applications.