Scholarlink

<p>Simultaneous localization and mapping (SLAM) is a pivotal challenge in mobile robotics. Traditional SLAM solutions primarily focus on achieving rapid and accurate localization and mapping while typically neglecting environmental object identification. This paper introduces an innovative SLAM system enhanced with YOLO-based open-vocabulary object detection. It leverages visual-language alignment to identify both known and novel objects using extensive image-text pairs. Our approach employs YOLOv8 as a teacher model, balancing speed and accuracy for object detection and bounding box prediction. These predictions are processed via CLIP encoders to generate high-dimensional vectors, teaching a student model robust image and text embeddings. Novel loss functions align augmented embeddings with supervisory signals, greatly enhancing detection accuracy and generalization. Additionally, the system integrates depth map-based scale extraction, 3D mapping of target object positions, and efficient relative pose estimation for loop detection. The direct method used improves accuracy and robustness, especially in poorly textured environments. Extensive ablation studies show significant improvements in precision and recall metrics. Our advanced SLAM system not only ensures accurate localization and mapping but also enables mobile robots to recognize and interact with a wide variety of objects, making it ideal for practical applications in complex environments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Open environments-aware SLAM based on YOLO-enhanced open-vocabulary object detection

  • 期刊论文

  • 发表日期：

    2025年10月11日

查看全文
  • Chengqun Song,
  • Fuxiang Wu,
  • Xiangyang Gao,
  • Jun Cheng,
  • Mengjie Yang,
  • Qiao Liu,
  • Lei Wang

摘要

Simultaneous localization and mapping (SLAM) is a pivotal challenge in mobile robotics. Traditional SLAM solutions primarily focus on achieving rapid and accurate localization and mapping while typically neglecting environmental object identification. This paper introduces an innovative SLAM system enhanced with YOLO-based open-vocabulary object detection. It leverages visual-language alignment to identify both known and novel objects using extensive image-text pairs. Our approach employs YOLOv8 as a teacher model, balancing speed and accuracy for object detection and bounding box prediction. These predictions are processed via CLIP encoders to generate high-dimensional vectors, teaching a student model robust image and text embeddings. Novel loss functions align augmented embeddings with supervisory signals, greatly enhancing detection accuracy and generalization. Additionally, the system integrates depth map-based scale extraction, 3D mapping of target object positions, and efficient relative pose estimation for loop detection. The direct method used improves accuracy and robustness, especially in poorly textured environments. Extensive ablation studies show significant improvements in precision and recall metrics. Our advanced SLAM system not only ensures accurate localization and mapping but also enables mobile robots to recognize and interact with a wide variety of objects, making it ideal for practical applications in complex environments.