Scholarlink

Abstract <p>Visual Semantic Navigation (VSN) is a fundamental problem in robotics, where an agent must navigate toward a target object in an unknown environment, mainly using visual information. Most state-of-the-art VSN models are trained in simulation environments, where rendered scenes of the real world are used, at best. These approaches typically rely on raw RGB data from the virtual scenes, which limits their ability to generalize to real-world environments due to domain adaptation issues. To tackle this problem, in this work, we propose <span>SemNav</span>, a novel approach that leverages semantic segmentation as the main visual input representation of the environment to enhance the agent’s perception and decision-making capabilities. By explicitly incorporating this type of high-level semantic information, our model learns robust navigation policies that improve generalization across unseen environments, both in simulated and real-world settings. We also introduce the <span>SemNav</span> dataset, a newly curated dataset designed for training semantic segmentation-aware navigation models like <span>SemNav</span>. Our approach is evaluated extensively in both simulated environments and with real-world robotic platforms. Experimental results demonstrate that <span>SemNav</span> outperforms existing state-of-the-art VSN models, achieving higher success rates in the Habitat 2.0 simulation environment, using the HM3D dataset. Furthermore, our real-world experiments highlight the effectiveness of semantic segmentation in mitigating the sim-to-real gap, making our model a promising solution for practical VSN-based robotic applications. The code and datasets are accessible at <a href="https://github.com/gramuah/semnav">https://github.com/gramuah/semnav</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SemNav: Enhancing visual semantic navigation in robotics through semantic segmentation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月19日

查看全文
  • Rafael Flor-Rodríguez-Rabadán,
  • Carlos Gutiérrez-Álvarez,
  • Francisco Javier Acevedo-Rodríguez,
  • Sergio Lafuente-Arroyo,
  • Roberto Javier López-Sastre

摘要

Abstract

Visual Semantic Navigation (VSN) is a fundamental problem in robotics, where an agent must navigate toward a target object in an unknown environment, mainly using visual information. Most state-of-the-art VSN models are trained in simulation environments, where rendered scenes of the real world are used, at best. These approaches typically rely on raw RGB data from the virtual scenes, which limits their ability to generalize to real-world environments due to domain adaptation issues. To tackle this problem, in this work, we propose SemNav, a novel approach that leverages semantic segmentation as the main visual input representation of the environment to enhance the agent’s perception and decision-making capabilities. By explicitly incorporating this type of high-level semantic information, our model learns robust navigation policies that improve generalization across unseen environments, both in simulated and real-world settings. We also introduce the SemNav dataset, a newly curated dataset designed for training semantic segmentation-aware navigation models like SemNav. Our approach is evaluated extensively in both simulated environments and with real-world robotic platforms. Experimental results demonstrate that SemNav outperforms existing state-of-the-art VSN models, achieving higher success rates in the Habitat 2.0 simulation environment, using the HM3D dataset. Furthermore, our real-world experiments highlight the effectiveness of semantic segmentation in mitigating the sim-to-real gap, making our model a promising solution for practical VSN-based robotic applications. The code and datasets are accessible at https://github.com/gramuah/semnav.