Scholarlink

<p>The 3D visual grounding aims to match textual instructions with target objects in a 3D scene. Existing approaches typically leverage high-performance point cloud and text encoders to extract features and involve feature interactions to grounding target objects. However, these methods often capture limited spatial information, and the target object is often misidentified when parsing instructions, both of which pose challenges to the accuracy of object recognition. To mitigate the above issues, we propose a novel framework to enhance spatial-semantic coherence via <b>S</b>patial-<b>A</b>ware encoder and <b>T</b>arget <b>R</b>efinement (SATRefer). Specifically, we first design a spatial-aware object encoder that employs self-attention to aggregate features from surrounding objects and utilizes positional encoding to represent spatial relationships, thereby enriching both object features and spatial information. Secondly, to alleviate target object recognition errors, we propose a target refinement strategy based on a large language model (LLM), which replaces the target object in the instruction for verification, ensuring greater accuracy. Finally, to effectively integrate multi-modal information, we introduce a multi-modal feature fusion module to improve object grounding accuracy by fully utilizing the rich texture information in the image, along with position and geometric cues in the point cloud, as well as semantic knowledge of the instructions. Extensive experiments show that our method outperforms previous methods on multiple datasets, especially achieving competitive performance on spatially related metrics.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing spatial-semantic coherence of 3D visual grounding via spatial-aware encoder and target refinement

  • 期刊论文

  • 发表日期：

    2026年3月21日

查看全文
  • Shucheng Wan,
  • Mingwen Shao,
  • Lingzhuang Meng,
  • Jie Zhang

摘要

The 3D visual grounding aims to match textual instructions with target objects in a 3D scene. Existing approaches typically leverage high-performance point cloud and text encoders to extract features and involve feature interactions to grounding target objects. However, these methods often capture limited spatial information, and the target object is often misidentified when parsing instructions, both of which pose challenges to the accuracy of object recognition. To mitigate the above issues, we propose a novel framework to enhance spatial-semantic coherence via Spatial-Aware encoder and Target Refinement (SATRefer). Specifically, we first design a spatial-aware object encoder that employs self-attention to aggregate features from surrounding objects and utilizes positional encoding to represent spatial relationships, thereby enriching both object features and spatial information. Secondly, to alleviate target object recognition errors, we propose a target refinement strategy based on a large language model (LLM), which replaces the target object in the instruction for verification, ensuring greater accuracy. Finally, to effectively integrate multi-modal information, we introduce a multi-modal feature fusion module to improve object grounding accuracy by fully utilizing the rich texture information in the image, along with position and geometric cues in the point cloud, as well as semantic knowledge of the instructions. Extensive experiments show that our method outperforms previous methods on multiple datasets, especially achieving competitive performance on spatially related metrics.