Scholarlink

As part of our ongoing development of ADAPT (Adaptive Dynamics and Active Perception for Thought), a cognitive architecture for robotics [5], we present a framework for constructing a real-time virtual model of the physical world. The goal is to enable robots to perceive and simulate dynamic environments for planning and reasoning. We integrate a ZED 2i stereo camera with Unreal Engine 5 [9] using spatial mapping, Hierarchical Level of Detail (HLOD) mesh rendering, and a custom Blueprint-driven update loop. The system supports runtime 3D reconstruction and pose synchronization using streamed (Red Green Blue - Depth) RGB-D data. Initial results show live spatial mapping at 45–55 FPS (Frames Per Second) with frame times ranging between 17 and 25 milliseconds, under standard indoor conditions. Granularity is a tunable parameter, with mesh resolution set at 4 cm voxel size. While the system supports granularity control, trade-off evaluations are planned for future work. This platform supports the construction of a continuously updating mental model in ADAPT, forming the basis for simulation-driven, autonomous behavior in complex environments.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Realtime Rendering of Physical Environment to the Robot Mental Model Using Unreal Engine 5 (Phase I)

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Milind Kumar Choudhary,
  • Gunjan Asrani,
  • David P. Benjamin

摘要

As part of our ongoing development of ADAPT (Adaptive Dynamics and Active Perception for Thought), a cognitive architecture for robotics [5], we present a framework for constructing a real-time virtual model of the physical world. The goal is to enable robots to perceive and simulate dynamic environments for planning and reasoning. We integrate a ZED 2i stereo camera with Unreal Engine 5 [9] using spatial mapping, Hierarchical Level of Detail (HLOD) mesh rendering, and a custom Blueprint-driven update loop. The system supports runtime 3D reconstruction and pose synchronization using streamed (Red Green Blue - Depth) RGB-D data. Initial results show live spatial mapping at 45–55 FPS (Frames Per Second) with frame times ranging between 17 and 25 milliseconds, under standard indoor conditions. Granularity is a tunable parameter, with mesh resolution set at 4 cm voxel size. While the system supports granularity control, trade-off evaluations are planned for future work. This platform supports the construction of a continuously updating mental model in ADAPT, forming the basis for simulation-driven, autonomous behavior in complex environments.