Scholarlink

Robot learning aims to complete diverse tasks. End-to-end VLA models, achieving significant performance, but struggling on data dependency. Recently, video generation models (VGMs) as a world model provides a new perspective, enabling robots to generalize across tasks by ‘imagining’ future states. However, computing bottleneck leading to limited-length video output, not applicable for long-term tasks. In this paper, we train a image-text conditioned robotic video generation model, named RoboImagine, aiming to generate long-term robotic manipulation videos, with visual-semantic-dynamic conformity. We build an autoregressive long-term video generation pipeline based on a VLM as task-complete-verifier, in which RoboImagine is designed with dynamic and geometric consistency augmentation to get continuous and smooth motions between generated clips. Systematic experiments are implemented, showing that we are able to generate longe-term robotic manipulation videos with continuous motion, achieveing average success rate increment of 150% than that of w/o augmentation method. Our method effectively generalize on unseen simulation and real world cases. The generated video is mapped into end-effector actions, through a visual inverse dynamic model. We open source our work with link: https://github.com/Egbert-Lannister/Robo-Imagine .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

RoboImagine: A Robotic Video Generation Model, for Autoregressive Long-Term Task Video Generation with Geometric and Dynamic Consistency Augmentation

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Conglin Wang,
  • Hongkun Yang,
  • Chuanjiang Li,
  • Siqi Wen,
  • Yiming Gan,
  • Shuai Liang

摘要

Robot learning aims to complete diverse tasks. End-to-end VLA models, achieving significant performance, but struggling on data dependency. Recently, video generation models (VGMs) as a world model provides a new perspective, enabling robots to generalize across tasks by ‘imagining’ future states. However, computing bottleneck leading to limited-length video output, not applicable for long-term tasks. In this paper, we train a image-text conditioned robotic video generation model, named RoboImagine, aiming to generate long-term robotic manipulation videos, with visual-semantic-dynamic conformity. We build an autoregressive long-term video generation pipeline based on a VLM as task-complete-verifier, in which RoboImagine is designed with dynamic and geometric consistency augmentation to get continuous and smooth motions between generated clips. Systematic experiments are implemented, showing that we are able to generate longe-term robotic manipulation videos with continuous motion, achieveing average success rate increment of 150% than that of w/o augmentation method. Our method effectively generalize on unseen simulation and real world cases. The generated video is mapped into end-effector actions, through a visual inverse dynamic model. We open source our work with link: https://github.com/Egbert-Lannister/Robo-Imagine .