Recent breakthroughs in multimodal large language models (MLLMs) have driven remarkable progress in embodied intelligence, particularly for sequential decision-making and long-horizon planning tasks. Besides, reinforcement fine-tuning (RFT) approaches exemplified by DeepSeek-R1 have further facilitated self-iteration and reasoning optimization in MLLMs. However, embodied tasks present unique spatio-temporal challenges that distinguish them from conventional multimodal reasoning tasks, requiring simultaneous consideration of temporal sequence logic, spatial precision, and operational safety. Current RFT paradigms fail to adequately address these critical dimensions. To bridge this gap, we propose ST-Reasoner, a reasoning-decision-action unified framework for multimodal large language model fine-tuning. Our framework employs a two-stage fine-tuning approach for MLLMs in diverse tabletop manipulation tasks. In the first stage, we conduct supervised fine-tuning (SFT) to equip the model with preliminary reasoning and planning capabilities. Subsequently, we perform multi-objective optimization via GRPO using diverse rule-based reward functions, thereby achieving holistic improvement of the model’s capabilities. Experimental results across diverse desktop manipulation tasks demonstrate that ST-Reasoner outperforms existing state-of-the-art methods on LoHoRavens benchmark. This work establishes a new paradigm for developing spatially-aware and temporally-consistent embodied intelligence systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improving Spatio-Temporal Awareness of Multimodal Large Language Models via Reinforcement Fine-Tuning

  • Jingyi Zhang,
  • Hefeng Wu,
  • Liang Lin

摘要

Recent breakthroughs in multimodal large language models (MLLMs) have driven remarkable progress in embodied intelligence, particularly for sequential decision-making and long-horizon planning tasks. Besides, reinforcement fine-tuning (RFT) approaches exemplified by DeepSeek-R1 have further facilitated self-iteration and reasoning optimization in MLLMs. However, embodied tasks present unique spatio-temporal challenges that distinguish them from conventional multimodal reasoning tasks, requiring simultaneous consideration of temporal sequence logic, spatial precision, and operational safety. Current RFT paradigms fail to adequately address these critical dimensions. To bridge this gap, we propose ST-Reasoner, a reasoning-decision-action unified framework for multimodal large language model fine-tuning. Our framework employs a two-stage fine-tuning approach for MLLMs in diverse tabletop manipulation tasks. In the first stage, we conduct supervised fine-tuning (SFT) to equip the model with preliminary reasoning and planning capabilities. Subsequently, we perform multi-objective optimization via GRPO using diverse rule-based reward functions, thereby achieving holistic improvement of the model’s capabilities. Experimental results across diverse desktop manipulation tasks demonstrate that ST-Reasoner outperforms existing state-of-the-art methods on LoHoRavens benchmark. This work establishes a new paradigm for developing spatially-aware and temporally-consistent embodied intelligence systems.