Scholarlink

<p>Multimodal Large Language Models (MLLMs) has recently demonstrated superior multimodal comprehension abilities, heralding a new era for artificial general intelligence (AGI). However, achieving AGI necessitates more than just comprehension. A crucial capability required is effective planning in diverse scenarios, which involves making reasonable decisions based on complex environments to solve real-world problems. Despite its importance, the planning abilities of current MLLMs in varied scenarios remain underexplored, leaving a significant gap in our understanding of their full potential. In this paper, we introduce EgoPlan-Bench2, a rigorous and comprehensive benchmark designed to <i>assess the planning capabilities of MLLMs across a wide range of real-world scenarios</i>. EgoPlan-Bench2 encompasses everyday tasks spanning 4 major domains and 24 detailed scenarios, closely aligned with human daily life. It is constructed through a semi-automatic process utilizing egocentric videos, complemented by manual verification. Grounded in a first-person perspective, it mirrors the way humans approach problem-solving in everyday life. We evaluate 25 competitive MLLMs and provide an in-depth analysis of their limitations, revealing that they face significant challenges in real-world planning. To diagnose the underlying bottlenecks, we investigate the effectiveness of various prompts via a training-free multimodal prompting method. We find that MLLMs’ planning performance on EgoPlan-Bench2 is critically dependent on temporally structured action sequences in historical task progress and interactions between objects and humans in current observation state. This dependency also underscores the necessity for strong reasoning abilities to integrate diverse multimodal cues and analysis before making final decision. Building on this insight, we demonstrate that EgoPlan-Bench2 is also an effective video reasoning benchmark. Experiments with Gemini-2.5-Flash and a post-trained Qwen-2.5-VL confirm its ability to distinguish between models with and without explicit deliberate reasoning mechanisms, showcasing the tangible impact of DeepSeek-R1 paradigm reasoning on planning tasks. We have made data and code available at <a href="https://qiulu66.github.io/egoplanbench2/">https://qiulu66.github.io/egoplanbench2/</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月8日

查看全文
  • Lu Qiu,
  • Yi Chen,
  • Yuying Ge,
  • Yixiao Ge,
  • Ying Shan,
  • Xihui Liu

摘要

Multimodal Large Language Models (MLLMs) has recently demonstrated superior multimodal comprehension abilities, heralding a new era for artificial general intelligence (AGI). However, achieving AGI necessitates more than just comprehension. A crucial capability required is effective planning in diverse scenarios, which involves making reasonable decisions based on complex environments to solve real-world problems. Despite its importance, the planning abilities of current MLLMs in varied scenarios remain underexplored, leaving a significant gap in our understanding of their full potential. In this paper, we introduce EgoPlan-Bench2, a rigorous and comprehensive benchmark designed to assess the planning capabilities of MLLMs across a wide range of real-world scenarios. EgoPlan-Bench2 encompasses everyday tasks spanning 4 major domains and 24 detailed scenarios, closely aligned with human daily life. It is constructed through a semi-automatic process utilizing egocentric videos, complemented by manual verification. Grounded in a first-person perspective, it mirrors the way humans approach problem-solving in everyday life. We evaluate 25 competitive MLLMs and provide an in-depth analysis of their limitations, revealing that they face significant challenges in real-world planning. To diagnose the underlying bottlenecks, we investigate the effectiveness of various prompts via a training-free multimodal prompting method. We find that MLLMs’ planning performance on EgoPlan-Bench2 is critically dependent on temporally structured action sequences in historical task progress and interactions between objects and humans in current observation state. This dependency also underscores the necessity for strong reasoning abilities to integrate diverse multimodal cues and analysis before making final decision. Building on this insight, we demonstrate that EgoPlan-Bench2 is also an effective video reasoning benchmark. Experiments with Gemini-2.5-Flash and a post-trained Qwen-2.5-VL confirm its ability to distinguish between models with and without explicit deliberate reasoning mechanisms, showcasing the tangible impact of DeepSeek-R1 paradigm reasoning on planning tasks. We have made data and code available at https://qiulu66.github.io/egoplanbench2/.