Scholarlink

Despite their widespread adoption, contemporary voice assistants remain limited in their ability to handle complex, multi-step interactions, primarily due to their reliance on inflexible, predefined intent-based systems. We introduce Large Action Models (LAMs), an architecture enabling autonomous task execution from natural language through integrated reasoning, perception, and action. Our modular design combines speech-to-text (Whisper), multi-step planning (xLAM), device control (Mobile-MCP), visual feedback (CogVLM), and safety mechanisms to convert free-form speech into contextually-aware device actions. LAMs achieve flexible task automation through continuous perception-driven replanning, adapting to changing conditions and maintaining cross-application context. This work establishes the foundation for LAMs, proposes a novel modular architecture, and explores how perception-action loops enable task automation beyond current assistant limitations.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Toward Human-Level Task Automation with Large Action Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Keane Scouton Crasto,
  • A. Priyadharshini

摘要

Despite their widespread adoption, contemporary voice assistants remain limited in their ability to handle complex, multi-step interactions, primarily due to their reliance on inflexible, predefined intent-based systems. We introduce Large Action Models (LAMs), an architecture enabling autonomous task execution from natural language through integrated reasoning, perception, and action. Our modular design combines speech-to-text (Whisper), multi-step planning (xLAM), device control (Mobile-MCP), visual feedback (CogVLM), and safety mechanisms to convert free-form speech into contextually-aware device actions. LAMs achieve flexible task automation through continuous perception-driven replanning, adapting to changing conditions and maintaining cross-application context. This work establishes the foundation for LAMs, proposes a novel modular architecture, and explores how perception-action loops enable task automation beyond current assistant limitations.