Scholarlink

<p>Transformers have become the dominant architectural paradigm in modern artificial intelligence. Originally introduced for sequence modeling in natural language processing, the transformer architecture has since been extended to different input modalities through innovations in tokenization, attention mechanisms, and training objectives. This article examines the evolution of transformer models across input modalities by analyzing these models along five dimensions: (1) architectural designs, (2) training paradigms, (3) computational requirements, (4) performance benchmarks, and (5) evolutionary trends. Through this comparative analysis, we find that recent developments in transformer models are driven by shared system-level constraints rather than modality-specific design choices. We observe a convergence toward unified systems in which language models serve as the central reasoning and coordination backbone across modalities. These insights also highlight challenges related to deliberative reasoning, grounding, alignment, and persistent memory, which cannot be addressed through scaling alone. Addressing these limitations points to future research directions aimed at developing systems that function as interactive, multimodal agents capable of deeper reasoning, cross-modal grounding, and adaptive alignment beyond static prediction.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A survey of transformers based on their input modalities

  • 期刊论文

  • 发表日期：

    2026年7月11日

查看全文
  • Priyanka Sharma,
  • Sanjay Kumar Jain

摘要

Transformers have become the dominant architectural paradigm in modern artificial intelligence. Originally introduced for sequence modeling in natural language processing, the transformer architecture has since been extended to different input modalities through innovations in tokenization, attention mechanisms, and training objectives. This article examines the evolution of transformer models across input modalities by analyzing these models along five dimensions: (1) architectural designs, (2) training paradigms, (3) computational requirements, (4) performance benchmarks, and (5) evolutionary trends. Through this comparative analysis, we find that recent developments in transformer models are driven by shared system-level constraints rather than modality-specific design choices. We observe a convergence toward unified systems in which language models serve as the central reasoning and coordination backbone across modalities. These insights also highlight challenges related to deliberative reasoning, grounding, alignment, and persistent memory, which cannot be addressed through scaling alone. Addressing these limitations points to future research directions aimed at developing systems that function as interactive, multimodal agents capable of deeper reasoning, cross-modal grounding, and adaptive alignment beyond static prediction.