Scholarlink

<p>The transformer architecture with self attention mechanism has fundamentally reshaped machine learning by replacing sequential recurrence with parallelisable self-attention, enabling scale-driven performance improvements across diverse task domains. This survey traces the evolution of transformer architectures from their origins in machine translation through their current deployment in Large Language Models (LLMs), computer vision, speech recognition, healthcare, finance, and recommendation systems. We organise this review around three principal domains: Natural Language Processing and Generative Modelling, Pattern Recognition, and Expert and Scientific Applications. Within each domain, we discuss architectural innovations including encoder-only pre-training, decoder-only scaling and instruction alignment, encoder-decoder unification, vision transformers, and efficient architectures and provide benchmark data on standardised evaluation suites. To ground the mathematical exposition, we present a condensed, step-by-step numerical illustration of the encoder-decoder mechanism using simplified hyperparameters. We also examine persistent challenges including quadratic attention complexity, context length scaling, alignment, interpretability, bias, and privacy, and identify five concrete open research directions. This survey is distinguished from prior transformer surveys by its simultaneous coverage of efficient architecture variants (sparse attention, state space models, Mixture-of-Experts (MoE)), its cross-domain scope including recommendation systems and genomics, and its numerical illustration. The survey includes architectures published through early 2026. It recognizes rapidly changing areas, such as test-time compute scaling and diffusion-based language models, but does not cover them in detail. Companion code for the numerical walkthrough is available at <a href="https://github.com/karan3819/Transformer_Translation_Visualization">https://github.com/karan3819/Transformer_Translation_Visualization</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Transformer Models and their Cross-Domain Applications: A Systematic Review

  • 期刊论文

  • 发表日期：

    2026年7月27日

查看全文
  • Karan Pathania,
  • Sukhdeep Singh,
  • Anuj Sharma

摘要

The transformer architecture with self attention mechanism has fundamentally reshaped machine learning by replacing sequential recurrence with parallelisable self-attention, enabling scale-driven performance improvements across diverse task domains. This survey traces the evolution of transformer architectures from their origins in machine translation through their current deployment in Large Language Models (LLMs), computer vision, speech recognition, healthcare, finance, and recommendation systems. We organise this review around three principal domains: Natural Language Processing and Generative Modelling, Pattern Recognition, and Expert and Scientific Applications. Within each domain, we discuss architectural innovations including encoder-only pre-training, decoder-only scaling and instruction alignment, encoder-decoder unification, vision transformers, and efficient architectures and provide benchmark data on standardised evaluation suites. To ground the mathematical exposition, we present a condensed, step-by-step numerical illustration of the encoder-decoder mechanism using simplified hyperparameters. We also examine persistent challenges including quadratic attention complexity, context length scaling, alignment, interpretability, bias, and privacy, and identify five concrete open research directions. This survey is distinguished from prior transformer surveys by its simultaneous coverage of efficient architecture variants (sparse attention, state space models, Mixture-of-Experts (MoE)), its cross-domain scope including recommendation systems and genomics, and its numerical illustration. The survey includes architectures published through early 2026. It recognizes rapidly changing areas, such as test-time compute scaling and diffusion-based language models, but does not cover them in detail. Companion code for the numerical walkthrough is available at https://github.com/karan3819/Transformer_Translation_Visualization.