Scholarlink

<p>This work presents a comparative experimental study of selected transformer-based language model architectures: GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), and BLOOM (BigScience Large Open-science Open-access Multilingual Language Model). The evaluation spans execution time, accuracy, computational efficiency, cost-effectiveness, and task-specific performance across six key Natural Language Processing (NLP) benchmarks: Stanford Question Answering Dataset (SQuAD), General Language Understanding Evaluation (GLUE), Super General Language Understanding Evaluation (SuperGLUE), Cross-lingual Natural Language Inference (XNLI), AG News Text Classification Dataset (AG News), and the Massive Multitask Language Understanding benchmark (MMLU). BERT-Large demonstrates strong performance in classification-oriented tasks, achieving 94.1% accuracy on AG News and an 89.2% F1 score on SQuAD. GPT-3 excels in high-quality text generation but carries the highest training cost ($4.6M) and inference cost ($0.060 per 1K tokens). BLOOM-7B provides relatively stronger multilingual capabilities with a 91.2% accuracy on XNLI while maintaining moderate computational requirements. GPT-2 offers the most cost-efficient option for resource-constrained applications with a training cost of only $40K. Model selection should therefore be guided by the intended task requirements, computational budget, and deployment constraints. This analysis provides experimentally grounded insights under a controlled yet task-aligned evaluation setting for researchers, practitioners, and organizations aiming to determine the most suitable LLM architecture for their specific use cases,while acknowledging practical differences in evaluation methodology across model types rather than a fully uniform benchmarking pipeline.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A comprehensive comparative analysis of transformer based large language models across natural language processing benchmarks

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月8日

查看全文
  • Sudhindra B. Deshpande,
  • Priyanka Desai,
  • Kah Ong Michael Goh,
  • Pratijnya Ajawan,
  • Kadayya Mathad,
  • Uttam U. Deshpande

摘要

This work presents a comparative experimental study of selected transformer-based language model architectures: GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), and BLOOM (BigScience Large Open-science Open-access Multilingual Language Model). The evaluation spans execution time, accuracy, computational efficiency, cost-effectiveness, and task-specific performance across six key Natural Language Processing (NLP) benchmarks: Stanford Question Answering Dataset (SQuAD), General Language Understanding Evaluation (GLUE), Super General Language Understanding Evaluation (SuperGLUE), Cross-lingual Natural Language Inference (XNLI), AG News Text Classification Dataset (AG News), and the Massive Multitask Language Understanding benchmark (MMLU). BERT-Large demonstrates strong performance in classification-oriented tasks, achieving 94.1% accuracy on AG News and an 89.2% F1 score on SQuAD. GPT-3 excels in high-quality text generation but carries the highest training cost ($4.6M) and inference cost ($0.060 per 1K tokens). BLOOM-7B provides relatively stronger multilingual capabilities with a 91.2% accuracy on XNLI while maintaining moderate computational requirements. GPT-2 offers the most cost-efficient option for resource-constrained applications with a training cost of only $40K. Model selection should therefore be guided by the intended task requirements, computational budget, and deployment constraints. This analysis provides experimentally grounded insights under a controlled yet task-aligned evaluation setting for researchers, practitioners, and organizations aiming to determine the most suitable LLM architecture for their specific use cases,while acknowledging practical differences in evaluation methodology across model types rather than a fully uniform benchmarking pipeline.