Scholarlink

A large-scale language model generally refers to models based on Transformer architecture, such as BERT, GPT-X (X \(=\) 1, 2, 3, 3.5, 4), BART, T5, and Llama. Large-scale language models are fundamentally based on pre-training through self-supervised learning and fine-tuning for specific tasks. The aim of pre-training is to acquire general knowledge about language, and compared to pre-training, fine-tuning requires less data.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Large-Scale Language Models

  • 章节

  • 发表日期：

    2025年

查看全文
  • Takeshi Okadome

摘要

A large-scale language model generally refers to models based on Transformer architecture, such as BERT, GPT-X (X \(=\) 1, 2, 3, 3.5, 4), BART, T5, and Llama. Large-scale language models are fundamentally based on pre-training through self-supervised learning and fine-tuning for specific tasks. The aim of pre-training is to acquire general knowledge about language, and compared to pre-training, fine-tuning requires less data.