This chapter surveys the conceptual and technical scaffolding of modern large language models (LLMs) and their impact on language services. It first problematizes “computable linguistic units,” showing how segmentation and tokenization (from words to subwords and bytes) operationalize theory while trading off sparsity, semantic integrity, and engineering efficiency. It then contrasts dynamic sequence modeling with static representation learning, tracing evolution from n‑grams to neural and Transformer architectures and explaining the rise of autoregressive (decoder–only) scaling over masked objectives for unified generation and emergent in-context learning. The pretraining–fine–tuning paradigm has shifted toward prompt–driven interaction, supplemented by continued pretraining, supervised fine‑tuning, knowledge distillation, retrieval–augmented generation, and agent frameworks that orchestrate multistep tool use. Core service-oriented capabilities now hinge on efficient adaptation, controllability, and integration with external knowledge. Technical and socio–ethical limitations—hallucination, bias, responsibility attribution, interpretability, and control—are analyzed as structural consequences of probabilistic next‑token prediction and data provenance rather than isolated defects. The chapter frames LLMs as transformative yet fallible linguistic infrastructures, emphasizing the emerging professional skill set: critical evaluation, prompt and data curation, bias mitigation, and system orchestration for trustworthy language services.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Language Modeling and Large Language Models

  • Jingsong Shawn Yu,
  • Yazhi Yao

摘要

This chapter surveys the conceptual and technical scaffolding of modern large language models (LLMs) and their impact on language services. It first problematizes “computable linguistic units,” showing how segmentation and tokenization (from words to subwords and bytes) operationalize theory while trading off sparsity, semantic integrity, and engineering efficiency. It then contrasts dynamic sequence modeling with static representation learning, tracing evolution from n‑grams to neural and Transformer architectures and explaining the rise of autoregressive (decoder–only) scaling over masked objectives for unified generation and emergent in-context learning. The pretraining–fine–tuning paradigm has shifted toward prompt–driven interaction, supplemented by continued pretraining, supervised fine‑tuning, knowledge distillation, retrieval–augmented generation, and agent frameworks that orchestrate multistep tool use. Core service-oriented capabilities now hinge on efficient adaptation, controllability, and integration with external knowledge. Technical and socio–ethical limitations—hallucination, bias, responsibility attribution, interpretability, and control—are analyzed as structural consequences of probabilistic next‑token prediction and data provenance rather than isolated defects. The chapter frames LLMs as transformative yet fallible linguistic infrastructures, emphasizing the emerging professional skill set: critical evaluation, prompt and data curation, bias mitigation, and system orchestration for trustworthy language services.