In this chapter, we review the recent developments in Large Language Model (LLM) fine-tuning, a process that involves updating model parameters to enhance their performance. Fine-tuning is essential for adapting models to improve abilities such as instruction following, grounding, and generalization to new domains. The chapter examines four key fine-tuning approaches: Supervised Instruction Tuning, which enhances task generalization through supervised data; Continual Learning, which integrates new information while avoiding catastrophic forgetting; Parameter-Efficient Fine-Tuning, which allows efficient model adaptation by updating only a small subset of parameters; and Semi-Supervised Fine-Tuning, which tackles the challenge of training models with unlabeled data. Each section explores the methods, challenges, and applications of these approaches, highlighting their importance in interactive natural language processing systems. Through fine-tuning, LLMs can more effectively leverage interaction data, enabling iterative improvements in their interactions with external entities.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fine-Tuning

  • Zekun Wang,
  • Qingqing Zhu,
  • Xiuying Chen,
  • Mong Yuan Sim,
  • Wangchunshu Zhou,
  • Shaochun Hao,
  • Jie Fu

摘要

In this chapter, we review the recent developments in Large Language Model (LLM) fine-tuning, a process that involves updating model parameters to enhance their performance. Fine-tuning is essential for adapting models to improve abilities such as instruction following, grounding, and generalization to new domains. The chapter examines four key fine-tuning approaches: Supervised Instruction Tuning, which enhances task generalization through supervised data; Continual Learning, which integrates new information while avoiding catastrophic forgetting; Parameter-Efficient Fine-Tuning, which allows efficient model adaptation by updating only a small subset of parameters; and Semi-Supervised Fine-Tuning, which tackles the challenge of training models with unlabeled data. Each section explores the methods, challenges, and applications of these approaches, highlighting their importance in interactive natural language processing systems. Through fine-tuning, LLMs can more effectively leverage interaction data, enabling iterative improvements in their interactions with external entities.