Instruction Tuning
摘要
Pre-training forms the foundational stage of large language models (LLMs), primarily built on the objective of predicting the next token. While pre-training is highly effective, models immediately after this stage often struggle to adapt effectively to task-solving scenarios. To enhance their task-solving capabilities, instruction tuning or supervised fine-tuning is employed, teaching LLMs to solve tasks described in natural language instructions. This approach has proven effective in eliciting and unlocking the task-solving abilities of LLMs. This chapter provides a comprehensive introduction to instruction fine-tuning methods, covering various aspects from dataset construction to training strategies. We begin by exploring methods for constructing instruction datasets, including the adaptation of existing datasets and the synthesis of high-quality data. We also discuss key factors in instruction data construction and their impact on model performance. Next, we examine typical training strategies for improving instruction tuning, such as optimization settings and data organization. Additionally, we provide an overview of parameter-efficient fine-tuning methods, particularly in low-resource scenarios, with a focus on the widely used lightweight fine-tuning technique, LoRA (Low-Rank Adaptation). We also briefly introduce several variants of parameter-efficient fine-tuning. Finally, we present code implementations and conduct an empirical study to analyze the effects of different instruction datasets.