Finetuning: Hands on

In Chapter 5, you learned about fine-tuning and model alignment in a very theoretical manner. It was the foundation to being able to fine-tune your own models. You learned about the whys, whats, and hows of fine-tuning. You learned that fine-tuning can be less resource and time consuming than building and training a model from scratch. The previous chapter talked to you about what happens to the neural network during the fine-tuning process – specifically that most layers are “frozen” and the final few layers are updated to adapt the model to a new task. The focus was on Reinforcement Learning with Human Feedback (RLHF) and Parameter-Efficient Fine-Tuning (PEFT).