With the rise of powerful code generation large language models (LLMs), there is increasing interest in distilling the capabilities of these LLMs to smaller LLMs. Due to the difference between code and text, many methods use reinforcement learning for guidance in distillation. However, most of these methods only utilize the results of unit tests or are too complex and difficult to apply in practice, resulting in poor performance after model distillation. Therefore, we propose a method to make the granularity in reinforcement learning more nuanced and simple to apply. We incorporated the process of generating code for the model into reinforcement learning as a way to guide the distillation process. We evaluated on benchmark datasets HumanEval and MBPP, and experimented with different types of LLMs. The experimental results show that our method achieves good performance.