<p>Instruction tuning aims to improve cross-task generalization of language models. However, it is still challenging for language models to complete the target tasks following the instructions because the instructions are general and lack intermediate steps. To solve this issue, step-by-step instructions are introduced to help language models decompose tasks, which can offer detailed and specific procedures for completing the target tasks. The step-by-step instructions are obtained automatically by prompting ChatGPT, and are further merged with the original instructions to tune the language models. Extensive experiments on S<span>up</span>-N<span>at</span>I<span>nst</span> reveal that high-quality step-by-step instructions can enhance cross-task generalization across different model sizes. Further analysis indicates the significance of the order of steps in the proposed instruction for improvement. To promote future research, step-by-step instructions and human quality evaluation results will be released.</p>