This paper proposes a novel imitation learning approach for the whole-body motion control of humanoid robots. Based on the instruction learning framework proposed in our prior work, we integrate human motion capture data as a feedforward action in this paper, which is combined with a feedback action driven by reinforcement learning to achieve human-like whole-body movements. Compared to other imitation learning methods, the proposed method can significantly enhance the training efficiency due to the application of a feedforward action. Furthermore, since the motion-mimic capability is mainly determined by the feedforward action while the neural network only plays a role as a stabilizer, it enables the control of multiple motion skills using a single neural network. The effectiveness of the proposed method in whole-body motion imitation learning is verified through several simulation tasks performed on the Unitree H1 robot. The attached video can be found at https://linqi-ye.github.io/video/icira25.mp4 .