With the development of large neural models, the scale of their parameters is increasing rapidly. The amount of their training computation increases almost exponentially, leading to high costs. To address the problem, this paper proposes a new accelerated method called Grad-Grow (Gradient-based Growth Learning) to improve training efficiency and obtain competitive performance. This method adopts the local-to-global learning strategy, which first optimizes a sub-model and then growingly trains larger sub-models step-by-step. Eventually, the whole model is learned. We design the growth strategy from the perspective of progressive optimization and use gradient signals to guide the growth direction. With this growth learning way, the small sub-model can be trained quickly and provides a better initialization to learn the larger sub-model in the next stage. That can boost convergence and reduce computational costs. Extensive experiments are conducted on two tasks’ datasets with different network architectures. The results verify the effectiveness of our approach, with 30% training acceleration on average.