The transferability of adversarial examples under the black-box setting piques the interest of deep learning practitioners, which also offers an effective direction to identify the deficiencies of DNNs, especially for safety-critical applications. Due to its wide range of expertise, ensemble adversarial attack shows promise in synthesizing more transferable adversarial samples while being under-explored, as existing ensemble methods simply sum the contributions of sub-models or just reduce their gradient variance. Therefore, this paper develops a novel Gradients Reweighing method (dubbed GREW) for ensemble attack to steer adversarial example generation through reweighing the importance of gradients from different sub-models. Specifically, the proposed GREW method can adaptively adjust the importance of the gradient from different sub-models in each iteration, thus helping the model escape from the local optimum. Comprehensive experimental results on several widely used datasets including CIFAR100 and ImageNet confirm the superiority and effectiveness of our transferability and stability when compared to the recent methods.