Recently, convolutional neural networks (CNNs) based methods have achieved great success in the field of image super-resolution. However, most methods produce over-smoothed outputs, and suffer from degradation for very low resolution images. We present an accurate CNN-based structure based on shearlet transform in this paper. Firstly, we propose a multi-scale fusion network (MSFN) to fully explore features from low resolution images. And then, we integrate shearlet transform to make MSFN further get the texture details for super-resolved images. The experiments demonstrate that the proposed network achieves superior super resolution results and outperforms the state-of-the-art.