Due to the diversity and complexity of snowfall, removing snow from a single image has become a challenging image restoration task. Existing methods struggle to effectively handle the diverse and complex scenarios of snow cover in different environments. To overcome these limitations, we propose a novel transformer model named MSA-Former. Compared with the traditional transformer-based approach, MSA-Former significantly enhances the understanding of image features under complex weather conditions. It achieves this by dynamically adjusting the feature fusion process with the guidance of a significance map derived from multi-scale attention. In addition, MSA-Former introduces the Fusion Enhancement Module (FEM) and the Global Attention Mechanism (GAM) to further improve the model’s ability to represent both global and local features in images. This enhancement is crucial for recovering snow-covered regions and enhancing the local details of the image. We conducted extensive experimental validation of MSA-Former on several synthetic datasets and the newly constructed Night Snow Dataset (NSD). The results of the experiment demonstrate that MSA-Former can greatly enhance the overall quality of recovered images while removing snow, surpassing existing snow removal methods and general image restoration techniques. Experiments evaluating public synthesized and real snow images validate the proposed method’s superiority, providing quantitative and qualitative results.