Dusty image enhancement has attracted wide attention due to its practicability in autonomous and monitoring systems. However, little methods focus on advanced learning-based dedusting models stemming from the challenge of collecting paired training data. To bridge this problem, this paper introduces a novel large-scale benchmark dataset synthesized by the proposed synthetic method, named Realistic Single Image Dust Removal (RSIDR), for image dedusting tasks, which consisting both synthetic and corresponding real-world dusty images. In addition, we present a comprehensive study and evaluation of the state-of-the-art image enhancement methods on image dedusting tasks. Furthermore, we offer a diverse array of criteria and metrics for evaluating images, spanning from full-reference Image Quality Assessment (IQA) to no-reference IQA. Experiments on RSIDR reveal the limitations and advantages of the existing image enhancement algorithms and suggest promising research directions.