New Insights on the Generation of Rain Streaks: Generating-Removing United Unpaired Image Deraining Network

Most existing deraining methods use synthetic rainy images to train models. They focus on extracting the features to establish mapping models from rainy images to clean background images, while ignoring the domain gap between synthetic and real rainy images. Further more, the rain streaks generation is not considered as important as the rain removal. Hence, we propose a novel Generating-Removing United Unpaired image deraining Network(GRUUNet) to generate and remove rain streaks unitedly. It helps to reduce the difference between synthetic and real rain streaks to improve the performance of deraining. Specifically, (1)we adopt a dual-way transform strategy between real and synthetic rainy images. There are both rain streaks generation and removal network for real and synthetic rainy images respectively; (2)our model learns the prototypes of rainy degradation images by self-supervised sparse-addressing memory modules; (3)we align the domain gap between real and synthetic rainy images with shared rain streaks generation network. We empirically evaluate our GRUUNet on Cityscape and RainHQ, and the outstanding results prove the promising performance of our method on both real and synthetic rainy images.