Controllable Rain Image Generation: Balance Between Diversity and Controllability

Currently, there is a lack of DL-based rain image generation methods that simultaneously consider diversity and controllability. However, these methods are essential for conveniently generating realistic and diverse rain layers for rainy images synthesis, which can provide training datasets for DL-based image rain removal models. In this paper, to balance diversity and controllability in rain image generation, we novelly construct a generative model within the Bayesian framework to depict the generation process of real rainy images and rendered rain images, thus achieving a combination of explicit and implicit control. To provide realistic rain images with prior supervision of controllable rain attributes, we introduce dual discriminator adversarial learning for model training. Through the adversarial learning between the generated rainy images and real rainy images, the rain generator can capture the statistical distribution of rain from real rainy images, generating non-repetitive and diverse rain streaks. The adversarial learning between the generated rain images and rendered rain images force the generator to imitate the rain rendering process, thereby improving its controllability. Rain generation and disentanglement experiments show that our model has explicit control and disentanglement capabilities on rain attributes. We also validate the diversity of our generated rain streaks by dataset augmentation and rain removal experiments on real rainy image dataset. This coincides with our goal of having a good balance between diversity and controllability in rain image generation.