A Low-Rank Appearance Recurrent Network for Single Image Rain Removal

Deep learning (DL) methods have achieved state-of-the-art performance for removing single image rain streaks. However, most existing DL models directly learn the residual between rainy images and clear images, ignoring the physical characteristics of rain streaks. This weakness makes these networks need to identify rain streaks and fix pixels simultaneously, which complicates the rain removal process and limits the rain removal performance, especially for heavy rain. To tackle this issue, we propose a Low-rank Appearance Recurrent Network called LRARNet based on the low-rank property of rain streak patterns in rainy images. Specifically, we first develop a low-rank layer to estimate the rain streak according to the principles of the low-rank appearance model, reducing the network’s pressure of rain streak identification. Subsequently, we design a multi-scale gated ConvLSTM (MSG-ConvLSTM) block to restore the clean image based on the estimated rain streak. LRARNet iteratively updates rain streaks and clean images in both temporal and spatial domains through the stacking of low-rank layers and MSG-ConvLSTM blocks. Extensive experiments demonstrate that our model has strong rain streak removal ability and certain interpretability, especially for heavy rain removal.