Deep Unfolding Warping Network for Image Rectangling

Rectangling is a technique designed for stitched images, aiming for a neat rectangular view presentation like common natural images while maintaining high content fidelity. Previous mesh-based methods often cause either distortion or local void areas (ranging from small to large) in the final output image. This is due to poor mesh prediction and the characteristic of rigid transformation. To address these issues, we propose the Deep Unfolding Warping Network (DUWN) by decomposing the rectangling process into two stages: rectangling deformation and alignment. In the first stage, our goal is to estimate the crude mesh as good as possible to mediate anamorphose or big void districts, which is achieved in the way of unfolding it into a series of estimation steps. Concretely, a direct mesh predictor and a deformer are designed to perform in an alternating way during these steps. Then, a specialized decorator network is introduced to handle those unwanted small voids in the alignment stage. It can provide valuable pixels around these areas, which are highly correlated with the pixels determined by the deformation. Moreover, our decorator can rectify the degradation caused by the former warping operation, improving the visual effects. We evaluate our proposed approach on the public rectangling benchmark, and numerous experiments have demonstrated the superiority of our designs. Our algorithms have achieved state-of-the-art performance compared with other methods.