Two-Stage Unsupervised Disentangled Realism Enhancement for Rendered Indoor Scene Images
摘要
Photo-realistic rendering aims to render realistic material models using physically-based light transport models, resulting in a considerable need for computational resources and storage. Current techniques also utilize deep learning methods to enhance image realism. However, conventional unpaired synthesis-to-real transformation networks may lead to visible artifacts due to substantial discrepancies between input and output data distributions. To address these issues, this paper proposes a two-stage method to enhance the realism of indoor scene images, dividing the process into: Local-to-Global illumination rendered image enhancement (L2G) and Global illumination rendered-to-real image enhancement (G2R). In the L2G stage, global illumination images are introduced as reference images to learn the global illumination style. First, local illumination content features are extracted, and multi-scale auxiliary and skip feature information is integrated through Feature Fusion Module (FFM). The Dynamic Instance Normalization module is used for global illumination style matching, ultimately obtaining the predicted global illumination rendered image. After that, G2R further enhances texture and illumination. For texture enhancement, self-calibrated convolutional modules are adopted to better capture long-distance spatial dependencies and inter-channel dependencies across different spatial locations in the image. For illumination enhancement in G2R, we employ the same structure with a smoothing module to mitigate minor structural deformations. The experiment shows that, the method effectively enhances image realism, achieving promising results.