Single Image Highlight Removal via Innovative Pseudo Image Bases Fusion with a Dual-Network

Specular highlights in images can obscure or completely destroy color and texture details, posing significant challenges to various visual tasks. Traditional highlight removal methods often struggle with complex surfaces or require strict shooting conditions. Deep learning-based methods excel at handling intricate surfaces but still suffer from texture distortion in highlight regions. This paper proposes a novel highlight removal method based on the observation that specular highlights increase brightness and decrease saturation. We generate pseudo-SV (saturation-value) modulated image bases that effectively constructing a discrete color space that closely approximates the brightness, saturation, and hues of highlight-free pixels. We propose a dual-network architecture that jointly trains a highlight detection sub-network and a highlight removal sub-network. By integrating image bases and highlight positional priors, the removal network effectively learns to restore texture and color details through a weighted fusion process. Our results indicate that our approach effectively generates high-quality, highlight-free images, surpassing existing methods in PSNR and SSIM scores. The source code can be accessed at: https://github.com/XufangPANG/Highlight-Removal-based-on-Pesudo-image-bases-fusion .