Document Specular Highlight Removal with Coarse-to-Fine Strategy

Specular highlight detection and removal are fundamental challenges in computer vision and image processing, with the detection results serving as a precursor to guide the model in achieving better removal of specular highlights. This paper introduces a novel highlight removal model, which presents an efficient end-to-end deep learning framework designed to automatically remove specular highlights from a single image. Our architecture comprises three key modules: the Coarse Predictor (CP), Refinement Predictor (RP), and Global Discriminator (GD). The CP utilizes a novel Transformer-based Unet architecture to recover the primary content, while the GD incorporates a discriminator to ensure the coarse result is more feasible in a global context. Lastly, the RP is based on conditional Denoising Diffusion Probabilistic Models (DDPM) and is responsible for predicting the residual information between the ground-truth and the CP-predicted image. Experimental results on four public benchmark images demonstrate that our method surpasses state-of-the-art methods in the task of highlight removal.