A Mutual Enhancement Framework for Specular Highlight Detection and Removal

Specular highlights, generated by direct light reflection from surfaces, can significantly reduce the image quality and impair various computer vision applications. Recently, the existing approaches for jointly detecting and removing specular highlights attempted to use the detection as guidance for highlight removal. However, they ignored that this kind of unidirectional enhancement was susceptible to detection tasks. To achieve mutual enhancement, we assume that discriminative features would benefit the highlight detection task which needs to distinguish between highlight areas and highlight-free areas, while coherent features would facilitate the learning of highlight removal since it requires converting highlight areas to highlight-free areas. Specifically, we propose a mutual enhancement framework (MEF-SHDR) that addresses both specular highlight detection and removal in a unified manner. The proposed framework designs a Feature Decomposition and Aggregation Module (FDAM) that separates highlight and highlight-free features explicitly and aggregates them for improved detection and removal performance. Comprehensive experiments are implemented on five widely used datasets, i.e., SHIQ, LIME, SD1, SD2, and RD, demonstrating the superiority of the proposed approach over previous state-of-the-art methods, as well as the effectiveness of jointly detecting and removing highlights. Code is available at https://github.com/drafly/MEF-SHDR .