Msc-Net: multi-stage colorization network for real-world images with specular highlights

Image coloring has always been a challenging problem. Current coloring methods cannot eliminate the ambiguity of bright material surfaces from specular highlight images, as the presence of specular highlights introduces complex brightness and color changes. To tackle this issue, we propose a novel multi-stage colorization network (MSC-Net) to identify and eliminate color ambiguity caused by specular highlights accurately. Firstly, we design a new method for detecting and removing specular highlights in images, including a specular highlight segmentation network(SHSNet) and a highlight-eliminating module, in order to accurately detect and eliminate specular highlight areas for subsequent coloring. Subsequently, we give a deep learning-based coloring network(ColorNet) and integrated highlight features to color the image. After coloring, the completed image will be combined with the original specular highlight information to achieve a comprehensive restoration of the highlighted area. Experiments on multiple public datasets are conducted, validating that our MSC-Net achieves favorable results in specular highlight segmentation and performs excellently in coloring images with specular highlights. Our MSC-Net provides a mature solution for image coloring with specular highlights, which has significant theoretical and practical implications. Our code is available at: https://github.com/ymm0304/MSC-Net.