Region-assisted line drawing colorization through diffusion model

Line drawing colorization is a crucial step in the image creation process, yet traditional manual coloring demands considerable time and effort from skilled artists. While deep learning advancements have enabled colorization through various user prompts and text inputs, these approaches still often require some degree of human intervention. The challenge lies in finding fully automated methods that can achieve high-quality results without manual assistance, maintaining both efficiency and artistic integrity. In this paper, we propose a region assisted for reference-based line drawing colorization, which uses a more stable diffusion model to automatically colorize line drawing and introduces a skeleton map as an additional guide to reduce the bleeding problem encountered during colorization and improve the quality of the generated images. To further improve the model’s ability to capture the colors from reference images and enhance the overall quality of the colorized output, we adopt a two-stage training strategy. In the first stage, a pre-trained model designed to capture cartoon-like features was trained on a large-scale dataset. The second stage involved fine-tuning the model on a smaller, specialized dataset. Additionally, we have created a paired dataset of fashion line drawings and illustrations, which can be utilized in the fashion design industry. The effectiveness of our reference-based automatic coloring approach was validated through extensive qualitative and quantitative experiments, demonstrating its robustness and adaptability across various contexts. Our code will be released at (https://github.com/jackmeme1/colorization).