Scholarlink

<p>Image colorization, a challenging task in computer vision, aims to restore color to grayscale images. This study introduces a novel diffusion-based framework that integrates semantic conditioning for high-fidelity colorization. Our approach involves training a grayscale-adapted encoder and a CLIP model separately to extract and align semantic features. The U-Net component of a pre-trained stable diffusion model is fine-tuned using low-rank adaptation (LoRA) for efficient and high-quality color generation. Experimental results demonstrate strong performance, achieving an FID of 13.53 and a PSNR of 24.42, outperforming existing baselines in both accuracy and perceptual quality. These findings underscore the effectiveness of combining diffusion models with semantic guidance and lightweight fine-tuning for robust and generalizable image colorization. Our implementation is publicly available at <a href="https://github.com/mohsenosl99/colorization_SD_Lora">https://github.com/mohsenosl99/colorization_SD_Lora</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing image colorization with semantic-guided diffusion and lightweight fine-tuning

  • 期刊论文

  • 发表日期：

    2026年6月26日

查看全文
  • Mohsen Osooli,
  • Mostafa Tavassolipour

摘要

Image colorization, a challenging task in computer vision, aims to restore color to grayscale images. This study introduces a novel diffusion-based framework that integrates semantic conditioning for high-fidelity colorization. Our approach involves training a grayscale-adapted encoder and a CLIP model separately to extract and align semantic features. The U-Net component of a pre-trained stable diffusion model is fine-tuned using low-rank adaptation (LoRA) for efficient and high-quality color generation. Experimental results demonstrate strong performance, achieving an FID of 13.53 and a PSNR of 24.42, outperforming existing baselines in both accuracy and perceptual quality. These findings underscore the effectiveness of combining diffusion models with semantic guidance and lightweight fine-tuning for robust and generalizable image colorization. Our implementation is publicly available at https://github.com/mohsenosl99/colorization_SD_Lora.