In recent years, diffusion models brought unprecedented high-quality image generation, alongside high inference costs. We explore variants of FLUX, a state-of-the-art family of models: baseline FLUX.1-dev and distilled FLUX.1-schnell. We hypothesize that differences between outputs of baseline and distilled models are consistent within a specialized domain, such as portrait generation. Then, we suggest training a domain-specific image-to-image (I2I) fast translation model from schnell to dev domain. The paper discusses two potential backbones for the model: UNet, requiring a pairwise dataset of low- and high-quality images of the same scene and subject, and non-pairwise CycleGAN. We demonstrate that results produced by a distilled schnell model with our I2I head are perceptually close to what a baseline dev model would produce, while cutting 82% computational cost. We also show that results generated by CycleGAN are superior to UNet, which suggests that training images need not be paired in order to achieve satisfactory results.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing AI Face Realism: Cost-Efficient Quality Improvement in Distilled Diffusion Models with a Fully Synthetic Dataset

  • Jakub Wąsala,
  • Bartłomiej Wrzalski,
  • Kornelia Noculak,
  • Yuliia Tarasenko,
  • Oliwer Krupa,
  • Jan Kocoń,
  • Grzegorz Chodak

摘要

In recent years, diffusion models brought unprecedented high-quality image generation, alongside high inference costs. We explore variants of FLUX, a state-of-the-art family of models: baseline FLUX.1-dev and distilled FLUX.1-schnell. We hypothesize that differences between outputs of baseline and distilled models are consistent within a specialized domain, such as portrait generation. Then, we suggest training a domain-specific image-to-image (I2I) fast translation model from schnell to dev domain. The paper discusses two potential backbones for the model: UNet, requiring a pairwise dataset of low- and high-quality images of the same scene and subject, and non-pairwise CycleGAN. We demonstrate that results produced by a distilled schnell model with our I2I head are perceptually close to what a baseline dev model would produce, while cutting 82% computational cost. We also show that results generated by CycleGAN are superior to UNet, which suggests that training images need not be paired in order to achieve satisfactory results.