In the field of image editing, inpainting tasks that aim to integrate customized elements into a given image context pose significant challenges. Existing methods often fail to adequately incorporate contextual features from the source image and exhibit limitations in maintaining subject consistency. This paper presents a novel approach to subject-driven image inpainting, leveraging a context-aware framework to achieve superior semantic integration of subjects. Our method introduces two key innovations. First, we employ an online augmented dataset constructed through image stitching, enabling the diffusion inpainting model to learn contextual embeddings of subject images. Second, we enhance the model’s semantic understanding by integrating subject image encoding into the diffusion UNet via cross-attention mechanisms and incorporating a prompt identifier. Through extensive experimentation, we demonstrate state-of-the-art performance across overall quality score metrics, surpassing recent methods in the field.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Online Data Augmentation and Subject Enhancement for Context-Aware Image Inpainting

  • Huanghao Yin,
  • Jing Zhu,
  • Junhai Yong,
  • Bin Wang

摘要

In the field of image editing, inpainting tasks that aim to integrate customized elements into a given image context pose significant challenges. Existing methods often fail to adequately incorporate contextual features from the source image and exhibit limitations in maintaining subject consistency. This paper presents a novel approach to subject-driven image inpainting, leveraging a context-aware framework to achieve superior semantic integration of subjects. Our method introduces two key innovations. First, we employ an online augmented dataset constructed through image stitching, enabling the diffusion inpainting model to learn contextual embeddings of subject images. Second, we enhance the model’s semantic understanding by integrating subject image encoding into the diffusion UNet via cross-attention mechanisms and incorporating a prompt identifier. Through extensive experimentation, we demonstrate state-of-the-art performance across overall quality score metrics, surpassing recent methods in the field.