Scholarlink

<p>Customizing diffusion models for multiple concepts remains challenging due to cross-concept interference. This paper introduces Multi-SBoRA, a novel method for customizing diffusion models for multiple concepts. By leveraging orthogonal standard basis vectors, Multi-SBoRA constructs low-rank matrices for LoRA fine-tuning, enabling regional and non-overlapping weight updates that effectively mitigate crosstalk between different concepts. This approach preserves the knowledge embedded in the pre-trained model and reduces interference between customized concepts, thereby ensuring that each concept is learned independently without compromising its integrity. The localized weight updates also reduce computational overhead and enhance model flexibility. Experimental results demonstrate the optimal quantitative performance of Multi-SBoRA, showcasing its efficacy in addressing multi-concept customization while maintaining independence via orthogonality and localized updates, and mitigating crosstalk effects.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-SBoRA: regional and non-overlapping weight updates for multi-concept customization of diffusion models

  • 期刊论文

  • 发表日期：

    2025年10月9日

查看全文
  • Haoxuan Wu,
  • Lai-Man Po,
  • Yuyang Liu,
  • Wing-Yin Yu,
  • Tianqi Zhang,
  • Zeyu Jiang,
  • Kun Li

摘要

Customizing diffusion models for multiple concepts remains challenging due to cross-concept interference. This paper introduces Multi-SBoRA, a novel method for customizing diffusion models for multiple concepts. By leveraging orthogonal standard basis vectors, Multi-SBoRA constructs low-rank matrices for LoRA fine-tuning, enabling regional and non-overlapping weight updates that effectively mitigate crosstalk between different concepts. This approach preserves the knowledge embedded in the pre-trained model and reduces interference between customized concepts, thereby ensuring that each concept is learned independently without compromising its integrity. The localized weight updates also reduce computational overhead and enhance model flexibility. Experimental results demonstrate the optimal quantitative performance of Multi-SBoRA, showcasing its efficacy in addressing multi-concept customization while maintaining independence via orthogonality and localized updates, and mitigating crosstalk effects.