Scholarlink

<p>In recent years, text-to-image generation technology based on diffusion models has made significant progress, but extending it to the field of video generation, especially under few-shot conditions, still faces huge challenges. Existing methods usually rely on a large amount of text-video pair data or consume a lot of training resources. Based on this, this paper proposes a new few-shot video generation framework, <b>STAN-SNR</b>, which combines spatio-temporal feature regulation, feature scrolling enhancement and dynamic signal-to-noise ratio (SNR) weighting strategies, using 8–16 videos on a single A6000 training, effectively improving the quality and efficiency of video generation and reducing the amount of calculation. Specifically, the spatio-temporal feature regulation module effectively extracts spatio-temporal features and reduces computational complexity. The feature scrolling enhancement module enhances the ability to capture local features to avoid overfitting. In addition, the dynamic SNR weighting strategy adjusts the loss calculation according to the time step, which improves the convergence speed of the model, which is 2.44 times faster compared with the baseline model. The experimental results show that the STAN-SNR framework generates videos with higher text alignment, consistency, and diversity under few-shot conditions.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

STAR-SNR: spatial–temporal adaptive regulation and SNR optimization for few-shot video generation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年5月9日

查看全文
  • Xian Yu,
  • Jianxun Zhang,
  • Siran Tian,
  • Hongyu Yi

摘要

In recent years, text-to-image generation technology based on diffusion models has made significant progress, but extending it to the field of video generation, especially under few-shot conditions, still faces huge challenges. Existing methods usually rely on a large amount of text-video pair data or consume a lot of training resources. Based on this, this paper proposes a new few-shot video generation framework, STAN-SNR, which combines spatio-temporal feature regulation, feature scrolling enhancement and dynamic signal-to-noise ratio (SNR) weighting strategies, using 8–16 videos on a single A6000 training, effectively improving the quality and efficiency of video generation and reducing the amount of calculation. Specifically, the spatio-temporal feature regulation module effectively extracts spatio-temporal features and reduces computational complexity. The feature scrolling enhancement module enhances the ability to capture local features to avoid overfitting. In addition, the dynamic SNR weighting strategy adjusts the loss calculation according to the time step, which improves the convergence speed of the model, which is 2.44 times faster compared with the baseline model. The experimental results show that the STAN-SNR framework generates videos with higher text alignment, consistency, and diversity under few-shot conditions.