Generative AI has emerged as a focal point for research in recent times and has also achieved significant success. Text-to-Image (T2I) generation is one such field. Many efforts have been made to replicate the success of T2I models in the realm of Text-to-Video (T2V). This paper dwells on how T2I models can be leveraged to T2V models with the use of diffusion-based schedulers like Denoising Diffusion Implicit Models (DDIM) and Denoising Diffusion Probabilistic Models (DDPM) which help the model in pertaining the temporal consistency and fluidity between generated frames of the video. Additionally we also dwell on the paradigms on how the T2V models are trained using zero-shot and one-shot training methods. Zero-shot training generalizes the model across diverse dataset which enables the model to generate videos from unseen text prompts, whereas one-shot training fine tunes the model with minimal data which enables the model to generalize within the trained domain. Through this research, we aim to highlight the emerging potential of T2V models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Advancing Text-to-Video Models: A Study of Schedulers and Learning Techniques in Video Synthesis

  • Nikhil D. Bharadwaj,
  • C. S. Chaithra,
  • V. N. Manjunath Aradhya

摘要

Generative AI has emerged as a focal point for research in recent times and has also achieved significant success. Text-to-Image (T2I) generation is one such field. Many efforts have been made to replicate the success of T2I models in the realm of Text-to-Video (T2V). This paper dwells on how T2I models can be leveraged to T2V models with the use of diffusion-based schedulers like Denoising Diffusion Implicit Models (DDIM) and Denoising Diffusion Probabilistic Models (DDPM) which help the model in pertaining the temporal consistency and fluidity between generated frames of the video. Additionally we also dwell on the paradigms on how the T2V models are trained using zero-shot and one-shot training methods. Zero-shot training generalizes the model across diverse dataset which enables the model to generate videos from unseen text prompts, whereas one-shot training fine tunes the model with minimal data which enables the model to generalize within the trained domain. Through this research, we aim to highlight the emerging potential of T2V models.