Image animation brings static objects in source images to life based on driving video. Recent studies commonly employ unsupervised methods for transferring motion to arbitrary objects. However, these approaches encounter challenges due to the absence of prior knowledge about the object’s structure, which critically impacts the preservation of structural integrity during the motion transfer, ultimately influencing the quality of the results. To address this issue, we introduce an additional gradient branch that generates a warped gradient map, providing valuable structural priors for the motion transfer process. Furthermore, we incorporate a gradient loss to encourage the generator to focus more on preserving the geometric structure. Additionally, existing prior-based motion model methods typically predict low-resolution, single-scale motion flows, making it challenging to learn precise motion and adapt to multi-scale feature fusion. To address this, we propose a motion flow optimization module to generate higher-accuracy multi-scale motion flows. Experimental results demonstrate that our method exhibits high scalability and can be applied to various unsupervised motion models. It outperforms state-of-the-art methods across most benchmarks, leading to significant improvements in motion-related metrics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Gradient-Guided Motion Transfer with Motion Optimization

  • Qingwei Wang,
  • Tingting Xiao,
  • Jianjun Li

摘要

Image animation brings static objects in source images to life based on driving video. Recent studies commonly employ unsupervised methods for transferring motion to arbitrary objects. However, these approaches encounter challenges due to the absence of prior knowledge about the object’s structure, which critically impacts the preservation of structural integrity during the motion transfer, ultimately influencing the quality of the results. To address this issue, we introduce an additional gradient branch that generates a warped gradient map, providing valuable structural priors for the motion transfer process. Furthermore, we incorporate a gradient loss to encourage the generator to focus more on preserving the geometric structure. Additionally, existing prior-based motion model methods typically predict low-resolution, single-scale motion flows, making it challenging to learn precise motion and adapt to multi-scale feature fusion. To address this, we propose a motion flow optimization module to generate higher-accuracy multi-scale motion flows. Experimental results demonstrate that our method exhibits high scalability and can be applied to various unsupervised motion models. It outperforms state-of-the-art methods across most benchmarks, leading to significant improvements in motion-related metrics.