Scholarlink

<p>Tracking shot videos, where a third person camera follows a moving subject, are central to games and cinematography and are increasingly useful for simulation, content creation, and embodied evaluation. However, existing controllable video generators often rely on text or coarse global cues. In addition, their motion transfer pipelines typically assume a fixed viewpoint, and game specific systems frequently discretize movement. Consequently, producing third person videos with free human level motion and consistent camera tracking from minimal inputs remains underexplored. To address these issues, we present PlayLife, a tracking shot <i>image to video</i> generator. Specifically, given a single exocentric image and a human motion sequence, it synthesizes third person videos in which a virtual camera consistently follows the actor while the generated motion adheres to the provided actions across AAA game and real world scenes. At the core is a <i>geometry-aware action injection</i> module. Concretely, articulated keypoints from the motion sequence are projected into the first frame using estimated camera parameters and are fused with video latents through cross attention. Through this design, the model achieves part level correspondence, stable trajectories, and scene aware view control. To learn from scarce and noisy supervision, we adopt <i>hierarchical training</i>. In the first stage, lightweight temporal adapters are pretrained on a large corpus with paired motion and video data; in the second stage, they are finetuned on a curated subset with a <i>pose aware reconstruction</i> loss that emphasizes articulated foreground regions, thereby improving temporal synchronization and motion fidelity. Finally, we curate tracking shot data and establish evaluation protocols in both game and real world settings. Furthermore, experiments with visualizations, automatic metrics, and user studies show strong controllability, accurate alignment between motion and camera view, high perceptual realism, and robust generalization from minimal inputs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

PlayLife: Tracking-Shot Video Generation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月11日

查看全文
  • Yuanpeng Tu

摘要

Tracking shot videos, where a third person camera follows a moving subject, are central to games and cinematography and are increasingly useful for simulation, content creation, and embodied evaluation. However, existing controllable video generators often rely on text or coarse global cues. In addition, their motion transfer pipelines typically assume a fixed viewpoint, and game specific systems frequently discretize movement. Consequently, producing third person videos with free human level motion and consistent camera tracking from minimal inputs remains underexplored. To address these issues, we present PlayLife, a tracking shot image to video generator. Specifically, given a single exocentric image and a human motion sequence, it synthesizes third person videos in which a virtual camera consistently follows the actor while the generated motion adheres to the provided actions across AAA game and real world scenes. At the core is a geometry-aware action injection module. Concretely, articulated keypoints from the motion sequence are projected into the first frame using estimated camera parameters and are fused with video latents through cross attention. Through this design, the model achieves part level correspondence, stable trajectories, and scene aware view control. To learn from scarce and noisy supervision, we adopt hierarchical training. In the first stage, lightweight temporal adapters are pretrained on a large corpus with paired motion and video data; in the second stage, they are finetuned on a curated subset with a pose aware reconstruction loss that emphasizes articulated foreground regions, thereby improving temporal synchronization and motion fidelity. Finally, we curate tracking shot data and establish evaluation protocols in both game and real world settings. Furthermore, experiments with visualizations, automatic metrics, and user studies show strong controllability, accurate alignment between motion and camera view, high perceptual realism, and robust generalization from minimal inputs.