This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our method achieves real-time editing with a feedforward network (i.e., \(\sim \) 0.04 s per image), over 100 \(\times \) faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with \(\sim \) 5 min fine-tuning per style). Project page can be found here .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Time 3D-Aware Portrait Editing from a Single Image

  • Qingyan Bai,
  • Zifan Shi,
  • Yinghao Xu,
  • Hao Ouyang,
  • Qiuyu Wang,
  • Ceyuan Yang,
  • Xuan Wang,
  • Gordon Wetzstein,
  • Yujun Shen,
  • Qifeng Chen

摘要

This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our method achieves real-time editing with a feedforward network (i.e., \(\sim \) 0.04 s per image), over 100 \(\times \) faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with \(\sim \) 5 min fine-tuning per style). Project page can be found here .