Learning 3D-aware generators from 2D image collections has attracted significant attention in the field of generative modeling. However, there are several challenges in generating high-resolution multi-view consistent images, e.g., 2D CNN-based approaches leverage upsampling layers to generate high-resolution images, easily resulting in inconsistencies across multi-view images; methods that generate images based on NeRF require tremendous memory space and a long time to converge. To this end, we propose a novel 3D-aware generative method named 3D-HRFC to generate high-resolution consistent images with faster convergence. Specifically, we first propose a depth fusion based super-resolution module that integrates the depth maps into the low-resolution images in order to generate consistent multi-view images. And then a skip super-resolution module is devised to enhance the generation of the high-resolution images. To generate high-resolution consistent images and accelerate the model convergence, we devise a composite loss function that consists of adversarial loss, super-resolution loss, and content consistency. Extensive experiments conducted on FFHQ and AFHQ-v2 Cats datasets illustrate that our proposed method can generate high-quality 3D-consistent images.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

3D-HRFC: 3D-Aware Image Generation at High Resolution with Faster Convergence

  • Qiqiang Xia,
  • Junhong Chen,
  • Tianxiao Li,
  • Yiheng Huang,
  • Muhammad Asim,
  • Nick Michiels,
  • Wenyin Liu

摘要

Learning 3D-aware generators from 2D image collections has attracted significant attention in the field of generative modeling. However, there are several challenges in generating high-resolution multi-view consistent images, e.g., 2D CNN-based approaches leverage upsampling layers to generate high-resolution images, easily resulting in inconsistencies across multi-view images; methods that generate images based on NeRF require tremendous memory space and a long time to converge. To this end, we propose a novel 3D-aware generative method named 3D-HRFC to generate high-resolution consistent images with faster convergence. Specifically, we first propose a depth fusion based super-resolution module that integrates the depth maps into the low-resolution images in order to generate consistent multi-view images. And then a skip super-resolution module is devised to enhance the generation of the high-resolution images. To generate high-resolution consistent images and accelerate the model convergence, we devise a composite loss function that consists of adversarial loss, super-resolution loss, and content consistency. Extensive experiments conducted on FFHQ and AFHQ-v2 Cats datasets illustrate that our proposed method can generate high-quality 3D-consistent images.