View transformation robustness (VTR) is critical for deep learning-based single- or multi-view 3D object reconstruction, serving as a key metric to evaluate model stability under diverse view transformations. Despite its importance, VTR remains underexplored in existing 3D reconstruction research. While data augmentation with varied view transformations is a straightforward approach to enhance VTR, the rapid development of large vision models, particularly Stable Diffusion models, offers considerable potential for generating 3D models or synthesizing new view images from single inputs for related tasks. In this work, we propose leveraging Stable Diffusion models to generate novel views without incurring additional inference costs, thereby significantly improving the performance of single-view 3D object reconstruction. Specifically, shifting our focus from traditional neural radiance field methods, we explore view selection strategies in 3D reconstruction. First, we optimize viewpoint quality via the farthest point sampling algorithm. Then, we use a generative model to expand single views into multiple views, enhancing single-view reconstruction performance without retraining the implicit field. Extensive experiments show our method outperforms state-of-the-art 3D reconstruction techniques and other VTR-focused approaches, validating its superiority and effectiveness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Robust Single-View 3D Object Reconstruction with Stable Diffusion Generation and Farthest View Selection

  • Zhouhang Luo,
  • Qian Yu,
  • Qi Zhang

摘要

View transformation robustness (VTR) is critical for deep learning-based single- or multi-view 3D object reconstruction, serving as a key metric to evaluate model stability under diverse view transformations. Despite its importance, VTR remains underexplored in existing 3D reconstruction research. While data augmentation with varied view transformations is a straightforward approach to enhance VTR, the rapid development of large vision models, particularly Stable Diffusion models, offers considerable potential for generating 3D models or synthesizing new view images from single inputs for related tasks. In this work, we propose leveraging Stable Diffusion models to generate novel views without incurring additional inference costs, thereby significantly improving the performance of single-view 3D object reconstruction. Specifically, shifting our focus from traditional neural radiance field methods, we explore view selection strategies in 3D reconstruction. First, we optimize viewpoint quality via the farthest point sampling algorithm. Then, we use a generative model to expand single views into multiple views, enhancing single-view reconstruction performance without retraining the implicit field. Extensive experiments show our method outperforms state-of-the-art 3D reconstruction techniques and other VTR-focused approaches, validating its superiority and effectiveness.