Recent advances in deep learning-based Multi-View Stereo (MVS) methods have significantly improved reconstruction quality. However, current deep MVS approaches still exhibit limitations in feature extraction. Most methods rely on traditional convolutional layers for feature extraction, resulting in relatively weak discriminative power and a failure to fully leverage the critical structural information embedded in the depth maps and their corresponding features. This often leads to reconstruction artifacts such as noise and blurred edges in challenging regions. To address these issues, we propose the ZC-MVSNet. Specifically, we first design a novel zero-sum convolution layers to replace the standard convolutions in the traditional feature pyramid network. Additionally, we introduce an entropy maximization constraint for invalid depth-annotated regions. By applying an entropy maximization loss, the network is encouraged to learn more discriminative feature distributions in challenging areas. Furthermore, we design a geometry-guided cross-stage feature enhancement module to improve geometric reasoning. This module decodes the cost volume from the coarse stage into a geometric feature map and adaptively fuses it with the features from the fine-stage feature pyramid network. To further enhance feature representation, we employ a geometric self-attention mechanism that utilizes coarse depth as a pseudo-depth modality for effective feature modulation. Extensive experiments on the DTU and Tanks & Temples benchmark datasets demonstrate that our method achieves state-of-the-art performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ZC-MVSNet: Zero-Sum Convolution and Prior Fusion for Multi-view Stereo

  • Weibin Qiu,
  • Suping Wu,
  • Hao Xu,
  • Jie Yang,
  • Xiang Zhang

摘要

Recent advances in deep learning-based Multi-View Stereo (MVS) methods have significantly improved reconstruction quality. However, current deep MVS approaches still exhibit limitations in feature extraction. Most methods rely on traditional convolutional layers for feature extraction, resulting in relatively weak discriminative power and a failure to fully leverage the critical structural information embedded in the depth maps and their corresponding features. This often leads to reconstruction artifacts such as noise and blurred edges in challenging regions. To address these issues, we propose the ZC-MVSNet. Specifically, we first design a novel zero-sum convolution layers to replace the standard convolutions in the traditional feature pyramid network. Additionally, we introduce an entropy maximization constraint for invalid depth-annotated regions. By applying an entropy maximization loss, the network is encouraged to learn more discriminative feature distributions in challenging areas. Furthermore, we design a geometry-guided cross-stage feature enhancement module to improve geometric reasoning. This module decodes the cost volume from the coarse stage into a geometric feature map and adaptively fuses it with the features from the fine-stage feature pyramid network. To further enhance feature representation, we employ a geometric self-attention mechanism that utilizes coarse depth as a pseudo-depth modality for effective feature modulation. Extensive experiments on the DTU and Tanks & Temples benchmark datasets demonstrate that our method achieves state-of-the-art performance.