Vision foundation models have exhibited exceptional generalization capabilities across various tasks through large-scale pretraining. In the realm of monocular depth estimation, existing models have excelled at predicting relative depth from pinhole camera images. However, while pinhole cameras are widely used, they are not always ideal for scenarios requiring a broader field of view, such as autonomous driving and surveillance. Fisheye cameras, which provide nearly a 180-degree field of view, serve as a cost-effective alternative to LiDAR for close-range depth sensing. However, the scarcity of publicly available fisheye image datasets for depth estimation limits their application. In this paper, we present DepthFisheye, an efficient fine-tuning method that adapts existing depth estimation models from pinhole to fisheye cameras. Our approach addresses both input adaptation—transitioning from pinhole to fisheye images—and output adaptation, transforming relative depth into metric depth. We propose the Distortion-Aware Adapter (DAA) to manage fisheye distortions without network forgetting and introduce the ScaleFormer Head (SFH) to predict global depth scale. Experimental results demonstrate that DepthFisheye significantly enhances the performance of depth estimation models on fisheye images, enabling accurate depth predictions with minimal computational cost. Our code is available at https://github.com/world-executed/DepthFisheye.git .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DepthFisheye: Efficient Fine-Tuning of Depth Estimation Models for Fisheye Cameras

  • Wenbin Wu,
  • Zhiwei Zhang,
  • Xin Tan,
  • Zhizhong Zhang,
  • Lizhuang Ma

摘要

Vision foundation models have exhibited exceptional generalization capabilities across various tasks through large-scale pretraining. In the realm of monocular depth estimation, existing models have excelled at predicting relative depth from pinhole camera images. However, while pinhole cameras are widely used, they are not always ideal for scenarios requiring a broader field of view, such as autonomous driving and surveillance. Fisheye cameras, which provide nearly a 180-degree field of view, serve as a cost-effective alternative to LiDAR for close-range depth sensing. However, the scarcity of publicly available fisheye image datasets for depth estimation limits their application. In this paper, we present DepthFisheye, an efficient fine-tuning method that adapts existing depth estimation models from pinhole to fisheye cameras. Our approach addresses both input adaptation—transitioning from pinhole to fisheye images—and output adaptation, transforming relative depth into metric depth. We propose the Distortion-Aware Adapter (DAA) to manage fisheye distortions without network forgetting and introduce the ScaleFormer Head (SFH) to predict global depth scale. Experimental results demonstrate that DepthFisheye significantly enhances the performance of depth estimation models on fisheye images, enabling accurate depth predictions with minimal computational cost. Our code is available at https://github.com/world-executed/DepthFisheye.git .