DepthFisheye: Efficient Fine-Tuning of Depth Estimation Models for Fisheye Cameras
摘要
Vision foundation models have exhibited exceptional generalization capabilities across various tasks through large-scale pretraining. In the realm of monocular depth estimation, existing models have excelled at predicting relative depth from pinhole camera images. However, while pinhole cameras are widely used, they are not always ideal for scenarios requiring a broader field of view, such as autonomous driving and surveillance. Fisheye cameras, which provide nearly a 180-degree field of view, serve as a cost-effective alternative to LiDAR for close-range depth sensing. However, the scarcity of publicly available fisheye image datasets for depth estimation limits their application. In this paper, we present DepthFisheye, an efficient fine-tuning method that adapts existing depth estimation models from pinhole to fisheye cameras. Our approach addresses both input adaptation—transitioning from pinhole to fisheye images—and output adaptation, transforming relative depth into metric depth. We propose the Distortion-Aware Adapter (DAA) to manage fisheye distortions without network forgetting and introduce the ScaleFormer Head (SFH) to predict global depth scale. Experimental results demonstrate that DepthFisheye significantly enhances the performance of depth estimation models on fisheye images, enabling accurate depth predictions with minimal computational cost. Our code is available at https://github.com/world-executed/DepthFisheye.git .