Recently, the first foundation models for monocular depth estimation such as Depth Anything have emerged . However, by being trained to make affine-invariant predictions, these methods rely on fine-tuning for making metric depth predictions and therefore perform poorly on zero-shot metric depth estimation. In a real use case, the fine-tuning stage is costly because a dedicated dataset with ground truth depth must be created and used as a training set. Additionally, fine-tuning can compromise the model’s generalization ability. This paper proposes to leverage 2D LiDARs to rescale Depth Anything’s predictions in the context of indoor scenes so as to prevent expensive fine-tuning or harming the model capacity. Our experiments demonstrate similar performance with fine-tuned approaches and enhanced results over zero-shot metric depth estimation methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Recovering Dense Metric Depth in Indoor Scenes from Monocular Depth Foundation Models and 2D LiDARs

  • Rémi Marsal,
  • Alexandre Chapoutot,
  • Philippe Xu,
  • David Filliat

摘要

Recently, the first foundation models for monocular depth estimation such as Depth Anything have emerged . However, by being trained to make affine-invariant predictions, these methods rely on fine-tuning for making metric depth predictions and therefore perform poorly on zero-shot metric depth estimation. In a real use case, the fine-tuning stage is costly because a dedicated dataset with ground truth depth must be created and used as a training set. Additionally, fine-tuning can compromise the model’s generalization ability. This paper proposes to leverage 2D LiDARs to rescale Depth Anything’s predictions in the context of indoor scenes so as to prevent expensive fine-tuning or harming the model capacity. Our experiments demonstrate similar performance with fine-tuned approaches and enhanced results over zero-shot metric depth estimation methods.