We present MonoBite, a scale-aware 3D reconstruction and volume estimation pipeline for monocular multi-food images. Accurate food volume estimation from RGB images is critical for dietary tracking and nutritional assessment, yet remains challenging due to the lack of depth information and physical references. Existing methods typically rely on multi-view inputs or explicit physical references such as checkerboards, limiting their applicability in real-world settings. In contrast, MonoBite performs single-view, multi-object 3D reconstruction without requiring explicit references or camera calibration. Our approach introduces three key modules: (1) Single-View Multi-Object 3D Reconstruction (SVMOR), which leverages 3D foundation models enhanced with an implicit plane hint; (2) Scaffold Point Cloud Mesh Alignment (SPCMA), which enables accurate local scaling by aligning reconstructed meshes with metric depth predictions; and (3) Metric Depth Refinement (MDR), which refines global scale using contextual cues from real-world dining scenes. MonoBite achieves state-of-the-art performance with a MAPE of 0.23 and a Chamfer Distance of 5.85 on the CVPR 2025 2nd MetaFood Workshop challenge dataset, winning first place in the official competition.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MonoBite: Scale-Aware 3D Reconstruction and Volume Estimation from Monocular Multi-food Images

  • Songen Gu,
  • Lina Liu,
  • Binjie Liu,
  • Sanyi Zhang,
  • Lei Yang,
  • Yanwei Fu

摘要

We present MonoBite, a scale-aware 3D reconstruction and volume estimation pipeline for monocular multi-food images. Accurate food volume estimation from RGB images is critical for dietary tracking and nutritional assessment, yet remains challenging due to the lack of depth information and physical references. Existing methods typically rely on multi-view inputs or explicit physical references such as checkerboards, limiting their applicability in real-world settings. In contrast, MonoBite performs single-view, multi-object 3D reconstruction without requiring explicit references or camera calibration. Our approach introduces three key modules: (1) Single-View Multi-Object 3D Reconstruction (SVMOR), which leverages 3D foundation models enhanced with an implicit plane hint; (2) Scaffold Point Cloud Mesh Alignment (SPCMA), which enables accurate local scaling by aligning reconstructed meshes with metric depth predictions; and (3) Metric Depth Refinement (MDR), which refines global scale using contextual cues from real-world dining scenes. MonoBite achieves state-of-the-art performance with a MAPE of 0.23 and a Chamfer Distance of 5.85 on the CVPR 2025 2nd MetaFood Workshop challenge dataset, winning first place in the official competition.