Scholarlink

<p>Accurate ingredient-level segmentation is critical for intelligent dietary assessment, yet it remains challenging due to severe class imbalance and the visual ambiguity of cooked foods. This study aims to enhance segmentation accuracy, particularly for low-prevalence ingredients, by incorporating domain-specific culinary knowledge into deep learning models. We propose a general framework that leverages ingredient co-occurrence relationship constraints to guide the training of deep learning segmentation models. By statistically analyzing ingredient pairing patterns in large-scale datasets (FoodSeg103 and Recipe1M +), we construct co-occurrence matrices that capture the likelihood of ingredients appearing together. These matrices are integrated into the loss function of a U-Net architecture with various state-of-the-art encoders, providing contextual priors to improve model learning. Our method achieves significant and statistically significant performance gains across multiple encoder backbones. On the FoodSeg103 dataset, it improves the mean Intersection over Union (mIoU) by up to 3.72% and the F1-score by 3.65%. More notably, when utilizing the larger Recipe1M + dataset to construct the co-occurrence matrix, the mIoU improvement reaches a remarkable 16.54%, demonstrating strong generalization capabilities. Critically, for the EfficientNet-b7 model using the Recipe1M + matrix, the approach achieves a dramatic 755% improvement in mIoU for the least frequent 10% of ingredient classes, effectively mitigating the class imbalance problem. The approach maintains real-time inference capability with a latency of 15.68&#xa0;ms on a standard GPU. This work effectively addresses the class imbalance problem in food segmentation by integrating external culinary knowledge through co-occurrence matrices. The consistent and statistically significant performance improvements validate the framework's robustness. The method offers strong practical value for applications in automated dietary monitoring systems, smart dining platforms, and nutritional analysis tools.</p> Graphical Abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deep learning-enhanced food ingredient segmentation with co-occurrence relationship constraints

  • 期刊论文

  • 发表日期：

    2025年9月24日

查看全文
  • Huijiang Du,
  • Minghan Wang,
  • Qinlu Fang,
  • Mengbi Shen,
  • Liping Sun

摘要

Accurate ingredient-level segmentation is critical for intelligent dietary assessment, yet it remains challenging due to severe class imbalance and the visual ambiguity of cooked foods. This study aims to enhance segmentation accuracy, particularly for low-prevalence ingredients, by incorporating domain-specific culinary knowledge into deep learning models. We propose a general framework that leverages ingredient co-occurrence relationship constraints to guide the training of deep learning segmentation models. By statistically analyzing ingredient pairing patterns in large-scale datasets (FoodSeg103 and Recipe1M +), we construct co-occurrence matrices that capture the likelihood of ingredients appearing together. These matrices are integrated into the loss function of a U-Net architecture with various state-of-the-art encoders, providing contextual priors to improve model learning. Our method achieves significant and statistically significant performance gains across multiple encoder backbones. On the FoodSeg103 dataset, it improves the mean Intersection over Union (mIoU) by up to 3.72% and the F1-score by 3.65%. More notably, when utilizing the larger Recipe1M + dataset to construct the co-occurrence matrix, the mIoU improvement reaches a remarkable 16.54%, demonstrating strong generalization capabilities. Critically, for the EfficientNet-b7 model using the Recipe1M + matrix, the approach achieves a dramatic 755% improvement in mIoU for the least frequent 10% of ingredient classes, effectively mitigating the class imbalance problem. The approach maintains real-time inference capability with a latency of 15.68 ms on a standard GPU. This work effectively addresses the class imbalance problem in food segmentation by integrating external culinary knowledge through co-occurrence matrices. The consistent and statistically significant performance improvements validate the framework's robustness. The method offers strong practical value for applications in automated dietary monitoring systems, smart dining platforms, and nutritional analysis tools.

Graphical Abstract