Food nutrition estimation with RGB-D fusion module and bidirectional feature pyramid network
摘要
With the development of society and the economy, the demand for food nutrition evaluation is increasing. Consequently, various nutritional estimation methods have been proposed. However, there are still some issues that need to be further considered. Specifically, (1) Traditional methods often rely on specialized biochemical instruments and are typically confined to laboratory environments, thereby making them difficult to popularize; (2) Some vision-based methods only leverage RGB images as input, missing some necessary spatial information. To solve the above problems, we propose a novel Fusion and Bidirectional Feature Pyramid Network (FBFPN) for nutrition estimation. The proposed FBFPN is an end-to-end approach, which simultaneously takes RGB and depth images as input, the spatial information can be effectively utilized. Besides, we develop an RGB-D fusion module to excavate richer vision features. A multi-scale fusion module is proposed to fuse feature maps with different resolutions. Compared with state-of-the-art methods, the mean value of the PMAE for our method reaches 17.3