FoodLens: Fine-Grained and Multi-label Classification of Indian Food Images
摘要
India has rich cultural diversity which is reflected in its variety of food. In recent years, computer vision has played a key role in classifying food images for automated tagging, nutrition profiling and many other tasks. However, the existing state-of-the-art AI-based food classification models trained on global food images have subpar performance on Indian food images. This is due to the lack of representation of Indian food in existing food datasets and unique image classification challenges specific to Indian food, such as cuisines having multiple dishes within a single image and regional fine grained varieties of the dishes. To address these challenges, a dataset with 30K food images consisting of popular dishes from restaurant menus across India was curated and annotated with multi-label and fine-grained labels for each dish in the image. All the dishes were mapped onto a hierarchical tree which models a categorical breakdown of Indian food. Custom loss function was tuned to learn from hierarchical and multi-label information contained in the Indian food images. Augmenting our loss on existing methods gives 13% improvement on average AUPRC and shows better classification performance on Indian food dataset compared to state of art food classification models with comparable results for other food benchmark datasets. More than 100k photos which are submitted each day on Google Maps on Indian restaurants and many more on social media channels were utilized for the project.