Robot Vision and Deep Learning for Automated Planogram Compliance in Retail

In this paper, automated planogram compliance technique is proposed for retail applications. A mobile robot with camera vision capabilities provides the images of the products on shelves, which are processed to reconstruct an overall image of the shelves to be compared to the planogram. The image reconstruction includes image frames extraction from live video stream, images stitching and concatenation. Object detection, for the products, is achieved using a deep learning tool based on YOLOv5 model. Dataset, for algorithm training and testing, is built to identify the products based on their image identification, number, and location on the shelf. A small scale of shelves with products is built and different cases of products on shelves are tested in a laboratory environment. It was found that YOLOv5 algorithm detects various products on shelves with a precision of 0.98, recall of 0.99, F-measure of 0.98, and clarification loss of 0.006.