Expiration Date Recognition System Using Spatial Transformer Network for Visually Impaired

In this paper, we propose a system for recognizing an expiration date of a beverge package. A visually impaired user takes a photo of the product. Then, the system automatically recognizes the expiration date of the product and tells it to the user. The dates in the image were sometimes skewed, misaligned, or partially missing. To solve these problems, we use Spatial Transformer Network (STN) to recognize skewed, misaligned, or partially missing date images. STN explicitly allows the spatial manipulation of data within the neural network. We propose a dedicated CNN with STN expiration date recognition. The input to this network is an image of the expiration date. The network has six STNs to detect each digit of the date. Each STN outputs the rectified image of the digits. The image of the digits is then recognized by the traditional CNN. We trained the expiration date recognition network. The network converged quickly, and the training was stopped in 10 epochs. We tested this network on 1088 date images that were not used during training. The experimental results showed that the system was able to achieve a high recognition rate of 99.42% for the dataset without dates with spaces and 98.44% for the dataset with dates with spaces.