PIXEMO: Captioning Images Along with Emojis for Social Media Applications

This research paper focuses on incorporating emojis into image captioning systems using deep learning CNN and RNN architecture, providing a more engaging and emotionally expressive narrative for images. It opens up new avenues for enhancing the communication of visual content and enriching the user experience in various applications, such as social media, image sharing platforms, and assistive technologies for visually impaired individuals. We have taken into consideration ResNet50 as CNN and LSTM as RNN for image description purposes and compared LSTM, Stacked LSTM, GRU, and Bidirectional LSTM for emoji prediction. Data preprocessing is a crucial step in preparing datasets for research work, but it can often be a cumbersome task. That’s why in this research paper, we have also explained the data preprocessing steps needed in detail for the Flickr8K dataset. The datasets used are Flickr8K and custom dataset for image captioning and emoji prediction respectively. BLEU score is used for evaluation purposes.