Performance Analysis of Image Caption Generation Techniques Using CNN-Based Encoder–Decoder Architecture

Image captioning is the method of generating textual descriptions for an image using deep neural networks. Its objective is to produce accurate results to specify the hidden features and to satisfy its wide applications. There are various Convolutional Neural Network (CNN)-based encoder architectures available in the literature for image caption generation and there is a need to empirically evaluate the best-performing architecture on multiple and diverse datasets to check their efficacy and generalization capability. To address this, we performed the experiments using the Flickr30K dataset containing 31,783 images along with the commonly used Flickr8K dataset consisting of 8091 images. In this study, we aim to discover the best-suited CNN architecture models for caption generation. The study evaluated various encoder architectures, including VGG16, VGG19, InceptionV3, and InceptionResNetV2, for extracting image features and Long Short-Term Memory (LSTM) as a decoder for generating accurate captions. The models are analyzed based on accuracy variation and value loss metrics on both datasets. The results depict that all models perform better on the larger dataset, i.e., Flickr30K achieves better accuracy with minimum loss values and VGG19 shows the best results among all.