Analysing Various CNNs for Image Caption Generation

Deep Learning has accelerated the efficacy of generating descriptive captions for images. Much of the work done in this field includes the use of Convolutional Neural Networks (CNNs) for extracting features from input images, and subsequently a Text Generator (typically Recurrent Neural Networks are used) for generating sentences that describe the image. In this study, we focus on comparing 12 different CNNs for the Image Captioning Task and compare their performance through metrics such as BLEU and METEOR. Our study employs the Flickr Dataset, a relatively small dataset compared to larger collections like MS-COCO, to investigate the performance of CNNs when trained on limited data. Furthermore, we leverage pre-trained CNN models, initially trained on the ImageNet Dataset, and use the Long Short-Term Memory (LSTM) as the Text Generator, to understand their impact on image captioning. Notably, our research refrains from employing an attention mechanism to study the inherent behaviour of CNNs in generating image captions, utilizing pure feature vectors extracted from images by CNNs. Our observations conclude that recently developed CNNs, such as ConvNeXt and EfficientNetV2, perform better than fairly older ones, which include VGG16-19 and ResNetV2.