A Novel Method for Hybrid Deep Learning-Based Image Captioning
摘要
The automatic generation of the image caption resulted from a thorough study of the image, which included recognizing items and their relationships. Our paper propose a unique hybrid architecture that combines the benefits of ResNet50 and MobileNet to enhance image caption performance, evaluated on Flickr30k dataset, for photo captioning tasks. ResNet50, which is well known for its deep residual learning and potent feature extraction capabilities, is coupled with efficient and lightweight MobileNet to strike a compromise between model accuracy and computational economy. By merging characteristics from both networks, our design combines the efficient and fast processing of MobileNet with the sophisticated semantic understanding of ResNet50. For image caption, which is the automatic generation of natural language descriptions for pictures, accurate feature extraction using convolutional neural networks is essential. The hybrid model’s capacity to maximize the trade-off between computational cost and accuracy might be advantageous for real-time applications. Here, we merge features derived from input images using two contemporary pre-trained convolutional neural networks models, ResNet50 and MobileNet, utilizing a deep feature concatenation approach. To enhance general prediction abilities, we therefore suggest MobiRes-Net, a neural network that combines the ResNet50 and MobileNet models. In terms of BLEU, METEOR, and ROUGE scores, we evaluated the proposed model on the Flickr30k dataset and achieve competitive results, i.e., 99% accuracy, beating standalone models and reducing inference time. This approach provides a workable solution for image caption workloads in resource-constrained environments by striking a compromise between accuracy and efficiency.